Feed tracking control method for pigtail wishbone deboning operation

Abstract

The invention provides a feed tracking control method for pigtail wishbone deboning operation, and the method comprises the steps: obtaining an X-ray image of a pigtail position, and obtaining an optimal deboning track according to the image information in the X-ray image; according to the inclination angle of the deboning knife and the deboning force difference value range, the optimal deboning track is divided into a deboning complex section and a deboning simple section, it can be ensured that the deboning robot adopts a proper deboning method in different deboning road sections, the deboning difficulty of the deboning complex section is ranked, color mapping is conducted on the optimal deboning track, and the deboning accuracy of the deboning robot is improved. Matching the trajectory information with the force information to obtain an optimal deboning trajectory with force sense form points; the cutting position, the cutting force and the cutting angle of the deboning cutter are adjusted in real time according to the force sense form points on the deboning track, deboning is carried out according to the optimal deboning track, and the deboning accuracy of the pigtail wishbone and the smoothness of the deboning section are improved.

Description

Technical Field

[0001] The invention relates to the technical field of a pigtail wishbone deboning robot, and in particular to a tool tracking control method for a pigtail wishbone deboning operation. Background Art

[0002] Pork has multiple nutritional values ​​and health benefits. With the growth of population and the improvement of people's living standards, the demand for pork consumption has shown an upward trend. Consumers have higher and higher requirements for the quality, taste, nutritional value and other aspects of pork. By accurately cutting different parts of pork, different cooking needs and market demands can be met. The shape of the pigtail fork bone is complex, and the bones are irregularly distributed in the pig's hind legs. Manual removal of the pigtail fork bone requires superb operating skills and experience, and the worker's physical strength, endurance and proficiency will affect the quality and efficiency of deboning. Using a robot to remove the pigtail fork bone can solve the above problems. However, when the deboning robot is deboning, due to the complex bone structure, it is easy to have defects such as knife jamming and incomplete deboning. Therefore, the present invention proposes a tool tracking control method for deboning pigtail fork bones. Specifically, the deboning robot first obtains the deboning trajectory according to the X-ray image, and then controls the tool to deboning through the mechanical arm. When the actual deboning trajectory of the tool deviates due to cutting bones or pork being compressed, the cutting direction of the knife is changed according to the guidance of the force sense morphological point, and the knife position is found again for precise cutting. Summary of the invention

[0003] In view of the problems in the prior art, the present invention provides a tool tracking control method for pigtail forkbone deboning operation, aiming to realize autonomous control of the tool tracking trajectory, thereby realizing the pigtail forkbone deboning operation.

[0004] A tool tracking control method for pig tail wishbone deboning operation comprises the following steps: Step 1: Obtain an X-ray image of the pig's tail, and obtain the optimal deboning trajectory based on the image information in the X-ray image; Step 2: Roughly segment the deboning trajectory based on the bone pit where the pig's hind leg bone connects with the coccyx; Step 3: subdividing each coarse segment according to the inclination angle of the deboning knife during deboning and the difference between the maximum value and the minimum value of the deboning force of each coarse segment, and obtaining a complex segment and a simple segment; wherein the complex segment and the simple segment both correspond to corresponding difference ranges and angle ranges; Step 4: According to the inclination angle of the deboning knife during deboning and the real-time force feedback information of the six-dimensional force sensor, the deboning trajectory points of the complex segment are color mapped and the deboning trajectory with force morphological points is obtained; Step 5: For simple segments, the boning knife is used to perform deboning according to the optimal deboning trajectory; for complex segments, the boning knife is used to perform deboning according to the optimal deboning trajectory with force-sensing morphological points; thereby completing the deboning operation of the pigtail forkbone.

[0005] Further: In step 3, the angle at which the boning knife is tilted during deboning for: in: is the vector from the Cartesian coordinate origin to the tool tip position; is the vector from the Cartesian origin to the tool handle position.

[0006] Further: the difference between the maximum and minimum values ​​of the coarse segmented bone removal force The angle of inclination of the deboning knife is set at 10N. As a benchmark, each coarse segment is subdivided as shown in the following table: Thus, complex segments and simple segments are obtained.

[0007] Further: Step 4 includes the following steps: Step 4.1: Sort the deboning difficulty of the deboning trajectory points to obtain the original sorting data ; Step 4.2: Raw sorted data Normalize to [0, 1]: in: is the pixel coordinate point; , They are the minimum and maximum values ​​of the sorted values ​​respectively; is a normalized value, indicating the relative position of the value in the range [0, 1]; Step 4.3: Mapped to color values, in: , , is the obtained pixel color value; Each pixel in the image contains the optimal deboning trajectory The data points corresponding to the color value data are mapped to RGB values: Thus, a deboned color trajectory segmentation graph with color trajectory is obtained; Step 4.4: adjusting the depth of the trajectory color in the deboning color trajectory segmentation map according to the weights of the force feature and the visual feature, and obtaining the final deboning trajectory map; Step 4.5: Convert the image coordinate system of the final deboning trajectory map into the deboning trajectory coordinate system, thereby obtaining a deboning trajectory with force-sensing morphological points.

[0008] Further: In step 4.4, the correlation between force features and visual features is calculated: in: The first The vector and the visual feature The correlation of vectors; The first vectors; The visual features vectors; represents the relevance scoring function; Calculate the attention weights: in: The first The vector and the visual feature The attention weights of the vector; is the number of vectors; Calculate the weighted fusion force features: in: It is the force feature after weighted fusion; in: is the visual feature after weighted fusion; The weighted visual features and force features are concatenated into a comprehensive feature vector: in: is the comprehensive feature vector; when When , the comprehensive feature vector L is set to be larger, and the color of the bone removal trajectory points is deepened; when When , the comprehensive feature vector L is set to be normal, and the color of the deboning trajectory points is calibrated to be normal; when When , the comprehensive feature vector L is set to be smaller, and the color of the bone removal trajectory points becomes lighter.

[0009] Further, in step 4.5, the coordinates of the upper left corner of the deboning trajectory segmentation map are used as the origin to establish an image coordinate system, and the image coordinate system of the final deboning trajectory map is converted into the deboning trajectory coordinate system, that is: in: is the image coordinate system; is the bone removal trajectory coordinate system; is the rotation matrix, i.e., the rotation of the image coordinate system relative to the deboning trajectory coordinate system; is the translation vector, i.e., the translation of the image coordinate system relative to the bone removal trajectory coordinate system; The origin of the bone removal trajectory coordinate system is the center point of the bone socket of the pig hind leg bone.

[0010] Further: original sorting data The table is: Among them, the difference is the difference between the maximum and minimum values ​​of the deboning force, angle And adjust the angle of the boning knife when deboning.

[0011] Further, at the junction of the simple segment and the complex segment, the priority of the optimal deboning trajectory and the optimal deboning trajectory with force-sensing morphological points is adjusted in real time according to the system operation status, and the output is: in: Control strategy for the boning knife; is the time-varying weight, which changes gradually as the deboning trajectory switches; The deboning knife follows the optimal deboning trajectory. The deboning knife performs deboning according to the optimal deboning trajectory with force-sensing morphological points.

[0012] The beneficial effects of the present invention are as follows: according to the inclination angle of the boning knife and the range of the difference in the boning force, the optimal boning trajectory is divided into a complex boning section and a simple boning section, which can ensure that the boning robot adopts an appropriate boning method in different boning sections, sorts the boning difficulty of the complex boning section, performs color mapping on the optimal boning trajectory, matches the trajectory information with the force information, and obtains an optimal boning trajectory with force-sensing morphological points; enables the boning tool to adjust the knife position, knife force and knife angle in real time according to each force-sensing morphological point on the boning trajectory, follows the optimal boning trajectory for boning, and improves the accuracy of pig tail forkbone boning and the flatness of the boning section. BRIEF DESCRIPTION OF THE DRAWINGS

[0013] Figure 1 It is a schematic diagram of the process of the present invention; Figure 2 It is the trajectory diagram of bone removal; Figure 3 This is a rough segmentation diagram of the bone removal trajectory. DETAILED DESCRIPTION

[0014] The present invention is described in detail below in conjunction with the accompanying drawings. The embodiments of the present invention are described in detail below, and examples of the embodiments are shown in the accompanying drawings, wherein the same or similar reference numerals throughout represent the same or similar elements or elements with the same or similar functions. The embodiments described below with reference to the accompanying drawings are exemplary and are only used to explain the present invention, and cannot be interpreted as limiting the present invention. The directional terms such as left, middle, right, top, and bottom in the embodiments of the present invention are only relative concepts or are based on the normal use state of the product, and should not be considered as restrictive.

[0015] A tool tracking control method for pig tail wishbone deboning operation, such as Figure 1 As shown, the following steps are included: Step 1: Obtain an X-ray image of the pig's tail and obtain the optimal deboning trajectory based on the image information in the X-ray image, specifically: Step 1.1: Use an X-ray machine to irradiate the pig's tail furcula to obtain an X-ray image of the tail furcula; Step 1.2: Use depth cameras, industrial cameras and other visual sensors to process the X-ray image of the pig's tail before deboning. Based on the yolov5 network, first extract the key features of the X-ray image, obtain the tail fork bone meat boundary box and bone meat category data information, amplify the image of the deboned area, denoise the image and detect the edge, and train the image information of the key parts of the tail fork bone to obtain the optimal deboning trajectory, such as Figure 2 As shown; Step 2: The deboning trajectory is roughly segmented based on the bone pit where the pig's hind leg bone and the coccyx are connected; that is, the deboning trajectory is roughly segmented based on the bone pit where the pig's hind leg bone and the coccyx are connected, a vertical line A of the deboning trajectory is drawn with the center of the bone pit as the origin o, and two lines parallel to line A are drawn in the directions of H cm on both sides, line B and line C, and the intersection points of line B and line C with the deboning trajectory are set to b and c, the starting point of the deboning trajectory is a, and the end point is d; Figure 3 ,Therefore, the bone removal trajectory is roughly segmented into three segments: ab, bc, and cd; Step 3: According to the inclination angle of the boning knife during deboning and the difference between the maximum and minimum values ​​of the deboning force of each coarse segment, each coarse segment is subdivided to obtain a complex segment and a simple segment; wherein the complex segment and the simple segment both have corresponding difference ranges and angle ranges; the inclination angle of the boning knife during deboning for: in: is the vector from the Cartesian coordinate origin to the tool tip position; is the vector from the Cartesian coordinate origin to the tool handle position; Deboning is performed with the deboning tool according to the deboning trajectory segments ab, bc and cd, the deboning force of each deboning segment is collected and analyzed, and the difference between the maximum and minimum deboning force of each segment is set as , and the difference between the maximum and minimum values ​​of the coarse segmented deboning force The angle of inclination of the deboning knife is set at 10N. As a benchmark, each coarse segment is subdivided as shown in the following table: Thus, complex segments and simple segments are obtained; It is necessary to convert the three-dimensional pixel coordinates of the handle position and the tip position of the boning tool obtained by the camera, that is, the camera coordinate system with the camera as the origin, into the Cartesian coordinate system of the robot; in order to calculate and ; Specifically: in: is the Cartesian coordinate system of the robot; is the camera coordinate system; is the rotation matrix (rotation of the camera coordinate system relative to the Cartesian coordinate system); is the translation vector (the translation of the camera coordinate system relative to the Cartesian coordinate system); Through the coordinate system conversion, the handle position and tip position of the deboning tool in the Cartesian coordinate system, as well as the three-dimensional coordinate position point of the deboning trajectory are obtained. , , ), , , ), , , ); When deboning is performed strictly according to the optimal deboning trajectory, the three-dimensional coordinates of the tool tip position and the deboning trajectory always remain coincident, and the vector from the Cartesian coordinate origin to the tool tip position and the tool handle position is calculated: , ; Step 4: According to the inclination angle of the deboning knife during deboning and the real-time force feedback information of the six-dimensional force sensor, color mapping is performed on the deboning trajectory points of the complex segment to obtain a deboning trajectory with force morphological points; specifically, the following steps are included: Step 4.1: Sort the deboning difficulty of the deboning trajectory points to obtain the original sorting data ; Original sequencing data The table is: Among them, according to the precise bone-removing force numerical information of the six-dimensional force sensor, a digital low-pass filter is used to reduce the noise of the transmitted force numerical information: in: For the The filtered output at the moment refers to the force value signal after low-pass filtering; is the original signal Sample values; is the covariance matrix caused by noise on the observation results; is the length of the sliding window; The force value after noise reduction is , let the maximum value be , the minimum value is , then the difference range is: Difference is the difference between the maximum and minimum values ​​of the deboning force, angle The present invention also adjusts the angle of inclination of the bone removal knife during bone removal. and The smallest ranking value is considered And the ranking value is 1, which means that the difficulty of bone removal is relatively low. and and and The maximum sort value is considered And the ranking value 6 means that the difficulty of bone removal is relatively high; Step 4.2: Raw sorted data Normalize to [0, 1]: in: is the pixel coordinate point; , They are the minimum and maximum values ​​of the sorted values ​​respectively; is a normalized value, indicating the relative position of the value in the range [0, 1]; Step 4.3: Mapped to color values, in: , , is the pixel color value obtained; Each pixel in the image contains the optimal deboning trajectory The data points corresponding to the color value data are mapped to RGB values: Thus, a deboned color trajectory segmentation graph with color trajectory is obtained; Step 4.4: Adjust the depth of the trajectory color in the deboning color trajectory segmentation map according to the weights of the force feature and the visual feature, and obtain the final deboning trajectory map; specifically: pass the optimal deboning trajectory and color mapping information into the convolutional neural network and the recurrent neural network to extract features, and output represents the visual feature vector, Represents the force feature vector; after feature extraction, the attention mechanism is introduced to realize multimodal fusion. During the fusion process, the visual information is used to weight the force information. Specifically, the correlation between the force feature and the visual feature is calculated: in: The first The vector and the visual feature The correlation of vectors; The first vectors; The visual features vectors; represents the relevance scoring function; Calculate the attention weights: in: The first The vector and the visual feature The attention weights of the vector; is the number of vectors; Calculate the weighted fusion force features: in: It is the force feature after weighted fusion; in: is the visual feature after weighted fusion; The weighted visual features and force features are concatenated into a comprehensive feature vector: in: is the comprehensive feature vector; when When , the comprehensive feature vector L is set to be larger, and the color of the bone removal trajectory points is deepened; when When , the comprehensive feature vector L is set to be normal, and the color of the deboning trajectory points is calibrated to be normal; when When , the comprehensive feature vector L is set to be smaller, and the color of the bone removal trajectory points becomes lighter; Step 4.5: Convert the image coordinate system of the final deboning trajectory map to the deboning trajectory coordinate system, so as to obtain the deboning trajectory with force-sensing morphological points; specifically, take the coordinates of the upper left corner of the deboning trajectory segment map as the origin, establish an image coordinate system, and convert the image coordinate system of the final deboning trajectory map to the deboning trajectory coordinate system, that is: in: is the image coordinate system; is the bone removal trajectory coordinate system; is the rotation matrix, i.e., the rotation of the image coordinate system relative to the deboning trajectory coordinate system; is the translation vector, i.e., the translation of the image coordinate system relative to the bone removal trajectory coordinate system; The origin of the bone removal trajectory coordinate system is the center point of the bone pit of the pig hind leg bone; Step 5: For simple segments, the deboning knife is used to debone according to the optimal deboning trajectory; for complex segments, the deboning knife is used to debone according to the optimal deboning trajectory with force-sensing morphological points; at the junction of simple and complex segments, the priority of the optimal deboning trajectory and the optimal deboning trajectory with force-sensing morphological points is adjusted in real time according to the system operation status, and the output is: in: Control strategy for the boning knife; is the time-varying weight, which changes gradually as the deboning trajectory switches; The deboning knife follows the optimal deboning trajectory. The deboning knife performs deboning according to the optimal deboning trajectory with force-sensing morphological points, thereby completing the deboning operation of the pig tail fork bone.

[0016] In this embodiment, the X-ray image of the pig's tail before deboning is processed by visual sensors such as a depth camera and an industrial camera. The depth camera is a camera that can obtain the depth information of the pig's hind legs, and can calculate the position and posture of the pig's tail in three-dimensional space, thereby helping the deboning robot to perceive the surrounding environment more accurately and plan a deboning trajectory. The industrial camera is used to identify and track the deboning route to ensure that the deboning knife follows the optimal deboning trajectory. Then, according to the inclination angle of the deboning knife and the deboning force difference range, the optimal deboning trajectory is divided into a complex deboning section and a simple deboning section, which can ensure that the deboning robot adopts a suitable deboning method in different deboning sections, sort the deboning difficulty of the complex deboning section, perform color mapping on the optimal deboning trajectory, match the trajectory information with the force information, and obtain an optimal deboning trajectory with force morphological points; the deboning tool is enabled to adjust the knife position, knife force and knife angle in real time according to each force morphological point on the deboning trajectory, follow the optimal deboning trajectory for deboning, and improve the accuracy of deboning the pig's tail forkbone and the flatness of the deboned section.

[0017] The above shows and describes the basic principles, main features and advantages of the present invention. It should be understood by those skilled in the art that the present invention is not limited to the above embodiments. The above embodiments and descriptions are only for explaining the principles of the present invention. Without departing from the spirit and scope of the present invention, the present invention may have various changes and improvements, which fall within the scope of the present invention. The scope of protection of the present invention is defined by the attached claims and their equivalents.

Claims

1. A tool tracking control method for pig tail wishbone deboning operation, characterized in that: The following steps are involved: Step 1: Obtain an X-ray image of the pig's tail, and obtain the optimal deboning trajectory based on the image information in the X-ray image; Step 2: Roughly segment the deboning trajectory based on the bone pit where the pig's hind leg bone connects with the coccyx; Step 3: subdividing each coarse segment according to the inclination angle of the deboning knife during deboning and the difference between the maximum value and the minimum value of the deboning force of each coarse segment, and obtaining a complex segment and a simple segment; wherein the complex segment and the simple segment both correspond to corresponding difference ranges and angle ranges; Step 4: According to the inclination angle of the deboning knife during deboning and the real-time force feedback information of the six-dimensional force sensor, the deboning trajectory points of the complex segment are color mapped and the deboning trajectory with force morphological points is obtained; Step 5: For simple segments, the boning knife is used to perform deboning according to the optimal deboning trajectory; for complex segments, the boning knife is used to perform deboning according to the optimal deboning trajectory with force-sensing morphological points; thereby completing the deboning operation of the pigtail forkbone.

2. The method for controlling the tool tracking of pigtail wishbone deboning according to claim 1, characterized in that: In step 3, the angle at which the boning knife is tilted during deboning for: ； ； in: is the vector from the Cartesian coordinate origin to the tool tip position; is the vector from the Cartesian origin to the tool handle position.

3. The method for controlling the tool tracking of pigtail wishbone deboning according to claim 2, characterized in that: The difference between the maximum and minimum values ​​of the coarse segment deboning force The angle of inclination of the deboning knife is set at 10N. As a benchmark, each coarse segment is subdivided as shown in the following table: ； Thus complex segments and simple segments are obtained.

4. The method for controlling the tool tracking of pigtail wishbone deboning according to claim 1, characterized in that: Step 4 includes the following steps: Step 4.1: Sort the deboning difficulty of the deboning trajectory points to obtain the original sorting data ; Step 4.2: Raw sorted data Normalize to [0, 1]: ； in: is the pixel coordinate point; , They are the minimum and maximum values ​​of the sorted values ​​respectively; is a normalized value, indicating the relative position of the value in the range [0, 1]; Step 4.3: Mapped to color values, ； ； ； in: , , is the pixel color value obtained; Each pixel in the image contains the optimal deboning trajectory The data points corresponding to the color value data are mapped to RGB values: ； Thus, a deboned color trajectory segmentation graph with color trajectory is obtained; Step 4.4: adjusting the depth of the trajectory color in the deboning color trajectory segmentation map according to the weights of the force feature and the visual feature, and obtaining the final deboning trajectory map; Step 4.5: Convert the image coordinate system of the final deboning trajectory map into the deboning trajectory coordinate system, thereby obtaining a deboning trajectory with force-sensing morphological points.

5. The method for controlling the tool tracking in the pigtail wishbone deboning operation according to claim 4, characterized in that: In step 4.4, the correlation between force features and visual features is calculated: ； in: The first The vector and the visual feature The correlation of vectors; The first vectors; The visual features vectors; represents the relevance scoring function; Calculate the attention weights: ； in: The first The vector and the visual feature The attention weights of the vector; is the number of vectors; Calculate the weighted fusion force features: ； in: It is the force feature after weighted fusion; ； in: is the visual feature after weighted fusion; The weighted visual features and force features are concatenated into a comprehensive feature vector: ； in: is the comprehensive feature vector; when When , the comprehensive feature vector L is set to be larger, and the color of the bone removal trajectory points is deepened; when When , the comprehensive feature vector L is set to be normal, and the color of the deboning trajectory points is calibrated to be normal; when When , the comprehensive feature vector L is set to be smaller, and the color of the bone removal trajectory points becomes lighter.

6. The method for controlling the tool tracking in the pigtail wishbone deboning operation according to claim 4, characterized in that: In step 4.5, the coordinates of the upper left corner of the deboning trajectory segmentation map are used as the origin to establish an image coordinate system, and the image coordinate system of the final deboning trajectory map is converted into the deboning trajectory coordinate system, that is: ； in: is the image coordinate system; is the bone removal trajectory coordinate system; is the rotation matrix, i.e., the rotation of the image coordinate system relative to the deboning trajectory coordinate system; is the translation vector, i.e., the translation of the image coordinate system relative to the bone removal trajectory coordinate system; The origin of the bone removal trajectory coordinate system is the center point of the bone socket of the pig hind leg bone.

7. The method for controlling the tool tracking in the pigtail wishbone deboning operation according to claim 4, characterized in that: Raw sequencing data The table is: ； Among them, the difference is the difference between the maximum and minimum values ​​of the deboning force, angle And adjust the angle of the boning knife when deboning.

8. The method for controlling the tool tracking in pigtail wishbone deboning according to claim 1, characterized in that: At the junction of the simple segment and the complex segment, the priority of the optimal deboning trajectory and the optimal deboning trajectory with force-sensing morphological points are adjusted in real time according to the system operation status, and the output is: ； in: Control strategy for the boning knife; is the time-varying weight, which changes gradually as the deboning trajectory switches; The deboning knife follows the optimal deboning trajectory. The deboning knife performs deboning according to the optimal deboning trajectory with force-sensing morphological points.

Images