Deep learning-based binocular vision measurement method for fat thickness of cross section of middle square pork of pig

Abstract

A deep learning-based binocular vision measurement method for cross-section fat thickness of pig middle square meat is characterized in that after left and right images are obtained through a binocular vision system, feature points in feature regions of the left and right images are matched and reconstructed to restore three-dimensional depth information of points to be measured, and more accurate cross-section fat thickness is obtained. According to the method, deep learning and digital image processing technologies are combined, section data information of the middle square meat is marked, an improved UNET network model is input, feature regions are segmented, feature point information is obtained, and then classification processing calculation is carried out on the middle square meat through a middle square meat triangular meat feature region judgment method. The method has the advantages of being non-contact, high in measurement precision, rich in feature information acquisition, high in speed and the like, and can effectively avoid the problems that in the field flow line production process, due to the fact that the placement position of the cut meat on the flow line is random, three-dimensional reconstruction information precision loss is caused, and then the precision of the measurement result of the section fat thickness is not high. And the automation degree in the industrial processing of the split pork is improved.

Description

Technical Field

[0001] The present invention belongs to the technical field of pig cut meat grading detection, and relates to a binocular vision measurement method for fat thickness of a pig square meat section based on deep learning. Background Art

[0002] With the development of economy and the increase of income, consumers are pursuing higher and higher food quality. Due to the problem of soft meat and deformation of the position to be measured due to external pressure or hanging position and angle in the process of measuring fat thickness of cut meat, conventional non-contact measurement methods are easily affected by the above interference factors, resulting in the inability to accurately characterize the fat thickness of the cross section. At present, the production lines of most pork slaughtering and processing enterprises in my country are still in the status quo of semi-automatic production lines and mixed production by workers. The relevant classification special machinery and instruments generally need to be imported from abroad. Once applied, it not only requires a lot of funds to introduce equipment, but also requires large-scale transformation of existing production lines and the introduction of relevant professional talents for the use and maintenance of equipment. The huge investment and the reliability of its profit creation have deterred many enterprises. In such an industry environment, most pork companies still choose to grade by manual visual and rough measurement. The manual measurement method with a ruler is highly subjective, the accuracy is difficult to guarantee, and the operation process directly contacts pork, there are problems such as pork contamination and squeezing pork to cause measurement errors, and it is difficult to achieve stable and accurate non-destructive detection and grading of fat thickness of the cross section of pork. The research team has previously authorized an invention patent, "A method for detecting the maximum fat thickness of a cut pork section based on a binocular vision system", with patent number ZL 2023 1 0039130.5, which uses ROI region fitting and edge line fitting methods to find the location information of feature points and obtain the fat thickness result. This method combines mathematical models with high measurement accuracy, but it needs to be performed under a specific experimental environment. It is difficult to meet the requirements of online real-time measurement, is not widely applicable, and is difficult to apply on the actual pig slaughtering line. Therefore, a non-destructive detection and grading method for fat thickness of pig square meat sections based on deep learning is provided. Summary of the invention

[0003] The present invention provides a binocular vision measurement method for the cross-sectional fat thickness of Chinese pork based on deep learning. After obtaining the left and right images through the binocular vision system, the feature points in the feature areas of the left and right images are matched and reconstructed to restore the three-dimensional depth information of the points to be measured, and finally a more accurate cross-sectional fat thickness is obtained. The method combines deep learning and digital image processing technology, by marking the cross-sectional data information of Chinese pork, inputting the improved UNET network model, performing feature area segmentation, and obtaining feature point information. After that, the Chinese pork is classified, processed and calculated through the Chinese pork triangle meat feature area determination method.

[0004] The technical solution of the present invention:

[0005] A binocular vision measurement method for fat thickness of a pig's middle square meat section based on deep learning comprises the following steps:

[0006] Step 1: Use a binocular acquisition system to collect images of the cross section of the middle pork meat, and obtain the cross section images of the middle pork meat as a data set. The data set is divided into a training set, a test set, and a validation set. According to the definition and regulations of cut meat in the national standard NY / T3380-2018, combined with the actual grading and measurement needs of the enterprise, the fat thickness measurement position of the cross section image of the middle pork meat is determined as the subcutaneous fat thickness below the triangular muscle near the rib side, that is, the subcutaneous fat thickness at the tip of the triangular meat.

[0007] Step 2: Build an improved UNet network by adding an attention mechanism;

[0008] The unet network is used as the basic network framework. CA modules and SA modules are added in each downsampling process. The CA modules and SA modules are connected in series to construct an improved UNet network. After the cross-section image of pork in the training set is input into the improved U-Net network, the convolution feature map before each layer of downsampling in the encoding network is used as the input of the CA module. The CA module first performs global average pooling on the input convolution feature map to obtain a 1×1×c vector, where c is the number of channels of the input feature map, leaving only the dimensional information of each channel. Then the weight of each channel is obtained through self-learning, and finally the convolution feature map is multiplied by the corresponding channel weight to obtain the weighted feature map. The obtained weighted feature map is input into the SA module to compress the output feature map. The compressed feature map is subjected to maximum pooling and average pooling to obtain two one-dimensional feature maps. Then the two one-dimensional feature maps are convolved to reduce them to one channel, and then the feature map is output through an activation function. Finally, the output feature map is multiplied by the original image to return to the original dimension size.

[0009] Step 3: After the improved UNet network reaches a convergence state after training iteration, a trained improved UNet network is obtained; the cross-section image of the pork in the verification set is introduced into the trained improved UNet network to obtain the segmentation result; the edge of the segmented meat is preliminarily extracted, and the binary image segmented by the trained improved UNet network is defined as a horizontal line structure and horizontal line corrosion is performed to prevent the loss of boundary pixels; the connected areas with an area less than 500 are removed to prevent small noise points from existing in the binary image; the processed binary image is smoothed to prevent excessive jagged edges;

[0010] Step 4: Extract the edge of the characteristic area of ​​the middle section image of the pig in step 1, and obtain the triangular meat characteristic area and pig skin edge data on the middle section of the pig; after feature recognition through the trained improved Unet network, establish a triangular meat characteristic area determination method for hierarchical classification according to the situation where the triangular meat characteristic area on the middle section of the pig will appear: obtain the upper and lower edges L of the triangular meat characteristic area on the middle section of the pig through the edge extraction of the triangular meat characteristic area 1 , L 2 The characteristic points of the straight line l are obtained by fitting 1 , l 2 , and find the intersection point A of the upper and lower edge fittings 1 (a 1 , b 1 ); The rightmost end point A of the triangular meat feature area on the middle section of the pig meat in the binary image segmented by the trained improved Unet network 2 (a 2 , b 2 ) and the upper and lower edge fitting intersection point A 1 The horizontal axis difference u=|a 1 -a 2 |, if u is less than the threshold 2, this type of cut meat is positioned at level I; if 2≤u<3, this type of cut meat is positioned at level II; if u≥3, this type of cut meat is positioned at level III;

[0011] Step 5: Based on the analysis of the distribution of feature points of the middle section of pig meat, a feature point matching method based on polar line constraint combined with brute force matching is proposed, that is, the tip feature points of the triangular meat feature area are matched with the feature points on the upper edge of the pig skin, and a regional classification feature matching method is proposed: for the tip feature points of the triangular meat feature area on the middle section of pig meat, brute force matching is directly used, and polar line constraint matching is performed on the feature points on the upper edge of the pig skin, so as to solve the problem of incorrect matching results due to the small number of tip feature points in the triangular meat feature area, and greatly improve the matching efficiency; then the triangular reconstruction method is used to realize the reconstruction of the three-dimensional information of the middle section of pig meat at the real scale;

[0012] Step 6: Obtain the tip feature point A of the triangular meat feature area on the middle section of the pig meat 1After that, it is necessary to extract the feature points on the upper edge of the pig skin. Combined with the definition of fat thickness measurement of the middle section of the pig in step 1, the upper edge coordinate point B of the pig skin that is in a normal vector direction relationship with the tip feature point A of the triangular meat feature area on the middle section of the pig must fall near the pig skin boundary below point A; on the two-dimensional section of the middle section of the pig, a plane rectangular coordinate system is established with the tip feature point A of the triangular meat feature area as the common origin, and two mutually perpendicular axes are established through point A, wherein the horizontal axis is the x-axis, the right direction is the positive direction, and the vertical axis is the y-axis, the upward direction is the positive direction; search vertically downward along the y-axis until the coordinate point that intersects with the contour of the upper boundary of the pig skin is marked as C 0 ; C 0 Point is the reference point. Search for n points to the left and right along the upper boundary of the pig skin at the same time. That is, search along the upper boundary L of the pig skin instead of horizontally along the x-axis direction. Then the feature point of the upper boundary of the pig skin is C 0-n To C 0+n ; and point B, which is in a normal direction to point A, will fall on point C 0-n To C 0+n Within the range; after multiple experiments, it is found that when n is 50, the measurement results after the system completes the 3D reconstruction are more accurate and the stability is higher;

[0013] Step 7: After completing the reconstruction of the three-dimensional information of the feature points, calculate the spatial distance from the tip feature point of the triangular meat feature area to the feature point on the upper edge of the pig skin in the normal vector direction, and obtain the measurement result of the fat thickness from the spatial distribution relationship; determine the three-dimensional coordinates of the tip key feature point of the triangular meat area feature as L (x, y, z), and perform a straight line fitting on the upper edge point of the pig skin to obtain a three-dimensional straight line equation ax + by + cz + d = 0; where a, b, c, d are constants used to describe the slope, direction and distance from the origin of the fitted straight line; define the upper edge feature point of the pig skin as a straight line m after fitting, and calculate the goodness of fit R of the straight line m 2 :

[0014]

[0015] Among them, y i is the ordinate of the discrete point, is the average value of the vertical coordinates of the discrete points, is the vertical coordinate of the discrete points after fitting; R 2 is a dimensionless coefficient with a value range of (0, 1), R 2 The closer it is to 1, the better the model fit is. 2 The closer it is to 0, the worse the model fitting effect;

[0016] If the straight line m fits well R 2 ≥0.7, it means that the spatial distribution of the feature points on the upper edge of the pigskin is close to a linear distribution, and the fitting effect is good. At this time, the calculation method of fat thickness is:

[0017]

[0018] Among them, P is any point on the fitting line m on the upper edge of the pig skin, A is the feature point of the tip of the triangular meat, S 0 is the unit direction vector of line m;

[0019] If the straight line m fits well R 2 <0.7, it means that the linear fitting effect of the edge points is poor, and the spatial position of the edge feature points on the pig skin is not distributed in a linear manner. The calculation formula for the cross-sectional fat thickness result is as follows:

[0020]

[0021] Among them, (x, y, z) is the three-dimensional coordinates of the feature point of the triangle meat tip, (x i ,y i ,z i ) are the three-dimensional coordinates of the discrete feature points on the upper edge of the pigskin.

[0022] Beneficial effects of the present invention:

[0023] (1) The attention fusion module is introduced into the U-Net network structure to construct a fusion attention pig cut feature area segmentation model, which fully extracts the feature information between each channel, improves the segmentation accuracy of the cross-sectional features of pig cut meat, and achieves the purpose of reducing redundant information and improving the convergence speed of the model, thereby improving the segmentation effect of the characteristic region of pig cut meat. By improving the U-Net network structure, the effects of supplementing the feature information of complex feature regions and removing interference noise can be achieved, and the maximum fat thickness of the cross-sectional surface of pig cut meat can be measured more accurately and objectively by locating feature points. Compared with traditional methods, the method of combining deep learning with digital image measurement has stronger universality, is more suitable for complex production environments, and has more accurate measurement results.

[0024] (2) The method for determining the characteristic area of ​​triangular meat is to classify the data of the actual triangular meat of pork cuts based on the complex segmentation results of the actual characteristic area. The definition and regulations of cut meat in the national standard are combined with the actual grading and measurement needs of the enterprise to meet the grading requirements of pork cut meat and lay the foundation for the subsequent grading of pork cut meat. BRIEF DESCRIPTION OF THE DRAWINGS

[0025] Figure 1 A flow chart of a binocular vision measurement method for fat thickness of a Chinese pork cross section based on deep learning provided in an embodiment;

[0026] Figure 2 A schematic diagram of image training annotation in an embodiment;

[0027] Figure 3This is a schematic diagram of the thickness of subcutaneous fat at the tip of the triangular meat;

[0028] Figure 4 This is a schematic diagram of the thickness in the normal direction from the tip of the triangular meat to the upper edge of the pig skin;

[0029] Figure 5 The schematic diagram of the improved Unet network model structure for the embodiment;

[0030] Figure 6 This is a diagram of the segmentation result of the verification set of the embodiment;

[0031] Figure 7 This is an edge extraction effect diagram of the embodiment;

[0032] Figure 8 , 10 12 is a schematic diagram of the complex situation of the actual production process of the embodiment;

[0033] Fig. 9 , 11 , 13 is a partial enlarged schematic diagram of a complex situation;

[0034] Fig.14 , 15 16 is a schematic diagram of a method for determining a characteristic region of triangular meat according to an embodiment;

[0035] Fig.17 Schematic diagram of stereo matching effect of the embodiment;

[0036] Fig.18 A schematic diagram of the three-dimensional reconstruction effect of the embodiment;

[0037] Fig.19 It is a schematic diagram of measuring the normal vector direction of the tip of the triangular meat of the meat section in the embodiment. DETAILED DESCRIPTION

[0038] The specific implementation of the present invention is further described below in conjunction with the accompanying drawings and technical solutions.

[0039] Example

[0040] A binocular vision measurement method for fat thickness of a Chinese pork cross section based on deep learning, comprising the following steps:

[0041] Step 1: Use the binocular acquisition system to collect images of the cross-section of the pig meat to obtain a data set. The data set annotation is as follows: Figure 2 As shown; According to the definition and regulations of cut meat in the national standard NY / T 3380-2018, combined with the actual grading and measurement needs of the enterprise, the measurement position of fat thickness of the middle meat section is determined as: the subcutaneous fat thickness below the triangular muscle near the rib side, such as Figure 3 The middle triangle shows the thickness of subcutaneous fat at the tip of the triangular meat. Figure 4 In the partial enlarged image on the right, ΔH is a schematic representation of the thickness in the normal vector direction from the tip of the triangle meat to the upper edge of the pig skin.

[0042] Step 2: Use the method of adding attention mechanism to build an improved UNet framework, such as Figure 5 ; Based on the unet model, the CA module and the SA module are added in each downsampling process, and the CA module and the SA module are connected in series; after the pork square meat image in the training set is input into the U-Net network, the convolution feature map before each layer of downsampling in the encoding network is the input of the CA module. The CA module first takes global average pooling for the input convolution feature map to obtain a 1×1×c vector, where c is the number of channels of the input feature map, leaving only the dimensional information of each channel; then the weight of each channel is obtained through self-learning, and finally the convolution feature map is multiplied by the corresponding channel weight to obtain the weighted feature map; the obtained feature map is continuously input into the SA module to compress the output feature map; perform a maximum pooling and an average pooling respectively to obtain two one-dimensional feature maps. The maximum pooling operation is to extract the maximum value on the channel, and the average pooling operation is to extract the average value on the channel, so that two 2-channel feature maps can be obtained. Then the two feature maps are convolved and reduced to one channel, and then the feature map is output through the activation function. Finally, the output result is multiplied by the original image to return to the original dimension size;

[0043] Step 3: After the experimental model reaches convergence through training iterations, the validation set images are introduced into the trained improved UNet model to obtain the segmentation results as follows: Figure 6 As shown. On this basis, the edge of the segmented meat is initially extracted. The binary image segmented by the model defines the horizontal line structure and performs horizontal line corrosion to prevent the loss of boundary pixels; the connected areas with an area less than 500 are removed to prevent small noise points from existing in the image; the image is smoothed to prevent excessive jagged edges; the edge extraction results of the feature area are shown in Figure 7 Step 4: Extract the edges of the feature areas of the left and right images of the collected Chinese meat section to obtain the edge data information of the triangular meat and pig skin on the Chinese meat section. In the actual production process, after the improved UNet model is used for feature recognition, the triangular meat area of ​​the Chinese meat section will appear as follows: Figure 8 (Good feature recognition, clear feature edges, Fig. 9 is a partial enlarged view), Fig.10 (feature missing, feature edge extension, Fig.11 is a partial enlarged view), Fig.12 (Noise interference near the feature area affects feature edge detection, Fig.13 For these three complex situations, we can establish Fig. 9 , 10 11. Triangle meat characteristic area determination method for grading and classification:

[0044] The upper and lower edges L of the triangle meat of the middle meat section are obtained by extracting the edge of the feature area 1、 L 2 The feature point information is obtained by fitting the straight line l 1 , l 2 , and find the intersection point A 1 (a 1 , b 1 ).

[0045] 1. The rightmost endpoint A of the triangle meat of the square meat section in the binary image segmented by the model 2 (a 2 , b 2 ) and the upper and lower edge fitting intersection point A 1 The horizontal axis difference u=|a 1 -a 2 |, if u is less than the threshold 2, then according to the definition and regulations of cut meat in the national standard and the actual grading and measurement requirements of the enterprise, this type of cut meat is positioned as Grade I. The actual cut meat image is as follows Fig.14 .

[0046] 2. The rightmost point A of the triangle meat in the square meat section in the binary image segmented by the model 2 (a 2 , b 2 ) and the upper and lower edge fitting intersection point A 1 (a 1 , b 1 ) make the difference u=|a 1 -a 2 |, if 2≤u<3, then according to the definition and regulations of cut meat in the national standard and the actual grading and measurement needs of the enterprise, this type of cut meat is positioned as level II. The actual cut meat image is as follows Fig.15 .

[0047] 3. The rightmost point A of the triangle meat in the square meat section in the binary image segmented by the model 2 (a 2 , b 2 ) and the upper and lower edge fitting intersection point A 1 (a 1 , b 1 ) make the difference u=|a 1 -a 2 |, if u≥3, then according to the definition and regulations of cut meat in the national standard and the actual grading and measurement requirements of the enterprise, this type of cut meat is positioned as grade III. The actual cut meat image is as follows Fig.16 .

[0048] Step 5: Based on the analysis of the distribution of feature points on the cross-section of Chinese meat, a feature point matching scheme based on polar line constraints combined with brute force matching is proposed, that is, the tip point of the triangular meat is matched with the feature point of the upper edge of the pig skin. A regional classification feature matching method is proposed, and feature points of the left and right images are accurately extracted, and feature point matching and reconstruction are performed.

[0049] The epipolar constraint matching is used to perform stereo matching on the feature point extraction results of the upper edge of the pigskin in the left and right images. The biggest advantage of the epipolar constraint matching is that it changes the search for feature points from a two-dimensional search of the entire image to a one-dimensional search along the epipolar direction, which greatly improves the matching efficiency. However, due to the small number of feature points at the tip of the triangle meat in the middle meat section, the results of stereo matching using the epipolar constraint method are too random, and it is very easy to have wrong matching results. Therefore, it is impossible to reconstruct three-dimensional information to obtain measurement results. Therefore, a feature point matching scheme based on epipolar constraint combined with brute force matching is proposed, and a regional classification feature matching method is proposed. For the feature points of the tip of the triangle meat in the middle meat section, brute force matching is directly used to solve the problem of wrong stereo matching results.

[0050] After the feature point matching is completed, in order to achieve the reconstruction of the three-dimensional information of the cross section of the Chinese meat at the real scale, the triangulation reconstruction method is used to improve the robustness and accuracy of the reconstruction. Fig.17 The matching scheme 3D reconstruction effect is shown in Fig.18 shown.

[0051] Step 6: Extract the feature point A at the tip of the triangle meat 1 After that, the feature points on the upper edge of the pig skin need to be extracted. Combined with the definition of fat thickness measurement of the middle meat section in step 1, the upper edge coordinate point B of the pig skin that is in a normal vector direction relationship with the tip point A of the triangular meat feature area must fall near the pig skin boundary below point A; Fig.19 As shown in the figure, a plane rectangular coordinate system is established with the tip feature point A of the triangular meat feature area as the origin, and a vertical search is performed downward along the y-axis until the coordinate point intersecting with the upper boundary contour of the pig skin is marked as C. 0 ; C 0 Point is the reference point. Search for n points to the left and right along the upper boundary of the pig skin (along the upper boundary L of the pig skin instead of searching horizontally along the x-axis direction). The feature point of the upper boundary of the pig skin is C 0-n To C 0+n ; and point B, which is in a normal direction to point A, will fall on point C 0-n To C 0+n range; through multiple experiments, it is found that when n is 50, the measurement results after the system completes 3D reconstruction are more accurate and the stability is higher.

[0052] Step 7: After completing the reconstruction of the three-dimensional information of the feature points, it is necessary to calculate the spatial distance from the tip of the triangular meat to the feature point on the upper edge of the pig skin in the normal direction, and obtain the measurement result of the fat thickness from the spatial distribution relationship. Determine the three-dimensional coordinates of the key feature point at the tip of the triangular meat as L (x, y, z), and perform a straight line fitting on the upper edge points of the pig skin to obtain the three-dimensional straight line equation ax + by + cz + d = 0. Among them, a, b, c, d are constants used to describe the slope, direction and distance from the origin of the fitted straight line. Define the feature points on the upper edge of the pig skin as a straight line m after fitting, and calculate the goodness of fit R of the straight line m 2 :

[0053]

[0054] Among them, y i is the ordinate of the discrete point, is the average value of the vertical coordinates of the discrete points, is the vertical coordinate of the discrete points after fitting. 2 is a dimensionless coefficient with a value range of (0, 1), R 2 The closer it is to 1, the better the model fit is. 2 The closer it is to 0, the worse the model fitting effect.

[0055] If the straight line m fits well R 2 ≥0.7, it means that the spatial distribution of the feature points on the upper edge of the pigskin is close to a linear distribution, and the fitting effect is good. At this time, the calculation method of fat thickness is:

[0056]

[0057] Among them, P is any point on the fitting line m on the upper edge of the pig skin, A is the feature point of the tip of the triangular meat, S 0 is the unit direction vector of line m.

[0058] If the straight line m fits well R 2 <0.7, it means that the linear fitting effect of the edge points is poor, and the spatial position of the edge feature points on the pig skin is not distributed in a linear manner. The calculation formula for the cross-sectional fat thickness result is as follows:

[0059]

[0060] Among them, (x, y, z) is the three-dimensional coordinates of the feature point of the triangle meat tip, (x i ,y i ,z i ) are the three-dimensional coordinates of the discrete feature points on the upper edge of the pigskin.

[0061] Step 11: Test and evaluate the accuracy of the segmentation model and the fat thickness of the cut meat section. In order to test the effect of the improved Unet model in the segmentation task of this embodiment, the segmentation results of the original U-Net neural network and the pig cut meat fat segmentation model with the added attention module constructed in this application are compared and tested. The cross-sectional images of pig cut meat in the test set are tested, and the test results of the automatic positioning method of the pig carcass measurement position are shown in Table 1 below, where the Mean Intersection over Union (Mean Intersection over Union, Miou) and the Dice Similarity Coefficient (Dice Similariy Coefficient, DSC) are commonly used evaluation indicators for segmentation tasks, which are used to test the accuracy of the segmentation model results. The positioning accuracy is used to test the accuracy of the cut meat fat measurement position obtained after the segmentation result passes through the feature analysis module. The standard is that the deviation from the manually marked measurement position is within 1 cm, which is accurate.

[0062] Table 1 Model test results

[0063]

[0064] According to the test results, the pig meat feature recognition and segmentation model constructed by combining Unet and attention modules achieved the best segmentation accuracy, and the final cross-sectional fat thickness measurement accuracy was also improved to a certain extent.

[0065] Therefore, the present invention adopts the above-mentioned binocular vision measurement method of the cross-sectional fat thickness of Chinese meat based on deep learning. After obtaining the left and right images through the binocular vision system, the feature points in the feature areas of the left and right images are matched and reconstructed to restore the three-dimensional depth information of the points to be measured, and finally a more accurate cross-sectional fat thickness is obtained. The method combines deep learning and digital image processing technology, by marking the cross-sectional data information of Chinese meat, inputting the improved UNET network model, segmenting the feature area, and obtaining the feature point information. After that, the Chinese meat is classified, processed and calculated through the Chinese meat triangle meat feature area determination method, which can effectively improve the degree of automation in the industrial processing of pig cut meat.

Claims

1. A binocular vision measurement method for fat thickness of middle section of pig meat based on deep learning, characterized in that: The following steps are involved: Step 1: Use a binocular acquisition system to collect images of the cross section of the middle pork meat, and obtain the cross section images of the middle pork meat as a data set. The data set is divided into a training set, a test set, and a validation set. According to the definition and regulations of cut meat in the national standard NY / T3380-2018, combined with the actual grading and measurement needs of the enterprise, the fat thickness measurement position of the cross section image of the middle pork meat is determined as the subcutaneous fat thickness below the triangular muscle near the rib side, that is, the subcutaneous fat thickness at the tip of the triangular meat. Step 2: Build an improved UNet network by adding an attention mechanism; The unet network is used as the basic network framework. CA modules and SA modules are added in each downsampling process. The CA modules and SA modules are connected in series to construct an improved UNet network. After the cross-section image of pork in the training set is input into the improved U-Net network, the convolution feature map before each layer of downsampling in the encoding network is used as the input of the CA module. The CA module first performs global average pooling on the input convolution feature map to obtain a 1×1×c vector, where c is the number of channels of the input feature map, leaving only the dimensional information of each channel. Then the weight of each channel is obtained through self-learning, and finally the convolution feature map is multiplied by the corresponding channel weight to obtain the weighted feature map. The obtained weighted feature map is input into the SA module to compress the output feature map. The compressed feature map is subjected to maximum pooling and average pooling to obtain two one-dimensional feature maps. Then the two one-dimensional feature maps are convolved to reduce them to one channel, and then the feature map is output through an activation function. Finally, the output feature map is multiplied by the original image to return to the original dimension size. Step 3: After the improved UNet network reaches a convergence state through training iterations, a trained improved UNet network is obtained; the cross-section image of the pork in the validation set is introduced into the trained improved UNet network to obtain the segmentation result; the edge of the segmented meat is preliminarily extracted, and the binary image segmented by the trained improved UNet network is defined as a horizontal line structure and horizontal line corrosion is performed to prevent the loss of boundary pixels; Remove connected areas with an area less than 500 to prevent small noise points from existing in the binary image; smooth the processed binary image to prevent excessive jagged edges; Step 4: extract the edge of the feature area of ​​the pig middle meat cross-section image collected in step 1, and obtain the triangular meat feature area and pig skin edge data on the pig middle meat cross-section; after feature recognition through the trained improved Unet network, according to the situation where the triangular meat feature area on the pig middle meat cross-section will appear, establish a triangular meat feature area determination method for hierarchical classification: obtain the feature points of the upper and lower edges L1 and L2 of the triangular meat feature area on the pig middle meat cross-section through the edge extraction of the triangular meat feature area, obtain straight lines l1 and l2 through fitting, and obtain the upper and lower edge fitting intersection A1 (a1, b1); the difference u=|a1-a2| is made between the rightmost endpoint A2 (a2, b2) of the triangular meat feature area on the pig middle meat cross-section in the binary image segmented by the trained improved Unet network and the horizontal coordinate of the upper and lower edge fitting intersection A1, if u is less than the threshold value 2, this type of segmented meat is positioned at level I; if 2≤u<3, this type of segmented meat is positioned at level II; if u≥3, this type of segmented meat is positioned at level III; Step 5: Based on the analysis of the distribution of feature points of the middle section of pig meat, a feature point matching method based on polar line constraint combined with brute force matching is proposed, that is, the tip feature points of the triangular meat feature area are matched with the feature points on the upper edge of the pig skin, and a regional classification feature matching method is proposed: for the tip feature points of the triangular meat feature area on the middle section of pig meat, brute force matching is directly used, and polar line constraint matching is performed on the feature points on the upper edge of the pig skin, so as to solve the problem of incorrect matching results due to the small number of tip feature points in the triangular meat feature area, and greatly improve the matching efficiency; then the triangular reconstruction method is used to realize the reconstruction of the three-dimensional information of the middle section of pig meat at the real scale; Step 6: After obtaining the tip feature point A1 of the triangular meat characteristic area on the middle section of the pig, it is necessary to extract the feature points of the upper edge of the pig skin. Combined with the definition of the fat thickness measurement of the middle section of the pig in step 1, the pig skin upper edge coordinate point B that is in a normal vector direction relationship with the tip feature point A of the triangular meat characteristic area on the middle section of the pig must fall near the pig skin boundary below point A; on the two-dimensional section of the middle section of the pig, a plane rectangular coordinate system is established with the tip feature point A of the triangular meat characteristic area as the common origin, and two mutually perpendicular axes are established through point A, wherein the horizontal axis is the x-axis, the right direction is the positive direction, and the vertical axis is the y-axis, the upward direction is the positive direction; search vertically downward along the y-axis until the coordinate point intersecting with the contour of the upper boundary of the pig skin is marked as C0; with point C0 as the reference point, search for n points to the left and right along the upper boundary of the pig skin at the same time, that is, search along the upper boundary L of the pig skin instead of horizontally along the x-axis direction, then the feature point of the upper boundary of the pig skin is C 0-n To C 0+n ; and point B, which is in a normal direction to point A, will fall on point C 0-n To C 0+n Within the range; after multiple experiments, it is found that when n is 50, the measurement results after the system completes the 3D reconstruction are more accurate and the stability is higher; Step 7: After completing the reconstruction of the three-dimensional information of the feature points, calculate the spatial distance in the normal direction of the tip feature point of the triangular meat feature area to the feature point on the upper edge of the pig skin, and obtain the measurement result of the fat thickness from the spatial distribution relationship; determine the three-dimensional coordinates of the tip key feature point of the triangular meat area feature as L (x, y, z), and perform a straight line fitting on the upper edge point of the pig skin to obtain the three-dimensional straight line equation ax + by + cz + d = 0; where a, b, c, d are constants used to describe the slope, direction and distance from the origin of the fitted line; define the upper edge feature points of the pigskin as a straight line m after fitting, and calculate the goodness of fit R of the straight line m 2 : Among them, yi is the vertical coordinate of the discrete point, is the average value of the vertical coordinates of the discrete points, is the vertical coordinate of the discrete points after fitting; R 2 is a dimensionless coefficient with a value range of (0, 1), R 2 The closer it is to 1, the better the model fit is. 2 The closer it is to 0, the worse the model fitting effect; If the straight line m fits well R 2 ≥0.7, it means that the spatial distribution of the feature points on the upper edge of the pigskin is close to a linear distribution, and the fitting effect is good. At this time, the calculation method of fat thickness is: Among them, P is any point on the pig skin upper edge fitting line m, A is the feature point of the triangle meat tip, and S0 is the unit direction vector of the line m; If the straight line m fits well R 2 <0.7, it means that the straight line fitting effect of the edge point is poor, and the spatial position of the edge feature points on the pig skin is not distributed in a linear manner. The calculation formula for the cross-sectional fat thickness result is as follows: Among them, (x, y, z) is the three-dimensional coordinates of the feature point of the triangle meat tip, (x i ,y i ,z i ) are the three-dimensional coordinates of the discrete feature points on the upper edge of the pigskin.

Images