Sheep intramuscular fat content estimation method and system based on ultrasonic image

Abstract

The invention provides a sheep intramuscular fat content estimation method and system based on an ultrasonic image. The method comprises the following steps: acquiring the ultrasonic image; performing target area segmentation and marking on the ultrasonic image to obtain a target area mask; splicing the ultrasonic image with the target area mask to obtain a feature image; extracting image features of the feature image through a convolutional neural network; and on the basis of the image features, predicting through an MLP model to obtain an intramuscular fat content value. The IMF content can be objectively analyzed, and the accuracy and reliability of IMF content prediction are improved.

Description

Technical Field

[0001] The invention relates to the technical field of intramuscular fat content determination, and in particular to a method for estimating sheep intramuscular fat content based on ultrasonic images. Background Art

[0002] Intramuscular fat (IMF) refers to the fat present in muscle tissue, which is one of the key factors affecting the quality of mutton. In animal husbandry production, farmers can detect and increase the IMF content in mutton by selecting breeds with high IMF genetic potential or through specific feeding and management methods, thereby improving the quality and market competitiveness of mutton.

[0003] Traditional IMF determination methods include sensory evaluation and chemical extraction. Among them, the sensory evaluation method is inevitably affected by multiple factors such as the environment and the subjective feelings of the grader, and errors are inevitable; the chemical extraction method uses Soxhlet extraction, which is accurate but requires the destruction of the sample, so it cannot be used for live testing, and the extraction process takes several hours, which is not conducive to large-scale meat testing. In addition, organic solvents such as ether and petroleum ether are also required during the determination process, which is not only costly, but also requires special safety measures. Summary of the invention

[0004] The invention provides a method for estimating the intramuscular fat content of sheep based on ultrasonic images, so as to solve the defects of the prior art.

[0005] The present invention provides a method for estimating the intramuscular fat content of sheep based on ultrasonic images, comprising: S1: Acquire ultrasound images; S2: segmenting and marking the target area of ​​the ultrasound image to obtain a target area mask; S3: splicing the ultrasound image with the target area mask to obtain a feature image; S4: extracting image features of the feature image through a convolutional neural network; S5: Based on the image features, the intramuscular fat content value is predicted by the MLP model.

[0006] According to the method for estimating intramuscular fat content in sheep based on ultrasound images provided by the present invention, the target area in step S2 is the eye muscle area.

[0007] According to a method for estimating intramuscular fat content of sheep based on ultrasound images provided by the present invention, step S2 further comprises: S21: obtaining a region of interest from the ultrasound image; S22: marking the target area within the area of ​​interest to obtain a target area label; S23: Training a U-Net image segmentation model using the target region label, inputting the ultrasound image into the image segmentation model, and obtaining a target region mask.

[0008] According to the method for estimating intramuscular fat content in sheep based on ultrasound images provided by the present invention, the characteristic image in step S3 specifically includes a first characteristic image and a second characteristic image.

[0009] According to a method for estimating intramuscular fat content in sheep based on ultrasound images provided by the present invention, step S3 further comprises: S31: converting the ultrasound image into grayscale values ​​to obtain a single-channel grayscale image; S32: Channel-joining the ultrasound image and the target area mask to obtain the first feature image; channel-joining the grayscale image and the target area mask to obtain the second feature image.

[0010] According to the method for estimating intramuscular fat content in sheep based on ultrasound images provided by the present invention, the image features in step S4 specifically include texture features, brightness features, and area features.

[0011] According to a method for estimating intramuscular fat content of sheep based on ultrasound images provided by the present invention, step S4 further comprises: S41: Establish a convolutional neural network based on convolution and channel attention mechanism; S42: inputting the first feature image into the convolutional neural network to obtain the texture feature; inputting the second feature image into the convolutional neural network to obtain the brightness feature; S43: Counting pixels of the target region on the mask feature map with the target region mask to obtain the area of ​​the target region; S44: Input the target region area into the fully connected layer to obtain area features.

[0012] According to a method for estimating intramuscular fat content of sheep based on ultrasound images provided by the present invention, step S5 further comprises: S51: fusing multiple features in the image features, and training the MLP model through the fused features; S52: Input the feature image into the MLP model to predict the intramuscular fat content value.

[0013] According to a sheep intramuscular fat content estimation method based on ultrasound images provided by the present invention, in step S51, the expression of the objective function of the MLP model is:

[0014] in, is the training target of the MLP model, is the intramuscular fat content value predicted by the MLP model, It is the actual intramuscular fat content value.

[0015] The present invention also provides a sheep intramuscular fat content estimation system based on ultrasonic images, comprising: Acquisition module: used to acquire ultrasound images; Annotation module: used for segmenting and annotating the target area of ​​the ultrasound image obtained by the acquisition module to obtain a target area mask; A splicing module: used for splicing the ultrasound image with the target area mask obtained by the annotation module to obtain a feature image; Extraction module: used for extracting image features of the feature image through a convolutional neural network; Prediction module: used to predict the intramuscular fat content value through the MLP model based on the image features obtained by the extraction module.

[0016] The present invention provides a method and system for estimating the intramuscular fat content of sheep based on ultrasound images, which aims to seek a faster and more cost-effective live detection IMF technology, which is of great significance for real-time monitoring and evaluation of IMF content. As a non-invasive and real-time measurement method, ultrasound technology is widely used in the detection of biological signs. The propagation speed of ultrasound in muscle and fat is different, and its wavelength is smaller than the minimum diameter of intramuscular fat particles. Therefore, ultrasound can measure all intramuscular fat particles and reflect them in the image to form a texture image. In addition, deep learning technology can effectively extract and process image features and has been widely used in image-based classification and regression tasks. The present invention uses a deep learning method and sheep back ultrasound images to achieve the estimation of IMF content in live sheep.

[0017] Compared with the traditional chemical extraction method, the present invention does not need to destroy the sample, so it can be used for living body detection, which not only reduces the detection cost, but also avoids the inconvenience caused by sample destruction. In addition, the present invention uses ultrasonic images and machine learning technology, which can significantly shorten the detection time and improve the detection efficiency, which is conducive to large-scale meat detection. Secondly, the present invention adopts image processing and machine learning technology, which can provide more objective and accurate IMF content estimation and reduce errors caused by human factors. Compared with the chemical extraction method that needs to use organic solvents, the cost and safety risks are reduced. The present invention can extract multiple features of ultrasonic images, such as texture features, brightness features, area features, etc., to more comprehensively analyze the IMF content and improve the accuracy and reliability of prediction. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] In order to more clearly illustrate the technical solutions in the present invention or the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0019] Figure 1 A schematic flow chart of a method for estimating intramuscular fat content in sheep based on ultrasound images provided in an embodiment of the present invention; Figure 2 A schematic diagram of the structure of a sheep intramuscular fat content estimation system based on ultrasound images provided by an embodiment of the present invention; Figure 3 Schematic diagram of the segmentation effect of target area segmentation in an embodiment of the present invention; Figure 4 A schematic diagram showing the comparison between the IMF estimation value and the true value for the test set provided by an embodiment of the present invention; Figure 5 An IMF estimation error histogram for a test set provided by an embodiment of the present invention.

[0020] Attached figures: 100, acquisition module; 200, labeling module; 300, splicing module; 400, extraction module; 500, prediction module. DETAILED DESCRIPTION

[0021] In order to make the purpose, technical solution and advantages of the present invention clearer, the technical solution of the present invention will be clearly and completely described below in conjunction with the drawings in the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments, and they should not be understood as limitations on the present invention. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention. In the description of the present invention, it should be understood that the terms used are only for descriptive purposes and cannot be understood as indicating or implying relative importance.

[0022] Combine the following Figures 1 to 3 Embodiments of the present invention are described.

[0023] like Figure 1 As shown, the present invention provides a method for estimating the intramuscular fat content of sheep based on ultrasonic images, comprising: S1: Acquire ultrasound images.

[0024] Furthermore, in step S1, an ultrasonic image of the sheep to be tested is first obtained. When capturing the ultrasonic image of the sheep, an ultrasonic device specially designed for animals is used. In this embodiment, a veterinary B-ultrasound machine is used. High-frequency sound waves are sent and received by the device to form a real-time dynamic image of the internal structure of the sheep on the display.

[0025] S2: Segment and label the target area of ​​the ultrasound image to obtain a target area mask.

[0026] Wherein, the target area in step S2 is the eye muscle area.

[0027] Furthermore, the loin area refers to the cross-sectional area of ​​the longissimus dorsi muscle of livestock. Since the loin area trait has a strong correlation with the meat production performance of livestock, it is particularly important in breeding. Therefore, the loin area is used as the target area in this embodiment.

[0028] Wherein, step S2 further comprises: S21: Obtaining a region of interest from the ultrasound image.

[0029] Furthermore, since there is a noise area at a fixed position of the user interface in the original image output by the ultrasound device, in order to avoid interference from the noise area, a 642×518 pixel area is cropped from the original ultrasound image as a region of interest (ROI).

[0030] S22: Mark the target area in the region of interest to obtain a target area label.

[0031] Based on the image corresponding to the region of interest (ROI) obtained in step S21, the loin area of ​​the ROI is annotated in step S22, the region edge is divided, and the region mask is obtained as a label for subsequent segmentation model training.

[0032] S23: Training a U-Net image segmentation model using the target region label, inputting the ultrasound image into the image segmentation model, and obtaining a target region mask.

[0033] Specifically, according to the input image specification of 3×642×518 pixels, a U-Net image segmentation model is first established in step S23. In this embodiment, each module in the U-Net model is modified as follows to adapt to the input image specification: EncoderBlock 1: the convolution kernel size is 4×4, 4×4, and the padding specification is (0, 0), (0, 0); Encoder Block 2: the convolution kernel size is 4×3, 4×3, and the padding specification is (0, 1), (0, 1); Encoder Block 3: the convolution kernel size is 4×3, 4×3, and the padding specification is (0, 1), (0, 1); Encoder Block 4: the convolution kernel size is 4×3, 4×3, and the padding specification is (0, 1), (0, 1); Bottleneck: the convolution kernel size is 3×3, 3×3, and the padding specification is (1, 1), (1, 1); Decoder Block 4: The convolution kernel size is 4×3, 4×3, and the padding specification is (3, 1), (3, 1); Decoder Block 3: The convolution kernel size is 4×3, 4×3, and the padding specification is (3, 1), (3, 1); Decoder Block 2: The convolution kernel size is 4×3, 4×3, and the padding specification is (3, 1), (3, 1); Decoder Block 1: The convolution kernel size is 4×3, 4×3, and the padding specification is (3, 3), (3, 3).

[0034] Finally, the output of the U-Net model is a 1×642×518 feature map, in which an element value of 1 indicates that the pixel at the corresponding position is located in the loin region, and an element value of 0 indicates that the pixel at the corresponding position is located outside the loin region. A schematic diagram of the loin region segmentation effect obtained in a specific embodiment is shown in FIG. Figure 3 As shown, Figure 3 A is the original image and the image of the region of interest obtained in step S22, Figure 3 B is the true loin mask corresponding to the original image. The red area is the target area, i.e. the eye muscle area. Figure 3 C is a schematic diagram of the comparison between the mask output by the model and the original image, where the green part is the predicted target area. Figure 3 D is the final output mask of the model.

[0035] Furthermore, the objective function of the above image segmentation model is expressed as:

[0036] in, Train the objective function for the image segmentation model, is the predicted mask of the image segmentation model, is the true mask of the image segmentation model, is the image area corresponding to the predicted mask of the image segmentation model, is the image area corresponding to the true mask of the image segmentation model, is the intersection area of ​​the predicted mask and the true mask of the image segmentation model, A constant used to prevent the result from being 0.

[0037] In the above formula, let and Denote the predicted mask and the true mask respectively, represents the intersection area, represents the union region, and Represent their areas respectively, It is set to 1.0 to prevent the denominator from being zero. In this embodiment, the segmentation model is trained separately.

[0038] Furthermore, the Dice loss function and Intersection over Union (IoU) are used as evaluation indicators in this stage. The Dice loss function is used to measure the similarity between the predicted mask and the true mask. The smaller the value of the Dice loss function, the closer the predicted result is to the true result. The Intersection over Union (IoU) is another important indicator for evaluating the accuracy of the segmentation result. Its value is between 0 and 1. The larger the value, the more accurate the prediction result. In this embodiment, the average Dice coefficient of the model on the test set is 0.9461, and the average IoU is 0.8994. The model can better segment the Loin area of ​​the sheep back ultrasound image.

[0039] S3: splicing the ultrasound image with the target area mask to obtain a feature image.

[0040] The characteristic image in step S3 specifically includes a first characteristic image and a second characteristic image.

[0041] Wherein, step S3 further comprises: S31: converting the grayscale value of the ultrasound image to obtain a single-channel grayscale image; In step S31, the three-channel values ​​of the original RGB ultrasound image are first converted as follows to obtain a single-channel grayscale ultrasound image: .

[0042] S32: Channel-joining the ultrasound image and the target area mask to obtain the first feature image; channel-joining the grayscale image and the target area mask to obtain the second feature image.

[0043] In step S32, the original RGB ultrasound image and the loin mask obtained in step S2 are first channel-joined to obtain a 4×642×518 feature map; secondly, the grayscale ultrasound image and the loin mask are channel-joined to obtain a 2×642×518 feature map.

[0044] S4: Extracting image features of the feature image through a convolutional neural network.

[0045] The image features in step S4 specifically include texture features, brightness features, and area features.

[0046] Wherein, step S4 further comprises: S41: Establish a convolutional neural network based on convolution and channel attention mechanism.

[0047] Furthermore, in step S41, a convolutional neural network with four convolutional modules stacked is first established, each convolutional module is composed of a deep convolutional layer and a point-by-point convolutional layer, the convolution kernel size of the deep convolutional layer is 3×3, the padding specification is (1, 1), and the step size is 2; the convolution kernel size of the point-by-point convolutional layer is 1×1, the padding specification is (1, 1), and the step size is 1. The output specification of the established network model is C×5×4, where C is a variable related to the number of input feature map channels. When the number of input feature map channels is 4, C=32, and when the number of input feature map channels is 2, C=16.

[0048] Secondly, the present invention also sets an SE Block substructure in the above-mentioned convolutional neural network for adaptively reweighting the features of each channel. The features of each channel are weighted through global average pooling and two fully connected layers. The value of each channel of the input feature map is multiplied by the channel weight. In addition, the feature map output by the SE Block needs to undergo a linear transformation, which is implemented by a fully connected layer. Finally, the input of the SE Block is the feature map output by the convolutional neural network model, and the output specification is consistent with the input specification.

[0049] S42: Input the first feature image into the convolutional neural network to obtain the texture feature; input the second feature image into the convolutional neural network to obtain the brightness feature.

[0050] In step S42, the 4×642×518 feature map obtained in step S32, that is, the first feature map, is first input into the established convolutional neural network model to obtain a 64-dimensional texture feature. Subsequently, the 2×642×518 feature map obtained in step S32, that is, the second feature map, is input into the convolutional neural network model to obtain a 64-dimensional brightness feature. After obtaining the two features, the texture feature and the brightness feature need to be spliced ​​and input into a 3-layer MLP for feature fusion. In a specific embodiment, each layer of the MLP is composed of a fully connected layer and a LeakyReLU activation function, and the specifications of each fully connected layer are as follows: (128, 64), (64, 64), (64, 64).

[0051] S43: Count the target region pixels on the mask feature map with the target region mask to obtain the target region area.

[0052] S44: Input the target region area into the fully connected layer to obtain area features.

[0053] In steps S33 to S34, the mask feature map obtained in step S23 is used to count the pixels in the loin area as the area of ​​the loin area, and then input into a fully connected layer to obtain the area feature. The specification of the fully connected layer is (1,16).

[0054] S5: Based on the image features, the intramuscular fat content value is predicted by the MLP model.

[0055] Wherein, step S5 further comprises: S51: Fusing multiple features in the image features, and training the MLP model using the fused features.

[0056] In step S51, the fused texture, brightness and area features are first input into a three-layer MLP for feature fusion. Each layer of the MLP consists of a fully connected layer and a LeakyReLU activation function, and the specifications of each fully connected layer are (80, 64), (64, 64), (64, 1).

[0057] Wherein, in step S51, the expression of the objective function of the MLP model is:

[0058] in, is the training target of the MLP model, is the intramuscular fat content value predicted by the MLP model, It is the actual intramuscular fat content value.

[0059] S52: Input the feature image into the MLP model to predict the intramuscular fat content value.

[0060] Subsequently, the MLP model in step S51 can output an estimated value of IMF. It should be noted that the IMF estimation model training is performed separately from the segmentation model training, that is, the segmentation model is trained in advance, and then the regional mask output by the trained segmentation model is used to generate a feature map, which is input into the IMF estimation model and trained.

[0061] like Figure 2 As shown, the present invention also provides a sheep intramuscular fat content estimation system based on ultrasound images, comprising: Acquisition module 100: used to acquire ultrasound images; The labeling module 200 is used to segment and label the target area of ​​the ultrasound image obtained by the acquisition module 100 to obtain a target area mask; The stitching module 300 is used to stitch the ultrasound image with the target area mask obtained by the labeling module 200 to obtain a feature image; Extraction module 400: used to extract image features of the feature image through a convolutional neural network; Prediction module 500: used to predict the intramuscular fat content value through the MLP model based on the image features obtained by the extraction module 400.

[0062] The device embodiments described above are merely illustrative, wherein the units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, they may be located in one place, or they may be distributed on multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the scheme of this embodiment. Ordinary technicians in this field can understand and implement it without paying creative labor.

[0063] Through the description of the above implementation methods, those skilled in the art can clearly understand that each implementation method can be implemented by means of software plus a necessary general hardware platform, and of course, can also be implemented by hardware. Based on this understanding, the above technical solution is essentially or the part that contributes to the prior art can be embodied in the form of a software product, and the computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, a disk, an optical disk, etc., including a number of instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to execute the methods described in each embodiment or some parts of the embodiments.

[0064] According to the following Figures 4 to 5 Describe the performance effects of the methods and models provided by the present invention in specific embodiments.

[0065] The various indicators of the model obtained by the method provided by the present invention on the test set are as follows: MAE is 0.322, MAPE is 0.070, MSE is 0.178, and RMSE is 0.422.

[0066] Figure 4 and Figure 5 The results of IMF estimation using the model of the present invention are shown. Figure 4 In the figure, the blue dots represent the true and predicted IMF values ​​of each test sample, the horizontal axis is the percentage of the true IMF, the vertical axis is the percentage of the predicted IMF, and the red dotted line is the best fit line, which depicts the linear trend between the predicted value and the true value. It can be observed from the figure that most of the blue dots are closely around the red dotted line, which shows that the IMF predicted value of the model is well consistent with the true value, and the model prediction is relatively accurate. Figure 4 The positive correlation between the predicted value and the true value is intuitively demonstrated, showing the ability of the model to fit the true value of the IMF. Figure 5 The mean absolute error by interval is shown, showing the distribution of IMF estimation errors in different ground truth mean absolute error (MAE) intervals. The horizontal axis represents six different intervals of true MAE, increasing from 3.0-3.5% to 6.0-6.5%, and the vertical axis reflects the percentage of IMF estimation errors in each interval, which is represented by Figure 5 It can be seen that the forecast errors in each interval show a high degree of consistency, which indicates that there is no obvious bias in the model's IMF estimates for samples in different intervals.

[0067] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A method for estimating intramuscular fat content in sheep based on ultrasound images, characterized in that: include: S1: Acquire ultrasound images; S2: segmenting and marking the target area of ​​the ultrasound image to obtain a target area mask; S3: splicing the ultrasound image with the target area mask to obtain a feature image; S4: extracting image features of the feature image through a convolutional neural network; S5: Based on the image features, the intramuscular fat content value is predicted by the MLP model.

2. The method for estimating intramuscular fat content of sheep based on ultrasonic images according to claim 1, characterized in that: The target area in step S2 is the eye muscle area.

3. The method for estimating intramuscular fat content of sheep based on ultrasonic images according to claim 1, characterized in that: Step S2 further comprises: S21: obtaining a region of interest from the ultrasound image; S22: marking the target area within the area of ​​interest to obtain a target area label; S23: Training a U-Net image segmentation model using the target region label, inputting the ultrasound image into the image segmentation model, and obtaining a target region mask.

4. The method for estimating intramuscular fat content of sheep based on ultrasonic images according to claim 1, characterized in that: The characteristic image in step S3 specifically includes a first characteristic image and a second characteristic image.

5. The method for estimating intramuscular fat content of sheep based on ultrasonic images according to claim 4, characterized in that: Step S3 further comprises: S31: converting the ultrasound image into grayscale values ​​to obtain a single-channel grayscale image; S32: Channel-joining the ultrasound image and the target area mask to obtain the first feature image; channel-joining the grayscale image and the target area mask to obtain the second feature image.

6. The method for estimating sheep intramuscular fat content based on ultrasonic images according to claim 5, characterized in that: The image features in step S4 specifically include texture features, brightness features, and area features.

7. The method for estimating intramuscular fat content of sheep based on ultrasonic images according to claim 6, characterized in that: Step S4 further comprises: S41: Establish a convolutional neural network based on convolution and channel attention mechanism; S42: inputting the first feature image into the convolutional neural network to obtain the texture feature; inputting the second feature image into the convolutional neural network to obtain the brightness feature; S43: Counting pixels of the target region on the mask feature map with the target region mask to obtain the area of ​​the target region; S44: Input the target region area into the fully connected layer to obtain area features.

8. The method for estimating intramuscular fat content of sheep based on ultrasonic images according to claim 1, characterized in that: Step S5 further comprises: S51: fusing multiple features in the image features, and training the MLP model through the fused features; S52: Input the feature image into the MLP model to predict the intramuscular fat content value.

9. The method for estimating intramuscular fat content of sheep based on ultrasonic images according to claim 1, characterized in that: In step S51, the objective function of the MLP model is expressed as: in, is the training target of the MLP model, is the intramuscular fat content value predicted by the MLP model, It is the actual intramuscular fat content value.

10. A sheep intramuscular fat content estimation system based on ultrasound images, used to execute a sheep intramuscular fat content estimation method based on ultrasound images as claimed in any one of claims 1 to 9, characterized in that: include: Acquisition module: used to acquire ultrasound images; Annotation module: used for segmenting and annotating the target area of ​​the ultrasound image obtained by the acquisition module to obtain a target area mask; A splicing module: used for splicing the ultrasound image with the target area mask obtained by the annotation module to obtain a feature image; Extraction module: used for extracting image features of the feature image through a convolutional neural network; Prediction module: used to predict the intramuscular fat content value through the MLP model based on the image features obtained by the extraction module.

Images