Method, system and computer readable medium for assessing meat quality

Abstract

The described embodiments relate to a method of determining a quality assessment metric for a carcass. The method includes determining video data comprising a sequence of frames, wherein at least some of the frames depict a portion of a carcass to be quality assessed. For a first frame of the sequence of frames, the method comprises a) determining a frame suitability score for assessing a first trait; b) in response to the frame suitability score being greater than a threshold frame suitability score for the first trait, determining the frame as a suitable frame for assessing the first trait; c) determining a region of interest for evaluating the first character in the frame through a segmentation model; d) determining a predicted score of the first character based on the determined region of interest by a first character prediction model; and e) determining a rating confidence rating for the first trait. The method further comprises: in response to determining that the rating confidence rating of the first trait has not exceeded a rating confidence threshold of the first trait, performing steps a) to e) on subsequent frames in the sequence; and in response to determining that the rating confidence rating for the first trait has exceeded the rating confidence threshold for the first trait, determining a quality rating metric for the first trait based on the determined predicted score for the first trait suitable for a frame.

Description

Technical Field

[0001] The described embodiments generally relate to methods, systems, and computer-readable media for assessing the meat quality of animal carcasses, such as frozen carcasses. Background Technology

[0002] In the field of meat quality assessment, carcasses are typically evaluated through inspection to determine the quality of the meat they contain.

[0003] Existing technologies for assessing meat quality often involve using specialized imaging devices designed for meat quality assessment to capture images of the assessment area of ​​the carcass.

[0004] The aim is to address or improve one or more drawbacks or deficiencies associated with such existing technologies, or at least provide a useful alternative.

[0005] Throughout this specification, the word “comprise” or variations such as “comprises” or “comprising” shall be understood to imply inclusion of the stated elements, integers or steps, or groups of elements, groups of integers or groups of steps, but not to exclude any other elements, integers or steps, or groups of elements, groups of integers or groups of steps.

[0006] Any discussion of documents, actions, materials, devices, articles, or the like included in this specification shall not be construed as an admission that any or all of these matters constitute part of the prior art or common knowledge in the field relating to this disclosure prior to the priority date of each appended claim. Summary of the Invention

[0007] Some embodiments relate to a method comprising: determining video data comprising a sequence of frames, wherein at least some of these frames depict a portion of a body to be quality rated; for a first frame of the frame sequence: a) determining a frame fitness score for rating a first trait; b) in response to the frame fitness score being greater than a threshold frame fitness score for the first trait, determining the frame as a fit frame for rating the first trait; c) determining a region of interest in the frame for rating the first trait using a segmentation model; d) determining a predicted score for the first trait based on the determined region of interest using a first trait prediction model; and e) determining a rating confidence rating for the first trait; in response to determining that the rating confidence rating for the first trait has not yet exceeded the rating confidence threshold for the first trait, performing steps a) through e) on subsequent frames in the sequence; and in response to determining that the rating confidence rating for the first trait has exceeded the rating confidence threshold for the first trait, determining a quality rating metric for the first trait based on the predicted score for the first trait of the determined fit frame.

[0008] The method may further include determining the assessment confidence rating for the first trait, which may include: determining one or more of the following: (i) the number of fit frames determined for the first trait; and (ii) a function of the frame fit rating for the first trait for the determined fit frames.

[0009] The method may further include determining the confidence rating of the first trait, which may include: determining a function for the predicted score of the first trait for a given fit frame. The function for determining the predicted score of the first trait for a given fit frame may include determining an arithmetic mean taken over the following intervals: [ B min , B max ],in B min Defined by the following functions:

[0010] And among them B max Defined by the following functions:

[0011] in P A The predicted value is the first percentile of the predicted score for the primary trait; where P B The predicted value is the second percentile of the predicted score for the primary trait; where Here, is the tolerance parameter; and where IQR It is by IQR = P A – P B Defined interquartile range.

[0012] This method may further include excluding the lowest predicted score from one or more of the following. p 丢弃 Percentages: (i) a measure of glare deficiency; (ii) a measure of sharpness; and (iii) a measure of segment size; where p 丢弃 This is a configurable parameter. The method may further include: determining a frame count threshold for the frame sequence; and, in response to the determination that the number of frames exceeds the frame count threshold, discarding the frame fitness score, prediction score, and confidence rating of the earliest determined frame. The first percentile may be a smaller percentile than the second percentile.

[0013] The method may further include: performing steps c) and d) in response to determining the frame as a suitable frame for assessing the first trait; and omitting steps c) and d) in response to determining the frame as not a suitable frame for assessing the first trait.

[0014] Determining the video data may include receiving the video stream.

[0015] Frame fit scoring can be based on one or more of the following: (i) a measure of glare deficiency, (ii) a measure of sharpness, (iii) a measure of segment size, (iv) a measure of segment edges of the region of interest, and (v) a measure of segment roundness of the region of interest.

[0016] Determining a frame fit score may include determining the frame fit score according to the following equation: , in For suitability scoring; Rating type for the first frame r The value; For targeted traits t rating types r The proportionality coefficient; and For targeted traits t rating r The offset coefficient.

[0017] The method may further include: outputting the quality assessment metric of the primary trait to the user interface.

[0018] The primary trait may include any of the following: marbling; marbling fineness; rib eye area; rib fat ratio; intramuscular fat; fat color; and meat color.

[0019] The method may further include: for the first frame of the frame sequence: f) determining a frame fitness score for assessing a second trait, wherein the second trait is different from the first trait; g) in response to the frame fitness score being greater than a threshold frame fitness score for the second trait, determining the frame as a fit frame for assessing the second trait; h) determining a region of interest (ROI) in the frame for assessing the second trait using a second segmentation model; i) determining a predicted score for the second trait based on the determined ROI using a second trait prediction model; and j) determining a rating confidence level for the second trait; in response to determining that the rating confidence level for the second trait has not yet exceeded the rating confidence threshold for the second trait, performing steps a) to e) on subsequent frames in the sequence; and in response to determining that the rating confidence level for the second trait has exceeded the rating confidence threshold for the second trait, determining a quality assessment metric for the second trait based on the predicted score for the second trait of the determined fit frame. Secondary traits may differ from primary traits and include any of the following: marbling; marbling fineness; rib eye area; rib fat thickness; intramuscular fat; fat color and meat color.

[0020] The frame fitness score may be based on one or more frame fitness ratings, each frame fitness rating indicating the fitness of a frame relative to a specific metric. For each of the one or more frame fitness ratings, the method further includes: comparing the frame fitness rating with a corresponding metric-specific rating threshold; and in response to determining that the frame fitness rating does not meet the threshold, determining that the frame is unsuitable and excluding the frame from further processing.

[0021] In some embodiments, the step of determining the region of interest (ROI) in the frame for evaluating a first trait using a segmentation model includes: receiving a point cloud and isolating regions in the point cloud corresponding to the ROI; determining a plane fitted to the points in the point cloud; projecting the points in the point cloud onto the plane to generate a plurality of projected plane points; rasterizing the projected plane points into a high-resolution image, wherein each pixel unit of the high-resolution image corresponds to a region of the plane in the real-world region; converting the high-resolution image into a binary image; applying morphological operations to fill holes in the binary image; identifying the contour of the ROI in the binary image; and, in response to identifying only a single contour, converting the area inside the contour into a real-world area estimate of the ROI.

[0022] Some embodiments relate to a meat assessment system comprising: at least one processor; and a memory accessible by the at least one processor and including computer-executable instructions that, when executed by the at least one processor, cause the system to perform the described methods.

[0023] The system may further include: a user interface configured to display determined quality assessment metrics for the first trait. The system may further include a video capture device controlled by at least one processor for capturing video data.

[0024] Some embodiments relate to a non-transitory machine-readable medium that stores instructions that, when executed by one or more processors, cause an electronic device to perform the described methods. Attached Figure Description

[0025] Figure 1 This is an example illustration of animal products undergoing meat quality assessment based on some embodiments; Figure 2 This is a block diagram of an analytical apparatus for assessing meat quality according to some embodiments; Figure 3 This is a flowchart of a method for assessing meat quality according to some embodiments; and Figure 4A and Figure 4b According to some embodiments Figure 2 Example screenshot of the user interface display of the analysis device. Detailed Implementation

[0026] The described embodiments generally relate to methods, systems, and computer-readable media for assessing the meat quality of animal carcasses, such as frozen carcasses.

[0027] In meat processing environments, it is often beneficial to grade meat carcasses quickly and reliably for further processing. Figure 1 This is an example schematic diagram of a meat product 140 used for meat quality assessment according to some embodiments of an analytical device 110. The analytical device 110 is configured to assess the meat product 140 to determine a quality assessment of one or more traits or characteristics of the meat product 140. For example, traits may include one or more of marbling, marbling fineness, ribeye area, rib fat thickness, intramuscular fat ratio, meat color, fat color, and eye muscle area (EMA). The eye muscle area may be determined in squares (e.g., cm²). 2 The measurement of the costovertebral muscle. Rib fat thickness can refer to the absolute real-world rib fat thickness of meat product 140. In some embodiments, rib fat thickness can include a proportional value compared to another aspect of the geometry of the cut surface of meat product 140. In some embodiments, marbling fineness can include a measure of the size and / or uniformity of fat particles within the EMA. Intramuscular fat ratio may also be referred to as intramuscular fat or IMF.

[0028] Typically, in the described embodiments, a user using the analysis device 110 uses the analysis device 110 to capture video data of the meat product 140. For example, the analysis device 110 may include a video capture component 112. The video data includes a sequence of frames 104, each frame depicting a view or image of the meat product 140. The analysis performed by the analysis device 110 can be performed simultaneously with or near real-time of the video data capture.

[0029] The analysis device 110 is configured to analyze frames of video data to determine quality assessment metrics for a trait.

[0030] Machine learning (ML) models, such as neural networks, can be used to determine quality assessment metrics for traits. The model determines a predicted score for the corresponding trait for each of multiple captured frames. A frame fitness score can be determined for each frame to determine whether the frame is suitable for assessing meat quality. In some embodiments, the determination of frame fitness and the determination of the predicted score for the corresponding trait are performed in parallel or substantially simultaneously.

[0031] The analysis device 110 can determine a confidence rating to determine whether a quality assessment metric should be determined based on the information obtained. For example, the confidence rating may depend on whether a sufficient number of frames (e.g., at least ten) have been obtained and / or whether a sufficient number of suitable frames have been obtained and / or whether the function of the predicted score for the suitable frames meets a threshold, e.g., sufficiently high or sufficiently low.

[0032] Once the analysis device 110 determines the quality assessment metrics for a trait, it provides or outputs these metrics to its user interface 120. For example, the analysis device 110 may present the determined quality assessment metrics on the display screen 130 of the user interface 120, allowing the user to see the determined quality metrics for each trait of the meat product 140. The analysis device 110 may store the determined quality assessment metrics in its memory 210. The analysis device 110 may also send the determined quality assessment metrics to an external device via its communication interface 226.

[0033] Examples of screenshots of the user interface 120 and display screen 130 of the analysis device 110 are shown in Figures 4A to 4B. Figure 4b As shown below, and discussed in more detail.

[0034] Using video for meat quality assessment of meat product 140 allows for high accuracy and / or efficiency. When assessing a series of frames 104, data can be acquired relatively quickly, and / or the determination of desired traits can have improved accuracy when determining them from an array or multiple frames 104. This may be preferable to using a single or individual image, for example, because a single image may be limited by factors such as ambient lighting, lack of sharpness, glare, and / or other factors that may affect the ability to accurately assess meat quality based on a single image. Furthermore, capturing a high-quality single image typically requires pausing the meat processing line, which can result in costly delays in refrigeration time and / or processing output. In contrast, assessing more than one frame from a set of images or videos can provide the advantage of allowing operators to assess meat quality under various lighting and / or imaging conditions without significantly affecting the accuracy and / or efficiency of the assessment.

[0035] The described method provides an effective and / or reliable technique for assessing meat quality. The ability to assess a video stream comprising a sequence of images or frames 104 means that relatively large amounts of data can be captured for analysis within short time intervals. Determining frame suitability on an individual or “frame-by-frame” basis means that operators may not require specialized training to operate the camera, e.g., to accurately position it for image capture, as in cases where assessment is performed on a single image. Therefore, the benefit of using a trained machine learning model of the meat quality assessment module 214 means that operators can be able to capture video of a general area and rely on the functionality of the meat quality assessment module 214 to assess which portions of the video are unsuitable for analysis. This approach can have significant benefits in slaughterhouses, for example, where environmental conditions (such as ambient lighting) are uncontrollable and, in some cases, carcasses may move during the grading process. Furthermore, the described system and method can be particularly beneficial to further processing plants that can benefit from reliably and / or effectively determining the meat quality in the products they receive for further processing. The described system and method can be particularly beneficial to retailers for identifying meat quality at individual cut levels. Furthermore, the described systems and methods can be beneficial in consumer applications to allow for easy grading of purchased meat products, assessment of the value of meat products, and / or assistance in determining optimal cooking times and / or procedures.

[0036] Figure 2 An analysis apparatus 110 according to some embodiments is depicted. As shown, the analysis apparatus 110 may include a video capture device 112 for capturing video data of a meat product to be evaluated. The analysis apparatus 110 may include a user interface 120 for, for example, outputting quality assessment metrics for determining the characteristics of the meat product to a user. The analysis apparatus 110 may be mounted on a bracket 115.

[0037] In some embodiments, the analysis device 110 may be a smartphone, such as a Samsung Galaxy S10. TM A smartphone model. In this type of embodiment, the video capture device 112 may include a digital camera built into the smartphone. In other embodiments, the video capture device 112 may be a separate unit mounted away from the controller 202. In this type of embodiment, the video capture device 112 may include a separate video camera.

[0038] In some embodiments, the video capture device 112 may include a 2D camera configured to capture video. The video capture device 112 may be coupled to a housing (such as a metal housing) configured to hold or retain the video capture device 112 in a specific position such that it captures video from a fixed angle and / or orientation. The fixed angle and / or orientation can be selected to ensure that the video capture device 112 captures an appropriate or suitable view of the meat product. However, it should be understood that the use of a housing is optional, not required.

[0039] In some embodiments, the video capture device 112 may include a 3D camera configured to capture video data with depth information. In such embodiments, the 3D camera may include an Intel Realsensetm camera or an equivalent camera.

[0040] Such 3D cameras can be configured to capture frames of meat carcasses from a variety of different angles and / or orientations. Therefore, the use of a 3D camera eliminates the need for a shield to maintain a fixed angle and / or orientation as described above, and in some embodiments, eliminates the need for a static restraint device 115. For example, a video capture device 112 can be deployed on a robotic arm (not shown), which can help acquire frames at a range of angles and / or orientations. This is particularly useful in meat processing conditions that may include carcasses moving along a chain, as the realization of a larger set of angles and / or orientations can mean that suitable frame acquisition and / or accurate meat assessment can be reliably maintained as the carcass moves along the chain.

[0041] Furthermore, using a 3D camera in the video capture device 112 not only allows for the capture of a series of frames from different angles and / or orientations, but also allows for the capture of relatively high-resolution images, and thus allows for the capture of a much larger amount of information that a 2D camera might allow. This can improve the accuracy of downstream processes, such as selecting genetic populations and determining appropriate treatments for the population based on assessed traits.

[0042] In some embodiments, where at least one of the traits being assessed or measured is the extraocular muscle area (EMA), marbling, costal eye area, costal fat thickness, meat color, fat color, marbling fineness, and / or intramuscular fat percentage, the use of a 3D camera can eliminate the need for a mesh (such as a point mesh, e.g., a point planimeter) that needs to be placed or overlaid on the region of interest of the carcass, such as the costal eye area, to allow for the determination of that region. Using such a mesh can be cumbersome and inefficient, requiring operator skill and time for placement, necessitating hygienic practices to ensure the mesh remains clean, and requiring intervention when moving the mesh from one carcass to another. The use of a 3D camera can be particularly advantageous where the EMA is only one of several different traits being assessed, in which case it may be necessary to capture at least two separate video streams; one video stream with a point mesh positioned appropriately on the region of interest of the carcass, while the other video stream lacks a point mesh (it is neither needed nor suitable for assessing other traits).

[0043] In some embodiments, the video capture device 112 captures video streams to better enable near real-time processing and reduce the impact of memory storage.

[0044] like Figure 2 As shown, the analysis device 110 includes a controller 202 configured to perform analysis and output results. For this purpose, the controller 202 can communicate with the video capture device 112 and / or the user interface 120. The controller 202 includes one or more processors 205 that communicate with the memory 210.

[0045] Processor 205 may be arranged to retrieve data from memory 210 and execute program code stored in memory 210 to perform the described quality assessment functions. Processor 205 may include more than one electronic processing device and / or additional processing circuitry. For example, processor 205 may include multiple processing chips, digital signal processors (DSPs), analog-to-digital or digital-to-analog converters, and / or other circuitry or processing chips with the processing capability to perform the functions described herein. Processor 205 may perform all the processing functions described herein locally on analysis device 110.

[0046] The memory 210 may include a UI module 212, which sends and receives instructions from the user interface 120 when executed by the processor 205, and allows information stored in the memory 210, such as visual displays and / or audio output, to be output on the user interface 120.

[0047] The memory 210 includes a meat quality assessment module 214, which, when executed by the processor 205, determines meat quality assessment metrics for corresponding traits of the meat product. In some embodiments, the meat quality assessment module 214 receives video data including a sequence of frames 104 as input. For example, at least some of the frames 104 may depict a portion of the carcass to be quality assessed. The quality assessment module 214 may output determined corresponding quality assessment metrics for one or more traits related to meat quality.

[0048] As described above, the video capture device 112 may include a 2D camera or a 3D camera. In embodiments where the video capture device 112 includes a 3D camera, the video capture device 112 may be external to or different from the analysis device 110. The memory 210 may include a 3D camera interface module 213. The 3D camera interface module 213 may include a code library that, when executed by the processor 205, enables the 3D camera interface module 213 to send instructions to and receive instructions from the 3D camera. In embodiments utilizing the depth capabilities of the 3D camera, such as when assessing traits like EMA, marbling, ribeye area, rib fat thickness, meat color, fat color, marbling fineness, and / or intramuscular fat ratio, processed depth data from the 3D camera can be performed through the camera interface module 213 and / or the meat quality assessment module 214. In some embodiments, a 2D camera may be used to assess traits including EMA, marbling, ribeye area, rib fat thickness, meat color, fat color, marbling fineness, and / or intramuscular fat ratio. In some embodiments, a non-depth analysis mode can be used to evaluate video data acquired by a 2D camera. In some embodiments, monocular depth estimation, stereo depth estimation from two different lenses, or similar techniques can be used to evaluate video data acquired by a 2D camera to perform depth-based analysis using 2D images as a data source.

[0049] The meat quality assessment module 214 includes the object determination module 216 and the object quality determination module 218.

[0050] The object determination module 216 may include program code that, when executed by the processor 205, determines regions of interest (ROIs) or regions 132 within frames of video data. The object determination module 216 may determine multiple ROIs or regions 132 for any given frame 104. For example, different or unique ROIs or regions 132 may be determined for evaluating each trait.

[0051] In some embodiments, the object determination module 216 may include one or more segmentation models 220. Each segmentation model 220 may be configured to determine a region of interest (ROI) or region 132 within a frame for analysis. In some embodiments, the determined ROI or region may be suitable for measuring meat quality for more than one or even all traits of interest. The segmentation model 220 may be a machine learning model trained on a dataset of processed images of carcasses and / or meat products and configured to determine the presence, boundaries, and / or segmentation of meat products in frame 104.

[0052] Segmentation model 220 may include a convolutional neural network. Segmentation model 220 may be configured to identify regions of interest 132 in frame 104. After identifying region of interest 132, segmentation model 220 may crop frame 104 to focus on a proportion of region of interest 132. In some cases, other parts of the image are masked, and the cropped (and / or masked) frame 104 may be scaled. The cropped and / or masked and / or scaled frame 104 may be fed into another convolutional network within segmentation model 220 to produce scalar characteristic predictions.

[0053] When the assessed trait is EMA, segmentation model 220 can be configured to process frames characterized by a quadrature that is located or placed on a region of interest (such as the costal eye area) of the carcass. In some embodiments, segmentation model 220 can be configured to operate according to a first operating state or operating mode for assessing traits other than EMA, and when acquiring video frames, it does not require placing a point grid on the region of interest of the carcass, such as marbling, costal eye area, costal fat thickness, meat color, fat color, marbling fineness, or intramuscular fat ratio. Segmentation model 220 can be configured to operate according to a second operating state or operating mode for assessing traits such as EMA, marbling, costal eye area, costal fat thickness, meat color, fat color, marbling fineness, or intramuscular fat ratio, and when acquiring video frames, it does require placing a point grid on the region of interest of the carcass, such as, for example, where a 2D camera is used. User interface 120 can be configured to allow a user to select a desired operating mode for segmentation model 220. For example, user interface 120 can be configured to present or display user interface elements to allow the user to select the desired operating mode for segmentation model 220.

[0054] For the second operating mode, the segmentation model 220 can be trained on a meat product image with a grid to identify eye muscles and recognize the presence of integrator points within frame 104.

[0055] In some embodiments, the object determination module 216 may include a MobileNetV3 encoder with a simplified Atrous spatial pyramid pooling segmentation decoder. In other embodiments, the object determination module 216 may include other machine learning models, such as DeepLabv3, Mask R-CNN, EAR-Net, etc. In some embodiments, a binary cross-entropy loss function is used. However, it should be understood that other loss functions, such as Dice loss or sparse classification cross-entropy, may be used. In some embodiments, training data augmentation techniques may be used to enhance or adjust images, such as: image rotation and / or flipping and / or contrast adjustment and / or brightness adjustment and / or cropping and / or resizing.

[0056] In some embodiments, when performing the EMA segmentation process, the segmentation model 220 may include YOLOv4-tiny. The loss function used may be a generalized IoU loss model.

[0057] The object quality determination module 218 may include program code defining one or more trait prediction models, which, when executed by the processor 205, determine a predicted score for each corresponding trait based on a region of interest or area as determined by the object quality determination module 216. The object quality determination module 216 may include a machine learning model trained on a dataset of processed images of carcasses and / or meat products, and configured to determine traits such as marbling, marbling fineness, ribeye area, rib fat thickness, intramuscular fat, fat color, and meat color within images depicting meat products.

[0058] In some embodiments, the object quality determination module 216 may include a MobileNetV3 encoder with custom pooling. This encoder may utilize fully connected layers. In other embodiments, the object quality determination module 216 may include an EfficientNetv2 model. In other embodiments, the object quality determination module 216 may include an FBNet model. In other embodiments, the object quality determination module 216 may include a ShuffleNetv2 model. The loss function used in the object quality determination module 216 may include a mean absolute error function. In other embodiments, the loss function may be a root mean square error function.

[0059] In some embodiments, the training data of the machine learning model used in the object quality determination module 216 can be enhanced by rotating the input data. In some embodiments, the training data of the object quality determination module 216 can be enhanced by flipping the input data. In some embodiments, the training data of the object quality determination module 216 can be enhanced by changing the contrast of the input data. In some embodiments, the training data of the object quality determination module 216 can be enhanced by changing the brightness of the input data. In some embodiments, the training data of the object quality determination module 216 can be enhanced by changing the brightness of the input data. In some embodiments, the training data of the object quality determination module 216 can be enhanced by cropping the input data. In some embodiments, the training data of the object quality determination module 216 can be enhanced by adjusting the size of the input data. Enhancing data in this way can create a more resilient and accurate dataset.

[0060] The meat quality assessment module 214 includes a frame fit determination module 224. The frame fit determination module 224 can be configured to determine whether frames of acquired video data are suitable or sufficient to assess a specific trait of the meat product 140. In some embodiments, the frame fit determination module 224 is configured to determine a frame fit score for predicting each corresponding trait. Some frames may be suitable or sufficient to assess a first trait in the meat product 140, but may be insufficient or unsuitable for assessing a second trait in the meat product 140. Therefore, the frame fit determination module 224 can apply different criteria and / or thresholds when determining frame fit scores for different traits. Similarly, the frame fit determination module 224 can apply the same criteria and / or thresholds when determining frame fit scores for different traits. In some embodiments, the frame fitness determination module 224 can compare a frame fitness score for a trait with a threshold frame fitness score for the trait, and in response to a frame fitness score greater than the threshold frame fitness score for the trait, the frame fitness determination module 224 can determine the frame as a suitable frame for evaluating or predicting the trait. In response to a frame fitness score less than the threshold frame fitness score for the trait, the frame fitness determination module 224 can determine the frame as unsuitable for predicting the trait. In some embodiments, the frame fitness determination module 224 may discard or disregard unsuitable frames for further evaluation. However, it should be understood that frames deemed unacceptable or unsuitable for evaluating the first trait may still be suitable for evaluating the second trait. In some embodiments, frames deemed unacceptable for the first trait but suitable for the second trait may be discarded based on their unacceptability for the first trait. The threshold fitness score can be configurable. The threshold fitness score can be configured differently for each trait. For example, each trait may have a trait-specific threshold; a frame that may be suitable for evaluating the first trait may not be suitable for evaluating the trait.

[0061] Meat quality assessment module 214 can be configured to determine an assessment confidence rating for each trait. The assessment confidence rating can be based on the fitness and / or number of acquired fit frames, and / or on a predicted score determined for the fit frames. In some embodiments, meat quality assessment module 214 determines the assessment confidence rating for a trait by determining one or more of the following: (i) the number of fit frames determined for the trait; and (ii) a function of the frame fitness score for the trait in the determined fit frames. In some embodiments, meat quality assessment module 214 determines the assessment confidence rating for a trait by determining a function of the predicted score for a first trait in the determined fit frames.

[0062] The meat quality assessment module 214 can determine whether to determine or generate a quality assessment metric for a trait based on a corresponding assessment confidence rating. For example, the meat quality assessment module 214 can compare the assessment confidence rating with a threshold for the corresponding trait. If the assessment confidence rating exceeds the relevant threshold, the meat quality assessment module 214 can determine or generate a quality assessment metric for the trait. The quality assessment metric for the trait can be based on the output from the object quality determination module 218, such as a predicted score for the trait in a determined suitable frame. However, if the assessment confidence rating does not exceed the relevant threshold, the meat quality assessment module 214 can acquire and assess other frames of the video data for analysis.

[0063] Meat quality assessment module 214 can be configured to determine a carcass identifier. For example, the carcass identifier may be provided within image frame 104. The carcass identifier may be located on meat product 140. The carcass identifier may be located in the vicinity of meat product 140. The carcass identifier may be one or more of the following: a one-dimensional barcode; a two-dimensional barcode (such as a QR code or, for example, the DotMatrix system); an alphanumeric code; a near-field communication (NFC) tag; an RFID tag. In some embodiments, the carcass identifier may be input via user interface 120. In such embodiments, the user interface receives the carcass identifier and transmits it to memory 210 via processor 205. In some embodiments, meat quality assessment module 214 sends a request to processor 205 to retrieve the carcass identifier from an external application using an application programming interface (API). The carcass identifier may be stored in memory 210. The carcass identifier may be stored within meat quality assessment module 214. Meat quality assessment module 214 can associate carcass identifiers with image frame 104. Meat quality assessment module 214 can also associate carcass identifiers with meat product 140. The use of carcass identifiers allows for accurate retrieval of meat quality assessments corresponding to a specific carcass. Processor 205 can determine carcass identifiers via application programming interface (API). The API can allow communication with a processing facility network storing carcass identifiers. User interface 120 can include display 130, which is configured to display the determined meat quality assessment metrics. User interface 120 can include a keyboard, touchscreen, and / or button-based interface. Display 130 can include an LED or LCD display, such as a smartphone touchscreen display.

[0064] In some embodiments, the analysis device 110 may be mounted on a robotic arm (not shown). In such embodiments, the robotic arm (not shown) may be controlled by a controller 202 to orient the video capture device 112 of the analysis device 110 to match, align, or adapt to a predetermined viewpoint of a moving carcass in a meat processing environment, for example, to allow the analysis device 110 to capture video images consistently and / or relatively quickly. The robotic arm (not shown) can improve the accuracy and / or efficiency of the analysis process because the robotic arm mount can better match the speed and orientation of the carcass as it moves during processing, and / or maintain a consistent viewpoint. Furthermore, such embodiments can allow for reduced labor costs in the plant.

[0065] In some embodiments, the analysis device 110 may be mounted as part of a meat slicing assembly (not shown). In such embodiments, the video capture device 112 of the analysis device 110 may be oriented to capture video images of the facets of the meat raw material. The output of the analysis device 110 can be used to determine the quality of the facets of the meat raw material. After determining the quality of the facets of the meat raw material, it can then be sliced ​​by the slicing assembly (not shown) and conveyed along one or more different conveyor belts. For example, each conveyor belt may lead to a separate or different processing area where the slices are processed, labeled, and / or packaged according to their assessed meat quality. Thus, such embodiments allow for relatively rapid processing of meat products and / or relatively greater meat quality specificity throughout the meat raw material. This is an improvement over existing solutions for high-volume meat processing, which typically assign a single meat quality grade to the processed meat raw material, potentially failing to capture the higher quality portions present throughout the raw material.

[0066] Communication interface 226 is accessible by processor 205 and configured to allow information exchange with devices external to analysis device 110. Communication interface 226 may include components for receiving a SIM card to facilitate communication over 2G, GSM, EDGE, CDMA, EVDO, 3G, GPRS, 4G, 5G, or other suitable telecommunications networks. Communication interface 226 may include an Ethernet port for wired communication. Communication interface 226 may include a wireless internet interface. Communication interface 226 may include a Bluetooth interface. Communication interface 226 may include one or more of the features described in any of the above embodiments. Communication interface 226 may enable communication via an API. Communication interface 226 may enable communication via Hypertext Transfer Protocol (HTTP, HTTPS).

[0067] Figure 3 This is a flowchart of a method 300 for assessing meat quality according to some embodiments. Method 300 can be performed by executing program code in memory 210, such as meat quality assessment module 214, through analysis device 110.

[0068] At 302, the analysis device 110 determines the video data. The video data includes a sequence of frames 104 or a series of images. At least some of the frames depict a portion of the carcass or meat product 140 to be quality assessed. In some embodiments, the video data is captured by the video capture device 112 and provided to the meat quality assessment module 214. The sequence of frames 104 may be stored in memory 210 for assessment by the meat quality assessment module 214. During method 300, the sequence of frames 104 may be temporarily and in near real-time stored to avoid retaining large image files that could adversely affect storage on memory 210. Therefore, in some embodiments, each frame 104 of the video data may be analyzed as it is received and while the video data is still being captured by the video capture device 112.

[0069] In some embodiments, the trait being assessed is the EMA. However, it should be understood that the trait being assessed may include one or more of the following: EMA, marbling, ribeye area, rib fat thickness, meat color, fat color, marbling fineness, or intramuscular fat ratio. As described above, the analysis device 110 may be configured to operate in a second operating mode, for example, as selected by a user providing input to a user interface, or automatically when a grid is detected placed on a region of the captured carcass. The second operating mode may be specifically configured to assess the EMA of the meat product 140 on which the grid is placed, such that frames of the meat product 140 captured by the analysis device 110 also depict the grid. The grid may be a planimeter, such as a translucent or substantially translucent point planimeter. The grid allows the analysis device 110 to determine the area of ​​the region of interest of the carcass.

[0070] At positions 304 to 312, the meat quality assessment module 214 performs a series of actions on the first image frame 104 of the frame sequence.

[0071] At 304, meat quality assessment module 214 determines a frame fit score for assessing or predicting a primary trait. The frame fit score may relate to the fit, appropriateness, or adequacy of a specific or candidate frame used to assess a particular trait. For example, a higher score may indicate a greater degree of fit for accurate prediction of the trait. The frame fit score may be based on one or more frame fit ratings.

[0072] In some embodiments, the frame fit score may depend on a frame fit rating indicating glare deficiency. For example, the meat quality assessment module 214 may be configured to convert frames to grayscale. After converting frames to grayscale, the meat quality assessment module 214 may determine or calculate the number of pixels exceeding a configurable brightness value as a proportion of the total number of pixels in the image. If the number of grayscale pixels exceeds a configurable brightness value (a frame fit rating threshold for glare deficiency), the frame may be determined to have too much glare to be used for meat quality assessment. Such frames may be discarded or not considered from further analysis.

[0073] In some embodiments, the frame fit score may depend on a frame fit rating that indicates image sharpness. For example, the meat quality assessment module 214 may be configured to perform a method based on Fast Fourier Transform (FFT) that measures the high-frequency components of an image. If the high-frequency components of an image are determined to correspond to an image with sufficient sharpness (a frame fit rating threshold for sharpness), then the frame can be used for meat quality assessment. Frames that do not have sufficient sharpness may be considered too blurry to accurately assess meat quality and may be discarded or not considered for further analysis.

[0074] In some embodiments, the frame fit score may depend on a frame fit rating indicating the segment size. This rating measures the size of the identified region of interest 132 relative to the size of the entire frame. The meat quality assessment module 214 can determine the size of the identified segment as the number of pixels as a segment pixel score using the segmentation model 220. If the segment pixel score is below a threshold, the frame can be determined to lack sufficient segment size and can be discarded or not considered for further analysis. The frame fit rating threshold for segment size can be the minimum ratio of the determined segment pixel score to the total number of pixels in the frame.

[0075] In some embodiments, the frame fit score may depend on a frame fit rating that indicates segmentation edges. A segmentation edge is a rating associated with the shortest distance between the segmented region of interest 132 and any edge of the frame. Values ​​that are too small may indicate that the region of interest 132 is cut off by frame edges. If the meat quality assessment module 214 determines that a frame has a segmentation edge score below a threshold (the frame fit rating threshold used for segmentation edges), the frame may be discarded or ignored for further analysis.

[0076] In some embodiments, the frame fit score may depend on a frame fit rating that indicates the roundness of the segmentation. The segmentation edge is a rating that measures the roundness of the region of interest 132. This can be done by the meat quality assessment module 214 by fitting an ellipse to the region and taking its aspect ratio. The angle of inclination relative to the relative height of the animal cutting surface can be determined as undesirable for the prediction of some traits, and this value can be used as a representative of the non-tilt nature of the camera angle.

[0077] In some embodiments, frame fitness rating may include the application or output of a frame fitness model or a frame quality model. The frame fitness model may include a neural network configured to receive input from a frame image. The neural network may be a convolutional neural network. The frame image may be segmented or unsegmented. In some embodiments, the frame image may be rescaled or not. The frame fitness model is configured to rate the quality or fitness of a frame and output a representative indicator. The representative indicator may be an indicator representing overall quality or fitness, a rating of quality or fitness, or different aspects of quality or fitness. In some embodiments, the frame fitness model is configured to output multiple representative indicators. The representative indicators may be numerical or categorical. In some embodiments, the rating module 214 may be configured to discard frames based on the output from the frame fitness model.

[0078] In some embodiments, the frame fitness model may utilize other frame fitness ratings (including those discussed herein) as input to provide an assessment of quality or fitness, such as frame fitness ratings indicating segmentation roundness, segmentation edges, segmentation size, image sharpness, lack of glare, etc. In some embodiments, the frame fitness model may be used in conjunction with other frame fitness measures to identify or determine high-quality or fit frames. For example, depending on the frame fitness model used, the frame fitness measure may relate to the quality or content of the frame image. In some embodiments, the frame fitness model may be used in combination with or together with per-frame feature predictions. For example, specific frames (or frames) from a plurality of frames in video data may be selected for downstream processing or for visual display.

[0079] The relevance of each of the above ratings varies for each animal trait. For example, glare is extremely detrimental to some traits but has a smaller impact on others. Therefore, for each trait, a trait-specific threshold can be configured for each of the different ratings, where if any of these thresholds is not met, the frame is considered unsuitable for accurate prediction of that trait and is excluded from further processing.

[0080] In some embodiments, one or more of the ratings may be used to calculate an overall score or frame fitness score for each frame for each trait.

[0081] In some embodiments, for specific traits t Frame fit score of the frame S t It can be calculated as:

[0082] in: v r The rating value for rating type r. For targeted traits t rating types r The proportionality coefficient, and For targeted traits t rating types r The offset coefficient (where, and Both are constant across frames. For each trait / rating type pair ( t, r (You can choose the coefficient) and This makes higher values ​​indicate a greater fit for accurate predictions, and S t Always non-negative. The scaling factor and offset factor for each rating type can be specifically selected or configured for each trait. This allows one or more rating types to be given greater weight than others, thus allowing different criteria to be applied when considering whether a frame is suitable for rating two different types of traits.

[0083] In some embodiments, a multiplication method may be used. For example, for specific traits. t Frame fit score of the frame S t It can be calculated as:

[0084] In some embodiments, for specific traits t Frame fit score of the frame S t It can be calculated as The geometric mean of the components.

[0085] In some embodiments, for specific traits t Frame fit score of the frame S t Calculated as each The minimum or maximum value of a component.

[0086] In some embodiments, for specific traits t Frame fit score of the frame S t Calculated as The median of the components.

[0087] In some embodiments, each rating v r a r t + b r t It can be included in any of the above embodiments for targeting a specific characteristic. t Calculate the frame fit score for frame 104. S t This eliminates the need to first determine the maximum value of each rating with zero to produce non-negative values.

[0088] At 306, in response to a frame suitability score greater than a threshold frame suitability score, the meat quality assessment module 214 determines that the frame is a suitable frame for assessing or predicting a first trait. The threshold can be configured according to the trait of interest, as different traits may have different requirements to determine whether frame 104 is suitable for assessing or predicting the trait. When operating in a second operating mode for assessing EMA, the analysis device 110 can determine whether frame 104 is suitable for EMA assessment by determining whether a dot grid is depicted in frame 104. In some embodiments, if the meat quality assessment module 214 detects eye muscles in frame 104, the analysis device 110 can determine that frame 104 is suitable for EMA assessment. In some embodiments, the analysis device 110 can determine that frame 104 is suitable for EMA assessment by determining the presence of both a dot grid and eye muscles within frame 104.

[0089] At 308, the meat quality assessment module 214 (e.g., object determination module 216 or segmentation model 220) determines the region of interest (ROI) or region 132 in frame 104 for assessment or prediction of a first trait. For example, a frame of video data can be provided to the object determination module 216 as input, and the object determination module 216 can provide a numerical value indicating the area of ​​the RIO as output. In some embodiments, the object determination module 216 can generate a modified image or frame that depicts the RIO or region 132 on the frame as, for example, a region with contours or boundaries. Figure 4a and Figure 4b An example of a corrected frame is shown in the image.

[0090] When operating in the second operating mode for evaluating the EMA, the analysis device 110 can determine the region of interest 132 within frame 104 by determining the eye muscle boundary within frame 104. In some embodiments, determining the region of interest 132 may further include determining the point grid covering the eye muscle boundary within frame 104. In some embodiments, the segmentation model 220 may mask frame 104 to remove portions of the image that are not within the region of interest 132. In such cases, masking frame 104 can reduce the chance of points from the point grid falling outside the determined region of interest being miscounted.

[0091] At 310, the meat quality assessment module 214 (e.g., object quality determination module 218 or trait prediction model 222) determines a predicted score for the first trait based on the determined region of interest or region 132. The predicted score for the first trait may include a scalar value for the first trait corresponding to the first frame 104.

[0092] When operating in the second operating mode used for assessing EMA, the meat quality assessment module 214 can determine the number of points within the identified region of interest 132. Since each point corresponds to one square centimeter, this determination provides information in centimeters. 2 This is a measurement of the area of ​​the eye muscles per unit. In some embodiments, different grid sizes can be used, such as smaller or larger grids. In embodiments using a smaller grid, the points may be spaced less than one square centimeter. In embodiments using a larger grid, the points may be spaced more than one square centimeter. In such embodiments, the user can select the grid size via user interface 120. In other embodiments, the user can select the area measurement value corresponding to each point.

[0093] At point 312, meat quality assessment module 214 determines the assessment confidence rating for the primary trait. The assessment confidence rating indicates whether sufficient information has been determined from the video data to generate a quality assessment metric for the primary trait.

[0094] In some embodiments, the meat quality assessment module 214 determines the assessment confidence rating of the first trait by determining whether a sufficient or threshold number of suitable frames have been identified. If the total number of identified suitable frames does not exceed the threshold (minimum), the meat quality assessment module 214 determines that the assessment confidence rating is insufficient.

[0095] In some embodiments, the meat quality assessment module 214 determines the assessment confidence rating of the first trait based on a function of the frame fitness scores of the determined fit frames. For example, the frame fitness scores of all unexcluded frames can be summed or added together and compared with a threshold (minimum). If the sum of the frame fitness scores does not exceed the threshold, the meat quality assessment module 214 determines that the assessment confidence rating is insufficient.

[0096] In some embodiments, the meat quality assessment module 214 determines the assessment confidence rating of the first trait based on a function of the predicted score of the first trait for a determined fit frame. For example, the function of the predicted score could be the sample standard deviation of the predicted score for the trait for a determined fit frame. The sample standard deviation value can be compared with a threshold (maximum value). If the sample standard deviation value exceeds the threshold, the meat quality assessment module 214 determines that the assessment confidence rating is insufficient.

[0097] In some embodiments, the meat quality assessment module 214 is configured to determine whether a frame is a suitable frame (e.g., 304 and 306) and to determine the predicted score of the frame substantially simultaneously or substantially in parallel (e.g., 308 and 310).

[0098] In some embodiments, the meat quality assessment module 214 determines only the predicted score for frames that have already been identified as suitable frames. For example, in response to determining a frame as a suitable frame for predicting or assessing a first trait (e.g., at 304 and 306), the meat quality assessment module 214 determines the predicted score for the frame (e.g., at 308 and 310). In response to determining that a frame is not a suitable frame for predicting or assessing a first trait, the meat quality assessment module 214 may select an uncertain predicted score for the frame (e.g., not performing 308 and 310).

[0099] At step 314, in response to determining that the confidence rating for the first trait has not exceeded the confidence threshold for the first trait, the meat quality assessment module 214 performs steps 304 to 310 for subsequent frames in the frame sequence, such as those that can be received from the video capture device 112.

[0100] At step 316, in response to determining that the assessment confidence rating for the first trait has exceeded the assessment confidence threshold for the first trait, the meat quality assessment module 214 determines a quality assessment metric for the first trait based on the predicted score of the first trait in the determined suitable frame. Furthermore, at step 316, after determining the quality assessment metric for the first trait, the meat quality assessment module 214 may send an instruction to the processor 205 to terminate video capture via the video capture device 112. In embodiments, for example, when assessing more than one trait, the meat quality assessment module 214 may send an instruction to the processor 205 to terminate video capture via the video capture device 112 after completing the quality assessment of each or all traits. This allows video capture to be stopped without user input. The meat quality assessment may be stored in memory 210. The meat quality assessment module 214 may send an instruction to the processor 205 to transmit the meat quality assessment via communication interface 226. In some embodiments, image frames 104 stored in memory 210 may be transmitted via communication interface 226. Communication interface 226 transmits data to an external source. In this way, once a sufficient number of frames have been determined for accurate assessment of a trait, no further frames are analyzed for that trait. This means that once a sufficient number of suitable frames have been acquired to allow the device to properly assess meat quality, no more frames are acquired and analyzed. This contrasts with techniques where a predetermined number of frames are acquired and assessed, in which case it can be determined that the number of frames acquired is actually insufficient to allow assessment with sufficient accuracy, or that too many frames are acquired, where a subset of the frames would actually be sufficient to allow assessment with sufficient accuracy.

[0101] In some embodiments, the meat quality assessment module 214 uses the following outlier exclusion and averaging procedure to determine the quality assessment measure of the first trait: Calculate the second percentile value P of the predicted score respectively. B and the first percentile value P A (in P B ≤ P A For example, the first percentile value and the second percentile value can be the 25th and 75th percentiles (the first quartile and the third quartile), respectively. In some embodiments, the second percentile value P B and the first percentile value P A These can be set to 20 and 80, or 10 and 90, or 40 and 60 respectively. However, it should be understood that any suitable value can be used.

[0102] Then, calculate the interquartile range (IQR) = P A– P B .

[0103] Then, by multiplying the interquartile range by the tolerance parameter tol IQR (Configurable, default value is 1.5) and subtract this value from the 25th percentile value to calculate the lower limit. B min :

[0104] Similarly, upper limit B max The calculation is as follows:

[0105] Finally, for those falling within the range [ B min , B max The arithmetic mean of all individual frame predictions within the range is used to produce a quality rating metric for the first trait.

[0106] In some embodiments, when determining quality assessment metrics, the meat quality assessment module 214 may determine and ignore or disregard the “worst” from the frame based on frame suitability ratings (e.g., for each of glare, sharpness, and segmentation size). p 丢弃 Predicted scores derived as a percentage. In other words, in such embodiments, these predicted scores are not used to determine a quality assessment metric for the trait.

[0107] In some embodiments, a quality rating metric can be determined by averaging the predicted scores obtained without exclusion. In some embodiments, a quality rating metric can be determined by taking the median or geometric mean (the opposite of the arithmetic mean) of the predicted scores. In some embodiments, a quality rating metric can be determined by weighting the corresponding fitness scores; this allows frames 104 with higher fitness scores to have a greater influence on the resulting combinations. In some embodiments, outliers can be excluded from the quality rating metric by determining a multiple of the (sample) standard deviation of each prediction from the mean and comparing these to a fitness threshold. In some embodiments, a quality rating metric can be determined by clustering the predictions and selecting determined clusters based on criteria. This criterion can be the largest cluster, or the number of largest clusters, such as the largest... n One cluster.

[0108] Once the meat quality assessment module 214 has determined the quality assessment metrics, it can output the results to the user via the user interface 120. For example, the UI module 212 can enable the presentation of quality assessment metrics for one or more traits on the display 130 and / or store the quality assessment metrics in the memory 210.

[0109] When all animal traits have reached the point, video capture from video capture device 112 can be terminated by processor 205, and the user can be prompted by an alarm at user interface 120 to move to the next carcass for analysis.

[0110] In some embodiments, setting a maximum number of frames for analysis may be included. N F The option (which applies to all traits and includes all frames, including frames that have been excluded from the processing of one or more traits) is used. If the total number of frames analyzed exceeds this limit, the first frame (the oldest frame) of the received sequence is discarded from future calculations, and only the most recent frame is used. N F Each frame is used for traits for which quality assessment metrics have not yet been generated.

[0111] In some embodiments, the meat quality assessment module 214 may assess frames of a second trait and / or an additional trait according to method 300. The second trait and / or the additional trait are different from the first trait and are different from each other. This assessment using method 300 may be performed substantially simultaneously with the assessment of the first trait using method 300.

[0112] As described above, in some embodiments, the video capture device 112 may include a 3D camera configured to determine depth information that can be used to assess the characteristics of the EMA. However, it should be understood that the depth information may also be used to assess characteristics such as marbling, rib eye area, rib fat thickness, meat color, fat color, marbling fineness, or intramuscular fat ratio. The processor 205 may receive instructions from the user interface 120 to activate the 3D camera to record 3D video data. The 3D video data may include video image frames 104 captured from more than one orientation. The 3D video image data including video image frames 104 captured from more than one orientation may be processed by the 3D camera interface module 213. The 3D camera interface module 213 may determine a 3D point cloud based on the captured 3D video data. The 3D camera interface module 213 may cross-reference the image frames 104 to generate a 3D model of an object in the camera's field of view. This may include meat product 140. In some embodiments, this may include a portion of meat product 140. The 3D camera interface module 213 can communicate with the meat quality assessment module 214. The 3D camera interface module can send a 3D point cloud to the object determination module 216. The object determination module 216 can receive the point cloud from the 3D camera interface module 213 and isolate the region of the point cloud corresponding to the region of interest 132. In some embodiments, another target region of the meat product 140 can be identified in the point cloud. This can include the region of interest 132. The 3D camera interface module 213 can send the 3D point cloud to the meat quality assessment module 214. The meat quality assessment module 214 can determine a plane fitted to the points of the 3D point cloud. The meat quality assessment module 214 can project the points of the 3D point cloud onto the plane. This can generate multiple projection plane points that can be further processed. The area of ​​the plane can be determined by the meat quality assessment module 214 through the placement of the projected points on the plane. The area of ​​the determined region of interest 132 can correspond to the eye muscle area.

[0113] The projected planar points can then be rasterized into a high-resolution image, where each pixel unit of the image corresponds to a square region of a plane of known real-world size, i.e., a real-world region. For example, the real-world region can be determined by the meat quality assessment module 214 based on video data. In some embodiments, the size of the pixel unit can be 0.5 mm. 2 With 0.01mm 2 Between. In some embodiments, the size of the pixel unit can be 0.25mm. 2 A square region on a plane of known real size can contain "holes," which are pixel units within the region of interest that do not contain corresponding points in the planar projection of the point cloud. The high-resolution image can then be converted to binary, where each pixel has a value of 1 or 0, depending on whether it contains a point in the planar projection. The resulting binary image can then identify the region of interest.

[0114] The meat quality assessment module 214 can then be configured to fill holes in the image using morphological closing operations. For example, a dilation operation followed by an erosion operation can be applied, using the same structured elements for both operations. Morphological operations can use kernels of appropriate sizes. Morphological closing operations can be effective for filling small holes in an image while preserving the shape and size of any large holes and objects in the image. The meat quality assessment module 214 can then use contour detection or contour recognition to obtain the boundaries of the regions of interest in the filled binary image. In some embodiments, the detection of individual contours is confirmed and / or verified. That is, multiple possible failure modes are filtered out when the filled regions of interest are confirmed to be continuous in the binary image. In response to the identification of a single contour, the known correspondence between pixels and real-world regions is used to obtain the region within the contour and convert it into an area estimate of the real-world region of interest.

[0115] Therefore, the meat quality assessment module 214 can determine the size of the eye muscle area (EMA) based on the area of ​​the defined region of interest 132. Since the cut surface of the meat product 140 can be assumed to be approximately planar, errors caused by noise or depth measurements can be corrected based on the approximately planar surface.

[0116] In some embodiments, the video capture device 112 may include a 3D camera configured to determine depth information that can be used to assess the characteristics of rib fat thickness. A meat quality assessment module 214 may be configured to measure the characteristics of rib fat thickness. Similar to the process for isolating regions of interest, a 3D camera interface module 213 may communicate with the meat quality assessment module 214 and may send the determined 3D point cloud to an object determination module 216. The object determination module 216 may receive the point cloud from the 3D camera interface module 213 and isolate both the region of the point cloud corresponding to the extraocular muscle (EMA) and the region of the point cloud corresponding to the rib fat. In some embodiments, EMA and rib fat points are isolated in the 3D point cloud by segmentation derived from a neural network of the color image components of a depth map camera frame referencing the video data, where each pixel corresponds to a point in the 3D point cloud that has previously been cross-referenced with the depth map image. In some embodiments, the separation of EMA and rib fat may be performed sequentially or in parallel.

[0117] The 3D camera interface module 213 can send a 3D point cloud to the meat quality assessment module 214. The meat quality assessment module 214 can determine a plane fitted to the points of the 3D point cloud. The meat quality assessment module 214 can project the points of the 3D point cloud onto the plane. This can generate multiple projected plane points that can be further processed. The area of ​​the plane can be determined by the meat quality assessment module 214 through the placement of the projected points on the plane. In this case, the determined area corresponds to the EMA and rib fat. The multiple projected plane points represent a 2D point cloud with both EMA and rib fat points. Since these are separated and isolated from each other, the projected points of the EMA are different from the projected points of the rib fat.

[0118] Identify the first principal component (PC1) of the EMA and align the 2D point cloud such that PC1 is vertical. In some embodiments, it is assumed that the camera is held upwards in the correct manner and the image is taken from the bottom of the EMA at a 180-degree angle. This can be used to select which of the two possible "upward" methods is correct.

[0119] The projected planar points can then be rasterized into an image. In some embodiments, a pixel grid with a coarser pixel unit size than the grid used in calculating the area of ​​the region of interest is employed. For example, a 1 mm grid is used. 2 Pixel unit. The size of the pixel unit used can be selected to minimize or reduce the number of holes. In some embodiments, the roughness of the pixel unit mesh can be selected such that it is coarse enough to anticipate no holes. That is, each pixel can contain or surround at least one point from a planar projection. This provides a lower-resolution 2D image (color, non-binary) with both EMA and rib fat, where it is known which pixels correspond to the EMA and which pixels correspond to the rib fat, and where the correspondence between pixels and real-world geometry is known.

[0120] In some embodiments, the method for measuring rib fat thickness may vary depending on the jurisdiction in which the method is performed.

[0121] To measure rib fat thickness, the y-axis line is identified at an appropriate percentage of the distance between the bottom and top of the EMA (Embedded Expectorant) that meets the requirements for measuring rib fat thickness. As previously mentioned, the top of the EMA can be vertically aligned along its first principal component. In some embodiments, rib fat thickness can be measured at approximately three-quarters of the way up the EMA towards the top. In this case, the value of the y-axis line can be approximately 75%. In some embodiments, it can be in the range of 70% to 85%. In some cases, the value of the y-axis line can be 79%. However, it should be understood that this value can be determined empirically and can vary depending on video data and meat products.

[0122] The meat quality assessment module 214 can then be configured to measure the thickness of rib fat along a y-axis row, and a predetermined number of surrounding rows around the y-axis row. For example, rows between 74% and 83% above the EMA. In some embodiments, the surrounding rows can be determined based on a pre-selected threshold or buffer. The number of surrounding rows below the y-axis row value can differ from the number of surrounding rows above the y-axis row value. In some embodiments, the thickness can be measured in pixels. In some embodiments, the number of surrounding rows along which the thickness of the rib fat is measured can be determined empirically and can vary depending on video data and the meat product.

[0123] The pixel thickness can then be converted back into a real-world estimate of rib fat thickness. The rows can then be aggregated into an overall estimate for the frame using the outlier exclusion and averaging procedures described herein. In some embodiments, a similar procedure is used to aggregate the frames themselves. In some embodiments, frames may be aggregated after fitness filtering, e.g., only frames determined to meet a specific fitness rating are aggregated. In some embodiments, a frame may be rejected for rib fat thickness measurement if the difference between the maximum and minimum per-row thickness estimates within a given frame is too large. In some embodiments, a frame may be rejected for rib fat thickness measurement if too many rows within the used percentage range do not contain any rib fat points.

[0124] It should be understood that procedures similar to those described herein with reference to EMA and rib fat thickness can also be used to determine measurements related to other traits, such as marbling, rib eye area, meat color, fat color, marbling fineness, or intramuscular fat percentage. In some embodiments, EMA, rib eye area, rib fat thickness, meat color, fat color, marbling fineness, or intramuscular fat percentage can be assessed by analyzing video data from a 2D or 3D camera. In some embodiments, depth and / or non-depth analysis modes can be used to assess EMA, marbling, rib eye area, rib fat thickness, meat color, fat color, marbling fineness, or intramuscular fat percentage.

[0125] Figure 4a and Figure 4b According to some embodiments Figure 2 Example screenshot of the user interface display of the analysis device. Figure 4aA display 130 depicts frames with video data 402, which depicts a portion of a carcass to be quality assessed. Video data 402 may be live output from a video capture device 112. Video data 402 can be depicted using segments 410, 412 representing the output of segmentation model 220. Segment 410 may represent a defined segment corresponding to the fat portion of the carcass. Segment 412 may represent a defined meat product 140 on the carcass. Figure 4a and Figure 4b In this embodiment, the identified meat product 140 is a ribeye incision. The segment to be identified may be shown adjacent to video data 402, as shown at 404 on display 130. 406 may include at least one progress indicator of the confidence threshold for the assessed animal trait. 406 may be a progress bar 406, which depicts a graphical representation of the progress in determining the number of fit frames for each assessed trait. In some embodiments, 406 is a percentage indicator. In some embodiments, 406 is a numeric indicator. In some embodiments, 406 is a text indicator.

[0126] At 408, the display 130 may depict operation icons that allow the display 130 to provide the functions of the user interface 120, such as on-screen buttons that control the operation of the analysis device 110 or provide the ability to control the operation of the meat quality assessment module 214. For example, in Figure 4a In this process, the analysis device 120 can be positioned in the middle of performing method 300. Therefore, the operation button at 408 allows the display 130 to act as the user interface 120, enabling the user to stop the meat assessment operation. Figure 4b In one embodiment, the same carcass portion is depicted after meat grading is completed, and the operation button at 408 allows for reinitialization of method 300 on the new carcass, marking of videos for further grading, or changing the grading mode (e.g., between EMA and normal operation). Therefore, as... Figure 4b A complete meat quality assessment is described, and 406 describes a complete assessment for the corresponding animal traits. Figure 4b In this embodiment, these are marbling (both Australian and MSA grades), meat color, fat color, and intramuscular fat percentage. By presenting the portion 402 of the carcass being assessed, displaying the segmented portions 412, 410, having a context-sensitive operation button at 408, and the progress / results of the meat quality assessment in 406, 418, the user can easily and efficiently operate the analysis device 110 to perform method 300. Figure 4a and Figure 4b The operations and displays of the user interface features described herein are provided by executable code stored in UI module 212.

[0127] exist Figure 4a and Figure 4b In some embodiments, the user interface is optimized to work with touchscreen displays, such as smartphone displays. However, in other embodiments, display 130 may be a simple display screen without touchscreen functionality. In such embodiments, the operation buttons at 408 and 416 may be physical buttons that are part of the analysis device 110. Alternatively, more than one display 130 may be provided to display different information – such as a progress indicator at 406 or the results of meat quality assessment at 418. In some embodiments, this information may be presented as part of the user interface 120 via lights, an LCD screen, or other visual indicators to provide the operator with an indication of the results of method 300 for a given animal trait.

[0128] In some embodiments, 3D point clouds generated by a 3D depth camera can be used to capture frames or sets of frames from any angle relative to the cut surface of the meat product to generate a high-resolution composite view of the cut surface. For example, as if it were captured from a top-down, vertical, or bird's-eye view angle. In some embodiments, the angle relative to the cut surface can be within a reasonable range. In some embodiments, the 3D point cloud can also be used to generate a high-resolution composite view of EMA or EMA and rib fat thickness measurement, rib eye area, meat color, fat color, marbling fineness, or intramuscular fat percentage. In some embodiments, video data (or one or more frames from video data) captured by a 2D camera can be used to create a high-resolution composite image of a top-down view. In some embodiments, video data obtained from a 2D camera can be used to generate a series of high-resolution composite images of a top-down view of the cut surface of the meat product. In some embodiments, neural networks and / or machine learning methods can be used to generate images, for example, by interpolating, extrapolating, or generating composite images of the cut surface based on the acquired video data. In some embodiments, a combination of video data obtained from both 3D and 2D cameras can be used to create a high-resolution composite image (or a series of high-resolution composite images). In other words, both depth video data and regular video data can be used to create a composite image of the cut surface.

[0129] High-resolution composite images can be generated directly on the analysis device 110, which includes the video capture device 112. In some embodiments, the composite image can be generated outside the analysis device 110, for example on a server, where point cloud data is transmitted to an external server and then configured to receive the generated composite image from the server. The composite image can be generated in real time during data capture or after data capture has been performed.

[0130] Generating a synthetic image from point cloud data may include acquiring a low-resolution rasterized image as an output of a measurement determining rib fat thickness, and applying upscaling techniques to acquire it from the low resolution imposed by the source pixel grid to a high resolution. In some embodiments, applying upscaling techniques may include using a generative ML model. Generating a synthetic image from point cloud data may include acquiring a planar projected 2D point cloud and applying one or more interpolation techniques before converting the point cloud into a high-resolution rasterized image using a finer pixel grid. In some embodiments, the interpolation techniques may be conventional or machine learning-based techniques. In some embodiments, generating a synthetic image may include acquiring a higher-resolution rasterized image as an output of a determination of the area of ​​a region of interest as described herein, which is obtained using a finer pixel grid, and utilizing a patching technique to “fill in” holes. In some embodiments, the patching technique may use conventional or machine learning methods, such as a diffusion machine learning model.

[0131] In some embodiments, generating a composite image may include acquiring a 3D point cloud (or a separated EMA or EMA and rib fat thickness points, or any other separated portion or morphological point within the point cloud) and obtaining a rasterized image at high resolution using direct means. For example, a neural network may be used to obtain a rasterized image of the 3D point cloud at high resolution. In some embodiments, generating a composite image may include acquiring a 2D planar projected point cloud (or a separated EMA or EMA and rib fat thickness points, or any other separated portion or morphological point within the point cloud) and obtaining a rasterized image of the 2D point cloud at high resolution using direct means. For example, a neural network may be used to obtain a rasterized image at high resolution. In some embodiments, generating a composite image may include using any of the techniques described herein while considering more than one frame at a time to produce a single resulting high-resolution composite image.

[0132] In some embodiments, all frames in the video data can be used to generate a single resulting high-resolution image. Using multiple frames can leverage information about the time series of the frames used in the process. In some embodiments, multiple frames can be used without using frame ordering or temporal information. The resulting high-resolution composite image can be in the form of a video of a sequence of frames and / or high-resolution images, each high-resolution image derived from one frame. In some embodiments, generating the composite image can include a combination of two or more techniques described herein.

[0133] The generated high-resolution synthetic image can be used for further downstream processing, such as for additional trait assessment / analysis. This further processing can be performed on or outside the analysis device 110. The generated high-resolution synthetic image can be used for visual display, such as on or outside the analysis device 110, through hardware accessories equipped with a display, or through an application such as an online portal. In some embodiments, the synthetic image can be displayed in real time during or after data capture.

[0134] Those skilled in the art will understand that many variations and / or modifications can be made to the above embodiments without departing from the broad general scope of this disclosure. Therefore, these embodiments should be considered illustrative rather than restrictive in all respects.

Claims

1. A method comprising: determining video data comprising a sequence of frames, wherein at least some of the frames depict a portion of a carcass to be quality graded; for a first frame of the sequence of frames: a) determining a frame suitability score for grading a first trait; b) in response to the frame suitability score being greater than a threshold frame suitability score for the first trait, determining the frame to be a suitable frame for grading the first trait; c) determining, by a segmentation model, a region of interest in the frame for grading the first trait; d) determining, by a first trait prediction model, a predicted score for the first trait based on the determined region of interest; and e) determining a grading confidence rating for the first trait; in response to determining that the grading confidence rating for the first trait has not exceeded a grading confidence threshold for the first trait, performing steps a) to e) for subsequent frames in the sequence; and in response to determining that the grading confidence rating for the first trait has exceeded the grading confidence threshold for the first trait, determining a quality grading metric for the first trait based on the predicted scores for the first trait for the determined suitable frames.

2. The method of claim 1, wherein determining the grading confidence rating for the first trait comprises: determining one or more of: (i) a number of determined suitable frames for the first trait; and (ii) a function of the frame suitability scores for the first trait for the determined suitable frames.

3. The method of claim 1 or claim 2, wherein determining the grading confidence rating for the first trait comprises: determining a function of the predicted scores for the first trait for the determined suitable frames.

4. The method of claim 3, wherein determining the function of the predicted scores for the first trait for the determined suitable frames comprises determining an arithmetic mean taken over an interval where IQR is an interquartile range defined by 5. The method of claim 4, further comprising:

6. The method of claim 4 or 5, further comprising: determining a frame number threshold for the sequence of frames; [B min , B max ] where B min defined by the function: And where B max defined by the function: where P A is a predicted value of a first percentile of the predicted score for the first trait; where P B is the predicted value of the second percentile of the predicted scores for the first trait; wherein is a tolerance parameter; and in response to determining that a number of frames exceeds the frame number threshold, discarding the frame suitability scores, predicted scores, and confidence ratings for the earliest determined frames. IQR = P A – P B .

7. The method of any one of claims 3 to 6, wherein the first percentile is a smaller percentile than the second percentile. The lowest p of the prediction score excluding one or more of 丢弃 Percentages: (i) a measure of glare absence; (ii) a measure of sharpness; and (iii) a measure of segmentation size; where p 丢弃 is a configurable parameter.

8. The method of claim 1 or claim 2, further comprising: in response to determining the frame to be a suitable frame for grading the first trait, performing steps c) and d); and in response to determining the frame to not be a suitable frame for grading the first trait, omitting steps c) and d).

9. The method of any one of the preceding claims, wherein determining video data comprises receiving a video stream. ​ ​ ​ ​ 10. The method of any one of the preceding claims, wherein the frame suitability score is based on one or more of: (i) a measure of lack of glare, (ii) a measure of sharpness, (iii) a measure of segmentation size; (iv) a measure of segmentation edges of the region of interest; (v) a measure of segmentation circularity of the region of interest.

11. The method of any one of the preceding claims, wherein determining the frame suitability score comprises determining the frame suitability score according to the equation: ， wherein is the fitness score; a value for the rating type r for the first frame; a proportionality coefficient for rating type r for trait t; and Offset coefficient for rating r for trait t.

12. The method of any one of the preceding claims, further comprising: outputting the quality assessment metric for the first trait to a user interface.

13. The method of any one of the preceding claims, wherein the first trait comprises any one of: eye muscle area, marbling, marbling fineness, rib eye area, rib fat thickness, intramuscular fat, fat color, and meat color.

14. The method of any one of the preceding claims, further comprising: for the first frame of the sequence of frames: f) determining a frame suitability score for assessing a second trait, wherein the second trait is different from the first trait; g) responsive to the frame suitability score being greater than a threshold frame suitability score for the second trait, determining the frame as a suitable frame for assessing the second trait; h) determining, by a second segmentation model, a region of interest area in the frame for assessing a second trait; i) determining, by a second trait prediction model, a predicted score for the second trait based on the determined region of interest; and j) determining an assessment confidence rating for the second trait; responsive to determining that the assessment confidence rating for the second trait has not exceeded an assessment confidence threshold for the second trait, performing steps a) through e) on subsequent frames in the sequence; and responsive to determining that the assessment confidence rating for the second trait has exceeded the assessment confidence threshold for the second trait, determining a quality assessment metric for the second trait based on the predicted score for the second trait for the determined suitable frame.

15. The method of claim 14, wherein the second trait is different from the first trait and comprises any one of: eye muscle area, marbling, marbling fineness, rib eye area, rib fat thickness, intramuscular fat, fat color, and meat color.

16. The method of any one of the preceding claims, wherein the frame suitability score is based on one or more frame suitability ratings, each frame suitability rating indicating suitability of a frame with respect to a particular measure, for each frame suitability rating of the one or more frame suitability ratings, the method further comprising: comparing the frame suitability rating to a respective measure-specific rating threshold; and responsive to determining that the frame suitability rating does not satisfy the threshold, determining the frame as unsuitable and excluding the frame from further processing.

17. A meat assessment system, comprising: at least one processor; a memory accessible by the at least one processor and comprising computer- executable instructions that, when executed by the at least one processor, cause the system to perform the method of any one of claims 1 to 16.

18. The system of claim 17, further comprising: a user interface configured to display the determined quality rating metric for the first trait.

19. The system of claim 17 or claim 18, further comprising a video capture device controlled by the at least one processor to capture video data.

20. A non-transitory machine-readable medium storing instructions that, when executed by one or more processors, cause an electronic device to perform the method of any one of claims 1 to 16.

Images