Meat quality detection method and computer readable storage medium

Abstract

The application provides a meat quality detection method and a computer readable storage medium. The meat quality detection method comprises: acquiring a spectral image of a meat detection sample; determining a meat region according to the spectral image; and determining a thawing state or freshness of the meat according to the meat region. The meat quality detection method can detect the thawing state or freshness of the meat in the thawing process.

Description

TECHNICAL FIELD

[0001] The present application generally relates to the field of spectral technology, and particularly relates to a meat quality detection method and a computer readable storage medium. BACKGROUND

[0002] The conventional meat quality detection method is limited to detecting the quality of thawed meat, for example, it is limited to distinguishing frozen meat and chilled meat, and cannot realize meat quality detection during thawing. Therefore, how to detect the thawing state or freshness of meat during thawing is a technical problem to be solved by the present application.

[0003] The content of the background section merely represents the technical knowledge of the inventors, and does not necessarily represent the prior art in the field. SUMMARY

[0004] In view of one or more of the problems in the prior art, the present application provides a meat quality detection method and a computer readable storage medium, which can detect the thawing state or freshness of meat during thawing.

[0005] In a first aspect, the present application provides a meat quality detection method. The meat quality detection method comprises: acquiring a spectral image of a meat detection sample; determining a meat region according to the spectral image; and determining a thawing state or freshness of the meat according to the meat region.

[0006] Optionally, determining the thawing state of the meat comprises: determining a first region of the meat region; and determining the thawing state of the meat according to the first region, comprising: determining the thawing state of the meat according to a pixel parameter of the first region; wherein a gray value of a pixel of the first region is higher than a preset gray value; and the pixel parameter comprises at least one of a pixel number, a pixel area, a pixel distribution, and a pixel proportion.

[0007] Optionally, determining the thawing state of the meat comprises: calculating a proportion of a pixel number of the first region to a total pixel number of the spectral image; and determining the thawing state based on the proportion.

[0008] Optionally, the thawing state comprises a completely thawed state, a semi-thawed state, or an un-thawed state, and determining the thawing state comprises: if the proportion is less than or equal to a first proportion threshold, the meat is in a completely thawed state; if the proportion is greater than the first proportion threshold and less than or equal to a second proportion threshold, the meat is in a semi-thawed state; and if the proportion is greater than the second proportion threshold, the meat is in an un-thawed state, wherein the first proportion threshold is less than the second proportion threshold.

[0009] Optionally, the determining the freshness of the meat includes: determining a second region of the meat region; and determining the freshness of the meat according to the second region, wherein a gray value of a pixel of the second region is not higher than a preset gray value.

[0010] Optionally, the determining the freshness of the meat includes: performing spectral inversion based on the pixels of the second region to obtain a spectral curve; constructing a regression model of the spectral curve and a preset component; and determining the freshness of the meat based on the regression model.

[0011] Optionally, the constructing the regression model of the spectral curve and the preset component includes: constructing the regression model of the spectral curve and the preset component based on at least one of a partial least squares regression algorithm, a support vector machine algorithm, or a decision tree algorithm, wherein the preset component includes at least one of volatile basic nitrogen, total bacterial count, myoglobin, or moisture.

[0012] Optionally, the constructing the regression model of the spectral curve and the preset component includes: extracting image features and spectral features based on the second region; obtaining fusion features according to the image features and the spectral features; and constructing the regression model according to the fusion features.

[0013] Optionally, the extracting the image features includes: extracting the image features based on a multi-stage convolutional neural network.

[0014] Optionally, the multi-stage convolutional neural network includes a first-stage convolutional neural network, a second-stage convolutional neural network, a third-stage convolutional neural network, and a fourth-stage convolutional neural network, and the extracting the image features includes: performing convolution processing on the second region based on the first-stage convolutional neural network, the second-stage convolutional neural network, and the third-stage convolutional neural network to extract the image features to obtain a first feature map; determining a channel feature vector of the first feature map based on a channel attention mechanism; obtaining a channel weight of the first feature map based on the channel feature vector and a parameter mapping function; adjusting the channel weight of the first feature map based on the channel weight to obtain a second feature map; and performing convolution processing on the second feature map based on the fourth-stage convolutional neural network to obtain a third feature map, wherein the third feature map includes a global image feature vector.

[0015] Optionally, the extracting the spectral features includes: dividing a wave band of the spectral curve into a plurality of local wave bands; performing convolution processing on the local wave bands to extract local features; performing convolution processing on the local features to obtain cross-wave band features; and obtaining a spectral feature map based on the cross-wave band features, wherein the spectral feature map includes a global spectral feature vector.

[0016] Optionally, the convolution processing on the local features comprises: performing convolution processing on the local features by using convolution kernels of different sizes.

[0017] Optionally, the fusion of the global image feature vector and the global spectral feature vector comprises: constructing a first sequence based on the global image feature vector and the global spectral feature vector and performing position encoding: injecting learnable or fixed position information into each position in the first sequence to obtain a second sequence; performing feature interaction on the image features and the spectral features of the second sequence based on a multi-head attention mechanism and a feedforward neural network to obtain a third sequence; and performing attention pooling based on the global spectral feature vector and the third sequence to generate a fusion feature vector sequence; wherein the fusion feature vector sequence comprises a fusion feature vector.

[0018] Optionally, the fusion of the global image feature vector and the global spectral feature vector comprises: constructing a first sequence based on the global image feature vector and the global spectral feature vector and performing position encoding: injecting learnable or fixed position information into each position in the first sequence to obtain a second sequence; performing feature interaction on the image features and the spectral features of the second sequence based on a multi-head attention mechanism and a feedforward neural network to obtain a third sequence; and performing attention pooling based on the global spectral feature vector and the third sequence to generate a fusion feature vector sequence; wherein the fusion feature vector sequence comprises a fusion feature vector.

[0019] Optionally, determining the freshness of the meat comprises: inputting the fusion feature vector sequence into a classifier; mapping the fusion feature vector sequence to a plurality of category spaces by the classifier; obtaining scores of the fusion feature vector sequence in the plurality of category spaces; and determining the freshness of the meat based on the scores.

[0020] Optionally, the plurality of category spaces correspond to a plurality of freshness, and the plurality of freshness comprises fresh, less fresh, and not fresh, and determining the freshness of the meat comprises: determining the highest score among the scores of the fusion feature vector sequence in the plurality of category spaces; and determining the freshness of the meat based on the freshness corresponding to the category space corresponding to the highest score.

[0021] Optionally, the meat quality detection method further comprises: performing morphological processing on the meat region, including at least one of the following operations: performing an opening operation on a first region of the meat region; and performing a closing operation on a second region of the meat region.

[0022] In a second aspect, the application provides a computer-readable storage medium. The computer-readable storage medium comprises computer-executable instructions stored thereon, which, when executed by a processor, implement the meat quality detection method as described above.

[0023] The meat quality detection method can monitor the change of frozen meat in a thawing process in real time, can exclude ice crystal interference, supports detection of a plurality of preset components, and can monitor the thawing state or freshness of the frozen meat in real time, accurately and reliably, maximally retains the nutrition and taste of the meat, maintains the meat quality and safety, is suitable for planned cooking requirements, and is helpful to improving user experience.

[0024] The meat quality detection device is small in size, low in cost, and convenient to integrate into a refrigerator refrigerating chamber, so that a user can monitor the thawing state or freshness of meat in the refrigerating chamber in real time. BRIEF DESCRIPTION OF DRAWINGS

[0025] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings used in the embodiment description will be exemplarily introduced below, and the drawings in the following description are only embodiments of the present application, and those skilled in the art can also obtain other drawings according to the provided drawings without creative labor. The drawings are used to provide further understanding of the present application, and constitute a part of the specification. Together with the embodiments of the present application, they are used to explain the present application, and do not constitute a limitation on the present application.

[0026] Figure 1 A flowchart of a meat quality detection method according to some embodiments of the present application is shown.

[0027] Figure 2 A schematic diagram of a meat quality detection device according to some embodiments of the present application is shown.

[0028] Figure 3 A flowchart of determining the thawing state of meat according to some embodiments of the present application is shown.

[0029] Figure 4 A flowchart of determining the thawing state of meat based on a pixel ratio according to some embodiments of the present application is shown.

[0030] Figure 5 A flowchart of determining the freshness of meat according to some embodiments of the present application is shown.

[0031] Figure 6 A flowchart of determining the freshness of meat according to some embodiments of the present application is shown.

[0032] Figure 7 A flowchart of operation S1342 according to some embodiments of the present application is shown.

[0033] Figure 8 A schematic diagram of obtaining fused features according to some embodiments of the present application is shown.

[0034] Figure 9A flowchart showing generation of a sequence of fused feature vectors according to some embodiments of the application.

[0035] Figure 10 A flowchart showing determination of freshness of meat according to some embodiments of the application.

[0036] Figure 11 A flowchart showing determination of freshness of meat according to some embodiments of the application. DETAILED DESCRIPTION

[0037] In the following, only some exemplary embodiments are simply described. As those skilled in the art can recognize, the described embodiments can be modified in various different ways without departing from the spirit or scope of the application. Therefore, the drawings and the description are to be considered exemplary in nature but not limiting.

[0038] In the description of the present application, it is to be understood that the orientations or positional relationships indicated by the terms "center", "longitudinal", "lateral", "length", "width", "thickness", "upper", "lower", "front", "rear", "left", "right", "vertical", "horizontal", "top", "bottom", "inner", "outer", "clockwise", "counterclockwise", and the like are based on the orientations or positional relationships shown in the drawings, and are merely for the purpose of facilitating the description of the present application and simplifying the description, and are not intended to indicate or imply that the devices or elements referred to must have a particular orientation, be constructed and operated in a particular orientation, and therefore cannot be understood as limiting the present application. In addition, the terms "first", "second" are only for the purpose of description, and cannot be understood as indicating or implying relative importance or implicitly indicating the number of the technical features referred to. Therefore, the features defined with "first", "second" can explicitly or implicitly include one or more of the features. In the description of the present application, the meaning of "a plurality of" is two or more, unless otherwise explicitly specified and limited.

[0039] In the description of the present application, it should be noted that, unless otherwise explicitly specified and limited, the terms "mounting", "connecting", "connection", "coupling" should be understood broadly, for example, can be fixed connection, can also be detachable connection, or integrally connected; can be mechanical connection, or electrical connection or can communicate with each other; can be directly connected, or indirectly connected through an intermediate medium, can be the internal communication of two elements or the interaction relationship between two elements. For those skilled in the art, the specific meaning of the above terms in the present application can be understood according to the specific circumstances.

[0040] In the present application, unless otherwise explicitly specified and limited, "on" or "under" of a first feature to a second feature can include that the first and second features are in direct contact, or can include that the first and second features are not in direct contact but are in contact through another feature between them. Moreover, "on", "above" and "over" of a first feature to a second feature includes that the first feature is directly above and obliquely above the second feature, or only means that the first feature is higher in horizontal height than the second feature. "Under", "below" and "underneath" of a first feature to a second feature includes that the first feature is directly below and obliquely below the second feature, or only means that the first feature is lower in horizontal height than the second feature.

[0041] Many different embodiments or examples of the application are provided below. To simplify the present application, the components and settings of specific examples are described below. Of course, they are only examples, and the purpose is not to limit the application. In addition, the application can repeatedly refer to numbers and / or letters in different examples, and such repetition is for the purpose of simplification and clarity, which does not indicate the relationship between the various embodiments and / or settings discussed. In addition, the application provides examples of various specific processes and materials, but those skilled in the art can realize the application of other processes and / or the use of other materials.

[0042] The preferred embodiments of the present application are described below in conjunction with the accompanying drawings, it should be understood that the preferred embodiments described herein are only used to illustrate and explain the present application, and are not used to limit the present application.

[0043] The present application provides a meat quality detection method. The meat quality detection method comprises: acquiring a spectral image of a meat detection sample; determining a meat region according to the spectral image; and determining a thawing state or freshness of the meat according to the meat region. The meat quality detection method of the present application can detect the thawing state or freshness of the meat during thawing.

[0044] Figure 1 A flowchart of a meat quality detection method 10 according to some embodiments of the present application is shown. As shown in Figure 1 The meat quality detection method 10 includes steps S11-S13. In step S11, a spectral image of a meat detection sample is acquired. In step S12, a meat region is determined according to the spectral image. In step S13, a thawing state or freshness of the meat is determined according to the meat region. It should be noted that in step S13, only the thawing state of the meat can be determined, only the freshness of the meat can be determined, or both the thawing state and the freshness of the meat can be determined.

[0045] In some embodiments, the meat quality detection method 10 can be executed by a processor. In some embodiments, the processor can be the processor of the meat quality detection device 20. For example, the meat quality detection device 20 may include electronic devices such as mobile phones, tablets, laptops, wearable devices, in-vehicle devices, augmented reality (AR) / virtual reality (VR) devices, ultramobile personal computers (UMPCs), netbooks, personal digital assistants (PDAs), and smart home devices. Smart home devices may include refrigerators, etc. In some embodiments, the processor may be an external processor to the meat quality detection device 20.

[0046] Figure 2 A schematic diagram of a meat quality testing device 20 according to some embodiments of the present invention is shown. Figure 2 As shown, the meat quality testing device 20 includes a processor 21, a light source 22, a spectral camera 23, a memory 24, and a communication module 25. The processor 21 can control the operation of at least one of the light source 22, the spectral camera 23, the memory 24, and the communication module 25.

[0047] Understandably, the meat quality testing device 20 may also include components such as a power module and a housing.

[0048] In some embodiments, the processor 21 may include processing circuitry, driving circuitry, a central processing unit (CPU), a microcontroller unit (MCU), a digital signal processor (DSP), a graphics processing unit (GPU), an accelerator, a neural processing unit (NPU), a tensor processing unit (TPU), an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA), or other programmable logic devices, gate devices, or transistor logic devices, or similar devices.

[0049] In some embodiments, the light source 22 can have a spectrum covering 385~1200 nm. The visible light band (400~700 nm) is sensitive to pigment changes (such as myoglobin), and the near-infrared band (700~1100 nm) is sensitive to the molecular vibration of water, fat, and protein. The spectrum of the light source 22 covers the visible light band and the near-infrared band, which helps to achieve detection of multiple preset components such as volatile basic nitrogen, total bacterial count, myoglobin, or water.

[0050] In some embodiments, the spectral camera 23 has a spectrum covering 350~950 nm, covering the visible light band and the near-infrared band, and supporting multiple preset component detection. It has a spectral resolution of 1 nm and a spatial resolution of 1600x1200 pixels (pixel size 1.75 m x 1.75 m). The spectral camera 23 can collect spectral information in the 350-1100 nm band, support high-speed acquisition of 30 frame / s, have strong real-time performance, and be designed with a matching module (a multispectral imaging device with both spectral and imaging functions), which helps to achieve real-time detection. The spectral camera 23 is small in size and low in cost, and can be integrated into the refrigerator compartment to monitor the thawing state or freshness of meat in the refrigerator compartment in real time. In some embodiments, the spectral camera 23 and the processor 21 can be independently arranged. In some embodiments, the spectral camera 23 and the processor 21 can be integrated.

[0051] In some embodiments, the spectral camera 23 can capture a spectral image. The spectral camera 23 can include a lens (lens group), a microlens array, a filter unit array, and a photosensor array arranged along an optical path. The microlens can converge light incident thereon on a filter unit. The filter unit can include a filter subunit array. Each filter subunit can have a specific transmittance curve. Each filter subunit can form a spectral channel to allow light of a preset wavelength band to pass through. The working wavelength bands of the filter subunits in each filter unit can be different or partially the same. The filter subunit array in each filter unit can form multiple spectral channels to allow light of multiple preset wavelength bands to pass through. In some embodiments, the filter subunit can include a filter film, a filter sheet, a nanoarray, a grating, or the like to achieve wavelength selection. It should be noted that the present application does not limit the number, arrangement, transmittance, channel number, device type, and the like of the filter subunits in each filter unit, which can be configured according to requirements in actual applications. The photosensor can convert light signals incident thereon into electrical signals. The photosensor can include one or more picture elements. The picture element can serve as the smallest light sensing unit of the photosensor array to convert light signals incident thereon into electrical signals. The picture element can include a photosensitive element such as a photodiode or a phototransistor, for example, a charge-coupled device (CCD) or a complementary metal-oxide-semiconductor (CMOS) device, or the like. The processor 21 can generate a multispectral image according to the electrical signals output by the photosensor. Each pixel in the spectral image can include a three-dimensional data cube. A single shot of the spectral camera 23 can obtain image data of multiple spectral channels. It should be noted that the present application does not limit the correspondence between the microlens, the filter unit, the filter subunit, the photosensor, the picture element, and the image pixel, which can be one-to-many or many-to-one, and can be configured according to requirements in actual applications. In some embodiments, the meat quality detection device 20 can include a refrigerator. The refrigerator includes a refrigeration chamber. The refrigeration chamber can be a closed space, and the spectral camera 23 can be arranged at the top of the closed space. In some embodiments, the light source 22 can be independently arranged at the top of the refrigerator. In some embodiments, the light source 22 can be integrated with the spectral camera 23 and arranged together at the top of the closed space.

[0052] In some embodiments, the memory 24 can include a random access memory (RAM) and can also include a non-volatile memory (NVM). Further, the memory can include at least one of a phase-change random access memory (PRAM), a static random access memory (SRAM), a dynamic random access memory (DRAM), a read-only memory (ROM), an electrically erasable programmable read only memory (EEPROM), and the like.

[0053] In some embodiments, the communication module 25 can include at least one of a Bluetooth module, a WiFi module, a 4G module, a 5G module, a 6G module, and the like. The meat quality detection device 20 can communicate with the user terminal through the communication module 25. For example, the user terminal can send instructions to the meat quality detection device 20 to control the operation of the meat quality detection device 20. The meat quality detection device 20 can feed back the detection results to the user terminal, so that the user can grasp the thawing state or freshness of the meat in real time, and will not miss the best taste and nutritional ingredients, which helps to improve the user experience.

[0054] In some embodiments, the meat quality detection device 20 can include a display. The processor 21 can be coupled to the display to visually output the detection results, so that the user can intuitively grasp the thawing state or freshness of the meat, which helps to improve the user experience.

[0055] In some embodiments, in step S11, a spectral image of a meat detection sample is obtained.

[0056] In some embodiments, the meat detection sample can include frozen meat. The meat detection sample can include meat transferred from a freezer compartment to a refrigeration compartment. For example, the freezer compartment and the refrigeration compartment can be provided in the same device, such as a refrigerator. The freezer compartment and the refrigeration compartment can be provided in different devices, such as the freezer compartment provided in a freezer and the refrigeration compartment provided in a refrigerator. Hereinafter, the meat quality detection method 10 of the present application is introduced by taking the meat transferred from the freezer compartment of the refrigerator to the refrigeration compartment of the refrigerator as an example.

[0057] In some embodiments, the processor 21 can control the light source 22 to turn on, and can control the spectral camera 23 to capture a spectral image in the light source 22 turning-on state. In the light source 22 turning-on state, the light emitted by the light source 22 can be incident on the meat detection sample, the light reflected by the meat detection sample can be sensed by the spectral camera 23, and after photoelectric conversion by the spectral camera 23, the spectral image can be obtained after processing by the processor 21. The spectral image includes a plurality of pixels. Each pixel includes three-dimensional data (x, y, λ). Wherein, x and y represent spatial dimensions, representing the horizontal and vertical positions of the pixel in the image, and λ represents the pixel coordinates. λ represents the spectral dimension, representing the wavelength. The spectral image can reflect the reflection, absorption or reflection characteristics of the object at different wavelengths.

[0058] In some embodiments, the processor 21 can receive a user input instruction, control the light source 22 to turn on or off, and control the spectral camera 23 to capture or not capture an operation based on the user input instruction. For example, the input instruction can include at least one of a voice instruction, a motion instruction, a gesture instruction, an image instruction, or a text instruction. For example, the motion instruction can include opening the refrigerator door, pressing the refrigerator button, etc. For example, when the user opens the refrigerator door, the light source 22 can be triggered to turn on and the spectral camera 23 can be triggered to capture a spectral image.

[0059] In step S12, the meat region is determined according to the spectral image. It can be understood that the spectral image can not only include the meat detection sample, but also can include the environment image in which the meat detection sample is located. For example, the meat detection sample is located in the refrigerator freezer, and the spectral image can include the meat detection sample and the freezer image. The processor determines the meat region according to the spectral image, which can exclude environmental noise such as the freezer, and facilitate subsequent accurate determination of the thawing state or freshness of the meat.

[0060] In some embodiments, the processor can use an image segmentation algorithm to perform image segmentation on the spectral image, distinguish the meat region from the background region, and determine the meat region.

[0061] For example, the processor can determine the meat region based on formula (1).

[0062] M=S(I) ……(Formula 1)

[0063] Wherein, I is the spectral image data (pixel coordinate, wavelength and spectral intensity), S(I) is the image segmentation algorithm, and M is a binary mask (meat region marked as 1 and background marked as 0).

[0064] In some embodiments, at step S13, the processor can determine the thawing state of the meat according to the meat region. The thawing state can reflect the thawing degree. The thawing state can include a plurality of thawing states. For example, the thawing state can include an unthawed state and a thawed state. For example, the thawing state can include a completely thawed state, a half-thawed state, or an unthawed state. It should be noted that these thawing states are only exemplary and the present application is not limited thereto. In actual applications, more thawing states can be set according to actual needs.

[0065] Figure 3 A flowchart for determining the thawing state of the meat according to some embodiments of the present application is shown. As shown, determining the thawing state of the meat includes sub-steps S131 and S133. Figure 3

[0066] At sub-step S131, a first region of the meat region is determined. The meat region includes an ice crystal region and a fat region. The first region is the ice crystal region. The second region is the fat region. It can be understood that the brightness of the ice crystal region is higher than that of the fat region. The gray value can reflect the brightness. The processor can determine the first region based on the gray value of the pixels of the meat region. For example, the processor can read the gray value of the pixels of the meat region, and determine the first region based on the gray value of the pixels of the meat region. For example, the processor can compare the gray value of the pixels of the meat region with a preset gray value, and determine the pixels with a gray value higher than the preset gray value. The first region includes a set of pixels with a gray value higher than the preset gray value.

[0067] It should be understood that determining the first region of the meat region can be understood as detecting the oversaturation point of the meat region. The oversaturation point is the pixel with a gray value higher than the preset gray value. The first region is a set of oversaturation points. The first region is a saturation region. For example, the processor can determine the first region of the meat region based on equation (2).

[0068] … (equation 2).

[0069] wherein, Past (x, y) is the pixel point marker value; τ(λ) is the saturation threshold value (preset gray value); (x, y) is the pixel point coordinate; I(x, y, λ) is the spectral signal of the pixel point (x, y).

[0070] In some embodiments, the saturation threshold value (preset gray value) τ(λ) can be set according to empirical values, which is related to the camera exposure time, the degree of object reflection, and the CMOS sensor itself parameters. For example, τ(λ) can be derived from the physical model of photoelectric conversion and adjusted in combination with experience. The modeling process of τ(λ) is exemplarily described below.

[0071] ​In some embodiments, the processor can determine the basic relationship of the photoelectric signal. For example, the processor can determine the pixel output value S(λ) of the CMOS sensor based on (Formula 3).

[0072] … (Formula 3).

[0073] where G is the sensor gain (including quantum efficiency, ADC conversion coefficient, etc.). Φ(λ) is the incident light flux (related to object reflectivity and illumination). t is the exposure time. D is the dark current noise (which can be ignored or deducted after calibration).

[0074] In some embodiments, the processor can decompose the incident light flux Φ(λ) based on (Formula 4). The incident light flux Φ(λ) depends on:

[0075] … (Formula 4).

[0076] where E(λ) is the scene illumination spectrum (unit: W / m 2 ·nm). R(λ) is the object reflectivity (dimensionless, 0~1). A is the pixel photosensitive area (m 2 ). N is the lens aperture value (f-number).

[0077] In some embodiments, the processor can determine the saturation threshold (preset gray value) τ(λ) based on (Formula 5). The saturation threshold (preset gray value) τ(λ) is defined as: when S(λ) reaches the maximum output Smax of the sensor, the corresponding reflectivity threshold is τ(λ).

[0078] … (Formula 5).

[0079] The processor solves (Formula 5) to obtain: .

[0080] In some embodiments, the processor can introduce empirical corrections to τ(λ), for example, considering the following empirical factors in combination with the actual scene. For example, nonlinear response, the sensor gain G may vary with the intensity of illumination, a nonlinear function can be introduced. For example, noise compensation, an offset b can be added to avoid misjudgment caused by noise. For example, wavelength dependence, the quantum efficiency Q(λ) of the CMOS can be explicitly modeled.

[0081] In some embodiments, the processor can simplify τ(λ). For example, if it is assumed that G and E(λ) are constant under short-time exposure, and the nonlinearity is ignored, then τ(λ) can be simplified to (Formula 6),

[0082] … (Formula 6).

[0083] wherein, C is a camera-related constant, which can be determined by a calibration experiment.

[0084] In some embodiments, the processor can determine the constant C by a calibration method as follows. For example, using a calibration board with a known reflectivity Rcal, adjust the exposure time t until just saturated. Record the tcal at this time, and substitute it into (Formula 7), repeat the experiment for other wavelengths, and fit the Q(λ) curve.

[0085] … (Formula 7).

[0086] wherein, represents a reflectivity threshold of calibration, represents the reflectivity of the calibration board. represents the quantum efficiency of the CMOS sensor, represents the exposure time at just saturation.

[0087] In some embodiments, the general expression of the saturation threshold τ(λ) can be (Formula 8).

[0088] … (Formula 8).

[0089] wherein, b is an empirical offset, and Q(λ) can be obtained by the sensor manual or calibration.

[0090] It should be noted that the foregoing exemplary introduces the determination method of the saturation threshold (preset gray value) τ(λ), and the present application is not limited thereto. In actual application, the saturation threshold (preset gray value) τ(λ) can also be determined by other methods.

[0091] Sub-step S133, determining the thawing state of the meat according to the first region. The first region is an ice crystal region. The gray value of the pixels in the first region is higher than the preset gray value. In some embodiments, the processor can determine the thawing state of the meat according to the pixel parameters of the first region. The pixel parameters can include at least one of the pixel number, the pixel area, the pixel distribution, and the pixel proportion.

[0092] In some embodiments, the processor can determine the pixel number of the first region, and determine the thawing state of the meat based on the pixel number of the first region. For example, when the pixel number of the first region is relatively large (e.g., greater than a preset number), the processor can determine that the meat is in an unthawed state (low thawing degree, most of which is unthawed or even completely unthawed). For another example, when the pixel number of the first region is relatively small (e.g., not greater than the preset number), the processor can determine that the meat is in a thawed state (high thawing degree, most of which is thawed or even completely thawed). By the size relationship between the pixel number of the first region and the preset number, the processor can determine the thawing state of the meat.

[0093] In some embodiments, the processor can determine the thawing state of the meat based on the pixel area of ​​the first region. For example, when the pixel area of ​​the first region is large (e.g., larger than a preset area), the processor can determine that the meat is in an unthawed state (low degree of thawing, mostly thawed or even completely thawed). Conversely, when the pixel area of ​​the first region is small (e.g., not larger than the preset area), the processor can determine that the meat is in a thawed state (high degree of thawing, mostly thawed or even completely thawed). By comparing the pixel area of ​​the first region with the preset area, the processor can determine the thawing state of the meat.

[0094] In some embodiments, the processor can determine the thawing state of the meat based on the pixel distribution of the first region. For example, when the pixel distribution of the first region is relatively uniform (e.g., the distance between adjacent pixels is no greater than a preset distance), the processor can determine that the meat is in an unthawed state (low degree of thawing, mostly thawed or even completely thawed). Conversely, when the pixel distribution of the first region is relatively uneven (e.g., the distance between adjacent pixels is less than a preset distance), the processor can determine that the meat is in a thawed state (high degree of thawing, mostly thawed or even completely thawed). By comparing the distance between adjacent pixels in the first region with the preset distance, the processor can determine the pixel distribution of the first region. Based on the pixel distribution of the first region, the processor can determine the thawing state of the meat.

[0095] In some embodiments, the processor can determine the thawing status of the meat based on the pixel ratio of the first region. Figure 4 This diagram illustrates a process for determining the thawing state of meat based on pixel ratio according to some embodiments of the present invention. Figure 4 As shown, sub-step S133 includes operations S1335 and S1337. In operation S1335, the processor can calculate the proportion of pixels in the first region to the total number of pixels in the spectral image. In operation S1337, the processor can determine the thawing state based on the proportion of pixels in the first region to the total number of pixels in the spectral image.

[0096] In some embodiments, the thawing state can include a fully thawed state and an unthawed state. For example, if the proportion of pixels in the first region to the total number of pixels in the entire spectral image is less than or equal to a first proportion threshold, the processor determines the meat is in a fully thawed state. For example, if the proportion of pixels in the first region to the total number of pixels in the entire spectral image is greater than a second proportion threshold, the processor determines the meat is in an unthawed state. The first proportion threshold is less than the second proportion threshold.

[0097] In some embodiments, the thawing state can include a completely thawed state, a semi-thawed state, or an un-thawed state. For example, if the number of pixels in the first region accounts for a proportion of the total number of pixels in the entire spectral image that is less than or equal to a first proportion threshold, the processor determines that the meat is in a completely thawed state. For example, if the number of pixels in the first region accounts for a proportion of the total number of pixels in the entire spectral image that is greater than the first proportion threshold and less than or equal to a second proportion threshold, the processor determines that the meat is in a semi-thawed state. For example, if the number of pixels in the first region accounts for a proportion of the total number of pixels in the entire spectral image that is greater than the second proportion threshold, the processor determines that the meat is in an un-thawed state. The first proportion threshold is less than the second proportion threshold.

[0098] It should be noted that the foregoing embodiments are exemplarily described by taking two or three thawing states as examples, and the present application is not limited thereto. In actual applications, two, four or more thawing states can be set. In addition, the present application does not limit the specific values of the proportion thresholds, which can be set according to requirements in actual applications.

[0099] In some embodiments, the processor can determine the thawing state of the meat based on one of the pixel parameters including the number of pixels, the pixel area, the pixel distribution, and the pixel proportion of the first region. In some embodiments, the processor can determine the thawing state of the meat based on multiple or all of the pixel parameters including the number of pixels, the pixel area, the pixel distribution, and the pixel proportion of the first region.

[0100] In some embodiments, in step S13, the processor can determine the freshness of the meat according to the meat region. The freshness can reflect the freshness degree. The freshness can include multiple freshnesses. For example, the freshness can include fresh and not fresh. For example, the freshness can include fresh, sub-fresh, and not fresh. It should be noted that these freshnesses are exemplarily described, and the present application is not limited thereto. In actual applications, more freshnesses can be set according to actual requirements.

[0101] Figure 5 A flowchart for determining the freshness of meat according to some embodiments of the present application is shown. As shown in FIG. 13, determining the freshness of meat includes sub-steps S132 and S134. Figure 5

[0102] ​In sub-step S132, a second region of the meat region is determined. The second region is a fat region. The processor can determine the second region based on the gray value of the pixels of the meat region. For example, the processor can read the gray value of the pixels of the meat region, and determine the second region based on the gray value of the pixels of the meat region. For example, the processor can compare the gray value of the pixels of the meat region with a preset gray value, and determine the pixels with a gray value not higher than the preset gray value. The second region includes a set of pixels with a gray value not higher than the preset gray value. It should be understood that determining the second region of the meat region can be understood as detecting non-over-saturation points of the meat region. The non-over-saturation points are pixels with a gray value not higher than the preset gray value. The second region is a set of non-over-saturation points. The second region is a non-saturation region. For example, the processor can determine the second region of the meat region based on equation (2).

[0103] In sub-step S134, the freshness of the meat is determined according to the second region. The second region is a fat region. The second region is a non-saturation region. The gray value of the pixels of the second region is not higher than the preset gray value. The processor can determine the freshness of the meat according to the pixels of the second region.

[0104] When the frozen meat is subjected to spectral detection, ice crystals will interfere due to the effect of surface frost. The strong absorption of ice at 1020 nm and 1230 nm will completely mask the characteristic peaks of ammonia substances. The ice crystal absorption peak (1020 nm and 1230 nm): the O-H bond vibration in the ice crystal (the cooperative vibration of the hydrogen bond of water molecules) produces strong absorption at 1020 nm, which directly overlaps with the N-H bond secondary frequency (1050 nm) and C-N stretching vibration (1130 nm) of TVB-N, resulting in the TVB-N signal being masked. TVB-N is an important indicator for detecting the freshness of meat, and the characteristic peak of the TVB-N signal is masked by the ice crystal, which adversely affects the freshness detection. The present application detects the freshness of meat based on the second region, which can exclude the interference of ice crystals, and helps to accurately determine the freshness of meat.

[0105] Figure 6 A flowchart for determining the freshness of meat according to some embodiments of the present application is shown. As shown in FIG. 13, sub-step S134 includes operations S1340-S1344. Figure 6

[0106] ​Operation S1340, based on the pixels of the second region, the spectral inversion is performed to obtain a spectral curve. For example, the processor can perform spectral inversion based on the pixel data Data of the second region to obtain a spectral curve. In some embodiments, each pixel can include data of multiple channels. For example, the multiple channels can include 2, 3, 4, 5, 6, 7, 8, 9 or even more channels. The processor can average the data of the corresponding channels of the pixels of the second region to obtain a channel mean value data. The processor can perform spectral inversion on the second region based on the channel mean value data to obtain a spectral curve Spec by (Formula 9). For example, the wavelength range of the spectral curve can be 350-950nm.

[0107] Spec = sprinv(light, data) …… (Formula 9).

[0108] Wherein, sprinv is a spectral inversion function, light is a light source spectrum, and data is a plurality of channel mean values of the non-saturated region.

[0109] Operation S1342, a regression model of the spectral curve and the preset component is constructed. In some embodiments, the preset component (detection index or detection component) can include at least one of volatile basic nitrogen, total bacterial count, myoglobin or moisture. One regression model corresponds to one preset component. The processor can construct a regression model of the spectral curve and at least one of volatile basic nitrogen, total bacterial count, myoglobin or moisture. For example, the processor can construct a regression model of the spectral curve and volatile basic nitrogen. For example, the processor can construct a regression model of the spectral curve and total bacterial count. For example, the processor can construct a regression model of the spectral curve and myoglobin. For example, the processor can construct a regression model of the spectral curve and moisture.

[0110] TVB-N is the total of basic nitrogen compounds produced by the decomposition of protein in the meat during the spoilage process. These compounds include ammonia, amine compounds and other volatile nitrogen-containing substances. TVB-N content has a significant correlation with meat freshness, and the national meat safety standard stipulates TVB-N as the main detection index. TVB-N content is negatively correlated with meat freshness, the higher the content, the less fresh the meat; the lower the content, the fresher the meat.

[0111] TVC represents the total number of bacterial colonies per gram or per milliliter of the sample under certain conditions. TVC content has a significant correlation with meat freshness, and TVC content is negatively correlated with meat freshness, the higher the content, the less fresh the meat; the lower the content, the fresher the meat.

[0112] The number of coliform bacteria (ANC) is not a certain kind of bacteria or a certain genus of bacteria, but a group of bacteria with certain characteristics related to fecal contamination, which is often present in animal intestines, feces and meat contaminated by feces, is a kind of microorganism that can cause meat spoilage, and is a microbial index that must be detected by meat food enterprises when leaving the factory. The ANC content is negatively correlated with meat freshness. The number of coliform bacteria (ANC) can be detected by an instrument.

[0113] The water holding capacity is the ability of muscle to retain water when subjected to external forces such as pressure, heating, freezing and chopping. The water holding capacity can reflect the tenderness of meat. The water holding capacity is positively correlated with the tenderness of meat. The higher the water holding capacity, the more tender the meat. The lower the water holding capacity, the older the meat.

[0114] The pH value can reflect the acidity and alkalinity of meat. The glycogen in the meat will ferment to produce lactic acid at the early stage of storage, and adenosine triphosphate will decompose into phosphoric acid. The accumulation of lactic acid and phosphoric acid will reduce the pH value. The smaller the pH value, the more acidic and more corrupt the meat. The pH value can be detected by a pH meter.

[0115] Myoglobin can reflect the color of meat. With the oxidation time of myoglobin becoming longer, the color of meat gradually changes from red to brown.

[0116] In some embodiments, each TVB-N content determination is performed according to GB 5009.228-2016 National Food Safety Standard Determination of Volatile Nitrogen in Food; the pH value determination is performed according to GB 5009.237-2016 National Food Safety Standard Determination of pH Value of Food; and the color determination of pork refers to NY / T 2793-2015 Objective Evaluation Method for Meat Edible Quality. It should be noted that the detection results of different detection methods may be slightly different for the same detection index. Each detection index and its sensitive spectral band in Table 1 is only exemplary, and the present application is not limited thereto.

[0117] Table 1

[0118]

[0119] The present application does not limit the construction method of the regression model. In some embodiments, the processor can construct a regression model of the spectral curve and the preset component based on at least one of a partial least squares regression algorithm (Plsr), a support vector machine algorithm (SVM) or a decision tree algorithm. The decision tree algorithm can include a random decision tree and the like.

[0120] In some embodiments, the processor can preprocess the data. For example, the processor can align and label the data, ensuring that the multispectral image bands of each meat sample are accurately matched with its thawing time (e.g., 1-12h, 1-24h, etc.) and the corresponding values of the detection indicators. For example, the processor can normalize the data, i.e., calculate the mean and standard deviation of each band (channel) of the image separately and normalize it. For example, the processor can extract the spectrum, calculate the average spectrum of the entire region of interest (ROI), and obtain a one-dimensional spectrum vector. In some embodiments, the ROI region can be the entire second region. In some embodiments, the ROI region can be a local region of the second region.

[0121] It should be noted that the present application does not limit the specific type of meat. For example, it can be pork, chicken, fish, shrimp, duck, beef, mutton, etc. Different types of meat may have differences between parameters, but the method of the present application is universal and can be used for thawing state or freshness detection of various types of meat.

[0122] Figure 7 A flowchart illustrating operation S1342 according to some embodiments of the present application is shown. As shown, operation S1342 includes sub-operations OP1-OP3. Figure 7

[0123] Sub-operation OP1 extracts image features and spectral features based on the second region. The image features can include one or more of color features, texture features, shape features, spatial relationship features, edge features, contour features, corner features, semantic features, etc. In some embodiments, the processor can extract the image features through an image feature extraction module. For example, the image feature extraction module can include a multi-stage convolutional neural network. The processor can extract image features based on the multi-stage convolutional neural network. Based on the multi-stage neural network, the processor can extract and compress the image features layer by layer to obtain multiple image features.

[0124] Figure 8 A schematic diagram of obtaining fused features according to some embodiments of the present application is shown. As shown, Figure 8 ​As shown, the image feature extraction module Mi can include a multi-stage convolutional neural network. The multi-stage convolutional neural network includes a first-stage convolutional neural network NET1, a second-stage convolutional neural network NET2, a third-stage convolutional neural network NET3, and a fourth-stage convolutional neural network NET4. In some embodiments, an output end of the first-stage convolutional neural network NET1 is connected to an input end of the second-stage convolutional neural network NET2. An output end of the second-stage convolutional neural network NET2 is connected to an input end of the third-stage convolutional neural network NET3. The architectures of the first-stage convolutional neural network NET1, the second-stage convolutional neural network NET2, and the third-stage convolutional neural network NET3 can be the same. Each stage neural network can include a two-dimensional convolutional neural network. Each stage neural network can include a convolution operation module, a normalization module, a nonlinear activation module, and a pooling module, for extracting and compressing spatial features layer by layer. By scanning the input data using a sliding filter through the convolution operation module, the processor can extract local spatial features. By scanning the input data using multiple filters, the processor can extract feature map data of different dimensions. By standardizing the feature map data through the normalization module, the processor can stabilize the data distribution before activation. By using a nonlinear function to map the normalized features through the nonlinear activation module, the processor can perform down-sampling on the activated feature maps through the pooling module, compress the spatial size, reduce the computational load, and enhance the spatial invariance of the features. While saving computing power, the reliability and robustness of the feature extraction results are taken into account.

[0125] In some embodiments, extracting the image features includes: performing convolution processing on the second region based on the first-stage convolutional neural network, the second-stage convolutional neural network, and the third-stage convolutional neural network, to extract the image features and obtain a first feature map. For example, as shown in FIG. 4, the processor can input the pixel data D2 (Data / data) of the second region into the first-stage convolutional neural network NET1, and extract image features of different dimensions through the convolution kernels of the first-stage convolutional neural network NET1, the second-stage convolutional neural network NET2, and the third-stage convolutional neural network NET3, respectively, to obtain the first feature map Fimg1 and output the first feature map Fimg1. Figure 8 As shown, the processor can input the pixel data D2 (Data / data) of the second region into the first-stage convolutional neural network NET1, and extract image features of different dimensions through the convolution kernels of the first-stage convolutional neural network NET1, the second-stage convolutional neural network NET2, and the third-stage convolutional neural network NET3, respectively, to obtain the first feature map Fimg1 and output the first feature map Fimg1.

[0126] In some embodiments, extracting the image features further includes: determining a channel feature vector of the first feature map based on a channel attention mechanism. For example, as shown in FIG. 5, the processor can determine the channel feature vector of the first feature map Fimg1 based on the channel attention mechanism. Figure 8As shown, the output end of the third-stage convolutional neural network NET3 is connected to the input end of the channel attention module SE. The output of the third-stage convolutional neural network NET3 can be used as the input of the channel attention module SE. The first feature map Fimg1 is output from the third-stage convolutional neural network NET3 and input to the channel attention module SE. The processor can learn the feature relationship between the channels of the first feature map Fimg1 through the channel attention module SE and based on the channel attention mechanism, and determine the channel feature vector Z based on the feature relationship between the channels. For example, the first feature map Fimg1 (X ∈ R, B * C * H * W). Wherein, X ∈ R indicates that the data type of the data X of the first feature map Fimg1 is real number. B represents batch size, which is the number of image samples processed at a time. C represents the number of channels. H represents the height of the image. W represents the width of the image. The processor can perform global average pooling on the first feature map Fimg1 in the spatial dimension to obtain the statistical description Zc of each channel.

[0127] .

[0128] Wherein, i represents the height direction H index of the first feature map Fimg1, and the value range of i is 1 to H. j represents the width direction W index of the first feature map Fimg1, and the value range of j is 1 to W. b represents the batch index. c represents the channel index.

[0129] Based on the formula, the processor can perform global average pooling on the feature map of a certain batch and a certain channel of the first feature map Fimg1 to obtain the statistical description Zc of each channel. According to the statistical description Zc of each channel, the processor can obtain the channel feature vector Z of each channel. It should be noted that Zc is a scalar (one scalar for one channel), and Z is a vector, which is a collection of multiple or even all Zc.

[0130] In some embodiments, the extracting the image features further includes: obtaining a channel weight of the first feature map based on the channel feature vector and a parameter mapping function. For example, the processor can input the feature vector Z into the parameter mapping function F(Z) and perform normalization processing through the normalization function σ to obtain the channel weight S = σ(F(Z)). For example, the parameter mapping function F(Z) can be a nonlinear activation function.

[0131] In some embodiments, the extracting the image features further includes: adjusting the channel weights of the first feature map based on the channel weights to obtain a second feature map. For example, the processor can adjust the channel data of the first feature map Fimg1 based on the channel weights S to obtain a second feature map Fimg2 and output. The processor assigns a weight coefficient to each channel based on the channel attention mechanism, which can highlight important spectral bands and suppress redundant bands, helping to more accurately determine the freshness of meat subsequently.

[0132] In some embodiments, the extracting the image features further includes: performing convolution processing on the second feature map based on a fourth stage convolutional neural network to obtain a third feature map. The third feature map includes image features. The third feature map includes a global image feature vector. As shown in Figure 8 The output end of the channel attention module SE is connected to the input end of the fourth stage convolutional neural network NET4. The output of the channel attention module SE can be used as the input of the fourth stage convolutional neural network NET4. The second feature map Fimg2 is output from the channel attention module SE and input to the fourth stage convolutional neural network NET4. The processor can perform a global pooling operation on the third feature map Fimg3 through the fourth stage convolutional neural network NET4 to compress the spatial dimension to a preset size Fimg3 (Fimg3 ∈ R, B*C1*1*1), thereby obtaining a global image feature vector Fimg. In an example, the global pooling operation can include a global average pooling operation. It should be understood that the global pooling operation is essentially a pooling operation on the entire spatial range (i.e., the H × W dimension) of the third feature map Fimg3. In some embodiments, the extracting the spectral features includes: dividing the spectral curve into a plurality of local bands; performing convolution processing on the local bands to extract local features; performing convolution processing on the local features to obtain cross-band features; obtaining a spectral feature map based on the cross-band features; and the spectral feature map includes a global spectral feature vector. Details are described below.

[0133] In some embodiments, as shown in Figure 8 The processor can extract spectral features through the spectral feature extraction module Ms. In an example, the spectral feature extraction module Ms can include a one-dimensional convolutional neural network. The spectral feature extraction module Ms includes a local band division module Ms1, a local feature extraction module Ms2, a cross-band feature extraction module Ms3, and a spectral feature generation module Ms4.

[0134] In an example, the processor can divide the 350-950nm spectral curve Spec into a plurality of local bands λ (assuming λ = [λ1, λ2, λ3…λ601] ∈ R, B*601) by the local band division module Ms1 with a unit of 1nm.

[0135] The local feature extraction module Ms2 may, for example, include a one-dimensional convolutional neural network. The processor may use the local feature extraction module Ms2 to perform one-dimensional convolutional processing on the input spectrum (local waveband λ) to extract local features H(1) and obtain fine-grained changes in the short waveband. The local feature extraction by convolutional processing on the local waveband may be understood as modeling the local feature waveband.

[0136] H(1) = σ(BN(W * X + b)).

[0137] where σ is a nonlinear activation function, * is one-dimensional convolution operation, W is a convolution kernel parameter, b is a bias term, and B is batch normalization.

[0138] The cross-waveband feature extraction module Ms3 may, for example, include a one-dimensional convolutional neural network based on the local features. The processor may use the cross-waveband feature extraction module Ms3 to perform convolutional processing on the local features H(1) to obtain cross-waveband features H(2). In some embodiments, the processor may use the cross-waveband feature extraction module Ms3 to perform convolutional processing on the local features using convolution kernels of different sizes to obtain the cross-waveband features and obtain overall trends in the cross-waveband. In some embodiments, the processor may perform parallel convolutional processing on the local features using convolution kernels of different sizes to improve processing efficiency. The convolutional processing on the local features to obtain the cross-waveband features may be understood as modeling the cross-waveband features.

[0139] H(2) = Concat(σ(BN(W * X + b)), σ(BN(W * X + b)), σ(BN(W * X + b))).

[0140] The processor may, for example, use the spectral feature generation module Ms4 to process the cross-waveband features H(2) to obtain a spectral feature map. The spectral feature map includes spectral features. The spectral feature map includes a global spectral feature vector. The cross-waveband features may be presented in a sequence. The spectral feature generation module Ms4 may include a pooling layer Ms41 and a fully connected layer Ms42. The processor may use the pooling layer Ms41 to perform global average pooling on the cross-waveband feature sequence to reduce the dimensionality and obtain a global spectral feature H(3).

[0141] H(3) = .

[0142] where L is the length of the spectral sequence. H(3) may represent calculating the average value over the entire spectral sequence L for each feature channel.

[0143] For example, the processor can use a fully connected layer Ms42 to map the global spectral features H(3) into a low-dimensional feature vector, resulting in the global spectral feature vector Fspec. Here, Fspec ∈ R, B*C1. It should be understood that the core operation of the fully connected layer is matrix multiplication, which flattens the input global spectral features H(3) into a vector and performs dimensionality reduction mapping to obtain the global spectral feature vector Fspec.

[0144] Fspec=W*H3+b.

[0145] Operation OP2 obtains the fused features based on image features and spectral features.

[0146] In some embodiments, such as Figure 8 As shown, the processor can input image features and spectral features into the fusion module Mf, which then fuses the image features and spectral features to obtain fused features. In some embodiments, the fusion module Mf can be a multimodal fusion module. The multimodal fusion module, based on the transformer architecture, can achieve deep, bidirectional interaction and fusion of image features and spectral features. In some embodiments, the processor can use the fusion module Mf, based on a transformer-based multimodal fusion algorithm, to fuse the global image feature vector Fimg and the global spectral feature vector Fspec to obtain the fused feature vector Ffused. The deep learning module based on the transformer architecture can achieve deep, bidirectional, and dynamic interaction and fusion between two heterogeneous modal features (such as images and spectra), overcoming the limitations of simple splicing or unidirectional activation. Through attention mechanisms, it allows modalities to query and complement each other, generating richer joint representations.

[0147] Figure 9 A schematic flowchart illustrating the generation of a fused feature vector sequence according to some embodiments of the present invention is shown. Figure 9 As shown, generating the fused feature vector sequence includes operations OP21~OP24. See below for reference. Figure 8 , 9 Let me introduce it.

[0148] The operation OP21 is to construct a first sequence and perform position encoding based on the global image feature vector and the global spectral feature vector. For example, the processor can fuse the global image feature vector Fimg and the global spectral feature vector Fspec through a multi-modal fusion module. The input of the multi-modal fusion module Mf is connected to the output of the image feature extraction module Mi and the spectral feature extraction module Ms. The processor fuses the features of the two modalities of the image and the spectrum based on the global image feature vector and the global spectral feature vector through the multi-modal fusion module Mf to obtain the first sequence Z1. The first sequence Z1 can be regarded as a sequence with a length of 2.

[0149] The first sequence Z1 includes a plurality of elements. Each element of the first sequence Z1 includes an image feature and a spectral feature. The processor can perform position encoding on each element of the first sequence Z1. Each element of the first sequence Z1 corresponds to a position encoding, so that the image feature, the spectral feature, and the position encoding of each element are associated.

[0150] Z1 = stack([Fspec, Fimg], dim = 1), Z1 ∈ R B×2×D .

[0151] In the sequence construction operation, the processor can concatenate the projected global image feature vector Fimg and the global spectral feature vector Fspec in the sequence length dimension (dim = 1). For example, the global spectral feature vector Fspec can be located at the first position (index 0) of the sequence, and the global image feature vector Fimg can be located at the second position (index 1) of the sequence to obtain the concatenated feature vector [Fspec, Fimg]. Alternatively, the global image feature vector Fimg can be located at the first position (index 0) of the sequence, and the global spectral feature vector Fspec can be located at the second position (index 1) of the sequence to obtain the concatenated feature vector [Fimg, Fspec]. In addition, the sequence length dimension is not limited to (dim = 1). In actual applications, it can be adjusted according to requirements.

[0152] The operation OP22 is to inject learnable or fixed position information into each position in the first sequence to obtain a second sequence. For example, the processor can inject learnable or fixed position information into each position in the first sequence to obtain the second sequence Z2. The second sequence includes a plurality of elements. Each element of the second sequence corresponds to a position encoding, and a position encoding corresponds to a position information, so that the image feature, the spectral feature, the position encoding, and the position information of each element are associated.

[0153] Z2 = Z1 + P, where P ∈ R 2×D .

[0154] Here, P represents the positional encoding matrix with a shape of (2, D), where 2 corresponds to the length of the first sequence Z1. The processor can perform positional encoding addition operations on the first two positions (index 0 and index 1) of the first sequence Z1, assigning a unique encoding vector of dimension D to each position (i.e., the 0th and 1st positions) of the first sequence Z1. For example, the processor can use a broadcast mechanism to add the positional encoding matrix P of shape (2, D) to the first sequence Z1 of shape (B, 2, D). For each sample in the batch, the features of position 0 (index 0) are fused with P[0, :], and the features of position 1 (index 1) are fused with P[1, :]. P is equivalent to a 2*D matrix, where P[0,:] means the first row has dimension D, and P[1,:] means the second row has dimension D. In this way, the processor can inject learnable or fixed positional information into each position in the first sequence Z1 to obtain the second sequence Z2. The shape of the second sequence Z2 is (B, 2, D). The features of each element in the second sequence Z2 have been incorporated into its positional information within the sequence. This positional information can be absolute (fixed) positional information or learnable positional information.

[0155] Operation OP23, based on a multi-head attention mechanism and a feedforward neural network, performs feature interaction on the image and spectral features of the second sequence to obtain the third sequence. For example, the processor can... l A series of cascaded Transformer encoding layers, based on a multi-head attention mechanism and a feedforward neural network, interact with the features of each element in the second sequence to obtain the third sequence Z3.

[0156] In some embodiments, the processor can interact with the features of each element of the second sequence Z2 based on a multi-head attention mechanism to obtain sequence Z3'.

[0157] Z3'= LayerNorm(Z2 l-1 + MHSA(Z2 l-1 )).

[0158] Among them, Z2 l-1 Indicates the first l The output of the -1 layer Transformer encoder, which is also the... l The input to the Transformer encoder. For example, for the first Transformer encoding layer, its input is the second sequence Z2. MHSA represents the multi-head self-attention function. LayerNorm represents the layer normalization function. The processor uses the layer normalization function LayerNorm... lThe Transformer encoding layer and the multi-head self-attention function MHSA in series process the second sequence Z2, which can normalize the feature dimension of each element of the second sequence Z2 to obtain a sequence Z3'. It should be understood that the sequence Z3' is an intermediate sequence for obtaining the third sequence Z3 based on the second sequence Z2. The feature of each position of the sequence Z3' is a "mixed feature" that fuses the image and spectral modal information. In some embodiments, the processor can be based on the sequence Z3' and based on a feedforward neural network to obtain a sequence Z3''through l The Transformer encoding layer processes the features of each element of the sequence Z3' to obtain a third sequence Z3. The shape of the third sequence Z3 is still (B, 2, D), but the feature of each position fuses the information of another position. The feature of each position of the third sequence Z3 is a "mixed feature" that fuses the image and spectral modal information. It should be noted that the third sequence Z3 is a sequence obtained by performing a nonlinear transformation on the sequence Z3' (intermediate sequence), and the representation ability of the fused feature of each position in the third sequence Z3 is stronger.

[0159] Z3 = LayerNorm(Z3' + FFN(Z3')).

[0160] Where FFN represents a feedforward neural network.

[0161] Operation OP24, based on the global spectral feature vector and the third sequence, performs attention pooling to generate a fused feature vector sequence.

[0162] For example, the processor can generate an attention weight matrix Attention(Q, K, V) based on the third sequence Z3. For each element Z3 in the third sequence, Z3 ∈ R 2xD The multi-head attention mechanism enables it to "focus" on all other elements (including itself) in the sequence and dynamically aggregate information based on the correlation. For the input third sequence, the processor can map it to a query matrix Query (Q), a key matrix Key (K), and a value matrix Value (V). The query matrix is used to generate a query vector. The key matrix is used to generate a key vector. The value matrix is used to generate a value vector. The three matrices are used to calculate the attention weight, so as to perform weighted summation on the input sequence to obtain the final representation. The attention mechanism is determined by the similarity of the query matrix Q and the key matrix K, and the output is the weighted sum of the value matrix V, which realizes deep bidirectional interaction, so that the image feature can be updated based on the spectral feature, and vice versa.

[0163] The attention weight matrix Attention(Q, K, V) = softmax( ) V.

[0164] where sofmax represents an activation function. The activation function sofmax can normalize a numerical vector into a probability distribution vector, and the sum of each probability is 1.

[0165] In an example, the processor can perform attention pooling based on the attention weight matrix Attention(Q, K, V) and the global spectral feature vector Fspec to generate a fused feature vector sequence. The fused feature vector sequence includes a fused feature vector Ffused. The core of generating the fused feature vector sequence is a cross-attention operation. The spectral feature contains more global and abstract semantic information (such as object category, etc.). The processor can use the global spectral feature vector Fspec as the Q matrix, use the third sequence Z3 after the depth interaction as the K matrix and the V matrix, dynamically extract important information (for example, sensitive spectral bands of preset components, etc.) from the fused knowledge base according to the requirement of the original spectral feature, and summarize into the fused feature vector Ffused.

[0166] Ffused = CrossAttention(Q=Fspec, K=Z3, V=Z3).

[0167] where CrossAttention represents a cross-attention module.

[0168] ‌Operation OP3, constructing a regression model according to the fused feature. In some embodiments, the processor can construct a regression model according to the fused feature vector Ffused and a classifier. In an example, the fused feature vector Ffused includes information such as sensitive spectral bands of preset components. The classifier includes a plurality of category spaces. Each category space corresponds to a content range of a preset component. The plurality of category spaces one-to-one correspond to a plurality of freshness. The processor can associate the sensitive spectral bands of the preset components, the content range, the category space, and the freshness according to the fused feature vector Ffused and the classifier, so as to obtain the regression model. In some embodiments, one regression model can correspond to one preset component. For example, a regression model of a spectral curve and volatile basic nitrogen, a regression model of a spectral curve and total bacterial count, a regression model of a spectral curve and myoglobin, a regression model of a spectral curve and moisture, etc. In some embodiments, one regression model can correspond to a plurality of preset components. For example, a regression model of a spectral curve and volatile basic nitrogen, total bacterial count, myoglobin, and moisture. In some embodiments, one regression model can correspond to two freshness. In some embodiments, one regression model can correspond to three or even more freshness. In some embodiments, the fused feature vector Ffused can also fuse information such as the number of coliform bacteria (ANC) or the PH value, and the regression model can also increase the detection dimension of the number of coliform bacteria (ANC) or the PH value.

[0169] Operation S1344, based on the regression model, determine the freshness of the meat. In some embodiments, the processor can determine the freshness of the meat based on one regression model. For example, the processor can determine the freshness of the meat based on the regression model of the spectral curve and the volatile basic nitrogen. For example, the processor can determine the freshness of the meat based on the regression model of the spectral curve and the total number of colonies. For example, the processor can determine the freshness of the meat based on the regression model of the spectral curve and the myoglobin. For example, the processor can determine the freshness of the meat based on the regression model of the spectral curve and the moisture. In some embodiments, the processor can determine the freshness of the meat based on multiple regression models. For example, the processor can determine the freshness of the meat based on the regression model of the spectral curve and the volatile basic nitrogen, the regression model of the spectral curve and the total number of colonies, the regression model of the spectral curve and the myoglobin, the regression model of the spectral curve and the moisture, or several or even all of the regression models. Each regression model can have a corresponding weight, and the weights of different regression models can be the same or different, and can be set according to requirements. One regression model corresponds to one index, and multiple regression models correspond to multiple indexes. Based on multiple regression models to determine the freshness, multiple indexes can be determined, and the freshness of the meat can be more rich, accurate, reliable, multi-dimensional and comprehensive.

[0170] In some embodiments, the processor can input the fusion feature vector sequence into the classifier; map the fusion feature vector sequence to multiple category spaces through the classifier; obtain the scores of the fusion feature vector sequence in the multiple category spaces; and determine the freshness of the meat based on the scores. The multiple category spaces correspond to multiple freshness. The processor can determine the highest score in the scores of the fusion feature vector sequence in the multiple category spaces, and determine the freshness of the meat based on the freshness corresponding to the category space corresponding to the highest score.

[0171] For example, the multiple freshness can include fresh and not fresh. Figure 10 A schematic diagram for determining the freshness of meat according to some embodiments of the present application is shown. As shown in FIG. 13, the processor can determine the freshness of the meat based on the regression model of the spectral curve and the volatile basic nitrogen, the regression model of the spectral curve and the total number of colonies, the regression model of the spectral curve and the myoglobin, the regression model of the spectral curve and the moisture, or several or even all of the regression models. Figure 10As shown, freshness corresponds to the first category space c1, and staleness corresponds to the second category space c2. The processor can input the fused feature vector sequence ZFfused into the classifier Mc, which maps the ZFfused feature vector sequence to the first category space c1 and the second category space c2. The processor can obtain the score Score1 of the ZFfused feature vector sequence in the first category space c1 and the score Score2 in the second category space c2, compare their magnitudes, and determine the highest score ScoreMAX based on the comparison result. Based on the highest score ScoreMAX, the freshness Lfresh of the meat is determined. For example, if Score1 is greater than Score2, then Score1 is the highest score ScoreMAX, and the processor can determine the freshness of the meat as freshness corresponding to the first category space. For example, if Score2 is greater than Score1, then Score2 is the highest score ScoreMAX, and the processor can determine the freshness of the meat as staleness corresponding to the second category space.

[0172] For example, various levels of freshness can include fresh, slightly fresh, and not fresh. Figure 11 A schematic diagram illustrating the determination of meat freshness according to some embodiments of the present invention is shown. Figure 11 As shown, freshness corresponds to the first category space c1. Staleness corresponds to the second category space c2. Second freshness corresponds to the third category space c3. The processor can input the fused feature vector sequence ZFfused into the classifier Mc, which maps the ZFfused feature vector sequence to the first category space c1, the second category space c2, and the third category space c3. The processor can obtain the scores Score1 (first category space c1), Score2 (second category space c2), and Score3 (third category space c3) of the fused feature vector sequence ZFfused, determine the highest score ScoreMAX, and determine the freshness of the meat based on the freshness corresponding to the category space of the highest score ScoreMAX. For example, if Score1 is the highest score ScoreMAX, the processor can determine the freshness of the meat as freshness corresponding to the first category space c1. For example, if Score2 is the highest score ScoreMAX, the processor can determine the freshness of the meat as staleness corresponding to the second category space c2. For example, if Score3 is the highest score ScoreMAX, the processor can determine that the freshness of the meat is the second freshness corresponding to the third category space c3.

[0173] It should be noted that the present application does not limit the types of freshness and the number of spatial categories, and in actual application, the number can be set according to the needs. It can be understood that these scores represent the prediction strength of each category by the regression model. In some embodiments, the processor can convert the scores into a probability distribution based on the Softmax function, so that the output of the regression model can be interpreted as the probability of belonging to each category. The sum of the probabilities of multiple categories is 1.

[0174] In some embodiments, the meat quality detection method 10 further comprises: performing morphological processing on the meat region, including at least one of the following operations: performing an open operation on a first region of the meat region; performing a close operation on a second region of the meat region. The processor can only perform an open operation on the first region of the meat region, or can only perform a close operation on the second region of the meat region, or can perform an open operation on the first region of the meat region and a close operation on the second region. The first region is an ice crystal region. The processor applies morphological processing to perform an open operation (first erosion and then dilation) on the ice crystal region, which can optimize the ice crystal region, remove noise points, and help more accurately determine the thawing state of the meat. The second region is a fat region. The processor applies morphological processing to perform a close operation (first dilation and then erosion) on the fat region, which can optimize the fat region, fill small holes, enhance continuity, and help more accurately determine the freshness of the meat.

[0175] The meat quality detection method of the present application can monitor the changes of frozen meat during thawing in real time, can exclude ice crystal interference, supports multiple preset component detection, monitors the thawing state or freshness of frozen meat in real time, accurately and reliably, maximizes the preservation of the nutrition and taste of meat, maintains the quality and safety of meat, and is suitable for planned cooking needs, which helps to improve user experience.

[0176] The meat quality detection device of the present application is small in size, low in cost, and convenient to integrate into a refrigerator refrigeration chamber, so that users can monitor the thawing state or freshness of meat in the refrigeration chamber in real time.

[0177] The present application also provides a computer readable storage medium. The computer readable storage medium includes computer executable instructions stored thereon, which, when executed by a processor, implement the meat quality detection method 10.

[0178] In some embodiments, the present invention may take the form of a computer program product implemented on one or more storage media containing program code. Computer-usable storage media include permanent and non-permanent, removable and non-removable media, and information storage can be implemented by any method or technology. Information may be computer-readable instructions, data structures, program modules, or other data. Examples of computer storage media include, but are not limited to: PRAM, SRAM, DRAM, other types of RAM, ROM, EEPROM, flash memory or other memory technologies, compact disc read-only memory (CD-ROM), digital video disc (DVD) or other optical storage, magnetic tape, magnetic disk storage or other magnetic storage devices, or any other non-transfer medium that can be used to store information accessible by a computing device.

[0179] It should be noted that this specification provides method operation steps as shown in the embodiments or diagrams, but based on conventional or non-inventive labor, more or fewer operation steps may be included. The order of steps listed in the embodiments is merely one possible execution order among many steps and does not represent the only execution order. In actual system or device products, the methods shown in the embodiments or flowcharts can be executed sequentially or in parallel.

[0180] It should be noted that although several modules of the meat quality testing device have been mentioned in the detailed description above, this division is not mandatory. In fact, according to embodiments of the present invention, the features and functions of two or more modules described above can be implemented in one module. Conversely, the features and functions of one module described above can be further divided and embodied by multiple modules. Furthermore, the various modules mentioned in this invention can be implemented in hardware, in software, or a combination of hardware and software.

[0181] It should be noted that the present invention may include Figures 1-11 Any one or more features of any one or more embodiments. In other words, not all features shown in the figures need to be implemented simultaneously in the meat quality testing method / meat quality testing device / computer-readable storage medium of the present invention.

[0182] Finally, it should be noted that the above only describes the preferred embodiments of the present application and is not intended to limit the present application. Although the present application has been described in detail with reference to the foregoing embodiments, those skilled in the art will appreciate that the technical solutions described in the foregoing embodiments can be modified or some technical features thereof can be replaced by equivalent ones. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present application shall be included in the protection scope of the present application.

Claims

1. A method for detecting meat quality, characterized in that, include: Obtain spectral images of meat samples for testing; Based on the spectral image, the meat region was determined; Based on the meat section, determine the thawed state or freshness of the meat; The process of determining the thawing state of the meat includes: determining a first region of the meat area; and determining the thawing state of the meat based on the first region, including: determining the thawing state of the meat based on the pixel parameters of the first region; wherein the grayscale value of the pixels in the first region is higher than a preset grayscale value; and the pixel parameters include at least one of the following: number of pixels, pixel area, pixel distribution, and pixel ratio. The determination of the freshness of the meat includes: determining a second region of the meat area; determining the freshness of the meat based on the second region, including: performing spectral inversion based on the pixels of the second region to obtain a spectral curve; constructing a regression model of the spectral curve and preset components; determining the freshness of the meat based on the regression model; wherein the grayscale value of the pixels in the second region is not higher than a preset grayscale value.

2. The meat quality testing method according to claim 1, characterized in that, Determining the thawed state of the meat includes: Calculate the proportion of the number of pixels in the first region to the total number of pixels in the spectral image; The thawing state is determined based on the ratio.

3. The meat quality testing method according to claim 2, characterized in that, The thawing state includes a fully thawed state, a partially thawed state, or an unthawed state. Determining the thawing state includes: If the ratio is less than or equal to the first ratio threshold, the meat is in a completely thawed state; If the ratio is greater than the first ratio threshold and less than or equal to the second ratio threshold, the meat is in a semi-thawed state; If the ratio is greater than the second ratio threshold, the meat is in an unthawed state. Wherein, the first proportional threshold is less than the second proportional threshold.

4. The meat quality testing method according to any one of claims 1-3, characterized in that, Constructing the regression model for the spectral curve and the preset components includes: A regression model for the spectral curve and preset components is constructed based on at least one of the partial least squares regression algorithm, support vector machine algorithm, or decision tree algorithm. The preset components include at least one of volatile basic nitrogen, total bacterial count, myoglobin, or water.

5. The meat quality testing method according to claim 4, characterized in that, Constructing the regression model for the spectral curve and the preset components includes: Based on the second region, image features and spectral features are extracted; Based on the image features and the spectral features, the fusion features are obtained; Based on the fusion features, the regression model is constructed.

6. The meat quality testing method according to claim 5, characterized in that, Image feature extraction includes: The image features are extracted based on a multi-stage convolutional neural network.

7. The meat quality testing method according to claim 6, characterized in that, The multi-stage convolutional neural network includes a first-stage convolutional neural network, a second-stage convolutional neural network, a third-stage convolutional neural network, and a fourth-stage convolutional neural network. Extracting the image features includes: Based on the first-stage convolutional neural network, the second-stage convolutional neural network, and the third-stage convolutional neural network, the second region is convolved to extract the image features and obtain the first feature map. Based on the channel attention mechanism, the channel feature vectors of the first feature map are determined; Based on the channel feature vectors and parameter mapping functions, the channel weights of the first feature map are obtained; Based on the channel weights, the channel weights of the first feature map are adjusted to obtain the second feature map; Based on the fourth-stage convolutional neural network, the second feature map is convolved to obtain the third feature map, which includes the global image feature vector.

8. The meat quality testing method according to claim 7, characterized in that, Extracted spectral features include: The spectral curve is divided into multiple local bands; The local bands are convolutionally processed to extract local features; The local features are convolved to obtain cross-band features; Based on the cross-band features, a spectral feature map is obtained; wherein, the spectral feature map includes a global spectral feature vector.

9. The meat quality testing method according to claim 8, characterized in that, The convolution processing of the local features includes: performing convolution processing on the local features using convolution kernels of different sizes.

10. The meat quality testing method according to claim 9, characterized in that, Based on the image features and the spectral features, the fusion features obtained include: A multimodal fusion algorithm based on the Transformer architecture fuses the global image feature vector and the global spectral feature vector to obtain a fused feature vector.

11. The meat quality testing method according to claim 10, characterized in that, The fusion of the global image feature vector and the global spectral feature vector includes: Based on the global image feature vector and the global spectral feature vector, a first sequence is constructed and its position is encoded: Learnable or fixed positional information is injected into each position of the first sequence to obtain the second sequence; Based on a multi-head attention mechanism and a feedforward neural network, feature interaction is performed on the image features and spectral features of the second sequence to obtain a third sequence; and Based on the global spectral feature vector and the third sequence, attention pooling is performed to generate a fused feature vector sequence; wherein, the fused feature vector sequence includes a fused feature vector.

12. The meat quality testing method according to claim 11, characterized in that, Determining the freshness of the meat includes: The fused feature vector sequence is input into the classifier; The classifier maps the fused feature vector sequence to multiple class spaces. Obtain the scores of the fused feature vector sequence in the multiple category spaces; The freshness of the meat is determined based on the score.

13. The meat quality testing method according to claim 12, characterized in that, The multiple categories correspond to multiple freshness levels, including fresh, slightly fresh, and not fresh. Determining the freshness of the meat includes: Determine the highest score of the fused feature vector sequence in the scores of the multiple class spaces; and The freshness of the meat is determined based on the freshness corresponding to the category space of the highest score.

14. The meat quality testing method according to any one of claims 1-3, characterized in that, Also includes: Morphological processing of the meat region includes at least one of the following operations: Perform an opening operation on the first region of the meat area; Perform a closing operation on the second region of the meat region.

15. A computer-readable storage medium, characterized in that, It includes computer-executable instructions stored thereon, which, when executed by a processor, implement the meat quality detection method as described in any one of claims 1-14.

Images