Food sorting method and apparatus based on image processing

Abstract

The application relates to the field of food sorting, and provides a food sorting method and device based on image processing, which comprises the following steps: acquiring a front image of food; performing edge recognition and part segmentation on the front image to obtain a plurality of part mark images of the food, and performing pretreatment on the plurality of part mark images to obtain part images; extracting fresh contrast features based on the part images of the parts of the food; splicing the fresh contrast features of the part images in a preset order to obtain a feature vector, and inputting the feature vector into a sorting model to obtain a sorting category of the food. The fresh contrast features are extracted by parts, and a multi-scale convolutional neural network model with a fusion attention mechanism is used for classification, so that the technical problem of low sorting precision of the prior art in the freshness of food sorting is solved, automatic and accurate sorting based on multiple parts and multiple features is realized, and the accuracy and efficiency of food sorting are significantly improved.

Description

Technical Field

[0001] This application relates to the field of food sorting, and more particularly to a food sorting method and apparatus based on image processing. Background Technology

[0002] Food sorting is a crucial step in food processing and quality control. Its purpose is to classify and grade raw materials or finished products based on indicators such as appearance, internal quality, and safety to meet diverse market demands and enhance product added value and distribution efficiency. However, sorting largely relies on mechanical sorting equipment. This equipment only filters food based on its physical characteristics (size and weight) and cannot directly assess quality, resulting in limited sorting accuracy. For example, abalone, a high-end seafood, has a commodity value directly correlated with its size and freshness. Abalone sorting is a core process in the distribution chain, directly determining product grading, pricing, and market competitiveness.

[0003] Traditional mechanical sorting methods can only roughly divide food based on physical dimensions and cannot effectively identify changes in freshness or quality within the food. During storage, the edges of food often experience oxidation, browning, or moisture loss before the center, leading to a decline in freshness. Existing image recognition technologies typically extract single features from the entire food, ignoring the differences and correlations in freshness changes between different parts of the food (such as the edge and center areas). This results in low sorting accuracy and makes it difficult to meet the sorting requirements of high-quality food. Summary of the Invention

[0004] This application provides a food sorting method and apparatus based on image processing, which solves the technical problem of low sorting accuracy of food based on freshness in the prior art.

[0005] To achieve the above objectives, this application adopts the following technical solution: Firstly, an image processing-based food sorting method is provided, including: Acquire a frontal image of the food; wherein the frontal image is a projection surface that can reflect the freshness of the food; Edge recognition and part segmentation are performed on the frontal image to obtain several part identification images of the food. These part identification images are then preprocessed to obtain part images. Freshness contrast features are extracted from the part images of each part of the food. These freshness contrast features are features directly related to the freshness of the food. The freshness contrast features of images from various parts are spliced ​​together in a preset order to obtain a feature vector, which is then input into a sorting model to obtain the sorting category of the food. The sorting model is a multi-scale convolutional neural network model with an attention mechanism that has been trained.

[0006] Based on the above technical solution, the food sorting method based on image processing provided in this application acquires a frontal image of the food that reflects its freshness, performs edge recognition and part segmentation, extracts contrast features directly related to freshness from each part, and then stitches the features together and inputs them into a multi-scale convolutional neural network model with an attention mechanism for classification. This achieves automated and accurate sorting based on multiple parts and features, which can not only distinguish sizes but also accurately identify freshness, solving the problem that traditional mechanical sorting cannot judge quality, improving sorting accuracy and efficiency, and reducing the rate of human error.

[0007] In conjunction with the first aspect above, in one possible implementation, the edge recognition and part segmentation of the frontal image includes: The frontal image is converted into a single-channel grayscale image, and then smoothed by a 3×3 Gaussian filter to obtain a denoised image. By calculating the gradient magnitude and gradient direction of the denoised image, pixel-level brightness abrupt points are located, and false edge points are eliminated through non-maximum suppression to obtain edge contour points. By setting dual thresholds to connect edge contour points, a closed contour of the food is obtained. Then, the edge lines of the sub-parts of the food are extracted from the overall closed contour of the food to obtain the edge recognition result. The sub-parts of the food are the constituent parts of the food. Based on edge recognition results and prior morphological features of the food, the frontal image is segmented to obtain images with identification of several parts.

[0008] The above scheme, through steps such as Gaussian filtering smoothing, gradient calculation, non-maximum suppression, and double threshold connection, can accurately extract the overall outline and sub-part edge lines of food. Combined with morphological prior features for segmentation, it provides an accurate image basis for subsequent sub-part feature extraction.

[0009] In conjunction with the first aspect above, in one possible implementation, the segmentation of the frontal image based on edge recognition results and prior morphological features of the food includes: Extract the largest closed connected edge from the edge recognition results and mark it as the overall outer contour; The frontal image is segmented based on the overall outer contour to obtain the foreground image; The image between the edge lines of the sub-parts in the foreground image is segmented and matched based on the morphological prior features of the food to obtain segmented images of several food sub-parts; wherein, the morphological prior features are the hierarchical relationship between the food parts. Shrink the segmented image inward by a preset number of n pixels to obtain the edge region; The segmented image with edge regions is labeled as a part identification image.

[0010] It should be noted that the morphological prior features are the positional relationships and / or size hierarchy among the various sub-parts of the food, while the n preset pixels mentioned above are preset according to the parts of each food, the shooting distance, the shooting pixels, and the width and length of the pixels occupied.

[0011] In conjunction with the first aspect above, in one possible implementation, the preprocessing of several part identification images to obtain part images includes: The images of each part are uniformly cropped to the same resolution as the frontal image, and median filtering is applied to remove edge noise. Histogram equalization is then used to enhance contrast and highlight color and texture edge details; Finally, the dimensions are normalized to obtain the part image.

[0012] In conjunction with the first aspect above, in one possible implementation, the extraction of freshness features from images of different parts of the food includes: The images of each part are converted from RGB space images to HSV color space images, and the corresponding H channel single-channel images are separated. Based on the segmentation binary mask of edge and non-edge regions in a single-channel image, the mean values ​​of H, S and V channels of edge and non-edge regions are calculated respectively, and the absolute value of the difference between corresponding channels of edge and non-edge regions is calculated to obtain the hue feature value. Construct HSV 3D color vector matrices for edge and non-edge regions and Through calculation formula Calculate the distribution divergence; where, in the formula, This represents the difference in distribution divergence between the edge region and the non-edge region. To calculate the covariance matrix of the matrix; To calculate the trace of the matrix; The distribution divergence difference and hue feature value are integrated into a fresh contrast feature.

[0013] It should be noted that when food is not fresh, oxidation usually occurs first in the edge areas, leading to browning, blackening, graying, and dulling. This causes a shift in hue, with the difference between the hue and the non-edge areas that tend towards the center increasing significantly. The hue characteristic value can well reflect the characteristics of the aforementioned stale food.

[0014] In conjunction with the first aspect above, in one possible implementation, the freshness contrast feature further includes: standard offset deviation feature and the difference between the mean values ​​of the H, S, and V channels of the edge region and the non-edge region; The standard offset deviation characteristic is the Euclidean distance between the mean vectors of the H, S, and V channels of the edge region and the mean vectors of the standard H, S, and V channels of the corresponding part of the food in the fresh benchmark library.

[0015] Examples of methods for obtaining standard offset deviation characteristics include: The mean values ​​of the H, S, and V channels of the edge regions of each part of the image are statistically analyzed and then concatenated into a three-dimensional vector. ; Extract the standard HSV 3D vector of the corresponding edge from the fresh benchmark library. ; Through calculation formula The standard offset deviation characteristic is obtained by calculating the Euclidean distance. ; In conjunction with the first aspect above, in one possible implementation, the construction of the HSV three-dimensional color vector matrix of the edge region and the non-edge region... and ,include: Traverse every pixel of the normalized HSV 3D image and synchronously match the corresponding pixel positions of the edge region binary mask and the non-edge region binary mask. The effective edge pixels with a value of 255 in the binary mask of the edge region are retained, and the invalid edge pixels are removed; among them, invalid edge pixels include background pixels and non-edge region pixels; The three-dimensional color vector of the effective pixels at the edge [ , , Construct a two-dimensional matrix by arranging rows. ; In the binary mask of non-edge regions, retain the valid non-edge pixels with a pixel value of 255 and discard the invalid non-edge pixels; where invalid non-edge pixels include background pixels and edge region pixels; where, , , These are the raw pixel values ​​of a single HSV channel. , , , , , These are the original pixel values ​​for a single HSV channel; The three-dimensional color vector of the non-edge effective pixels [ , , Construct a two-dimensional matrix by arranging rows. .

[0016] In conjunction with the first aspect above, in one possible implementation, the sorting model includes an input layer, a feature recombination layer, a multi-scale convolutional module, a channel attention module, a global average pooling layer, a fully connected layer, and a Softmax classification output layer; The input layer is used to receive feature vectors; The feature reorganization layer is used to expand and rearrange the dimension of the input one-dimensional feature vector V, and then map it to a fully connected layer. Spatial dimensions, forming feature maps ,in, This is the number of basic channels; it enables one-dimensional statistical features to have spatial structure, making it easier for subsequent convolution operations to extract spatial relationships between features. The multi-scale convolution module uses 1×1, 3×3 and 5×5 convolution kernel sizes, and each branch contains a convolutional layer, a batch normalization layer and a Leaky ReLU activation function; The channel attention module includes a global average pooling branch and a global max pooling branch; The global average pooling layer is used to perform global average pooling on the weighted features output by the channel attention module, pooling the features of each channel. Features are compressed into feature vectors; The fully connected layer is used to compress the feature vector and output it to the Softmax classification output layer; The Softmax classification output layer is used to calculate... The output determines the probability value of the food item belonging to each sorting category, and the category corresponding to the highest probability value is taken as the sorting result; where, This represents the probability that the input feature vector X is classified into the c-th class, where N is the total number of classes. and These are the weight vector and bias term corresponding to the c-th class, respectively. The compressed feature vector, and The weight vector and bias term corresponding to the j-th class are respectively, where j is the index of the traversed class in the summation operation.

[0017] In conjunction with the first aspect above, in one possible implementation, the training method of the sorting model includes: An initial model was constructed based on a CNN model and a part-of-sector attention mechanism. Select several fresh foods of various sizes from the database that require freshness assessment as sample data, and label them with the actual sorting category labels. ; For each sample data, edge recognition, part segmentation, and freshness comparison feature extraction steps are performed to generate the corresponding feature vector X; Constructing a training sample set The dataset is then randomly divided into training, validation, and test sets according to a preset ratio. The feature vector X of the training set is input into the initial model, which passes through a feature reorganization layer, a multi-scale convolutional module, a channel attention module, a global average pooling layer, and a fully connected layer. Finally, the Softmax classification output layer outputs the predicted probability distribution matrix of the training set. Where B is the number of samples in the training set, N is the number of freshness categories, and for the i-th sample in the training set, the probability of it being predicted as the c-th category is: ; The loss value of the initial model output is calculated using the loss function, and the loss value is calculated relative to the parameters and gradients in the initial model using the backpropagation algorithm. The parameters are updated using the Adam optimizer to obtain the trained model, and the classification accuracy of the trained model is measured using the validation set. If the validation set accuracy does not improve for 10 consecutive rounds, training is stopped, and the training model corresponding to the parameters of the model with the highest validation set accuracy is retained.

[0018] The final saved training model is tested using the test set. If the classification accuracy reaches the preset value, the final saved training model is marked as a sorting model.

[0019] In conjunction with the first aspect above, in one possible implementation, the loss function is calculated as follows: ; in, is the total loss value; i is the sample index in the training set; N is the number of freshness levels; c is the freshness category index; Let be the indicator value of the true label of the i-th sample in class c, when the true class of the i-th sample is c. ,otherwise, ; Let be the probability value that the i-th sample is predicted by the model to be of class c, output by the Softmax layer, satisfying . and ; Pre-set hyperparameters, ; This refers to the set of trainable parameters in the sorting model. For set A single parameter variable in; For parameter variables The square of the L2 norm.

[0020] Secondly, a food sorting method and apparatus based on image processing are provided, including: a communication unit and a processing unit; The communication unit is used to acquire a frontal image of the food. The processing unit is used to perform edge recognition and part segmentation on the front image to obtain several part identification images of the food, preprocess the several part identification images to obtain part images, and extract freshness contrast features based on the part images of each part of the food. The freshness contrast features of images from various parts are stitched together in a preset order to obtain a feature vector, which is then input into the sorting model to obtain the sorting category of the food.

[0021] This application provides a food sorting method and apparatus based on image processing. By acquiring a frontal image of the food and performing edge recognition and part segmentation, the food is divided into different parts, and the freshness contrast features of the edge areas and non-edge areas are extracted. Since the edge areas of food often undergo oxidative browning first when the freshness of the food decreases, resulting in an increased difference in color and texture between them and the central area, this invention uses HSV color space features and parameters such as distribution dispersion difference to quantify this difference, and combines it with standard offset deviation features to achieve a refined characterization of food freshness.

[0022] Furthermore, by using a multi-scale convolutional neural network model that integrates attention mechanisms to classify features, it can adaptively focus on important feature channels, solving the problem that traditional methods rely solely on size sorting or have low accuracy in recognizing single overall features, and significantly improving the accuracy and efficiency of food sorting.

[0023] It should be understood that the descriptions of technical features, technical solutions, beneficial effects, or similar language in this application do not imply that all features and advantages can be achieved in any single embodiment. Rather, it is understood that the description of a feature or beneficial effect means that a specific technical feature, technical solution, or beneficial effect is included in at least one embodiment. Therefore, the descriptions of technical features, technical solutions, or beneficial effects in this specification do not necessarily refer to the same embodiment. Furthermore, the technical features, technical solutions, and beneficial effects described in this embodiment can be combined in any suitable manner. Those skilled in the art will understand that embodiments can be implemented without one or more specific technical features, technical solutions, or beneficial effects of a particular embodiment. In other embodiments, additional technical features and beneficial effects may be identified in specific embodiments that do not embody all embodiments. Attached Figure Description

[0024] Figure 1 This is an architecture diagram of the food sorting method based on image processing provided in the embodiments of this application; Figure 2 A schematic diagram illustrating the process of obtaining a part identification image provided in an embodiment of this application; Figure 3 This is a schematic diagram illustrating the process of segmenting a frontal image according to an embodiment of this application. Detailed Implementation

[0025] In the description of this application, unless otherwise stated, " / " means "or," for example, A / B can mean A or B. The "and / or" in this document is merely a description of the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A alone, A and B simultaneously, and B alone. Furthermore, "at least one" means one or more, and "multiple" means two or more. The terms "first," "second," etc., do not limit the quantity or order of execution, and "first," "second," etc., do not necessarily imply differences.

[0026] It should be noted that, in this application, the terms "exemplary" or "for example" are used to indicate that something is being described as an example, illustration, or illustration. Any embodiment or design described as "exemplary" or "for example" in this application should not be construed as being more preferred or advantageous than other embodiments or design solutions. Specifically, the use of terms such as "exemplary" or "for example" is intended to present the relevant concepts in a concrete manner.

[0027] Currently, the industry mainly relies on screening abalone based solely on size or weight. While this method can sort abalone by size, the freshness of the abalone also affects its price. During storage, the edges of abalone often experience oxidation, browning, or moisture loss before the center, which can significantly impact freshness and quality, thus affecting its grading and pricing.

[0028] Please see Figures 1-3 To address the aforementioned technical problems, this application provides an image processing-based food sorting method. The method includes: acquiring a frontal image of the food; wherein the frontal image is a projection surface that reflects the freshness of the food; performing edge recognition and part segmentation on the frontal image to obtain several part identification images of the food; preprocessing the several part identification images to obtain part images; and extracting freshness contrast features based on the part images of each part of the food; wherein the freshness contrast features are features directly related to the freshness of the food. The freshness contrast features of images from various parts are concatenated in a preset order to obtain a feature vector, which is then input into a sorting model to determine the sorting category of the food. The sorting model is a multi-scale convolutional neural network model with an attention mechanism that has been trained. Based on this, the aforementioned problems can be solved.

[0029] like Figure 1 As shown in the embodiments of this application, the food sorting method based on image processing includes: S101. Obtain a frontal image of the food.

[0030] The front image is a projection surface that reflects the freshness of the food.

[0031] Specifically, for abalone, a frontal image typically refers to a full view of the abalone with its ventral side facing upwards. This projection surface can fully display key parts of the abalone, such as the foot muscles, visceral mass, and mantle. Because during storage, the edges of abalone (such as the mantle edge) often oxidize, brown, or lose moisture before the center, a full ventral image most directly reflects these changes in freshness. In practical applications, this frontal image can be obtained by vertically capturing the image above a conveyor belt using an industrial camera, or by using a handheld device, as long as the surface features of the food are clearly visible.

[0032] For example, in an automated sorting scenario for live abalone, a conveyor belt transports the abalone at a constant speed through an image acquisition station. An industrial camera, illuminated by a specific light source, captures a real-time frontal image of each abalone. In this embodiment, a high color rendering index (CRI) LED light source with a color temperature of 5500K-6500K is preferably used to accurately reproduce the true color of the abalone flesh.

[0033] S102. Perform edge recognition and part segmentation on the front image to obtain several part identification images of the food. Preprocess the several part identification images to obtain part images. Extract freshness contrast features based on the part images of each part of the food. The freshness comparison feature is a feature that is directly related to the freshness of food.

[0034] Specifically, edge recognition aims to separate the abalone body from the background and further identify the boundaries between sub-parts such as the shell and meat, and the edges and center. Part segmentation involves dividing the image into several independent regions based on the identified edge lines, such as edge region images and center region images. Preprocessing steps may include image cropping, noise reduction, and contrast enhancement to improve the accuracy of subsequent feature extraction. Freshness contrast features refer to feature parameters that can quantify differences in the freshness of food. For abalone, freshness contrast features may specifically include color and texture differences between the edge and center regions. For example, the edge and center regions of fresh abalone have relatively uniform color with little difference; while in stale abalone, the edge regions often show browning, resulting in a significantly increased color difference between the edge and center regions. By extracting these difference features, the freshness of abalone can be effectively characterized.

[0035] S103. The freshness contrast features of the images of each part are spliced ​​together in a preset order to obtain a feature vector, and then input into the sorting model to obtain the sorting category of the food; wherein, the sorting model is a multi-scale convolutional neural network model with an attention mechanism that has been trained.

[0036] Specifically, multiple extracted freshness comparison features (such as color difference values ​​and texture difference values) are arranged in a preset order to form a one-dimensional feature vector. This feature vector comprehensively reflects the freshness information of different parts of the abalone. The sorting model receives this feature vector as input and outputs the sorting category of the abalone through complex internal nonlinear operations. The sorting category can be set according to actual needs, such as premium (freshest), first-grade (relatively fresh), and second-grade (generally fresh). The sorting model uses a multi-scale convolutional neural network with an attention mechanism, which can adaptively focus on important information in the feature vector and extract the correlation between features from different scales, thereby improving the accuracy and robustness of classification. This model is pre-trained on a large number of abalone sample images labeled with freshness levels and has the ability to accurately determine the freshness level based on the input feature vector.

[0037] Through the above steps, this embodiment achieves automated and precise sorting of abalone based on freshness. Compared to traditional mechanical sorting methods that rely solely on size or weight, this embodiment uses image processing technology to delve into subtle changes in the surface features of food, particularly the contrast between the edges and the center. This effectively identifies products that, while meeting size standards, have deteriorated in freshness, significantly improving sorting accuracy and meeting the demands of the high-quality food market.

[0038] Example 2: This embodiment, based on Embodiment 1, provides a detailed explanation of the specific process of edge recognition and part segmentation. Step S200, which involves edge recognition and part segmentation of the frontal image, specifically includes the following steps: Step S201: Convert the frontal image into a single-channel grayscale image, and smooth the grayscale image using a 3×3 Gaussian filter to obtain a denoised image.

[0039] Specifically, raw abalone images captured by industrial cameras are typically RGB three-channel color images, containing a large amount of color information, but also accompanied by environmental noise. Converting them to single-channel grayscale images can reduce computational load and highlight brightness variation information. Since the abalone surface may have subtle textures or water droplet reflections, direct edge detection can easily produce false edges. Therefore, this embodiment uses a 3×3 Gaussian filter kernel to perform convolution operations on the grayscale image. Gaussian filtering, by weighting and averaging neighboring pixels, can effectively smooth high-frequency noise in the image while preserving the overall gradient trend of the abalone edges, laying the foundation for accurate edge extraction later.

[0040] Step S202: Pixel-level brightness abrupt change points are located by calculating the gradient magnitude and gradient direction of the denoised image, and false edge points are eliminated by non-maximum suppression to obtain edge contour points.

[0041] Specifically, this embodiment uses the Sobel operator or a similar algorithm to calculate the gradient of each pixel in the image in the horizontal and vertical directions, and then determines the gradient magnitude and direction. The magnitude of the gradient reflects the degree of abrupt change in brightness at that point, i.e., the edge strength. However, edges extracted solely based on gradient magnitude are often coarse and contain many non-edge local extrema. The non-maximum suppression process compares the gradient magnitude of the current pixel with the two adjacent pixels in its gradient direction. If the current pixel is not a local maximum, it is suppressed (set to zero). This step effectively refines coarse edges to a single pixel width and eliminates a large number of false edge points caused by noise or weak textures, significantly improving the accuracy of edge localization.

[0042] Step S203: By setting dual thresholds, edge contour points are connected to obtain a closed contour of the food, and the edge lines of the sub-parts of the food are extracted from the closed contour of the food to obtain the edge recognition result; wherein, the sub-parts of the food are the constituent parts of the food.

[0043] Specifically, edge points obtained after non-maximum suppression may exhibit breakage, especially in areas with low contrast at the abalone edge. This embodiment introduces a dual-threshold mechanism, setting a high threshold and a low threshold. Points with gradient magnitudes higher than the high threshold are identified as strong edge points and considered true edges; points with gradient magnitudes lower than the low threshold are discarded; points in between are considered weak edge points. When connecting edges, strong edge points are connected first, and then weak edge points connected to the strong edge points are checked. If a connection exists, the weak edge point is retained, thereby achieving closed edge connection and obtaining the overall closed contour of the abalone. Simultaneously, by detecting gradient changes within the contour, the edge lines of sub-parts such as the boundary between the abalone shell and meat can be further extracted.

[0044] Step S204: Based on the edge recognition results and the morphological prior features of the food, the frontal image is segmented to obtain several part identification images.

[0045] As one specific implementation method, this step further includes: Extract the largest closed connected edge from the edge recognition results and mark it as the overall outer contour; segment the frontal image based on the overall outer contour to obtain the foreground image.

[0046] Specifically, the image may contain background clutter or shadows that create interfering contours, while the abalone itself typically occupies the main area of ​​the image. Therefore, by extracting the closed connected edges with the largest area or longest perimeter, the overall outer contour of the abalone can be accurately located. A mask is then generated using this outer contour to block out the background area, retaining only the area containing the abalone, thus obtaining the foreground image and effectively eliminating background interference.

[0047] The image between the edge lines of the sub-parts in the foreground image is segmented and matched based on the morphological prior features of the food to obtain segmented images of several food sub-parts.

[0048] It should be noted that morphological prior features refer to the inherent positional relationships and size hierarchy among the various sub-parts of a food. Taking abalone as an example, its morphological prior features include: the shell is usually located at the edge and is arc-shaped; the foot muscles are located in the center and have a large area; and the visceral mass is located on one side of the foot. During segmentation, the system matches and classifies the extracted sub-part edge lines based on this prior knowledge. For example, a ring-shaped area located inside the overall outer contour is identified as the mantle edge, and the central highlighted area is identified as the foot. This prior feature-based matching strategy can effectively correct segmentation errors caused by edge breaks or blurring, improving the accuracy of part identification.

[0049] Shrink the segmented image inward by a preset number of n pixels along its outer edge to obtain the edge region; mark the segmented image with the edge region as a part identification image.

[0050] Specifically, to accurately quantify the freshness changes at the abalone's edges, a precise definition of the "edge region" is needed. This embodiment uses morphological erosion to shrink the image inward by n pixels along the outer edge of the segmented image. The value of n is pre-set based on the actual physical size and image resolution. For example, if the image resolution is 0.1 mm / pixel and the physical width of the edge region is set to 0.5 mm, then n is 5; if the width is set to 1.0 mm, then n is 10. By shrinking inward, the extracted edge region is ensured to closely adhere to the outer edge of the abalone, avoiding the inclusion of too many pixels in the central region while completely covering the edge areas prone to oxidation and browning. This definition method has clear physical meaning, adapts to the sorting needs of abalone of different sizes or under different shooting conditions, and enhances the robustness of the algorithm. The final generated part identification image not only contains the segmentation information of the sub-parts but also clearly distinguishes between the edge region and the non-edge region (i.e., the central region), providing a precise spatial mask for subsequent extraction of freshness contrast features.

[0051] Example 3: This embodiment, based on Embodiment 2, provides a detailed explanation of the extraction process for freshness contrast features. Step S200, which involves extracting freshness features from images of different parts of the food, specifically includes the following steps: Step S301: Convert the part images of each part from RGB space images to HSV color space images, and separate the corresponding H channel single-channel images.

[0052] Specifically, while the RGB color space is suitable for display imaging, it is often significantly affected by brightness when describing color differences. The HSV color space, however, separates color information (hue H) from brightness information (lightness V), which better aligns with human intuitive color perception. For detecting the freshness of abalone, the browning at its edges is primarily manifested as a change in hue (e.g., from bright to dull). Therefore, converting an RGB image to an HSV image can more accurately capture this change. The conversion formula uses a standard geometric transformation algorithm to obtain component images for the three channels: H (hue), S (saturation), and V (lightness).

[0053] Step S302: Based on the binary mask of the segmentation of edge region and non-edge region in single-channel image, the mean values ​​of H channel, S channel and V channel of edge region and non-edge region are calculated respectively, and the absolute value of the difference between corresponding channels of edge region and non-edge region is calculated to obtain hue feature value.

[0054] Specifically, using the edge region mask obtained in Example 2, all pixel values ​​of the edge region and the non-edge region (i.e., the center region) can be extracted respectively. The mean values ​​of these pixels in the H, S, and V channels are calculated and denoted as follows: and , , The hue characteristic value is the absolute value of the difference between the two, for example... When abalone is fresh, the color between the edge and the center is relatively uniform, and the difference is small. When the edge of the abalone turns brown, its hue (H) value shifts significantly, causing the difference to increase. Therefore, this feature can intuitively reflect the color contrast between the edge and the center.

[0055] Step S303: Construct the HSV three-dimensional color vector matrix for both edge and non-edge regions. and Through calculation formula Calculate the distribution divergence difference. Where, in the formula, The difference in distribution divergence between the edge region and the non-edge region is represented by cov(), which calculates the covariance matrix of the matrix; tr() calculates the trace of the matrix.

[0056] A more in-depth explanation is needed regarding the covariance matrix (cov()), which describes the distribution pattern of multidimensional data (here, a three-dimensional vector of H, S, and V). The trace of the covariance matrix, the sum of its diagonal elements, is statistically equal to the sum of the variances of each dimension. Variance reflects the dispersion of the data; therefore, the size of the trace directly quantifies the dispersion of the color distribution. For fresh abalone, the color distribution in its edge areas is relatively concentrated, with a smaller variance and a smaller trace. However, for stale abalone, due to uneven distribution of edge oxidation, black spots, or browning, the color distribution becomes more dispersed, increasing the variance and the trace. By calculating the absolute value of the difference in dispersion between the edge and center areas, the difference in color distribution patterns can be quantified. This feature not only considers the mean difference in color but also captures variations in texture and mottledness, effectively complementing hue feature values.

[0057] Step S304: Integrate the distribution divergence difference and hue feature values ​​into fresh contrast features.

[0058] As a specific implementation method, an HSV three-dimensional color vector matrix is ​​constructed for both edge and non-edge regions. and The process specifically includes: Iterate through each pixel of the normalized HSV 3D image and simultaneously match the corresponding pixel positions of the edge region binary mask and the non-edge region binary mask.

[0059] The effective edge pixels with a pixel value of 255 in the binary mask of the edge region are retained, and the invalid edge pixels are removed; among them, the invalid edge pixels include background pixels and non-edge region pixels.

[0060] Construct a two-dimensional matrix by arranging the three-dimensional color vectors of the effective pixels at the edge in rows. .

[0061] Non-edge valid pixels with a pixel value of 255 are retained in the binary mask of non-edge regions, and non-edge invalid pixels are removed; among them, non-edge invalid pixels include background pixels and edge region pixels.

[0062] Construct a two-dimensional matrix by arranging the three-dimensional color vectors of the non-edge effective pixels in rows. .

[0063] Specifically, pixel filtering is necessary before constructing the matrix. Including background pixels or pixels from non-target areas in the calculation will severely interfere with the covariance matrix results, leading to distorted distribution dispersion. For example, the background is typically black or white, and its HSV value differs significantly from that of abalone meat; failing to remove these will artificially increase the variance. By using a binary mask's AND operation, pixels belonging only to the edge or center regions can be precisely extracted. After normalizing the HSV values ​​of these pixels, they are arranged in rows into a matrix, with each row representing the color vector of a single pixel. This results in a matrix... and It accurately reflects the color distribution of the target area, providing a reliable data foundation for subsequent accurate calculation of the distribution divergence difference.

[0064] Example 4: This embodiment, based on the above embodiments, provides a detailed description of the specific network architecture of the sorting model. The sorting model used in this embodiment is a multi-scale convolutional neural network model that incorporates an attention mechanism. Its specific structure includes an input layer, a feature recombination layer, a multi-scale convolutional module, a channel attention module, a global average pooling layer, a fully connected layer, and a Softmax classification output layer.

[0065] Specifically, the input layer receives the feature vector. This feature vector is the one-dimensional vector obtained by extracting and concatenating fresh contrast features in the aforementioned embodiment. As the entry point to the model, the input layer is responsible for transmitting data to subsequent network layers for processing.

[0066] The feature reorganization layer is one of the key components of the model in this embodiment. Since the input feature vector is a one-dimensional vector, although it contains rich statistical feature information, it lacks spatial structure information, making it difficult to capture the local correlations between features when directly applied to a convolutional neural network. Therefore, the feature reorganization layer is used to expand and rearrange the dimensions of the input one-dimensional feature vector V, mapping it to a spatial dimension of H×W through a fully connected layer to form a feature map. ,in, This is the base number of channels. This process gives the originally flat, one-dimensional statistical features a two-dimensional spatial structure, making it easier for subsequent convolution operations to extract the spatial relationships between features. For example, the color features of the edge region and the texture features of the center region can be arranged spatially in proximity, allowing the convolution kernel to simultaneously perceive the contrast information between the edge and the center, thereby more effectively uncovering the intrinsic connections between freshness features.

[0067] The multi-scale convolution module employs 1×1, 3×3, and 5×5 kernel sizes, with each branch containing convolutional layers, batch normalization layers, and the Leaky ReLU activation function. Different kernel sizes offer different receptive fields. The 1×1 kernel is primarily used for channel information fusion and dimensionality reduction, integrating features from different channels; the 3×3 kernel captures feature changes in local neighborhoods, suitable for extracting detailed texture features; and the 5×5 kernel has a larger receptive field, capturing feature relationships over a wider area. By using these three kernel sizes in parallel, the model can extract multi-scale feature information from the same feature map, preserving both detailed and global features, thus improving the model's adaptability to features of abalone of different sizes and shapes. Batch normalization layers accelerate model convergence and prevent gradient vanishing, while the Leaky ReLU activation function introduces non-linearity, enhancing the model's expressive power.

[0068] The channel attention module includes global average pooling and global max pooling branches. After feature extraction through multi-scale convolution, the feature maps of different channels have varying importance for the final classification. For example, some channels may be more sensitive to color difference features, while others may be more sensitive to texture features. The channel attention module compresses the two-dimensional feature map of each channel into a scalar value through global average pooling and global max pooling operations, thereby aggregating information from the entire spatial dimension. Subsequently, weight coefficients for each channel are generated through a shared fully connected layer and a sigmoid activation function. These weight coefficients are used to weight the channels of the original feature map, i.e., enhancing the response of important feature channels and suppressing the response of irrelevant or redundant feature channels. This adaptive weighting mechanism allows the model to focus on the features most critical to freshness judgment, effectively improving the model's discriminative ability and robustness.

[0069] The global average pooling layer is used to perform global average pooling on the weighted features output by the channel attention module, compressing the H×W features of each channel into a feature vector. This step flattens the multi-dimensional feature map into a one-dimensional vector, preparing it for subsequent classification in the fully connected layer. Compared to traditional flattening operations, global average pooling can reduce the number of parameters and has a certain degree of resistance to overfitting.

[0070] Fully connected layers are used to compress the feature vectors before outputting them to the Softmax classification output layer. The fully connected layers further perform linear transformations and non-linear mappings on the pooled feature vectors, mapping the high-dimensional features to a low-dimensional space equal to the number of classification categories.

[0071] The Softmax classification output layer is used to classify data according to the formula. The output determines the probability value of the food item belonging to each sorting category, and the category corresponding to the highest probability value is labeled as the sorting result. In the formula, P(y=c|X) represents the probability that the input feature vector X is classified into the c-th category, and N is the total number of sorting categories. and These are the weight vector and bias term corresponding to the c-th class, respectively. This is the compressed feature vector. The Softmax function transforms the output of the fully connected layer into a probability distribution, where the probability value of each category is between 0 and 1, and the sum of the probabilities of all categories is 1. The model ultimately selects the category with the highest probability value as the sorting result for the abalone, such as premium, first-grade, or second-grade. Through the synergistic effect of the above network layers, the sorting model in this embodiment can fully utilize the spatial correlation and channel importance of freshness contrast features to achieve high-precision automated classification of abalone freshness.

[0072] Example 5: This embodiment, based on the above embodiments, provides a detailed explanation of the training method for the sorting model. The high-precision classification capability of the sorting model is not generated out of thin air, but rather obtained through iterative optimization and training on a large number of labeled samples. The training method for this sorting model specifically includes the following steps: Step S401: Construct an initial model based on a CNN model and a part attention mechanism.

[0073] Specifically, the initial model architecture adopts the network structure described in Example 4, including an input layer, a feature reorganization layer, a multi-scale convolutional module, a channel attention module, a global average pooling layer, a fully connected layer, and a Softmax classification output layer. Before training begins, the weight parameters in the model need to be initialized, for example, using Xavier initialization or He initialization methods, to avoid gradient vanishing or exploding and ensure the stability of the training process.

[0074] Step S402: Select several fresh foods of several sizes from the database that require freshness assessment as sample data, and label them with the true sorting category label y∈{1,2,...,N}.

[0075] Specifically, the diversity of sample data directly determines the model's generalization ability. For abalone sorting models, the database should contain samples of different freshness levels, such as premium, first-grade, and second-grade, and each level should cover image data of different sizes (e.g., large, medium, small), different origins, and different lighting conditions. The labeling process is completed by experienced quality inspectors to ensure the accuracy of the label y. This construction of a training set covering multi-dimensional variations can prevent the model from overfitting to a specific size or lighting condition, enabling it to have good recognition ability for abalone of different sizes in actual production environments.

[0076] Step S403: Perform edge recognition, part segmentation, and freshness comparison feature extraction on each sample data to generate the corresponding feature vector X.

[0077] It should be noted that the processing logic of this step is completely consistent with the methods described in Examples 1 to 3. That is, for each sample image in the database, operations such as edge detection, part segmentation, HSV conversion, and distribution divergence calculation are performed to transform it into a fixed-length one-dimensional feature vector X. This step transforms unstructured image information into structured numerical features, providing standardized input data for subsequent neural network training.

[0078] Step S404: Construct a training sample set [X,y] and randomly divide it into a training set, a validation set, and a test set according to a preset ratio.

[0079] As a specific implementation method, the preset ratio is usually set to 7:2:1 or 8:1:1. The training set is used to learn the model parameters, the validation set is used to evaluate the model's generalization performance and assist in hyperparameter tuning during training, and the test set is used to finally evaluate the model's performance after training. The purpose of random partitioning is to shuffle the sample order, avoid bias caused by the sample collection order (e.g., collecting all fresh samples first and then collecting stale samples), and ensure that the sample distribution in each subset is as uniform as possible.

[0080] Step S405: Input the feature vector X of the training set into the initial model. After passing through the feature reorganization layer, multi-scale convolution module, channel attention module, global average pooling layer, and fully connected layer, the Softmax classification output layer finally outputs the prediction probability distribution matrix of the training set. Where B is the number of samples in the training set, N is the number of freshness categories, and for the i-th sample in the training set, the probability of it being predicted as the c-th category is: This process is known as forward propagation. The feature vector X undergoes nonlinear transformations through layers of the network, ultimately being mapped into an N-dimensional probability distribution vector. In the early stages of training, since the model parameters are randomly initialized, the output predicted probabilities often differ significantly from the true labels. A loss function is needed to quantify this difference and guide the direction of parameter updates.

[0081] Step S406: Calculate the loss value output by the initial model using the loss function, and use the backpropagation algorithm to calculate the loss value relative to the parameters and gradients in the initial model.

[0082] Specifically, the loss function is a measure of the difference between the model's predicted values ​​and the actual values. This embodiment uses the cross-entropy loss function with L2 regularization, and its calculation formula is: Where L is the total loss value; i is the sample index in the training set; N is the number of freshness levels; and c is the freshness category index. Let be the true label of the i-th sample in class c, where c is the true class of the i-th sample. ,otherwise, ; Let be the probability value that the i-th sample is predicted by the model to be of class c, output by the Softmax layer, satisfying . and λ is a pre-set hyperparameter. Θ represents the set of trainable parameters in the sorting model; θ represents a single parameter variable in the set Θ; ||θ||_2^2 represents the square of the L2 norm of the parameter variable θ. It's important to explain in more detail that the first part of the formula is the cross-entropy loss, used to measure the difference between the predicted probability distribution and the true label distribution, prompting the model to output more accurate classification results. The second part of the formula is the L2 regularization term, whose core function is to prevent overfitting. During training, if the model parameter θ is too large, it often means that the model has overfitted to noise or special cases in the training samples, resulting in poor performance on unseen test samples. L2 regularization forces the model parameters to tend towards smaller values ​​by penalizing the sum of squares of the parameters, thereby limiting the model's complexity and enabling it to learn smoother and more generalizable feature representations. The hyperparameter λ controls the weight of the regularization term; if λ is too small, the regularization effect is not significant, and if λ is too large, it will lead to underfitting. This embodiment preferably uses... The goal is to strike a balance between preventing overfitting and maintaining the model's fit.

[0083] Step S407: Update the parameters using the Adam optimizer to obtain the training model, and evaluate the classification accuracy of the training model using the validation set.

[0084] Specifically, the Adam optimizer combines the advantages of momentum optimization and adaptive learning rates, dynamically adjusting the learning rate for different parameters to accelerate convergence. Its parameter update formula involves the calculation of first-order and second-order moment estimates, adaptively adjusting the update step size for each parameter. It uses a larger learning rate in the early stages of training to accelerate convergence and automatically reduces the learning rate in the later stages to accurately find the optimal solution. This adaptive optimization strategy is particularly effective for handling sparse features (such as certain dimensions in fresh contrastive features). Its parameter update formula is as follows:

[0085]

[0086]

[0087]

[0088]

[0089] The current iteration step (time step) is a positive integer; For the current batch loss function, the parameters are adjusted. The gradient, i.e. ; For the exponential moving average of the gradient (first moment estimate), the initial value is... ; For the exponential moving average of the squared gradient (second moment estimate), the initial value is... ; Let be the exponential decay rate estimated by moments, and be a hyperparameter, typically taken as 0.9 and 0.999. These are deviation corrections for the first and second moments, respectively, to offset the estimation deviations in the initial stage; Here, is the learning rate, and is a hyperparameter used to control the step size for each parameter update. This is a numerical stability constant to prevent the denominator from being zero. This is the updated parameter value.

[0090] Step S408: If the accuracy of the validation set does not improve for 10 consecutive rounds, stop training and retain the training model corresponding to the model parameters with the highest accuracy on the validation set.

[0091] This is an early stopping mechanism. During training, the model's loss on the training set typically decreases continuously, but the accuracy on the validation set often shows an initial increase followed by a decrease, indicating overfitting. By monitoring the validation set accuracy, if the accuracy does not improve for 10 consecutive epochs, it indicates that the model performance has reached a bottleneck or has begun to overfit. At this point, training is forcibly stopped, and the model parameters are rolled back to the level where the validation set accuracy was highest. This mechanism effectively solves the key problem of when to stop training, saving computational resources while ensuring model performance.

[0092] Step S409: Use the test set to perform performance testing on the finally saved training model. If the classification accuracy reaches the preset value, mark the finally saved training model as a sorting model.

[0093] Finally, the model is evaluated using an independent test set. This test set data was not used during training and validation and objectively reflects the model's performance in real-world application scenarios. Only when the classification accuracy reaches a preset threshold (e.g., 95% or 98%) is the model considered a qualified sorting model and used for subsequent actual sorting tasks. This rigorous testing process ensures the reliability and practicality of the sorting model.

[0094] Example 6: This embodiment, based on the above embodiments, provides a detailed description of another configuration of the freshness contrast feature. In addition to the distribution divergence difference and hue feature value mentioned in the previous embodiments, the freshness contrast feature also includes the standard offset deviation feature and the difference between the mean H, S, and V channels of the edge region and the non-edge region.

[0095] Specifically, the standard offset deviation feature refers to the Euclidean distance between the mean vectors of the H, S, and V channels of the edge region and the mean vectors of the standard H, S, and V channels of the corresponding part of the food in the fresh benchmark library. The introduction of this feature expands the freshness assessment of the abalone under test from "self-comparison" (comparison between the edge and the center) to "standard comparison" (comparison with the ideal fresh state), providing a more objective evaluation dimension.

[0096] As a specific implementation method, the construction of a fresh benchmark library is a prerequisite for calculating the standard offset deviation characteristics. The fresh benchmark library stores characteristic parameters of various foods under standard fresh conditions. For abalone, standard freshness typically refers to the state of being freshly caught or within its optimal consumption period after standardization treatment. During the construction process, a large number of abalone samples confirmed to be at the highest freshness level were selected, and the mean HSV channels of each part (especially the edge areas) were extracted and statistically averaged to obtain the standard HSV three-dimensional vector. This vector represents the baseline color characteristics of this abalone variety when it is fresh.

[0097] During the sorting process, the system first calculates the mean values ​​of the H, S, and V channels of the edge regions of the abalone images from different parts of the abalone to be tested, and then stitches them together into a three-dimensional vector. Subsequently, through calculation formula The standard offset deviation characteristic is obtained by calculating the Euclidean distance. .

[0098] It's important to explain further that when abalone freshness declines, its edge areas often undergo oxidative browning first, causing its HSV value to deviate from the standard fresh state. For example, fresh abalone typically has a light yellow or light brown edge, while oxidation may turn it dark brown or black. This color change is represented in the HSV space as a shift in the H value (hue) and a decrease in the V value (lightness). (Euclidean distance) It can comprehensively quantify the degree of deviation in this multi-dimensional color space. The higher the value, the greater the difference between the abalone being tested and the standard fresh state, and the lower its freshness. Compared with single-channel comparison, the Euclidean distance comprehensively considers the combined changes in hue, saturation, and brightness, and has stronger robustness to complex freshness decay phenomena.

[0099] Furthermore, the differences between the mean values ​​of the H, S, and V channels in the edge and non-edge regions are also important supplementary features. Although the aforementioned embodiments calculated the hue feature value (i.e., the absolute value of the difference in the mean H channel value), this embodiment further extends this to the S and V channels. Differences in the S channel (saturation) reflect the vibrancy of the abalone's surface color; fresh abalone has a strong sheen and a high S value, while spoiled abalone often appears grayish or dark, with a lower S value. Differences in the V channel (brightness) directly reflect changes in surface brightness. Through calculation... , , These three differences can comprehensively quantify the contrast between the edge and the center from three dimensions: hue, saturation, and brightness, capturing different types of freshness decay characteristics (such as browning, dehydration, and decreased gloss). Finally, in the feature integration stage, the system integrates the above-mentioned standard offset deviation characteristics. The mean difference of the H / S / V channels, along with the distribution divergence difference and hue feature values ​​from the aforementioned embodiments, are concatenated in a preset order to construct a feature vector with higher dimensionality and richer information. This feature vector not only includes the edge-center contrast information of the abalone itself but also the deviation information from the standard fresh state, thus enabling a more comprehensive and accurate characterization of the abalone's freshness. This provides high-quality input data for subsequent sorting models, further improving the accuracy of classification.

[0100] Example 7: This embodiment provides a food sorting device based on image processing, which is used to execute the food sorting method based on image processing described in any one of Embodiments 1 to 6 above. The device mainly includes a communication unit and a processing unit in its hardware architecture.

[0101] Specifically, the communication unit is used to acquire a frontal image of the food. In practical industrial applications, the communication unit can be implemented by various hardware entities. For example, the communication unit may include a high-resolution industrial camera positioned above the sorting line, equipped with a ring light source or strip light source, to quickly acquire a full-view image of the abalone's ventral side, i.e., the frontal image, as the abalone is conveyed by the conveyor belt. Furthermore, the communication unit may also include a network interface card (NIC) or a wireless communication module (such as a Wi-Fi module or a 5G communication module). When the image acquisition equipment is separated from the main processing equipment—for example, cameras distributed throughout the production line while the processing equipment is located in the control room—the communication unit receives image data transmitted from remote cameras via a network interface. The core function of the communication unit is to establish a data link between the image acquisition front-end and the back-end processing system, ensuring that image data can be transmitted to the processing unit in real time and stably.

[0102] The processing unit is used to perform edge recognition and part segmentation on the frontal image to obtain several part identification images of the food. The part identification images are preprocessed to obtain part images. Freshness contrast features are extracted based on the part images of each part of the food. The freshness contrast features of each part image are spliced ​​in a preset order to obtain a feature vector, which is then input into the sorting model to obtain the sorting category of the food.

[0103] In terms of hardware implementation, the processing unit is the core computing engine of the entire sorting device. Because this invention involves numerous image processing algorithms (such as Gaussian filtering, gradient calculation, and morphological operations) and inference operations of deep neural network models (sorting models), it places high demands on computing performance. Therefore, the processing unit typically employs an architecture combining a high-performance embedded processor (such as a multi-core processor based on ARM architecture) and a graphics processing unit (GPU), or uses a dedicated artificial intelligence inference acceleration chip (such as an FPGA or ASIC chip). The embedded processor is responsible for overall logic control, data scheduling, and simple preprocessing operations, while the GPU or acceleration chip is specifically used to perform forward inference calculations for the sorting model, ensuring real-time output of sorting results during high-speed pipeline operation.

[0104] The processing unit is logically configured to execute computer program instructions stored in memory (such as ROM, Flash, or RAM). When executed, these instructions specifically implement the edge recognition and part segmentation algorithm described in Example 2, the freshness comparison feature extraction algorithm described in Example 3, and the sorting model inference process described in Example 4. The processing unit is connected to the communication unit via an internal bus. When the communication unit receives image data, the processing unit immediately reads the data and stores it in a memory buffer, then calls the corresponding algorithm module for processing. Finally, the sorting category signal output by the processing unit can be sent to an external actuator via the communication unit, such as controlling a pneumatic push rod or robotic arm to sort abalone of different freshness levels into corresponding collection boxes.

[0105] Through the hardware architecture design described above, the device in this embodiment integrates complex image processing algorithms and deep learning models into a dedicated hardware system, achieving a fully automated process from image acquisition and feature extraction to classification decision-making. Compared to traditional mechanical sorting equipment, this device possesses intelligent decision-making capabilities, enabling accurate identification of food freshness. Compared to general-purpose computer software solutions, this device adopts an embedded and GPU-accelerated architecture, offering advantages such as small size, low power consumption, fast response speed, and ease of integration into production lines, thus meeting the stringent real-time and stability requirements of the food processing industry.

[0106] Example 8: This embodiment provides a specific application scenario of an image processing-based food sorting method in an automated abalone sorting production line. This application scenario aims to demonstrate the deployment and operational effects of the methods and apparatus described in Embodiments 1 to 7 in a real industrial production environment.

[0107] Specifically, this application scenario is set in the sorting workshop of a seafood processing plant. The assembly line equipment mainly includes a conveyor belt, an industrial camera, a light source system, an edge computing processing unit, and a sorting execution mechanism. The conveyor belt moves at a preset uniform speed, transporting the abalone to be sorted one by one to the image acquisition area. To ensure image quality, the light source system uses a high color rendering ring LED light source to evenly illuminate the conveyor belt and eliminate shadow interference. The industrial camera is vertically mounted directly above the conveyor belt, and its field of view covers the width of the conveyor belt. When the abalone moves with the conveyor belt to a position directly below the camera, a photo is taken to capture a frontal image of the abalone.

[0108] The system automatically executes the methods described in Examples 1 to 6. Specifically, the edge computing processing unit receives a frontal image captured by an industrial camera, first performing edge recognition and segmentation to separate the abalone from the conveyor belt background and accurately locate its edge and center regions. Subsequently, the system extracts fresh contrast features, including hue feature values ​​of the edges and center, distribution divergence difference, and standard offset deviation features. These features are concatenated into a feature vector and input into a pre-trained sorting model. The sorting model, based on a multi-scale convolutional neural network with an attention fusion mechanism, performs fast inference on the feature vector and outputs the sorting category to which the abalone belongs, such as premium, first-grade, or second-grade.

[0109] The processing unit generates corresponding control signals while outputting the sorting category and sends them to the sorting actuator. The sorting actuator can be a pneumatic pusher, a robotic arm, or an airflow nozzle. In one specific implementation, when the sorting model determines the abalone is "premium grade," the system does not trigger any action, and the abalone continues moving along the conveyor belt and falls into the premium grade collection box. When it is determined to be "grade 1," the system triggers the pneumatic pusher at a specific time point to push the abalone into the grade 1 collection box. When it is determined to be "grade 2" or unqualified, the system triggers another set of pneumatic pushers or airflow nozzles to sort it into the corresponding collection box. Through this hierarchical control, automated and precise sorting of abalone is achieved.

[0110] In practical applications, the technical solution of this embodiment demonstrates significant technical effectiveness. Traditional mechanical sorting can only grade abalone based on size or weight, failing to identify slight browning at the edges or internal quality changes. This embodiment, however, by extracting freshness contrast features between edge and non-edge areas, can keenly capture the subtle characteristic of preferential oxidation and browning at the edges of abalone during storage. For example, for an abalone that meets size standards but shows slight browning at the edges, traditional methods might misclassify it as premium grade. The method of this embodiment, by calculating the hue difference and dispersion difference between the edge and center, can accurately identify the characteristic of decreased freshness and correctly sort it into first or second grade. Verified in actual production lines, this system significantly improves the sorting accuracy for abalone freshness, effectively reducing the cost of manual re-inspection and enhancing product market competitiveness and consistency. Furthermore, due to the use of a feature reconstruction layer and channel attention mechanism, the model exhibits good robustness when processing images under different lighting conditions, ensuring the stability of the sorting results.

[0111] Although this application has been described in conjunction with specific features and embodiments, it is obvious that various modifications and combinations can be made thereto without departing from the spirit and scope of this application. Accordingly, this specification and drawings are merely exemplary illustrations of this application as defined by the appended claims, and are considered to cover any and all modifications, variations, combinations, or equivalents within the scope of this application. Clearly, those skilled in the art can make various alterations and modifications to this application without departing from the spirit and scope of this application. Thus, if such modifications and modifications of this application fall within the scope of the claims of this application and their equivalents, this application is also intended to include such modifications and modifications.

Claims

1. A food sorting method based on image processing, characterized in that, include: Acquire a frontal image of the food; wherein the frontal image is a projection surface that can reflect the freshness of the food; Edge recognition and part segmentation are performed on the frontal image to obtain several part identification images of the food. These part identification images are then preprocessed to obtain part images. Freshness contrast features are extracted from the part images of each part of the food. These freshness contrast features are features directly related to the freshness of the food. The freshness contrast features of images from various parts are spliced ​​together in a preset order to obtain a feature vector, which is then input into a sorting model to obtain the sorting category of the food. The sorting model is a multi-scale convolutional neural network model with an attention mechanism that has been trained.

2. The food sorting method based on image processing according to claim 1, characterized in that, The process of edge recognition and part segmentation of the frontal image includes: The frontal image is converted into a single-channel grayscale image, and then smoothed by a 3×3 Gaussian filter to obtain a denoised image. By calculating the gradient magnitude and gradient direction of the denoised image, pixel-level brightness abrupt points are located, and false edge points are eliminated through non-maximum suppression to obtain edge contour points. By setting dual thresholds to connect edge contour points, a closed contour of the food is obtained, and the edge lines of the sub-parts of the food are extracted to obtain the edge recognition result; where the sub-parts of the food are the constituent parts of the food. Based on edge recognition results and prior morphological features of the food, the frontal image is segmented to obtain images with identification of several parts.

3. The food sorting method based on image processing according to claim 2, characterized in that, The segmentation of the frontal image based on edge recognition results and prior morphological features of the food includes: Extract the largest closed connected edge from the edge recognition results and mark it as the overall outer contour; The frontal image is segmented based on the overall outer contour to obtain the foreground image; The image between the edge lines of the sub-parts in the foreground image is segmented and matched based on the morphological prior features of the food to obtain segmented images of several food sub-parts. Shrink the segmented image inward by a preset number of n pixels to obtain the edge region; The segmented image with edge regions is labeled as a part identification image.

4. The food sorting method based on image processing according to claim 3, characterized in that, The extraction of freshness features from images of different parts of food includes: The images of each part are converted from RGB space images to HSV color space images, and the corresponding H channel single-channel images are separated. Based on the segmentation binary mask of edge and non-edge regions in a single-channel image, the mean values ​​of H, S and V channels of edge and non-edge regions are calculated respectively, and the absolute value of the difference between corresponding channels of edge and non-edge regions is calculated to obtain the hue feature value. Construct HSV 3D color vector matrices for edge and non-edge regions and Through calculation formula Calculate the distribution divergence; where, in the formula, This represents the difference in distribution divergence between the edge region and the non-edge region. To calculate the covariance matrix of the matrix; To calculate the trace of the matrix; The distribution divergence difference and hue feature value are integrated into a fresh contrast feature.

5. The food sorting method based on image processing according to claim 4, characterized in that, The construction of HSV three-dimensional color vector matrices for edge and non-edge regions. and ,include: Traverse every pixel of the normalized HSV 3D image and synchronously match the corresponding pixel positions of the edge region binary mask and the non-edge region binary mask. The effective edge pixels with a value of 255 in the binary mask of the edge region are retained, and the invalid edge pixels are removed; among them, invalid edge pixels include background pixels and non-edge region pixels; Construct a two-dimensional matrix by arranging the three-dimensional color vectors of the effective pixels at the edge in rows. ; Non-edge valid pixels with a pixel value of 255 are retained in the binary mask of non-edge regions, while non-edge invalid pixels are removed; among them, non-edge invalid pixels include background pixels and edge region pixels; Construct a two-dimensional matrix by arranging the three-dimensional color vectors of the non-edge effective pixels in rows. .

6. The food sorting method based on image processing according to claim 1, characterized in that, The sorting model includes an input layer, a feature recombination layer, a multi-scale convolutional module, a channel attention module, a global average pooling layer, a fully connected layer, and a Softmax classification output layer. The input layer is used to receive feature vectors; The feature reorganization layer is used to expand and rearrange the dimension of the input one-dimensional feature vector V, and then map it to a fully connected layer. Spatial dimensions, forming feature maps ,in, This is the number of basic channels; it enables one-dimensional statistical features to have spatial structure, making it easier for subsequent convolution operations to extract spatial relationships between features. The multi-scale convolution module uses 1×1, 3×3 and 5×5 convolution kernel sizes, and each branch contains a convolutional layer, a batch normalization layer and a Leaky ReLU activation function; The channel attention module includes a global average pooling branch and a global max pooling branch; The global average pooling layer is used to perform global average pooling on the weighted features output by the channel attention module, pooling the features of each channel. Features are compressed into feature vectors; The fully connected layer is used to compress the feature vector and output it to the Softmax classification output layer; The Softmax classification output layer is used to calculate... The output determines the probability value of the food item belonging to each sorting category, and the category corresponding to the highest probability value is taken as the sorting result; where, This represents the probability that the input feature vector X is classified into the c-th class, where N is the total number of classes. and These are the weight vector and bias term corresponding to the c-th class, respectively. This is the compressed feature vector.

7. The food sorting method based on image processing according to claim 1, characterized in that, The training method for the sorting model includes: An initial model was built based on a CNN model and an attention mechanism; Select several fresh foods of various sizes from the database that require freshness assessment as sample data, and label them with the actual sorting category labels. ; For each sample data, edge recognition, part segmentation, and freshness comparison feature extraction steps are performed to generate the corresponding feature vector X; Constructing a training sample set The dataset is then randomly divided into training, validation, and test sets according to a preset ratio. The feature vector X of the training set is input into the initial model, which passes through a feature reorganization layer, a multi-scale convolutional module, a channel attention module, a global average pooling layer, and a fully connected layer. Finally, the Softmax classification output layer outputs the predicted probability distribution matrix of the training set. Where B is the number of samples in the training set, N is the number of freshness categories, and for the i-th sample in the training set, the probability of it being predicted as the c-th category is: ; The loss value of the initial model output is calculated using the loss function, and the loss value is calculated relative to the parameters and gradients in the initial model using the backpropagation algorithm. The parameters are updated using the Adam optimizer to obtain the trained model, and the classification accuracy of the trained model is measured using the validation set. If the validation set accuracy does not improve for 10 consecutive rounds, training is stopped, and the training model corresponding to the parameters of the model with the highest validation set accuracy is retained. The final saved training model is tested using the test set. If the classification accuracy reaches the preset value, the final saved training model is marked as a sorting model.

8. The food sorting method based on image processing according to claim 7, characterized in that, The formula for calculating the loss function is: ; in, is the total loss value; i is the sample index in the training set; N is the number of freshness levels; c is the freshness category index; Let be the indicator value of the true label of the i-th sample in class c, when the true class of the i-th sample is c. ,otherwise, ; Let be the probability value that the i-th sample is predicted by the model to be of class c, output by the Softmax layer, satisfying . and ; Pre-set hyperparameters, ; This refers to the set of trainable parameters in the sorting model. For set A single parameter variable in; For parameter variables The square of the L2 norm.

9. The food sorting method based on image processing according to claim 1, characterized in that, The freshness contrast features also include: standard offset deviation features and the difference between the mean values ​​of H, S, and V channels of the edge region and the non-edge region; The standard offset deviation feature is the Euclidean distance between the mean vectors of the H, S, and V channels of the edge region and the mean vectors of the standard H, S, and V channels of the corresponding part of the food in the fresh benchmark library.

10. A food sorting device based on image processing, applied to the method as described in any one of claims 1-9, characterized in that, The device includes: a communication unit and a processing unit; The communication unit is used to acquire a frontal image of the food. The processing unit is used to perform edge recognition and part segmentation on the front image to obtain several part identification images of the food, preprocess the several part identification images to obtain part images, and extract freshness contrast features based on the part images of each part of the food. The freshness contrast features of images from various parts are stitched together in a preset order to obtain a feature vector, which is then input into the sorting model to obtain the sorting category of the food.

Images