A product freshness detection processing method, system, device and medium

Abstract

The application provides a product freshness detection processing method, comprising: obtaining a target image data set and a to-be-detected image data set; sequentially performing module replacement processing, attention mechanism embedding processing, function change processing and training processing on an initial target detection network model to generate a final target detection network model; inputting the to-be-detected image data set into the final target detection network model for detection processing to generate product freshness data, so as to complete the detection of the product freshness. Through the product freshness detection processing method, system, equipment and medium disclosed by the application, the detection accuracy and detection efficiency of the product freshness can be improved.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of computer vision processing and deep learning, in particular to a product freshness detection processing method, system, device and medium. BACKGROUND

[0002] With the improvement of the living standards of residents, the demand for improvement is growing, and the agricultural product industry is developing well. At the same time, people's requirements for the quality of agricultural products are also getting higher and higher, especially the freshness of the product. The existing method for detecting the freshness of the product is mainly through artificial detection and recognition and machine vision detection. However, whether it is artificial detection or existing machine vision detection, the detection process is easy to cause damage to the product, and the detection efficiency of artificial detection is low, and the detection accuracy of machine detection is low. SUMMARY

[0003] In view of the above-mentioned shortcomings of the prior art, the purpose of the present application is to provide a product freshness detection processing method, system, device and medium, which can improve the detection accuracy and detection efficiency of product freshness.

[0004] To achieve the above-mentioned purpose and other related purposes, the present application provides a product freshness detection processing method, comprising:

[0005] obtaining a target image data set and a to-be-detected image data set;

[0006] performing module replacement processing on an initial target detection network model to generate a first target detection network model;

[0007] performing attention mechanism embedding processing on the first target detection network model to generate a second target detection network model;

[0008] performing function change processing on the second target detection network model to generate an intermediate target detection network model;

[0009] training the intermediate target detection network model according to the target image data set to generate a final target detection network model; and

[0010] inputting the to-be-detected image data set into the final target detection network model for detection processing to generate product freshness data, so as to complete the detection of product freshness.

[0011] In an embodiment of the present application, the step of obtaining a target image data set and a to-be-detected image data set comprises:

[0012] obtaining an initial image data set and the to-be-detected image data set;

[0013] annotating the initial image dataset to generate an image annotation dataset; and

[0014] performing format conversion processing on the image annotation dataset to generate a target image dataset.

[0015] In an embodiment of the present application, the step of performing module replacement processing on the initial target detection network model to generate a first target detection network model comprises:

[0016] obtaining an interoperability protocol module; and

[0017] replacing a residual feature extraction module in a feature extraction layer in the initial target detection network model according to the interoperability protocol module to generate a first target detection network model.

[0018] In an embodiment of the present application, the step of performing attention mechanism embedding processing on the first target detection network model to generate a second target detection network model comprises:

[0019] constructing a global attention mechanism; and

[0020] embedding the global attention mechanism into a feature fusion layer of the first target detection network model to generate a second target detection network model.

[0021] In an embodiment of the present application, the global attention mechanism comprises a channel attention mechanism and a spatial attention mechanism.

[0022] In an embodiment of the present application, the step of performing function change processing on the second target detection network model to generate an intermediate target detection network model comprises:

[0023] obtaining a target detection loss function;

[0024] modifying an initial loss function in the second target detection network model according to the target detection loss function to generate an intermediate target detection network model.

[0025] In an embodiment of the present application, the step of performing training processing on the intermediate target detection network model according to the target image dataset to generate a final target detection network model comprises:

[0026] performing preprocessing on the target image dataset to generate a preprocessed image dataset;

[0027] performing stochastic gradient descent processing on the intermediate target detection network model to generate the intermediate target detection network model with optimal parameters; and

[0028] The preprocessed image dataset is input into the intermediate target detection network model with optimal input parameters and is trained and processed to generate a final target detection network model.

[0029] The application also provides a product freshness detection processing system, which comprises:

[0030] An image acquisition module is configured to acquire a target image dataset and a to-be-detected image dataset.

[0031] A replacement processing module is configured to perform module replacement processing on an initial target detection network model to generate a first target detection network model.

[0032] An embedding processing module is configured to perform attention mechanism embedding processing on the first target detection network model to generate a second target detection network model.

[0033] A function modification module is configured to perform function change processing on the second target detection network model to generate an intermediate target detection network model.

[0034] A model training module is configured to perform training processing on the intermediate target detection network model according to the target image dataset to generate a final target detection network model.

[0035] A detection judgment module is configured to input the to-be-detected image dataset into the final target detection network model to perform detection processing and generate product freshness data, so as to complete the detection of the product freshness.

[0036] The application also provides an electronic device, which comprises:

[0037] One or more processors;

[0038] A storage device configured to store one or more programs, which, when executed by the one or more processors, cause the electronic device to implement the product freshness detection processing method.

[0039] The application also provides a computer-readable storage medium, which stores a computer program, which, when executed by a processor of a computer, causes the computer to execute the product freshness detection processing method.

[0040] As described above, the application provides a product freshness detection processing method, system, device and medium, which sequentially performs module replacement processing, attention mechanism embedding processing, function change processing and training processing on an initial target detection network model to generate a final target detection network model with high detection precision, thereby improving the detection precision and efficiency of the product freshness. BRIEF DESCRIPTION OF DRAWINGS

[0041] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the drawings required to be used in the description of the embodiments will be briefly introduced. Obviously, the drawings in the following description are only some embodiments of the present application, and other drawings can also be obtained by those skilled in the art without creative labor.

[0042] Figure 1 A flowchart showing a product freshness detection processing method according to an embodiment of the present application;

[0043] Figure 2 For Figure 1 A flowchart of step S210 in the embodiment shown in an exemplary embodiment;

[0044] Figure 3 For Figure 1 A flowchart of step S220 in the embodiment shown in an exemplary embodiment;

[0045] Figure 4 For Figure 1 A flowchart of step S230 in the embodiment shown in an exemplary embodiment;

[0046] Figure 5 A structural diagram of a global attention mechanism in an exemplary embodiment of a product freshness detection processing method according to an embodiment of the present application;

[0047] Figure 6 A structural diagram of a channel attention mechanism in an exemplary embodiment of Figure 5 ;

[0048] Figure 7 A structural diagram of a spatial attention mechanism in an exemplary embodiment of Figure 5 ;

[0049] Figure 8 For Figure 1 A flowchart of step S240 in the embodiment shown in an exemplary embodiment;

[0050] Figure 9 For Figure 1 A flowchart of step S250 in the embodiment shown in an exemplary embodiment;

[0051] Figure 10 A comparison diagram of mAP@0.5 results of YOLOv5 network model and YOLOv5-CGS network model in an exemplary embodiment of a product freshness detection processing method according to an embodiment of the present application;

[0052] Figure 11 A comparison diagram of mAP@0.5:0.95 results of the YOLOv5 network model and the YOLOv5-CGS network model in an exemplary embodiment of the product freshness detection processing method of the present application is shown;

[0053] Figure 12 A comparison diagram of Precision results of the YOLOv5 network model and the YOLOv5-CGS network model in an exemplary embodiment of the product freshness detection processing method of the present application is shown;

[0054] Figure 13 A comparison diagram of Recall results of the YOLOv5 network model and the YOLOv5-CGS network model in an exemplary embodiment of the product freshness detection processing method of the present application is shown;

[0055] Figure 14 A structural diagram of a product freshness detection processing system according to an exemplary embodiment of the present application is shown;

[0056] Figure 15 A structural diagram of a computer system of an electronic device suitable for implementing the embodiments of the present application is shown.

[0057] Element number explanation:

[0058] 700, computer system; 701, central processing unit; 702, read-only memory; 703, random access memory; 704, bus; 705, input / output interface; 706, input unit; 707, output unit; 708, storage unit; 709, communication unit; 710, driver; 711, removable medium. DETAILED DESCRIPTION

[0059] The technical solutions in the embodiments of the present application will be described clearly and completely below with reference to the drawings in the embodiments of the present application. Obviously, the described embodiments are only a part of the embodiments of the present application, rather than all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative labor fall within the scope of protection of the present application.

[0060] First, it's important to note that with the development of the big data era, artificial intelligence technology has advanced rapidly, with deep learning being applied to numerous fields such as image segmentation, object detection, and speech segmentation. Agricultural products, as an indispensable industry globally, urgently need to utilize cutting-edge technology to detect product freshness. Deep learning (DL) is a new research direction within machine learning (ML), bringing it closer to its original goal—artificial intelligence (AI). Deep learning learns the inherent patterns and hierarchical representations of sample data; the information gained during this learning process greatly aids in interpreting data such as text, images, and sound. Existing methods for detecting product freshness primarily rely on manual inspection and machine vision inspection. However, both manual inspection and existing machine vision inspection methods are prone to damaging the product during the process, and manual inspection is inefficient, while existing machine vision inspection has relatively low accuracy. The product freshness detection method involves acquiring a target image dataset and a dataset of images to be detected. Then, an initial target detection network model undergoes sequential module replacement, attention mechanism embedding, and function modification to obtain an intermediate target detection network model. This intermediate model is then trained on the target image dataset to generate a final target detection network model. Finally, the dataset of images to be detected is input into the final target detection network model for detection, generating product freshness data to complete the product freshness detection. In other application scenarios, the product freshness detection method can be configured according to actual conditions, and the embodiments of this application do not impose limitations on this.

[0061] Please see Figure 1 , Figure 1 This is a flowchart illustrating a product freshness detection process in an exemplary embodiment of this application. It should be understood that this method can also be applied to other exemplary implementation environments and specifically executed by devices in other implementation environments. This embodiment does not limit the implementation environment to which this method is applicable.

[0062] like Figure 1 As shown, in an exemplary embodiment, the product freshness detection method includes at least steps S210 to S260, which are described in detail below:

[0063] Step S210: Obtain the target image dataset and the image dataset to be detected.

[0064] Step S220: Perform module replacement processing on the initial target detection network model to generate the first target detection network model.

[0065] Step S230: The first object detection network model is embedded with an attention mechanism to generate the second object detection network model.

[0066] Step S240: Modify the function of the second target detection network model to generate an intermediate target detection network model.

[0067] Step S250: Train the intermediate target detection network model based on the target image dataset to generate the final target detection network model.

[0068] Step S260: Input the image dataset to be detected into the final target detection network model for detection processing to generate product freshness data, thereby completing the detection of product freshness.

[0069] like Figure 2 As shown, in an exemplary embodiment, step S210, namely obtaining the target image dataset and the image dataset to be detected, may include steps S211 to S213, which are described in detail below:

[0070] Step S211: Obtain the initial image dataset and the image dataset to be detected.

[0071] Step S212: Perform annotation processing on the initial image dataset to generate an image annotation dataset.

[0072] Step S213: Perform format conversion processing on the image annotation dataset to generate the target image dataset.

[0073] In one exemplary embodiment, the initial image dataset can be acquired via a camera, but is not limited to this; it can also be acquired via other photographic devices. The initial image dataset can be images of fresh fruit and, corresponding to fresh fruit, rotten fruit; it is not limited to this; the initial image dataset can also be images of fresh and rotten products. Annotation processing can utilize a visual image labeling tool (LabelImage) to label the target images in the initial image dataset, but is not limited to this; other labeling tools can also be used to perform the annotation processing on the initial image dataset. The image annotation dataset obtained after annotation by the visual image labeling tool can be in XML file format. Format conversion processing of the image annotation dataset involves using a corresponding format conversion script file to convert the XML file format image annotation dataset to a format suitable for training the initial object detection network.

[0074] like Figure 3As shown, in an exemplary embodiment, step S220, which involves performing module replacement processing on the initial target detection network model to generate the first target detection network model, may include steps S221 to S222, which are described in detail below:

[0075] Step S221: Obtain the interoperability protocol module.

[0076] Step S222: Replace the residual feature extraction module in the feature extraction layer of the initial target detection network model according to the interoperability protocol module to generate the first target detection network model.

[0077] In one exemplary embodiment, the interoperability protocol module can be a pure convolutional neural network module (ConvNeXt), which can be used to improve the ability to identify product freshness. The ConvNeXt module has faster inference speed and higher accuracy compared to the Swin Transformer module. Specifically, the ConvNeXt module can be trained and optimized based on the original ResNet-50 structure. In terms of macro-design, the ConvNeXt module changes the block ratio of the four stages in the ResNet residual neural network structure from [3,4,6,3] to [3,3,9,3], and replaces the Stem stage with a 4×4 convolution with a stride of 4, adjusting the fast downsampling strategy in the initial stage of the network. Regarding depthwise separable convolution, the ConvNeXt module uses depthwise separable convolution, meaning the number of groups and channels in the ConvNeXt module are the same. The ConvNeXt module uses a small-dimensional-large-dimensional-small-dimensional inverse bottleneck structure, which can avoid information loss due to dimensional compression when transforming information between different dimensional feature spaces. Regarding convolutional kernels, the ConvNeXt module can use a large 7×7 convolutional kernel to process feature information. Furthermore, the ConvNeXt module replaces the non-linear activation function (ReLU) with a Gaussian error linear unit activation function (GELU), reducing the number of activation functions used and improving processing efficiency. Through the use of fewer activation functions, fewer normalization layers, and split downsampling layers, the ConvNeXt module achieves superior feature extraction capabilities. The initial object detection network can be a single-stage object detection network model (YOLOv5). The feature extraction layer in the initial object detection network model can be the backbone layer in the YOLOv5 network model, and the residual feature extraction module can be the C3 module in the backbone layer of the YOLOv5 network model. That is, according to the interoperability protocol module, the residual feature extraction module in the feature extraction layer of the initial object detection network model can be replaced by replacing the C3 module in the backbone layer of the YOLOv5 network model with the ConvNeXt module to generate a YOLOv5-C network model. In other words, the first object detection network model can be a YOLOv5-C network model, which is used to improve the feature extraction capability of the YOLOv5 network model.

[0078] like Figure 4 , Figure 5 , Figure 6 and Figure 7 As shown, in an exemplary embodiment, step S230, which involves embedding an attention mechanism into the first object detection network model to generate the second object detection network model, may include steps S231 to S232, which are detailed below:

[0079] Step S231: Construct a global attention mechanism.

[0080] Step S232: Embed the global attention mechanism into the feature fusion layer of the first object detection network model to generate the second object detection network model.

[0081] In an exemplary embodiment, a Global Attention Mechanism (GAM) can be constructed to reduce information reduction and amplify global dimensional interactive features. The GAM can include channel attention and spatial attention mechanisms. Specifically, the channel attention mechanism can first transform the initial image feature information F1 in the target image dataset to generate new dimensional image feature information. Then, spatial dependency enhancement processing is applied to the new dimensional image feature information to obtain enhanced image feature information. Finally, activation processing is applied to the enhanced image feature information to obtain intermediate image feature information F2. For example, an image with height H, width W, and number of channels C (i.e., an image feature dimension of C*W*H) can be used as an example. The channel attention mechanism can perform a three-dimensional arrangement transformation on the image with a feature dimension of C*W*H to generate new dimensional image feature information with a feature dimension of W*H*C. Then, a Multi-Layer Perception (MLP) is used to perform spatial dependency enhancement processing on the new dimensional image feature information to obtain enhanced image feature information. Specifically, spatial dependency enhancement of new-dimensional image features can include first encoding the new-dimensional image features using a Multilayer Perceptron (MLP), reducing the number of channels C to C / R, where R represents the channel compression ratio. Then, the MLP decodes the new-dimensional image features with the C / R channel count to obtain enhanced image features with C channels. The activation function used for activation processing can be the Sigmoid function. Spatial attention mechanisms can be used to spatially fuse intermediate image features F2 to obtain fused image features. The number of channels in the fused image features is then reduced to generate fused image features with reduced channel counts. The number of channels in the fused feature features is then smoothed to generate fused image features with the original number of channels. Finally, the original number of channels in the fused image features is activated using an activation function to obtain the target image features F3. For example, the spatial attention mechanism can spatially fuse intermediate image feature information F2 using two 7×7 convolutional layers. The reduction ratio for the number of channels in the fused image feature information can be R. Then, the number of channels in the fused feature information is smoothed to generate fused image feature information with the original number of channels. Finally, the original number of channels in the fused image feature information is activated using the Sigmoid activation function to obtain the target image feature information F3. The first object detection network model can be the YOLOv5-C network model, which replaces the C3 module in the backbone layer of the YOLOv5 network model with the ConvNeXt module. However, it is not limited to this; the first object detection network model can also be other object detection network models.The feature fusion layer in the first object detection network model can be the Neck layer of the YOLOv5-C network model. For example, taking the YOLOv5-C network model as the first object detection network model as an example, the channel attention mechanism and the spatial attention mechanism can be embedded sequentially at the end of the Neck layer of the YOLOv5-C network model. The second object detection network model can be the YOLOv5-CG network model, but it is not limited to this; the second object detection network model can also be other object detection network models.

[0082] like Figure 8 As shown, in an exemplary embodiment, step S240, which involves performing function modification processing on the second object detection network model to generate an intermediate object detection network model, may include steps S241 to S242, which are detailed below:

[0083] Step S241: Obtain the target detection loss function.

[0084] Step S242: Modify the initial loss function in the second target detection network model according to the target detection loss function to generate an intermediate target detection network model.

[0085] In an exemplary embodiment, the object detection loss function can be the SIOU loss function, used to improve the fitting efficiency between the predicted bounding box and the ground truth bounding box of the product image during the training of the object detection network model. The intermediate object detection network model can be the YOLOv5-CGS ​​network model, but is not limited to this; other object detection network models are also possible. The SIOU loss function can include an angle loss sub-function, a distance loss sub-function, a shape loss sub-function, and an intersection over union (IOU) loss sub-function. The angle loss sub-function can satisfy the following formula:

[0086]

[0087]

[0088]

[0089]

[0090] Among them, c h This can be represented as the height difference between the center points of the predicted bounding box and the ground truth bounding box. This can be represented as the coordinates of the center point of the true bounding box in the image. This can be represented as the coordinates of the center point of the image prediction bounding box. It can be represented as the maximum of the absolute values ​​of the ordinates of the center points of the ground truth bounding box and the center points of the predicted bounding box. The distance loss function can be expressed as the minimum of the absolute values ​​of the ordinates of the center points of the ground truth bounding box and the predicted bounding box. σ can be represented as the distance from the center point of the ground truth bounding box to the center point of the predicted bounding box. α can be represented as the angle between the line connecting the center points of the ground truth bounding box and the predicted bounding box and the horizontal direction. x can be represented as the sine of the angle between the line connecting the center points of the ground truth bounding box and the predicted bounding box and the horizontal direction. Λ can be represented as the angular loss value between the predicted and ground truth bounding boxes. The distance loss function can satisfy the following formula:

[0091]

[0092]

[0093]

[0094] γ=2-Λ

[0095] Among them, (c w c h ρ is the width and height of the smallest bounding rectangle between the ground truth bounding box and the predicted bounding box of the image. x ρ refers to the distance along the x-axis between the center points of the ground truth bounding box and the center point of the predicted bounding box in the image. y γ refers to the y-axis distance between the center points of the ground truth bounding box and the predicted bounding box, γ refers to the angular cost between them, and Δ is the distance loss value. When the angle α between the line connecting the center points of the ground truth and predicted bounding boxes in the angular loss approaches 0, the distance loss value Δ decreases. Conversely, when the angle α between the line connecting the center points of the ground truth and predicted bounding boxes in the angular loss approaches 45°, the distance loss value Δ increases. The shape loss sub-function satisfies the following formula:

[0096]

[0097]

[0098] Among them, w gt and h gt The width and height of the ground truth bounding box can be represented by w and h respectively, and the width and height of the predicted bounding box can be represented by θ. The shape loss can be represented by max(w, w) = 0. gtThe value of ) can be represented as the maximum of the width of the predicted bounding box and the width of the ground truth bounding box, max(h, h gt The height of the predicted bounding box and the height of the ground truth bounding box can be represented as the maximum of the two heights, w. w This can be expressed as the width loss coefficient between the predicted bounding box and the ground truth bounding box in the image, w h The height loss coefficient between the predicted and ground truth bounding boxes can be represented, and Ω can represent the shape loss between the ground truth and predicted bounding boxes. The IOU loss function satisfies the following formula:

[0099]

[0100] Where B can be represented as the area of ​​the image prediction box, B GT The area of ​​the predicted bounding box in the image can be represented as the area of ​​the ground truth bounding box, while the Intersection over Union (IoU) can represent the degree of overlap between the predicted bounding box and the ground truth bounding box. The object detection loss function can satisfy the following formula:

[0101]

[0102] Where IoU represents the degree of overlap between the predicted and ground truth bounding boxes, Δ is the distance loss between the ground truth and predicted bounding boxes, Ω represents the shape loss between the ground truth and predicted bounding boxes, and Loss... sIOU It can be represented as the loss value of the target image in the SIOU loss function.

[0103] like Figure 9 As shown, in an exemplary embodiment, step S250, which involves training the intermediate object detection network model based on the target image dataset to generate the final object detection network model, may include steps S251 to S253, which are detailed below:

[0104] Step S251: Preprocess the target image dataset to generate a preprocessed image dataset.

[0105] Step S252: Perform stochastic gradient descent processing on the intermediate target detection network model to generate the intermediate target detection network model with optimal parameters.

[0106] Step S253: Input the preprocessed image dataset into the intermediate target detection network model with optimal parameters and train it to generate the final target detection network model.

[0107] In an exemplary embodiment, before preprocessing the target image dataset, the batch size of the intermediate object detection network model for each training iteration can be set to 24 images. However, this is not limited to this; the batch size can be any other number. The training epochs of the intermediate object detection network model can be set to 200, but this is not limited to this; other epochs can also be used. The preprocessed image dataset can have a pixel size of 640×640. Stochastic gradient descent (SGD) can be used to adjust the parameters of the intermediate object detection network model to obtain the optimal intermediate object detection network model. For example, taking a batch size of 24 and a training epoch of 200 as an example, the 24 images from the preprocessed image dataset can be used as the input sample set to train the optimal intermediate object detection network model for 200 iterations, where each training iteration requires 24 training samples.

[0108] like Figure 1 , Figure 10 , Figure 11 , Figure 12 ,and Figure 13 As shown, in an exemplary embodiment, when step S260 is executed, the image dataset to be detected is input into the final target detection network model for detection processing to generate product freshness data, thereby completing the detection of product freshness. Specifically, the image dataset to be detected is the image data of the product to be detected, which can be obtained through photographic or video recording devices such as cameras or webcams.

[0109] Figure 14 This is a schematic diagram illustrating the structure of a product freshness detection processing system according to an exemplary embodiment of this application. This system can be applied to other exemplary implementation environments and specifically configured in other devices; this embodiment does not limit the implementation environment to which the system is applicable.

[0110] The product freshness detection processing system may include an image acquisition module 310, a replacement processing module 320, an embedding processing module 330, a function modification module 340, a model training module 350, and a detection and judgment module 360.

[0111] In an exemplary embodiment, the image acquisition module 310 can be used to acquire a target image dataset and a target image dataset. The initial image dataset can be acquired via a camera, but is not limited to this; it can also be acquired via other photographic devices. The initial image dataset can be images of fresh fruit and images of rotten fruit corresponding to fresh fruit, but is not limited to this; it can also be images of fresh and rotten products. Annotation processing can be performed using a visual image labeling tool (LabelImage) to label the target images in the initial image dataset, but is not limited to this; other labeling tools can also be used to perform the annotation processing on the initial image dataset. The image annotation dataset obtained after annotation by the visual image labeling tool can be in XML file format. Format conversion processing of the image annotation dataset involves using a corresponding format conversion script file to convert the XML file format image annotation dataset to a format suitable for training the initial target detection network.

[0112] In an exemplary embodiment, the replacement processing module 320 can be used to perform module replacement processing on the initial object detection network model to generate a first object detection network model. The module replacing the initial object detection network model can be an interoperability protocol module, which can be a pure convolutional neural network module (ConvNeXt), used to improve the ability to identify product freshness. The ConvNeXt module has faster inference speed and higher accuracy compared to the Swin Transformer module. The ConvNeXt module can be trained and optimized based on the original ResNet-50 structure. In terms of macro-design, the ConvNeXt module changes the block ratio of the four stages in the ResNet residual neural network structure from [3,4,6,3] to [3,3,9,3], and changes the Stem stage to a 4×4 convolution with a stride of 4, adjusting the fast downsampling strategy in the initial stage of the network. Regarding depthwise separable convolution, the ConvNeXt module uses depthwise separable convolution, meaning the number of groups and channels in the ConvNeXt module are the same. The ConvNeXt module employs a small-dimensional-large-dimensional-small-dimensional inverse bottleneck structure, avoiding information loss due to dimensionality compression during information transformation between different dimensional feature spaces. For convolutional kernels, ConvNeXt can use large 7×7 kernels to process feature information. Furthermore, ConvNeXt replaces the non-linear activation function (ReLU) with the Gaussian error linear unit activation function (GELU), reducing the use of activation functions and improving processing efficiency. Through fewer activation functions, fewer normalization layers, and split downsampling layers, ConvNeXt achieves superior feature extraction capabilities. The initial object detection network can be a single-stage object detection network model (YOLOv5). The feature extraction layer in the initial object detection network model can be the backbone layer in the YOLOv5 network model, and the residual feature extraction module can be the C3 module in the backbone layer of the YOLOv5 network model. That is, according to the interoperability protocol module, the residual feature extraction module in the feature extraction layer of the initial object detection network model can be replaced by replacing the C3 module in the backbone layer of the YOLOv5 network model with the ConvNeXt module to generate a YOLOv5-C network model, which is used to improve the feature extraction capability of the YOLOv5 network model.

[0113] In an exemplary embodiment, the embedding processing module 330 can be used to perform attention mechanism embedding processing on the first object detection network model to generate a second object detection network model. Specifically, a global attention mechanism (GAM) can be constructed to reduce information reduction and amplify global dimensional interactive features. The global attention mechanism GAM can include channel attention mechanism and spatial attention mechanism. The channel attention mechanism can be used to first transform the initial image feature information F1 in the target image dataset to generate new dimensional image feature information, then perform spatial dependency enhancement processing on the new dimensional image feature information to obtain enhanced image feature information, and finally activate the enhanced image feature information to obtain intermediate image feature information F2. For example, an image with height H, width W, and number of channels C (i.e., an image feature dimension of C*W*H) can be used as an example. The channel attention mechanism can perform three-dimensional arrangement transformation processing on the image with image feature dimension C*W*H to generate new dimensional image feature information with image feature dimension W*H*C. Then, a Multi-Layer Perceptron (MLP) is used to perform spatial dependency enhancement processing on the new dimension image feature information to obtain enhanced image feature information. Specifically, spatial dependency enhancement processing on the new dimension image feature information can include first encoding the new dimension image feature information using a MLP, that is, reducing the number of channels C in the channel dimension to C / R, where R can represent the channel compression ratio. Then, the MLP decodes the new dimension image feature information with the number of channels C / R to obtain the enhanced image feature information with the number of channels C. The activation function used for activation processing can be the Sigmoid function. Spatial attention mechanism can be used to perform spatial fusion processing on the intermediate image feature information F2 to obtain fused image feature information. Then, the number of channels of the fused image feature information is reduced to generate fused image feature information with reduced channel number. Then, the number of channels of the fused feature information is smoothed to generate fused image feature information with the original number of channels. Finally, the fused image feature information with the original number of channels is activated by using an activation function to obtain the target image feature information F3. For example, the spatial attention mechanism can use two 7×7 convolutional layers to spatially fuse the intermediate image feature information F2. The reduction ratio of the number of channels of the fused image feature information can be R. Then, the number of channels of the fused feature information is smoothed to generate the fused image feature information with the original number of channels. Finally, the fused image feature information with the original number of channels is activated by using the sigmoid activation function to obtain the target image feature information F3.The first object detection network model can be an optimized YOLOv5-C network model by replacing the C3 module in the backbone layer of the YOLOv5 network model with the ConvNeXt module. However, it is not limited to this; the first object detection network model can also be other object detection network models. The feature fusion layer in the first object detection network model can be the Neck layer in the optimized YOLOv5-C network model. For example, taking the optimized YOLOv5-C network model as an example, the channel attention mechanism and spatial attention mechanism can be sequentially embedded into the end of the Neck layer of the optimized YOLOv5-C network model to generate the second object detection network model, the YOLOv5-CG network model.

[0114] In an exemplary embodiment, the function modification module 340 can be used to modify the function of the second object detection network model to generate an intermediate object detection network model. Specifically, the modified function of the second object detection network model can be an object detection loss function, which can be the SIOU loss function, used to improve the fitting efficiency between the image prediction box and the ground truth box of the product during the training process of the object detection network model. The intermediate object detection network model can be a YOLOv5-CGS ​​network model, and the SIOU loss function can include an angle loss sub-function, a distance loss sub-function, a shape loss sub-function, and an intersection over union (IOU) loss sub-function.

[0115] In an exemplary embodiment, the model training module 350 can be used to train an intermediate object detection network model based on a target image dataset to generate a final object detection network model. Specifically, before preprocessing the target image dataset, the batch size of the intermediate object detection network model for each training iteration can be set to 24 samples, but this is not limited to it; the batch size can be other numbers. The number of training epochs for the intermediate object detection network model can be set to 200, but this is not limited to it; the number of training epochs can be other numbers. The pixel resolution of the preprocessed image dataset can be 640×640, and the stochastic gradient descent processing can be performed by adjusting the parameters of the intermediate object detection network model using the stochastic gradient descent (SGD) method to obtain an intermediate object detection network model with optimal parameters. For example, we can take a training sample size of 24 and a training epoch of 200 as an example. That is, we can use 24 images from the preprocessed image dataset as the sample set to train the intermediate object detection network model with the optimal input parameters for 200 epochs. In each training epoch, the number of training samples required is 24.

[0116] In an exemplary embodiment, the detection and judgment module 360 ​​can be used to input the image dataset to be detected into the final target detection network model for detection processing, generating product freshness data to complete the detection of product freshness. Specifically, the image dataset to be detected is the image data of the product to be detected, which can be obtained by photographing or recording devices such as cameras or webcams.

[0117] It should be noted that the product freshness detection processing system and the product freshness detection processing method provided in the above embodiments belong to the same concept. The specific operation methods of each module and unit have been described in detail in the method embodiments and will not be repeated here. In practical applications, the product freshness detection processing system provided in the above embodiments can be assigned to different functional modules as needed, that is, the internal structure of the device can be divided into different functional modules to complete all or part of the functions described above. This is not a limitation here.

[0118] Embodiments of this application also provide an electronic device, including: one or more processors; and a storage device for storing one or more programs, which, when executed by the one or more processors, cause the electronic device to implement the product freshness detection processing method provided in the above embodiments.

[0119] Figure 15 A schematic diagram of a computer system suitable for implementing the embodiments of this application is shown. It should be noted that... Figure 15 The computer system 700 of the electronic device shown is merely an example and should not impose any limitation on the functionality and scope of use of the embodiments of this application.

[0120] like Figure 15 As shown, the computer system 700 includes a Central Processing Unit (CPU) 701, which can perform various appropriate actions and processes based on programs stored in Read-Only Memory (ROM) 702 or programs loaded from storage portion 708 into Random Access Memory (RAM) 703, such as performing the methods described in the above embodiments. The RAM 703 also stores various programs and data required for system operation. The CPU 701, ROM 702, and RAM 703 are interconnected via a bus 704. An Input / Output (I / O) interface 705 is also connected to the bus 704.

[0121] The following components are connected to the I / O interface 705: an input unit 706 including a keyboard, mouse, etc.; an output unit 707 including a cathode ray tube (CRT), liquid crystal display (LCD), and speakers, etc.; a storage section 708 including a hard disk, etc.; and a communication unit 709 including a network interface card such as a LAN (Local Area Network) card and a modem, etc. The communication unit 709 performs communication processing via a network such as the Internet. A drive 710 is also connected to the I / O interface 705 as needed. A removable medium 711, such as a disk, optical disk, magneto-optical disk, semiconductor memory, etc., is installed on the drive 710 as needed so that computer programs read from it can be installed into the storage section 708 as needed.

[0122] It should be noted that the computer-readable medium shown in the embodiments of this application can be a computer-readable signal medium or a computer-readable storage medium, or any combination of the two. A computer-readable storage medium can be, for example, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination thereof. More specific examples of a computer-readable storage medium may include, but are not limited to: an electrical connection having one or more wires, a portable computer disk, a hard disk, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM), flash memory, optical fiber, portable compact disc read-only memory (CD-ROM), optical storage device, magnetic storage device, or any suitable combination thereof. In this application, a computer-readable signal medium may include a data signal propagated in baseband or as part of a carrier wave, carrying a computer-readable computer program. Such propagated data signals can take various forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination thereof. Computer-readable signal media can also be any computer-readable medium other than computer-readable storage media, which can send, propagate, or transmit a program for use by or in connection with an instruction execution system, apparatus, or device. The computer program contained on the computer-readable medium can be transmitted using any suitable medium, including but not limited to wireless, wired, etc., or any suitable combination thereof.

[0123] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of this application. Each block in a flowchart or block diagram may represent a module, segment, or portion of code, which contains one or more executable instructions for implementing a specified logical function. It should also be noted that in some alternative implementations, the functions indicated in the blocks may occur in a different order than those indicated in the drawings. For example, two consecutively indicated blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in a block diagram or flowchart, and combinations of blocks in a block diagram or flowchart, can be implemented using a dedicated hardware-based system that performs the specified function or operation, or using a combination of dedicated hardware and computer instructions.

[0124] The units described in the embodiments of this application can be implemented in software or hardware, and the described units can also be located in a processor. The names of these units do not necessarily limit the specific unit itself.

[0125] Another aspect of this application provides a computer-readable storage medium storing a computer program thereon, which, when executed by a computer's processor, causes the computer to perform the aforementioned product freshness detection processing method. This computer-readable storage medium may be included in the electronic device described in the above embodiments, or it may exist independently and not assembled into that electronic device.

[0126] In summary, the product freshness detection method, system, device, and medium provided by this invention, through sequential module replacement, attention mechanism embedding, function modification, and training of the initial target detection network model, generates a final target detection network model with high detection accuracy, thereby improving the detection accuracy and efficiency of product freshness.

[0127] In the description of this specification, the references to terms such as "this embodiment," "example," "specific example," etc., indicate that a specific feature, structure, material, or characteristic described in connection with that embodiment or example is included in at least one embodiment or example of the present invention. In this specification, the illustrative expressions of the above terms do not necessarily refer to the same embodiment or example. Furthermore, the specific features, structures, materials, or characteristics described may be combined in any suitable manner in one or more embodiments or examples.

[0128] The embodiments of the present invention disclosed above are merely illustrative of the invention. The embodiments do not exhaustively describe all details, nor do they limit the invention to the specific implementations described. Clearly, many modifications and variations can be made based on the content of this specification. This specification selects and specifically describes these embodiments to better explain the principles and practical applications of the invention, thereby enabling those skilled in the art to better understand and utilize the invention. The invention is limited only by the claims and their full scope and equivalents.

Claims

1. A processing method for detecting product freshness, characterized in that, The method includes: Obtain the target image dataset and the image dataset to be detected; Obtain the interoperability protocol module, and replace the residual feature extraction module in the feature extraction layer of the initial target detection network model according to the interoperability protocol module to generate the first target detection network model; The first object detection network model is embedded with an attention mechanism to generate a second object detection network model. Obtain the target detection loss function, and modify the initial loss function in the second target detection network model according to the target detection loss function to generate an intermediate target detection network model; The intermediate target detection network model is trained based on the target image dataset to generate the final target detection network model; and The dataset of images to be detected is input into the final target detection network model for detection processing to generate product freshness data, thereby completing the detection of product freshness.

2. The processing method for product freshness detection according to claim 1, characterized in that, The steps for obtaining the target image dataset and the image dataset to be detected include: Obtain the initial image dataset and the image dataset to be detected; The initial image dataset is annotated to generate an image annotation dataset; and The image annotation dataset is converted to a new format to generate the target image dataset.

3. The processing method for product freshness detection according to claim 1, characterized in that, The step of performing attention mechanism embedding processing on the first object detection network model to generate the second object detection network model includes: Constructing a global attention mechanism; and The global attention mechanism is embedded into the feature fusion layer of the first object detection network model to generate the second object detection network model.

4. The processing method for product freshness detection according to claim 3, characterized in that, The global attention mechanism includes channel attention mechanism and spatial attention mechanism.

5. The processing method for product freshness detection according to claim 1, characterized in that, The step of training the intermediate target detection network model based on the target image dataset to generate the final target detection network model includes: The target image dataset is preprocessed to generate a preprocessed image dataset; The intermediate target detection network model is subjected to stochastic gradient descent processing to generate the intermediate target detection network model with optimal parameters; and The preprocessed image dataset is input into the intermediate target detection network model with optimal parameters and trained to generate the final target detection network model.

6. A processing system for detecting product freshness, characterized in that, The system employs the product freshness detection method as described in any one of claims 1 to 5, wherein the system comprises: The image acquisition module is used to acquire the target image dataset and the image dataset to be detected; The replacement processing module is used to perform module replacement processing on the initial object detection network model to generate the first object detection network model; An embedding processing module is used to perform attention mechanism embedding processing on the first object detection network model to generate a second object detection network model; The function modification module is used to modify the function of the second target detection network model to generate an intermediate target detection network model. A model training module is used to train the intermediate object detection network model based on the target image dataset to generate a final object detection network model; and The detection and judgment module is used to input the image dataset to be detected into the final target detection network model for detection processing, generate product freshness data, and complete the detection of product freshness.

7. An electronic device, characterized in that, The electronic device includes: One or more processors; A storage device for storing one or more programs, which, when executed by one or more processors, cause the electronic device to implement the processing method for detecting product freshness as described in any one of claims 1 to 5.

8. A computer-readable storage medium, characterized in that, It stores a computer program, which, when executed by the computer's processor, causes the computer to perform the product freshness detection processing method according to any one of claims 1 to 5.

Images