A product information entry method and system based on image recognition

Abstract

The present invention discloses a product information entry method and system based on image recognition, which relate to the field of image processing technology. An initial image is acquired, and an area where a barcode is located in the initial image is determined to obtain a target area image. The target area image is substituted into an improved YOLOv8 model to obtain a first enhanced image. The first enhanced image is substituted into a blur removal model to obtain a second enhanced image. The second enhanced image is scanned to enter product information. By acquiring the initial image and determining the target area of the barcode, the scanning content can be accurately focused and the interference of irrelevant information can be reduced. The target area image is substituted into the improved YOLOv8 model and the blur removal model to gradually enhance the image quality, reduce image blur and noise problems caused by factors such as illumination and jitter, make the scanned image clearer, thereby greatly improving the accuracy of product information entry and improving the efficiency of product information entry.

Description

Technical Field

[0001] The present invention belongs to the technical field of image processing, and in particular relates to a product information entry method and system based on image recognition. Background Art

[0002] In the process of entering goods, the traditional conveyor belt method has significant defects. In order to complete the scanning and entry of product information, the goods need to be paused during the transmission process. This operation seriously restricts the efficiency of data entry and makes it difficult for the entire process to operate efficiently. To break through this bottleneck, the industry has begun to explore dynamic scanning technology, trying to complete the scanning work directly while the goods are moving. However, this innovative technology has encountered many external environmental interferences in actual application, among which the influence of lighting factors is the most difficult. Some auxiliary light sources configured to improve the static scanning speed will cause artifacts and interference in the scanned image due to the continuous changes in lighting angle and intensity when the goods are dynamically transmitted. This greatly reduces the accuracy of dynamic recognition and makes it difficult for dynamic scanning technology to fully utilize its high efficiency advantages.

[0003] Patent CN109784810A discloses a device communication system and an information entry method based on the system. The system uses a preset scanning device to scan the delivery note number on the delivery note, and enters the delivery note information into the preset delivery note location on the whole order entry function interface; determines whether the confirmation of shelving is received; if the confirmation of shelving is received, receives the warehouse barcode information, cargo compartment barcode information, and preset quantity box information corresponding to the delivery note, and enters the warehouse barcode information, cargo compartment barcode information, and preset quantity box information corresponding to the received delivery note into the preset shelving information location on the whole order entry function interface; saves the whole order entry function interface and uploads it to the backend server. This method saves time and effort, improves production efficiency, and enhances user experience. However, when faced with dynamic transmission of goods, this solution generates problems such as artifacts and interference in the scanned image, which makes it impossible to enter product information normally, resulting in low efficiency in product information entry. Summary of the Invention

[0004] The purpose of the present invention is to solve the problem of low efficiency in product information entry and to propose a product information entry method and system based on image recognition.

[0005] In a first aspect of the present invention, a method for entering product information based on image recognition is first proposed, the method comprising:

[0006] Acquire an initial image, determine the area where the barcode is located in the initial image to obtain a target area image;

[0007] Substituting the target area image into a first enhancement model to obtain a first enhanced image;

[0008] Substituting the first enhanced image into a blur removal model to obtain a second enhanced image;

[0009] Scanning the second enhanced image to enter product information;

[0010] The first enhanced model is an improved YOLOv8 model. The original YOLOv8 model includes a backbone network, a neck network, and a head network. The specific improvements include:

[0011] Replacing the C2f layer in the backbone network with a lightweight fusion layer;

[0012] The lightweight fusion layer includes a first branch and a second branch; the first branch includes a first convolutional layer and an MLP layer; the second branch includes a second convolutional layer;

[0013] The lightweight fusion layer obtains input features and performs channel segmentation on the input features to obtain first input features and second input features;

[0014] Substituting the first input feature into the first branch, the first input feature passes through the first convolutional layer and the MLP layer in sequence to obtain a first output feature;

[0015] Substituting the second input feature into the second branch, and passing the second input feature through the second convolutional layer to obtain a second output feature;

[0016] Feature fusion is performed on the first output feature and the second output feature to obtain the output feature of the lightweight fusion layer.

[0017] Optionally, the first enhanced model is an improved YOLOv8 model, and the model improvement further includes:

[0018] Replacing the convolutional layer in the neck network with a convolutional fusion layer; the convolutional fusion layer includes a third convolutional layer and a depth convolutional layer;

[0019] The convolution fusion layer obtains input features, and substitutes the input features into the third convolution layer to obtain the first generation fusion features;

[0020] Substituting the first generation fusion features into the depth convolution layer to obtain the second generation fusion features;

[0021] The first-generation fusion features and the second-generation fusion features are fused to obtain the output features of the convolutional fusion layer.

[0022] Optionally, substituting the first enhanced image into a blur removal model to obtain a second enhanced image includes:

[0023] Downsampling the first enhanced image to obtain a first downsampled image and a second downsampled image;

[0024] Performing a 3×3 convolution operation on the first enhanced image and inputting the result into a feature extraction module to obtain a first feature;

[0025] Downsampling the first feature to obtain a first downsampled feature, and substituting the first downsampled feature and the first downsampled image into a feature supervision module to obtain a first supervised feature;

[0026] Substituting the first supervised feature into a feature extraction module to obtain a second feature;

[0027] Downsampling the second feature to obtain a second downsampled feature, and substituting the second downsampled feature and the second downsampled image into a feature supervision module to obtain a second supervised feature;

[0028] Substituting the second supervised feature into two feature extraction modules successively and then upsampling to obtain a third feature;

[0029] Substituting the third feature into a feature extraction module to obtain a fourth feature, and fusing the fourth feature with the second feature to obtain a fifth feature;

[0030] Upsampling the fifth feature and inputting it into a feature extraction module to obtain a sixth feature, fusing the sixth feature with the first feature to obtain a seventh feature, and performing a 3×3 convolution operation on the seventh feature to obtain a first target feature;

[0031] Performing element-wise addition on the first enhanced image and the first target feature to obtain a target image.

[0032] Optionally, the feature extraction module includes:

[0033] Get the input features, perform 1×1 partial convolution on the input features and then feed them into the Gaussian error linear unit to obtain the first activation features;

[0034] The first activation feature and the input feature are fused to obtain a first fused activation feature, and the first fused activation feature is convolved with a 3×3 convolution and then fed into a Gaussian error linear unit to obtain a second activation feature;

[0035] Performing a fast Fourier transform on the input features to obtain frequency domain features, performing two 1×1 partial convolutions on the frequency domain features and then performing an inverse fast Fourier transform to obtain a third activation feature;

[0036] The second activation feature and the third activation feature are concatenated and then subjected to 1×1 convolution to obtain the output feature of the feature extraction module.

[0037] Optionally, the feature supervision module input includes a sampled image and a sampled feature; the feature supervision module includes:

[0038] Substituting the sampled image into two consecutive convolutional layers continuously to obtain a first continuous convolutional feature, and fusing the sampled image and the first continuous convolutional feature to obtain a first fused convolutional feature;

[0039] Performing a 1×1 convolution on the first fused convolutional feature and then normalizing it to obtain a first normalized feature;

[0040] Normalizing the sampled features and substituting them into the continuous convolution layer to obtain a first continuous convolutional sampling feature, and multiplying the first normalized feature and the first continuous convolutional sampling feature to obtain a first convolutional fusion feature;

[0041] Substituting the first convolution fusion feature into a continuous convolution layer to obtain a second continuous convolution feature, and fusing the first continuous convolution sampling feature and the second continuous convolution feature to obtain a second convolution fusion feature;

[0042] Perform 1×1 convolution on the second convolutional fusion feature to obtain the output feature of the feature supervision module.

[0043] In a second aspect of the present invention, a product information entry system based on image recognition is provided, comprising:

[0044] A target area image determination module is used to obtain an initial image, determine the area where the barcode is located in the initial image, and obtain a target area image;

[0045] An image enhancement module, configured to substitute the target area image into the improved first enhancement model to obtain a first enhanced image;

[0046] a blur removal module, configured to substitute the first enhanced image into a blur removal model to obtain a second enhanced image;

[0047] A product information entry module, configured to scan the second enhanced image to enter product information;

[0048] The first enhanced model is an improved YOLOv8 model. The original YOLOv8 model includes a backbone network, a neck network, and a head network. The specific improvements include:

[0049] Replacing the C2f layer in the backbone network with a lightweight fusion layer;

[0050] The lightweight fusion layer includes a first branch and a second branch; the first branch includes a first convolutional layer and an MLP layer; the second branch includes a second convolutional layer;

[0051] The lightweight fusion layer obtains input features and performs channel segmentation on the input features to obtain first input features and second input features;

[0052] Substituting the first input feature into the first branch, the first input feature passes through the first convolutional layer and the MLP layer in sequence to obtain a first output feature;

[0053] Substituting the second input feature into the second branch, and passing the second input feature through the second convolutional layer to obtain a second output feature;

[0054] Feature fusion is performed on the first output feature and the second output feature to obtain the output feature of the lightweight fusion layer.

[0055] Optionally, the first enhanced model is an improved YOLOv8 model, and the model improvement further includes:

[0056] A target area image determination module, configured to replace the convolutional layer in the neck network with a convolutional fusion layer; the convolutional fusion layer includes a third convolutional layer and a depth convolutional layer;

[0057] A target area image determination module is used for the convolution fusion layer to obtain input features and substitute the input features into the third convolution layer to obtain first-generation fusion features;

[0058] A target area image determination module, configured to substitute the first-generation fusion features into the deep convolution layer to obtain second-generation fusion features;

[0059] The target area image determination module is used to fuse the first-generation fusion features and the second-generation fusion features to obtain the output features of the convolutional fusion layer.

[0060] Optionally, the blur removal module includes:

[0061] a downsampling module, configured to downsample the first enhanced image to obtain a first downsampled image and a second downsampled image;

[0062] A first feature determination module is configured to perform a 3×3 convolution operation on the first enhanced image and then input the result into a feature extraction module to obtain a first feature;

[0063] A first supervised feature determination module is configured to downsample the first feature to obtain a first downsampled feature, and substitute the first downsampled feature and the first downsampled image into a feature supervision module to obtain a first supervised feature;

[0064] A second feature determination module, configured to substitute the first supervised feature into a feature extraction module to obtain a second feature;

[0065] A second supervised feature determination module is configured to downsample the second feature to obtain a second downsampled feature, and substitute the second downsampled feature and the second downsampled image into a feature supervision module to obtain a second supervised feature;

[0066] A third feature determination module is used to successively substitute the second supervised feature into two feature extraction modules and then perform upsampling to obtain a third feature;

[0067] a fifth feature determination module, configured to substitute the third feature into a feature extraction module to obtain a fourth feature, and fuse the fourth feature with the second feature to obtain a fifth feature;

[0068] a first target feature determination module, configured to upsample the fifth feature and input the resultant feature into a feature extraction module to obtain a sixth feature, fuse the sixth feature with the first feature to obtain a seventh feature, and perform a 3×3 convolution operation on the seventh feature to obtain a first target feature;

[0069] A target image determination module is configured to perform element-wise addition on the first enhanced image and the first target feature to obtain a target image.

[0070] Optionally, the feature extraction module includes:

[0071] A first activation feature determination module is used to obtain input features, perform 1×1 partial convolution on the input features, and then input them into a Gaussian error linear unit to obtain a first activation feature;

[0072] To the second activation feature determination module, which is used to fuse the first activation feature and the input feature to obtain a first fused activation feature, perform a 3×3 convolution on the first fused activation feature, and then input the convolution into a Gaussian error linear unit to obtain a second activation feature;

[0073] A third activation feature determination module is configured to perform a fast Fourier transform on the input feature to obtain a frequency domain feature, perform two 1×1 partial convolutions on the frequency domain feature, and then perform an inverse fast Fourier transform on the frequency domain feature to obtain a third activation feature;

[0074] An output feature determination module is used to perform 1×1 convolution on the second activation feature and the third activation feature and obtain the output feature of the feature extraction module.

[0075] Optionally, the feature supervision module input includes a sampled image and a sampled feature; the feature supervision module includes:

[0076] a first fused convolution feature determination module, configured to successively substitute the sampled image into two consecutive convolution layers to obtain a first continuous convolution feature, and fuse the sampled image and the first continuous convolution feature to obtain a first fused convolution feature;

[0077] a first normalized feature determination module, configured to perform a 1×1 convolution on the first fused convolution feature and then perform normalization processing to obtain a first normalized feature;

[0078] a first convolution fusion feature determination module, configured to perform normalization on the sampled features and substitute the normalized features into the continuous convolution layer to obtain a first continuous convolution sampling feature, and multiply the first normalized features and the first continuous convolution sampling features to obtain a first convolution fusion feature;

[0079] a second convolution fusion feature determination module, configured to substitute the first convolution fusion feature into a continuous convolution layer to obtain a second continuous convolution feature, and fuse the first continuous convolution sampling feature and the second continuous convolution feature to obtain a second convolution fusion feature;

[0080] An output feature determination module is used to perform 1×1 convolution on the second convolution fusion feature to obtain the output feature of the feature supervision module.

[0081] Beneficial effects of the present invention:

[0082] The present invention proposes a product information entry method based on image recognition, which comprises the following steps: acquiring an initial image, determining the area where a barcode is located in the initial image to obtain a target area image; substituting the target area image into a first enhancement model to obtain a first enhanced image; substituting the first enhanced image into a blur removal model to obtain a second enhanced image; scanning the second enhanced image to enter product information; by acquiring the initial image and determining the target area of the barcode, the scanning content can be accurately focused and irrelevant information interference can be reduced; substituting the target area image into an improved YOLOv8 model and a blur removal model, gradually enhancing the image quality, reducing image blur and noise problems caused by factors such as lighting and jitter, making the scanned image clearer, thereby greatly improving the accuracy of product information entry and improving the efficiency of product information entry. BRIEF DESCRIPTION OF THE DRAWINGS

[0083] The present invention will be further described below with reference to the accompanying drawings.

[0084] Figure 1 A flowchart of a method for entering product information based on image recognition provided by an embodiment of the present invention;

[0085] Figure 2 A processing flow chart of the feature extraction module provided by the present invention is provided in an embodiment of the present invention;

[0086] Figure 3 A framework diagram of a product information entry system based on image recognition provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0087] The technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, rather than all the embodiments.

[0088] Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making any creative work shall fall within the scope of protection of the present invention.

[0089] The embodiment of the present invention provides a method for entering product information based on image recognition. Figure 1 , Figure 1 A flowchart of a method for entering product information based on image recognition is provided in an embodiment of the present invention. The method comprises the following steps:

[0090] S101, obtaining an initial image, determining the area where the barcode is located in the initial image to obtain a target area image;

[0091] S102, substituting the target area image into the first enhancement model to obtain a first enhanced image;

[0092] S103, substituting the first enhanced image into a blur removal model to obtain a second enhanced image;

[0093] S104, scanning the second enhanced image to enter product information;

[0094] The first enhanced model is an improved YOLOv8 model. The original YOLOv8 model includes a backbone network, a neck network, and a head network. The specific improvements include:

[0095] Replace the C2f layer in the backbone network with a lightweight fusion layer;

[0096] The lightweight fusion layer includes a first branch and a second branch; the first branch includes a first convolutional layer and an MLP layer; the second branch includes a second convolutional layer;

[0097] The lightweight fusion layer obtains input features and performs channel segmentation on the input features to obtain the first input features and the second input features;

[0098] Substitute the first input feature into the first branch, and the first input feature passes through the first convolutional layer and the MLP layer in sequence to obtain the first output feature;

[0099] Substitute the second input feature into the second branch, and the second input feature passes through the second convolution layer to obtain the second output feature;

[0100] The first output feature and the second output feature are fused to obtain the output feature of the lightweight fusion layer.

[0101] A product information entry method based on image recognition provided by an embodiment of the present invention can accurately focus on the scanned content and reduce interference from irrelevant information by acquiring an initial image and determining the target area of the barcode; the target area image is substituted into an improved YOLOv8 model and a blur removal model to gradually enhance the image quality, reduce image blur and noise problems caused by factors such as lighting and jitter, and make the scanned image clearer, thereby greatly improving the accuracy of product information entry and improving the efficiency of product information entry.

[0102] In one implementation, the C2f layer in the backbone network is replaced with a lightweight fusion layer. This layer processes features through channel segmentation and employs different branches. Compared to the original model, the lightweight fusion layer reduces network parameters and computational complexity, making the model more lightweight. In practical applications, especially when deployed on edge devices or in resource-constrained environments, this can reduce the demand for hardware computing resources and operating costs while ensuring real-time performance and enabling fast scanning and recognition.

[0103] In one implementation, the convolutional layer and MLP layer in the first branch are combined, along with the convolutional layer in the second branch, to process input features from different perspectives. The convolutional layer extracts local features of the image, while the MLP layer learns the nonlinear relationships between features. By fusing the output features of different branches, barcode features can be extracted more comprehensively and accurately. This allows for effective barcode recognition even in complex lighting conditions and when the barcode is damaged, improving the model's robustness and adaptability.

[0104] In one implementation, the initial image is a product image with a barcode. Image processing techniques (such as edge detection and color filtering) are used to precisely locate the barcode area in the initial image (the purpose is to determine the area where the barcode is located, using existing technologies). This avoids invalid processing of the entire image, improves processing efficiency, and reduces computing resource consumption. Processing only the barcode area can reduce background interference and lay the foundation for subsequent enhancement operations.

[0105] In one implementation, all features in the scheme are fused by averaging; the convolution kernels of the first and second convolution layers are both 1×1.

[0106] In one implementation, an improved detection model is used to enhance the features of the barcode area, further eliminating motion blur and Gaussian blur from the first-stage output, restoring barcode details (stripe boundaries, character clarity), and ensuring the accuracy of scanning and recognition.

[0107] In one implementation, the original C2f layer uses cross-stage local fusion, which has a large number of parameters. The lightweight fusion layer divides the input features into two branches through channel segmentation to reduce the computational complexity of a single branch. The first branch extracts local features through convolution, and MLP implements cross-channel interaction to enhance feature expression capabilities. The second branch only uses convolution to retain basic feature extraction, ensure feature integrity, and improve inference speed while maintaining detection accuracy.

[0108] In one implementation, the input features are divided into two parts by channel, so that different branches focus on features at different semantic levels (bottom-level texture and top-level semantics); the first branch uses MLP to implement cross-channel information interaction and capture the global structure of the barcode; the second branch retains local details through convolution, and the fused features have both global context and local details, making them particularly robust to detecting blurred and deformed barcodes.

[0109] In one implementation, a complete and optimized data processing pipeline is formed, from initial image acquisition to final product information entry. Each step works closely together, with the previous steps providing high-quality data for subsequent model processing. The improved model further enhances image quality and recognition, making the entire product entry process more intelligent and automated, reducing manual intervention and improving the company's digital management and operational efficiency.

[0110] In one embodiment, the first enhancement model is an improved YOLOv8 model, and the model improvement further includes:

[0111] Replace the convolutional layer in the neck network with a convolutional fusion layer; the convolutional fusion layer includes the third convolutional layer and the depth convolutional layer;

[0112] The convolution fusion layer obtains input features and substitutes the input features into the third convolution layer to obtain the first generation of fusion features;

[0113] Substitute the first generation fusion features into the deep convolutional layer to obtain the second generation fusion features;

[0114] The first-generation fusion features and the second-generation fusion features are fused to obtain the output features of the convolutional fusion layer.

[0115] In one implementation, the original neck network (Neck) of YOLOv8 uses convolutional layers and feature pyramid structures, which are mainly used for the fusion of features at different levels (such as high-level semantic features and low-level detail features). However, traditional convolutional layers can only realize single-scale local feature extraction and lack flexibility in multi-scale feature fusion. Especially in barcode detection scenarios, the feature expression may be insufficient due to changes in barcode size. The third convolutional layer (1×1) is used to extract basic global features and capture the overall structure of the barcode. The deep convolutional layer (3×3) extracts the detailed features of the barcode. The output of the third convolutional layer (first-generation fusion features) is fused with the output of the deep convolutional layer (second-generation fusion features) to achieve the complementarity of global context and local details.

[0116] In one implementation, deep convolution itself is a lightweight operation that greatly reduces computational overhead and parameter count compared to standard convolution, improving computational efficiency while maintaining accuracy. The first-generation fusion features retain strong inter-channel semantic information, while the second-generation fusion features emphasize local spatial patterns. Fusing the two helps the network better perceive targets at different scales and improve the recognition accuracy of targets in complex scenes.

[0117] In one embodiment, substituting the first enhanced image into a blur removal model to obtain a second enhanced image includes:

[0118] Downsampling the first enhanced image to obtain a first downsampled image and a second downsampled image;

[0119] Perform a 3×3 convolution operation on the first enhanced image and then input it into the feature extraction module to obtain the first feature;

[0120] Downsampling the first feature to obtain a first downsampled feature, and substituting the first downsampled feature and the first downsampled image into a feature supervision module to obtain a first supervised feature;

[0121] Substitute the first supervised feature into the feature extraction module to obtain the second feature;

[0122] Downsampling the second feature to obtain a second downsampled feature, and substituting the second downsampled feature and the second downsampled image into a feature supervision module to obtain a second supervised feature;

[0123] Substitute the second supervised feature into two feature extraction modules continuously and then upsample to obtain the third feature;

[0124] Substituting the third feature into the feature extraction module to obtain the fourth feature, and fusing the fourth feature with the second feature to obtain the fifth feature;

[0125] The fifth feature is upsampled and fed into the feature extraction module to obtain the sixth feature, the sixth feature is fused with the first feature to obtain the seventh feature, and a 3×3 convolution operation is performed on the seventh feature to obtain the first target feature;

[0126] The first enhanced image and the first target feature are added element-wise to obtain a target image.

[0127] In one implementation, different downsampling scales (first downsampling image and second downsampling image) and convolution operations are used to extract the underlying details (stripe edges) and high-level semantics (barcode structure) of the original image. By using supervised comparison between the downsampling image and the feature map, the feature extraction module is forced to learn details that are closer to the real image. Through multi-level feature fusion (such as the first feature and the fourth feature, the sixth feature and the first feature), the complementarity of features at different scales and levels is achieved, thereby improving the clarity and integrity of the barcode image.

[0128] In one implementation, the first downsampled image (1 / 2 scale) retains the medium-scale features of the barcode to supervise the first-stage feature extraction and ensure that the feature map contains sufficient structural information (such as the distribution of barcode modules). The second downsampled image (1 / 4 scale) focuses on the global structure of the barcode to supervise deeper feature extraction and avoid the loss of details due to excessive downsampling. Through multi-scale decomposition, the model can simultaneously process the local details and global layout of the barcode.

[0129] In one implementation, the downsampled features (first downsampled features) are compared with the downsampled images of the corresponding scale, so that the feature extraction module outputs a feature map that is closer to the real image, avoiding feature homogeneity (i.e., convergence of features in different regions) caused by gradient vanishing in deep networks.

[0130] In one implementation, the first feature is a shallow feature that contains rich underlying details; the fourth feature is a deep feature that has undergone multi-layer supervision and contains high-level semantics; the first feature is fused with the fourth feature to improve the semantic consistency of the barcode features and reduce false detections; the sixth feature is fused with the first feature to enhance the sharpness of details (clarity of stripe boundaries) while retaining the global constraints of the deep features to avoid noise amplification caused by over-enhancement.

[0131] In one implementation, the first target feature is obtained through multi-layer supervision and fusion, focusing on the part of the barcode that needs to be enhanced (details in the blurred area). Through element-level addition, that is, the target image is equal to the original enhanced image plus the optimized residual feature. This can gradually improve the image quality without destroying the original structure, avoiding the oversaturation or distortion problems that may be introduced by traditional enhancement algorithms (such as histogram equalization).

[0132] In one implementation, the supervision mechanism corresponding to the second downsampled image (1 / 4 scale) can force the model to learn the global outline of the barcode. Combined with the feature fusion after upsampling, it can effectively improve the feature resolution of small-size barcodes and avoid missed detections due to the low resolution of the original image. The feature supervision module can locate the blurred area (pixel blocks with blurred stripe boundaries) by comparing the downsampled image with the feature map, and through subsequent convolution and fusion operations, it can specifically enhance the high-frequency information (edge gradient) of the area, making the stripe boundaries of the blurred barcode sharper, thereby approaching the imaging effect of the "ideal barcode".

[0133] In one embodiment, see Figure 2 , Figure 2 The processing flow chart of the feature extraction module provided by the present invention includes:

[0134] S201, obtaining input features, performing 1×1 partial convolution on the input features, and then inputting the convolutional layers into a Gaussian error linear unit to obtain a first activation feature;

[0135] S202, fusing the first activation feature and the input feature to obtain a first fused activation feature, performing a 3×3 convolution on the first fused activation feature and then inputting the convolution into a Gaussian error linear unit to obtain a second activation feature;

[0136] S203, performing a fast Fourier transform on the input feature to obtain a frequency domain feature, performing two 1×1 partial convolutions on the frequency domain feature and then performing an inverse fast Fourier transform to obtain a third activation feature;

[0137] S204: concatenate the second activation feature and the third activation feature and perform a 1×1 convolution on them to obtain an output feature of the feature extraction module.

[0138] In one implementation, a 1×1 partial convolution is performed on the input features. Compared with full convolution, only part of the channels are convolved to achieve lightweight calculation while retaining key features; the first activated feature is added to the original input feature, retaining the low-frequency information in the original feature while injecting enhanced high-frequency details.

[0139] In one implementation, 3×3 convolution and GELU (Gaussian Error Linear Unit) are used to further extract local spatial features to enhance the texture expression capability of the barcode, while the probabilistic activation characteristics of GELU are used to suppress noise interference.

[0140] In one implementation, after FFT is used to convert the image from the spatial domain to the frequency domain, the periodic stripe structure of the barcode (such as equally spaced bars) will appear as an obvious peak in the frequency domain, making it easier for the model to capture global periodic features. Through two convolutions, the first convolution enhances the low-frequency components in the frequency domain; the second convolution selectively enhances the mid-frequency components while suppressing high-frequency noise. The processed frequency domain features are converted back to the spatial domain. The third activated feature obtained contains enhanced global structural information and suppressed noise, which complements the local detail features of the spatial domain branch.

[0141] In one implementation, the second activation feature (local details) in the spatial domain and the third activation feature (global structure) in the frequency domain are spliced in the channel dimension to form a multi-dimensional feature representation. The cross-domain features are integrated through 1×1 convolution, and the optimal weights of the spatial domain and frequency domain features are automatically learned, so that the output features have both accurate edge positioning and local texture details.

[0142] In one embodiment, the feature supervision module input includes a sampled image and sampled features; the feature supervision module includes:

[0143] Substituting the sampled image into two consecutive convolutional layers continuously to obtain the first continuous convolutional feature, and fusing the sampled image and the first continuous convolutional feature to obtain the first fused convolutional feature;

[0144] The first fused convolution feature is subjected to 1×1 convolution and then normalized to obtain the first normalized feature;

[0145] The sampling features are normalized and substituted into the continuous convolution layer to obtain the first continuous convolution sampling features, and the first normalized features and the first continuous convolution sampling features are multiplied to obtain the first convolution fusion features;

[0146] Substitute the first convolution fusion feature into the continuous convolution layer to obtain the second continuous convolution feature, and fuse the first continuous convolution sampling feature and the second continuous convolution feature to obtain the second convolution fusion feature;

[0147] Perform 1×1 convolution on the second convolutional fusion feature to obtain the output feature of the feature supervision module.

[0148] In one implementation, continuous convolution and cross-layer fusion are used to extract multi-level features from bottom-level details to mid-level semantics, adapting to the dual requirements of local edges and global structures in barcode detection; the normalization operation suppresses gradient vanishing or exploding, ensuring the stability of deep network training while reducing feature offsets between different samples.

[0149] In one implementation, the sampled image passes through two consecutive convolutional layers (3×3 convolution) to extract the underlying visual features, capture the basic structure of the barcode, and fuse the sampled image with the first consecutive convolutional features to retain the low-frequency information in the original input while superimposing high-frequency details to avoid information loss caused by pure convolution operations.

[0150] In one implementation, the first normalized feature (the fused middle-level feature) is multiplied by the first continuous convolution sampling feature (the bottom-level detail feature), which is equivalent to weighting the attention of the bottom-level feature through the middle-level feature. If the middle-level feature detects the barcode outline (high response), the corresponding area in the bottom-level feature (the edge of the stripe) is amplified. If it is a background area (low response, not a barcode feature area), the noise response in the bottom-level feature is suppressed to achieve adaptive feature calibration.

[0151] In one implementation, the first convolution fusion feature is further extracted through continuous convolution layers to extract high-level semantic features (module arrangement rules of the barcode), enhance the discriminability of the features, and fuse the first continuous convolution sampling features with the second continuous convolution features. Through cross-scale feature complementarity, the problem of detail loss caused by downsampling of the barcode is solved. Finally, the number of channels is adjusted through 1×1 convolution to match the output features with the input dimensions of the subsequent modules, ensuring the smoothness of the entire network.

[0152] Based on the same inventive concept, the present invention also provides a product information entry system based on image recognition. Figure 3 , Figure 3 A framework diagram of a product information entry system based on image recognition provided by an embodiment of the present invention includes:

[0153] The target area image determination module is used to obtain an initial image, determine the area where the barcode is located in the initial image, and obtain a target area image;

[0154] An image enhancement module, configured to substitute the target area image into a first enhancement model to obtain a first enhanced image;

[0155] a blur removal module, configured to substitute the first enhanced image into a blur removal model to obtain a second enhanced image;

[0156] A product information entry module, used to scan the second enhanced image to enter product information;

[0157] The first enhanced model is an improved YOLOv8 model. The original YOLOv8 model includes a backbone network, a neck network, and a head network. The specific improvements include:

[0158] Replace the C2f layer in the backbone network with a lightweight fusion layer;

[0159] The lightweight fusion layer includes a first branch and a second branch; the first branch includes a first convolutional layer and an MLP layer; the second branch includes a second convolutional layer;

[0160] The lightweight fusion layer obtains input features and performs channel segmentation on the input features to obtain the first input features and the second input features;

[0161] Substitute the first input feature into the first branch, and the first input feature passes through the first convolutional layer and the MLP layer in sequence to obtain the first output feature;

[0162] Substitute the second input feature into the second branch, and the second input feature passes through the second convolution layer to obtain the second output feature;

[0163] The first output feature and the second output feature are fused to obtain the output feature of the lightweight fusion layer.

[0164] Based on the framework diagram of a product information entry system based on image recognition provided by an embodiment of the present invention, by acquiring an initial image and determining the target area of the barcode, it can accurately focus on the scanning content and reduce interference from irrelevant information; the target area image is substituted into the improved YOLOv8 model and blur removal model to gradually enhance the image quality, reduce image blur and noise problems caused by factors such as lighting and jitter, and make the scanned image clearer, thereby greatly improving the accuracy of product information entry and improving the efficiency of product information entry.

[0165] In one embodiment, the first enhancement model is an improved YOLOv8 model, and the model improvement further includes:

[0166] The target area image determination module is used to replace the convolution layer in the neck network with a convolution fusion layer; the convolution fusion layer includes the third convolution layer and the depth convolution layer;

[0167] The target area image determination module is used to obtain input features in the convolution fusion layer and substitute the input features into the third convolution layer to obtain the first generation of fusion features;

[0168] The target area image determination module is used to substitute the first-generation fusion features into the deep convolution layer to obtain the second-generation fusion features;

[0169] The target area image determination module is used to fuse the first-generation fusion features and the second-generation fusion features to obtain the output features of the convolutional fusion layer.

[0170] In one embodiment, the blur removal module includes:

[0171] A downsampling module, configured to downsample the first enhanced image to obtain a first downsampled image and a second downsampled image;

[0172] A first feature determination module is configured to perform a 3×3 convolution operation on the first enhanced image and then input the result into a feature extraction module to obtain a first feature;

[0173] A first supervised feature determination module is configured to downsample the first feature to obtain a first downsampled feature, and substitute the first downsampled feature and the first downsampled image into a feature supervision module to obtain a first supervised feature;

[0174] A second feature determination module is used to substitute the first supervised feature into the feature extraction module to obtain a second feature;

[0175] A second supervised feature determination module is configured to downsample the second feature to obtain a second downsampled feature, and substitute the second downsampled feature and the second downsampled image into the feature supervision module to obtain a second supervised feature;

[0176] A third feature determination module is used to successively substitute the second supervised feature into two feature extraction modules and then perform upsampling to obtain a third feature;

[0177] a fifth feature determination module, configured to substitute the third feature into the feature extraction module to obtain a fourth feature, and fuse the fourth feature and the second feature to obtain a fifth feature;

[0178] A first target feature determination module is configured to upsample the fifth feature and input the resultant feature into a feature extraction module to obtain a sixth feature, fuse the sixth feature with the first feature to obtain a seventh feature, and perform a 3×3 convolution operation on the seventh feature to obtain a first target feature;

[0179] The target image determination module is used to perform element-wise addition on the first enhanced image and the first target feature to obtain the target image.

[0180] In one embodiment, the feature extraction module includes:

[0181] A first activation feature determination module is used to obtain input features, perform 1×1 partial convolution on the input features, and then input them into a Gaussian error linear unit to obtain a first activation feature;

[0182] To the second activation feature determination module, which is used to fuse the first activation feature and the input feature to obtain a first fused activation feature, perform a 3×3 convolution on the first fused activation feature, and then input the convolution into the Gaussian error linear unit to obtain a second activation feature;

[0183] A third activation feature determination module is used to perform a fast Fourier transform on the input feature to obtain a frequency domain feature, perform two 1×1 partial convolutions on the frequency domain feature, and then perform an inverse fast Fourier transform to obtain a third activation feature;

[0184] The output feature determination module is used to perform 1×1 convolution on the second activation feature and the third activation feature to obtain the output feature of the feature extraction module.

[0185] In one embodiment, the feature supervision module input includes a sampled image and sampled features; the feature supervision module includes:

[0186] a first fused convolution feature determination module, configured to successively substitute the sampled image into two consecutive convolution layers to obtain a first continuous convolution feature, and fuse the sampled image and the first continuous convolution feature to obtain a first fused convolution feature;

[0187] A first normalized feature determination module, configured to perform 1×1 convolution on the first fused convolution feature and then perform normalization processing to obtain a first normalized feature;

[0188] A first convolution fusion feature determination module is used to perform normalization on the sampled features and substitute the normalized features into the continuous convolution layer to obtain a first continuous convolution sampling feature, and to multiply the first normalized features and the first continuous convolution sampling features to obtain a first convolution fusion feature;

[0189] A second convolution fusion feature determination module is used to substitute the first convolution fusion feature into the continuous convolution layer to obtain a second continuous convolution feature, and fuse the first continuous convolution sampling feature and the second continuous convolution feature to obtain a second convolution fusion feature;

[0190] The output feature determination module is used to perform 1×1 convolution on the second convolution fusion feature to obtain the output feature of the feature supervision module.

[0191] The above is a detailed description of an embodiment of the present invention, but the content is only a preferred embodiment of the present invention and should not be considered to limit the scope of the present invention. All equivalent changes and improvements made within the scope of the present invention should still fall within the scope of the patent coverage of the present invention.

Claims

1. A product information entry method based on image recognition, characterized in that: The method comprises: Acquire an initial image, determine the area where the barcode is located in the initial image to obtain a target area image; Substituting the target area image into the YOLOv8 model improved by the first enhancement model to obtain a first enhanced image; Substituting the first enhanced image into a blur removal model to obtain a second enhanced image; Scanning the second enhanced image to enter product information; The first enhanced model is an improved YOLOv8 model. The original YOLOv8 model includes a backbone network, a neck network, and a head network. The specific improvements include: Replacing the C2f layer in the backbone network with a lightweight fusion layer; The lightweight fusion layer includes a first branch and a second branch; the first branch includes a first convolutional layer and an MLP layer; the second branch includes a second convolutional layer; The lightweight fusion layer obtains input features and performs channel segmentation on the input features to obtain first input features and second input features; Substituting the first input feature into the first branch, the first input feature passes through the first convolutional layer and the MLP layer in sequence to obtain a first output feature; Substituting the second input feature into the second branch, and passing the second input feature through the second convolutional layer to obtain a second output feature; Feature fusion is performed on the first output feature and the second output feature to obtain the output feature of the lightweight fusion layer.

2. The method for entering product information based on image recognition according to claim 1, characterized in that: The first enhanced model is an improved YOLOv8 model, and the model improvement also includes: Replacing the convolutional layer in the neck network with a convolutional fusion layer; the convolutional fusion layer includes a third convolutional layer and a depth convolutional layer; The convolution fusion layer obtains input features, and substitutes the input features into the third convolution layer to obtain the first generation fusion features; Substituting the first generation fusion features into the depth convolution layer to obtain the second generation fusion features; The first-generation fusion features and the second-generation fusion features are fused to obtain the output features of the convolutional fusion layer.

3. The method for entering product information based on image recognition according to claim 1, characterized in that: Substituting the first enhanced image into a blur removal model to obtain a second enhanced image includes: Downsampling the first enhanced image to obtain a first downsampled image and a second downsampled image; Performing a 3×3 convolution operation on the first enhanced image and inputting the result into a feature extraction module to obtain a first feature; Downsampling the first feature to obtain a first downsampled feature, and substituting the first downsampled feature and the first downsampled image into a feature supervision module to obtain a first supervised feature; Substituting the first supervised feature into a feature extraction module to obtain a second feature; Downsampling the second feature to obtain a second downsampled feature, and substituting the second downsampled feature and the second downsampled image into a feature supervision module to obtain a second supervised feature; Substituting the second supervised feature into two feature extraction modules successively and then upsampling to obtain a third feature; Substituting the third feature into a feature extraction module to obtain a fourth feature, and fusing the fourth feature with the second feature to obtain a fifth feature; Upsampling the fifth feature and inputting it into a feature extraction module to obtain a sixth feature, fusing the sixth feature with the first feature to obtain a seventh feature, and performing a 3×3 convolution operation on the seventh feature to obtain a first target feature; Performing element-wise addition on the first enhanced image and the first target feature to obtain a target image.

4. The method for entering product information based on image recognition according to claim 3, characterized in that: The feature extraction module includes: Get the input features, perform 1×1 partial convolution on the input features and then feed them into the Gaussian error linear unit to obtain the first activation features; The first activation feature and the input feature are fused to obtain a first fused activation feature, and the first fused activation feature is convolved with a 3×3 convolution and then fed into a Gaussian error linear unit to obtain a second activation feature; Performing a fast Fourier transform on the input features to obtain frequency domain features, performing two 1×1 partial convolutions on the frequency domain features and then performing an inverse fast Fourier transform to obtain a third activation feature; The second activation feature and the third activation feature are concatenated and then subjected to 1×1 convolution to obtain the output feature of the feature extraction module.

5. The method for entering product information based on image recognition according to claim 3, characterized in that: The input of the feature supervision module includes a sampled image and a sampled feature; The feature supervision module includes: Substituting the sampled image into two consecutive convolutional layers continuously to obtain a first continuous convolutional feature, and fusing the sampled image and the first continuous convolutional feature to obtain a first fused convolutional feature; Performing a 1×1 convolution on the first fused convolutional feature and then normalizing it to obtain a first normalized feature; Normalizing the sampled features and substituting them into the continuous convolution layer to obtain a first continuous convolutional sampling feature, and multiplying the first normalized feature and the first continuous convolutional sampling feature to obtain a first convolutional fusion feature; Substituting the first convolution fusion feature into a continuous convolution layer to obtain a second continuous convolution feature, and fusing the first continuous convolution sampling feature and the second continuous convolution feature to obtain a second convolution fusion feature; Perform 1×1 convolution on the second convolutional fusion feature to obtain the output feature of the feature supervision module.

6. A product information entry system based on image recognition, characterized in that: The system comprises: A target area image determination module is used to obtain an initial image, determine the area where the barcode is located in the initial image, and obtain a target area image; An image enhancement module, configured to substitute the target area image into the YOLOv8 model improved by the first enhancement model to obtain a first enhanced image; a blur removal module, configured to substitute the first enhanced image into a blur removal model to obtain a second enhanced image; A product information entry module, configured to scan the second enhanced image to enter product information; The first enhanced model is an improved YOLOv8 model. The original YOLOv8 model includes a backbone network, a neck network, and a head network. The specific improvements include: Replacing the C2f layer in the backbone network with a lightweight fusion layer; The lightweight fusion layer includes a first branch and a second branch; the first branch includes a first convolutional layer and an MLP layer; the second branch includes a second convolutional layer; The lightweight fusion layer obtains input features and performs channel segmentation on the input features to obtain first input features and second input features; Substituting the first input feature into the first branch, the first input feature passes through the first convolutional layer and the MLP layer in sequence to obtain a first output feature; Substituting the second input feature into the second branch, and passing the second input feature through the second convolutional layer to obtain a second output feature; Feature fusion is performed on the first output feature and the second output feature to obtain the output feature of the lightweight fusion layer.

7. The product information entry system based on image recognition according to claim 6, characterized in that: The first enhanced model is an improved YOLOv8 model, and the model improvement also includes: A target area image determination module, configured to replace the convolutional layer in the neck network with a convolutional fusion layer; the convolutional fusion layer includes a third convolutional layer and a depth convolutional layer; A target area image determination module is used for the convolution fusion layer to obtain input features and substitute the input features into the third convolution layer to obtain first-generation fusion features; A target area image determination module, configured to substitute the first-generation fusion features into the deep convolution layer to obtain second-generation fusion features; The target area image determination module is used to fuse the first-generation fusion features and the second-generation fusion features to obtain the output features of the convolutional fusion layer.

8. The product information entry system based on image recognition according to claim 6, characterized in that: The blur removal module includes: a downsampling module, configured to downsample the first enhanced image to obtain a first downsampled image and a second downsampled image; A first feature determination module is configured to perform a 3×3 convolution operation on the first enhanced image and then input the result into a feature extraction module to obtain a first feature; A first supervised feature determination module is configured to downsample the first feature to obtain a first downsampled feature, and substitute the first downsampled feature and the first downsampled image into a feature supervision module to obtain a first supervised feature; A second feature determination module, configured to substitute the first supervised feature into a feature extraction module to obtain a second feature; A second supervised feature determination module is configured to downsample the second feature to obtain a second downsampled feature, and substitute the second downsampled feature and the second downsampled image into a feature supervision module to obtain a second supervised feature; A third feature determination module is used to successively substitute the second supervised feature into two feature extraction modules and then perform upsampling to obtain a third feature; a fifth feature determination module, configured to substitute the third feature into a feature extraction module to obtain a fourth feature, and fuse the fourth feature with the second feature to obtain a fifth feature; a first target feature determination module, configured to upsample the fifth feature and input the resultant feature into a feature extraction module to obtain a sixth feature, fuse the sixth feature with the first feature to obtain a seventh feature, and perform a 3×3 convolution operation on the seventh feature to obtain a first target feature; A target image determination module is configured to perform element-wise addition on the first enhanced image and the first target feature to obtain a target image.

9. The product information entry system based on image recognition according to claim 8, characterized in that: The feature extraction module includes: A first activation feature determination module is used to obtain input features, perform 1×1 partial convolution on the input features, and then input them into a Gaussian error linear unit to obtain a first activation feature; To the second activation feature determination module, which is used to fuse the first activation feature and the input feature to obtain a first fused activation feature, perform a 3×3 convolution on the first fused activation feature, and then input the convolution into a Gaussian error linear unit to obtain a second activation feature; A third activation feature determination module is configured to perform a fast Fourier transform on the input feature to obtain a frequency domain feature, perform two 1×1 partial convolutions on the frequency domain feature, and then perform an inverse fast Fourier transform on the frequency domain feature to obtain a third activation feature; An output feature determination module is used to perform 1×1 convolution on the second activation feature and the third activation feature and obtain the output feature of the feature extraction module.

10. The product information entry system based on image recognition according to claim 8, characterized in that: The input of the feature supervision module includes a sampled image and a sampled feature; The feature supervision module includes: a first fused convolution feature determination module, configured to successively substitute the sampled image into two consecutive convolution layers to obtain a first continuous convolution feature, and fuse the sampled image and the first continuous convolution feature to obtain a first fused convolution feature; a first normalized feature determination module, configured to perform a 1×1 convolution on the first fused convolution feature and then perform normalization processing to obtain a first normalized feature; a first convolution fusion feature determination module, configured to perform normalization on the sampled features and substitute the normalized features into the continuous convolution layer to obtain a first continuous convolution sampling feature, and multiply the first normalized features and the first continuous convolution sampling features to obtain a first convolution fusion feature; a second convolution fusion feature determination module, configured to substitute the first convolution fusion feature into a continuous convolution layer to obtain a second continuous convolution feature, and fuse the first continuous convolution sampling feature and the second continuous convolution feature to obtain a second convolution fusion feature; An output feature determination module is used to perform 1×1 convolution on the second convolution fusion feature to obtain the output feature of the feature supervision module.

Images