A cigarette box detection method based on image enhancement and color perception

Abstract

This invention discloses a cigarette case detection method based on image enhancement and color perception, comprising: acquiring and preprocessing cigarette case images; and inputting the preprocessed cigarette case images into a trained detection model to obtain detection results. By enhancing images and integrating color perception vectors into the feature extraction portion of the detection network, the invention achieves good detection results for low-quality image data, even in complex environments.

Description

Technical Field

[0001] The present invention relates to the technical field of computer vision and image processing, and in particular to a cigarette box detection method based on image enhancement and color perception. Background Art

[0002] Radio frequency identification (RFID) technology is widely used as a primary tool for warehouse and logistics management in the tobacco manufacturing and warehousing industries. The core function of RFID is to automatically identify and read relevant data from tags, enabling real-time product tracking, optimizing inventory and logistics management, and improving overall operational efficiency. While RFID offers significant advantages, it also presents some issues and challenges. RFID systems may be affected by interference from radio signals in the environment and from the physical properties of the tags themselves, resulting in readers being unable to correctly read tag information and affecting management accuracy. Furthermore, RFID tags may be difficult to read due to errors, damage, or physical damage. These factors can affect the accuracy of RFID in product tracking and inventory management. The procurement and maintenance costs of RFID systems, especially in large tobacco warehouses, are also a significant issue.

[0003] To address these issues, tobacco manufacturers have introduced piece case inspection systems, specifically deep learning-based image detection methods. However, during the inspection process, camera obstructions or angle restrictions may prevent the clear display of parts of the image. Furthermore, warehouse lighting conditions are often limited, resulting in images that are too dim, making it difficult to discern details. Another potential problem is that alternating light and dark conditions or reflections within the warehouse can cause excessive image contrast, blurring the boundaries of some piece case areas. Summary of the Invention

[0004] The purpose of the present invention is to overcome the defects of the prior art and provide a cigarette box detection method based on image enhancement and color perception to solve the problems of blurred images and excessive contrast in cigarette box detection due to limited working environment in the prior art.

[0005] To solve the above technical problems, the present invention is implemented by adopting the following solutions:

[0006] In a first aspect, the present invention provides a cigarette box detection method based on image enhancement and color perception, comprising:

[0007] Obtain cigarette box images and perform preprocessing;

[0008] Input the pre-processed cigarette box image into the trained detection model to obtain the detection result;

[0009] The trained detection model includes a trained image enhancement network, a trained color perception network, and an improved Fast R-CNN network. The trained image enhancement network, the trained color perception network, and the improved Fast R-CNN network transmit data in sequence. The training of the detection model includes:

[0010] The image enhancement network is trained using the pre-processed cigarette box image data to obtain a trained image enhancement network;

[0011] The enhanced image data output by the trained image enhancement network is input into the color perception network for training to obtain a trained color perception network;

[0012] The image-corresponding color perception vector output by the trained color perception network is fused into the feature extraction part of the Fast R-CNN network to obtain an improved Fast R-CNN network.

[0013] Furthermore, the preprocessing includes standardization and zero-meaning processing;

[0014] The formula for standardization is as follows:

[0015]

[0016] Where x is the data sample, μ is the mean of the sample, σ is the standard deviation of the sample, and x * is the data sample after standardization;

[0017] The formula for zero mean processing is as follows:

[0018]

[0019] Where N is the total number of data, x i is the value of the i-th dimension of the data, is the value of the i-th dimension in the j-th data, x i ′ is the value of the i-th dimension of the processed data.

[0020] Furthermore, the image enhancement network includes a brightness equalization module and a deblurring module in parallel; the encoder of the brightness equalization module includes three serial downsampling convolution blocks, each downsampling convolution block includes two convolution layers and a pooling layer connected in sequence, the decoder of the brightness equalization module includes three serial upsampling convolution blocks, each upsampling convolution block includes a transposed convolution layer and two convolution layers connected in sequence, and the encoder and decoder of the brightness equalization module are connected in a channel connection manner; the encoder of the deblurring module includes three serial downsampling convolution blocks, each downsampling convolution block includes a convolution layer, a hole convolution layer and an average pooling layer with a window size of 2 connected in sequence, the decoder of the deblurring module includes three serial upsampling convolution blocks, each upsampling convolution block includes a transposed convolution layer, a hole convolution layer and a convolution layer connected in sequence, and the encoder and decoder of the deblurring module are connected in a channel connection manner.

[0021] Furthermore, the brightness equalization module and the deblurring module in the image enhancement network use the same input. The image data generated by the brightness equalization module and the deblurring module are respectively averaged with the same pixel position value and superimposed into one image data. After passing through three sequentially connected convolution layers and one convolution layer with a convolution kernel of 1, the feature map is obtained as the output of the image enhancement network.

[0022] Furthermore, the image enhancement network is trained by blurring the clear images. The loss function during image enhancement network training is as follows:

[0023]

[0024] Where N is the number of pixels in the image, P i The value of the i-th pixel in the real image, P i ′ To generate the value of the i-th pixel in the image.

[0025] Furthermore, the color perception network includes two sequentially connected dilated convolutional layers with a dilation rate of 2, a residual block, two fully connected layers, and a softmax layer; the residual block includes three sequentially connected convolutional layers; the color perception network is trained using the cross entropy loss function as follows:

[0026]

[0027] Among them, y i is the true value of the i-th color, p i is the predicted value of the i-th category color.

[0028] Furthermore, the feature extraction part of the improved Fast R-CNN network passes the obtained feature map through a convolution layer and multiplies it with the corresponding color perception vector of the image to obtain a new feature map as the output of the feature extraction part; the region proposal network of the improved Fast R-CNN network generates multiple anchor boxes of different sizes and proportions at each position in the feature map, and uses a classifier and a regressor to predict the probability that the anchor box is the foreground and the size of the bounding box that needs to be adjusted.

[0029] In a second aspect, the present invention provides a cigarette box detection device based on image enhancement and color perception, comprising:

[0030] An acquisition module is used to acquire cigarette box images and perform preprocessing;

[0031] The detection module is used to input the pre-processed cigarette box image into the trained detection model to obtain the detection result;

[0032] The trained detection model includes a trained image enhancement network, a trained color perception network, and an improved Fast R-CNN network. The trained image enhancement network, the trained color perception network, and the improved Fast R-CNN network transmit data in sequence. The training of the detection model includes:

[0033] The image enhancement network is trained using the pre-processed cigarette box image data to obtain a trained image enhancement network;

[0034] The enhanced image data output by the trained image enhancement network is input into the color perception network for training to obtain a trained color perception network;

[0035] The image-corresponding color perception vector output by the trained color perception network is fused into the feature extraction part of the Fast R-CNN network to obtain an improved Fast R-CNN network.

[0036] In a third aspect, the present invention provides an electronic device comprising a processor and a storage medium; the storage medium is used to store instructions; and the processor is used to operate according to the instructions to execute the steps of the method described in the first aspect.

[0037] In a fourth aspect, the present invention provides a computer-readable storage medium having a computer program stored thereon, which implements the steps of the method described in the first aspect when the computer program is executed by a processor.

[0038] Compared with the existing technology, the beneficial effects achieved by the present invention are as follows: the present invention achieves better detection effects on low-quality image data even in complex working environments through image enhancement and the fusion of color perception vectors into the feature extraction part of the detection network. BRIEF DESCRIPTION OF THE DRAWINGS

[0039] Figure 1 1 is a flow chart of the cigarette box detection method based on image enhancement and color perception according to the present invention;

[0040] Figure 2 Schematic diagram of the structure of the image enhancement network of the present invention;

[0041] Figure 3 It is a schematic diagram of the structure of the color perception network of the present invention;

[0042] Figure 4 It is a structural diagram of the improved Fast R-CNN network of the present invention. DETAILED DESCRIPTION

[0043] The present invention will be further described below in conjunction with the accompanying drawings. The following embodiments are only used to more clearly illustrate the technical solutions of the present invention and are not intended to limit the scope of protection of the present invention.

[0044] Example 1:

[0045] like Figure 1 As shown, this embodiment provides a cigarette box detection method based on image enhancement and color perception, including:

[0046] Obtain cigarette box images and perform preprocessing;

[0047] Input the pre-processed cigarette box image into the trained detection model to obtain the detection result;

[0048] The preprocessing process for the acquired cigarette box images is the same as the preprocessing process in the following training process. The trained detection model includes a trained image enhancement network, a trained color perception network, and an improved Fast R-CNN network. The trained image enhancement network, the trained color perception network, and the improved Fast R-CNN network transmit data in sequence. The training of the detection model includes:

[0049] The image enhancement network is trained using the pre-processed cigarette box image data to obtain a trained image enhancement network;

[0050] The enhanced image data output by the trained image enhancement network is input into the color perception network for training to obtain a trained color perception network;

[0051] The image-corresponding color perception vector output by the trained color perception network is fused into the feature extraction part of the Fast R-CNN network to obtain an improved Fast R-CNN network.

[0052] Specifically, the image enhancement network is trained using the preprocessed cigarette box image data to obtain a trained image enhancement network, including:

[0053] S1: Collect image data from various shooting angles and perform standardization and zero-mean preprocessing on them.

[0054] S1.1: Data standardization. The formula is as follows:

[0055]

[0056] Where x represents the data sample of the training set, μ is the mean value of the sample, σ is the standard deviation of the sample, and x * is the data sample after standardization.

[0057] S1.2: Zero-mean data. For each set of data, subtract the mean value of each dimension from the input data. This will center the data in each dimension to 0. The formula is as follows:

[0058]

[0059] Where N represents the total number of data in the training set, x i Indicates the value of the i-th dimension of the processed data, represents the value of the i-th dimension in the j-th data in the training set, x i ′ Represents the value of the i-th dimension of the processed data.

[0060] S2: Use the pre-processed cigarette box image data to enhance the image network (structure diagram as shown in the figure) Figure 2 As shown in Figure 2, the trained image enhancement network is obtained.

[0061] S2.1: The image enhancement network begins with the brightness equalization module. Its input is preprocessed cigarette box image data. The encoder consists of three serially connected downsampling convolutional blocks. Each downsampling convolutional block consists of two consecutive convolutional layers and a pooling layer. The output feature map is scaled down by half compared to the input feature map. The encoder extracts features from the cigarette box image. The extracted feature map is then fed into the decoder, which consists of three serially connected upsampling convolutional blocks. Each upsampling convolutional block consists of a transposed convolutional layer and two normal convolutional layers. Transposed convolutions increase the output feature map to twice the size of the input feature map. The decoder of the brightness equalization module is serially connected after the encoder. A branch connects the encoder and decoder, connecting the downsampling convolutional block in the middle of the encoder with the upsampling convolutional block in the middle of the decoder. The outputs of these two blocks are summed and fed into the final upsampling convolutional block, resulting in the generated brightness-balanced image data.

[0062] S2.2: Next is the deblurring module, connected in parallel with the brightness equalization module. This module shares the same inputs as the brightness equalization module and consists of an encoder and decoder based on dilated convolutions. The encoder consists of three downsampling convolution blocks, each of which consists of a normal convolution layer, a dilated convolution layer, and an average pooling layer with a window size of 2 in series. The decoder consists of three upsampling convolution blocks, each of which consists of a transposed convolution layer, a dilated convolution layer, and a normal convolution layer in series. The decoder of the deblurring module is connected in series after the encoder, and a branch is used to connect the encoder and decoder. The downsampling convolution block in the middle of the encoder is connected to the upsampling convolution block in the middle of the decoder. The outputs of the two blocks are added together as the input of the final upsampling convolution block, resulting in the generated deblurred image data.

[0063] S2.3: Finally, the image data generated by the two modules are superimposed into a single image, C3, by taking the average of the values ​​at the same pixel position. This image is then passed through three convolutional layers in series, one with a kernel size of 1, to increase the number of channels to 256. The resulting feature map, C, is then fed into subsequent modules.

[0064] In addition, during the training phase, clear images are taken and blurred before use in training. The loss function of this module is expressed as follows:

[0065]

[0066] In the back propagation process, the parameters of the network in the module are updated by the gradient descent method; where N is the number of pixels in the image, P i The value of the i-th pixel in the real image, P i ′ To generate the value of the i-th pixel in the image.

[0067] The enhanced image data output by the trained image enhancement network is input into the color perception network (structure diagram as shown in Figure 3 As shown), the trained color perception network is obtained, including:

[0068] SS1: The color perception network takes the data C obtained in S2.3 as input. The network first connects two dilated convolutional layers in series, with a dilation ratio of 2. This helps capture a wider range of image information, including subtle feature differences and texture details. A residual block consisting of three convolutional layers is then introduced to further learn features in the image. The second convolutional layer in the residual block is used to increase the number of channels in the feature map. Next, two fully connected layers and a softmax layer convert the feature map into a one-dimensional vector, and the relationship between the features and color is learned. The fully connected layers enable advanced combination and mapping of features, converting them into a vector l representing color perception.

[0069] During the training phase, the cross entropy loss function is used, which is expressed as follows:

[0070]

[0071] Among them, y i is the true value of the i-th color, p i is the predicted value of the i-th category color.

[0072] SS2: The image corresponding color perception vector l obtained by the color perception network is used to assist the subsequent detection network (Fast R-CNN network) to achieve better detection results.

[0073] The color perception vector of the image output by the trained color perception network is fused into the feature extraction part of the Fast R-CNN network to obtain the improved Fast R-CNN network (structure diagram as shown in the figure). Figure 4 shown), including:

[0074] SSS1: Improves the Fast R-CNN network using the color perception vectors corresponding to the images output by the trained color perception network. The main improvement is in the feature extraction part of the Fast R-CNN network. A convolutional layer is added after the feature map obtained by the original feature extraction part, and the number of channels is adjusted to the same dimension as the color perception representation vector l obtained by SS1. The feature map obtained by multiplying the feature map with the vector l is then used as the output of the feature extraction part of the improved Fast R-CNN network. This operation enhances the influence of the color information of the cigarette box in the obtained feature map on the detection effect, which helps to improve the detection effect.

[0075] SSS2: In the subsequent region proposal network of the improved Fast R-CNN network, multiple anchor boxes of different sizes and scales are generated at each position in the feature map. Then, the classifier and regressor are used to predict the probability that these anchor boxes are foregrounds and the size of the bounding boxes that need to be adjusted.

[0076] In summary, the present invention achieves better detection effect on low-quality image data even in complex working environments by enhancing images and fusing color perception vectors into the feature extraction part of the detection network.

[0077] Example 2:

[0078] This embodiment provides a cigarette case detection device based on image enhancement and color perception, including:

[0079] An acquisition module is used to acquire cigarette box images and perform preprocessing;

[0080] The detection module is used to input the pre-processed cigarette box image into the trained detection model to obtain the detection result;

[0081] The trained detection model includes a trained image enhancement network, a trained color perception network, and an improved Fast R-CNN network. The trained image enhancement network, the trained color perception network, and the improved Fast R-CNN network transmit data in sequence. The training of the detection model includes:

[0082] The image enhancement network is trained using the pre-processed cigarette box image data to obtain a trained image enhancement network;

[0083] The enhanced image data output by the trained image enhancement network is input into the color perception network for training to obtain a trained color perception network;

[0084] The image-corresponding color perception vector output by the trained color perception network is fused into the feature extraction part of the Fast R-CNN network to obtain an improved Fast R-CNN network.

[0085] Example 3:

[0086] This embodiment provides an electronic device, including a processor and a storage medium; the storage medium is used to store instructions; the processor is used to operate according to the instructions to execute the steps of the method described in the first embodiment.

[0087] Example 4:

[0088] This embodiment provides a computer-readable storage medium having a computer program stored thereon. When the computer program is executed by a processor, the steps of the method described in the first embodiment are implemented.

[0089] The present application is described with reference to the flowcharts and / or block diagrams of the methods, devices (systems), and computer program products according to the embodiments of the present application. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of the processes and / or boxes in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the steps in the process. Figure 1 a process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0090] These computer program instructions may also be stored in a computer readable memory that can direct a computer or other programmable data processing device to work in a specific manner, so that the instructions stored in the computer readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 a process or multiple processes and / or boxes Figure 1 The function specified in one or more boxes.

[0091] Although the preferred embodiments of the present invention have been described, those skilled in the art may make additional changes and modifications to these embodiments once they have learned the basic creative concept. Therefore, the appended claims are intended to be interpreted as including the preferred embodiments and all changes and modifications that fall within the scope of the present invention.

[0092] Obviously, those skilled in the art may make various changes and modifications to the present invention without departing from the spirit and scope of the present invention. Thus, if such changes and modifications fall within the scope of the claims and their equivalents, the present invention is intended to include such changes and modifications.

Claims

1. A cigarette box detection method based on image enhancement and color perception, characterized in that: include: Obtain cigarette box images and perform preprocessing; Input the pre-processed cigarette box image into the trained detection model to obtain the detection result; The trained detection model includes a trained image enhancement network, a trained color perception network, and an improved Fast R-CNN network. The trained image enhancement network, the trained color perception network, and the improved Fast R-CNN network transmit data in sequence. The training of the detection model includes: The image enhancement network is trained using the pre-processed cigarette box image data to obtain a trained image enhancement network; The enhanced image data output by the trained image enhancement network is input into the color perception network for training to obtain a trained color perception network; The image-corresponding color perception vector output by the trained color perception network is fused into the feature extraction part of the Fast R-CNN network to obtain an improved Fast R-CNN network. The image enhancement network includes a brightness equalization module and a deblurring module in parallel; The color perception network consists of two sequentially connected dilated convolutional layers with a dilation rate of 2, a residual block, two fully connected layers, and a softmax layer. The residual block consists of three sequentially connected convolutional layers. The cross entropy loss function is used for color perception network training as follows: ； Where, For the The true value of the class color, For the predicted The value of the class color; The feature extraction part of the improved Fast R-CNN network is to pass the obtained feature map through a convolution layer to adjust the number of channels to the same as the dimension of the color perception vector corresponding to the image, and multiply it with the color perception vector corresponding to the image to obtain a new feature map as the output of the feature extraction part; the region proposal network of the improved Fast R-CNN network generates multiple anchor boxes of different sizes and proportions at each position in the feature map, and uses a classifier and regressor to predict the probability that the anchor box is the foreground and the size of the bounding box that needs to be adjusted.

2. The cigarette box detection method based on image enhancement and color perception according to claim 1, characterized in that: Preprocessing includes standardization and zero-meaning; The formula for standardization is as follows: ； In the formula x is the data sample, is the mean value of the sample, is the standard deviation of the sample, is the data sample after standardization; The formula for zero mean processing is as follows: ； In the formula N is the total number of data, For the data Dimension value, For the The first Dimension value, The processed data Dimension value.

3. The cigarette box detection method based on image enhancement and color perception according to claim 1, characterized in that: The encoder of the brightness equalization module consists of three serially connected downsampling convolution blocks, each of which consists of two sequentially connected convolution layers and a pooling layer. The decoder of the brightness equalization module consists of three serially connected upsampling convolution blocks, each of which consists of a sequentially connected transposed convolution layer and two convolution layers. In the brightness equalization module, the decoder is connected in series after the encoder and a branch is used to connect the encoder and decoder. The downsampling convolution block in the middle of the encoder is connected to the upsampling convolution block in the middle of the decoder, and the outputs of the two blocks are added as the output of the last upsampling convolution block; the encoder of the deblurring module includes three downsampling convolution blocks connected in series, each downsampling convolution block includes a convolution layer, a hole convolution layer and an average pooling layer with a window size of 2 connected in sequence, and the decoder of the deblurring module includes three upsampling convolution blocks connected in series, each upsampling convolution block includes a transposed convolution layer, a hole convolution layer and a convolution layer connected in sequence; the decoder in the deblurring module is connected in series after the encoder and a branch is used to connect the encoder and decoder. The downsampling convolution block in the middle of the encoder and the upsampling convolution block in the middle of the decoder are connected, and the outputs of the two blocks are added as the output of the last upsampling convolution block.

4. The cigarette box detection method based on image enhancement and color perception according to claim 3, characterized in that: The brightness equalization module and the deblurring module in the image enhancement network use the same input. The image data generated by the brightness equalization module and the deblurring module are averaged with the same pixel position value and superimposed into one image data. After passing through three sequentially connected convolution layers and one convolution layer with a convolution kernel of 1, the feature map is obtained as the output of the image enhancement network.

5. The cigarette box detection method based on image enhancement and color perception according to claim 1, characterized in that: The image enhancement network is trained by blurring the clear images. The loss function during image enhancement network training is as follows: ； in, is the number of pixels in the image, For the real image Pixel value, To generate the image Pixel value.

6. A cigarette box detection device based on image enhancement and color perception, characterized in that: include: An acquisition module is used to acquire cigarette box images and perform preprocessing; The detection module is used to input the pre-processed cigarette box image into the trained detection model to obtain the detection result; The trained detection model includes a trained image enhancement network, a trained color perception network, and an improved Fast R-CNN network. The trained image enhancement network, the trained color perception network, and the improved Fast R-CNN network transmit data in sequence. The training of the detection model includes: The image enhancement network is trained using the pre-processed cigarette box image data to obtain a trained image enhancement network; The enhanced image data output by the trained image enhancement network is input into the color perception network for training to obtain a trained color perception network; The image-corresponding color perception vector output by the trained color perception network is fused into the feature extraction part of the Fast R-CNN network to obtain an improved Fast R-CNN network. The image enhancement network includes a brightness equalization module and a deblurring module in parallel; The color perception network consists of two sequentially connected dilated convolutional layers with a dilation rate of 2, a residual block, two fully connected layers, and a softmax layer. The residual block consists of three sequentially connected convolutional layers. The cross entropy loss function is used for color perception network training as follows: ； Where, For the The true value of the class color, For the predicted The value of the class color; The feature extraction part of the improved Fast R-CNN network is to pass the obtained feature map through a convolution layer to adjust the number of channels to the same as the dimension of the color perception vector corresponding to the image, and multiply it with the color perception vector corresponding to the image to obtain a new feature map as the output of the feature extraction part; the region proposal network of the improved Fast R-CNN network generates multiple anchor boxes of different sizes and proportions at each position in the feature map, and uses a classifier and regressor to predict the probability that the anchor box is the foreground and the size of the bounding box that needs to be adjusted.

7. An electronic device, characterized in that: The method comprises a processor and a storage medium; the storage medium is used to store instructions; the processor is used to operate according to the instructions to execute the steps of the method according to any one of claims 1 to 5.

8. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 5 are implemented.

Images