Cabinet beverage display detection method based on YOLOX

Abstract

The application discloses a showcase beverage display detection method based on YOLOX. The backbone network and feature pyramid network of YOLOX are used as a feature extraction module, and an output head is used as a prediction module; a lightweight neural network is used as a category weighting module to extract a category weighting vector, reassign weights to the feature extraction module and output several feature tensors; the feature extraction module, the category weighting module and the prediction module are jointly trained on a basic dataset; the parameters of the feature extraction module are frozen, the category weighting module and the prediction module are trained on a dataset of interest and the basic dataset, and the category weighting vector and a category name are added to a weighting vector library; a picture to be detected is added and a feature tensor is extracted, the feature tensor is reweighted by using the weighting vector, and whether there is a conflict between a showcase trademark and a beverage is judged. The category weighting vector is used for reweighting the feature tensor, so that the detection method does not depend on a large amount of sample data.

Description

Technical Field

[0001] This invention relates to a YOLOX-based method for detecting beverage displays in display cases, belonging to the field of target detection technology. Background Technology

[0002] Since its inception, the concept of artificial intelligence (AI) has sparked a surge in AI development, experiencing initial growth, a period of stagnation, and a phase of application development, and is currently in a period of rapid growth. Object detection, a branch of the vast field of AI, is one of the most fundamental and challenging problems in computer vision, and has received considerable attention in recent years. Its development over the past two decades can be seen as a microcosm of the history of computer vision. As one of the basic problems in computer vision, object detection forms the foundation of computer vision tasks such as instance segmentation, image captioning, and object tracking. In recent years, the rapid development of deep learning has injected new vitality into object detection, achieving remarkable breakthroughs.

[0003] With economic development, global business districts are increasingly demanding stricter regulations on goods. Standardized management of a wide variety of products is essential for scientific and standardized operations, facilitating categorized procurement, sales, and inventory, thereby improving management efficiency and economic benefits. To make purchasing convenient for customers, supermarkets and department stores have introduced display cases in addition to standard shelves. Driven by the need to serve their target customers and secure a place in the fiercely competitive market, brands of all sizes have begun to launch their own branded product display cases. This phenomenon is particularly evident in the beverage industry, becoming a new trend. Beverage manufacturers of all sizes have launched their own beverage display cases and placed their displays and products in various department stores, supermarkets, and retail outlets. To comply with the mall's merchandise display regulations and the rules of each brand, only products of their own brand can be placed in any beverage display case. Furthermore, brand owners and mall personnel conduct frequent assessments. The specific assessment method involves relevant staff going to designated locations to inspect each display case visually to check whether the beverages are displayed in accordance with regulations. However, there are many types of beverages; for example, one company has as many as 160 different beverage products. This process consumes a significant amount of human resources.

[0004] Currently, object detection technology is being applied to beverage display inspection in showcases. However, the wide variety of beverage categories results in low detection accuracy for algorithms such as YOLOv5, YOLOX, and Faster R-CNN, failing to meet basic requirements. Existing mainstream detection algorithms rely on a large number of training samples; as the number of target types increases, the required training samples increase dramatically, making their acquisition more difficult. Therefore, there is an urgent need for a beverage display inspection system.

[0005] Therefore, it is necessary to propose a detection method for beverages to solve the above problems. Summary of the Invention

[0006] The purpose of this invention is to provide a YOLOX-based method for detecting beverage displays in display cases. This method can achieve good detection accuracy even with a large number of categories, and does not rely on a large number of training samples, thus solving the problem of non-standard beverage display in display cases.

[0007] To achieve the above objectives, the present invention provides a YOLOX-based method for detecting beverage displays in display cases, comprising:

[0008] S1. Collect images of beverage display cases through on-site or online methods, and label the collected images to create datasets of interest and basic datasets.

[0009] S2. Use YOLOX's backbone network and feature pyramid network as feature extraction modules, and use YOLOX's output head as prediction modules;

[0010] S3. Use a lightweight neural network as the category weighting module. The category weighting module can extract the category weighting vector that can represent the category from the image, and use the category weighting vector to redistribute the weights of the output of the feature extraction module, outputting several feature tensors responsible for predicting different categories.

[0011] S4. Jointly train the feature extraction module, the category weighting module, and the YOLOX prediction module on the base dataset;

[0012] S5. Freeze the parameters of the feature extraction module, train the class weighting module and the prediction module on the dataset of interest and the basic dataset, maintain a class weighting library for storing class weighting vectors, and update the class weighting vectors and the class names they represent to the weighting vector library.

[0013] S6. The image to be detected is processed by the feature extraction module to obtain the feature tensor representing the target features. All weighted vectors are taken from the weighted vector library. Channel-level multiplication is performed on the weighted vectors and the feature tensor. Then the result is sent to the prediction module. The prediction module performs detection and post-processing. After post-processing, it determines whether there is a conflict between the trademark of the display case and the beverage inside.

[0014] As a further improvement of the present invention, S1 includes:

[0015] S11. On the one hand, use Python web crawlers to collect images of beverage display cases on the Internet, or take photos of beverage display cases on-site in supermarkets and shopping malls to obtain a full-view image containing the display case trademarks and beverage products.

[0016] S12. Crops out several small images containing only the display case logo or only a single beverage from the overall image to create single-target images, while retaining the overall image.

[0017] S13. Use LabelImg open-source annotation software to annotate all images. When annotating, use rectangles to mark the trademarks and brands of the display cases, the positions of the beverages in the display cases, and add category labels.

[0018] S14. Use single-target images as the dataset of interest and panoramic images as the base dataset.

[0019] As a further improvement of the present invention, S2 includes:

[0020] S21. Replace the spatial pyramid pooling in the YOLOX backbone network with fast spatial pyramid pooling;

[0021] S22. Use the feature pyramid to aggregate the information of the outputs of each branch of the YOLOX backbone network, and together with the YOLOX backbone network, form the feature extraction module F;

[0022] S23, Feature extraction module F uses the full-view image M from the basic dataset. i Given the input image, output the feature representation F(M). i ), which is a multidimensional tensor.

[0023] As a further improvement of the present invention, S3 includes:

[0024] S31. The category weighting module R includes 7 CBL components, and each CBL component is followed by a max pooling operation. The last CBL component is followed by a global max pooling operation. The CBL components include convolution units, normalization units, and linear correction units.

[0025] S32. When inputting the category weighting module R, a single target image M is required. j And single-target image M j Based on the annotation information, create a mask L representing the target's location. j Mask L j The dimensions are the same as the image, but the pixel values ​​are only 0 and 255. In the annotation information, the area corresponding to the rectangle is 0 pixels, and the rest is 255 pixels. Let S = (M1, L1) ∪ (M2, L2) … (M j ,L j S is the input to the category weighting module;

[0026] S33. The category weighting module outputs the category weight R(S) corresponding to the target in the image. The function of the category weight R(S) is to weight the feature F(M).i Reassign weights;

[0027] S34, the category weighting module is a lightweight network structure that only plays the role of allocating weights. It only needs to extract the weight coefficients of a specific category from a small number of samples.

[0028] As a further improvement of the present invention, S4 includes:

[0029] S41. Jointly train the feature extraction module F, the category weighting module R, and the prediction module P on the training set;

[0030] S42. Perform a series of image preprocessing on the images in the training set, including four-image stitching, image blending enhancement, random horizontal mirror flipping, random saturation transformation, random hue transformation, random brightness transformation, affine transformation, width and height scale transformation, image normalization, and grayscale pixel filling.

[0031] S43. Extract the overall image M from the base dataset. i The input is fed into the feature extraction module F, and the feature extraction module F outputs the feature representation of the image as F(M). i );

[0032] S44. Extract a single target image M from the dataset of interest. j and its Mask L j S = (M1, L1) ∪ (M2, L2) … (M j ,L j The input is fed into the category weighting module R, and R outputs the weighted vector R(S) of each category in S;

[0033] S45. Apply the weighted vector R(S) to the feature tensor F(M). i Reweighting is performed, i.e., F(M) i The channel multiplication of the weighted vector R(S) with the weighted vector is denoted as...

[0034] S46, The data is input into the prediction module P, which outputs the target's location information (bbox), category information (cls), and confidence information (obj). The category information is processed using the Softmax function. The category loss uses Cross Entropy Loss, the localization loss uses IoU Loss, and the confidence loss uses BCE Loss.

[0035] As a further improvement of the present invention, S5 includes:

[0036] S51. Freeze the parameters of the feature extraction module so that the parameters of the feature extraction module will not be optimized or updated;

[0037] S52. Jointly train the class-weighted module R and the prediction module P on the training set;

[0038] S53. Maintain a weighted vector library. At the end, add the category weighted vectors output by the category weighting module and their category names to the weighted vector library.

[0039] S55. As the category weighting vectors in the weighted vector library increase, the number of categories that the network can detect increases accordingly.

[0040] As a further improvement of the present invention, when it is necessary to detect a new beverage category, it is only necessary to add the new category data to the basic dataset and the dataset of interest, and then perform S5 separately.

[0041] As a further improvement of the present invention, S6 includes:

[0042] S61. Perform image preprocessing on the image to be detected, including grayscale filling, normalization, width and height scale transformation, and horizontal mirror flipping.

[0043] S62. Input the image to be detected x into the feature extraction module F to obtain the feature tensor F(x) of the image;

[0044] S63. Extract all category weighted vectors R(S) from the weighted vector library. Each category weighted vector is responsible for detecting one type of target.

[0045] S64. Perform channel-level multiplication on the category-weighted vector and feature tensor, i.e.

[0046] S65, will The results are sent to the prediction module P, which outputs target category information, location information, and confidence information.

[0047] S66. For the output of the prediction module, retrieve the category information of the display case trademark and the category information of all beverages in the display case. If the beverage category information matches the category of the display case trademark, use a green box to box it; otherwise, use a red box to box it.

[0048] The beneficial effects of this invention are as follows: The YOLOX-based beverage display detection method of this invention, by setting the category weighting module as a lightweight network structure, makes the detection method not dependent on a large amount of sample data, and significantly improves the detection accuracy when the sample is scarce; by establishing a weighted vector library and adding the category weighted vector to the weighted vector library, only the category weighted vector in the weighted vector library is used in the detection process, making the detector applicable to multi-category scenarios; furthermore, by detecting whether the beverage display brand corresponds to the beverage product brand displayed in the display case, the beverage display in the display case is standardized. Attached Figure Description

[0049] Figure 1 This is a full view of the display case trademark and beverage in the display case beverage display detection method based on YOLOX in this invention.

[0050] Figure 2 This is a single target image of the YOLOX-based display beverage display detection method in this invention.

[0051] Figure 3 This is a schematic diagram of a mask representing the target location.

[0052] Figure 4 This is a flowchart of S1 to S4.

[0053] Figure 5 This is the flowchart of image preprocessing in S42.

[0054] Figure 6 This is a flowchart of S1 to S5.

[0055] Figure 7 This is a flowchart of S6.

[0056] Figure 8 This is the flowchart of image preprocessing in S61. Detailed Implementation

[0057] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be described in detail below with reference to the accompanying drawings and specific embodiments.

[0058] This invention provides a YOLOX-based method for detecting beverage displays in display cases, used to check whether the beverages displayed in the case are in accordance with regulations. Specifically, it includes the following steps:

[0059] S1. Collect images of beverage display cases through on-site or online methods, and label the collected images to create datasets of interest and basic datasets.

[0060] Please see Figure 1 and Figure 2 As shown, S1 specifically includes:

[0061] S11. On the one hand, use Python web scraping to collect images of beverage display cases from the internet, or take photos of beverage display cases in supermarkets and shopping malls to obtain full-view images. The full-view images must include the brand logo of the display case and the beverage products displayed inside. Of course, other methods can also be used to obtain full-view photos; there are no restrictions here.

[0062] S12. Crop out several small images containing only the display case logo or only a single beverage from the overall image to create single-target images, while retaining the overall image.

[0063] S13. Use LabelImg open-source annotation software to annotate all images. When annotating, use rectangles to mark the trademarks and brands of the display cases, the positions of the beverages in the display cases, and add category labels.

[0064] S14. Use single-target images as the dataset of interest and panoramic images as the base dataset.

[0065] S2. Use YOLOX's backbone network and feature pyramid network as feature extraction modules, and use YOLOX's output head as prediction modules;

[0066] S2 specifically includes:

[0067] S21. Replace the spatial pyramid pooling in the YOLOX backbone network with fast spatial pyramid pooling; fast spatial pyramid pooling concatenates max pooling, which is faster than the parallel structure of max pooling in spatial pyramid pooling, thus improving training and detection speeds.

[0068] S22. Use the feature pyramid to aggregate the information of the outputs of each branch of the YOLOX backbone network, and together with the YOLOX backbone network, form the feature extraction module F;

[0069] S23, Feature extraction module F uses the full-view image M from the basic dataset. i Given the input image, output the feature representation F(M). i ), which is a multidimensional tensor.

[0070] S3. Use a lightweight neural network as the category weighting module. The category weighting module can extract the category weighting vector that can represent the category from the image, and use the category weighting vector to redistribute the weights of the output of the feature extraction module, outputting several feature tensors responsible for predicting different categories.

[0071] Please see Figure 3 and Figure 4 As shown, S3 specifically includes:

[0072] S31. The category weighting module R includes 7 CBL components, and each CBL component is followed by a max pooling operation. The last CBL component is followed by a global max pooling operation. The CBL components include Conv (convolutional unit), Normalize (normalization unit), and LeakyRelu (leakage linear correction unit).

[0073] S32. When using the category weighting module R, a single target image M needs to be input.j And single-target image M j Based on the annotation information, create a mask L representing the target's location. j Mask L j The dimensions are the same as the image, but the pixel values ​​are only 0 and 255. In the annotation information, the area corresponding to the rectangle is 0 pixels, and the rest is 255 pixels. Let S = (M1, L1) ∪ (M2, L2) … (M j ,L j S is the input to the category weighting module;

[0074] S33. The category weighting module outputs the category weight R(S) corresponding to the target in the image. The function of the category weight R(S) is to weight the feature F(M). i Reassign weights;

[0075] S34, the category weighting module is a lightweight network structure that only plays the role of allocating weights. It only needs to extract the weight coefficients of a specific category from a small number of samples.

[0076] Specifically, the feature extraction module outputs a feature tensor representing the target features, which is an N×H×W×C tensor, where H is the height, W is the width, and C is the number of channels. The class weighting module outputs several class weighted vectors, each corresponding to a target class. Channel-level multiplication is performed on the feature tensor and the weighted vectors, i.e., the weights are reassigned to the feature tensor so that it can detect specific classes. After this, several new reweighted feature tensors are obtained, each responsible for detecting a target of a certain class.

[0077] S4. Jointly train the feature extraction module, the category weighting module, and the YOLOX prediction module on the base dataset;

[0078] S4 specifically includes:

[0079] S41. Jointly train the feature extraction module F, the category weighting module R, and the prediction module P on the basic dataset;

[0080] S42. Perform a series of image preprocessing steps on the images in the basic dataset;

[0081] S43. Extract the overall image M from the base dataset. i The input is fed into the feature extraction module F, and the feature extraction module F outputs the feature representation of the image as F(M). i );

[0082] S44. Extract a single target image M from the dataset of interest. j and its Mask L jS = (M1, L1) ∪ (M2, L2) … (M j ,L j The input is fed into the category weighting module R, and R outputs the weighted vector R(S) of each category in S;

[0083] Please refer to details. Figure 4 and Figure 5 As shown, image preprocessing for the basic dataset includes four-image stitching, image blending enhancement, random horizontal mirroring, random saturation transformation, random tone transformation, random brightness transformation, affine transformation, aspect ratio and height scale transformation, image normalization, and grayscale pixel filling. After preprocessing, the overall image of the basic dataset is input into the feature extraction module. Masks are created for single-target images in the dataset of interest. Several single-target images and their annotation information are taken, with the annotation information containing target location information. Masks are calculated for these images using the target location information, i.e., using 0 pixels and 255 pixels to mark the target location in the image. The single-target image and its corresponding mask are input into the category weighting module.

[0084] S45. Apply the weighted vector R(S) to the feature tensor F(M). i Reweighting is performed, i.e., F(M) i The channel multiplication of the weighted vector R(S) with the weighted vector is denoted as...

[0085] S46, The data is input into the prediction module P, which outputs the target's location information (bbox), category information (cls), and confidence information (obj). The category information (cls) is processed using the Softmax function. Where i represents consecutive class numbers, e represents the natural constant, N represents the maximum value of the class number, the class loss uses Cross Entropy Loss, the localization loss uses IoU Loss, and the confidence loss uses BCE Loss.

[0086] Specifically, the new feature tensor is input into the prediction module, and the output of the prediction module and the labeled information are fed into the loss function, with parameters updated through backpropagation. Since the prediction module output includes target category information, location information, and confidence information, the training phase loss function includes three parts: category loss, localization loss, and confidence loss. Let the network predict the category information as c, the location information as b, and the confidence information as conf, with corresponding manually labeled information as label_c, label_b, and label_conf, respectively. Because the application scenario of this invention involves too many target categories, resulting in a large variance in the category scores for all targets, softmax processing is first applied to category c. Next, the category information output by the network. Using the cross-entropy loss function, record the class loss corresponding to the network's output class. Apply the intersection-union loss function to the location information, and denote the loss as Loss. b =torch.nn.IoULoss(b,label_b), similarly using the cross-entropy loss function for the confidence loss, denoted as Loss conf =torch.nn.CrossEntropyLoss(conf,label_conf). The overall loss function is Loss = Loss c +5×Loss b +Loss conf .

[0087] The parameters S1 to S3 are iterated continuously, and all parameters in the network are continuously updated and iterated using the corresponding learning rate scheduling strategy and optimizer grouping strategy. Specifically, in the parameter update part, the present invention uses the SGD optimizer, uses the cosine annealing learning rate to regulate the update rate, and precisely groups the parameters in the network: the weights of the convolutional layer are in the first group, the weights of the normalized layer are in the second group, and the biases of the convolutional layer and the normalized layer are in the third group, with different learning rate warmup strategies used for different groups.

[0088] S5. Freeze the parameters of the feature extraction module, train the class weighting module and the prediction module on the dataset of interest and the basic dataset, maintain a class weighting library for storing class weighting vectors, and update the class weighting vectors and the class names they represent to the weighting vector library.

[0089] Please see Figure 6 As shown, S5 specifically includes:

[0090] S51. Freeze the parameters of the feature extraction module so that the parameters of the feature extraction module will not be optimized or updated;

[0091] S52. Jointly train the class-weighted module R and the prediction module P on the training set;

[0092] S53. Maintain a weighted vector library. At the end, add the category weighted vectors output by the category weighting module and their category names to the weighted vector library.

[0093] S55. As the category weighting vectors in the weighted vector library increase, the number of categories that the network can detect increases accordingly.

[0094] When a new beverage category needs to be detected, simply add the new category data to the base dataset and the dataset of interest, and then perform S5 separately.

[0095] S6. The image to be detected is processed by the feature extraction module to obtain the feature tensor representing the target features. All weighted vectors are taken from the weighted vector library. Channel-level multiplication is performed on the weighted vectors and the feature tensor. Then the result is sent to the prediction module. The prediction module performs detection and post-processing. After post-processing, it determines whether there is a conflict between the trademark of the display case and the beverage inside.

[0096] Please see Figure 7 As shown, S6 specifically includes:

[0097] S61. Perform image preprocessing on the image to be detected, including grayscale filling, normalization, width and height scale transformation, and horizontal mirror flipping.

[0098] Specifically, before detection, the image needs to be preprocessed to ensure that the image size meets the output requirements of the object detection network model and to minimize the distortion rate of the feature information in the image.

[0099] Please see Figure 8 As shown, first, a horizontal mirror flip is performed, flipping the image horizontally; then, a width and height scale transformation is used to force the image's width and height to be set to 640×640; next, normalization is used to normalize the original pixel values ​​of 0-255 in the image to pixel values ​​of 0-1; finally, grayscale fill is used to make the image's length and width multiples of 32, and any insufficient parts are filled with pixel values ​​of 114.

[0100] S62. Input the image to be detected x into the feature extraction module F to obtain the feature tensor F(x) of the image;

[0101] S63. Extract all category weighted vectors R(S) from the weighted vector library. Each category weighted vector is responsible for detecting one type of target.

[0102] S64. Perform channel-level multiplication on the category-weighted vector and feature tensor, i.e.

[0103] S65, will The results are sent to the prediction module P, which outputs target category information, location information, and confidence information.

[0104] Specifically, after preprocessing, the image enters the network. First, it passes through a feature extraction module to extract image features. The feature extraction module outputs a feature tensor representing these features. All class-weighted vectors are retrieved from the weighted vector library. Channel multiplication is performed on the feature tensor and the class-weighted vectors. Finally, the prediction module obtains a tensor Pred containing the target confidence (Conf), target location (x, y, w, h), and target class information (Cls). It is important to emphasize again that the class-weighted module is not used in the detection phase; only the class-weighted vectors from the weighted vector library are used.

[0105] S66. For the output of the prediction module, retrieve the category information of the display case trademark and the category information of all beverages in the display case. If the beverage category information matches the category of the display case trademark, use a green box to box it; otherwise, use a red box to box it.

[0106] To verify the detection method mentioned in this invention, 160 types of beverages and 6 types of display cases were collected, totaling 166 target categories. Of these, 120 target categories had abundant data samples, while the remaining 46 target categories had only 1 to 10 samples per category. The 120 target categories with abundant samples were used to create a base dataset and a dataset of interest for the first stage of the training process. The remaining 46 target categories with scarce samples were added to the base dataset and the dataset of interest after the first stage for the second stage of the training process, with the dataset of interest in the second stage containing only 1, 5, and 10 samples per category, respectively. For comparison, experiments were conducted using the original YOLOX on the 166 target categories.

[0107] Experiment 1: For the 46 target classes, each class had only 1 sample. In the second stage of the training process, this invention uses the dataset of interest, with each class having only 1 sample. The experimental data is shown in the table below (the 120 target classes with abundant samples are referred to as the base classes, and the 46 target classes with scarce samples are referred to as the new classes. The evaluation metric used is mean average precision (mAP), with a threshold of 0.5):

[0108] Base class mAP New class mAP 166 types of mAP YOLOX 0.62 0.02 0.45 Method of the present invention 0.70 0.14 0.54

[0109] Experiment 2: For 46 target classes, each class had 5 samples. In the second stage of the training process, this invention used the dataset of interest, with each class having 5 samples. The experimental data is shown in the table below:

[0110] Base class mAP New class mAP 166 types of mAP YOLOX 0.61 0.09 0.46 Method of the present invention 0.73 0.26 0.59

[0111] Experiment 3: For 46 target classes, each class had 10 samples. In the second stage of the training process, this invention used the dataset of interest, with each class having 10 samples. The experimental data is shown in the table below:

[0112] Base class mAP New class mAP 166 types of mAP YOLOX 0.62 0.13 0.48 Method of the present invention 0.72 0.42 0.62

[0113] It can be seen that the detection accuracy of the YOLOX-based detection method provided by this invention is much higher than that of the traditional YOLOX detection method.

[0114] In summary, this invention provides a YOLOX-based method for detecting beverage displays in display cases. By setting the category weighting module as a lightweight network structure, the detection method does not rely on a large amount of sample data. By establishing a weighted vector library and adding category weighted vectors to it, the detection process only uses category weighted vectors from the library, thus improving detection accuracy. Furthermore, the invention standardizes the beverage display in the display case by detecting whether the brand of the beverage display case corresponds to the brand of the beverage products displayed.

[0115] The above embodiments are only used to illustrate the technical solutions of the present invention and are not intended to limit it. Although the present invention has been described in detail with reference to preferred embodiments, those skilled in the art should understand that modifications or equivalent substitutions can be made to the technical solutions of the present invention without departing from the spirit and scope of the technical solutions of the present invention.

Claims

1. A method for detecting beverage displays in display cases based on YOLOX, characterized in that, include: S1. Collect images of beverage display cases through on-site or online methods, and label the collected images to create datasets of interest and basic datasets. S2. Use YOLOX's backbone network and feature pyramid network as feature extraction modules, and use YOLOX's output head as prediction modules; S3. Use a lightweight neural network as the category weighting module. The category weighting module can extract the category weighting vector that can represent the category from the image, and use the category weighting vector to redistribute the weights of the output of the feature extraction module, outputting several feature tensors responsible for predicting different categories. S4. Jointly train the feature extraction module, the category weighting module, and the YOLOX prediction module on the base dataset; S5. Freeze the parameters of the feature extraction module, train the class weighting module and the prediction module on the dataset of interest and the basic dataset, maintain a class weighting library for storing class weighting vectors, and update the class weighting vectors and the class names they represent to the weighting vector library. S6. The image to be detected is processed by the feature extraction module to obtain the feature tensor representing the target features. All weighted vectors are taken from the weighted vector library. Channel-level multiplication is performed on the weighted vectors and the feature tensor. Then the result is sent to the prediction module. The prediction module performs detection and post-processing. After post-processing, it determines whether there is a conflict between the trademark of the display case and the beverage inside. S2 includes: S21. Replace the spatial pyramid pooling in the YOLOX backbone network with fast spatial pyramid pooling; S22. Use the feature pyramid to aggregate the information of the outputs of each branch of the YOLOX backbone network, and together with the YOLOX backbone network, form the feature extraction module F; S23, Feature extraction module F uses the full-view image M from the basic dataset. i Given the input image, output the feature representation F(M). i ), which is a multidimensional tensor.

2. The display beverage detection method based on YOLOX according to claim 1, characterized in that, S1 includes: S11. On the one hand, use Python web crawlers to collect images of beverage display cases on the Internet, or take photos of beverage display cases on-site in supermarkets and shopping malls to obtain a full-view image containing the display case trademarks and beverage products. S12. Crops out several small images containing only the display case logo or only a single beverage from the overall image to create single-target images, while retaining the overall image. S13. Use LabelImg open-source annotation software to annotate all images. When annotating, use rectangles to mark the trademarks and brands of the display cases, the positions of the beverages in the display cases, and add category labels. S14. Use single-target images as the dataset of interest and panoramic images as the base dataset.

3. The display beverage detection method based on YOLOX according to claim 1, characterized in that, S3 include: S31. The category weighting module R includes 7 CBL components, and each CBL component is followed by a max pooling operation. The last CBL component is followed by a global max pooling operation. The CBL components include convolution units, normalization units, and linear correction units. S32, The category weighting module R requires a single target image M as input. j And single-target image M j Based on the annotation information, create a mask L representing the target's location. j Mask L j The dimensions are the same as the image, but the pixel values ​​are only 0 and 255. In the annotation information, the area corresponding to the rectangle is 0 pixels, and the rest is 255 pixels. Let S = (M1, L1) ∪ (M2, L2) … (M j ,L j S is the input to the category weighting module; S33. The category weighting module outputs the category weight R(S) corresponding to the target in the image. The function of the category weight R(S) is to weight the feature F(M). i Reassign weights; S34, the category weighting module is a lightweight network structure that plays the role of allocating weights. It only needs to extract the weight coefficients of a specific category from a small number of samples.

4. The display beverage detection method based on YOLOX according to claim 1, characterized in that, S4 include: S41. Jointly train the feature extraction module F, the category weighting module R, and the prediction module P on the training set; S42. Perform a series of image preprocessing on the images in the training set, including four-image stitching, image blending enhancement, random horizontal mirror flipping, random saturation transformation, random hue transformation, random brightness transformation, affine transformation, width and height scale transformation, image normalization, and grayscale pixel filling. S43. Extract the overall image M from the base dataset. i The input is fed into the feature extraction module F, and the feature extraction module F outputs the feature representation of the image as F(M). i ); S44. Extract a single target image M from the dataset of interest. j and its Mask L j S = (M1, L1) ∪ (M2, L2) … (M j ,L j The input is fed into the category weighting module R, and R outputs the weighted vector R(S) of each category in S; S45. Apply the weighted vector R(S) to the feature tensor F(M). i Reweighting is performed, i.e., F(M) i The channel multiplication of the weighted vector R(S) with the weighted vector is denoted as... S46, The data is input into the prediction module P, which outputs the target's location information (bbox), category information (cls), and confidence information (obj). The category information (cls) is processed using the Softmax function. The category loss uses Cross Entropy Loss, the localization loss uses IoU Loss, and the confidence loss uses BCE Loss.

5. The display beverage detection method based on YOLOX according to claim 1, characterized in that, S5 include: S51. Freeze the parameters of the feature extraction module so that the parameters of the feature extraction module will not be optimized or updated; S52. Jointly train the class-weighted module R and the prediction module P on the training set; S53. Maintain a weighted vector library. At the end, add the category weighted vectors output by the category weighting module and their category names to the weighted vector library. S54. As the category weighting vectors in the weighted vector library increase, the number of categories that the network can detect increases accordingly.

6. The YOLOX-based beverage display detection method for display cases according to claim 5, characterized in that: When a new beverage category needs to be detected, simply add the new category data to the base dataset and the dataset of interest, and then perform S5 separately.

7. The display beverage detection method based on YOLOX according to claim 1, characterized in that, S6 include: S61. Perform image preprocessing on the image to be detected, including grayscale filling, normalization, width and height scale transformation, and horizontal mirror flipping. S62. Input the image to be detected x into the feature extraction module F to obtain the feature tensor F(x) of the image; S63. Extract all category weighted vectors R(S) from the weighted vector library. Each category weighted vector is responsible for detecting one type of target. S64. Perform channel-level multiplication on the category-weighted vector and feature tensor, i.e. S65, will The results are sent to the prediction module P, which outputs target category information, location information, and confidence information. S66. For the output of the prediction module, retrieve the category information of the display case trademark and the category information of all beverages in the display case. If the beverage category information matches the category of the display case trademark, use a green box to box it; otherwise, use a red box to box it.

Images