Single-sample learning lightweight neural network-based shelf commodity visual inventory method

Abstract

The invention discloses a shelf commodity visual checking method based on a single-sample learning lightweight neural network. According to the method, a lightweight convolutional neural network model which comprises a backbone network, feature fusion and a detection head and is based on single sample learning is constructed, a TrackingNet data set is introduced to construct a data set suitable for single sample learning, and a classification loss function, a target frame regression function and a centrality loss function are set as loss functions; therefore, training of the lightweight convolutional neural network model based on single sample learning is completed. And detecting a corresponding commodity in a newly input image according to a support image provided by a user through the trained lightweight convolutional neural network model based on single sample learning, and marking the commodity in a rectangular frame form. Therefore, the user can complete the inventory task through one commodity image, and meanwhile, the parameter quantity of the whole network is reduced, so that the system can be deployed on a computer with low computing resources.

Description

Technical Field

[0001] The present invention belongs to the technical field of computer vision and relates to a visual inventory method of shelf commodities using a single-sample learning lightweight neural network. Background Art

[0002] Offline retail supermarkets and daily necessities e-commerce generally use shelves to store daily necessities for warehousing. In order to effectively manage daily necessities, these commodities need to be queried and counted at any time. At present, with the help of existing camera equipment, real-time images of shelves can be easily obtained, which provides users with real-time visual information about inventory items. In the actual inventory process of stored items, users often need to directly query and count the inventory of the item based on a picture of the item they have, and the target detection method based on single-sample learning provides technical support for the visual inventory of items.

[0003] In response to the inventory needs of goods on shelves, the disclosed invention patent "Automatic Inventory Method and Device for Goods Inventory" with application number 202110926692.2 obtains the location information and category of each product on the shelf through image recognition. It requires setting the preset position of the camera so that the camera shooting area can cover the shelf area. At the same time, it manually marks the shelf grid to obtain the grid location information, and further performs image recognition on the goods in the grid to obtain the quantity of goods, and finally obtains the inventory information of the goods in each grid. The disclosed invention patent "Automatic Warehousing System, Goods Inventory Method and Server" with application number 202210863761.4 has built a warehousing system that uses multiple cameras to count the quantity of goods on the shelves. The system needs to obtain the category information of the goods by identifying the identification code on the surface of the goods being counted, and further uses multi-angle cameras to count rectangular goods. The disclosed invention patent with publication number CN202210414325.9, “A method, device and system for automatic warning and replenishment of Chinese herbal medicine inventory”, uses the Mask R-CNN image segmentation algorithm to detect the Chinese herbal medicine in the Chinese medicine cabinet, which ultimately needs to be combined with the weight data of the Chinese herbal medicine obtained by the pressure sensor at the bottom of the Chinese medicine cabinet to determine the inventory quantity of the Chinese herbal medicine. The disclosed invention patent with publication number CN202310021309.8, “An inventory counting management system based on image recognition”, uses two image acquisition units to shoot three-dimensional structure inventory items, and then directly extracts the image and compares it with the database image features to determine the same type of inventory items, thereby obtaining the number of inventory items.

[0004] The device proposed in the above invention patent requires manual setting of image recognition tags on the shelves, configuration of multiple cameras or installation of auxiliary sensors to count, inventory and query the goods, which results in a complex shelf structure and extremely inconvenient deployment. Secondly, the categories of goods on the shelves are currently pre-set manually, and the categories of items that can be counted are limited to the range of manually preset categories and cannot be extended to categories that are not pre-set. Finally, the inventory of goods on the shelves currently relies entirely on manual text input information for query, and does not provide the function of inventory by picture. Summary of the invention

[0005] In order to solve the problems existing in the background technology, the present invention proposes a shelf commodity visual inventory method based on a single-sample learning lightweight neural network.

[0006] In order to achieve the above object, the technical solution adopted by the present invention is as follows:

[0007] A method for visual inventory of shelf commodities using a single-sample learning lightweight neural network, including:

[0008] Build a lightweight convolutional neural network model based on single-sample learning, which consists of three parts: backbone network, feature fusion and detection head;

[0009] The TrackingNet dataset is introduced to construct a dataset suitable for single-sample learning, providing training data for the lightweight convolutional neural network model based on single-sample learning;

[0010] Set the loss function for the lightweight convolutional neural network model based on single-sample learning. The loss function includes classification loss function, target box regression function and center loss function.

[0011] Based on the trained single-sample learning-based lightweight convolutional neural network model, the corresponding products are detected in the new input image according to the support images provided by the user and marked in the form of a rectangular box.

[0012] Furthermore, the construction of a lightweight convolutional neural network model based on single-sample learning includes:

[0013] Select some neural network layers of MobileNetV3 to build the backbone network;

[0014] MobileNetV3 consists of 1 convolution layer, 1 batch normalization layer, a rectified linear activation function layer, 1 convolution layer, 1 batch normalization layer, 1 compression and excitation network layer, activation function layer, 1 convolution layer, and 1 batch normalization layer stacked in sequence;

[0015] The first convolution layer and the first 10 basic components of MobileNetV3 are cascaded to form a backbone network, where the convolution kernel size, span, boundary padding, and activation function layer type of the 10 basic components are different;

[0016] The compression and excitation network layers are embedded in the 2nd and 5th to 11th layer infrastructure of the backbone network;

[0017] The basic components of the outer 2nd to 4th layers use the modified linear activation function ReLU function as the type of activation function, and the modules of other layers use the improved Swish activation function. The specific formula of the improved Swish activation function is:

[0018]

[0019] Where x represents the input eigenvalue and >> represents the right shift operator.

[0020] Furthermore, the construction of a lightweight convolutional neural network model based on single sample learning also includes:

[0021] Feature fusion includes point-level correlation fusion module, spatial channel fusion module and inverse activation module;

[0022] Among them, the point-level correlation fusion module is responsible for multiplying the feature map of the support image and the image to be detected in matrix form to obtain the correlation map, which is sent to the compression and excitation network layer to enhance the neural network's attention to important channels;

[0023] The compression and excitation network first pools the correlation graph into a single output value through an adaptive average pooling layer, reduces the dimension through a fully connected layer, and performs a nonlinear transformation through a linear rectification function, then increases the dimension through a fully connected layer, and performs a nonlinear transformation through a linear rectification function, and finally outputs the weight coefficient of each channel through the Sigmod function, which is used to perform weighted calculations on different channels of the correlation graph itself;

[0024] The spatial channel fusion module contains 1 convolutional neural network layer, 1 batch normalization layer, 1 convolutional neural network layer, and 1 batch normalization layer;

[0025] The inverse activation module consists of 1 convolutional neural network layer, 1 batch normalization layer, 1 convolutional neural network layer, 1 Gaussian error linear unit layer, 1 convolutional neural network layer, 1 batch normalization layer, and 1 convolutional neural network layer.

[0026] Furthermore, the construction of a lightweight convolutional neural network model based on single sample learning also includes:

[0027] The detection head includes the classification head, the target box regression head and the centrality measurement;

[0028] Among them, the classification head contains 4 convolution blocks and 1 convolution classification layer. Each convolution block consists of 1 convolutional neural network layer, 1 group normalization layer, and 1 linear rectification function layer.

[0029] The last convolution classification layer has an output channel number of 1, a convolution span of 1, and a convolution kernel size of 3×3, which is used to estimate the confidence that each position point is the target object to be searched. The confidence map is finally input into the Sigmod function so that the final confidence value is compressed in the interval [0,1];

[0030] The frame regression head contains 4 convolution blocks and 1 convolution regression layer. Each convolution block consists of 1 convolutional neural network layer, 1 group normalization layer, and 1 linear rectification function layer.

[0031] The network parameter settings of the convolution block are the same as those of the convolution block in the classification head;

[0032] The last convolutional regression layer has 4 output channels, a convolution stride of 1, and a convolution kernel size of 3×3. The output images of the four channels represent the distance between the corresponding pixel of each point on the input image and the four sides of the rectangular target box.

[0033] The centrality measurement of the target box consists of one convolutional layer, which receives the feature maps output by the four convolutional blocks in the classification head and finally outputs a centrality map, which represents the confidence that the current position is the center of the target.

[0034] Furthermore, the specific method of introducing the TrackingNet dataset to construct a dataset suitable for single-sample learning and providing training data for a lightweight convolutional neural network model based on single-sample learning is as follows:

[0035] The TrackingNet dataset contains several short video clips. Each video clip is labeled with whether the target is visible and the spatial position of the target frame. Two frames of video are randomly extracted from each video clip to form a pair of target images to be detected and support images.

[0036] The support image is obtained by cropping the image region containing the target object in the frame image according to the marked target frame;

[0037] The support image and the image to be detected are used as the input of the lightweight convolutional neural network model based on single-sample learning. The size of the support image and the size of the image to be detected are uniformly scaled to 224×224.

[0038] Furthermore, the specific method for setting the loss function for the lightweight convolutional neural network model based on single-sample learning is:

[0039] The lightweight convolutional neural network model based on single-sample learning outputs a 7×7×1 confidence map, mapping each point on the confidence map to a pixel point on the 224×224 input image;

[0040] For each point, the true value label y is constructed. If the pixel corresponding to the true value label y falls within the target box, the point is considered to be a positive sample y=1, otherwise the true value label y is considered to be a negative sample y=0;

[0041] In the classification loss function, the actual output label for each true value label y is y', so the classification loss L cls For:

[0042]

[0043] Where α is the balance factor;

[0044] In the target box regression function, the predicted rectangular target box is constructed according to the distance from the true value label y of each positive sample to the four sides, the intersection and union ratio of the predicted target box and the true target box is calculated, and on this basis, the target box regression loss L is further calculated. reg :

[0045]

[0046] Where N pos Represents the number of true value labels y of positive samples, R pi Represents the rectangular box predicted at the current i-th point, R g represents the true target box, Represents the intersection-over-union ratio of the predicted target box and the true target box;

[0047] In the centrality loss, the loss between the predicted centrality of each true value label y and the true centrality of each true value label y is calculated in the form of binary cross entropy, where the true centrality is calculated as:

[0048]

[0049] In the above formula, r g , l g ,t g 、b g They represent the distance of the pixel corresponding to the current true value label y from the right side of the target box, the left side of the target box, the top of the target box, and the bottom of the target box respectively;

[0050] The final loss function L is:

[0051]

[0052] Among them, λ1 and λ2 are error weight parameters, and λ1>0 and λ2>0, and the entire neural network is trained by the error back propagation algorithm.

[0053] Furthermore, the specific method of detecting the corresponding commodity in the newly input image based on the trained single-sample learning-based lightweight convolutional neural network model according to the support image provided by the user and marking it in the form of a rectangular frame is as follows:

[0054] The classification head outputs a confidence map, in which the centrality value corresponding to each point is multiplied point by point to enhance the confidence value of the point falling within the target box;

[0055] Find points on the enhanced confidence map whose confidence is greater than a specific threshold T;

[0056] Find the distance (l, r, t, b) from the four sides of the candidate target box corresponding to the point whose confidence is greater than a specific threshold T, thereby determining the target box of the product.

[0057] Compared with the prior art, the present invention has the following beneficial effects:

[0058] A lightweight convolutional neural network model based on single-sample learning is used to meet the inventory needs of shelf goods in real life, allowing users to complete the inventory task with just one product image.

[0059] The network layer is selected from the lightweight convolutional neural network MobileNetV3 to construct the backbone network of the lightweight convolutional neural network model based on single sample learning, which ensures the feature's ability to characterize the target object and the efficiency of feature extraction. In particular, the activation function in MobileNetV3 is improved. Compared with the h-swish activation function used in MobileNetV3, the activation function improved by the present invention expands the representation of the input feature value interval, and its degree of fit to the Swish activation function is higher than that of the h-swish activation function. In addition, due to the use of the shift algorithm, the improved Swish activation function has high computational efficiency.

[0060] For the single-sample-based commodity visual inspection task, the backbone network, fusion network and detection head were integrated and optimized, reducing the number of parameters of the entire lightweight convolutional neural network model based on single-sample learning, so that it can be deployed on computers with low computing resources. BRIEF DESCRIPTION OF THE DRAWINGS

[0061] Figure 1 It is a schematic diagram of the structure of the lightweight convolutional neural network model based on single sample learning described in the present invention. DETAILED DESCRIPTION

[0062] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0063] The technical solution adopted by the present invention is as follows: a visual inventory method of shelf commodities using a single-sample learning lightweight neural network, comprising:

[0064] A lightweight convolutional neural network model based on single-sample learning is constructed, which consists of three parts: backbone network, feature fusion and detection head.

[0065] The TrackingNet dataset is introduced to construct a dataset suitable for single-sample learning, providing training data for the lightweight convolutional neural network model based on single-sample learning.

[0066] Set the loss function for the lightweight convolutional neural network model based on single-sample learning. The loss function includes classification loss function, target box regression function and center loss function.

[0067] Based on the trained single-sample learning-based lightweight convolutional neural network model, the corresponding products are detected in the new input image according to the support images provided by the user and marked in the form of a rectangular box.

[0068] In order to perform visual inspection on shelf merchandise, such as Figure 1 As shown in the figure, a lightweight convolutional neural network model based on single-sample learning is constructed. The neural network model consists of three modules: backbone network, feature fusion and detection head.

[0069] The structure of the lightweight convolutional neural network model based on single-sample learning is a twin form, which has two input branches. The first branch is used to receive the image of the goods to be counted provided by the user, and the second branch is used to receive the image to be detected. The images received by the two input branches are both color images. The purpose of the entire visual inspection model is to detect the goods to be counted from the image to be detected, so as to form the goods counting result.

[0070] In the backbone network design of the lightweight convolutional neural network model based on single-sample learning, some neural network layers of MobileNetV3 are selected to build the backbone network. The basic components of MobileNetV3 are composed of 1 convolution layer, 1 batch normalization layer, a rectified linear activation function layer, 1 convolution layer, 1 batch normalization layer, 1 compression and excitation network layer, activation function layer, 1 convolution layer, and 1 batch normalization layer stacked in sequence.

[0071] The first convolution layer and the first 10 basic components of MobileNetV3 are cascaded to form the backbone network. The convolution kernel size, span, boundary filling and activation function layer type of the 10 basic components are different. The compression and excitation network layers are embedded in the second and fifth to eleventh basic components of the backbone network. In addition, the modified linear activation function ReLU function is used as the activation function type for the second to fourth basic components. The improved Swish activation function is used in other modules, which replaces the h-swish activation function used in the original MobileNetV3 and enhances the nonlinear representation capability.

[0072] Among them, the specific form of the improved Swish activation function is:

[0073]

[0074] In the above formula, x represents the input eigenvalue, and >> represents the right shift operator.

[0075] The specific parameters set for each layer of the backbone network are shown in Table 1. The network parameters use the public pre-trained MobileNetV3 model parameters. During the training process, the parameters of the backbone network are frozen and do not participate in the calculation and reverse propagation of the error gradient. The support image and the image to be detected are respectively used to obtain feature maps through the backbone network. The parameters of the backbone network are the same, and the final network output is a feature map with 96 channels and a size of 7×7.

[0076] Table 1. Backbone network settings

[0077] Module Type Activation function type Is there a compression and incentive network layer? Convolutional layer (1) Improved Swish activation function no Basic components (2) ReLU have Basic components (3) ReLU no Basic components (4) ReLU no Basic components (5) Improved Swish activation function have Basic components (6) Improved Swish activation function have Basic components (7) Improved Swish activation function have Basic components (8) Improved Swish activation function have Basic components (9) Improved Swish activation function have Basic components (10) Improved Swish activation function have Basic components (11) Improved Swish activation function have

[0078] Feature fusion mainly consists of point-level correlation fusion module, spatial channel fusion module and inverse activation module.

[0079] Among them, the point-level correlation fusion module is responsible for multiplying the feature map of the support image and the image to be detected in matrix form to obtain the correlation map, which is further sent to the compression and excitation network layer to enhance the neural network's attention to important channels. The compression and excitation network first pools the correlation map into a single output value through an adaptive average pooling layer, then reduces the dimension through a fully connected layer, and performs a nonlinear transformation by a linear rectification function, and then increases the dimension through a fully connected layer, and performs a nonlinear transformation by a linear rectification function. Finally, the weight coefficient of each channel is output through the Sigmod function, which is used to perform weighted calculations on different channels of the correlation map itself.

[0080] The spatial channel fusion module contains 1 convolutional neural network layer, 1 batch normalization layer, 1 convolutional neural network layer, and 1 batch normalization layer.

[0081] The inverse activation module consists of 1 convolutional neural network layer, 1 batch normalization layer, 1 convolutional neural network layer, 1 Gaussian error linear unit layer, 1 convolutional neural network layer, 1 batch normalization layer, and 1 convolutional neural network layer.

[0082] The detection head includes the classification head, the target box regression head and the centrality measurement.

[0083] The classification head contains 4 convolution blocks and 1 convolution classification layer. Each convolution block consists of 1 convolutional neural network layer, 1 group normalization layer, and 1 linear rectifier function layer. The last convolution classification layer has 1 output channel, 1 convolution span, and 3×3 convolution kernel size, which is used to estimate the confidence of each position point as the target object to be searched. The confidence map is finally input into the Sigmod function so that the final confidence value is compressed in the interval [0,1].

[0084] The target box regression head contains 4 convolution blocks and 1 convolution regression layer. Each convolution block consists of 1 convolutional neural network layer, 1 group normalization layer, and 1 linear rectifier function layer. The network parameter settings of the convolution block are the same as those of the convolution block in the classification head. The last convolution regression layer has 4 output channels, a convolution span of 1, and a convolution kernel size of 3×3. The output images of the 4 channels represent the distance between the corresponding pixel of each point on the input image and the four sides of the rectangular target box.

[0085] The centrality measure of the target box consists of one convolutional layer, which receives the feature maps output by the four convolutional blocks in the classification head and finally outputs a centrality map, which represents the confidence that the current position is the center of the target.

[0086] In order to be able to use the model for visual inspection of goods on shelves, it is necessary to train it on a calibrated image dataset and design a loss function for the training of the entire neural network.

[0087] In terms of preparation of training data sets, the present invention introduces the TrackingNet data set to construct a data set suitable for single-sample learning.

[0088] The TrackingNet dataset contains 30,643 short video clips, each of which is labeled with whether the target is visible and the spatial position of the target frame. Two frames of video are randomly selected from each video clip to form a pair of target images to be detected and support images, where the support image is the image area containing the target object cropped from the frame image according to the labeled target frame. The support image and the image to be detected are used as the input of the neural network, and the size of the support image and the image to be detected are uniformly scaled to 224×224.

[0089] When using the TrackingNet dataset to train a lightweight convolutional neural network model based on single-sample learning, a loss function is also set to control the training effect. In terms of loss function setting, a classification loss function, a target box regression function, and a center loss function are set.

[0090] In the classification loss function, the network classification output is a 7×7×1 confidence map, and each point on the confidence map is mapped to a pixel on the 224×224 input image. For the construction of the true value label y of each point, if its corresponding pixel falls within the target box, the point is considered to be a positive sample y=1, otherwise it is considered to be a negative sample y=0. The actual output label of the entire network for each true value label y is y', so the classification loss L cls For:

[0091]

[0092] In the above formula, the specific value of the balance factor α is 0.25.

[0093] In terms of the target box regression function setting, each true value label y on the confidence map is mapped to a pixel point on the 224×224 input image. The point whose corresponding pixel point falls within the target box is considered a positive sample. The predicted rectangular target box is constructed according to the distance from each positive sample pixel point to the four sides, and the intersection and union ratio of the predicted target box and the true target box is calculated. On this basis, the target box regression loss L is further calculated. reg :

[0094]

[0095] In the above formula, N pos Represents the number of true value labels y of positive samples, Represents the rectangular box predicted at the current i-th point, R g Represents the true target box. Represents the intersection-over-union ratio of the predicted target box and the true target box.

[0096] In terms of centrality loss, the loss between the predicted centrality of each true value label y and the true centrality of each true value label y is calculated in the form of binary cross entropy. The calculation of the true centrality is:

[0097]

[0098] In the above formula, r g , l g ,t g 、b g They respectively represent the distance of the pixel corresponding to the current true value label y from the right side of the target box, the left side of the target box, the top of the target box, and the bottom of the target box.

[0099] The final loss function L is:

[0100]

[0101] Among them, λ1 and λ2 are error weight parameters, and λ1>0 and λ2>0, and the entire neural network is trained by the error back propagation algorithm.

[0102] After the training of the lightweight convolutional neural network model based on single-sample learning is completed, the lightweight convolutional neural network model based on single-sample learning is used for practical application.

[0103] Based on the trained lightweight convolutional neural network model based on single-sample learning, the corresponding products are detected in the new input image according to the support image provided by the user and marked in the form of a rectangular box. The confidence map output by the classification head is multiplied by the centrality value corresponding to each point to enhance the confidence value of the points falling within the target box, and further search for points with confidence greater than a specific threshold T on the enhanced confidence map. These points are considered to fall within the candidate target box, and the distances (l, r, t, b) corresponding to these points from the four sides of the candidate target box are found, thereby determining the target box of the product.

[0104] During the commodity inventory process, the user provides an existing picture of the commodity to be counted, which is input as a supporting image into the lightweight convolutional neural network model based on single-sample learning. In addition, a picture of a shelf with commodities stored is taken, which is input as an image to be detected into the lightweight convolutional neural network model based on single-sample learning. Finally, the target box of the commodity to be counted on the picture of the shelf with commodities stored is output as the inventory result.

[0105] Although the present invention has been described in detail with reference to the aforementioned embodiments, it is still possible for those skilled in the art to modify the technical solutions described in the aforementioned embodiments, or to make equivalent substitutions for some of the technical features therein. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present invention should be included in the protection scope of the present invention.

Claims

1. A visual inventory method for shelf commodities using a single-sample learning lightweight neural network, characterized in that: Included are: Build a lightweight convolutional neural network model based on single-sample learning, which consists of three parts: backbone network, feature fusion and detection head; The TrackingNet dataset is introduced to construct a dataset suitable for single-sample learning, providing training data for the lightweight convolutional neural network model based on single-sample learning; Set the loss function for the lightweight convolutional neural network model based on single-sample learning. The loss function includes classification loss function, target box regression function and center loss function. Based on the trained single-sample learning-based lightweight convolutional neural network model, the corresponding products are detected in the new input image according to the support images provided by the user and marked in the form of a rectangular box.

2. The method for visual inventory of shelf commodities using a single-sample learning lightweight neural network according to claim 1 is characterized in that: The construction of a lightweight convolutional neural network model based on single-sample learning includes: Select some neural network layers of MobileNetV3 to build the backbone network; MobileNetV3 consists of 1 convolution layer, 1 batch normalization layer, a rectified linear activation function layer, 1 convolution layer, 1 batch normalization layer, 1 compression and excitation network layer, activation function layer, 1 convolution layer, and 1 batch normalization layer stacked in sequence; The first convolution layer and the first 10 basic components of MobileNetV3 are cascaded to form a backbone network, where the convolution kernel size, span, boundary padding, and activation function layer type of the 10 basic components are different; The compression and excitation network layers are embedded in the 2nd and 5th to 11th layer infrastructure of the backbone network; The basic components of the outer 2nd to 4th layers use the modified linear activation function ReLU function as the type of activation function, and the modules of other layers use the improved Swish activation function. The specific formula of the improved Swish activation function is: Where x represents the input eigenvalue and >> represents the right shift operator.

3. The method for visual inventory of shelf commodities using a single-sample learning lightweight neural network according to claim 2 is characterized in that: The construction of a lightweight convolutional neural network model based on single sample learning also includes: Feature fusion includes point-level correlation fusion module, spatial channel fusion module and inverse activation module; Among them, the point-level correlation fusion module is responsible for multiplying the feature map of the support image and the image to be detected in matrix form to obtain the correlation map, which is sent to the compression and excitation network layer to enhance the neural network's attention to important channels; The compression and excitation network first pools the correlation graph into a single output value through an adaptive average pooling layer, reduces the dimension through a fully connected layer, and performs a nonlinear transformation through a linear rectification function, then increases the dimension through a fully connected layer, and performs a nonlinear transformation through a linear rectification function, and finally outputs the weight coefficient of each channel through the Sigmod function, which is used to perform weighted calculations on different channels of the correlation graph itself; The spatial channel fusion module contains 1 convolutional neural network layer, 1 batch normalization layer, 1 convolutional neural network layer, and 1 batch normalization layer; The inverse activation module consists of 1 convolutional neural network layer, 1 batch normalization layer, 1 convolutional neural network layer, 1 Gaussian error linear unit layer, 1 convolutional neural network layer, 1 batch normalization layer, and 1 convolutional neural network layer.

4. The method for visual inventory of shelf commodities using a single-sample learning lightweight neural network according to claim 3 is characterized in that: The construction of a lightweight convolutional neural network model based on single sample learning also includes: The detection head includes the classification head, the target box regression head and the centrality measurement; Among them, the classification head contains 4 convolution blocks and 1 convolution classification layer. Each convolution block consists of 1 convolutional neural network layer, 1 group normalization layer, and 1 linear rectification function layer. The last convolution classification layer has an output channel number of 1, a convolution span of 1, and a convolution kernel size of 3×3, which is used to estimate the confidence that each position point is the target object to be searched. The confidence map is finally input into the Sigmod function so that the final confidence value is compressed in the interval [0,1]; The frame regression head contains 4 convolution blocks and 1 convolution regression layer. Each convolution block consists of 1 convolutional neural network layer, 1 group normalization layer, and 1 linear rectification function layer. The network parameter settings of the convolution block are the same as those of the convolution block in the classification head; The last convolutional regression layer has 4 output channels, a convolution stride of 1, and a convolution kernel size of 3×3. The output images of the four channels represent the distance between the corresponding pixel of each point on the input image and the four sides of the rectangular target box. The centrality measurement of the target box consists of one convolutional layer, which receives the feature maps output by the four convolutional blocks in the classification head and finally outputs a centrality map, which represents the confidence that the current position is the center of the target.

5. The method for visual inventory of shelf commodities using a single-sample learning lightweight neural network according to claim 1 is characterized in that: The specific method of introducing the TrackingNet dataset to construct a dataset suitable for single-sample learning and providing training data for a lightweight convolutional neural network model based on single-sample learning is as follows: The TrackingNet dataset contains several short video clips. Each video clip is labeled with whether the target is visible and the spatial position of the target frame. Two frames of video are randomly extracted from each video clip to form a pair of target images to be detected and support images. The support image is obtained by cropping the image region containing the target object in the frame image according to the marked target frame; The support image and the image to be detected are used as the input of the lightweight convolutional neural network model based on single-sample learning. The size of the support image and the size of the image to be detected are uniformly scaled to 224×224.

6. The method for visual inventory of shelf commodities using a single-sample learning lightweight neural network according to claim 1 is characterized in that: The specific method for setting the loss function for the lightweight convolutional neural network model based on single-sample learning is: The lightweight convolutional neural network model based on single-sample learning outputs a 7×7×1 confidence map, mapping each point on the confidence map to a pixel point on the 224×224 input image; For each point, the true value label y is constructed. If the pixel corresponding to the true value label y falls within the target box, the point is considered to be a positive sample y=1, otherwise the true value label y is considered to be a negative sample y=0; In the classification loss function, the actual output label for each true value label y is y', so the classification loss L cls For: Where α is the balance factor; In the target box regression function, the predicted rectangular target box is constructed according to the distance from the true value label y of each positive sample to the four sides, the intersection and union ratio of the predicted target box and the true target box is calculated, and on this basis, the target box regression loss L is further calculated. reg : Where N pos Represents the number of true value labels y of positive samples, Represents the rectangular box predicted at the current i-th point, R g represents the true target box, Represents the intersection-over-union ratio of the predicted target box and the true target box; In the centrality loss, the loss between the predicted centrality of each true value label y and the true centrality of each true value label y is calculated in the form of binary cross entropy, where the true centrality is calculated as: In the above formula, r g , l g ,t g 、b g They represent the distance of the pixel corresponding to the current true value label y from the right side of the target box, the left side of the target box, the top of the target box, and the bottom of the target box respectively; The final loss function L is: Among them, λ1 and λ2 are error weight parameters, and λ1>0 and λ2>0, and the entire neural network is trained by the error back propagation algorithm.

7. The method for visual inventory of shelf commodities using a single-sample learning lightweight neural network according to claim 1 is characterized in that: The specific method of detecting the corresponding product in the new input image based on the support image provided by the user and marking it in the form of a rectangular frame based on the trained single-sample learning lightweight convolutional neural network model is as follows: The classification head outputs a confidence map, in which the centrality value corresponding to each point is multiplied point by point to enhance the confidence value of the point falling within the target box; Find points on the enhanced confidence map whose confidence is greater than a specific threshold T; Find the distance (l, r, t, b) from the four sides of the candidate target box corresponding to the point whose confidence is greater than a specific threshold T, thereby determining the target box of the product.

Images