A display fullness detection method and system

Abstract

The display fullness detection method and system provided by the present invention is to extract multiple image samples from the original video stream after screening; annotate the multiple image samples to obtain annotated image samples; divide the annotated image samples into a training set and a test set; input the image-enhanced training set into the constructed initial display fullness detection model for training to obtain a trained display fullness detection model; input the test set into the trained display fullness detection model to obtain the display fullness of each shelf area; judge whether the display fullness of each shelf area is less than a preset threshold at the current time, and if so, judge whether the display fullness of each shelf area is still less than the preset threshold within a preset time period, and if so, generate an alarm message. It can effectively reduce the reliance on manual labor, thereby reducing the cost and subjectivity of manual store inspections, improving the efficiency of store management, and reducing overall operating costs.

Description

Technical Field

[0001] The present invention relates to the technical field of image recognition, and in particular to a display fullness detection method and system. Background Art

[0002] At present, the national chain stores in the fast-moving consumer goods industry have expanded significantly, with tens of thousands of stores. The existing store management generally combines online and offline manual store inspections to manage stores. However, this management method has the following problems: With the increase in the number of stores, manual store inspections require a lot of manpower, time and travel costs, which is a considerable expense for enterprises; manual store inspections may not be able to achieve real-time monitoring, and the sample size of spot checks is limited, resulting in untimely discovery and resolution of problems, affecting store operation efficiency; manual store inspections often rely on the subjective judgment of store inspectors, and there may be problems such as inconsistent standards and inadequate implementation, resulting in uneven management results; manual store inspections are difficult to achieve comprehensive and objective data collection and analysis, thus affecting the accurate evaluation of store operations and decision support. Manual store inspections may involve cheating, such as reshooting photos, which will affect the authenticity and reliability of store inspection results. It is difficult to uniformly manage personnel in stores across the country, and it is difficult to adapt business strategies to local conditions, resulting in rising operating costs.

[0003] Therefore, it is an urgent problem to be solved by technical personnel in this field to provide a display fullness detection method and system that can effectively solve the above problems and improve store management efficiency. Summary of the invention

[0004] The purpose of the present invention is to provide a display fullness detection method, which has clear logic, is safe, effective, reliable and easy to operate, and can effectively reduce dependence on manual labor, thereby reducing the cost and subjectivity of manual store inspections, improving the efficiency of store management, and reducing overall operating costs.

[0005] Based on the above objectives, the technical solution provided by the present invention is as follows:

[0006] A display fullness detection method comprises the following steps:

[0007] Extracting multiple image samples from the filtered original video stream, and performing labeling processing on the multiple image samples to obtain labeled image samples;

[0008] Dividing the labeled image samples into a training set and a test set;

[0009] The image-enhanced training set is input into the initial display fullness detection model for training to obtain a trained display fullness detection model;

[0010] Inputting the test set into the trained display fullness detection model to obtain the display fullness of each shelf area;

[0011] At the current time, determine whether the display fullness of each shelf area is less than a preset threshold;

[0012] If yes, within a preset time period, determining whether the display fullness of each shelf area is still less than the preset threshold;

[0013] If so, generate an alarm message.

[0014] Preferably, extracting a plurality of image samples from the filtered original video stream comprises the following steps:

[0015] Acquire a plurality of the original video streams of preset locations of the store;

[0016] Determining whether the image sample exists in the original video stream;

[0017] If so, the original video stream is extracted by fixed-frequency frame sampling, and multiple image samples are obtained after review.

[0018] Preferably, the step of labeling the plurality of image samples to obtain labeled image samples comprises the following steps:

[0019] Marking the plurality of image samples as incomplete according to preset reference objects and marking the plurality of image samples as empty surfaces according to preset rules, and integrating them to obtain marked image samples;

[0020] The step of marking the empty surfaces of the plurality of image samples according to a preset rule comprises the following steps:

[0021] Label single countertop samples according to individual product columns;

[0022] According to the single-table sample annotation, the pixel segment in the short side area with the highest statistical frequency is used as the minimum detection frame;

[0023] When the width of the empty frame corresponding to the empty countertop is greater than a preset length, the image sample containing the entire display empty countertop is labeled according to the minimum detection frame to obtain an image sample after the empty countertop is labeled.

[0024] Preferably, obtaining the training set after image enhancement is specifically:

[0025] The training set is subjected to corresponding image enhancement operations of horizontal flipping, scaling, and cropping to obtain an image-enhanced training set.

[0026] Preferably, the step of inputting the image-enhanced training set into the initial display fullness detection model for training to obtain the trained display fullness detection model comprises the following steps:

[0027] Build an initial display fullness detection model based on the YOLO model framework;

[0028] According to the enhanced training set, the initial display fullness detection model is iteratively trained according to the weighted sum of classification loss, bounding box loss and DFL loss as the loss function until the model converges to obtain a trained display fullness detection model.

[0029] Preferably, after generating the warning information, the following steps are also included:

[0030] Adding the alarm information to the alarm list;

[0031] Determine whether the number of alarm information in the alarm list is greater than a preset number;

[0032] If yes, determining whether the alarm information is a new alarm;

[0033] If so, update the alarm ID and output the alarm information.

[0034] Preferably, after determining whether the number of alarm information in the alarm list is greater than a preset number, the method further includes:

[0035] If not, clear the alarm list.

[0036] Preferably, after determining whether the alarm information is a new alarm, the method further includes:

[0037] If not, determine whether the time between two adjacent alarm messages meets the alarm interval;

[0038] If so, the warning information is output.

[0039] A display fullness detection system, comprising:

[0040] An image sample annotation module is used to extract multiple image samples from the filtered original video stream, and annotate the multiple image samples to obtain annotated image samples;

[0041] A classification module, used for dividing the labeled image samples into a training set and a test set;

[0042] A model training module, used for inputting the image-enhanced training set into the initial display fullness detection model for training, so as to obtain a trained display fullness detection model;

[0043] A display fullness module, used for inputting the test set into the trained display fullness detection model to obtain the display fullness of each shelf area;

[0044] A judgment module, used for judging whether the display fullness of each shelf area is less than a preset threshold at the current time;

[0045] The judging module is further configured to judge whether the display fullness of each shelf area is still less than the preset threshold within a preset time period;

[0046] The alarm module is used to generate an alarm message if so.

[0047] The display fullness detection method provided by the present invention is to extract multiple image samples from the original video stream after screening; annotate the multiple image samples to obtain annotated image samples; divide the annotated image samples into a training set and a test set; input the image-enhanced training set into the constructed initial display fullness detection model for training to obtain a trained display fullness detection model; input the test set into the trained display fullness detection model to obtain the display fullness of each shelf area; judge whether the display fullness of each shelf area is less than a preset threshold at the current time, and if so, further judge whether the display fullness of each shelf area is still less than the preset threshold within a preset time period, and if so, generate an alarm message.

[0048] Compared with the existing technology, this case uses image processing, image recognition and incomplete display detection model to determine whether each shelf area in the image is incomplete, which can effectively reduce dependence on manual labor, thereby reducing the cost and subjectivity of manual store inspections, improving the efficiency of store management and reducing overall operating costs.

[0049] The present invention also discloses a display fullness detection system, which has the same technical concept as the method, solves the same technical problem, and should have the same beneficial effects, so it will not be described in detail here. BRIEF DESCRIPTION OF THE DRAWINGS

[0050] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0051] Figure 1 A flow chart of a display fullness detection method provided by an embodiment of the present invention;

[0052] Figure 2A flowchart of obtaining a filtered original video stream provided by an embodiment of the present invention;

[0053] Figure 3 A flow chart of labeling multiple image samples with empty surfaces provided by an embodiment of the present invention;

[0054] Figure 4 A flowchart of step S4 provided in an embodiment of the present invention;

[0055] Figure 5 A flowchart after step S8 provided for an embodiment of the present invention;

[0056] Figure 6 A schematic diagram of the structure of a display fullness detection system provided in an embodiment of the present invention. DETAILED DESCRIPTION

[0057] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0058] The embodiments of the present invention are written in a progressive manner.

[0059] The embodiment of the present invention provides a display fullness detection method and system, which mainly solves the technical problems in the prior art that manual store inspections are high in cost, highly subjective, and result in difficulties in store management and low efficiency.

[0060] like Figure 1 As shown, a display fullness detection method includes the following steps:

[0061] S1. extracting multiple image samples from the filtered original video stream, and annotating the multiple image samples to obtain annotated image samples;

[0062] S2. Divide the labeled image samples into a training set and a test set;

[0063] S3. Inputting the image-enhanced training set into the initial display fullness detection model for training to obtain a trained display fullness detection model;

[0064] S4. Input the test set into the trained display fullness detection model to obtain the display fullness of each shelf area;

[0065] S5. Determine whether the display fullness of each shelf area is less than a preset threshold at the current time;

[0066] S6. If yes, within the preset time period, determine whether the display fullness of each shelf area is still less than the preset threshold;

[0067] S7. If yes, generate an alarm message.

[0068] In step S1, the original video stream is called from the store, and after filtering the original video stream, multiple image samples are extracted from it; the multiple image samples are annotated, specifically, the placement status of the goods in the shelf area is annotated to obtain the annotated image samples;

[0069] It should be noted that with the rapid development of artificial intelligence technology, image annotation, as an important part of data annotation, has become increasingly important. Image annotation is not only related to the accuracy and effectiveness of data, but also directly affects the training effect of computer vision models;

[0070] In this embodiment, rectangular frame annotation is used. Rectangular frame annotation is the most widely used image annotation method. It draws a rectangular frame in the image to frame the specified target object. It is simple and fast and widely used in the fields of autonomous driving, security monitoring, etc. Rectangular frame annotation can quickly mark the main object in the image and provide basic data support for subsequent target detection, tracking and other tasks;

[0071] In step S2, the labeled samples are classified according to a ratio of 4:1 to obtain a training set and a test set;

[0072] In step S3, an initial display fullness detection model is created, and the training set is subjected to image enhancement and then input into the detection model for training to obtain a trained display fullness detection model;

[0073] In step S4, the display fullness detection model after training is tested by a test set to see whether it has reached the optimal level, and the display fullness of each corresponding shelf area is obtained;

[0074] In step S5, at the current time, it is determined whether the display fullness of each shelf area exceeds a certain proportion;

[0075] In this embodiment, the threshold ratio is set to 80%;

[0076] In step S6, due to the existence of manual replenishment, in order to make the detection more accurate, when the display fullness of each shelf area at a certain moment does not exceed the preset threshold, it is determined whether the display fullness of each shelf area has exceeded the preset threshold within a preset time period;

[0077] In this embodiment, the preset time period can be set to 10 minutes, that is, assuming that at 8 o'clock, the display fullness of each shelf area does not exceed 80%, and there are a large number of empty spaces without goods placed. At this time, it is recorded and determined in real time whether the display fullness of each shelf area is still less than 80% during the time period from 8 o'clock to 8:10;

[0078] In step S7, if the price is still less than the preset threshold, an alarm message is generated to remind the store clerk to replenish the stock.

[0079] like Figure 2 As shown, preferably, extracting multiple image samples from the filtered original video stream includes the following steps:

[0080] A1. Obtain multiple original video streams at preset locations of the store;

[0081] A2. Determine whether there is an image sample in the original video stream;

[0082] A3. If so, extract the original video stream by fixed-frequency frame sampling and obtain multiple image samples after review.

[0083] In step A1 to step A3, the original video stream captured by the camera installed at the preset position in each store is called, and the trained shelf area segmentation model is used to determine whether there is an image sample in the original video stream, that is, whether the core area (shelf area) is captured in the video stream. If the shelf area is not captured in the video stream, it is defined that there is no image sample, and the original video stream without image samples is filtered out, and the video stream with image samples is retained as the filtered original video stream;

[0084] Step A3, selecting a preset frequency to extract a specific number of frames in the video stream to obtain image sample frames, and selecting usable image samples from them through manual review or other review methods.

[0085] In this embodiment, the specific steps of obtaining the trained shelf area segmentation model are as follows: obtaining screenshots of all points (cameras) of 1,000 stores, each of which contains all shelf display information under the corresponding camera; using point images for area segmentation annotation, and training the shelf area segmentation model yolov8s-seg, with a model accuracy of 92%, for segmenting different shelf areas at each point. Apply the trained shelf area segmentation model yolov8s-seg to extract the shelf area information of each image, and save it to the point area configuration file .json for real-time detection algorithm to call.

[0086] like Figure 3 As shown, preferably, the multiple image samples are annotated to obtain annotated image samples, including the following steps:

[0087] Marking multiple image samples as incomplete according to preset reference objects and marking multiple image samples as empty surfaces according to preset rules, and integrating them to obtain marked image samples;

[0088] Among them, the empty table surface marking of multiple image samples according to preset rules includes the following steps:

[0089] B1. Label the single countertop samples according to the individual product columns;

[0090] B2. Based on the single-table sample annotation, the short-side area pixel segment with the highest statistical frequency is used as the minimum detection frame;

[0091] B3. When the width of the empty frame corresponding to the empty countertop is greater than the preset length, the image sample containing the entire display empty countertop is labeled according to the minimum detection frame to obtain the image sample after the empty countertop is labeled.

[0092] In actual application, the fullness samples of a preset single product column are used as references to mark multiple image samples as incomplete, and image samples after incomplete marking are obtained;

[0093] Incomplete annotations have reference objects to refer to, and products are used as clear boundaries. However, there are no reference objects for large empty countertops. If there are no boundary restrictions, many countertops are of different lengths, which requires more samples of different lengths to improve the recall rate of the model in various sizes. This is achieved through steps B1 to B3.

[0094] In step B1 to step B3, appropriate image samples are selected from multiple image samples according to the time period when empty counters are most likely to appear in the store; the empty counters are labeled according to single counters (single commodity column) through manual labeling or other labeling schemes to obtain single counter sample labels; according to the empty counter sample labels, statistical segmentation is performed according to the short edge according to the pixel segment to obtain the highest frequency pixel as a fixed minimum detection frame. When the width of the empty frame is greater than a certain length L, the minimum frame labeling is started, and the large empty counters are labeled according to the minimum frame. Finally, the empty counters that are less than a minimum are labeled with the real boundary.

[0095] In this embodiment, step B2 can be classified according to pixel segments such as 16-20, 21-25, 26-30, 31-35, 36-40, and the short side area segment with the highest frequency is counted. Assuming that the pixel segment with the highest frequency is 26-30, bar statistics are performed according to 26, 27, 28, 29, 30 to obtain the pixel with the highest frequency.

[0096] It should be noted that using 1,000 empty table data samples as the annotation objects, according to the blank random annotation, the number of empty table samples is only 1,250. According to the minimum sample annotation, the number of valid empty table samples increases to 6,280, and the number of samples increases by more than 4 times compared with before.

[0097] Preferably, the training set after image enhancement is obtained, specifically:

[0098] The training set is subjected to corresponding image enhancement operations of horizontal flipping, scaling, and cropping to obtain an image-enhanced training set.

[0099] In actual application, the labeled image samples in the training set are enhanced through image processing operations such as horizontal flipping, scaling, cropping, etc. to obtain the image enhanced training set.

[0100] like Figure 4 As shown, preferably, step S4 includes the following steps:

[0101] C1. Build the initial display fullness detection model based on the YOLO model framework;

[0102] C2. Based on the enhanced training set, the initial display fullness detection model is iteratively trained according to the weighted sum of classification loss, bounding box loss and DFL loss as the loss function until the model converges to obtain the trained display fullness detection model.

[0103] In step C1 to step C2, an initial display fullness detection model is constructed based on the YOLO model framework, and the enhanced training set is input into the model for training until the preset regression evaluation index (specifically, the recall rate) reaches the preset threshold, the model converges, and the trained display fullness detection model is obtained.

[0104] In this embodiment, the enhanced training set is used for training, the model parameters and the initial values ​​of the model are kept unchanged, the minimum box annotation model is used to end the training 15 epochs in advance, and the model accuracy is improved by 4%, and the recall rate is improved by 5%.

[0105] It should be noted that the YOLO model is a fast and accurate target detection algorithm, which is mainly used in real-time video surveillance, obstacle detection for self-driving cars, drone navigation, and sports event analysis. The YOLO model can complete the recognition and positioning of objects in an image with a single view, and has the advantages of fast speed, high accuracy, strong interpretability, and wide applicability. The basic principle of the YOLO model is to extract image features through convolutional layers, use multiple convolution kernels to perform sliding convolution operations on the image, and capture local features in the image such as edges, textures, colors, etc. With the stacking of convolutional layers, the model can extract more and more abstract and advanced features, which are critical for the recognition and positioning of targets.

[0106] The YOLO model is mainly composed of the following parts: ‌Convolutional layer‌: used to extract image features. ‌Pooling layer‌: used to reduce the resolution of the feature map, reduce the amount of calculation and the number of parameters. ‌Activation function‌: introduces nonlinear characteristics so that the model can learn complex functional relationships. ‌Neck network‌: responsible for fusing features at different levels to improve the detection ability of small targets. ‌Detection head‌: responsible for predicting the bounding box and category of the target.

[0107] It should also be noted that BCE (binary cross entropy) is used as the classification loss. Each category determines "whether it is of this type" and outputs the confidence. Simply put, BCE calculates the entropy of both this type and not this type, and adds them together, so that this loss function can still express whether the classification is correct or not when it is not of this type. The classification loss here is to sum up the binary cross entropy losses of N targets and then take the average. The formula is as follows:

[0108] .

[0109] Wise-IoU, a bounding box loss based on a dynamic non-monotonic focusing mechanism, uses "outlier degree" instead of IoU to evaluate the quality of anchor boxes, and provides a wise gradient gain allocation strategy. This strategy reduces the competitiveness of high-quality anchor boxes while also reducing the harmful gradients generated by low-quality examples. This allows WIoU to focus on anchor boxes of ordinary quality and improve the overall performance of the detector. The formula is as follows:

[0110] ;

[0111] ;

[0112] ;

[0113] ;

[0114] Where (x, y, W, H) represents the model prediction value of the center point and width and height of the box, (x gt , y gt ,W gt ,H gt ) are the true values ​​of the center point, width and height.

[0115] Distribution Focal Loss (DFL) was proposed in Generalized Focal Loss (GLF) to allow the network to quickly focus on the value near the label, making the probability density at the label as large as possible. The idea is to use the cross entropy function to optimize the probabilities of the two positions around the label y, so that the network distribution focuses on the label value. The formula is as follows:

[0116] .

[0117] In this embodiment, the classification loss, bounding box loss and DFL loss are weighted and summed according to coefficients of 0.5, 7.5 and 1.5, and then iterative training is performed until the model converges.

[0118] In this embodiment, 10,000 image samples are used, and the data set is divided into 5:1, 8,000 images are used as the training set, and 2,000 images are used as the test set. The model parameters are used to set the initial training parameters, and 8,000 images are trained. When the model is at epoch=128, the loss of the validation set no longer decreases, and the mAP no longer increases. The model is tested on the validation set, and the mAP of the validation set reaches 85.2%.

[0119] like Figure 5 As shown, preferably, after step S8, the following steps are also included:

[0120] D1. Add the alarm information to the alarm list;

[0121] D2. Determine whether the number of alarm information in the alarm list is greater than the preset number;

[0122] D3. If yes, determine whether the alarm information is a new alarm;

[0123] D4. If yes, update the alarm ID and output the alarm information.

[0124] In steps D1 to D4, considering that not every alarm message needs to be output, the alarm message is added to the alarm list after it is generated, and it is determined whether the number of alarm messages in the alarm list is greater than the preset number. For example, if the preset number is 10, if the number of alarm messages in the alarm list exceeds 10, it is further determined whether the alarm message is a new alarm. If so, the alarm ID is updated in the alarm list, and the alarm message is output at the same time.

[0125] Preferably, after D2, it further includes:

[0126] If not, clear the alarm list.

[0127] In actual use, if the alarm information in the alarm list does not exceed 10, the alarm list will be cleared.

[0128] Preferably, after D3, it also includes:

[0129] If not, determine whether the time between two adjacent alarm messages meets the alarm interval;

[0130] If so, output a warning message.

[0131] During actual application, if the alarm information is not a new alarm, it is determined whether the interval between two adjacent alarm information in the alarm list meets the alarm interval. If the alarm interval is met, the alarm information is output. If not, it returns to step D2.

[0132] like Figure 6 As shown, a display fullness detection system includes:

[0133] An image sample module is used to extract multiple image samples from the filtered original video stream;

[0134] A labeling module is used to label multiple image samples to obtain labeled image samples;

[0135] The classification module is used to divide the labeled image samples into training sets and test sets;

[0136] A model training module, used for inputting the image-enhanced training set into the initial display fullness detection model for training, so as to obtain a trained display fullness detection model;

[0137] The display fullness module is used to input the test set into the trained display fullness detection model to obtain the display fullness of each shelf area;

[0138] A judgment module is used to judge whether the display fullness of each shelf area is less than a preset threshold at the current time;

[0139] The judgment module is further used to judge whether the display fullness of each shelf area is still less than a preset threshold within a preset time period;

[0140] The alarm module is used to generate an alarm message if so.

[0141] During actual application, the present invention also discloses a display fullness detection system, which is provided with an image sample module, a labeling module, a classification module, a model training module, a display fullness module, a judgment module and an alarm module. The functions and beneficial effects of each module correspond to the various steps in the display fullness detection method, and will not be repeated again.

[0142] In the embodiments provided in the present application, it should be understood that the disclosed methods and systems can be implemented in other ways. The system embodiments described above are merely schematic. For example, the division of modules is only a logical function division. There may be other division methods in actual implementation, such as: multiple modules or components can be combined, or can be integrated into another system, or some features can be ignored, or not executed. In addition, the coupling, direct coupling, or communication connection between the components shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or modules, which can be electrical, mechanical or other forms.

[0143] In addition, all functional modules in the embodiments of the present invention may be integrated into one processor, or each module may be a separate device, or two or more modules may be integrated into one device; each functional module in the embodiments of the present invention may be implemented in the form of hardware or in the form of hardware plus software functional units.

[0144] A person of ordinary skill in the art can understand that all or part of the steps of implementing the above method embodiment can be completed by program instructions and related hardware. The aforementioned program instructions can be stored in a computer-readable storage medium. When the program instructions are executed, the steps of the above method embodiment are executed; and the aforementioned storage medium includes: a mobile storage device, a read-only memory (ROM), a magnetic disk or an optical disk, and other media that can store program codes.

[0145] It should be understood that the use of "system", "device", "unit" and / or "module" in this application is only a method for distinguishing different components, elements, parts, parts or assemblies at different levels. However, if other words can achieve the same purpose, the word can be replaced by other expressions.

[0146] As shown in this application and claims, unless the context clearly indicates an exception, the words "a", "an", "a kind" and / or "the" do not refer to the singular, but also include the plural. Generally speaking, the terms "include" and "comprise" only indicate the inclusion of clearly identified steps and elements, and these steps and elements do not constitute an exclusive list. The method or device may also include other steps or elements. The elements defined by the sentence "includes a..." do not exclude the existence of other identical elements in the process, method, commodity or device that includes the elements.

[0147] If a flow chart is used in the present application, the flow chart is used to illustrate the operations performed by the system according to the embodiment of the present application. It should be understood that the preceding or following operations are not necessarily performed accurately in order. On the contrary, each step can be processed in reverse order or simultaneously. At the same time, other operations can also be added to these processes, or a certain step or several steps of operations can be removed from these processes.

[0148] The above is a detailed introduction to a display fullness detection method and system provided by the present invention. The above description of the disclosed embodiments enables professionals and technicians in this field to implement or use the present invention. Various modifications to these embodiments will be obvious to professionals and technicians in this field, and the general principles defined in this article can be implemented in other embodiments without departing from the spirit or scope of the present invention. Therefore, the present invention will not be limited to the embodiments shown in this article, but will conform to the widest scope consistent with the principles and novel features disclosed in this article.

Claims

1. A display fullness detection method, characterized in that: The steps include: Extracting multiple image samples from the filtered original video stream, and performing labeling processing on the multiple image samples to obtain labeled image samples; Dividing the labeled image samples into a training set and a test set; The image-enhanced training set is input into the initial display fullness detection model for training to obtain a trained display fullness detection model; Inputting the test set into the trained display fullness detection model to obtain the display fullness of each shelf area; At the current time, determine whether the display fullness of each shelf area is less than a preset threshold; If yes, within a preset time period, determining whether the display fullness of each shelf area is still less than the preset threshold; If so, generate an alarm message; The step of inputting the image-enhanced training set into the initial display fullness detection model for training to obtain a trained display fullness detection model includes the following steps: Build an initial display fullness detection model based on the YOLO model framework; According to the enhanced training set, the initial display fullness detection model is iteratively trained according to the weighted sum of classification loss, bounding box loss and DFL loss as the loss function until the model converges to obtain a trained display fullness detection model.

2. The display fullness detection method according to claim 1, characterized in that: The step of extracting multiple image samples from the filtered original video stream includes the following steps: Acquire a plurality of the original video streams of preset locations of the store; Determining whether the image sample exists in the original video stream; If so, the original video stream is extracted by fixed-frequency frame sampling, and multiple image samples are obtained after review.

3. The display fullness detection method according to claim 1, characterized in that: The step of labeling the plurality of image samples to obtain labeled image samples comprises the following steps: Marking the plurality of image samples as incomplete according to preset reference objects and marking the plurality of image samples as empty surfaces according to preset rules, and integrating them to obtain marked image samples; The step of marking the empty surfaces of the plurality of image samples according to a preset rule comprises the following steps: Label single countertop samples according to individual product columns; According to the single-table sample annotation, the pixel segment in the short side area with the highest statistical frequency is used as the minimum detection frame; When the width of the empty frame corresponding to the empty countertop is greater than a preset length, the image sample containing the entire display empty countertop is labeled according to the minimum detection frame to obtain an image sample after the empty countertop is labeled.

4. The display fullness detection method according to claim 1, characterized in that: The training set after the image enhancement is obtained, specifically: The training set is subjected to corresponding image enhancement operations of horizontal flipping, scaling, and cropping to obtain an image-enhanced training set.

5. The display fullness detection method according to claim 1, characterized in that: After the alarm information is generated, the following steps are also included: Adding the alarm information to the alarm list; Determine whether the number of alarm information in the alarm list is greater than a preset number; If yes, determining whether the alarm information is a new alarm; If so, update the alarm ID and output the alarm information.

6. The display fullness detection method according to claim 5, characterized in that: After determining whether the number of alarm information in the alarm list is greater than a preset number, the method further includes: If not, clear the alarm list.

7. The display fullness detection method according to claim 6, characterized in that: After determining whether the alarm information is a new alarm, the method further includes: If not, determine whether the time between two adjacent alarm messages meets the alarm interval; If so, the warning information is output.

8. A display fullness detection system, characterized in that: include: An image sample annotation module is used to extract multiple image samples from the filtered original video stream, and annotate the multiple image samples to obtain annotated image samples; A classification module, used for dividing the labeled image samples into a training set and a test set; A model training module, used for inputting the image-enhanced training set into the initial display fullness detection model for training, so as to obtain a trained display fullness detection model; A display fullness module, used for inputting the test set into the trained display fullness detection model to obtain the display fullness of each shelf area; A judgment module, used to judge whether the display fullness of each shelf area is less than a preset threshold at the current time; The judging module is further configured to judge whether the display fullness of each shelf area is still less than the preset threshold within a preset time period; An alarm module is used to generate an alarm message if yes; The model training module is specifically used to build an initial display fullness detection model based on the YOLO model framework; according to the enhanced training set, the initial display fullness detection model is iteratively trained according to the weighted sum of classification loss, bounding box loss and DFL loss as the loss function until the model converges to obtain a trained display fullness detection model.

Images