A convenience store inventory behavior recognition method, storage medium and device

Abstract

The embodiment of the present invention provides a method, storage medium and device for identifying inventory counting behavior in a convenience store, and belongs to the field of computer technology. The method includes: obtaining the current frame of the convenience store's surveillance video; performing pedestrian detection and information comparison on the current frame to obtain pedestrian data of each pedestrian in the current frame; tracking the trajectory of each pedestrian based on the pedestrian data to obtain the trajectory of each pedestrian; and performing a first processing on the trajectory of each pedestrian, including: detecting inventory features in all image frames corresponding to the trajectory, and ending the current processing if the number of times the inventory features are detected is less than a preset number of detections; and comparing the trajectory with a preset standard commodity inventory trajectory to obtain trajectory similarity, and if the trajectory similarity is greater than a preset threshold, it is determined that the inventory work is completed. This method is used to automatically identify inventory behavior to avoid terminal stores from falsely reporting or concealing inventory frequency and content.

Description

Technical Field

[0001] The present invention relates to the field of computer technology, and in particular to a convenience store inventory counting behavior recognition method, storage medium, and device. Background Art

[0002] Inventory is a crucial step in checking and counting a company's inventory. It's a crucial tool for ensuring consistency between accounts and actual inventory and strengthening merchandise management. Therefore, wholesale inventory must be fully counted quarterly, while retail inventory must be checked and counted monthly. Inventory can help determine inventory quantity, whether the variety and specifications are consistent, whether the product quality is intact, whether there are sufficient reserves, any backlogs, expiration, damage, mildew, insect infestation, or rodent infestation, and whether the account balances match the actual inventory. Any discrepancies identified through re-inventory verification should be adjusted to ensure consistency between accounts and actual inventory.

[0003] Due to the large variety and quantity of goods, coupled with the need to ensure accurate data during inventory counting, inventory counting typically consumes a significant amount of staff time and effort. Existing inventory-related technologies primarily focus on automated inventory counting, such as adding RFID tags to items, installing millimeter-wave radar terminals on shelves, and using mobile inventory counting robots. However, due to the unique nature of convenience store products and cost factors, inventory counting in convenience stores is currently primarily done manually.

[0004] Although corporate management departments require stores to conduct regular inventory, terminal stores often falsely report or conceal the frequency and content of inventory due to heavy workload and lack of staff. This is a common problem faced by many retail and warehousing companies. Summary of the Invention

[0005] The purpose of the embodiments of the present invention is to provide a convenience store inventory counting behavior identification method, storage medium and device for automatically identifying inventory counting behavior to prevent terminal stores from falsely reporting or concealing inventory counting frequency and content.

[0006] In order to achieve the above-mentioned purpose, the first aspect of an embodiment of the present invention provides a convenience store inventory behavior recognition method, the method comprising: obtaining a current frame of the surveillance video of the convenience store; performing pedestrian detection and information comparison on the current frame to obtain pedestrian data of each pedestrian in the current frame; based on the pedestrian data, tracking the trajectory of each pedestrian to obtain the trajectory of each pedestrian; and performing a first processing on the trajectory of each pedestrian, including: detecting inventory features in all image frames corresponding to the trajectory, if the number of times the inventory features are detected is less than a preset number of detections, ending this processing; and comparing the trajectory with a preset standard commodity inventory trajectory to obtain trajectory similarity, if the trajectory similarity is greater than a preset threshold, determining that the inventory work is completed.

[0007] Optionally, the method further includes: detecting whether the inventory work is completed within a specified time, and issuing a warning if the inventory work is not completed.

[0008] Optionally, the method further includes: after obtaining the trajectory of each of the pedestrians, determining whether the trajectory contains a complete trajectory; if it contains a complete trajectory, performing the first processing on the complete trajectory; otherwise, not performing the first processing; wherein, when a certain pedestrian cannot be detected in a consecutive preset number of image frames, the trajectory corresponding to the pedestrian is determined to be a complete trajectory.

[0009] Optionally, the first processing also includes: before detecting the inventory features in all image frames corresponding to the trajectory, classifying the pedestrians corresponding to the trajectory as employees or customers. If the classification result is a customer, the first processing is terminated.

[0010] Optionally, classifying the pedestrian corresponding to the trajectory includes: obtaining all image frames of the surveillance video corresponding to the trajectory of the pedestrian; obtaining image blocks of the pedestrian in all the image frames, and labeling the image blocks as p1, p2, p3, ..., p N ; inputting the image blocks into the pre-trained yolov8 personnel classification model in sequence to obtain the classification category of the pedestrian contained in each image block, wherein the classification category includes employee and customer; counting the number of employees and the number of customers in the N image blocks of the pedestrian; and comparing the number of employees with the number of customers. If the number of customers is greater than the number of employees, classify the pedestrian as a customer; otherwise, classify the pedestrian as an employee, wherein the position of the image block in the image frame is obtained by the pedestrian detection.

[0011] Optionally, pedestrian detection and information comparison are performed on the current frame to obtain pedestrian data of each pedestrian in the current frame, including: pedestrian detection is performed on the current frame to obtain appearance information and position information of all people in the current frame; each pedestrian is compared with pedestrians that have appeared before the current frame, including: comparing the appearance information of each pedestrian with the appearance information of pedestrians that have appeared before the current frame to obtain appearance information similarity, comparing the position information of each pedestrian with the position information of pedestrians that have appeared before the current frame to obtain position information similarity; and if the appearance information similarity is not less than a preset first threshold and the position information similarity is not less than a preset second threshold, then the pedestrian is considered to be the same person as the pedestrian that appeared before the compared current frame, and the pedestrian in the current frame is assigned the same ID as the compared pedestrian; otherwise, the pedestrian is considered to be a newly appeared pedestrian, and the pedestrian in the current frame is assigned a new ID, wherein the appearance information is the feature vector of the image block corresponding to the pedestrian, and the position information is the coordinates of the preset position of the image block corresponding to the pedestrian.

[0012] Optionally, the pedestrian detection and information comparison performed on the current frame to obtain pedestrian data of each pedestrian in the current frame: comparing each pedestrian with pedestrians that have appeared before the current frame also includes: comparing the position information of each pedestrian with the predicted trajectory information of pedestrians that have appeared before the current frame to obtain the similarity of the predicted information; if the appearance information similarity is not less than a preset first threshold, the position information similarity is not less than a preset second threshold, and the predicted information similarity is not less than a preset third threshold, then the pedestrian is considered to be the same person as the pedestrian that appeared before the compared current frame, and the pedestrian in the current frame is assigned the same ID as the compared pedestrian; otherwise, the pedestrian is considered to be a newly appeared pedestrian, and the pedestrian in the current frame is assigned a new ID, wherein the predicted trajectory information is predicted by the position information of pedestrians that have appeared before the current frame.

[0013] Optionally, based on the pedestrian data, the trajectory of each pedestrian is tracked to obtain the trajectory of each pedestrian, including: if the pedestrian is a newly appeared pedestrian, a new trajectory is created for the pedestrian; if the pedestrian is a pedestrian that has appeared before the current frame, the position information of the pedestrian is added to the end of the trajectory of the pedestrian corresponding to its ID to obtain an updated trajectory of the pedestrian.

[0014] Optionally, detecting the inventory feature objects in all image frames corresponding to the trajectory includes: obtaining all image frames of the surveillance video corresponding to the trajectory; performing target detection on all image frames to detect whether the inventory feature objects are contained; and recording the number of times the inventory feature objects are detected.

[0015] Optionally, the inventory feature objects include a mobile phone and / or a white paper book.

[0016] A second aspect of an embodiment of the present invention provides a machine-readable storage medium, on which instructions are stored, and the instructions are used to enable a machine to execute any of the methods described above.

[0017] A third aspect of an embodiment of the present invention provides a device for identifying inventory counting behavior in a convenience store, the device comprising a memory and a processor, the processor being configured to run a program, wherein the program, when run, is configured to execute any of the methods described above.

[0018] Through the above technical solution, the present invention has the following beneficial effects compared with the prior art:

[0019] The proposed method for identifying convenience store inventory counting activity processes image frames from convenience store surveillance video to obtain pedestrian trajectories within the store. When a predetermined number of inventory counting features are detected in a particular trajectory, and the trajectory's similarity to a standard merchandise inventory trajectory exceeds a predetermined threshold, the store's inventory count is determined to be complete. This method automatically identifies convenience store inventory counting activity and determines whether the count has been completed, without requiring human intervention. This helps prevent retail stores from falsely reporting inventory count frequency and content, ensuring the rigor and accuracy of inventory counting and improving the efficiency and intelligence of enterprise management.

[0020] Other features and advantages of the embodiments of the present invention will be described in detail in the subsequent detailed description. BRIEF DESCRIPTION OF THE DRAWINGS

[0021] The accompanying drawings are used to provide a further understanding of the embodiments of the present invention and constitute a part of the specification. Together with the following detailed description, they are used to explain the embodiments of the present invention, but do not constitute a limitation of the embodiments of the present invention. In the accompanying drawings:

[0022] Figure 1 This is a flow chart of a convenience store inventory behavior recognition method provided by an embodiment of the present invention;

[0023] Figure 2 is a flow chart of a convenience store inventory counting behavior identification method provided by another embodiment of the present invention;

[0024] Figure 3It is a structural diagram of a device provided by an embodiment of the present invention.

[0025] Description of Reference Numerals

[0026] 101 processor 102 memory

[0027] 103 bus 10 devices DETAILED DESCRIPTION

[0028] The following describes the specific implementation of the embodiment of the present invention in detail with reference to the accompanying drawings. It should be understood that the specific implementation described herein is only used to illustrate and explain the embodiment of the present invention and is not used to limit the embodiment of the present invention.

[0029] Figure 1 FIG. 1 is a flow chart of a method for identifying inventory counting behavior in a convenience store provided by an embodiment of the present invention. Figure 1 As shown, the method includes the following steps S101 to S104.

[0030] Step S101: Acquire the current frame of the surveillance video of the convenience store.

[0031] Step S102: performing pedestrian detection and information comparison on the current frame to obtain pedestrian data of each pedestrian in the current frame.

[0032] Step S103: Based on the pedestrian data, track the trajectory of each pedestrian to obtain the trajectory of each pedestrian.

[0033] Step S104: Performing a first processing on the trajectory of each pedestrian, including: detecting inventory features in all image frames corresponding to the trajectory; if the number of times the inventory features are detected is less than a preset number of detections, ending the current processing; and comparing the trajectory with a preset standard commodity inventory trajectory to obtain trajectory similarity; if the trajectory similarity is greater than a preset threshold, it is determined that the inventory work is completed.

[0034] Specifically, surveillance video captured by a surveillance camera is acquired in real time. At any given moment, the most recent frame of the surveillance video is the current frame. When performing pedestrian detection on the current frame, the current frame is input into a pedestrian detection model. In one embodiment of the present invention, the pedestrian detection model utilizes a pre-trained YOLOv8 object detection model. This model is trained using a combination of publicly available pedestrian detection datasets and proprietary datasets derived from store surveillance data. This allows for accurate pedestrian detection while better aligning with the specific scenarios of each store.

[0035] After receiving the current frame image, the pedestrian detection model can detect pedestrian data in the image, including the location rectangle and confidence level. It then compares the information of the pedestrians in the current frame with the pedestrians in the previous image frame, distinguishes each pedestrian, assigns the same ID to the same pedestrian, and finally obtains the pedestrian data. For example, the pedestrian data of a certain pedestrian is: (Pedestrian 1, [200, 200, 250, 300], 0.95), indicating that the pedestrian ID is Pedestrian 1, the location in the image is [200, 200, 250, 300], and the confidence level is 0.95. It should be noted that if the confidence level in the pedestrian data of a certain pedestrian is less than 0.4, the detection target is considered not to be a pedestrian and the result is filtered out to avoid interference caused by false detection.

[0036] Based on the detected pedestrian data, all pedestrians in the current frame can be tracked. It should be noted that the trajectory tracking can select a variety of existing trajectory tracking algorithms, and the present invention does not limit the selection of the trajectory tracking algorithm.

[0037] After obtaining the trajectories of all pedestrians, it is determined whether each trajectory belongs to the trajectory of inventory counting behavior. For a certain trajectory, first detect whether all image frames corresponding to the trajectory contain inventory feature objects with a preset number of detections. Inventory feature objects are tools used when performing inventory work, such as mobile phones and / or white paper books, or other tools, which can be selected according to actual conditions. If inventory feature objects are detected with a number greater than or equal to the preset number of detections, then the trajectory may belong to the trajectory of inventory counting behavior. At this point, the trajectory is further compared with the preset standard commodity inventory trajectory to obtain the trajectory similarity. If the trajectory similarity is greater than the preset threshold, it can be understood that the pedestrian corresponding to the trajectory has performed an inventory work.

[0038] This technical solution can automatically identify convenience store inventory counts without human intervention, helping to prevent stores from falsely reporting inventory frequency and content, ensuring the rigor and accuracy of inventory checks, and improving the efficiency and intelligence of enterprise management. Furthermore, the results obtained through this technical solution can be automatically uploaded to the enterprise's control system, eliminating additional work for store employees and helping to reduce their workload.

[0039] Furthermore, the method further includes: detecting whether the inventory work is completed within a specified time, and issuing a warning if the inventory work is not completed.

[0040] To further improve the efficiency and intelligence of enterprise management, the system checks whether the inventory has been completed within a specified time. If the staff fails to complete the inventory within the specified time, a warning will be issued to the staff to remind them to complete the inventory as soon as possible.

[0041] Furthermore, the method also includes: after obtaining the trajectory of each of the pedestrians, determining whether the trajectory contains a complete trajectory; if it contains a complete trajectory, performing the first processing on the complete trajectory; otherwise, not performing the first processing; wherein, when a certain pedestrian cannot be detected in a consecutive preset number of image frames, the trajectory corresponding to the pedestrian is determined to be a complete trajectory.

[0042] It is understandable that if the first processing is performed on each trajectory, a large workload is required. Some of the trajectories may not have been tracked yet. For such trajectories, no processing is required and tracking can be continued until the trajectory tracking is completed. After the complete trajectory is obtained, the first processing can be performed, which can greatly reduce the workload.

[0043] In some embodiments of the present invention, if the number of times the inventory feature is detected in all image frames corresponding to the complete trajectory of a pedestrian is less than a preset number of detections, the pedestrian data and the complete trajectory corresponding to the pedestrian are deleted to reduce useless data occupying storage space.

[0044] Furthermore, the first processing also includes: before detecting the inventory features in all image frames corresponding to the trajectory, classifying the pedestrians corresponding to the trajectory as employees or customers. If the classification result is a customer, the first processing is terminated.

[0045] Understandably, in most cases, the trajectory of a customer entering a convenience store differs significantly from that of an inventory count. However, in rare cases, a customer's trajectory may be highly similar to a pre-set standard inventory count trajectory. To avoid misidentifying such trajectories as those associated with inventory counts, the system first identifies whether the person in the trajectory is an employee or a customer before detecting inventory features in all image frames corresponding to the trajectory. If the person is a customer, the first pass of the process is terminated, thereby improving the accuracy of inventory count identification.

[0046] In addition, whether the trajectory is a complete trajectory can be judged before the first processing. At this time, if the pedestrian classification result is a customer, the pedestrian data and the complete trajectory corresponding to the pedestrian are deleted to reduce useless data occupying storage space.

[0047] Furthermore, classifying the pedestrian corresponding to the trajectory includes: obtaining all image frames of the surveillance video corresponding to the trajectory of the pedestrian; obtaining image blocks of the pedestrian in all the image frames, and labeling the image blocks as p1, p2, p3, ..., p N; inputting the image blocks into the pre-trained yolov8 personnel classification model in sequence to obtain the classification category of the pedestrian contained in each image block, wherein the classification category includes employee and customer; counting the number of employees and the number of customers in the N image blocks of the pedestrian; and comparing the number of employees with the number of customers. If the number of customers is greater than the number of employees, classify the pedestrian as a customer; otherwise, classify the pedestrian as an employee, wherein the position of the image block in the image frame is obtained by the pedestrian detection.

[0048] It is understandable that in convenience stores, employees generally wear uniform uniforms of special colors, which are obviously different from ordinary customers in appearance. Therefore, employees and customers can be classified based on this difference.

[0049] In specific implementation, a special training set can be constructed based on the general yolov8_cls image classification model to train a special personnel classification model, wherein the training set includes a public pedestrian detection dataset and a proprietary dataset made from employee and customer data in the store's surveillance video. In step S102, after the pedestrian detection model detects the current frame image, pedestrian data is obtained, which includes the position rectangle of each pedestrian. Based on each position rectangle, the image block of each pedestrian can be cropped. Assuming that a trajectory or a complete trajectory P contains a target in N frames of image, all image blocks of the pedestrian corresponding to the trajectory can be recorded as P = (p1,…,p N ), and then sequentially transform the image blocks p1, p2, p3, ..., p N By inputting the pre-trained yolov8 person classification model, we can obtain the classification category of the pedestrian contained in each image block. Finally, we compare the number of people classified as employees with the number of people classified as customers. The larger number is the classification result of the pedestrian corresponding to the trajectory.

[0050] In reality, while the accuracy of existing image classification models continues to improve, classification errors still occur. The pedestrian classification proposed in this paper is based on all image frames along the entire trajectory. Even if a small number of image blocks cause classification errors due to occlusion or illumination changes, this does not affect the classification results for the pedestrians corresponding to the entire trajectory, thereby improving the accuracy of pedestrian classification and its robustness to occlusion and illumination changes.

[0051] Furthermore, pedestrian detection and information comparison are performed on the current frame to obtain pedestrian data of each pedestrian in the current frame, including: performing pedestrian detection on the current frame to obtain appearance information and position information of all people in the current frame; comparing each pedestrian with pedestrians that have appeared before the current frame, including: comparing the appearance information of each pedestrian with the appearance information of pedestrians that have appeared before the current frame to obtain appearance information similarity, comparing the position information of each pedestrian with the position information of pedestrians that have appeared before the current frame to obtain position information similarity; and if both the appearance information similarity is not less than a preset first threshold and the position information similarity is not less than a preset second threshold, then the pedestrian is considered to be the same person as the pedestrian that appeared before the compared current frame, and the pedestrian in the current frame is assigned the same ID as the compared pedestrian; otherwise, the pedestrian is considered to be a newly appeared pedestrian, and the pedestrian in the current frame is assigned a new ID, wherein the appearance information is the feature vector of the image block corresponding to the pedestrian, and the position information is the coordinates of the preset position of the image block corresponding to the pedestrian.

[0052] For location information, after obtaining the pedestrian's location rectangle, a point within the rectangle can be selected as the pedestrian's location information, such as the midpoint or bottom midpoint of the rectangle, or other locations can be selected based on actual conditions. For appearance information, after obtaining the location rectangle, the image block corresponding to the rectangle is input into the ResNet network to extract a 512-dimensional feature vector as the corresponding appearance information.

[0053] Each pedestrian is compared with pedestrians that appeared before the current frame to distinguish all pedestrians in the current frame, thereby subsequently tracking the trajectory of each pedestrian. During the comparison, the appearance information and position information of each pedestrian are compared. Specifically, the appearance information similarity between two pedestrians can be measured using the Euclidean distance between vectors, and the position information similarity can be measured using the Euclidean distance between coordinates. Depending on the actual situation, other calculation methods can be selected as long as they can characterize the similarity of appearance information and position information, and the present invention is not limited to this. When the appearance similarity and position information similarity between two pedestrians in two image frames both meet pre-set threshold conditions, the two pedestrians can be determined to be the same person, and the pedestrian in the current frame is assigned the same ID as the compared pedestrians. If the comparison result does not meet one or both threshold conditions, the two pedestrians can be determined to be different people. If a pedestrian in the current frame does not meet the above conditions after comparison with all pedestrians with existing IDs, the pedestrian is considered to be a new pedestrian and is assigned a new ID.

[0054] In some embodiments of the present invention, pedestrian data includes the pedestrian's ID, location rectangle, confidence, appearance information, and location information.

[0055] Furthermore, the pedestrian detection and information comparison are performed on the current frame to obtain the pedestrian data of each pedestrian in the current frame: comparing each pedestrian with the pedestrians that have appeared before the current frame also includes: comparing the position information of each pedestrian with the predicted trajectory information of the pedestrians that have appeared before the current frame to obtain the similarity of the predicted information; if the appearance information similarity is not less than a preset first threshold, the position information similarity is not less than a preset second threshold, and the predicted information similarity is not less than a preset third threshold, then the pedestrian is considered to be the same person as the pedestrian that appeared before the compared current frame, and the pedestrian in the current frame is assigned the same ID as the compared pedestrian; otherwise, the pedestrian is considered to be a newly appeared pedestrian, and the pedestrian in the current frame is assigned a new ID, wherein the predicted trajectory information is predicted by the position information of the pedestrians that have appeared before the current frame.

[0056] It is understandable that before the current frame, there may have been information on multiple pedestrians, and the position information and appearance information of each pedestrian before were detected. For any pedestrian, after the position information is detected once, the trajectory of the pedestrian can be created. After the position information is detected twice or three times, the trajectory of the pedestrian can be predicted based on the historical trajectory. If a pedestrian meets the first threshold and the second threshold, and the position in the current frame is similar to the predicted trajectory information of another pedestrian is not less than the pre-set third threshold, it can be further confirmed that the pedestrian and the other pedestrian are the same person. When comparing information, adding the predicted trajectory information further improves the accuracy of distinguishing each pedestrian. Specifically, the similarity of the predicted information between two pedestrians can be measured by the Euclidean distance between the coordinates. According to the actual situation, other calculation methods can also be selected to characterize the similarity of the predicted information. The present invention is not limited to this.

[0057] Furthermore, based on the pedestrian data, the trajectory of each pedestrian is tracked to obtain the trajectory of each pedestrian, including: if the pedestrian is a newly appeared pedestrian, a new trajectory is created for the pedestrian; if the pedestrian is a pedestrian that has appeared before the current frame, the position information of the pedestrian is added to the end of the trajectory of the pedestrian corresponding to its ID to obtain an updated trajectory of the pedestrian.

[0058] It is understandable that when it is possible to distinguish each pedestrian and the location information of each pedestrian is known, a variety of trajectory tracking algorithms can be used to track the trajectory of each pedestrian. The selection can be made according to the actual situation. The specific implementation method of trajectory tracking will not be elaborated here.

[0059] Furthermore, detecting the inventory feature objects in all image frames corresponding to the trajectory includes: obtaining all image frames of the surveillance video corresponding to the trajectory; performing target detection on all image frames to detect whether the inventory feature objects are contained; and recording the number of times the inventory feature objects are detected.

[0060] Similar to pedestrian detection, when detecting inventory features, all image frames corresponding to the trajectory are input into a pre-trained object detection model. The object detection model can be selected based on the actual situation and then trained using a specially prepared training set of inventory features.

[0061] Figure 2 FIG. 1 is a flow chart of a method for identifying inventory counting behavior in a convenience store provided by another embodiment of the present invention. Figure 2 As shown, the method includes the following steps S201-S207.

[0062] Step S201: Acquire the current frame of the convenience store surveillance video.

[0063] Step S202: performing pedestrian detection and information comparison on the current frame to obtain pedestrian data of each pedestrian in the current frame.

[0064] Step S203: Based on the ID and location information in the pedestrian data, track the trajectory of each pedestrian to obtain the trajectory of each pedestrian.

[0065] Step S204: Determine whether the trajectory contains a complete trajectory. If it does, continue processing; otherwise, end this processing.

[0066] Step S205: Classify the pedestrian corresponding to the detected complete trajectory. If the classification result is an employee, continue processing. If the classification result is a customer, delete the corresponding pedestrian data and the complete trajectory, and end this processing.

[0067] Step S206: Detect the inventory features in all image frames corresponding to the complete trajectory. If the number of times the inventory features are detected is greater than or equal to the preset number of detections, continue processing; otherwise, delete the corresponding pedestrian data and the complete trajectory, and end this processing.

[0068] Step S207: Compare the complete trajectory with a preset standard commodity inventory trajectory to obtain trajectory similarity. If the trajectory similarity is greater than a preset threshold, it is determined that the inventory work is completed; otherwise, it is determined that the inventory work is not completed.

[0069] An embodiment of the present invention further provides a machine-readable storage medium having stored thereon instructions for enabling a machine to execute the convenience store inventory counting behavior identification method.

[0070] The embodiment of the present invention also provides a device for identifying inventory counting behavior in convenience stores, for example, Figure 3 As shown, the device includes a memory and a processor, and the processor is used to run a program, wherein the program is used to execute the convenience store inventory behavior identification method when it is run.

[0071] The memory may include non-permanent memory in a computer-readable medium, random access memory (RAM) and / or non-volatile memory, such as read-only memory (ROM) or flash RAM, and the memory includes at least one memory chip.

[0072] The processor contains a kernel, which retrieves the corresponding program unit from the memory. One or more kernels can be set, and the kernel parameters can be adjusted to achieve convenience store inventory behavior recognition.

[0073] An embodiment of the present invention further provides a computer program product, which, when executed on a data processing device, is suitable for executing a program for initializing the convenience store inventory counting behavior identification method.

[0074] Those skilled in the art will appreciate that the embodiments of the present application can be provided as methods, systems, or computer program products. Therefore, the present application can adopt the form of a complete hardware embodiment, a complete software embodiment, or an embodiment in combination with software and hardware. Moreover, the present application can adopt the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to magnetic disk storage, CD-ROM, optical storage, etc.) that contain computer-usable program code.

[0075] The present application is described with reference to the flowcharts and / or block diagrams of the methods, devices (systems), and computer program products according to the embodiments of the present application. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of the processes and / or boxes in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the steps in the process. Figure 1 a process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0076] These computer program instructions may also be stored in a computer readable memory that can direct a computer or other programmable data processing device to work in a specific manner, so that the instructions stored in the computer readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 a process or multiple processes and / or boxes Figure 1 The function specified in one or more boxes.

[0077] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operational steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing the instructions executed on the computer or other programmable device for implementing the process. Figure 1 a process or multiple processes and / or boxes Figure 1 A step that specifies a function in one or more boxes.

[0078] In a typical configuration, a computing device includes one or more processors (CPUs), input / output interfaces, network interfaces, and memory.

[0079] The memory may include non-permanent memory in a computer-readable medium, random access memory (RAM) and / or non-volatile memory in the form of read-only memory (ROM) or flash RAM. The memory is an example of a computer-readable medium.

[0080] Computer-readable media includes permanent and non-permanent, removable and non-removable media that can be implemented by any method or technology to store information. The information can be computer-readable instructions, data structures, program modules or other data. Examples of computer storage media include, but are not limited to, phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technology, compact disc read-only memory (CD-ROM), digital versatile disc (DVD) or other optical storage, magnetic cassettes, magnetic disk storage or other magnetic storage devices or any other non-transmission media that can be used to store information that can be accessed by a computing device. As defined herein, computer-readable media does not include transitory computer-readable media (transitory media), such as modulated data signals and carrier waves.

[0081] It should also be noted that the terms "comprises," "includes," or any other variations thereof are intended to encompass non-exclusive inclusion, such that a process, method, commodity, or apparatus that includes a series of elements includes not only those elements but also other elements not explicitly listed, or includes elements inherent to such process, method, commodity, or apparatus. In the absence of further limitations, an element defined by the phrase "comprises a ..." does not exclude the presence of other identical elements in the process, method, commodity, or apparatus that includes the element.

[0082] The above are merely embodiments of the present application and are not intended to limit the present application. For those skilled in the art, the present application may have various changes and variations. Any modifications, equivalent replacements, improvements, etc. made within the spirit and principles of the present application should all be included within the scope of the claims of the present application.

[0083] The acquisition, transmission, storage, use, and processing of data in the technical solution of this application comply with the relevant provisions of national laws and regulations.

[0084] It should be noted that in the embodiments of the present application, certain software, components, models and other existing solutions in the industry may be mentioned. They should be regarded as exemplary. Their purpose is only to illustrate the feasibility of implementing the technical solution of the present application, but it does not mean that the applicant has or will necessarily use the solution.

Claims

1. A convenience store inventory behavior recognition method, characterized in that: The method comprises: Obtaining a current frame of the surveillance video of the convenience store; Performing pedestrian detection and information comparison on the current frame to obtain pedestrian data of each pedestrian in the current frame; Tracking each of the pedestrians based on the pedestrian data to obtain a trajectory of each of the pedestrians; and Performing a first processing on the trajectory of each pedestrian, including: Detecting the inventory feature objects in all image frames corresponding to the trajectory, and ending the current processing if the number of times the inventory feature objects are detected is less than a preset number of detections; and The trajectory is compared with a preset standard commodity inventory trajectory to obtain a trajectory similarity. If the trajectory similarity is greater than a preset threshold, it is determined that the inventory work is completed.

2. The method according to claim 1, characterized in that The method further comprises: Check whether the inventory work is completed within the specified time, and issue a warning if the inventory work is not completed.

3. The method according to claim 1, characterized in that The method further comprises: After obtaining the trajectory of each pedestrian, determining whether the trajectory contains a complete trajectory; if so, performing the first processing on the complete trajectory; otherwise, not performing the first processing; When a pedestrian cannot be detected in a preset number of consecutive image frames, the trajectory corresponding to the pedestrian is determined to be a complete trajectory.

4. The method according to claim 1, wherein The first process further includes: Before detecting the inventory features in all image frames corresponding to the trajectory, the pedestrian corresponding to the trajectory is classified as an employee or a customer. If the classification result is a customer, the first processing is terminated.

5. The method according to claim 4, characterized in that Classifying the pedestrian corresponding to the trajectory includes: Obtain all image frames of the surveillance video corresponding to the trajectory of the pedestrian; Obtain the image blocks of the pedestrian in all the image frames, and label the image blocks as p1, p2, p3, ..., p N ; Input the image blocks into the pre-trained yolov8 personnel classification model in sequence to obtain the classification category of the pedestrian contained in each image block, wherein the classification category includes employee and customer; Counting the number of employees and customers in the N image blocks of the pedestrian; and The number of employees is compared with the number of customers. If the number of customers is greater than the number of employees, the pedestrian is classified as a customer; otherwise, the pedestrian is classified as an employee. The position of the image block in the image frame is obtained by detecting the pedestrian.

6. The method according to claim 1, characterized in that Pedestrian detection and information comparison are performed on the current frame to obtain pedestrian data of each pedestrian in the current frame, including: Performing pedestrian detection on the current frame to obtain appearance information and position information of all persons in the current frame; Comparing each of the pedestrians with pedestrians that have appeared before the current frame, including: comparing the appearance information of each of the pedestrians with the appearance information of pedestrians that have appeared before the current frame to obtain appearance information similarity, and comparing the position information of each of the pedestrians with the position information of pedestrians that have appeared before the current frame to obtain position information similarity; and If the appearance information similarity is not less than a preset first threshold and the position information similarity is not less than a preset second threshold, the pedestrian is considered to be the same person as the pedestrian that appeared before the current frame being compared, and the pedestrian in the current frame is assigned the same ID as the pedestrian being compared. Otherwise, the pedestrian is considered to be a new pedestrian, and a new ID is assigned to the pedestrian in the current frame. The appearance information is a feature vector of an image block corresponding to the pedestrian, and the position information is the coordinates of a preset position of the image block corresponding to the pedestrian.

7. The method according to claim 6, characterized in that Pedestrian detection and information comparison are performed on the current frame to obtain pedestrian data of each pedestrian in the current frame: Comparing each of the pedestrians with pedestrians that have appeared before the current frame further includes: comparing the position information of each of the pedestrians with the predicted trajectory information of pedestrians that have appeared before the current frame to obtain the similarity of the predicted information; If the appearance information similarity is not less than a preset first threshold, the position information similarity is not less than a preset second threshold, and the prediction information similarity is not less than a preset third threshold, then the pedestrian is considered to be the same person as the pedestrian that appeared before the current frame being compared, and the pedestrian in the current frame is assigned the same ID as the pedestrian being compared. Otherwise, the pedestrian is considered to be a new pedestrian, and a new ID is assigned to the pedestrian in the current frame. The predicted trajectory information is predicted based on the position information of pedestrians that have appeared before the current frame.

8. The method according to claim 6, characterized in that Tracking each pedestrian based on the pedestrian data to obtain a trajectory of each pedestrian includes: If the pedestrian is a new pedestrian, a new trajectory is created for the pedestrian; If the pedestrian has appeared before the current frame, the position information of the pedestrian is added to the end of the trajectory of the pedestrian corresponding to its ID to obtain an updated trajectory of the pedestrian.

9. The method according to claim 1, characterized in that Detecting the inventory features in all image frames corresponding to the trajectory includes: Obtain all image frames of the surveillance video corresponding to the trajectory; Performing target detection on all the image frames to detect whether the inventory feature objects are contained; and The number of times the inventory feature is detected is recorded.

10. The method according to claim 9, characterized in that The inventory feature objects include a mobile phone and / or a white paper book.

11. A machine-readable storage medium having stored thereon instructions for causing a machine to execute the method according to any one of claims 1 to 10.

12. A device for identifying inventory counting behavior in a convenience store, the device comprising a memory and a processor, characterized in that: The processor is configured to run a program, wherein the program, when run, is configured to execute the method according to any one of claims 1 to 10.

Images