Robot shelf inspection method, device, equipment and system

Abstract

The invention relates to the field of goods shelf inspection, in particular to a robot goods shelf inspection method, device, equipment and system, and the method comprises the steps: obtaining an inspection task for a first target goods shelf; according to the store map of the target store, controlling the robot to carry out commodity stockout inspection on the first target shelf according to the inspection task, obtaining a single-point detection result, and determining a stockout position; based on a single point detection result, in combination with shelf standard information and historical detection data of the first target shelf, commodity information missing from the first target shelf can be accurately determined, the shelf standard information provides commodity information corresponding to each position, the historical detection data can reflect past conditions of the shelf, and the shelf standard information and the historical detection data complement each other. Therefore, the judgment of the missing commodity information is more accurate and reliable, thereby effectively improving the efficiency and accuracy of shelf inspection, finding out the stockout condition in time, and guaranteeing the normal supply of store commodities.

Description

Technical Field

[0001] This application relates to the field of shelf inspection, and in particular to a robotic shelf inspection method, apparatus, equipment and system. Background Technology

[0002] In today's highly competitive retail landscape, accurately identifying out-of-stock SKUs is crucial. However, current methods for determining out-of-stock SKUs heavily rely on algorithmic accuracy, which has significant limitations. Relying solely on a single detection result to infer out-of-stock SKUs is prone to error. Products on shelves are not always neatly arranged; some items may be obscured by other products due to customer movement or improper placement. In such cases, a single detection method may fail to identify obscured items, leading to incorrect SKU deeming them out of stock. This not only causes inventory management chaos and incorrect replenishment decisions for retailers but also prevents customers from purchasing the desired items due to misjudgment, reducing customer satisfaction. Summary of the Invention

[0003] The purpose of this application is to provide a robotic shelf inspection method, apparatus, equipment, and system that can effectively improve the efficiency and accuracy of shelf inspection.

[0004] In a first aspect, a robotic shelf inspection method is provided, comprising: acquiring an inspection task for a first target shelf, wherein the first target shelf is a shelf to be inspected in a target store; controlling a robot to perform out-of-stock inspection on the first target shelf according to the store map of the target store and the inspection task, thereby obtaining a single-point detection result; and determining the missing product information of the first target shelf based on the single-point detection result, according to the shelf standard information and historical detection data of the first target shelf.

[0005] In a preferred embodiment, this application may be further configured to include: responding to a mapping instruction, controlling a robot to explore and map a target shelf to obtain a store map; responding to a configuration operation triggered by a shelf configuration component for the store map, setting a configuration area on the store map, wherein the configuration operation includes a name and a location, and if the configuration operation is shelf configuration information, the configuration operation also includes a valid plane.

[0006] In a preferred embodiment, this application can be further configured such that the process of constructing shelf standard information includes: in response to the commodity standard input information triggered by the target position surface of the second target shelf of the target store, configuring the shelf standard information of the second target shelf, wherein the second target shelf is any one of the multiple shelves of the target store; or, in response to the shelf detection task, controlling the robot to inspect and obtain images of each location of each shelf in the target store; identifying the commodity information and commodity position of each location image of each shelf, and generating shelf standard information of each shelf based on the commodity information and commodity position.

[0007] In a preferred embodiment, this application can be further configured to: generate shelf standard information for each shelf based on the product information and product location, including: generating initial shelf standard information for each shelf based on the product information and product location; and generating shelf standard information for each shelf in response to a modification operation on the initial shelf standard information.

[0008] In a preferred embodiment, this application can be further configured to: obtain an inspection task for a first target shelf, including: obtaining an inspection task for the first target shelf in response to a task creation operation triggered by the first target shelf of the target store; or, determining whether the inspection cycle for the first target shelf of the target store has been reached, and if so, obtaining an inspection task for the first target shelf.

[0009] In a preferred embodiment, this application can be further configured as follows: based on the single-point detection results, and according to the shelf standard information and historical detection data of the first target shelf, after determining the missing product information of the first target shelf, the application further includes: displaying out-of-stock alert information on the store map on the map homepage; in response to a user triggering a details viewing operation of the area corresponding to the out-of-stock alert information, displaying the missing product information of the first target shelf; and / or, in response to a user triggering a shelf overview operation of the area corresponding to the out-of-stock alert information, displaying a shelf view, wherein the shelf view is displayed as a virtual shelf, and the out-of-stock location in the virtual shelf displays an out-of-stock indicator; the out-of-stock indicator... When the detection is triggered, the system can display information about the missing products at the out-of-stock location; and / or, in response to the user triggering an image display operation for the third target shelf, a composite image of the third target shelf is displayed, which is obtained by stitching together photos taken by the robot at different locations; and / or, in response to the user triggering a heatmap display operation for the fourth target shelf, a heatmap of the fourth target shelf is displayed, with different blocks of the virtual shelf in the heatmap of the fourth target shelf corresponding to corresponding colors; in response to the user triggering an operation to view abnormal colors in the heatmap of the fourth target shelf, a prompt message is displayed, which includes the number of times the product corresponding to the block is out of stock and product information.

[0010] In a preferred embodiment, this application can be further configured as follows: based on the single-point detection result, and according to the shelf standard information and historical detection data of the first target shelf, after determining the missing product information of the first target shelf, it further includes: in response to the user-triggered report viewing operation of the fifth target shelf, displaying the stockout analysis information of the fifth target shelf, wherein the stockout analysis information includes stockout trends and stockout statistics.

[0011] Secondly, a robotic shelf inspection device is provided, comprising: a task acquisition module for acquiring an inspection task for a first target shelf, wherein the first target shelf is a shelf to be inspected in a target store; a control inspection module for controlling a robot to perform out-of-stock inspections on the first target shelf according to the store map of the target store and the inspection task, thereby obtaining a single-point detection result; and a shelf detection module for determining the missing product information of the first target shelf based on the single-point detection result, the shelf standard information of the first target shelf, and historical detection data.

[0012] Thirdly, an electronic device is provided, the electronic device including a memory and a processor, the memory storing a computer program, the processor executing the robotic shelf inspection method according to any one of the first aspects when running the computer program.

[0013] Fourthly, a computer-readable storage medium is provided, wherein at least one piece of program code is stored therein, the program code being loaded and executed by a processor to implement the robotic shelf inspection method as described in any of the first aspects.

[0014] Fifthly, a computer program product is provided, including a computer program or instructions that, when executed by a processor, implement the robotic shelf inspection method as described in any of the first aspects.

[0015] In a sixth aspect, a robotic shelf inspection system is provided, comprising: a robot, an electronic device as described in the third aspect, and a terminal device; the robot is used to perform out-of-stock inspections on a first target shelf according to the store map and the inspection task, obtain single-point detection results, and send the single-point detection results and single-point images to the electronic device.

[0016] In summary, the robotic shelf inspection method provided in this application has the following beneficial technical effects:

[0017] The system acquires inspection tasks for the first target shelf; based on the store map of the target store, it controls a robot to inspect the first target shelf for out-of-stock items according to the inspection tasks and obtains single-point detection results to determine the location of the out-of-stock items; based on the single-point detection results, combined with the shelf standard information and historical detection data of the first target shelf, it can accurately determine the missing product information of the first target shelf. Among them, the shelf standard information provides the product information that should be in each location, and the historical detection data can reflect the past situation of the shelf. The two complement each other, making the judgment of missing product information more accurate and reliable, thereby effectively improving the efficiency and accuracy of shelf inspection, timely detection of out-of-stock situations, and ensuring the normal supply of goods to the store.

[0018] In addition, this application also provides a robotic shelf inspection device, equipment, and medium, all of which have the aforementioned beneficial technical effects. Attached Figure Description

[0019] To more clearly illustrate the technical solutions of the embodiments of this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0020] Figure 1 This is a schematic diagram of the structure of a robotic shelf inspection system provided in an embodiment of this application;

[0021] Figure 2 This is a schematic diagram of a robot shelf inspection method provided in an embodiment of this application;

[0022] Figure 3 This is a real-life image of a store provided in an embodiment of this application;

[0023] Figure 4 This is a schematic diagram of a construction provided in an embodiment of this application;

[0024] Figure 5 This is a schematic diagram of an image editing method provided in an embodiment of this application;

[0025] Figure 6 This is a schematic diagram of an effective detection surface of a labeling shelf provided in an embodiment of this application;

[0026] Figure 7 This is a schematic diagram of a shelf management page provided in an embodiment of this application;

[0027] Figure 8 This is a schematic diagram of a page for creating a task from the homepage, provided in an embodiment of this application;

[0028] Figure 9 This is a schematic diagram of a page for creating inspection tasks from a task list, provided in an embodiment of this application.

[0029] Figure 10 This is a schematic diagram of a page that triggers the display of out-of-stock items, provided in an embodiment of this application;

[0030] Figure 11 This is a schematic diagram of a shelf overview provided in an embodiment of this application;

[0031] Figure 12 This is a schematic diagram of a page displaying spliced ​​images provided in an embodiment of this application;

[0032] Figure 13 This is a schematic diagram of a heatmap switching page provided in an embodiment of this application;

[0033] Figure 14 This is a schematic diagram illustrating the process of switching to the out-of-stock analysis page provided in an embodiment of this application;

[0034] Figure 15 This is a schematic diagram of a stockout analysis page provided in an embodiment of this application;

[0035] Figure 16 This is a schematic diagram illustrating a specific process for robotic shelf inspection provided in an embodiment of this application;

[0036] Figure 17 This is a schematic diagram of the structure of a robotic shelf inspection device provided in an embodiment of this application;

[0037] Figure 18 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application. Detailed Implementation

[0038] This specific embodiment is merely an explanation of this application and is not intended to limit it. After reading this specification, those skilled in the art can make modifications to this embodiment without contributing any inventive step, but such modifications are protected by patent law as long as they are within the scope of this application.

[0039] It should be noted that, in the optional embodiments of this application, the data related to object information, when applied to specific products or technologies, requires the permission or consent of the object. Furthermore, the collection, use, and processing of this data must comply with the relevant laws, regulations, and standards of the relevant countries and regions. In other words, if the embodiments of this application involve data related to an object, it must be obtained with the permission and consent of the object, the permission and consent of relevant departments, and in accordance with the relevant laws, regulations, and standards of the country and region. If the embodiments involve personal information, the acquisition of all personal information requires the consent of the individual. If sensitive information is involved, the separate consent of the information subject is required. The embodiments also need to be implemented with the permission and consent of the object.

[0040] To make the objectives, technical solutions, and advantages of the embodiments of this application clearer, the technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0041] Furthermore, the term "and / or" in this article is merely a description of the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A existing alone, A and B existing simultaneously, or B existing alone. Additionally, the character " / " in this article, unless otherwise specified, generally indicates that the preceding and following related objects have an "or" relationship.

[0042] To facilitate understanding, some proper nouns are explained below.

[0043] SKU: Stock Keeping Unit. Here, it refers to a single product model displayed on the shelf. It is the smallest unit for the intelligent robot T1 to detect out-of-stock items and identify products. For example, "a certain soft drink 439ml" and "a certain fresh fruit tea 500ml" are both independent SKUs.

[0044] Local product detection algorithm: The product detection and recognition and out-of-stock area localization algorithm is deployed locally on the intelligent robot. It can process images captured by the three side cameras in real time, complete product frame segmentation, SKU recognition and out-of-stock hole detection, without relying on real-time cloud computing, thus improving detection efficiency.

[0045] Cloud-based shelf stitching algorithm: An image integration algorithm deployed in a cloud system that can stitch together shelf images collected by intelligent robots at different shooting points into a complete shelf image according to the actual shelf location order, and simultaneously integrate local detection results with backend shelf standard data.

[0046] This invention relates to the fields of intelligent robot inspection technology and retail shelf management technology. Specifically applied to retail scenarios such as unmanned convenience stores and chain supermarkets, it utilizes a hardware and software system based on intelligent robots. The intelligent robot can be equipped with three side-mounted cameras, LiDAR, and a touchscreen, enabling automatic inspection to detect out-of-stock items on shelves, perform data statistics, and provide visualized management.

[0047] In existing retail scenarios, current intelligent robot shelf inspection technologies suffer from the following problems: First, when intelligent robots use LiDAR to create maps, enclosed shelves against walls are easily misidentified as walls due to the lack of a clear boundary between them and the walls, resulting in incomplete shelf identification. Existing solutions require manual marking on the map based on experience, which can easily lead to missed or incorrect detections in subsequent inspections due to inaccurate marking positions. Second, on the one hand, the determination of out-of-stock SKUs is highly dependent on the accuracy of the algorithm. Inferring out-of-stock SKUs based solely on a single detection result is prone to errors due to product obstruction or interference from similar products. On the other hand, if manual configuration is used throughout the process, such as entering each SKU through a backend system, staff need to check the products on each side and layer of the shelf one by one, which is complex and labor-intensive. If the standard is automatically generated by the algorithm, inaccurate standards can be caused by issues such as shooting angle deviations and similar product packaging, further affecting the detection results. Third, even if the intelligent robot completes the shelf scanning, the standards need to be manually modified repeatedly when new products are added or the display is adjusted, which cannot adapt to the high-frequency changes in products in retail scenarios. Furthermore, the current inspection results from the intelligent robot are only displayed as a single image or simple text, without a complete picture of the entire shelf or highlighted areas of stock shortage. Staff need to check each image individually to trace the location of stock shortages. At the same time, the system lacks optimized stock shortage analysis, has poor scalability, and cannot provide constructive suggestions for optimizing store operations.

[0048] To address the aforementioned issues, this invention proposes a novel robotic SKU shelf inspection method, primarily comprising: establishing an automatic mapping combined with manual precision completion mode, supporting manual drawing of rectangular markers on a map to indicate wall-mounted shelves and specifying the effective inspection surface of the shelves, thus resolving the misjudgment problem of wall-mounted shelves. An optional shelf standard configuration process is designed, supporting backend preset standards, such as specifying the first row of shelf 1 as a certain product, or robot scanning combined with manual verification; the robot first generates an initial standard, which is then modified and confirmed manually, reducing the workload of manual data entry. A dual approach of historical data inference and standard comparison is employed: the local algorithm first infers out-of-stock products through adjacent products, and then the cloud calls the backend standard for secondary verification, improving the accuracy of out-of-stock SKU judgment. A comprehensive shelf map is generated through cloud-based image stitching, highlighting out-of-stock areas; clicking on the highlighted area allows viewing detailed images of the corresponding location; simultaneously, an interface for integration with the store's warehouse management system is reserved, supporting subsequent optimization of out-of-stock analysis based on inventory and time data.

[0049] To better understand the solution provided in the embodiments of this application, the solution will be described below in conjunction with a specific application scenario.

[0050] In one embodiment, this application provides a robotic shelf inspection system.

[0051] In some embodiments, see Figure 1 This robotic shelf inspection system includes electronic equipment, robots, and terminal devices. The robots are equipped with cameras to collect video information for use by the electronic equipment, which can be servers capable of implementing the robotic shelf inspection method. The terminal devices include, but are not limited to, mobile terminals such as mobile phones, laptops, digital radio receivers, PDAs (personal digital assistants), PADs (tablet computers), and PMPs (portable multimedia players), as well as fixed terminals such as digital TVs and desktop computers.

[0052] It is understood that the above is only one example, and this embodiment is not limited here.

[0053] Specifically, this application provides a robotic shelf inspection method, such as... Figure 2 As shown, the method provided in this application embodiment can be executed by an electronic device, and the method includes:

[0054] S101. Obtain the inspection task for the first target shelf, where the first target shelf is the shelf to be inspected in the target store.

[0055] The scenario primarily involves the daily operation and management of retail stores, including supermarkets, convenience stores, and other locations that sell goods. The target store refers to the specific retail store targeted in this inspection. The first target shelf is the shelf to be inspected; there must be at least one first target shelf. The inspection task includes the location of the first target shelf to be inspected, the inspection time, and the inspection requirements, which specify that out-of-stock items must be inspected.

[0056] S102. Based on the store map of the target store, control the robot to perform out-of-stock inspection on the first target shelf according to the inspection task, and obtain the single-point detection result.

[0057] The store map represents the spatial layout of the target store, graphically showing the location of shelves and aisles in various areas of the store. It provides navigation and positioning for the robot within the store. The robot represents the equipment used to perform out-of-stock inspection tasks. It can be equipped with three cameras on its side to capture images at different heights. Inspection points are set for the robot; for example, for a 2m long shelf, the robot can be programmed to move 20cm at a time, capturing an image at that point to obtain a single-point image. This single-point image can then be sent to electronic devices. The single-point detection result represents the result of detecting out-of-stock items at a single point on the first target shelf, including the location of the out-of-stock item. The single-point image represents the image captured at a single point on the first target shelf, recording the product display at a specific location or area on the shelf, used to assist in determining whether a product is out of stock.

[0058] Specifically, a store map of the target store is pre-programmed into the electronic device; based on the inspection task requirements, the specific location of the first target shelf and the range of single-point locations to be inspected are set in the program; the optimal path is planned based on the store map and target location, and the robot is controlled to move along the path to the first target shelf; upon reaching the target shelf, the robot's sensors are used to photograph the single-point products, with the robot taking pictures at each point using three side cameras. In one feasible approach, an algorithm is set within the robot to detect product frames and determine shelf vacancy locations, and then the robot sends the single-point detection results and single-point images to the electronic device. In another feasible approach, the robot sends the single-point images to the electronic device, which is equipped with an algorithm to detect product frames and shelf vacancy locations, accurately locate the coordinates of the out-of-stock area, and generate single-point detection results, including a product list and the coordinates of the out-of-stock area.

[0059] S103. Based on the single-point detection results, and according to the shelf standard information and historical detection data of the first target shelf, determine the missing product information of the first target shelf.

[0060] The standard shelf information refers to the product information that the first target shelf should have when it is normally displayed. This includes detailed standard data such as the types of products that should be displayed on the shelf and the display location of each product, serving as a reference for determining whether any products are missing. Historical inspection data refers to relevant data recorded during historical inspections of the first target shelf, including but not limited to single-point inspection results, single-point images, and historical records of confirmed product shortages. Missing product information indicates the types, specifications, quantities, and locations of currently missing products on the first target shelf, guiding the store in replenishment operations.

[0061] In one feasible approach, individual location images and detection results are uploaded to an electronic device, such as a cloud server. The cloud uses a mosaic algorithm to stitch the images together in the order of shelf positions to generate a complete shelf mosaic image. The coordinates of the out-of-stock areas are then mapped to the corresponding positions on the mosaic image, ensuring that the stitched image matches the actual number and position of the shelves. This allows the location of out-of-stock items to be displayed.

[0062] Furthermore, when making actual judgments, the system calls upon the standard information of the back-end shelves and historical detection data to make a comprehensive judgment, and finally displays the missing products on the first target shelf.

[0063] Specifically, based on the out-of-stock location identified by a single-point detection, corresponding sub-standard information is determined from the shelf standard information. Then, based on the out-of-stock location identified by the single-point detection, historical detection results corresponding to that location are filtered from historical detection data. The consistency between the sub-standard information and the historical detection results is then determined. If consistent, the missing product information is determined based on the sub-standard information / historical detection results. It's understandable that the historical detection results can be from the previous out-of-stock moment or from multiple out-of-stock moments. If from multiple out-of-stock moments, the product with the most frequent occurrences is used as the reference product. If inconsistent, in one possible scenario, multiple sub-standard information from the adjacent left and right positions is obtained. Then, based on the single-point image, the product information adjacent to the out-of-stock location is determined. Finally, based on the product information and multiple sub-standard information, the correct sub-standard information is determined, and the missing product information is determined based on the sub-standard information. Alternatively, a prompt message can be generated to facilitate manual confirmation of the out-of-stock product information.

[0064] The dual-scheme collaborative out-of-stock judgment mechanism, which combines local historical product inference with cloud-based standard comparison, integrates the advantages of real-time detection and historical standard data. It effectively overcomes the misjudgment caused by single algorithms due to product obstruction, similar packaging, and angle deviation, significantly improving the accuracy and reliability of out-of-stock SKU identification results. It breaks through the contradiction between accuracy and complexity and significantly improves the accuracy of out-of-stock judgment.

[0065] As can be seen, in this embodiment, an inspection task is obtained for the first target shelf; based on the store map of the target store, the robot is controlled to perform out-of-stock inspections on the first target shelf according to the inspection task and obtain single-point detection results to determine the out-of-stock location; based on the single-point detection results, combined with the shelf standard information and historical detection data of the first target shelf, the missing product information of the first target shelf can be accurately determined. Among them, the shelf standard information provides the product information that should be in each location, and the historical detection data reflects the past situation of the shelf. The two complement each other, making the judgment of missing product information more accurate and reliable, thereby effectively improving the efficiency and accuracy of shelf inspection, timely detection of out-of-stock situations, and ensuring the normal supply of goods in the store.

[0066] One possible implementation of this application embodiment further includes: responding to a mapping instruction, controlling a robot to explore and map the target shelf to obtain a store map; responding to a configuration operation triggered by a shelf configuration component for the store map, setting a configuration area on the store map, wherein the configuration operation includes a name and a location, and if the configuration operation is shelf configuration information, the configuration operation also includes a valid plane.

[0067] See Figure 3 , Figure 3 This is a real-world store image provided in an embodiment of this application. After the robot is connected to the network, see... Figure 4 , Figure 4 This is a mapping illustration provided in an embodiment of this application. The user clicks "Start Mapping" in the app system of the terminal device to enter mapping mode. At this time, a mapping instruction is generated, and the terminal device sends the instruction to an electronic device. The electronic device controls a robot to automatically begin exploring and mapping based on LiDAR, identifying obstacles in the map and generating a store map. Non-wall-mounted shelves, due to their clear boundaries, can be directly included in the map inspection range. Furthermore, if a wall-mounted shelf is misidentified as a wall in the map, it can be manually configured. Of course, besides wall-mounted shelves, it can also be any area; the user can manually edit the map.

[0068] See Figure 5 , Figure 5 This is a schematic diagram of a map editing method provided in an embodiment of this application. Workers access the shelf configuration page via an app on their terminal device. This page is a floating window design and can be moved freely. Using this shelf configuration component, rectangles are manually drawn at corresponding locations on the map to determine the shelf positions and dimensions. The shelf position coordinates are accurate to 0.1m, for example, X=9.2m, Y=3.4m. The dimensions include width W and height H, for example, W=2.0m, H=3.3m. See [link to relevant documentation]. Figure 6 , Figure 6This is a schematic diagram illustrating the marking of valid inspection surfaces on a shelf, as provided in an embodiment of this application. Based on the actual shelf structure, such as a cuboid shelf with only one side facing the aisle requiring inspection, staff can specify the valid inspection surfaces (e.g., surfaces A / C) on the app, excluding non-inspection surfaces (e.g., surfaces B / D), ensuring the robot only inspects the valid surfaces and avoiding invalid inspections.

[0069] After completing the map, click the "Save" button to complete the configuration process and set the configuration area for the electronic device at the corresponding location on the map. This modified map can be used in subsequent inspection tasks; if the store shelf location is adjusted, the "Rebuild Map" function can be used to re-execute the exploration and mapping steps to update the map data.

[0070] As can be seen, this embodiment solves the problem of identifying wall-mounted shelves and achieves full coverage of the inspection range. Through the mechanism of "automatic mapping combined with precise manual completion," it effectively distinguishes wall-mounted shelves from the wall boundary, solving the problem of misjudging wall-mounted shelves by robots and other equipment during mapping. This allows all types of shelves in the store to be included in the inspection range, increasing the inspection coverage to 100% and eliminating the errors and missed inspection risks caused by blind manual labeling.

[0071] Furthermore, in this embodiment of the application, if it is the first inspection and there are no historical standards, the process of building shelf standard information can be skipped, and the initial standards can be generated through subsequent inspection tasks; if it is necessary to fix the shelf display rules in advance, the following operations are performed based on the scanning capabilities of the intelligent robot: the background preset standards, and the standard settings for robot scanning and manual verification.

[0072] Regarding the preset standards in the background, one possible implementation of this application embodiment is that the process of constructing shelf standard information includes: in response to the commodity standard input information triggered by the target position of the second target shelf of the target store, configuring the shelf standard information of the second target shelf, wherein the second target shelf is any one of the multiple shelves of the target store.

[0073] Staff can log into the cloud-based backend management system via terminal devices, enter the "Standard Shelf Configuration" module, and quickly switch stores using the top filter bar; switch shelves, such as locating specific shelf groups like the "Beverage Section"; and switch shelf positions, such as selecting "A-side 01 facing the entrance." The map is pre-set for users to reference the actual location of the currently edited shelf on the map.

[0074] For the selected target shelves and locations, the following configurations can be performed: Basic information editing: Click "Edit Shelf and Location Name" to customize the name of the shelves and locations; click "Edit Shelf Rows" to set the shelf hierarchy, such as 1st floor, 2nd floor, etc. After configuration, click "OK" to save.

[0075] Furthermore, the "Edit whether this shelf position should be inspected" switch allows for flexible control over whether the current shelf position participates in the inspection task; it can also be combined with the "Equal distribution of storage space" function to make fine division of the shelf display space. In particular, to facilitate the management of the shelves, the shelves can be divided into small positions, i.e., storage spaces.

[0076] In this embodiment, SKU entry and association can also be implemented. Specifically, clicking "Edit Product" starts SKU entry, which triggers the entry of standard product information for the target shelf of the second target store. The entered information is used as the shelf standard information for the second target shelf. For example, at the shelf level, such as the 1st and 2nd shelves of "Beverage Section 1", product SKUs are entered according to the dimension of "row – position 1". For example, "a certain soft drink 439ml" is specified for the 1st shelf. The configuration data is automatically synchronized to the cloud database as the shelf standard information for that position, which is then used by inspection tasks. See also Figure 7 , Figure 7 This is a schematic diagram of a shelf management page provided in an embodiment of this application.

[0077] Furthermore, you can click on the image component to display the location of the currently edited shelf on the map. After finishing editing the product, you can click the exit editing component to exit the current editing page.

[0078] Regarding the standard settings for robot scanning and manual verification, one possible implementation of this application embodiment is as follows: in response to a shelf inspection task, the robot is controlled to inspect and obtain images of each location on each shelf in the target store; the product information and product location of each location image on each shelf are identified, and standard shelf information for each shelf is generated based on the product information and product location.

[0079] Furthermore, based on product information and product location, shelf standard information for each shelf is generated, including: generating initial shelf standard information for each shelf based on product information and product location; and generating shelf standard information for each shelf in response to modification operations on the initial shelf standard information.

[0080] Staff control the robot to perform "shelf inspection tasks" through the terminal device's APP. If there are no standards or inspection history in the background, a standard is generated by default. The robot takes pictures of various points on the shelf using three side cameras, and the local detection algorithm automatically identifies the products and generates initial shelf standard information. Staff can also view the shelf management - shelf standard configuration in the terminal device's web / APP, such as "Shelf 1, second layer: a certain juice 439ml". They can modify the incorrectly identified SKUs, and after confirmation, click "Save as official standard" to trigger the modification operation, thereby generating shelf standard information for each shelf and synchronizing the shelf standard information of each shelf to the cloud, i.e., electronic devices.

[0081] Based on the above two methods, by providing two flexible standard shelf configuration modes, namely "back-end preset" and "robot scanning combined with manual verification", the workload and complexity of manually entering SKU information are greatly reduced. Only manual intervention and verification are required at key stages, which is more efficient.

[0082] Furthermore, it can also respond to modifications to specific standard information and modify the shelf standard information. When new products are added or displays are adjusted, two efficient update methods are supported: 1. The robot updates the shelf standards at a certain frequency, such as once a month, during routine inspections; 2. Staff can directly modify the standards of a specific row of a designated shelf through the backend, without reconfiguring the entire shelf, adapting to the needs of high-frequency changes in products.

[0083] Based on the above, in this embodiment, the manual completion operation in the mapping stage is simple and intuitive, and the standard configuration stage supports the generation of a draft using data collected by the robot, which greatly reduces the manpower and time costs of manual configuration from scratch; it supports a standard and efficient update mechanism, such as automatic updates or targeted modifications on a periodic basis, which can flexibly adapt to the needs of high-frequency changes in goods in retail scenarios, avoid the need to repeat a large amount of manual configuration work due to display adjustments, and optimize operating costs; it significantly reduces reliance on manual labor and operating costs, and improves the level of process intelligence.

[0084] Furthermore, regarding the configuration of inspection tasks, this embodiment supports two convenient creation paths to adapt to different operating habits and meet the needs of rapid execution and refined management.

[0085] Thus, in one possible implementation, obtaining the inspection task for the first target shelf includes: in response to a task creation operation triggered by the first target shelf of the target store, obtaining the inspection task for the first target shelf.

[0086] For details, see Figure 8 You can quickly create and start the task from the "GO" button on the homepage. Specifically, on the app's "Map" page, click the "GO" button at the bottom, and the system will pop up a "Start Inspection" configuration pop-up. Select the "Shelf Vacancy Detection" task type, choose the shelf in the store to be inspected and the corresponding robot, and click "Start Now" at the bottom of the pop-up to trigger the task creation operation, obtaining the inspection task for the first target shelf. The robot immediately receives the task instructions and proceeds to the selected shelf to perform the inspection according to the preset logic.

[0087] In another possible implementation, the inspection task for the first target shelf is obtained, including: determining whether the inspection cycle for the first target shelf of the target store has been reached, and if so, obtaining the inspection task for the first target shelf.

[0088] In this embodiment, inspection tasks are created from a task list, supporting scheduled tasks and recurring task configurations. For details, see [link to relevant documentation]. Figure 9 Switch to the "Tasks" tab at the bottom of the app, enter the "All Tasks" list, click the "+" sign in the upper right corner, and select "Create New Task" to enter the task editing page. Select the "Shelf Vacancy Detection" task type, select the shelves in the store that need to be inspected and the corresponding robot, and configure the execution time and repetition rules, such as single / daily / weekly. Expand "Advanced Configuration" to set "Daily Cycle," such as cycling twice to meet periodic inspection needs, such as high-frequency inspections during peak hours. After configuration, click "Save" at the bottom. The task will then be saved to the "All Tasks" list and executed automatically according to the set time / rules. Check if the inspection cycle for the first target shelf in the target store has been reached; if so, the inspection task will be automatically received. Furthermore, created tasks support secondary editing, allowing modification of shelves, robots, execution rules, etc., to flexibly adapt to operational adjustments.

[0089] In this application embodiment, the two paths for creating inspection tasks cover the scenarios of "quick temporary tasks" and "planned periodic tasks", which not only meet the needs of sudden inspections, but also support the standardized and routine inspection process management, such as daily fixed-time full-shelf inspections. Through simple interaction and flexible configuration, the execution efficiency and scenario adaptability of robot inspection tasks are improved.

[0090] One possible implementation of this application, based on the single-point detection results, and after determining the missing product information of the first target shelf according to the shelf standard information and historical detection data, further includes:

[0091] Display out-of-stock alerts on the store map on the map homepage; in response to a user triggering a details view of the area corresponding to the out-of-stock alert, display information on the missing items on the first target shelf.

[0092] Once the task is configured, when the robot detects a shelf that is out of stock, an out-of-stock warning will appear on the map homepage of the client device's app. For example, the "beverage area" will display a red warning box and an out-of-stock sign.

[0093] See Figure 10 When a user clicks on a shelf area marked with an out-of-stock indicator, a "Out-of-Stock Items" pop-up window will appear on the screen. Clicking on the details will trigger a viewing process, displaying information about the missing items on the target shelf. This information includes a list of out-of-stock items for that shelf, allowing store staff to quickly review and replenish stock, ensuring sufficient inventory on store shelves.

[0094] One possible implementation of this application embodiment is to display a shelf view in response to a user triggering a shelf overview operation for the area corresponding to the out-of-stock notification information. The shelf view is displayed as a virtual shelf, and out-of-stock locations in the virtual shelf are marked with out-of-stock indicators. When the out-of-stock indicator is triggered, the missing product information at the out-of-stock location can be displayed.

[0095] Among them, see Figure 11 To view more detailed information about the shelf, click "Shelf Overview" to trigger the shelf overview operation, which will then display the shelf view. Specifically, Shelf Overview: Enter the shelf management page. The system abstracts the shelf goods into geometric objects based on the actual detection results. Through the virtual shelf diagram, it focuses on the actual distribution of goods and out-of-stock locations, helping users quickly focus on core information, such as out-of-stock locations and abnormal points in product distribution, and efficiently complete shelf status verification and problem localization.

[0096] In one possible scenario, clicking into the shelf management module will automatically filter out shelf data under the "Out of Stock" tag. The shelves are displayed according to the shelf area dimensions configured in the robot's backend (e.g., "Beverage Area - Side A - 2"), and the time the record was generated is shown, such as "2025-08-27 16:21:00", along with the number of out-of-stock items detected during inspection, such as "Out of Stock 1 type". The shelf view is displayed by default in a "virtual shelf" view, allowing you to view the shelf hierarchy and zoom in and slide to view product display details. Out-of-stock items will be marked with a red prompt box, and clicking on it will display the out-of-stock details, such as "Out of Stock 1 Type A Juice".

[0097] One possible implementation of this application embodiment is to display a real-shot stitched image of the third target shelf in response to a user triggering an image display operation of the third target shelf. The real-shot stitched image is obtained by stitching together photos taken by the robot at different points.

[0098] The third target shelf may be the same as or different from the first target shelf; this application embodiment does not limit this. See also Figure 12 Click the icon on the right side of the shelf item to trigger the image display operation, and you can view the details of the actual collage of the shelf. The collage is generated by integrating the images taken by the robot from different points, and fully presents the actual display scene of the shelf.

[0099] In one possible implementation of this application, in response to a user triggering a heatmap display operation for the fourth target shelf, a heatmap of the fourth target shelf is displayed, with different blocks of the virtual shelf in the heatmap of the fourth target shelf corresponding to corresponding colors; in response to a user triggering a viewing operation for abnormal colors in the heatmap of the fourth target shelf, a prompt message is displayed, including the number of times the product corresponding to the block is out of stock and product information.

[0100] The fourth target shelf may be the same as or different from the first and third target shelves; this application's embodiments do not limit this. See also Figure 13 When a user clicks "Heatmap," the heatmap display operation is triggered, and the user can switch to the shelf heatmap view. This map is based on the inspection data to count the number of times the product is out of stock on that day. When a user clicks on an area with abnormal color in the heatmap, the system will pop up a bubble prompt to display the number of times the corresponding product is out of stock on that day and the specific product information, such as "A fruit tea, out of stock 1 time on that day."

[0101] Shelf heatmaps use visualization to clearly show the number of times a product is out of stock each day. For store staff, this allows for a quick overview of the distribution and severity of stockouts in different areas of the shelf, eliminating the need to consult data reports line by line. They can then accurately focus on products with high stockout frequency. These frequently out-of-stock items are often best-selling, popular, and high-performing categories, allowing staff to replenish stock promptly and ensure a steady supply of popular items. Simultaneously, this data provides insights for optimizing subsequent shelf displays, such as moving best-selling products to more prominent positions and increasing restocking frequency. This helps stores better meet customer needs and improve inventory turnover efficiency and sales performance.

[0102] One possible implementation of this application, based on the single-point detection results, and after determining the missing product information of the first target shelf according to the shelf standard information and historical detection data, further includes:

[0103] In response to a user-triggered report viewing action on the fifth target shelf, the system displays out-of-stock analysis information for the fifth target shelf, including out-of-stock trends and statistics.

[0104] The fifth target shelf may be the same as or different from the first, third, and fourth target shelves; this embodiment does not limit this. Users can view multi-dimensional reports through either the app or the web robot module on their terminal device; the app and web functions are identical. After a user triggers a report viewing operation, the interface displays the out-of-stock analysis information for the fifth target shelf. The report data is generated based on the robot's inspection records. In one possible scenario, the user can access the shelf management module by clicking the icon on the map homepage in the app, and then access the report statistics module by clicking the data statistics icon in the upper right corner to trigger the report viewing operation. The user can directly view the report in the robot's out-of-stock analysis section on the web.

[0105] Furthermore, see Figure 14 , Figure 15Upon entering the "Out of Stock Analysis" page, you can quickly switch between time dimensions at the top using the "Today / Yesterday / Last 7 Days / Last 30 Days" tabs. The page first displays core data cards, including the number of inspections (e.g., 15 today, with the increase / decrease rate compared to yesterday); the number of out-of-stock shelves (e.g., 3 today, with a comparison to yesterday); and the number of out-of-stock occurrences (e.g., 13 today, with a comparison to yesterday). The "Out of Stock Trend" module displays out-of-stock statistics. Specifically, it can present different shelves or product sections in line chart format, such as "8 / 23 Shelf," "Snacks," "Potato Chips," and "Beverage Section," showing the out-of-stock trend changes within the selected time range. Clicking the "All" dropdown allows you to filter the trend data for a specific shelf or section.

[0106] The "Out-of-Stock Statistics" module displays out-of-stock statistics information. It can show the location of each shelf in detail in a list format, such as "Beverage Section A" and "Beverage Section B"; status, such as normal / out of stock, marked with green and red labels respectively; number of inspections and number of out-of-stock times. Clicking on a shelf item in the list can view more detailed out-of-stock items and historical inspection records for that shelf.

[0107] By providing multi-dimensional data statistics and visualizations on time, shelf space, and product zoning, operations staff can quickly grasp the overall stockout situation in stores, including the time distribution patterns of stockouts, shelves and areas with high-frequency stockouts, etc., which facilitates targeted replenishment and shelf optimization, thereby improving the stability of store product supply and operational efficiency.

[0108] Based on the above, this application embodiment provides intuitive, multi-dimensional result presentation and in-depth data analysis capabilities, empowering refined store management. A cloud-based jigsaw puzzle algorithm generates a visual view of the entire shelf, and combines highlighting, virtual shelves, heatmaps, and other methods to intuitively display the location and distribution of out-of-stock items. This allows staff to quickly locate problems, trace out-of-stock details, and make replenishment operations more targeted. Multi-dimensional statistical analysis reports and trend charts are provided to help managers grasp the patterns of store out-of-stock items from a macro perspective, identify frequently out-of-stock products and areas, and provide solid data support for optimizing replenishment strategies, adjusting product displays, and improving inventory management efficiency, thereby enhancing the system's decision-making support capabilities.

[0109] Furthermore, it not only applies to the robot's existing hardware capabilities, but its system architecture also reserves interfaces for integration with other systems such as store and warehouse management. This lays a solid foundation for future deeper intelligent analysis, such as predicting stockouts based on inventory data, optimizing order recommendations, and expanding functionality. It has excellent application prospects and promotional value, and possesses good scalability and adaptability.

[0110] Based on any of the above embodiments, the present invention provides a robot SKU shelf detection method, belonging to the field of intelligent inspection technology in retail scenarios. (See also...) Figure 16,include:

[0111] Step 11, Map Building and Maintenance: Automatic map building, the robot autonomously explores and generates a map, determines whether there are any misjudged front shelves, if so, manually completes the map, manually marks the shelf with a rectangle and specifies the valid detection surface and saves the map, synchronizing it to the cloud; if not, the map is saved and synchronized to the cloud.

[0112] Step 12, Standard Shelf Configuration. Select the configuration method. If you choose the backend preset, the SKU standards will be manually entered and synchronized to the cloud; if you choose robot scanning combined with manual verification, the robot will generate the initial standards, which will then be manually modified and saved.

[0113] Step 13, Inspection Task Configuration. Select the task creation method. If you select "Start Now," a temporary task will be quickly created. If you select "Scheduled Task," you will set the time and cycle rules. The robot receives the task and begins the inspection.

[0114] Step 14: Processing and Viewing Detection Results. The robot takes pictures of the shelves and detects out-of-stock areas locally. The images and results are uploaded to the cloud. The cloud-based image is stitched together to generate a complete shelf map, which is then mapped to the out-of-stock areas. Out-of-stock SKUs are determined by combining standard and historical data. Visualized displays include out-of-stock alerts, virtual shelves, and heat maps. Multi-dimensional reports are generated, supporting statistics by time, shelf, and product category.

[0115] In summary, the technical means adopted in this application are as follows: through the four core processes of "map creation and maintenance → standard rack configuration → inspection task configuration → inspection results and report viewing", combined with the robot's local detection algorithm and cloud collaborative processing capabilities, the entire rack inspection process is made intelligent.

[0116] 1. During the mapping phase, the system supports autonomous mapping by robots and manual completion of wall-mounted shelves, drawing rectangles to mark shelf locations and configuring shelf faces to be inspected. This addresses the issue that existing robots cannot recognize wall-mounted shelves. While the T1 intelligent robot has basic mapping capabilities, it has not resolved the problem of blurred boundaries between wall-mounted shelves and walls, preventing it from identifying them as objects requiring inspection. The system achieves accurate identification of wall-mounted shelves and assists in manual shelf configuration, resolving the issue of misjudging wall-mounted shelves during mapping by the T1 intelligent robot and improving inspection coverage.

[0117] 2. Shelf standards can be generated through backend presets or robot scanning combined with manual verification; this solves the problem of high labor costs if SKUs are manually configured and entered throughout the process, or the problem of inaccurate standard settings caused by relying entirely on algorithm recognition, thus achieving the goal of determining shelf standards with high efficiency and accuracy.

[0118] 3. Task configuration supports customizing the shelves to be inspected, where the inspection distance and shooting points are generated autonomously by the robot;

[0119] 4. During the detection phase, local product recognition and cloud-based image fusion are used, combined with "adjacent product inference" and "standard matching" to comprehensively determine out-of-stock SKUs. The detection results and visualized mosaic images are then transmitted back and displayed on a per-shelf basis, with multi-dimensional reports presenting the results. This addresses the problems of existing visual algorithm-based out-of-stock judgments being prone to false positives, lacking coordination with shelf standards, resulting in large errors due to the single factor of algorithm accuracy, and the lack of intuitive visualization and multi-dimensional statistical analysis capabilities for inspection results, hindering subsequent traceability and data application. Through a flexible standard configuration mode and dual-scheme out-of-stock judgment, the system reduces manual configuration workload while improving the accuracy of out-of-stock SKU judgment; it supports visualized traceability and multi-dimensional data statistics of inspection results, is compatible with the hardware capabilities of the T1 intelligent robot, and reserves expansion space for future integration with store warehouse management systems.

[0120] The following describes a device provided by an embodiment of this application. The device described below can be referred to in correspondence with the method described above. The device of this embodiment is installed in an electronic device. Figure 17 , Figure 17 This is a structural block diagram of an apparatus according to one embodiment of the present application, including: a task acquisition module 210, used to acquire an inspection task for a first target shelf, wherein the first target shelf is a shelf to be inspected in a target store; a control inspection module 220, used to control a robot to perform out-of-stock inspection on the first target shelf according to the store map of the target store and the inspection task, and obtain single-point detection results; and a shelf detection module 230, used to determine the missing product information of the first target shelf based on the single-point detection results, the shelf standard information of the first target shelf, and historical detection data.

[0121] In one possible implementation, it also includes: a map building module, which, in response to a mapping instruction, controls the robot to explore and map the target shelf to obtain a store map; and, in response to a configuration operation triggered by a shelf configuration component for the store map, sets a configuration area on the store map, wherein the configuration operation includes a name and a location, and if the configuration operation is shelf configuration information, the configuration operation also includes a valid plane.

[0122] In one feasible approach, the shelf standard information construction module is used to: configure the shelf standard information of the second target shelf in response to the commodity standard input information triggered by the target position surface of the second target shelf in the target store, wherein the second target shelf is any one of the multiple shelves in the target store; or, in response to the shelf detection task, control the robot to inspect and obtain images of each position of each shelf in the target store; identify the commodity information and commodity position of each position image of each shelf, and generate the shelf standard information of each shelf based on the commodity information and commodity position.

[0123] In one feasible approach, a shelf standard information construction module is used to: generate initial shelf standard information for each shelf based on product information and product location; and generate shelf standard information for each shelf in response to modification operations on the initial shelf standard information.

[0124] In one possible implementation, the task acquisition module 210 is configured to: obtain an inspection task for the first target shelf in response to a task creation operation triggered by the first target shelf of the target store; or, determine whether the inspection cycle for the first target shelf of the target store has been reached, and if so, obtain the inspection task for the first target shelf.

[0125] In one possible implementation, the system further includes: an out-of-stock notification display module, used to display out-of-stock notification information on the store map on the map homepage; in response to a user triggering a details view operation for the area corresponding to the out-of-stock notification, displaying information on the missing products on the first target shelf; and a shelf view display module, used to display a shelf view in response to a user triggering a shelf overview operation for the area corresponding to the out-of-stock notification, the shelf view being displayed as a virtual shelf, and out-of-stock locations on the virtual shelf displaying out-of-stock indicators; when an out-of-stock indicator is triggered, displaying information on the missing products at the out-of-stock location; and real-life footage. The image stitching display module is used to respond to the user's triggering of the image display operation for the third target shelf, displaying a real-shot stitched image of the third target shelf. The real-shot stitched image is obtained by stitching together photos taken by the robot at different points. The heat map display module is used to respond to the user's triggering of the heat map display operation for the fourth target shelf, displaying a heat map of the fourth target shelf. Different blocks of the virtual shelf in the heat map of the fourth target shelf correspond to corresponding colors. In response to the user's triggering of the viewing operation for abnormal colors in the heat map of the fourth target shelf, a prompt message is displayed, which includes the number of times the corresponding product is out of stock and product information.

[0126] In one possible implementation, it also includes: a report display module, used to respond to a user-triggered report viewing operation on the fifth target shelf, displaying out-of-stock analysis information for the fifth target shelf, including out-of-stock trends and out-of-stock statistics.

[0127] Figure 18 A structural diagram of an electronic device provided in an embodiment of the present invention, such as... Figure 18 As shown, the electronic device includes: a memory 60 for storing a computer program; and a processor 61 for executing the computer program to implement the steps of the method as described in the above embodiments.

[0128] The electronic devices provided in this embodiment may include, but are not limited to, smartphones, tablets, laptops, or desktop computers.

[0129] The processor 61 may include one or more processing cores, such as a quad-core processor or an octa-core processor. The processor 61 may be implemented using at least one hardware form selected from Digital Signal Processing (DSP), Field-Programmable Gate Array (FPGA), and Programmable Logic Array (PLA). The processor 61 may also include a main processor and a coprocessor. The main processor, also known as the Central Processing Unit (CPU), is used to process data in the wake-up state; the coprocessor is a low-power processor used to process data in the standby state. In some embodiments, the processor 61 may integrate a Graphics Processing Unit (GPU), which is responsible for rendering and drawing the content to be displayed on the screen. In some embodiments, the processor 61 may also include an Artificial Intelligence (AI) processor, which handles computational operations related to machine learning.

[0130] The memory 60 may include one or more computer-readable storage media, which may be non-transitory. The memory 60 may also include high-speed random access memory and non-volatile memory, such as one or more disk storage devices or flash memory devices. In this embodiment, the memory 60 is used to store at least the following computer program 601, which, after being loaded and executed by the processor 61, is capable of implementing the relevant steps of the method disclosed in any of the foregoing embodiments. In addition, the resources stored in the memory 60 may also include an operating system 602 and data 603, etc., and the storage method may be temporary storage or permanent storage. The operating system 602 may include Windows, Unix, Linux, etc.

[0131] In some embodiments, the electronic device may further include a display screen 62, an input / output interface 63, a communication interface 64, a power supply 65, and a communication bus 66.

[0132] Those skilled in the art will understand that Figure 18 The structures shown do not constitute a limitation on electronic devices and may include more or fewer components than those shown.

[0133] It is understood that if the methods in the above embodiments are implemented as software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, in essence, or the part that contributes to the current technology, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and executes all or part of the steps of the methods in the various embodiments of the present invention. The aforementioned storage medium includes: USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), electrically erasable programmable ROM, registers, hard disks, removable disks, CD-ROMs, magnetic disks, or optical disks, and other media capable of storing program code.

[0134] Based on this, embodiments of the present invention also provide a computer-readable storage medium storing a computer program, which, when executed by a processor, implements the steps of the method described above.

[0135] Based on this, embodiments of the present invention also provide a computer program product, including a computer program / instructions, which, when executed by a processor, implement the steps of the above-described method. It should be understood that although the steps in the flowcharts of the accompanying drawings are shown sequentially according to the arrows, these steps are not necessarily executed in the order indicated by the arrows. Unless explicitly stated herein, there is no strict order restriction on the execution of these steps, and they can be executed in other orders. Moreover, at least some steps in the flowcharts of the accompanying drawings may include multiple sub-steps or multiple stages. These sub-steps or stages are not necessarily completed at the same time, but can be executed at different times, and their execution order is not necessarily sequential, but can be performed alternately or in turn with other steps or at least a portion of the sub-steps or stages of other steps.

[0136] The above are only some embodiments of this application. It should be noted that for those skilled in the art, several improvements and modifications can be made without departing from the principle of this application, and these improvements and modifications should also be considered within the scope of protection of this application.

Claims

1. A robotic shelf inspection method, characterized in that, include: Obtain the inspection task for the first target shelf, which is the shelf to be inspected in the target store; Based on the store map of the target store, the robot is controlled to perform out-of-stock inspection on the first target shelf according to the inspection task, and the single-point detection result is obtained. Based on the single-point detection results, and according to the shelf standard information and historical detection data of the first target shelf, the missing product information of the first target shelf is determined.

2. The robotic shelf inspection method according to claim 1, characterized in that, Also includes: In response to mapping commands, the robot is controlled to explore and map the target shelves, thus obtaining a store map; In response to a configuration operation triggered by a shelf configuration component for the store map, a configuration area is set on the store map. The configuration operation includes a name and a location. If the configuration operation is shelf configuration information, the configuration operation also includes a valid location.

3. The robotic shelf inspection method according to claim 1, characterized in that, The process of building standard shelving information includes: In response to the product standard entry information triggered by the target position of the second target shelf in the target store, the shelf standard information of the second target shelf is configured, wherein the second target shelf is any one of the multiple shelves in the target store; or, In response to the shelf inspection task, the robot is controlled to conduct inspections and obtain images of each location on each shelf in the target store; the product information and product location of each location on each shelf are identified, and the standard shelf information for each shelf is generated based on the product information and product location.

4. The robotic shelf inspection method according to claim 3, characterized in that, Based on the product information and product location, generate standard shelf information for each shelf, including: Based on the product information and product location, generate initial standard shelf information for each shelf; In response to the modification operation on the initial shelf standard information, shelf standard information for each shelf is generated.

5. The robotic shelf inspection method according to claim 1, characterized in that, Obtain inspection tasks for the first target shelf, including: In response to a task creation operation triggered by a first target shelf in the target store, an inspection task for the first target shelf is obtained; or, Determine whether the inspection cycle for the first target shelf of the target store has been reached. If so, obtain the inspection task for the first target shelf.

6. The robotic shelf inspection method according to claim 1, characterized in that, Based on the single-point detection results, and according to the shelf standard information and historical detection data of the first target shelf, after determining the missing product information of the first target shelf, the method further includes: Display out-of-stock alerts on the store map on the map homepage; in response to a user triggering a details view operation for the area corresponding to the out-of-stock alert, display information on the missing products on the first target shelf; And / or, In response to a user triggering a shelf overview operation for the area corresponding to the out-of-stock notification information, a shelf view is displayed. The shelf view is shown as a virtual shelf, and out-of-stock locations in the virtual shelf are marked with out-of-stock indicators. When the out-of-stock indicator is triggered, the missing product information at the out-of-stock location can be displayed. And / or, In response to the user triggering the image display operation of the third target shelf, a real-shot stitched image of the third target shelf is displayed. The real-shot stitched image is obtained by stitching together photos taken by the robot at different points. And / or, In response to a user triggering the operation to display the heatmap of the fourth target shelf, the heatmap of the fourth target shelf is displayed, and different blocks of the virtual shelf in the heatmap of the fourth target shelf correspond to corresponding colors; in response to a user triggering the operation to view abnormal colors in the heatmap of the fourth target shelf, a prompt message is displayed, the prompt message including the number of times the product corresponding to the block is out of stock and product information.

7. The robotic shelf inspection method according to claim 1, characterized in that, Based on the single-point detection results, and according to the shelf standard information and historical detection data of the first target shelf, after determining the missing product information of the first target shelf, the method further includes: In response to a user-triggered report viewing operation on the fifth target shelf, the out-of-stock analysis information of the fifth target shelf is displayed, including out-of-stock trends and out-of-stock statistics.

8. A robotic shelf inspection device, characterized in that, include: The task acquisition module is used to acquire inspection tasks for the first target shelf, which is the shelf to be inspected in the target store. The inspection control module is used to control the robot to perform out-of-stock inspections on the first target shelf according to the store map of the target store and the inspection task, and obtain single-point detection results. The shelf detection module is used to determine the missing product information of the first target shelf based on the single-point detection results, according to the shelf standard information and historical detection data of the first target shelf.

9. An electronic device, characterized in that, The electronic device includes a memory and a processor, wherein the memory stores a computer program, and the processor executes the robotic shelf inspection method according to any one of claims 1 to 7 when running the computer program.

10. A robotic shelf inspection system, characterized in that, include: Robot, electronic device as described in claim 9, and terminal device; The robot is used to perform out-of-stock inspections on the first target shelf according to the store map and the inspection task, obtain single-point detection results, and send the single-point detection results and single-point images to electronic devices.

Images