Out-of-stock detection method, edge computing device and system for out-of-stock detection on shelf

Abstract

The present application discloses a method for out-of-stock detection, an edge computing device and a system for out-of-stock detection on a shelf, which collects a first shelf image, scans the goods in each grid in the first shelf image to obtain the first product border data, and obtains the first product name data based on all the first product border data to obtain the detection result, and sends the detection result message to the edge device management background so that the edge device management background generates the out-of-stock rate of each grid and marks the goods placed in the wrong grid, thereby increasing the amount of data for parallel out-of-stock processing and improving the efficiency of out-of-stock data processing.

Description

Technical Field

[0001] The present application belongs to the technical field of shelf detection, and in particular, relates to an out-of-stock detection method, an edge computing device and a system for out-of-stock detection on shelf arrays. Background Art

[0002] In recent years, smart new retail, which is closely related to people's lives, has developed rapidly. It uses technologies such as the Internet, Internet of Things, big data, and artificial intelligence to enable digital and intelligent management of supermarkets, convenience stores, etc., while optimizing the relationship between products, users, and payments, giving customers a faster, better, and more convenient shopping experience.

[0003] Digital shelves are an important part of the smart new retail business. Currently, the goods detection on the shelves is generally done through unified data processing in the background to determine the goods situation on the shelves, which leads to the problem of large backend data processing volume and slow processing speed. Summary of the invention

[0004] The purpose of this application is to provide a method for out-of-stock detection, aiming to solve the problem of slow processing speed in traditional shelf goods detection.

[0005] A first aspect of an embodiment of the present application provides a method for detecting out-of-stock, comprising:

[0006] Collect a first shelf image according to the out-of-stock detection task, scan the commodities in each shelf in the first shelf image to obtain first commodity border data, and obtain first commodity name data according to all the first commodity border data to obtain a detection result;

[0007] The detection result message is sent to the edge device management background so that the edge device management background generates the out-of-stock rate of each of the sheds and marks the wrong sheds for the goods.

[0008] In one embodiment, the out-of-stock detection method further includes:

[0009] Generate standard shelf layout and send it to the edge device management backend;

[0010] The edge device management backend generates the out-of-stock rate of the grid and marks the grid where the goods are placed incorrectly based on the detection result and the standard shelf layout.

[0011] In one embodiment, generating a standard shelf layout includes:

[0012] Collecting a second shelf image according to the row detection task, scanning the commodity in the second shelf image to obtain second commodity frame data, and obtaining second commodity name data according to the second commodity frame data;

[0013] The grid is drawn based on the same second product name data aggregation, and the standard shelf layout is generated.

[0014] A second aspect of an embodiment of the present application provides an edge computing device for out-of-stock detection on a shelf, including:

[0015] The main control unit is used to generate out-of-stock detection tasks at a regular interval;

[0016] An image acquisition unit, used for acquiring a first shelf image;

[0017] a digital shelf algorithm unit, configured to scan the commodities in each shelf grid in the first shelf image to obtain first commodity frame data, and to identify and obtain first commodity name data according to all the first commodity frame data to obtain a detection result; and

[0018] The communication unit is used to send the detection result message to the edge device management background.

[0019] In one embodiment, it is characterized in that

[0020] The main control unit is also used to generate row face detection tasks at a regular interval;

[0021] The image acquisition unit is further used to acquire the second shelf image;

[0022] The digital shelf algorithm unit is further used to scan the product in the second shelf image to obtain the second product border data, obtain the second product name data according to the second product border data, aggregate and draw the shelf grid based on the same second product name data, generate a standard shelf row and send it to the edge device management backend;

[0023] The communication unit is also used to send the standard shelf layout to the edge device management background.

[0024] In one embodiment, it further includes a data storage unit;

[0025] The data storage unit is used to store the execution steps of the digital shelf algorithm unit, the first shelf image, the second shelf image, the first product border data, the second product border data, the first product name data, the second product name data, the grid data, the standard shelf layout and the detection results.

[0026] In one embodiment, an online upgrade unit is also included;

[0027] The online upgrade unit is used to download and check the upgrade file.

[0028] In one embodiment, the communication unit is further used to receive upgrade task information issued by the edge device management background;

[0029] The main control unit is also used to parse the upgrade task information and restart to complete the upgrade after the online upgrade unit downloads and verifies the upgrade file.

[0030] In one embodiment, it also includes an expandable algorithm unit;

[0031] The extensible algorithm unit is used for function expansion.

[0032] A third aspect of the embodiments of the present application provides an edge computing system for out-of-stock detection on a shelf, including an edge device management backend, a message proxy middleware, and a plurality of edge computing devices for out-of-stock detection on a shelf as described in any one of the second aspects;

[0033] The edge device management background communicates with the plurality of edge computing devices for out-of-stock detection on the shelf rows through the message proxy middleware.

[0034] Compared with the prior art, the present application has the following beneficial effects: by collecting the first shelf image according to the out-of-stock detection task, scanning the goods in each grid in the first shelf image to obtain the first product border data, and obtaining the first product name data according to all the first product border data to obtain the detection result, the detection result message is sent to the edge device management background so that the edge device management background generates the out-of-stock rate of each grid and marks the goods in the wrong grid, which truly realizes the nearest service of data collection, data cleaning, and algorithm reasoning, increases the amount of data processed in parallel, and improves the efficiency of data processing. In the present application, each edge computing device monomer for out-of-stock detection on a shelf is an independent system, which has many advantages such as easy deployment, easy expansion, advancement, stability, accuracy, and online real-time upgrade, and can meet the personalized functional requirements of users of different scales and organizations. BRIEF DESCRIPTION OF THE DRAWINGS

[0035] Figure 1 A first method flow chart of the out-of-stock detection method provided in an embodiment of the present application;

[0036] Figure 2 A second method flow chart of the out-of-stock detection method provided in an embodiment of the present application;

[0037] Figure 3 A first exemplary principle block diagram of an edge computing device for out-of-stock detection on a shelf provided in an embodiment of the present application;

[0038] Figure 4 A second exemplary principle block diagram of an edge computing device for out-of-stock detection on a shelf provided in an embodiment of the present application;

[0039] Figure 5 An example principle block diagram of an edge computing system for out-of-stock detection on a shelf provided in an embodiment of the present application;

[0040] Figure 6 A first installation diagram of an edge computing device for out-of-stock detection on a shelf provided in an embodiment of the present application;

[0041] Figure 7 A second installation diagram of the edge computing device for out-of-stock detection on shelf rows provided in an embodiment of the present application. DETAILED DESCRIPTION

[0042] In order to make the technical problems, technical solutions and beneficial effects to be solved by the present application more clearly understood, the present application is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application.

[0043] In the following description, specific details such as specific system structures, technologies, etc. are provided for the purpose of illustration rather than limitation, so as to provide a thorough understanding of the embodiments of the present application. However, it should be clear to those skilled in the art that the present application may also be implemented in other embodiments without these specific details. In other cases, detailed descriptions of well-known systems, devices, circuits, and methods are omitted to prevent unnecessary details from obstructing the description of the present application.

[0044] It should be understood that when used in the present specification and the appended claims, the term "comprising" indicates the presence of described features, integers, steps, operations, elements and / or components, but does not exclude the presence or addition of one or more other features, integers, steps, operations, elements, components and / or combinations thereof.

[0045] In addition, in the description of the present application specification and the appended claims, the terms "first", "second", "third", etc. are only used to distinguish the descriptions and cannot be understood as indicating or implying relative importance.

[0046] References to "one embodiment" or "some embodiments" etc. described in the specification of this application mean that one or more embodiments of the present application include specific features, structures or characteristics described in conjunction with the embodiment. Therefore, the statements "in one embodiment", "in some embodiments", "in some other embodiments", "in some other embodiments", etc. that appear in different places in this specification do not necessarily refer to the same embodiment, but mean "one or more but not all embodiments", unless otherwise specifically emphasized in other ways. The terms "including", "comprising", "having" and their variations all mean "including but not limited to", unless otherwise specifically emphasized in other ways.

[0047] Figure 1 The first method flow chart of the out-of-stock detection method provided by the embodiment of the present application is shown. For the convenience of explanation, only the part related to the present embodiment is shown, which is described in detail as follows:

[0048] The out-of-stock detection method includes the following steps:

[0049] Step S01. According to the out-of-stock detection task, a first shelf image is collected, and the commodities in each shelf grid in the first shelf image are scanned to obtain first commodity frame data, and first commodity name data is obtained based on all first commodity frame data to obtain a detection result.

[0050] In the above step S01, the edge computing device 100 generates an out-of-stock detection task based on its own timed generation or receives an out-of-stock detection task sent by the edge device management background 300, and then collects images of the shelf according to the out-of-stock detection task to obtain a first shelf image, and then scans the goods in each shed in the first shelf image to obtain the first product border data corresponding to the goods in each shed, and obtains the first product name data corresponding to the first product border data according to the first product border data, and then generates a detection result according to the first product name data. In step S01, the scanning detection of the shelf and the data statistics of the goods on the shelf are completed, so that the edge computing device 100 processes the goods on the shelf first. Therefore, when the number of shelves is large, each edge computing device 100 processes the goods on the shelf it is responsible for, so all edge computing devices 100 can process data at the same time. Compared with the traditional method of aggregating data to the background for unified processing by the background, the efficiency of data processing is improved, and the data processing of the goods on the shelf is classified according to each shed, which can also provide data for subsequent product misplacement analysis.

[0051] The first shelf image may correspond to an image of one shelf or images of multiple shelves, depending on the performance requirements and / or design needs of the actual device for acquiring image data.

[0052] Step S02: Send the detection result message to the edge device management background 300 so that the edge device management background 300 generates the out-of-stock rate of each shed and marks the goods in the wrong shed.

[0053] In the above step S02, the edge computing device 100 sends the detection result to the edge device management background 300. The edge device management background 300 generates the out-of-stock rate of each shelf in the shelf based on the detection result analysis and marks the shelves where the goods are misplaced, so that the manager can obtain the information of the goods on the shelf.

[0054] Among them, after marking the goods placed in the wrong shed, the goods placed in the wrong shed can also be marked.

[0055] See also Figure 2 In one embodiment, the out-of-stock detection method further includes the following steps:

[0056] Step 1. Generate standard shelf layout.

[0057] Step S23: Send the standard shelf layout to the edge device management backend 300.

[0058] The edge device management backend 300 generates the out-of-stock rate of the shelf grid and marks the wrong shelf grid for the goods based on the detection results and the standard shelf layout.

[0059] In this embodiment, the edge computing device 100 generates a standard shelf layout and sends the standard shelf layout to the edge device management background 300, so that the edge device management background 300 compares the detection results with the standard shelf layout to obtain the out-of-stock rate of the shelf grid and mark the goods in the wrong grid. Generating the standard shelf layout by the edge computing device 100 saves the workload of manually entering the standard shelf layout.

[0060] See also Figure 2 In one embodiment, generating a standard shelf layout includes the following steps:

[0061] Step S21. According to the display detection task, a second shelf image is collected, the commodities in the second shelf image are scanned to obtain second commodity frame data, and second commodity name data is obtained according to the second commodity frame data.

[0062] In the above step S21, the edge computing device 100 generates an array detection task based on its own timing or receives an array detection task sent by the edge device management background 300, and then performs image acquisition on the shelf according to the array detection task to obtain a second shelf image, and then scans the goods in the second shelf image to obtain the second product border data of the goods in the shelf, and obtains the second product name data corresponding to the second product border data based on the second product border data.

[0063] Step S22: Aggregate and draw a shelf grid based on the same second product name data, and generate a standard shelf layout.

[0064] In the above step S22, the edge computing device 100 draws the grid based on the aggregation of the same second product name data. The same type of products will be placed in one area. Therefore, the second product name data corresponding to the scanned products in the area are the same. Therefore, the area where the same type of products are placed is divided into one grid, and then the standard shelf layout is generated according to the second product name data and the grid data.

[0065] See also Figure 3The embodiment of the present application also provides an edge computing device 100 for out-of-stock detection on shelf surfaces. The edge computing device 100 for out-of-stock detection on shelf surfaces includes a main control unit 110, an image acquisition unit 120, a digital shelf algorithm unit 140, and a communication unit 130.

[0066] The main control unit 110 is used to generate out-of-stock detection tasks at a regular interval.

[0067] The image acquisition unit 120 is used to acquire a first shelf image.

[0068] The digital shelf algorithm unit 140 is used to scan the commodities in each shelf in the first shelf image to obtain first commodity frame data, and to obtain first commodity name data according to all first commodity frame data to obtain a detection result.

[0069] The communication unit 130 is used to send a detection result message to the edge device management background 300.

[0070] In this embodiment, the main control unit 110 generates out-of-stock detection tasks at regular intervals. When generating the out-of-stock detection tasks, the image acquisition unit 120 is driven to collect the first shelf image of the shelf. The digital shelf algorithm unit 140 is used to scan and obtain the first commodity border data in each grid in the first shelf image, and to identify and obtain the corresponding first commodity name data based on the first commodity border data, so as to obtain the detection result based on the first commodity name data, and send the detection result to the edge device management background 300 through the communication unit 130.

[0071] The image acquisition unit 120 includes an image sensor and an image driver, wherein the image sensor and the image driver can be integrated into a camera.

[0072] Among them, the main control unit 110 is also used for task scheduling and information interaction management to ensure the stable and orderly operation of the edge computing device 100.

[0073] In one embodiment, the main control unit 110 is also used to generate array detection tasks at a regular interval.

[0074] The image acquisition unit 120 is further configured to acquire the second shelf image.

[0075] The digital shelf algorithm unit 140 is also used to scan the goods in the second shelf image to obtain the second product border data, and obtain the second product name data based on the second product border data, aggregate and draw the shelf grid based on the same second product name data, and generate a standard shelf layout and send it to the edge device management background 300.

[0076] The communication unit 130 is also used to send the standard shelf layout to the edge device management background 300.

[0077] In this embodiment, the main control unit 110 generates a display detection task at regular intervals. When generating the display detection task, the image acquisition unit 120 is driven to capture images of the shelf to obtain a second shelf image. The digital shelf algorithm unit 140 scans the products in the second shelf image to obtain second product border data of the products in the shelf, and obtains second product name data corresponding to the second product border data based on the second product border data, and aggregates and draws a grid based on the same second product name data, and generates a standard shelf display, and sends the standard shelf display to the edge device management background 300 through the communication unit 130.

[0078] Each edge computing device 100 for out-of-stock detection on each shelf in the present application is an independent system, which has many advantages such as easy deployment, easy expansion, advancement, stability, accuracy, and online real-time upgrade, and can meet the personalized functional requirements of users of different scales and organizations.

[0079] See also Figure 4 In one embodiment, the edge computing device 100 for shelf out-of-stock detection also includes a data storage unit 150.

[0080] The data storage unit 150 is used to store the execution steps of the digital shelf algorithm unit 140, the first shelf image, the second shelf image, the first product border data, the second product border data, the first product name data, the second product name data, the shelf grid data, the standard shelf layout and the detection results.

[0081] The data storage unit 150 is used to store the execution steps of the digital shelf algorithm unit 140, the first shelf image, the second shelf image, the first product border data, the second product border data, the first product name data, the second product name data, the grid data, the standard shelf layout and the detection results. It is also used to store information such as the application and system configuration of the main control unit 110 to ensure that the edge computing device 100 for out-of-stock detection on the shelf layout has a memory function and can continue the configuration work after power failure and restart.

[0082] See also Figure 4 In one embodiment, the edge computing device 100 for shelf out-of-stock detection also includes an online upgrade unit 160.

[0083] The online upgrade unit 160 is used to download and check the upgrade file.

[0084] In one embodiment, the communication unit 130 is further configured to receive upgrade task information issued by the edge device management backend 300 .

[0085] The main control unit 110 is also used to parse the upgrade task information and restart to complete the upgrade after the online upgrade unit 160 downloads and verifies the upgrade file.

[0086] In this embodiment, the edge device management background 300 publishes the upgrade task information, the communication unit 130 of the edge computing device 100 receives the upgrade task information, and then the main control unit 110 parses the upgrade task information, controls the online upgrade unit 160 to download the upgrade file and perform verification, and then controls the edge computing device 100 to restart to implement the upgrade, thereby realizing the remote download of the upgrade file by the edge computing device 100, ensuring that the edge computing device 100 has the update and iteration function to meet the needs of continuous upgrades after one deployment.

[0087] See also Figure 4 In one embodiment, the edge computing device 100 for shelf out-of-stock detection also includes an expandable algorithm unit 170.

[0088] The expandable algorithm unit 170 is used for function expansion.

[0089] In this embodiment, the extensible algorithm unit 170 can realize functional expansion, such as adding passenger flow statistics function, face recognition function, etc., so that functional expansion can be carried out later as needed.

[0090] See also Figure 5 The embodiment of the present application also provides an edge computing system for out-of-stock detection on shelf surfaces, including an edge device management backend 300, a message proxy middleware 200, and a plurality of edge computing devices for out-of-stock detection on shelf surfaces as in the above embodiments (indicated by reference numerals 100, 101...10n in the figure).

[0091] The edge device management backend 300 communicates with multiple edge computing devices 100 for out-of-stock detection on shelf rows through the message proxy middleware 200.

[0092] By having multiple edge computing devices 100 subscribe to messages published by the edge device management background 300 through the message proxy middleware 200, and the edge device management background 300 subscribes to messages published by multiple edge computing devices 100 through the message proxy middleware 200, information interaction between the edge device management background 300 and multiple edge computing devices 100 is realized.

[0093] Among them, the number of edge computing devices 100 is set according to actual needs and is not limited here.

[0094] See also Figure 6When the edge computing device 100 is used in a double-row shelf, the edge computing device 100 can be set on the top of the shelf opposite to the shelf, so that the image acquisition area of ​​the edge computing device 100 can cover the shelf, so that the edge computing device 100 can acquire the first shelf image and the second shelf image of the shelf.

[0095] See also Figure 7 When the edge computing device 100 is applied to a single-row shelf, the edge computing device 100 can be hung on the ceiling so that the image acquisition area of ​​the edge computing device 100 can cover the shelf, so that the edge computing device 100 can acquire the first shelf image and the second shelf image of the shelf.

[0096] In the above embodiments, the description of each embodiment has its own emphasis. For parts that are not described or recorded in detail in a certain embodiment, reference can be made to the relevant descriptions of other embodiments.

[0097] Those of ordinary skill in the art will appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of this application.

[0098] In the embodiments provided in the present application, it should be understood that the disclosed devices / network equipment and methods can be implemented in other ways. For example, the device / network equipment embodiments described above are only schematic. For example, the division of modules or units is only a logical function division. There may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms.

[0099] The above embodiments are only used to illustrate the technical solutions of the present application, rather than to limit them. Although the present application has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. These modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present application, and should all be included in the protection scope of the present application.

Claims

1. A method for detecting out-of-stock, characterized in that: include: Collect a first shelf image according to the out-of-stock detection task, scan the commodities in each shelf in the first shelf image to obtain first commodity border data, and obtain first commodity name data according to all the first commodity border data to obtain a detection result; Sending the detection result message to the edge device management background so that the edge device management background generates the out-of-stock rate of each shed and marks the shed where the goods are placed in the wrong place; Generate standard shelf layout and send it to the edge device management backend; The edge device management backend generates the out-of-stock rate of the shelf and marks the wrong shelf for the commodity according to the detection result and the standard shelf arrangement; The step of generating a standard shelf layout includes: Collecting a second shelf image according to the row detection task, scanning the commodity in the second shelf image to obtain second commodity frame data, and obtaining second commodity name data according to the second commodity frame data; The grid is drawn based on the same second product name data aggregation, and the standard shelf layout is generated.

2. An edge computing device for out-of-stock detection on a shelf, characterized in that: include: The main control unit is used to generate out-of-stock detection tasks at a regular interval; An image acquisition unit, used for acquiring a first shelf image; A digital shelf algorithm unit, configured to scan the commodities in each shelf grid in the first shelf image to obtain first commodity border data, and to obtain first commodity name data according to all the first commodity border data to obtain a detection result; as well as A communication unit, used to send the detection result message to the edge device management background; The main control unit is also used to generate row face detection tasks at a regular interval; The image acquisition unit is further used to acquire the second shelf image; The digital shelf algorithm unit is further used to scan the product in the second shelf image to obtain the second product border data, obtain the second product name data according to the second product border data, aggregate and draw the shelf grid based on the same second product name data, generate a standard shelf row and send it to the edge device management background; The communication unit is also used to send the standard shelf layout to the edge device management background.

3. The edge computing device for out-of-stock detection on a shelf as claimed in claim 2, characterized in that: Also included is a data storage unit; The data storage unit is used to store the execution steps of the digital shelf algorithm unit, the first shelf image, the second shelf image, the first product border data, the second product border data, the first product name data, the second product name data, the grid data, the standard shelf layout and the detection results.

4. The edge computing device for out-of-stock detection on a shelf as claimed in claim 2, characterized in that: Also includes online upgrade unit; The online upgrade unit is used to download and check the upgrade file.

5. The edge computing device for out-of-stock detection on a shelf as claimed in claim 4, characterized in that: The communication unit is also used to receive the upgrade task information issued by the edge device management background; The main control unit is also used to parse the upgrade task information and restart to complete the upgrade after the online upgrade unit downloads and verifies the upgrade file.

6. The edge computing device for out-of-stock detection on a shelf as claimed in claim 2, characterized in that: Also includes an extensible algorithm unit; The extensible algorithm unit is used for function expansion.

7. An edge computing system for out-of-stock detection on shelf rows, characterized in that: It includes an edge device management backend, a message proxy middleware, and a plurality of edge computing devices for shelf out-of-stock detection according to any one of claims 2 to 6; The edge device management background communicates with the plurality of edge computing devices for out-of-stock detection on the shelf rows through the message proxy middleware.

Images