Shelf layering method and device

Abstract

This application discloses a shelf stratification method and device, belonging to the field of smart retail technology. The shelf stratification method includes: acquiring shelf images; and stratifying the shelves using one or more of a product display detection method, a product price tag detection method, and a strip screen detection method. The shelf stratification is based on the product display, product price tag, and strip screen, avoiding over-reliance on the cross-sectional characteristics of the shelf layers in actual scenes. Accurate shelf stratification can still be achieved even when the shelf layers have no obvious outlines, are arranged too closely, are too thin, or are affected by lighting.

Description

Technical Field

[0001] The present application belongs to the field of smart retail technology, and specifically relates to a shelf stratification method and device. Background Art

[0002] With the rapid development of artificial intelligence (AI), smart retail is also rapidly developing, supported by AI technology. A range of AI devices, such as smart robots and smart cameras, are beginning to flood into large supermarkets, providing a variety of visual and intelligent solutions, such as shelf out-of-stock detection, shelf display supervision, shelf product placement recommendations, and product sales reporting.

[0003] Shelf visualization is essential in a range of intelligent solutions. Shelf visualization allows for real-time monitoring of shelf placement, sales, and data analysis. To enhance the efficiency of shelf visualization, tiering is crucial. Tiered shelf design not only enhances the visual experience but also reminds sales staff to promptly restock out-of-stock items on specific shelves and levels.

[0004] The current existing shelf stratification method is to first detect the cross section of the shelf layer, and then use the position of the cross section as the position of the layer, and finally achieve shelf stratification.

[0005] During the research process, the inventors found that the existing shelf layering technology currently has at least the following problems: the characteristics of the shelf layer cross-section in actual scenes determine the difficulty of layer detection. However, supermarket shelf cross-sections generally have the following problems in actual scenes: (1) the overall color of the layer is dark, the cross-section is integrated with the layer as a whole, and the outermost cross-section rectangle has no obvious border; (2) the shelves are arranged closely together, resulting in a close connection between adjacent layers, and there is no obvious distinguishing feature between adjacent layers; (3) the shelf layers are too thin, and during the image data collection process, no obvious outer cross-sectional features can be presented; (4) due to the influence of the internal environment of the supermarket, the layer cross-section may be reflected by the light, and the light may cast a shadow on the layer cross-section after the goods are illuminated, or there may even be a problem of goods blocking the layer cross-section. The above series of problems will affect the effect of layer detection. Summary of the Invention

[0006] The purpose of the embodiments of the present application is to provide a shelf stratification method and device that can solve the technical problem that the current existing technology relies too much on the characteristics of the cross-section of the shelf layer in actual scenarios, which affects the accuracy of shelf stratification when the shelf layers have no obvious contours, are arranged too closely, are too thin, or are affected by lighting.

[0007] In order to solve the above technical problems, this application is implemented as follows:

[0008] In a first aspect, an embodiment of the present application provides a shelf layering method, comprising:

[0009] S101: Acquire shelf image;

[0010] S102: The shelves are layered by using one or more of a product arrangement detection method, a product price tag detection method, and a bar screen detection method.

[0011] Furthermore, the shelves are layered using the commodity arrangement detection method, specifically including:

[0012] S201: Marking all products on the shelf with detection frames using a product arrangement detection algorithm;

[0013] S202: Calculate the vertical coordinate of the center point of each product detection frame;

[0014] S203: Select a target product, calculate the difference between the vertical coordinates of the center points of the target product detection frame and the remaining product detection frames, and take the absolute value of the difference;

[0015] S204: If the absolute value of the difference between the vertical coordinates of the center points of a certain product detection frame and the target product detection frame is less than a first threshold, determining that the product and the target product are on the same shelf layer;

[0016] S205: Select the next target product and repeat the above steps S203 and S204 until all products are classified into corresponding shelf layers;

[0017] S206: Sort the shelf layers in ascending order based on the minimum value of the vertical coordinate of the center point of each product detection frame in each shelf layer, and mark them as the first layer, the second layer, ..., the Nth layer in sequence;

[0018] S207: Determine a layer dividing line for each shelf layer according to the minimum value of the lower right corner vertical coordinate of each commodity detection frame in each shelf layer;

[0019] S208: The shelves are divided into layers according to the layer dividing lines.

[0020] Furthermore, the shelves are layered using the commodity price tag detection method, specifically including:

[0021] S301: Using a product price tag detection algorithm, mark all price tags on the shelf with detection frames;

[0022] S302: Calculate the vertical coordinate of the center point of each price tag detection frame;

[0023] S303: Select a target price tag, calculate the difference between the vertical coordinates of the center points of the target price tag detection frame and the remaining price tag detection frames, and take the absolute value of the difference;

[0024] S304: If the absolute value of the difference between the vertical coordinates of the center points of a certain price tag detection frame and the target price tag detection frame is less than a second threshold, determine that the price tag and the target price tag are on the same shelf layer;

[0025] S305: Select the next target price tag and repeat steps S303 and S304 until all price tags are classified into corresponding shelf layers;

[0026] S306: Sort the shelf layers in ascending order based on the minimum value of the vertical coordinate of the center point of each price tag detection frame in each shelf layer, and mark them as the first layer, the second layer, ..., the Mth layer in sequence;

[0027] S307: Determine the layer dividing line of each shelf layer according to the minimum value of the vertical coordinate of the center point of each price tag detection frame in each shelf layer;

[0028] S308: The shelves are divided into layers according to the layer dividing lines.

[0029] Furthermore, the shelves are layered using the strip screen detection method, specifically including:

[0030] S401: Using a strip screen detection algorithm, mark all strip screens on the shelf with detection frames;

[0031] S402: Calculating the vertical coordinates of the center points of each strip screen detection frame;

[0032] S403: Sort the shelves in ascending order based on the vertical coordinates of the center points of the bar screen detection frames in each shelf layer, marking them as the first layer, the second layer, ..., the Pth layer in sequence;

[0033] S404: Determine the layer dividing line of each shelf layer according to the vertical coordinate of the center point of each strip screen detection frame in each shelf layer;

[0034] S405: The shelves are divided into layers according to the layer dividing lines.

[0035] Furthermore, the shelves are layered by combining the product arrangement detection method and the product price tag detection method, specifically including:

[0036] S501: Through steps S201 to S208, the shelf is layered using the commodity arrangement detection method alone, and the vertical coordinates of the layer dividing lines of each shelf layer are recorded in a first list in ascending order;

[0037] S502: Through steps S301 to S308, the shelf is layered using the product price tag detection method alone, and the vertical coordinates of the layer dividing lines of each shelf layer are recorded in a second list in ascending order;

[0038] S503: When the number of shelf layers in the first list is consistent with the number of shelf layers in the second list, setting the layer dividing lines according to the commodity price tag detection method;

[0039] S504: When the number of shelf layers in the first list is inconsistent with the number of shelf layers in the second list, matching processing is performed on the second list based on the first list; when the absolute value of the difference between the basic value in the first list and the matching value in the second list is less than a third threshold value, it is determined that the basic value and the matching value are matched, and the shelf layer corresponding to the basic value and the shelf layer corresponding to the matching value are the same shelf layer; the shelf layer corresponding to the basic value that has not been matched and the shelf layer corresponding to the matching value that has not been matched are treated as separate shelf layers;

[0040] S505: If a layer dividing line obtained by the product price tag detection method exists on a certain shelf layer, the layer dividing line obtained by the product price tag detection method is used as the final layer dividing line; otherwise, the layer dividing line obtained by the product display detection method is used as the final layer dividing line.

[0041] Furthermore, the shelves are layered by combining the commodity arrangement detection method and the strip screen detection method, specifically including:

[0042] S601: Through steps S201 to S208, the shelf is layered using the commodity arrangement detection method alone, and the vertical coordinates of the layer dividing lines of each shelf layer are recorded in a third table in ascending order;

[0043] S602: Through steps S401 to S405, the shelves are layered using the strip screen detection method alone, and the vertical coordinates of the layer dividing lines of each shelf layer are recorded in a fourth table in ascending order;

[0044] S603: When the number of shelf layers in the third list is consistent with the number of shelf layers in the fourth list, setting the layer dividing lines according to the strip screen detection method;

[0045] S604: When the number of shelf layers in the third list is inconsistent with the number of shelf layers in the fourth list, matching processing is performed on the fourth list based on the third list; when the absolute value of the difference between the basic value in the third list and the matching value in the fourth list is less than a fourth threshold value, it is determined that the basic value and the matching value are matched, and the shelf layer corresponding to the basic value and the shelf layer corresponding to the matching value are the same shelf layer; the shelf layer corresponding to the basic value that has not been matched and the shelf layer corresponding to the matching value that has not been matched are treated as separate shelf layers;

[0046] S605: If a layer dividing line obtained by the strip screen detection method exists on a certain shelf layer, the layer dividing line obtained by the strip screen detection method is used as the final layer dividing line; otherwise, the layer dividing line obtained by the product display detection method is used as the final layer dividing line.

[0047] In a second aspect, an embodiment of the present application provides a shelf layering device, comprising:

[0048] An acquisition module, used to acquire shelf images;

[0049] The stratification module is used to stratify the shelves by using one or more of a product arrangement detection method, a product price tag detection method, and a bar screen detection method.

[0050] Furthermore, the layered module specifically includes:

[0051] A first marking submodule is configured to mark all commodities on the shelf with a detection frame using a commodity arrangement detection algorithm;

[0052] The first calculation submodule is used to calculate the vertical coordinate of the center point of each product detection frame;

[0053] A first difference submodule is configured to select a target product, calculate the difference in the vertical coordinates of the center points between the target product detection frame and the remaining product detection frames, and take the absolute value of the difference;

[0054] a first determining submodule, configured to determine that a product and the target product are on the same shelf layer when the absolute value of the difference between the vertical coordinates of the center points of a product detection frame and the target product detection frame is less than a first threshold;

[0055] A first classification submodule is used to select the next target product and call the first difference submodule and the first determination submodule until all products are classified into corresponding shelf layers;

[0056] The first sorting submodule is used to sort the shelf layers in ascending order according to the minimum value of the vertical coordinate of the center point of each product detection frame in each shelf layer, and mark them as the first layer, the second layer, ..., the Nth layer in sequence;

[0057] The first division submodule is used to determine the layer division line of each shelf layer according to the minimum value of the lower right corner vertical coordinate of each commodity detection frame in each shelf layer;

[0058] The first stratification submodule is configured to stratify the shelves according to the layer dividing lines.

[0059] Furthermore, the layering module further includes:

[0060] A second marking submodule is configured to mark all price tags on the shelf with a detection frame using a product price tag detection algorithm;

[0061] The second calculation submodule is used to calculate the vertical coordinate of the center point of each price tag detection frame;

[0062] A second difference submodule is configured to select a target price tag, calculate the difference in the vertical coordinates of the center points between the target price tag detection frame and the remaining price tag detection frames, and take the absolute value of the difference;

[0063] a second determining submodule, configured to determine that a price tag and the target price tag are located on the same shelf layer when the absolute value of the difference between the vertical coordinates of the center points of the detection frame of the price tag and the detection frame of the target price tag is less than a second threshold;

[0064] A second classification submodule is used to select the next target price tag and call the second difference submodule and the second determination submodule until all price tags are classified into corresponding shelf layers;

[0065] The second sorting submodule is used to sort the shelf layers in ascending order according to the minimum value of the vertical coordinate of the center point of each price tag detection frame in each shelf layer, and mark them as the first layer, the second layer, ..., the Mth layer in sequence;

[0066] The second division submodule is used to determine the layer division line of each shelf layer according to the minimum value of the vertical coordinate of the center point of each price tag detection frame in each shelf layer;

[0067] The second stratification submodule is used to stratify the shelves according to the layer dividing lines.

[0068] Furthermore, the layering module further includes:

[0069] A third marking submodule is configured to mark all the strip screens on the shelf with a detection frame using a strip screen detection algorithm;

[0070] The third calculation submodule is used to calculate the vertical coordinate of the center point of each strip screen detection frame;

[0071] The third sorting submodule is used to sort the shelf layers in ascending order according to the vertical coordinates of the center points of the bar screen detection frames in each shelf layer, marking them as the first layer, the second layer, ..., the Pth layer in sequence;

[0072] The third division submodule is used to determine the layer division line of each shelf layer according to the vertical coordinate of the center point of each strip screen detection frame in each shelf layer;

[0073] The third stratification submodule is configured to stratify the shelves according to the layer dividing lines.

[0074] Furthermore, the layered module specifically includes:

[0075] A first recording submodule is configured to separate the shelves into layers using the commodity arrangement detection method alone, and record the vertical coordinates of the layer dividing lines of each shelf layer in ascending order in a first list;

[0076] a second recording submodule, configured to separate the shelves into layers using the commodity price tag detection method alone, and record the vertical coordinates of the layer dividing lines of each shelf layer in ascending order in a second list;

[0077] A first setting submodule is configured to set the layer dividing line according to the commodity price tag detection method when the number of shelf layers in the first list is consistent with the number of shelf layers in the second list;

[0078] a first matching submodule configured to, when the number of shelf layers in the first list is inconsistent with the number of shelf layers in the second list, perform matching processing on the second list based on the first list; determine, when the absolute value of the difference between the basic value in the first list and the matching value in the second list is less than a third threshold, that the basic value and the matching value are matched, and that the shelf layer corresponding to the basic value and the shelf layer corresponding to the matching value are the same shelf layer; and treat the shelf layer corresponding to the basic value that has not been matched and the shelf layer corresponding to the matching value that has not been matched as separate shelf layers;

[0079] The fourth division submodule is used to use the layer division line obtained by the product price tag detection method as the final layer division line when there is a layer division line obtained by the product price tag detection method on a certain shelf layer; otherwise, use the layer division line obtained by the product display detection method as the final layer division line.

[0080] Furthermore, the layering module further includes:

[0081] a third recording submodule, configured to separate the shelves into layers using the commodity arrangement detection method alone, and record the vertical coordinates of the layer dividing lines of each shelf layer in ascending order in a third list;

[0082] A fourth recording submodule, configured to separate the shelves into layers using the strip screen detection method alone, and record the vertical coordinates of the layer dividing lines of each shelf layer in ascending order in a fourth list;

[0083] A second setting submodule is configured to set the layer dividing line according to the strip screen detection method when the number of shelf layers in the third list is consistent with the number of shelf layers in the fourth list;

[0084] a second matching submodule configured to, when the number of shelf layers in the third list is inconsistent with the number of shelf layers in the fourth list, perform matching processing on the fourth list based on the third list; when the absolute value of the difference between the basic value in the third list and the matching value in the fourth list is less than a fourth threshold, determine that the basic value and the matching value are matched, and the shelf layer corresponding to the basic value and the shelf layer corresponding to the matching value are the same shelf layer; and treat the shelf layer corresponding to the basic value that has not been matched and the shelf layer corresponding to the matching value that has not been matched as separate shelf layers;

[0085] The fifth division submodule is used to use the layer division line obtained by the strip screen detection method as the final layer division line when there is a layer division line obtained by the strip screen detection method on a certain shelf layer; otherwise, use the layer division line obtained by the product display detection method as the final layer division line.

[0086] In an embodiment of the present application, a shelf image is acquired, and the shelf is stratified by a combination of one or more of a product display detection method, a product price tag detection method, and a strip screen detection method. The shelf stratification is based on the product display, the product price tags, and the strip screen, thereby avoiding over-reliance on the characteristics of the shelf layer cross-section in actual scenarios. Therefore, accurate shelf stratification can still be performed when the shelf layers have no obvious contours, are arranged too closely, are too thin, or are affected by lighting. BRIEF DESCRIPTION OF THE DRAWINGS

[0087] Figure 1 This is one of the flow charts of a shelf stratification method provided in an embodiment of the present application;

[0088] Figure 2 This is one of the schematic diagrams of a specific image acquisition method provided in an embodiment of the present application;

[0089] Figure 3 This is a second schematic diagram of a specific image acquisition method provided in an embodiment of the present application;

[0090] Figure 4 This is the third schematic diagram of a specific image acquisition method provided in an embodiment of the present application;

[0091] Figure 5 This is the second flow chart of a shelf stratification method provided in an embodiment of the present application;

[0092] Figure 6 This is one of the shelf schematic diagrams provided in the embodiment of the present application;

[0093] Figure 7 This is the second schematic diagram of a shelf provided in an embodiment of the present application;

[0094] Figure 8 This is the third flow chart of a shelf stratification method provided in an embodiment of the present application;

[0095] Figure 9 This is the third schematic diagram of a shelf provided in an embodiment of the present application;

[0096] Figure 10 This is the fourth schematic diagram of a shelf provided in an embodiment of the present application;

[0097] Figure 11 This is a fourth flow chart of a shelf stratification method provided in an embodiment of the present application;

[0098] Figure 12 This is the fifth schematic diagram of a shelf provided in an embodiment of the present application;

[0099] Figure 13 This is the fifth flow chart of a shelf stratification method provided in an embodiment of the present application;

[0100] Figure 14 This is the sixth schematic diagram of a shelf provided in an embodiment of the present application;

[0101] Figure 15 This is a data matching diagram provided in an embodiment of the present application;

[0102] Figure 16 This is the sixth flow chart of a shelf stratification method provided in an embodiment of the present application;

[0103] Figure 17 This is the seventh schematic diagram of a shelf provided in an embodiment of the present application;

[0104] Figure 18 It is a structural schematic diagram of a shelf stratification device provided in an embodiment of the present application.

[0105] The realization of the objectives, functional features and advantages of the present invention will be further explained in conjunction with embodiments and with reference to the accompanying drawings. DETAILED DESCRIPTION

[0106] To make the objectives, technical solutions, and advantages of the present invention more clear, the technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments in the present invention, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of this application.

[0107] The terms "first," "second," and the like in the specification and claims of this application are used to distinguish similar objects, and are not used to describe a particular order or precedence. It should be understood that the terms used in this manner are interchangeable where appropriate, so that the embodiments of this application can be implemented in orders other than those illustrated or described herein. Furthermore, the terms "first," "second," and the like generally distinguish objects of a class and do not limit the number of objects. For example, the first object may be one or more.

[0108] The shelf stratification method provided in the embodiment of the present application is described in detail below through specific embodiments and their application scenarios in conjunction with the accompanying drawings.

[0109] Example 1

[0110] Reference Figure 1 , showing a flow chart of a shelf stratification method provided in an embodiment of the present application.

[0111] The present application provides a shelf layering method, comprising:

[0112] S101: Acquire shelf images.

[0113] Reference Figures 2 to 4 , showing the specific methods of obtaining three images.

[0114] Optionally, a fixed camera is provided opposite to the shelf, and the shelf image is captured by the fixed camera.

[0115] Optionally, a camera is provided on the robot to intelligently capture images of the shelves.

[0116] Optionally, the shelf image is taken manually using a mobile phone camera.

[0117] The embodiment of the present application does not limit the specific method of obtaining shelf images.

[0118] S102: Shelves are layered using one or more of a product display detection method, a product price tag detection method, and a bar screen detection method. The location information may be displayed in the form of coordinates.

[0119] Among them, the product display refers to the specific display method of the product 10 on the shelf; the product price tag 20 refers to the electronic price tag marked with information such as the product name and price; the bar screen 30 refers to a strip display screen set on the middle shelf layer for displaying information such as the product name and price.

[0120] It should be noted that any one of the product layout detection method, product price tag detection method, or bar screen detection method can be used independently to stratify shelves. This avoids over-reliance on the cross-sectional characteristics of shelf layers in actual scenarios, and can still accurately stratify shelves even when shelf layers have no obvious outlines, are arranged too closely, are too thin, or are affected by lighting.

[0121] Shelf stratification can also be achieved by combining product display detection with product price tag detection. Alternatively, shelf stratification can be achieved by combining product display detection with bar screen detection. This combination of methods allows for the use of a supplementary method when one of the products, price tags, or bar screen is obscured, allowing for a variety of complex real-world scenarios and further improving shelf stratification accuracy.

[0122] Reference Figure 5 , Figure 5 Another shelf layering method provided by an embodiment of the present application is shown.

[0123] When stratifying shelves solely by product placement detection, the following steps can be used:

[0124] S201: Using a product arrangement detection algorithm, all products 10 on the shelf are marked with detection frames.

[0125] Reference Figure 6 , Figure 6 A schematic diagram of a shelf provided in an embodiment of the present application is shown.

[0126] The commodity arrangement detection algorithm can add detection frames for all commodities 10 on the shelf.

[0127] Optionally, the detection frame is the minimum circumscribed rectangle of the commodity 10 in the shelf image.

[0128] S202: Calculate the vertical coordinate of the center point of each product detection frame.

[0129] The vertical coordinate of the center point of each commodity detection frame in the shelf image can be recorded in a statistical table to facilitate subsequent processing.

[0130] S203: Select a target product, calculate the difference d1 between the vertical coordinates of the center points of the target product detection frame and the remaining product detection frames, and take the absolute value of the difference.

[0131] The target products may be selected randomly or in a certain order. Specifically, the target products may be selected in order from top to bottom and from left to right.

[0132] Among them, the difference d1 of the vertical coordinates of the center points of the target product detection frame and the remaining product detection frames can be used to determine whether the target product and the remaining products are at the same height, and further determine whether they are located on the same shelf layer.

[0133] S204: When the absolute value of the difference d1 between the vertical coordinates of the center points of a certain product detection frame and the target product detection frame is less than a first threshold, it is determined that the product and the target product are on the same shelf layer.

[0134] Among them, the specific size of the first threshold can be selected according to actual conditions and is not limited in the embodiments of the present application.

[0135] When the absolute value of the difference d1 between the vertical coordinates of the center points of a certain product detection frame and the target product detection frame is less than the first threshold, it means that the product and the target product are at approximately the same height, and thus it can be determined that the product and the target product are on the same shelf layer.

[0136] Accordingly, when the absolute value of the difference d1 between the vertical coordinates of the center points of a certain product detection frame and the target product detection frame is greater than or equal to the first threshold, it can be determined that the product and the target product are not on the same shelf layer.

[0137] S205: Select the next target product and repeat the above steps S203 and S204 until all products are classified into corresponding shelf layers.

[0138] That is, in S205, it is necessary to determine whether all commodities 10 have been classified into corresponding shelf layers. If all commodities 10 have been classified into corresponding shelf layers, S206 is executed; otherwise, S203 and S204 are continued to be executed until all commodities 10 have been classified into corresponding shelf layers.

[0139] S206: Sort the shelf layers in ascending order based on the minimum value of the vertical coordinate of the center point of each product detection frame in each shelf layer, and mark them as the first layer, the second layer, ..., the Nth layer in sequence.

[0140] Optionally, sorting is also performed according to other sorting rules, for example, each shelf layer can be sorted from large to small.

[0141] like Figure 6 As shown in , the minimum vertical coordinate of the center point of each product on the top layer is greater than the minimum vertical coordinate of the center point of each product on the middle layer, and the minimum vertical coordinate of the center point of each product on the middle layer is greater than the minimum vertical coordinate of the center point of each product on the bottom layer. In this case, when sorting in ascending order, the bottom shelf is marked as the first shelf, the middle shelf is marked as the second shelf, and so on, until the Nth shelf is marked.

[0142] S207: Determine a layer dividing line for each shelf layer according to the minimum value of the vertical coordinate of the lower right corner of each commodity detection frame in each shelf layer.

[0143] The lower end of the product detection frame is closer to the middle shelf layer, so the dividing point is selected at the lower end of the product detection frame. In particular, the product detection frame with the smallest lower vertical coordinate value is closer to the middle shelf layer, so the minimum vertical coordinate value of the lower right corner of each product detection frame in each shelf layer is selected to determine the layer dividing line for each shelf layer.

[0144] Optionally, the minimum value of the lower left corner vertical coordinate of each commodity detection frame may be selected to determine the layer dividing line of each shelf layer.

[0145] Optionally, the layer dividing line is a horizontal line, and the horizontal line passes through the point with the smallest vertical coordinate in the lower right corner of each product detection frame.

[0146] Reference Figure 7 , Figure 7 Another shelf schematic diagram provided in an embodiment of the present application is shown.

[0147] Figure 7 In the figure, the second-to-last product detection frame from left to right is closer to the middle shelf board, and the vertical coordinate of the lower left corner of its product detection frame is the smallest. Therefore, the layer division line is determined based on the lower left corner of the second-to-last product detection frame.

[0148] S208: The shelves are layered according to the layer dividing lines.

[0149] Specifically, the area between two layer dividing lines is classified as one shelf layer.

[0150] The shelf stratification method is used to detect the product arrangement, avoiding over-reliance on the characteristics of the shelf layer cross-section in actual scenarios. Therefore, accurate shelf stratification can still be performed when the shelf layers have no obvious outline, are arranged too closely, are too thin, or are affected by lighting.

[0151] Reference Figure 8 , Figure 8 Another shelf layering method provided by an embodiment of the present application is shown.

[0152] Specifically, when stratifying shelves using the product price tag detection method alone, the following steps can be used:

[0153] S301: Using a product price tag detection algorithm, all price tags on the shelf are marked with detection frames.

[0154] Reference Figure 9 , Figure 9 A schematic diagram of another shelf provided in an embodiment of the present application is shown.

[0155] The product price tag detection algorithm can be used to add detection frames for all price tags 20 on the shelf.

[0156] Optionally, the detection frame is the minimum circumscribed rectangle of the price label 20 in the shelf image.

[0157] S302: Calculate the vertical coordinate of the center point of each price tag detection frame.

[0158] The vertical coordinates of the center points of the respective price tag detection frames in the shelf image may be recorded in a statistical table for subsequent processing.

[0159] S303: Select a target price tag, calculate the difference d2 between the vertical coordinates of the center points of the target price tag detection frame and the remaining price tag detection frames, and take the absolute value of the difference.

[0160] The target price tags may be selected randomly or in a certain order. Specifically, the target price tags may be selected in order from top to bottom and from left to right.

[0161] Among them, the difference d2 between the vertical coordinates of the center points of the target price tag detection frame and the remaining price tag detection frames can be used to determine whether the target price tag and the remaining price tags are at the same height, and further determine whether they are located on the same shelf layer.

[0162] S304: When the absolute value of the difference d2 between the vertical coordinates of the center points of a certain price tag detection frame and the target price tag detection frame is less than a second threshold, it is determined that the price tag and the target price tag are on the same shelf layer.

[0163] Among them, the specific size of the second threshold can be selected according to actual conditions and is not limited in the embodiments of the present application.

[0164] When the absolute value of the difference d2 between the vertical coordinates of the center points of a certain price tag detection frame and the target price tag detection frame is less than the second threshold, it means that the price tag and the target price tag are at approximately the same height, and it can be determined that the price tag and the target price tag are on the same shelf layer.

[0165] Accordingly, when the absolute value of the difference d2 between the vertical coordinates of the center points of a certain price tag detection frame and the target price tag detection frame is greater than or equal to the second threshold, it can be determined that the price tag and the target price tag are not on the same shelf layer.

[0166] S305: Select the next target price tag and repeat the above steps S303 and S304 until all price tags are classified into corresponding shelf layers.

[0167] That is, in S305, it is necessary to determine whether all price tags 20 have been classified into corresponding shelf layers. If all price tags 20 have been classified into corresponding shelf layers, S306 is executed; otherwise, S303 and S304 are continued until all price tags 20 have been classified into corresponding shelf layers.

[0168] S306: Sort the shelf layers in ascending order according to the minimum value of the vertical coordinate of the center point of each price tag detection frame in each shelf layer, and mark them as the first layer, the second layer, ..., the Mth layer in sequence.

[0169] Optionally, sorting is also performed according to other sorting rules, for example, each shelf layer can be sorted from large to small.

[0170] like Figure 9 As shown in , the minimum vertical coordinate of the center point of each price tag detection frame in the top layer is greater than the minimum vertical coordinate of the center point of each price tag detection frame in the middle layer, and the minimum vertical coordinate of the center point of each price tag detection frame in the middle layer is greater than the minimum vertical coordinate of the center point of each price tag detection frame in the bottom layer. In this case, when sorting in ascending order, the bottom shelf is marked as the first shelf, the middle shelf is marked as the second shelf, and so on, until the Mth shelf is marked.

[0171] S307: Determine a layer dividing line for each shelf layer according to the minimum value of the vertical coordinate of the center point of each price tag detection frame in each shelf layer.

[0172] The price tag is attached to the middle layer of the shelf, so it is most accurate to mark the middle layer of the shelf at the center point of the price tag detection frame.

[0173] Optionally, the layer dividing line is a horizontal line, and the horizontal line passes through the center point of each price tag detection frame with the smallest vertical coordinate.

[0174] Reference Figure 10 , Figure 10 Another shelf schematic diagram provided in an embodiment of the present application is shown.

[0175] Figure 10 In the figure, the center point of the last price tag detection frame from left to right has the smallest vertical coordinate, and the layer division line is determined based on the center point of the last price tag detection frame.

[0176] It should be noted that, generally, the height difference of electronic price tags on the same shelf layer is not very large, that is, the vertical coordinates of the center points of the price tag detection frames are very small. In actual application, the layer dividing line can be determined by using the minimum value of the vertical coordinate of the center point of the price tag detection frame or the maximum value of the vertical coordinate of the center point of the price tag detection frame. In this embodiment, the minimum value of the vertical coordinate of the center point of the price tag detection frame is used to determine the layer dividing line for example.

[0177] S308: The shelves are layered according to the layer dividing lines.

[0178] Specifically, the area between two layer dividing lines is classified as one shelf layer.

[0179] The shelf stratification method is used to detect product price tags, avoiding over-reliance on the characteristics of the shelf layer cross-section in actual scenarios. Therefore, accurate shelf stratification can still be performed when the shelf layers have no obvious outline, are arranged too closely, are too thin, or are affected by lighting.

[0180] Specifically, refer to Figure 11 , Figure 11 Another shelf layering method provided by an embodiment of the present application is shown.

[0181] Specifically, when the shelves are layered using the strip screen detection method alone, the following steps can be used:

[0182] S401: Using a strip screen detection algorithm, all strip screens on the shelf are marked with detection frames.

[0183] Reference Figure 12 , Figure 12 A schematic diagram of another shelf provided in an embodiment of the present application is shown.

[0184] The strip screen detection algorithm can be used to add detection frames for all the strip screens 30 on the shelf.

[0185] Optionally, the detection frame is the minimum circumscribed rectangle of the strip screen 30 in the shelf image.

[0186] S402: Calculate the vertical coordinate of the center point of each strip screen detection frame.

[0187] The vertical coordinates of the center points of the respective strip screen detection frames in the shelf image may be recorded in a statistical table for subsequent processing.

[0188] S403: Sort the shelf layers in ascending order according to the vertical coordinates of the center points of the bar screen detection frames in each shelf layer, and mark them as the first layer, the second layer, ..., the Pth layer in sequence.

[0189] Optionally, sorting is also performed according to other sorting rules, for example, each shelf layer can be sorted from large to small.

[0190] like Figure 12 As shown in , the vertical coordinate of the center point of the topmost strip screen detection frame is greater than the vertical coordinate of the center point of the middle strip screen detection frame, and the vertical coordinate of the center point of the middle strip screen detection frame is greater than the vertical coordinate of the center point of each price tag detection frame at the bottom layer. In this case, when sorting in ascending order, the bottom shelf is marked as the first shelf, the middle shelf is marked as the second shelf, and so on, until the Pth shelf is marked.

[0191] S404: Determine a layer dividing line for each shelf layer according to the vertical coordinate of the center point of each strip screen detection frame in each shelf layer.

[0192] The price tag is set on the middle layer of the shelf, so it is most accurate to mark the middle layer of the shelf at the center point of the bar screen detection frame.

[0193] Alternatively, as Figure 12 In the example, the layer dividing line is a horizontal line passing through the center point of the strip screen detection frame.

[0194] S405: The shelves are layered according to the layer division lines.

[0195] Specifically, the area between two layer dividing lines is classified as one shelf layer.

[0196] The use of a strip screen detection method to stratify shelves avoids over-reliance on the characteristics of the shelf layer cross-section in actual scenarios. As a result, accurate shelf stratification can still be performed when the shelf layers have no obvious outlines, are arranged too closely, are too thin, or are affected by lighting.

[0197] Reference Figure 13 , Figure 13 Another shelf layering method provided by an embodiment of the present application is shown.

[0198] Specifically, when stratifying shelves by combining the product layout detection method with the product price tag detection method, the following steps can be used:

[0199] S501: Through steps S201 to S208, the shelves are layered using the product arrangement detection method alone, and the vertical coordinates of the layer dividing lines of each shelf layer are recorded in the first list in ascending order.

[0200] The details of steps S201 to S208 can be found above and will not be repeated here.

[0201] It should be noted that the layer dividing line is a horizontal line. The specific position of the layer dividing line can be determined by the vertical coordinate of the point on the layer dividing line. After recording the vertical coordinate of the layer dividing line of each shelf layer in the first list in ascending order, the shelves can be layered according to the layer dividing line of each shelf layer.

[0202] S502: Through steps S301 to S308, the shelves are layered using the product price tag detection method alone, and the vertical coordinates of the layer dividing lines of each shelf layer are recorded in the second list in ascending order.

[0203] The details of steps S301 to S308 can be found above and will not be repeated here.

[0204] S503: When the number of shelf layers in the first list is consistent with the number of shelf layers in the second list, layer dividing lines are set according to the commodity price tag detection method.

[0205] That is to say, the layer dividing line is preferably set according to the position of the product price tag. This is because the product price tag 20 is set on the middle layer of the shelf, and the product 10 is set in the accommodating space formed by the two middle layers of the shelf. Obviously, the position of the product price tag 20 can better represent the position of the middle layer of the shelf.

[0206] S504: When the number of shelf layers in the first list is inconsistent with the number of shelf layers in the second list, the second list is matched based on the first list; when the absolute value of the difference between the basic value in the first list and the matching value in the second list is less than the third threshold value, it is determined that the basic value and the matching value are matched, and the shelf layer corresponding to the basic value and the shelf layer corresponding to the matching value are the same shelf layer; the shelf layer corresponding to the basic value that has not been matched and the shelf layer corresponding to the matching value that has not been matched are treated as separate shelf layers.

[0207] Among them, the specific size of the third threshold can be selected according to actual conditions and is not limited in the embodiments of the present application.

[0208] Optionally, the first list may be matched based on the second list, which is consistent with the final result of matching the second list based on the first list.

[0209] Reference Figure 14 , Figure 14 Another shelf schematic diagram provided in an embodiment of the present application is shown.

[0210] Figure 14 The table shows layer demarcation line 1, determined based on product layout detection, and layer demarcation line 2, determined based on product price tag detection. If the distance d3 between the two (i.e., the absolute value of the difference between the vertical coordinates recorded in the first list and the vertical coordinates recorded in the second list) is less than a third threshold, it can be determined that the two are matched and layer demarcation line 1 and layer demarcation line 2 represent the same shelf mid-layer. Otherwise, it should be determined that the two represent different shelf mid-layers and should be counted separately to avoid missing counts.

[0211] S505: If a layer dividing line obtained by the product price tag detection method exists on a certain shelf layer, the layer dividing line obtained by the product price tag detection method is used as the final layer dividing line; otherwise, the layer dividing line obtained by the product display detection method is used as the final layer dividing line.

[0212] It should be noted that when there are two corresponding layer dividing lines on the same middle layer of the shelf, the layer dividing line is preferably set according to the position of the product price tag 20. This is because the product price tag 20 is set on the middle layer of the shelf, and the product 10 is set in the accommodating space formed by the two layers of the middle layer. Obviously, the position of the product price tag 20 can better represent the position of the middle layer of the shelf.

[0213] Below through Figure 15 The matching process of the data in the first list and the second list is introduced in detail.

[0214] Reference Figure 15 , Figure 15 A data matching schematic diagram provided in an embodiment of the present application is shown.

[0215] The basic values in the first list are 100, 300, 400, and 500, representing the four layer dividing lines determined by the product layout detection method; the matching values in the second list are 102, 203, 302, and 503, representing the four layer dividing lines determined by the product price tag detection method.

[0216] Assuming the third threshold is 10, the successfully matched data are 100 and 102, 300 and 302, and 500 and 503. The price tag detection result is prioritized, so 102, 302, and 503 are recorded in the final layer division list. The unmatched data 400 and 203 are recorded separately in the final layer division line.

[0217] The combination of product display and price tag detection ultimately determines five layer demarcation lines. Data can be supplemented by price tag detection when products are obscured (e.g., 102 in the second list), or by display detection when price tags are obscured (e.g., 400 in the first list). This allows for adapting to various complex real-world scenarios and further improving shelf stratification accuracy.

[0218] Reference Figure 16 , Figure 16 Another shelf layering method provided by an embodiment of the present application is shown.

[0219] Specifically, when stratifying shelves by combining the product arrangement detection method with the strip screen detection method, the following steps can be used:

[0220] S601: Through steps S201 to S208, the shelves are layered using the product arrangement detection method alone, and the vertical coordinates of the layer dividing lines of each shelf layer are recorded in the third list in ascending order.

[0221] The details of steps S201 to S208 can be found above and will not be repeated here.

[0222] It should be noted that the layer dividing line is a horizontal line. The specific position of the layer dividing line can be determined by the vertical coordinate of the point on the layer dividing line. After recording the vertical coordinate of the layer dividing line of each shelf layer in the first list in ascending order, the shelves can be layered according to the layer dividing line of each shelf layer.

[0223] S602: Through steps S401 to S405, the shelves are layered using the strip screen detection method alone, and the vertical coordinates of the layer dividing lines of each shelf layer are recorded in a fourth list in ascending order.

[0224] The details of steps S401 to S405 can be found above and will not be repeated here.

[0225] S603: When the number of shelf layers in the third list is consistent with the number of shelf layers in the fourth list, layer dividing lines are set according to the bar screen detection method.

[0226] That is to say, the layer dividing line is preferably set according to the position of the strip screen. This is because the strip screen 30 is set on the middle layer of the shelf, and the goods 10 are placed in the accommodating space formed by the two middle layers of the shelf. Obviously, the position of the strip screen 30 is more representative of the position of the middle layer of the shelf.

[0227] S604: When the number of shelf layers in the third list is inconsistent with the number of shelf layers in the fourth list, the fourth list is matched based on the third list; when the absolute value of the difference between the basic value in the third list and the matching value in the fourth list is less than the fourth threshold value, it is determined that the basic value and the matching value are matched, and the shelf layer corresponding to the basic value and the shelf layer corresponding to the matching value are the same shelf layer; the shelf layer corresponding to the basic value that has not been matched and the shelf layer corresponding to the matching value that has not been matched are treated as separate shelf layers.

[0228] Among them, the specific size of the fourth threshold can be selected according to actual conditions and is not limited in the embodiments of the present application.

[0229] Optionally, the third list may be matched based on the fourth list, which is consistent with the final result of matching the fourth list based on the third list.

[0230] Reference Figure 17 , Figure 17 Another shelf schematic diagram provided in an embodiment of the present application is shown.

[0231] Figure 17 The table shows layer demarcation line 3, determined based on product layout detection, and layer demarcation line 4, determined based on bar screen detection. If the distance d4 between the two (i.e., the absolute value of the difference between the vertical coordinates recorded in the third list and the vertical coordinates recorded in the fourth list) is less than a fourth threshold, it can be determined that the two are matched successfully, and layer demarcation line 3 and layer demarcation line 4 represent the same shelf middle layer. Otherwise, it should be determined that the two do not represent the same shelf middle layer, and to avoid missing counts, they should be counted separately.

[0232] S605: If a layer dividing line obtained by the strip screen detection method exists on a certain shelf layer, the layer dividing line obtained by the strip screen detection method is used as the final layer dividing line; otherwise, the layer dividing line obtained by the product display detection method is used as the final layer dividing line.

[0233] It should be noted that when there are two corresponding layer dividing lines on the same shelf middle layer, the layer dividing line is preferably set according to the position of the strip screen 30. This is because the strip screen 30 is set on the shelf middle layer, and the goods 10 are set in the accommodating space formed by the two layers of the middle layer. Obviously, the position of the strip screen 30 is more representative of the position of the shelf middle layer.

[0234] The specific data matching process is similar to the data matching process in the first list and the second list above, and will not be repeated here.

[0235] The layer division lines finally determined by combining the product display detection method with the strip screen detection method can be supplemented and improved by strip screen detection when the product is blocked, or by product display detection when the strip screen is blocked, so as to cope with various complex actual scene environments and further improve the accuracy of shelf stratification.

[0236] In an embodiment of the present application, shelf images are acquired, and the shelves are stratified by a combination of one or more of a product display detection method, a product price tag detection method, and a strip screen detection method. The shelf stratification is based on the product display, product price tags, and strip screens, thereby avoiding over-reliance on the characteristics of the shelf layer cross-section in actual scenarios. Accurate shelf stratification can still be performed when the shelf layers have no obvious contours, are arranged too closely, are too thin, or are affected by lighting.

[0237] Example 2

[0238] Reference Figure 18 , showing a structural schematic diagram of a shelf stratification device 40 provided in an embodiment of the present application.

[0239] The shelf layering device 40 comprises:

[0240] An acquisition module 41 is used to acquire shelf images;

[0241] The stratification module 42 is used to stratify the shelves by using one or more of the following methods: a product display detection method, a product price tag detection method, and a bar screen detection method.

[0242] Furthermore, the layering module 42 specifically includes:

[0243] The first marking submodule 4201 is used to mark all products on the shelf with detection frames using a product arrangement detection algorithm;

[0244] The first calculation submodule 4202 is used to calculate the vertical coordinate of the center point of each product detection frame;

[0245] The first difference submodule 4203 is used to select a target product, calculate the difference in the vertical coordinates of the center points between the target product detection frame and the remaining product detection frames, and take the absolute value of the difference;

[0246] A first determining submodule 4204 is configured to determine that a product and a target product are on the same shelf layer when the absolute value of the difference between the vertical coordinates of the center points of a product detection frame and a target product detection frame is less than a first threshold;

[0247] The first classification submodule 4205 is used to select the next target product and call the first difference submodule 4203 and the first determination submodule 4204 until all products are classified into corresponding shelf layers;

[0248] The first sorting submodule 4206 is configured to sort the shelf layers in ascending order based on the minimum value of the vertical coordinate of the center point of each product detection frame in each shelf layer, and mark them as the first layer, the second layer, ..., the Nth layer in sequence;

[0249] The first division submodule 4207 is configured to determine a layer division line for each shelf layer based on the minimum value of the lower right corner vertical coordinate of each commodity detection frame in each shelf layer;

[0250] The first stratification submodule 4208 is used to stratify the shelves according to the layer dividing lines.

[0251] Furthermore, the layering module 42 further includes:

[0252] The second marking submodule 4209 is used to mark all price tags on the shelf with a detection frame using a product price tag detection algorithm;

[0253] The second calculation submodule 4210 is used to calculate the vertical coordinate of the center point of each price tag detection frame;

[0254] The second difference submodule 4211 is used to select a target price tag, calculate the difference between the vertical coordinates of the center points of the target price tag detection frame and the remaining price tag detection frames, and take the absolute value of the difference;

[0255] A second determining submodule 4212 is configured to determine that a price tag and a target price tag are located on the same shelf layer when the absolute value of the difference between the vertical coordinates of the center points of the detection frame of the price tag and the detection frame of the target price tag is less than a second threshold;

[0256] The second classification submodule 4213 is used to select the next target price tag and call the second difference submodule 4211 and the second determination submodule 4212 until all price tags are classified into corresponding shelf layers;

[0257] The second sorting submodule 4214 is configured to sort the shelf layers in ascending order based on the minimum value of the vertical coordinate of the center point of each price tag detection frame in each shelf layer, marking them as the first layer, the second layer, ..., the Mth layer in sequence;

[0258] The second dividing submodule 4215 is used to determine the layer dividing line of each shelf layer according to the minimum value of the vertical coordinate of the center point of each price tag detection frame in each shelf layer;

[0259] The second stratification submodule 4216 is used to stratify the shelves according to the layer dividing lines.

[0260] Furthermore, the layering module 42 further includes:

[0261] The third marking submodule 4217 is used to mark all the strip screens on the shelf with detection frames using a strip screen detection algorithm;

[0262] The third calculation submodule 4218 is used to calculate the vertical coordinate of the center point of each strip screen detection frame;

[0263] The third sorting submodule 4219 is used to sort the shelf layers in ascending order according to the vertical coordinates of the center points of the bar screen detection frames in each shelf layer, marking them as the first layer, the second layer, ..., the Pth layer in sequence;

[0264] The third division submodule 4220 is used to determine the layer division line of each shelf layer according to the vertical coordinate of the center point of each strip screen detection frame in each shelf layer;

[0265] The third stratification submodule 4221 is used to stratify the shelves according to the layer dividing lines.

[0266] Furthermore, the layering module 42 specifically includes:

[0267] The first recording submodule 4222 is configured to separate the shelves into layers using the product arrangement detection method alone, and record the vertical coordinates of the layer dividing lines of each shelf layer in ascending order in the first list;

[0268] The second recording submodule 4223 is configured to separate the shelves into layers using the product price tag detection method alone, and record the vertical coordinates of the layer dividing lines of each shelf layer in ascending order in a second list;

[0269] The first setting submodule 4224 is configured to set the layer dividing lines according to the commodity price tag detection method when the number of shelf layers in the first list is consistent with the number of shelf layers in the second list;

[0270] The first matching submodule 4225 is configured to, if the number of shelf layers in the first list is inconsistent with the number of shelf layers in the second list, perform matching processing on the second list based on the first list; if the absolute value of the difference between the basic value in the first list and the matching value in the second list is less than a third threshold, determine that the basic value and the matching value are matched, and the shelf layer corresponding to the basic value and the shelf layer corresponding to the matching value are the same shelf layer; and treat the shelf layer corresponding to the basic value that has not been matched and the shelf layer corresponding to the matching value that has not been matched as separate shelf layers;

[0271] The fourth division submodule 4226 is used to use the layer division line obtained by the product price tag detection method as the final layer division line when there is a layer division line obtained by the product price tag detection method on a certain shelf layer; otherwise, the layer division line obtained by the product display detection method is used as the final layer division line.

[0272] Furthermore, the layering module 42 further includes:

[0273] The third recording submodule 4227 is configured to separate the shelves into layers using the product arrangement detection method alone, and record the vertical coordinates of the layer dividing lines of each shelf layer in ascending order in a third list;

[0274] A fourth recording submodule 4228 is configured to separate the shelves into layers using the bar screen detection method alone, and record the vertical coordinates of the layer dividing lines of each shelf layer in ascending order in a fourth list;

[0275] The second setting submodule 4229 is configured to set the layer dividing line according to the strip screen detection method when the number of shelf layers in the third list is consistent with the number of shelf layers in the fourth list;

[0276] The second matching submodule 4330 is configured to, if the number of shelf layers in the third list is inconsistent with the number of shelf layers in the fourth list, perform matching processing on the fourth list based on the third list; if the absolute value of the difference between the basic value in the third list and the matching value in the fourth list is less than a fourth threshold, determine that the basic value and the matching value are matched, and the shelf layer corresponding to the basic value and the shelf layer corresponding to the matching value are the same shelf layer; and treat the shelf layer corresponding to the basic value that has not been matched and the shelf layer corresponding to the matching value that has not been matched as separate shelf layers;

[0277] The fifth division submodule 4331 is configured to use the layer division line obtained by the strip screen detection method as the final layer division line if a layer division line obtained by the strip screen detection method exists on a certain shelf layer; otherwise, use the layer division line obtained by the product display detection method as the final layer division line.

[0278] The shelf stratification device 40 provided in the embodiment of the present application can implement each process implemented in the above method embodiment. To avoid repetition, it will not be described here.

[0279] In an embodiment of the present application, shelf images are acquired, and the shelves are stratified by a combination of one or more of a product display detection method, a product price tag detection method, and a strip screen detection method. The shelf stratification is based on the product display, product price tags, and strip screens, thereby avoiding over-reliance on the characteristics of the shelf layer cross-section in actual scenarios. Accurate shelf stratification can still be performed when the shelf layers have no obvious contours, are arranged too closely, are too thin, or are affected by lighting.

[0280] The virtual device in the embodiment of the present application can be a device, or a component, integrated circuit, or chip in a terminal.

[0281] The foregoing is merely an embodiment of the present invention and is not intended to limit the present invention. It will be apparent to those skilled in the art that various modifications and variations of the present invention are possible. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present invention are intended to be included within the scope of the claims of the present invention.

Claims

1. A shelf layering method, characterized in that: include: S101: Acquire shelf image; S102: stratifying the shelves by using one or more of a product display detection method, a product price tag detection method, and a bar screen detection method; The shelves are layered using the commodity arrangement detection method, specifically comprising: S201: Marking all products on the shelf with detection frames using a product arrangement detection algorithm; S202: Calculate the vertical coordinate of the center point of each product detection frame; S203: Select a target product, calculate the difference between the vertical coordinates of the center points of the target product detection frame and the remaining product detection frames, and take the absolute value of the difference; S204: If the absolute value of the difference between the vertical coordinates of the center points of a certain product detection frame and the target product detection frame is less than a first threshold, determining that the product and the target product are on the same shelf layer; S205: Select the next target product and repeat the above steps S203 and S204 until all products are classified into corresponding shelf layers; S206: Sort the shelf layers in ascending order based on the minimum value of the vertical coordinate of the center point of each product detection frame in each shelf layer, and mark them as the first layer, the second layer, ..., the Nth layer in sequence; S207: Determine a layer dividing line for each shelf layer according to the minimum value of the lower right corner vertical coordinate of each commodity detection frame in each shelf layer; S208: Layering the shelves according to the layer dividing lines; The shelves are layered using the commodity price tag detection method, specifically comprising: S301: Using a product price tag detection algorithm, mark all price tags on the shelf with detection frames; S302: Calculate the vertical coordinate of the center point of each price tag detection frame; S303: Select a target price tag, calculate the difference between the vertical coordinates of the center points of the target price tag detection frame and the remaining price tag detection frames, and take the absolute value of the difference; S304: If the absolute value of the difference between the vertical coordinates of the center points of a certain price tag detection frame and the target price tag detection frame is less than a second threshold, determine that the price tag and the target price tag are on the same shelf layer; S305: Select the next target price tag and repeat steps S303 and S304 until all price tags are classified into corresponding shelf layers; S306: Sort the shelf layers in ascending order based on the minimum value of the vertical coordinate of the center point of each price tag detection frame in each shelf layer, and mark them as the first layer, the second layer, ..., the Mth layer in sequence; S307: Determine the layer dividing line of each shelf layer according to the minimum value of the vertical coordinate of the center point of each price tag detection frame in each shelf layer; S308: Layering the shelves according to the layer dividing lines; The shelves are layered by combining the commodity arrangement detection method and the commodity price tag detection method, specifically including: S501: Through steps S201 to S208, the shelf is layered using the commodity arrangement detection method alone, and the vertical coordinates of the layer dividing lines of each shelf layer are recorded in a first list in ascending order; S502: Through steps S301 to S308, the shelf is layered using the product price tag detection method alone, and the vertical coordinates of the layer dividing lines of each shelf layer are recorded in a second list in ascending order; S503: When the number of shelf layers in the first list is consistent with the number of shelf layers in the second list, setting the layer dividing lines according to the commodity price tag detection method; S504: When the number of shelf layers in the first list is inconsistent with the number of shelf layers in the second list, matching processing is performed on the second list based on the first list; when the absolute value of the difference between the basic value in the first list and the matching value in the second list is less than a third threshold value, it is determined that the basic value and the matching value are matched, and the shelf layer corresponding to the basic value and the shelf layer corresponding to the matching value are the same shelf layer; the shelf layer corresponding to the basic value that has not been matched and the shelf layer corresponding to the matching value that has not been matched are treated as separate shelf layers; S505: If a layer dividing line obtained by the product price tag detection method exists on a certain shelf layer, the layer dividing line obtained by the product price tag detection method is used as the final layer dividing line; otherwise, the layer dividing line obtained by the product display detection method is used as the final layer dividing line.

2. The shelf layering method according to claim 1, characterized in that: The shelves are layered using the strip screen detection method, specifically comprising: S401: Using a strip screen detection algorithm, mark all strip screens on the shelf with detection frames; S402: Calculating the vertical coordinates of the center points of each strip screen detection frame; S403: Sort the shelves in ascending order based on the vertical coordinates of the center points of the bar screen detection frames in each shelf layer, marking them as the first layer, the second layer, ..., the Pth layer in sequence; S404: Determine the layer dividing line of each shelf layer according to the vertical coordinate of the center point of each strip screen detection frame in each shelf layer; S405: The shelves are divided into layers according to the layer dividing lines.

3. The shelf layering method according to claim 2, characterized in that: The shelves are layered by combining the commodity arrangement detection method and the strip screen detection method, specifically including: S601: Through steps S201 to S208, the shelf is layered using the commodity arrangement detection method alone, and the vertical coordinates of the layer dividing lines of each shelf layer are recorded in a third table in ascending order; S602: Through steps S401 to S405, the shelves are layered using the strip screen detection method alone, and the vertical coordinates of the layer dividing lines of each shelf layer are recorded in a fourth table in ascending order; S603: When the number of shelf layers in the third list is consistent with the number of shelf layers in the fourth list, setting the layer dividing lines according to the strip screen detection method; S604: When the number of shelf layers in the third list is inconsistent with the number of shelf layers in the fourth list, matching processing is performed on the fourth list based on the third list; when the absolute value of the difference between the basic value in the third list and the matching value in the fourth list is less than a fourth threshold value, it is determined that the basic value and the matching value are matched, and the shelf layer corresponding to the basic value and the shelf layer corresponding to the matching value are the same shelf layer; the shelf layer corresponding to the basic value that has not been matched and the shelf layer corresponding to the matching value that has not been matched are treated as separate shelf layers; S605: If a layer dividing line obtained by the strip screen detection method exists on a certain shelf layer, the layer dividing line obtained by the strip screen detection method is used as the final layer dividing line; otherwise, the layer dividing line obtained by the product display detection method is used as the final layer dividing line.

4. A shelf layering device, characterized in that: include: An acquisition module, used to acquire shelf images; a stratification module for stratifying the shelves by using one or more of a product display detection method, a product price tag detection method, and a bar screen detection method; The layered module specifically includes: A first marking submodule is configured to mark all commodities on the shelf with a detection frame using a commodity arrangement detection algorithm; The first calculation submodule is used to calculate the vertical coordinate of the center point of each product detection frame; A first difference submodule is configured to select a target product, calculate the difference in the vertical coordinates of the center points between the target product detection frame and the remaining product detection frames, and take the absolute value of the difference; a first determining submodule, configured to determine that a product and the target product are on the same shelf layer when the absolute value of the difference between the vertical coordinates of the center points of a product detection frame and the target product detection frame is less than a first threshold; A first classification submodule is used to select the next target product and call the first difference submodule and the first determination submodule until all products are classified into corresponding shelf layers; The first sorting submodule is used to sort the shelf layers in ascending order according to the minimum value of the vertical coordinate of the center point of each product detection frame in each shelf layer, and mark them as the first layer, the second layer, ..., the Nth layer in sequence; The first division submodule is used to determine the layer division line of each shelf layer according to the minimum value of the lower right corner vertical coordinate of each commodity detection frame in each shelf layer; A first stratification submodule, configured to stratify the shelves according to the layer dividing lines; The layered module also includes: A second marking submodule is configured to mark all price tags on the shelf with a detection frame using a product price tag detection algorithm; The second calculation submodule is used to calculate the vertical coordinate of the center point of each price tag detection frame; A second difference submodule is configured to select a target price tag, calculate the difference in the vertical coordinates of the center points between the target price tag detection frame and the remaining price tag detection frames, and take the absolute value of the difference; a second determining submodule, configured to determine that a price tag and the target price tag are located on the same shelf layer when the absolute value of the difference between the vertical coordinates of the center points of the detection frame of the price tag and the detection frame of the target price tag is less than a second threshold; A second classification submodule is used to select the next target price tag and call the second difference submodule and the second determination submodule until all price tags are classified into corresponding shelf layers; The second sorting submodule is used to sort the shelf layers in ascending order according to the minimum value of the vertical coordinate of the center point of each price tag detection frame in each shelf layer, and mark them as the first layer, the second layer, ..., the Mth layer in sequence; The second division submodule is used to determine the layer division line of each shelf layer according to the minimum value of the vertical coordinate of the center point of each price tag detection frame in each shelf layer; A second stratification submodule is configured to stratify the shelves according to the layer dividing lines; The layered module specifically includes: A first recording submodule is configured to separate the shelves into layers using the commodity arrangement detection method alone, and record the vertical coordinates of the layer dividing lines of each shelf layer in ascending order in a first list; a second recording submodule, configured to separate the shelves into layers using the commodity price tag detection method alone, and record the vertical coordinates of the layer dividing lines of each shelf layer in ascending order in a second list; A first setting submodule is configured to set the layer dividing line according to the commodity price tag detection method when the number of shelf layers in the first list is consistent with the number of shelf layers in the second list; a first matching submodule configured to, when the number of shelf layers in the first list is inconsistent with the number of shelf layers in the second list, perform matching processing on the second list based on the first list; determine, when the absolute value of the difference between the basic value in the first list and the matching value in the second list is less than a third threshold, that the basic value and the matching value are matched, and that the shelf layer corresponding to the basic value and the shelf layer corresponding to the matching value are the same shelf layer; and treat the shelf layer corresponding to the basic value that has not been matched and the shelf layer corresponding to the matching value that has not been matched as separate shelf layers; The fourth division submodule is used to use the layer division line obtained by the product price tag detection method as the final layer division line when there is a layer division line obtained by the product price tag detection method on a certain shelf layer; otherwise, use the layer division line obtained by the product display detection method as the final layer division line.

5. The shelf layering device according to claim 4, characterized in that: The layered module also includes: A third marking submodule is configured to mark all the strip screens on the shelf with a detection frame using a strip screen detection algorithm; The third calculation submodule is used to calculate the vertical coordinate of the center point of each strip screen detection frame; The third sorting submodule is used to sort the shelf layers in ascending order according to the vertical coordinates of the center points of the bar screen detection frames in each shelf layer, marking them as the first layer, the second layer, ..., the Pth layer in sequence; The third division submodule is used to determine the layer division line of each shelf layer according to the vertical coordinate of the center point of each strip screen detection frame in each shelf layer; The third stratification submodule is configured to stratify the shelves according to the layer dividing lines.

6. The shelf layering device according to claim 5, characterized in that: The layered module also includes: a third recording submodule, configured to separate the shelves into layers using the commodity arrangement detection method alone, and record the vertical coordinates of the layer dividing lines of each shelf layer in ascending order in a third list; A fourth recording submodule, configured to separate the shelves into layers using the strip screen detection method alone, and record the vertical coordinates of the layer dividing lines of each shelf layer in ascending order in a fourth list; A second setting submodule is configured to set the layer dividing line according to the strip screen detection method when the number of shelf layers in the third list is consistent with the number of shelf layers in the fourth list; a second matching submodule configured to, when the number of shelf layers in the third list is inconsistent with the number of shelf layers in the fourth list, perform matching processing on the fourth list based on the third list; when the absolute value of the difference between the basic value in the third list and the matching value in the fourth list is less than a fourth threshold, determine that the basic value and the matching value are matched, and the shelf layer corresponding to the basic value and the shelf layer corresponding to the matching value are the same shelf layer; and treat the shelf layer corresponding to the basic value that has not been matched and the shelf layer corresponding to the matching value that has not been matched as separate shelf layers; The fifth division submodule is used to use the layer division line obtained by the strip screen detection method as the final layer division line when there is a layer division line obtained by the strip screen detection method on a certain shelf layer; otherwise, use the layer division line obtained by the product display detection method as the final layer division line.

Images