Method, terminal and storage device for obtaining shelf layers of commodities
By using the price tag set to determine the direction line of the shelf layer when shooting on the side of the shelf, the problem of poor accuracy of the position calculation of the commodity layer column in the prior art is solved, and more accurate commodity inventory and management is achieved.
Patent Information
- Application Number
- CN202110484479.0
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2021-04-30
- Publication Date
- 2024-11-12
- Estimated Expiration
- 2041-04-30
AI Technical Summary
When shooting on the side of the shelf, it is difficult to accurately obtain the layer position information of the product, resulting in poor identification accuracy and difficult to inventory and manage.
The price card collection is obtained through the layer column images, identify the layer where the price card is located, and use the price card of the same layer to determine the direction line of the shelf layer, thereby determining the products located at the same layer, reducing the impact of shooting angle on the calculation, and avoiding the products of different layers of products Mistaked.
Improve the accuracy of product layer column position calculation, making it easier to inventory and manage products.
Smart Images

Figure CN113269053B_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the technical field of shelf inventory, and in particular to a method, a terminal and a storage device for obtaining shelf layers of commodities. Background Art
[0002] Commodity layer and column calculation refers to calculating the layer (row) and column position of each commodity on the shelf. The existing technology is to take a layer and column image of the shelf, obtain the detection frame information of the commodity based on the layer and column image, search for adjacent frames based on the horizontal direction of the detection frame in the image and connect them to form the shelf layer where the commodity is placed, and then determine the column position based on the order in the shelf layer.
[0003] Among them, when the shooting angle is perpendicular to the shelf plane, the products on the same layer in the layered image are basically at the same horizontal height. The horizontal direction of the product detection frame is used as the standard for searching, and the layered information of the products can be calculated. The reality is that the space in front of many commodity shelves is small, and it is difficult to shoot vertically in front of the shelves. The shelf is shot from the side of the shelf, and the angle between the shooting direction and the shelf plane is much less than 90 degrees.
[0004] When photographing from the side of the shelf, the products on the same shelf layer in the layered image remain horizontal. If the search is based on the horizontal direction, it may lead to products on different layers. In addition, due to the different sizes of the placed products, the accuracy of the product detection frame identified by the image algorithm is low, and the product detection frames of adjacent layers and abnormal product detection frames are easily misclassified into the same shelf layer. Therefore, the existing recognition method has poor recognition accuracy and it is difficult to obtain the layer position information of the product, which is not conducive to inventory and management of the products. Summary of the invention
[0005] In order to overcome the shortcomings of the prior art, the present invention proposes a method, terminal and storage device for obtaining shelf layer arrays of goods, which obtain a set of goods and a set of price tags through layer array images, identify the layer where the price tags are located through the price tag set, obtain a directional straight line of the shelf layer using the price tags on the same layer, and use the directional straight line to determine the goods located on the same shelf layer. The directional straight line can be obtained by using the price tags to reduce the influence of the shooting angle on the calculation of the commodity layer array information, and the isolation effect of the directional straight line can prevent goods on different layers from being mistakenly classified into the same layer, thereby improving the accuracy of the commodity layer array position calculation and facilitating the inventory and management of the goods.
[0006] To solve the above problems, the present invention adopts a method for obtaining shelf layer columns of goods, and the method for obtaining shelf layer columns of goods comprises: S101: obtaining a layer column object set according to a layer column image of a shelf, the layer column object set storing position information and size information of the layer column objects, including a price tag set and a commodity set, and the layer column objects include price tags and commodities; S102: obtaining adjacent price tags of a price tag according to the price tag set, determining a first adjacent price tag according to a first connecting line formed by the price tag and the adjacent price tags, and putting price tags in the same layer into the same first layer candidate set, and obtaining a direction straight line of the layer where the price tag is located through the first layer candidate set, wherein the price tag is in the same layer as the first adjacent price tag; S103: obtaining adjacent commodities of the commodity according to the commodity set and the direction straight line, determining a first adjacent commodity according to a second connecting line between the commodity and the adjacent commodity, and putting commodities in the same layer into the same second layer candidate set, wherein the commodity is in the same layer as the first adjacent commodity; S104: obtaining the column position of the commodity according to the position information of the commodities in the second layer candidate set.
[0007] Furthermore, the step of acquiring a set of array objects according to the array image of the shelf specifically includes: identifying price tags and commodities in the array image, and forming a price tag set and a commodity set according to the identification result.
[0008] Furthermore, the step of obtaining adjacent price tags of a price tag based on the price tag set specifically includes: determining the center of the price tag based on the position information of the price tag, obtaining a price tag detection domain through the center and the height of the price tag, and using the price tag detection domain to obtain adjacent price tags of the price tag.
[0009] Furthermore, the step of determining the first adjacent price tag based on the first connecting line formed by the price tag and the adjacent price tag specifically includes: determining whether the angle of the first connecting line is less than a preset angle; if so, removing the price tag from the price tag set; if not, determining that the adjacent price tag is the first adjacent price tag.
[0010] Furthermore, the step of obtaining the direction straight line of the layer where the price signs are located through the first layer candidate set specifically includes: identifying and eliminating abnormal price signs through a linear regression algorithm, and fitting the direction straight line.
[0011] Furthermore, the step of obtaining adjacent products of the product based on the product set and the directional straight line specifically includes: determining the center of the product based on the location information of the product, forming a product detection domain through the directional straight line closest to the center, and searching for adjacent products of the product along the directional straight line.
[0012] Furthermore, the step of obtaining the column position of the product by sorting the position information of the products in the second-layer candidate set specifically includes: obtaining the coordinate axis having the smallest angle with the direction straight line in the two-dimensional coordinate system corresponding to the position information, sorting the coordinate values according to the coordinate values of each element in the second-layer candidate set on the coordinate axis, and determining the layer column position of the product according to the coordinate value sorting.
[0013] Furthermore, after the step of sorting and obtaining the column position of the product according to the position information of the products in the second-layer candidate set, the step also includes: obtaining a second-layer candidate set in which the number of elements is less than a preset threshold, sorting the elements in the second-layer candidate set according to their coordinate values on another coordinate axis, and determining the layer position of the product corresponding to the element through the sorting result.
[0014] Based on the same inventive concept, the present invention also proposes an intelligent terminal, which includes a processor and a memory, wherein the processor is communicatively connected to the memory, the memory stores a computer program, and the processor executes the above-mentioned method for obtaining shelf layer columns of goods according to the computer program.
[0015] Based on the same inventive concept, the present invention also proposes a storage device, which stores program data, and the program data is used to execute the above-mentioned method for acquiring the shelf layer column of commodities.
[0016] Compared with the prior art, the beneficial effects of the present invention are: obtaining a set of goods and a set of price tags through layer images, identifying the layer where the price tags are located through the price tag set, obtaining the direction line of the shelf layer using the price tags on the same layer, and determining the goods located on the same shelf layer using the direction line. The direction line can be obtained by using the price tags to reduce the influence of the shooting angle on the calculation of the commodity layer information, and the isolation effect of the direction line can prevent goods on different layers from being mistakenly classified into the same layer, thereby improving the accuracy of the calculation of the commodity layer position and facilitating the inventory and management of the goods. BRIEF DESCRIPTION OF THE DRAWINGS
[0017] Figure 1 It is a flow chart of an embodiment of a method for obtaining shelf layers of commodities of the present invention;
[0018] Figure 2 A flowchart of an embodiment of searching for adjacent price tags in a method for acquiring a shelf array of commodities according to the present invention;
[0019] Figure 3 This is a structural diagram of an embodiment of a smart terminal of the present invention;
[0020] Figure 4 FIG. 4 is a structural diagram of a storage device according to an embodiment of the present invention. DETAILED DESCRIPTION
[0021] The present invention is further described below in conjunction with the accompanying drawings and specific implementation methods. It should be noted that, under the premise of no conflict, the various embodiments or technical features described below can be arbitrarily combined to form a new embodiment.
[0022] See also Figure 1-2 ,in, Figure 1 It is a flow chart of an embodiment of a method for obtaining shelf layers of commodities of the present invention; Figure 2 The flowchart of an embodiment of searching for adjacent price tags in the method for obtaining the shelf layer of commodities of the present invention. Figure 1-2 The method for acquiring the shelf layer array of commodities according to the present invention is described.
[0023] In this embodiment, the device executing the method for obtaining the shelf layer of the commodity can be a computer, a server, a control platform, or other intelligent terminals that can obtain a price tag set and a commodity set, and determine the method for obtaining the shelf layer of the commodity according to the price tag set and the commodity set. The method for obtaining the shelf layer of the commodity includes:
[0024] S101: Acquire a layer column object set according to a layer column image of a shelf. The layer column object set stores position information and size information of the layer column objects, including a price tag set and a commodity set. The layer column objects include price tags and commodities.
[0025] In this embodiment, the step of acquiring a set of array objects according to the array image of the shelf specifically includes: identifying price tags and commodities in the array image, and forming a price tag set and a commodity set according to the identification result.
[0026] In this embodiment, the position information and size information of the price tag and the commodity are the position information and size information of the price tag, the price tag detection frame corresponding to the commodity, and the commodity detection frame in the layered image.
[0027] In a specific embodiment, the upper left corner of the layered image is taken as the origin, the horizontal direction of the layered image is the X axis, and the vertical direction is the Y axis, and a coordinate system is formed by the X axis and the Y axis. The price tag and the commodity in the layered image are identified by an image recognition algorithm or model, and the price tag detection frame and the commodity detection frame corresponding to the price tag and the commodity are respectively placed in the price tag set and the commodity set, wherein the price tag detection frame includes the position information and size information of the price tag detection frame in the coordinate system, and the commodity detection frame includes the position information and size information of the commodity in the coordinate system.
[0028] S102: Obtain the adjacent price tags of the price tag according to the price tag set, determine the first adjacent price tag according to the first connecting line formed by the price tag and the adjacent price tag, and put the price tags in the same layer into the same first layer candidate set, and obtain the direction straight line of the layer where the price tag is located through the first layer candidate set, wherein the price tag is in the same layer as the first adjacent price tag.
[0029] In this embodiment, the step of obtaining the adjacent price tags of the price tag according to the price tag set specifically includes: determining the center of the price tag according to the position information of the price tag, obtaining the price tag detection domain through the center and the height of the price tag, and obtaining the adjacent price tags of the price tag using the price tag detection domain. The coordinates of the center of the price tag are determined by the position information of the price tag detection frame corresponding to the price tag.
[0030] In this embodiment, the center line of the price tag detection domain passes through the center of the price tag, and the distance between the center of the price tag and both sides of the price tag detection domain is 1.5 times the height of the price tag. The price tag detection domain extends to both sides of the price tag along the horizontal direction of the layered image.
[0031] In other embodiments, the distance between the center of the price tag and the two sides of the price tag detection field is 1 times, 2 times, or other multiples of the height of the price tag.
[0032] In this embodiment, after obtaining the price tag detection domain, a searched price tag that overlaps with the price tag detection domain is horizontally searched along both sides of the price tag, and the distance between the searched price tag and the price tag serving as the search starting point in the axial direction of the layered image X is obtained, and the searched price tag with the smallest distance on both sides of the price tag is taken as the adjacent price tag of the price tag.
[0033] In a specific embodiment, any price sign a is selected as the starting point in the price sign set S. The size of the price sign detection frame corresponding to price sign a is (h, w), with the center point (x_mid, y_mid) of the detection frame as the center, (y_mid-3h / 2, y_mid+3h / 2) as the coordinates of the points on the two sides of the price sign detection domain closest to the center point, and search leftward from the horizontal direction to find the adjacent price sign l that overlaps with the neighborhood and is closest to the x-axis direction; search rightward in the above manner to find the adjacent price sign r on the right. Price signs l and r are regarded as the adjacent price signs of price sign a.
[0034] In this embodiment, the step of determining the first adjacent price tag based on the first connecting line formed by the price tag and the adjacent price tag specifically includes: determining whether the angle of the first connecting line is less than a preset angle; if so, removing the price tag from the price tag set; if not, determining that the adjacent price tag is the first adjacent price tag. The angle of the first connecting line is the angle formed by the price tag and the adjacent price tags on both sides thereof. The first adjacent price tag is determined to be located on the same layer of the shelf as the price tag serving as the starting point.
[0035] In other embodiments, when it is determined that the included angle is less than a preset angle, the price sign may be placed in a layer candidate set corresponding to the abnormal price sign.
[0036] In this embodiment, the preset angle is 160 degrees. In other embodiments, the preset angle may also be 150 degrees, 155 degrees, or other degrees, which are not limited here.
[0037] In this embodiment, after obtaining the first adjacent price tag of the price tag, the price tag is moved from the price tag set into the first-layer candidate set, and the search continues horizontally on both sides for price tags located at the same layer in the price tag set, starting with the first adjacent price tag, until the angle between the searched adjacent price tag and the first connecting line of the price tag serving as the starting point is less than a preset angle.
[0038] In a specific embodiment, the set of price tags is set S, the starting price tag is price tag a, and the adjacent price tags are price tags r and price tag l. When the angle between price tag a and the line connecting l and r is less than 160 degrees, a is deleted from set S, and another price tag is selected from set S for search. When the angle between price tag a and the line connecting l and r is greater than 160 degrees, a is added to the empty first-layer candidate set L. With l as the starting point, continue to search to the left until the angle between a certain price tag and the line connecting the left and right adjacent price tags is less than 160 degrees, and stop, and add the searched price tags that meet the angle condition to the first-layer candidate set L; with r as the starting point, continue to search to the right in the above manner until it stops, and also add the searched price tags that meet the angle condition to the first-layer candidate set L.
[0039] In this embodiment, the step of obtaining the direction straight line of the layer where the price signs are located through the first layer candidate set specifically includes: identifying and eliminating abnormal price signs through a linear regression algorithm, and fitting the direction straight line.
[0040] Before obtaining the direction line of the price sign through the first-layer candidate set, it is necessary to determine whether the number of elements in the first-layer candidate set is greater than a preset value. If it is greater than the preset value, the direction line is obtained; if not, the search for adjacent price signs continues and the price signs whose connecting line angle with the elements in the first-layer candidate set is less than the preset angle are put into the first-layer candidate set.
[0041] In this embodiment, the preset value is 3. In other embodiments, the preset value may also be 4, 5, 6 or other values.
[0042] In a specific embodiment, the preset value is 3. After determining that the number of price tags in the first layer candidate set is greater than 3, a linear regression algorithm with random sampling consistency is used to identify and eliminate abnormal price tags (price tags not in the same layer), and a direction line is fitted using the price tags that are not eliminated. The direction line can provide layer direction information for the adjacent layers of goods, and serve as isolation information between the layers of adjacent goods.
[0043] In the above embodiment, the adjacent price tags of the price tags are continuously searched, and the price tags at the same layer are put into the same first layer candidate set until it is determined that the element in the price tag candidate set is zero.
[0044] S103: Obtaining adjacent products of the product according to the product set and the directional line, determining the first adjacent product according to the second connecting line between the product and the adjacent product, and putting products in the same layer into the same second layer candidate set, wherein the product and the first adjacent product are in the same layer.
[0045] In this embodiment, the step of obtaining adjacent products of a product according to a product set and a directional line specifically includes: determining the center of the product according to the location information of the product, forming a product detection domain through a directional line closest to the center, and searching for adjacent products of the product along the directional line.
[0046] In this embodiment, the steps of determining the first adjacent commodity based on the second connecting line between the commodity and the adjacent commodities and placing the commodities on the same layer into the same second layer candidate set are the same as the steps of searching for the first price tag and placing the price tags on the same layer into the same first layer candidate set mentioned above, and will not be described in detail here.
[0047] In a specific embodiment, a product a is randomly selected from the product set S as the starting point. The size of the product detection frame corresponding to product a is (h, w), with the center point (x_mid, y_mid) of the detection frame as the center, and (y_mid-3h / 2, y_mid+3h / 2) as the coordinates of the points on the two sides of the product detection domain closest to the center point. The direction of the direction line closest to the point is selected as the extension method of the product detection domain. Search for adjacent products overlapping with the product detection domain within the product detection domain, start from the direction of the direction line and search to the left to find the product l that overlaps with the product detection domain and is closest to the x-axis direction; search to the right in the above manner to find the adjacent product r on the right. And when the angle between the second connecting lines of products a, l, and r is greater than the preset angle, add product a to the empty second layer candidate set L. Take product l as the starting point and continue to search to the left in the above manner until no product is found in the product detection range. Similarly, take r as the starting point and continue to search to the right in the above manner until it stops. Add the searched products to the layer candidate set.
[0048] In the above embodiment, the adjacent commodities of a commodity are continuously searched, and commodities at the same layer are put into the same second-layer candidate set until it is determined that the element in the commodity candidate set is zero.
[0049] S104: Sort the position information of the products in the second layer candidate set and obtain the column position of the products.
[0050] In this embodiment, the step of obtaining the column position of the product by sorting the position information of the products in the second-layer candidate set specifically includes: obtaining the coordinate axis with the smallest angle with the direction straight line in the two-dimensional coordinate system corresponding to the position information, sorting the coordinate values according to the coordinate values of each element in the second-layer candidate set on the coordinate axis, and determining the layer column position of the product according to the coordinate value sorting.
[0051] In a specific embodiment, the commodities in the second layer candidate set are sorted according to the X-axis coordinates of the upper left corners of the commodity detection boxes corresponding to the commodities in the layer column image, and the sorting results are used as the column positions of the commodities in the same layer on the shelf.
[0052] In this embodiment, since there are few commodities on some layers of the shelves, the commodities on this layer are identified as abnormal commodities. After the step of sorting and obtaining the column position of the commodities according to the position information of the commodities in the second-layer candidate set, the step also includes: obtaining a second-layer candidate set in which the number of elements is less than a preset threshold, sorting the elements in the second-layer candidate set according to their coordinate values on another coordinate axis, and determining the layer position of the commodity corresponding to the element through the sorting result.
[0053] In this embodiment, the preset threshold can be set according to the application environment and user needs, and is not limited here.
[0054] The beneficial effects of the present invention are as follows: a set of goods and a set of price tags are obtained through layer images, the layer where the price tags are located is identified through the price tag set, the direction line of the shelf layer is obtained using the price tags on the same layer, and the goods located on the same shelf layer are determined using the direction line. The direction line can be obtained through the price tags to reduce the influence of the shooting angle on the calculation of the commodity layer information, and the isolation effect of the direction line can prevent goods on different layers from being mistakenly classified into the same layer, thereby improving the accuracy of the commodity layer position calculation and facilitating the inventory and management of the goods.
[0055] Based on the same inventive concept, the present invention also proposes an intelligent terminal, see Figure 3 , Figure 3 This is a structural diagram of an embodiment of the intelligent terminal of the present invention, combined with Figure 3 The intelligent terminal of the present invention is described.
[0056] In this embodiment, the intelligent terminal includes a processor and a memory. The processor is communicatively connected to the memory. The memory stores a computer program. The processor implements the method for obtaining the shelf layer column of the commodity as described in the above embodiment according to the computer program.
[0057] The processor is used to control the overall operation of the smart terminal to complete all or part of the steps in the above-mentioned method for obtaining the shelf layer of the commodity. The memory is used to store various types of data to support the operation of the smart terminal, and these data may include, for example, instructions for any application or method used to operate on the smart terminal, and application-related data, such as contact data, sent and received messages, pictures, audio, video, etc. The memory can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic memory, flash memory, disk or optical disk.
[0058] Based on the same inventive concept, the present invention also proposes a storage device, see Figure 4 , Figure 4 FIG. 1 is a structural diagram of an embodiment of a storage device of the present invention, Figure 4 The storage device of the present invention will be described.
[0059] In this embodiment, the storage device stores program data, and the program data is used to execute the method for acquiring the shelf layer column of commodities as described in the above embodiment.
[0060] The various embodiments in this specification are described in a progressive manner, and each embodiment focuses on the differences from other embodiments. The same or similar parts between the various embodiments can be referenced to each other.
[0061] The above description of the disclosed embodiments enables those skilled in the art to implement or use the present invention. Various modifications to these embodiments will be apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the present invention. Therefore, the present invention will not be limited to the embodiments shown herein, but rather to the widest scope consistent with the principles and novel features disclosed herein.
Claims
1. A method for obtaining a shelf layer of a commodity, characterized in that: The method for obtaining the shelf layer of the commodity comprises: S101: acquiring a layer column object set according to a layer column image of a shelf, wherein the layer column object set stores position information and size information of the layer column objects, including a price tag set and a commodity set, wherein the layer column objects include price tags and commodities; S102: Acquire the adjacent price signs of the price sign according to the price sign set, determine the first adjacent price sign according to the first connecting line formed by the price sign and the adjacent price sign, put the price signs in the same layer into the same first layer candidate set, and acquire the direction straight line of the layer where the price signs are located through the first layer candidate set, wherein the price sign and the first adjacent price sign are in the same layer; S103: Acquire adjacent products of the product according to the product set and the directional straight line, determine the first adjacent product according to the second connecting line between the product and the adjacent product, and put products in the same layer into the same second layer candidate set, wherein the product and the first adjacent product are in the same layer; S104: Sort the position information of the products in the second-layer candidate set and obtain the column position of the products.
2. The method for obtaining the shelf layer of commodities according to claim 1, characterized in that: The step of obtaining a layer column object set according to the layer column image of the shelf specifically includes: The price tags and commodities in the layered image are identified, and a price tag set and a commodity set are formed according to the identification results.
3. The method for obtaining the shelf layer of commodities according to claim 1, characterized in that: The step of obtaining the adjacent price tags of the price tag according to the price tag set specifically includes: The center of the price tag is determined according to the position information of the price tag, a price tag detection domain is obtained through the center and the height of the price tag, and the price tag detection domain is used to obtain the adjacent price tags of the price tag.
4. The method for obtaining the shelf layer of commodities according to claim 1, characterized in that: The step of determining the first adjacent price sign according to the first connection line formed by the price sign and the adjacent price sign specifically includes: Determining whether the included angle of the first connecting line is less than a preset angle; If so, removing the price tag from the price tag set; If not, it is determined that the adjacent price sign is the first adjacent price sign.
5. The method for obtaining the shelf layer of commodities according to claim 1, characterized in that: The step of obtaining the direction straight line of the layer where the price sign is located through the first layer candidate set specifically includes: Identify and eliminate abnormal price signs through linear regression algorithm, and fit the directional straight line.
6. The method for obtaining the shelf layer of commodities according to claim 1, characterized in that: The step of obtaining adjacent commodities of the commodity according to the commodity set and the direction straight line specifically includes: The center of the commodity is determined according to the position information of the commodity, and a commodity detection domain is formed by a directional straight line closest to the center, and adjacent commodities of the commodity are searched along the directional straight line.
7. The method for obtaining the shelf layer of commodities according to claim 1, characterized in that: The step of obtaining the column position of the product according to the position information of the products in the second layer candidate set is specifically as follows: Obtain a coordinate axis having the smallest angle with the direction straight line in the two-dimensional coordinate system corresponding to the position information, and determine the column position of the product according to the coordinate value of each element in the second-layer candidate set on the coordinate axis and the coordinate value sorting.
8. The method for obtaining the shelf layer of commodities according to claim 7, characterized in that: After the step of sorting and obtaining the column position of the product according to the position information of the products in the second layer candidate set, the following step is further included: A second layer candidate set having an element number less than a preset threshold is obtained, the elements in the second layer candidate set are sorted according to their coordinate values on another coordinate axis, and the layer position of the commodity corresponding to the element is determined according to the sorting result.
9. An intelligent terminal, characterized in that: The intelligent terminal includes a processor and a memory, wherein the processor is in communication with the memory, the memory stores a computer program, and the processor implements the method for obtaining the shelf layer column of the commodity according to any one of claims 1 to 8 through the computer program.
10. A storage device, characterized in that: The storage device stores program data, and the program data is used to execute the method for acquiring shelf layers of commodities as described in any one of claims 1-8.
