Article Distribution Recognition Method, Device, Electronic Device and Readable Storage Medium

Abstract

The present application provides an article distribution recognition method, apparatus, electronic device, and readable storage medium. Among them, the method includes: recognizing a to-be-tested image of a to-be-tested area to determine article information of articles included in the to-be-tested area, where the article information includes article types and article coordinates; determining distribution data of each article in the to-be-tested area according to the article types and article coordinates; and determining a concentration result of each article according to the distribution data. Based on type recognition and concentration determination, it is possible to more efficiently determine whether the furnishings of articles are reasonable.

Description

Technical Field

[0001] The present application relates to the technical field of item distribution recognition, and more particularly, to an item distribution recognition method, apparatus, electronic device, and readable storage medium. Background Art

[0002] In a retail scenario, to form a good sales order, items are usually displayed according to their types. For items of the same type, they are displayed according to their brands. For example, items of the same brand are displayed together. However, inevitably, not all items of the same brand are exactly displayed together, or as items are being shown, it is possible that when users pick up items intermittently, the display arrangement of the items may be changed.

[0003] Currently, when items of the same brand or the same type are not displayed together, it is usually achieved by relevant staff making on-site inspections. However, although this on-site inspection method can check the distribution of items, the efficiency is relatively low. Summary of the Invention

[0004] In view of this, the purpose of the embodiments of the present application is to provide an item distribution recognition method, apparatus, electronic device, and readable storage medium, which can solve the problem of relatively low efficiency of the existing on-site inspection method.

[0005] In a first aspect, the embodiments of the present application provide an item distribution recognition method, including:

[0006] Identifying a test image of a test area to determine item information of items included in the test area, where the item information includes item type and item coordinates;

[0007] Determining distribution data of each item in the test area according to the item type and item coordinates;

[0008] Determining a concentration result of each item according to the distribution data.

[0009] In an optional implementation, the distribution data includes: an item coverage area; and determining a concentration result of each item according to the distribution data includes:

[0010] Determining an intersection coverage area between items of each item type according to the item coverage area of each item type;

[0011] Determining a concentration result of each item according to the intersection coverage area and the item coverage area.

[0012] In the above embodiments, the cross-coverage area can characterize the cross-situation of the furnishings of each item, so that it can be further determined whether the items are concentrated. Further, determining the cross-coverage area first can relatively quickly determine whether it is a concentrated furnishings.

[0013] In an alternative embodiment, the determining the distribution data of each item in the to-be-detected area according to the item type and the item coordinates includes:

[0014] For the items of each item type, based on the item coordinates, construct the coverage area of each item type.

[0015] In an alternative embodiment, the coverage area of each item type includes: the convex hull area formed by all the items of each item type; the constructing the coverage area of each item type based on the item coordinates includes:

[0016] Determine the first set of item coordinates of the outermost items among all the items of the target item type, where the target item type is any one of all the item types included in the to-be-detected image;

[0017] According to the first set of item coordinates, determine the convex hull area formed by the target item type.

[0018] In the above embodiments, based on the convex hull algorithm, the convex hull formed by the furnishings of the target item can be determined more accurately, so that the surrounding area where the target item is located can be determined better, so that it can be more accurately identified whether other items are within the target item area, and thus it can be more efficiently determined whether the target item is concentratedly furnished.

[0019] In an alternative embodiment, the first set of item coordinates includes: the first edge coordinate set, the second edge coordinate set, the third edge coordinate set, and the fourth edge coordinate set; the determining the first set of item coordinates of the outermost items among all the items of the target item type includes:

[0020] According to the first item area parameter, the second item area parameter, the third item area parameter, and the fourth item area parameter among all the items of the target item type, where each item area parameter includes at least two coordinate parameters;

[0021] According to the first item area parameter and the second item area parameter, determine the first edge coordinate set;

[0022] According to the second item area parameter and the third item area parameter, determine the second edge coordinate set;

[0023] Determine a third set of edge coordinates according to the third item area parameter and the fourth item area parameter;

[0024] Determine a fourth set of edge coordinates according to the first item area parameter and the fourth item area parameter.

[0025] In the above embodiment, the situation where the determined convex hull area is large enough to include some redundant areas can be reduced, so that the judgment on whether the items are centrally displayed can be realized more accurately.

[0026] In an alternative embodiment, the determining the convex hull area formed by the target item type according to the first set of item coordinates includes:

[0027] Connect the coordinates in the first set of item coordinates in sequence along the direction of the first axis and / or the direction of the second axis in the specified coordinate system to determine the convex hull area formed by the target item type.

[0028] In the above embodiment, it can be connected only in the two-axis direction, so that the determined convex hull area can more appropriately include the convex hull formed by the target item, and thus the result of whether the determined target item is centrally displayed can also be more accurate.

[0029] In an alternative embodiment, the determining the concentration result of each item according to the cross-coverage area and the item coverage area includes:

[0030] Calculate the ratio of the cross-coverage area of the first item and the second item to the item coverage area of the first item;

[0031] Compare the ratio with a preset value to determine whether the second item is concentrated;

[0032] Wherein, if it is greater than the preset value, it means that the second item is not centrally displayed, and if it is not greater than the preset value, it means that the second item is centrally displayed.

[0033] In a second aspect, an item distribution recognition device provided by an embodiment of the present application includes:

[0034] An identification module, configured to identify a to-be-tested image of a to-be-tested area to determine item information of items included in the to-be-tested area, where the item information includes item type and item coordinates;

[0035] A first determination module, configured to determine distribution data of each item in the to-be-tested area according to the item type and item coordinates;

[0036] The second determination module is used to determine the concentration result of each item according to the distribution data.

[0037] In a third aspect, an embodiment of the present application further provides an electronic device, comprising: a processor and a memory, wherein the memory stores machine-readable instructions executable by the processor, and when the electronic device is running, the machine-readable instructions are executed by the processor to perform the steps of the object distribution identification method in the above-mentioned first aspect, or any possible implementation of the first aspect.

[0038] In a fourth aspect, an embodiment of the present application further provides a computer-readable storage medium having a computer program stored thereon, and when the computer program is executed by a processor, the steps of the object distribution identification method according to the above-mentioned first aspect, or any possible implementation of the first aspect are executed.

[0039] The object distribution identification method, device, electronic device and readable storage medium provided in the embodiments of the present application use image recognition and determination of object distribution to detect the display of objects without manual visits.

[0040] In order to make the above-mentioned objects, features and advantages of the present application more obvious and understandable, embodiments are given below and described in detail with reference to the attached drawings. BRIEF DESCRIPTION OF THE DRAWINGS

[0041] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the drawings required for use in the embodiments will be briefly introduced below. It should be understood that the following drawings only show certain embodiments of the present application and therefore should not be regarded as limiting the scope. For ordinary technicians in this field, other related drawings can be obtained based on these drawings without paying creative work.

[0042] Figure 1 A block diagram of an electronic device provided in an embodiment of the present application;

[0043] Figure 2 A flowchart of the object distribution identification method provided in an embodiment of the present application;

[0044] Figure 3a A schematic diagram of an image to be tested processed by the object distribution recognition method provided in an embodiment of the present application;

[0045] Figure 3b A schematic diagram of another image to be tested processed by the object distribution recognition method provided in an embodiment of the present application;

[0046] Figure 3c A schematic diagram of another image to be tested processed by the object distribution recognition method provided in an embodiment of the present application;

[0047] Figure 4 It is a schematic diagram of the functional modules of the article distribution recognition device provided by the embodiment of the present application. Specific implementation manners

[0048] Next, the technical solutions in the embodiments of the present application will be described with reference to the accompanying drawings in the embodiments of the present application.

[0049] It should be noted that similar reference numerals and letters denote similar items in the following drawings. Therefore, once an item is defined in one drawing, it does not need to be further defined and explained in subsequent drawings. At the same time, in the description of the present application, the terms "first", "second", etc. are only used for descriptive distinction and cannot be understood as indicating or implying relative importance.

[0050] To facilitate the understanding of this embodiment, the electronic device that executes the article distribution recognition method disclosed in the embodiments of the present application will be introduced in detail first.

[0051] As Figure 1 shown, it is a block diagram of the electronic device. The electronic device 100 may include a memory 111, a storage controller 112, a processor 113, a peripheral interface 114, an input / output unit 115, and a display unit 116. Those of ordinary skill in the art can understand that Figure 1 the structure shown is only schematic and does not limit the structure of the electronic device 100. For example, the electronic device 100 may further include more or fewer components than Figure 1 shown, or have a different configuration from Figure 1 shown.

[0052] The above-mentioned memory 111, storage controller 112, processor 113, peripheral interface 114, input / output unit 115, and display unit 116 are directly or indirectly electrically connected to each other to realize data transmission or interaction. For example, these components may be electrically connected to each other through one or more communication buses or signal lines. The above-mentioned processor 113 is used to execute the executable module stored in the memory.

[0053] Among them, the memory 111 can be, but is not limited to, a Random Access Memory (RAM), a Read Only Memory (ROM), a Programmable Read-Only Memory (PROM), an Erasable Programmable Read-Only Memory (EPROM), an Electric Erasable Programmable Read-Only Memory (EEPROM), etc. Among them, the memory 111 is used to store a program. After receiving an execution instruction, the processor 113 executes the program. The method executed by the electronic device 100 defined by the process disclosed in any embodiment of the embodiments of the present application can be applied to the processor 113 or implemented by the processor 113.

[0054] The above-mentioned processor 113 may be an integrated circuit chip with signal processing capabilities. The above-mentioned processor 113 may be a general-purpose processor, including a Central Processing Unit (CPU), a Network Processor (NP), etc.; it may also be a digital signal processor (DSP), an Application Specific Integrated Circuit (ASIC), a Field Programmable Gate Array (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components. It can implement or execute the various methods, steps and logic block diagrams disclosed in the embodiments of the present application. The general-purpose processor may be a microprocessor or the processor may also be any conventional processor, etc.

[0055] The above-mentioned peripheral interface 114 couples various input / output devices to the processor 113 and the memory 111. In some embodiments, the peripheral interface 114, the processor 113, and the memory controller 112 may be implemented on a single chip. In some other instances, they may be implemented by separate chips respectively.

[0056] The above-mentioned input / output unit 115 is used to provide input data to the user. The input / output unit 115 can be, but is not limited to, a mouse, a keyboard, etc.

[0057] The above display unit 116 provides an interaction interface (such as a user operation interface) between the electronic device 100 and the user or is used to display image data for the user to refer to. In this embodiment, the display unit may be a liquid crystal display or a touch display. If it is a touch display, it may be a capacitive touch screen or a resistive touch screen that supports single-point and multi-point touch operations, etc. Supporting single-point and multi-point touch operations means that the touch display can sense touch operations generated simultaneously at one or more positions on the touch display and hand over the sensed touch operations to the processor for calculation and processing.

[0058] In this embodiment, the electronic device 100 may further include more or fewer components. For example, the electronic device 100 may further include a collection device, and the collection device is used to collect images of the area to be measured.

[0059] The electronic device 100 in this embodiment can be used to execute each step in the various methods provided in the embodiments of the present application. The implementation process of the item distribution recognition method will be described in detail through several embodiments below.

[0060] Please refer to Figure 2 , which is a flowchart of the item distribution recognition method provided in the embodiments of the present application. The following will elaborate in detail on the Figure 2 specific process shown.

[0061] Step 210, recognize the image to be measured of the area to be measured to determine the item information of the items included in the area to be measured.

[0062] In this embodiment, the item information may include the item type and the item coordinates. The item coordinates may be the coordinates of the item represented by the sub-image in the image to be measured.

[0063] In one implementation manner, the above image to be measured may be classified to determine the item types of the items in the sub-images in the image to be measured. Exemplarily, the above image to be measured may be classified through a classification model to determine the item types of the items in the sub-images in the image to be measured.

[0064] The above classification model may be a classification model pre-trained through a sample image set of the items to be recognized. Exemplarily, corresponding numbers of labels may be set according to the number of categories of the item types to be classified. In one instance, if it is necessary to recognize whether the furnishings of the items of the target item type are concentrated, two types of labels may be set. The two types of labels respectively represent the target item type and other item types. In another instance, if it is necessary to recognize whether the furnishings of the items of N item types are concentrated, N types of labels may be set, and each type of label is used to represent one item type.

[0065] In another embodiment, each sub-image in the above-mentioned image to be measured can be recognized to determine the type of item represented by each sub-image. Exemplarily, the type of item represented in each sub-image can be determined by recognizing information such as the item packaging graphics and trademarks in each sub-image.

[0066] In another embodiment, the identification codes in each sub-image of the above-mentioned image to be measured can be recognized to determine the type of item represented by each sub-image.

[0067] Step 220: Determine the distribution data of each item in the area to be measured according to the item type and the item coordinates.

[0068] Exemplarily, the distribution data can be the coordinate distribution of each item presented in the image to be measured.

[0069] Exemplarily, the coordinate distribution of each item can be presented as a closed area. For example, it can be a closed area surrounded by the coordinates of the outermost items of each item type.

[0070] Exemplarily, step 220 can be implemented as: for the items of each item type, based on the item coordinates, construct a coverage area for each item type.

[0071] In one example, the coverage area of each item type can be drawn by all the item coordinates in each item type.

[0072] In one example, as Figure 3a shown, the coverage area corresponding to item type A is area Oa1, and the coverage area corresponding to item type B is area Oa2.

[0073] In another example, as Figure 3b shown, the coverage area corresponding to item type C is area Oa3, and the coverage area corresponding to item type D is area Oa4 formed by the area Cr1 that intersects with area Oa3 and the area NCr1 that does not intersect with area Oa3.

[0074] In another example, as Figure 3c shown, another method is used to determine the coverage area corresponding to item type C. The coverage area corresponding to item type C is determined to be area Oa5, and the coverage area corresponding to item type D does not intersect with area Oa5.

[0075] Step 230: Determine the concentration result of each item according to the distribution data.

[0076] In one embodiment, it can be determined whether each item is concentrated and displayed according to the closed area formed by each item.

[0077] Optionally, it can be determined whether the items are centrally displayed according to whether there is an intersection in each enclosed area and the size of the intersection area.

[0078] Exemplarily, it can be determined whether the items are centrally displayed according to the ratio of the size of the intersection area to the size of the enclosed area formed by each item. For example, when the ratio of the size of the intersection area to the size of the enclosed area formed by each item is greater than a preset value, it can be determined that the items are not centrally displayed.

[0079] This preset value can be set according to requirements. For example, this preset value can be 80%, 70%, 75% and other values.

[0080] In an alternative embodiment, the above distribution data includes: the item coverage area. Step 230 may include: Step 231 and Step 232.

[0081] Step 231, determine the cross-coverage area between the items of each item type according to the item coverage area of each item type.

[0082] In Figure 3a the illustrated example, there is no intersection between the items of item type A and the items of item type B. Therefore, the cross-coverage area between the items of item type A and the items of item type B is zero.

[0083] In Figure 3b the illustrated example, there is an intersection between the items of item type C and the items of item type D, and the intersection area is Cr1. Therefore, the cross-coverage area between the items of item type A and the items of item type B is the area of the intersection area Cr1.

[0084] Step 232, determine the centralization result of each item according to the cross-coverage area and the item coverage area.

[0085] In Figure 3a the illustrated example, since there is no intersection between the items of item type A and the items of item type B, the centralization result of the items of item type A is centralized display.

[0086] In Figure 3b the illustrated example, there is an intersection between the items of item type C and the items of item type D, and the intersection area is Cr1. Therefore, and the area of this intersection area Cr1 accounts for a relatively large proportion of the coverage area Oa4 of the items of item type D, the centralization result of the items of item type A is non-centralized display.

[0087] In Figure 3c the illustrated example, there is no intersection between the items of item type C and the items of item type D. Therefore, the centralization result of the items of item type A is centralized display.

[0088] In an alternative embodiment, step 232 may include: calculating a ratio of the cross-over area of the first item and the second item to the item coverage area of the first item; comparing the ratio with a preset value to determine whether the second item is concentrated.

[0089] Wherein, if it is greater than the preset value, it indicates that the second item is not concentratedly displayed; if it is not greater than the preset value, it indicates that the second item is concentratedly displayed.

[0090] The preset value may be values such as 85%, 80%, 70%, 75%.

[0091] In Figure 3b In the illustrated example, the second item may be an item of item type C, and the first item is an item of item type D. The cross-over area of the first item and the second item is then the area of the cross-over region Cr1. If the ratio of the area of the cross-over region Cr1 to the area of the coverage region Oa4 of the item of item type D is greater than the preset value, it indicates that the item of item type C is not concentratedly displayed; if the ratio of the area of the cross-over region Cr1 to the area of the coverage region Oa4 of the item of item type D is not greater than the preset value, it indicates that the item of item type C is concentratedly displayed.

[0092] In an alternative embodiment, the coverage area of each item type includes: the convex hull area formed by all items of each item type.

[0093] On this basis, step 220 may include: step 221 and step 222.

[0094] Step 221, determining a first set of item coordinates of the outermost items among all items of the target item type.

[0095] The target item type is any one of all item types included in the to-be-tested image.

[0096] Exemplarily, the outermost items may refer to the items displayed at the edges.

[0097] In one example, taking Figure 3a as an example, the outermost items of item type A may be the items in the topmost first row, the items in the bottommost first row, the items in the leftmost first column, the two upper items in the third column from the left, and the bottommost item in the fourth column from the left.

[0098] In another example, taking Figure 3a as an example, the outermost items of item type A may be the items in the topmost first row, the items in the bottommost first row, the items in the leftmost first column, the uppermost item in the third column from the left, and the bottommost item in the fourth column from the left.

[0099] The first set of item coordinates may be a set of coordinates formed by the respective vertex coordinates of the peripheral items.

[0100] In this embodiment, the peripheral items may include peripheral items in four directions, and the first set of item coordinates includes: a first edge coordinate set, a second edge coordinate set, a third edge coordinate set, and a fourth edge coordinate set.

[0101] Exemplarily, the first edge coordinate set, the second edge coordinate set, the third edge coordinate set, and the fourth edge coordinate set may be four coordinate sets calculated using the XY convex hull calculation method.

[0102] Step 221 may include steps 2211 to 2215.

[0103] Step 2211, according to the first item region parameter, the second item region parameter, the third item region parameter, and the fourth item region parameter among all items of the target item type.

[0104] Wherein, each item region parameter includes at least two coordinate parameters.

[0105] The to-be-tested image may be first divided into multiple sub-regions, and the size of each sub-region may be the size of the region where one or more items are located. The above-mentioned first item region parameter, second item region parameter, third item region parameter, and fourth item region parameter are the coordinate parameters of one of the sub-regions.

[0106] Taking the size of each sub-region as the size of one item and each sub-region as a rectangle as an example below. Then the above-mentioned first item region parameter, second item region parameter, third item region parameter, and fourth item region parameter are all the coordinate parameters of an item of one of the target item types.

[0107] Exemplarily, the first item region parameter, the second item region parameter, the third item region parameter, and the fourth item region parameter may be the two diagonal vertex coordinates of the region where an item of one of the target item types is located, or may be the four diagonal vertex coordinates of the region where an item of one of the target item types is located.

[0108] Optionally, select the region parameter with the smallest x coordinate of the upper left vertex of all sub-regions as the first item region parameter. If there are multiple regions with the same smallest x coordinate of the upper left vertex, select the region parameter with the smallest y coordinate of the upper left vertex as the first item region parameter.

[0109] Optionally, select the region parameter of the region with the smallest y - coordinate of the upper - left vertex of all sub - regions as the second article region parameter. If there are multiple regions with the same smallest y - coordinate of the upper - left vertex, select the region parameter with the smallest x - coordinate of the upper - left vertex as the second article region parameter.

[0110] Optionally, select the region parameter of the region with the largest x - coordinate of the lower - right vertex of all sub - regions as the third article region parameter. If there are multiple regions with the same largest x - coordinate of the lower - right vertex, select the region parameter with the smallest y - coordinate of the upper - left vertex as the third article region parameter.

[0111] Optionally, select the region parameter of the region with the largest y - coordinate of the lower - right vertex of all sub - regions as the fourth article region parameter. If there are multiple regions with the same largest y - coordinate of the lower - right vertex, select the region parameter with the smallest x - coordinate of the upper - left vertex as the fourth article region parameter.

[0112] Step 2212: Determine the first edge coordinate set according to the first article region parameter and the second article region parameter.

[0113] Optionally, the first article region parameter and the second article region parameter can be connected to construct the first edge left - hand set. When connecting the first article region parameter and the second article region parameter, only connect along the x - axis direction or the y - axis direction.

[0114] In other instances, if the first article region parameter and the second article region parameter are region parameters of different sub - regions.

[0115] Exemplarily, represent the sub - region where the first article region parameter is located as B1, and the sub - region where the second article region parameter is located as B2.

[0116] Exemplarily, the upper - left vertex coordinates of the sub - region where the first article region parameter is located can be represented by (B1.x1, B1.y1), the lower - right vertex coordinates of the sub - region where the first article region parameter is located can be represented by (B1.x2, B1.y2), the upper - left vertex coordinates of the sub - region where the second article region parameter is located can be represented by (B2.x1, B2.y1), the lower - right vertex coordinates of the sub - region where the first article region parameter is located can be represented by (B2.x2, B2.y2), and P1 represents the first edge coordinate set.

[0117] First, (B1.x1, B1.y1) can be added to the first edge coordinate set P1.

[0118] Then, from all sub-regions in the sub-region where the first item region parameter is removed, select a first set of sub-regions whose upper left vertex y coordinate is less than the upper left vertex y coordinate of the sub-region where the first item region parameter is located, and whose upper left vertex x coordinate is between the upper left vertex x coordinate of the sub-region where the first item region parameter is located and the upper left vertex x coordinate of the sub-region where the second item region parameter is located.

[0119] Exemplarily, all sub-regions Bx1 can be screened out, where Bx1 satisfies Bx1.y1 ≤ B1.y1 and B1.x1 ≤ Bx1.x1 ≤ B2.x1. All the sub-regions Bx1 form the first set of sub-regions Phase1.

[0120] Optionally, the first set of sub-regions Phase1 formed by all the sub-regions Bx1 can be sorted. Exemplarily, it can be sorted in the order of the x coordinate of the upper left vertex as the main order. If the x coordinates of the upper left vertices are the same, then compare the y coordinates of the upper left vertices as the secondary order to obtain the first set of sub-regions Phase1.

[0121] Process the sorted first set of sub-regions Phase1 in the following way:

[0122] For each sub-region Bx1 in the first set of sub-regions Phase1:

[0123] Let (x, y) represent the coordinates of the point added to the first set of edge coordinates P1 in the previous item;

[0124] Add the point coordinates (Bx1.x1, y) to the first set of edge coordinates P1;

[0125] If Bx1.y1 is less than y, then add the point coordinates (Bx1.x1, Bx1.y1) to the first set of edge coordinates P1;

[0126] If the first set of sub-regions Phase1 is an empty set, then add the point coordinates (B2.x1, B2.y1) to the first set of edge coordinates P1.

[0127] Step 2213, determine a second set of edge coordinates according to the second item region parameter and the third item region parameter.

[0128] Optionally, the second item region parameter and the third item region parameter can be connected to construct a second set of edge coordinates. When connecting the second item region parameter and the third item region parameter, it is only connected along the x-axis direction or the y-axis direction.

[0129] Exemplarily, let B2 represent the sub-region where the second item region parameter is located, and B3 represent the sub-region where the third item region parameter is located.

[0130] Exemplarily, the upper left vertex coordinates of the sub-region where the second item area parameter is located can be represented by (B2.x1, B2.y1), the lower right vertex coordinates of the sub-region where the second item area parameter is located can be represented by (B2.x2, B2.y2), the upper left vertex coordinates of the sub-region where the third item area parameter is located can be represented by (B3.x1, B3.y1), the lower right vertex coordinates of the sub-region where the second item area parameter is located can be represented by (B3.x2, B3.y2), and P2 represents the second edge coordinate set.

[0131] Exemplarily, all sub-regions Bx2 can be filtered out, where Bx2 satisfies Bx2.y1 < B3.y1 and B2.x2 ≤ Bx2.x2 ≤ B3.x2. All the sub-regions Bx2 form the second sub-region set Phase2.

[0132] Optionally, the second sub-region set Phase2 formed by all the sub-regions Bx2 can be sorted. Exemplarily, it can be sorted in the reverse order of the main order of the x coordinate of the lower right vertex, and if the x coordinates of the lower right vertices are the same, then compare the y coordinates of the upper left vertex. The second sub-region set Phase2 is obtained.

[0133] Then the sorted second sub-region set Phase2 is processed in the following way:

[0134] For each sub-region Bx2 in the second sub-region set Phase1:

[0135] Let (x, y) represent the coordinates of the point added to the second edge coordinate set P2 last;

[0136] Add the point (Bx2.x2, y) to the second edge coordinate set P2;

[0137] If Bx2.y1 is less than y, then add the point (Bx2.x2, Bx2.y1) to the first edge coordinate set P1;

[0138] If the second sub-region set Phase2 is an empty set, then add the point (B2.x2, B2.y1) to the second edge coordinate set P2.

[0139] Step 2214, determine the third edge coordinate set according to the third item area parameter and the fourth item area parameter.

[0140] Optionally, the third item area parameter and the fourth item area parameter can be connected to construct the third edge left set. When connecting the third item area parameter and the fourth item area parameter, only connect along the x-axis direction or the y-axis direction.

[0141] Exemplarily, let B3 represent the sub-region where the third item area parameter is located, and B4 represent the sub-region where the fourth item area parameter is located.

[0142] Exemplarily, the upper left vertex coordinates of the sub-region where the third item region parameter is located can be represented by (B3.x1, B3.y1), the lower right vertex coordinates of the sub-region where the third item region parameter is located can be represented by (B3.x2, B3.y2), the upper left vertex coordinates of the sub-region where the fourth item region parameter is located can be represented by (B4.x1, B4.y1), the lower right vertex coordinates of the sub-region where the third item region parameter is located can be represented by (B4.x2, B4.y2), and P3 represents the third edge coordinate set.

[0143] Exemplarily, all sub-regions Bx3 can be filtered out, where Bx3 satisfies Bx3.y2 ≥ B3.y2 and B3.x2 ≤ Bx3.x2 ≤ B4.x2. All the sub-regions Bx3 form the third sub-region set Phase3.

[0144] Optionally, the third sub-region set Phase3 formed by all the sub-regions Bx3 can be sorted. Exemplarily, it can be sorted in the reverse order with the x coordinate of the lower right vertex as the main order, and if the x coordinates of the lower right vertices are the same, then compare the y coordinates of the lower right vertices, to obtain the third sub-region set Phase3.

[0145] Then the sorted third sub-region set Phase3 is processed in the following way:

[0146] For each sub-region Bx3 in the third sub-region set Phase3:

[0147] Let (x, y) represent the coordinates of the point added to the third edge coordinate set P3 last;

[0148] Add the point (Bx3.x2, y) to the third edge coordinate set P3;

[0149] If Bx3.y2 is greater than y, then add the point (Bx3.x2, Bx3.y2) to the third edge coordinate set P3;

[0150] If the third sub-region set Phase3 is an empty set, then add the point (B2.x2, B2.y2) to the third edge coordinate set P3.

[0151] Step 2215, determine the fourth edge coordinate set according to the first item region parameter and the fourth item region parameter.

[0152] Optionally, the first item region parameter and the fourth item region parameter can be connected to construct the fourth edge left set. When connecting the first item region parameter and the fourth item region parameter, it is only connected along the x-axis direction or the y-axis direction.

[0153] Exemplarily, let B1 represent the sub-region where the first item area parameter is located, and B4 represent the sub-region where the fourth item area parameter is located.

[0154] Exemplarily, the upper left vertex coordinates of the sub-region where the first item area parameter is located can be represented by (B1.x1, B1.y1), the lower right vertex coordinates of the sub-region where the first item area parameter is located can be represented by (B1.x2, B1.y2), the upper left vertex coordinates of the sub-region where the fourth item area parameter is located can be represented by (B4.x1, B4.y1), the lower right vertex coordinates of the sub-region where the first item area parameter is located can be represented by (B4.x2, B4.y2), and P4 represents the fourth edge coordinate set.

[0155] Exemplarily, all sub-regions BX4 can be filtered out, where BX4 satisfies BX4.y2 > B1.y2 and B1.x1 ≤ BX4.x1 ≤ B4.x1. All the sub-regions BX4 form the fourth sub-region set Phase4.

[0156] Optionally, the fourth sub-region set Phase4 formed by all the sub-regions BX4 can be sorted. Exemplarily, it can be sorted in the order of taking the x coordinate of the upper left vertex as the main order, and if the x coordinates of the upper left vertices are the same, then comparing the y coordinates of the lower right vertices as the secondary order, to obtain the fourth sub-region set Phase4.

[0157] Then the sorted fourth sub-region set Phase4 is processed in the following way:

[0158] For each sub-region BX4 in the fourth sub-region set Phase4:

[0159] Let (x, y) represent the coordinates of the point added to the fourth edge coordinate set P4 in the previous item;

[0160] Add the point (BX4.x1, y) to the first edge coordinate set P4;

[0161] If BX4.y2 is greater than y, then add the point (BX4.x1, BX4.y2) to the fourth edge coordinate set P4;

[0162] If the fourth sub-region set Phase4 is an empty set, then add the point (B4.x1, B4.y2) to the fourth edge coordinate set P4.

[0163] Step 222: Determine the convex hull region formed by the target item type according to the first item coordinate set.

[0164] Exemplarily, the coordinates in the first item coordinate set can be connected in sequence to form the convex hull region of the target item type.

[0165] Exemplarily, each coordinate in the first object coordinate set is sequentially connected along the direction of the first axis and / or the direction of the second axis in the specified coordinate system to determine the convex hull area formed by the target object type.

[0166] by Figure 3c For example, the items in the first column from the left are connected vertically in sequence, the first row from bottom to top are connected horizontally in sequence, the second row from bottom to top are connected horizontally in sequence, the items in the second column from left to right and the second row from bottom to top are connected to the items in the second column from left to right and the first row from bottom to top, the items in the second column from left to right and the first row from bottom to top are connected to the items in the third column from left to right, thereby forming a convex hull area of item type C.

[0167] In the embodiment of the present application, by using image recognition and determination of object distribution, the display of objects can be detected without manual visits.

[0168] Based on the same application concept, an item distribution identification device corresponding to the item distribution identification method is also provided in the embodiment of the present application. Since the principle of solving the problem by the device in the embodiment of the present application is similar to that in the aforementioned item distribution identification method embodiment, the implementation of the device in this embodiment can refer to the description in the embodiment of the above method, and the repeated parts will not be repeated.

[0169] See also Figure 4 , is a functional module diagram of the object distribution identification device provided in the embodiment of the present application. Each module in the object distribution identification device in this embodiment is used to execute each step in the above method embodiment. The object distribution identification device includes: an identification module 310, a first determination module 320 and a second determination module 330; wherein,

[0170] The recognition module 310 is used to recognize the image to be tested in the area to be tested to determine the object information of the object contained in the area to be tested, and the object information includes the object type and the object coordinates;

[0171] A first determination module 320, configured to determine distribution data of each object in the area to be detected according to the object type and the object coordinates;

[0172] The second determination module 330 is used to determine the concentration result of each item according to the distribution data.

[0173] In a possible implementation, the distribution data includes: an item coverage area; and a second determination module 330, including a cross determination unit and a result determination unit.

[0174] An intersection determination unit, configured to determine the intersection coverage area between items of each item type according to the item coverage area of each item type;

[0175] A result determination unit, configured to determine the concentration result of each item according to the cross-coverage area and the item coverage area.

[0176] In a possible implementation, the distribution data includes: the coverage area of each item type; the first determination module 320 is configured to:

[0177] For each item of each item type, based on the item coordinates, construct the coverage area of each item type.

[0178] In a possible implementation, the coverage area of each item type includes: the convex hull area formed by all items of each item type; the first determination module 320 includes: a coordinate set determination unit and a convex hull determination unit.

[0179] The coordinate set determination unit is configured to determine a first item coordinate set of the outermost items among all items of the target item type, where the target item type is any one of all item types included in the to-be-detected image;

[0180] The convex hull determination unit is configured to determine the convex hull area formed by the target item type according to the first item coordinate set.

[0181] In a possible implementation, the first item coordinate set includes: a first edge coordinate set, a second edge coordinate set, a third edge coordinate set, and a fourth edge coordinate set; the coordinate set determination unit is configured to:

[0182] According to the first item area parameter, the second item area parameter, the third item area parameter, and the fourth item area parameter among all items of the target item type, where each item area parameter includes at least two coordinate parameters;

[0183] Determine the first edge coordinate set according to the first item area parameter and the second item area parameter;

[0184] Determine the second edge coordinate set according to the second item area parameter and the third item area parameter;

[0185] Determine the third edge coordinate set according to the third item area parameter and the fourth item area parameter;

[0186] Determine the fourth edge coordinate set according to the first item area parameter and the fourth item area parameter.

[0187] In a possible implementation, the convex hull determination unit is configured to:

[0188] Connect each coordinate in the first item coordinate set successively along the direction of the first axis and / or the direction of the second axis in the specified coordinate system to determine the convex hull area formed by the target item type.

[0189] In a possible implementation manner, the second determination module 330 is configured to:

[0190] Calculate the ratio of the cross-coverage area of the first item and the second item to the item coverage area of the first item;

[0191] Compare the ratio with a preset value to determine whether the second item is concentrated;

[0192] Wherein, if it is greater than the preset value, it means that the second item is not concentratedly arranged, and if it is not greater than the preset value, it means that the second item is concentratedly arranged.

[0193] In addition, an embodiment of the present application further provides a computer-readable storage medium, on which a computer program is stored. When the computer program is run by a processor, it executes the steps of the item distribution recognition method described in the above method embodiment.

[0194] The computer program product of the item distribution recognition method provided by the embodiment of the present application includes a computer-readable storage medium storing program code. The instructions included in the program code can be used to execute the steps of the item distribution recognition method described in the above method embodiment. For details, refer to the above method embodiment, which will not be repeated here.

[0195] In several embodiments provided by the present application, it should be understood that the disclosed device and method can also be implemented in other ways. The device embodiments described above are only illustrative. For example, the flowcharts and block diagrams in the drawings show the possible architectures, functions, and operations of the device, method, and computer program product according to multiple embodiments of the present application. In this regard, each block in the flowchart or block diagram may represent a module, a program segment, or a part of code, and the module, program segment, or part of code includes one or more executable instructions for implementing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the blocks may occur in a different order than marked in the drawings. For example, two consecutive blocks may actually be executed substantially in parallel, and they may sometimes be executed in the reverse order, depending on the functions involved. It should also be noted that each block in the block diagram and / or flowchart, as well as the combination of blocks in the block diagram and / or flowchart, can be implemented by a dedicated hardware-based system for performing the specified functions or actions, or can be implemented by a combination of dedicated hardware and computer instructions.

[0196] In addition, in each embodiment of the present application, the functional modules can be integrated together to form an independent part, or each module can exist alone, or two or more modules can be integrated to form an independent part.

[0197] If the above-mentioned functions are implemented in the form of software functional modules and sold or used as an independent product, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application, in essence, or the part that contributes to the prior art, or a part of the technical solution, can be embodied in the form of a software product. The computer software product is stored in a storage medium and includes several instructions for causing a computer device (which can be a personal computer, a server, or a network device, etc.) to execute all or part of the steps of the methods described in each embodiment of the present application. The foregoing storage medium includes: various media such as USB flash drives, mobile hard disks, read-only memories (ROMs), random access memories (RAMs), magnetic disks, or optical discs that can store program codes. It should be noted that in this document, relational terms such as first and second are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any actual relationship or order between these entities or operations. Moreover, the term "comprises", "comprising", or any other variation thereof is intended to cover a non-exclusive inclusion, such that a process, method, article, or device comprising a series of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such process, method, article, or device. Without further limitation, the elements defined by the statement "comprising..." do not exclude the presence of additional identical elements in the process, method, article, or device comprising the said elements.

[0198] The foregoing are only the preferred embodiments of the present application and are not used to limit the present application. For those skilled in the art, the present application can have various changes and modifications. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present application shall be included within the protection scope of the present application. It should be noted that similar reference numerals and letters denote similar items in the following drawings. Therefore, once an item is defined in one drawing, it does not need to be further defined and explained in subsequent drawings.

[0199] The above is only the specific implementation manner of the present application, but the protection scope of the present application is not limited thereto. Any person skilled in the art can easily think of changes or substitutions within the technical scope disclosed by the present application, and all of them should be covered by the protection scope of the present application. Therefore, the protection scope of the present application shall be subject to the protection scope of the claims.

Claims

1. An article distribution recognition method, characterized in that, Including: Identifying a to-be-tested image of a to-be-tested area to determine item information of items included in the to-be-tested area, where the item information includes item type and item coordinates; Determining distribution data of each item in the to-be-tested area according to the item type and item coordinates; wherein, the distribution data includes: an item coverage area; Determining a concentration result of each item according to the distribution data, including: determining an intersection coverage area between items of each item type according to the item coverage area of each item type; determining a concentration result of each item according to the intersection coverage area and the item coverage area.

2. The method according to claim 1, wherein The determining the distribution data of each item in the to-be-tested area according to the item type and item coordinates includes: For items of each item type, constructing a coverage area of each item type based on the item coordinates.

3. The method according to claim 2, characterized in that, The coverage area of each item type includes: a convex hull area formed by all items of each item type; the constructing a coverage area of each item type based on the item coordinates includes: Determining a first set of item coordinates of peripheral items among all items of a target item type, where the target item type is any one of all item types included in the to-be-tested image; Determining a convex hull area formed by the target item type according to the first set of item coordinates.

4. The method according to claim 3, characterized in that, The first set of item coordinates includes: a first edge coordinate set, a second edge coordinate set, a third edge coordinate set, and a fourth edge coordinate set; the determining a first set of item coordinates of peripheral items among all items of a target item type includes: According to first item area parameters, second item area parameters, third item area parameters, and fourth item area parameters among all items of the target item type, where each item area parameter includes at least two coordinate parameters; Determining a first edge coordinate set according to the first item area parameter and the second item area parameter; Determining a second edge coordinate set according to the second item area parameter and the third item area parameter; Determining a third edge coordinate set according to the third item area parameter and the fourth item area parameter; Determining a fourth edge coordinate set according to the first item area parameter and the fourth item area parameter.

5. The method according to claim 3, characterized in that The determining a convex hull area formed by the target item type according to the first set of item coordinates includes: Connecting each coordinate in the first set of item coordinates in sequence along the direction of the first axis and / or the direction of the second axis in a specified coordinate system to determine a convex hull area formed by the target item type.

6. The method according to claim 1, wherein The determining a concentration result of each item according to the intersection coverage area and the item coverage area includes: Calculating a ratio of the intersection coverage area of a first item and a second item to the item coverage area of the first item; Comparing the ratio with a preset value to determine whether the second item is concentrated; Among them, if it is greater than a preset value, it indicates that the second item is not centrally displayed; if it is not greater than the preset value, it indicates that the second item is centrally displayed.

7. An article distribution recognition device, characterized in that, It includes: An identification module, configured to identify a to-be-tested image of a to-be-tested area to determine item information of items included in the to-be-tested area, where the item information includes item types and item coordinates; A first determination module, configured to determine distribution data of each item in the to-be-tested area according to the item types and item coordinates; among them, the distribution data includes: an item coverage area; A second determination module, configured to determine a centralized result of each item according to the distribution data; The second determination module is further configured to determine an intersection coverage area between items of each item type according to the item coverage area of each item type; and determine a centralized result of each item according to the intersection coverage area and the item coverage area.

8. An electronic device, characterized in that, It includes: A processor and a memory, where the memory stores machine-readable instructions executable by the processor, and when the electronic device runs, the machine-readable instructions are executed by the processor to perform the steps of the method according to any one of claims 1 to 6.

9. A computer-readable storage medium, characterized in that, A computer program is stored on the computer-readable storage medium, and when the computer program is run by a processor, it performs the steps of the method according to any one of claims 1 to 6.

Images