Commodity identification method, device and equipment in disordered environment and medium

Abstract

The invention relates to a commodity identification method and device in a disordered environment, equipment and a medium. The method comprises the steps of detecting category information and position information of a commodity surface in a commodity image, converting the position information into vertex coordinates, and outputting the category information and the vertex coordinates; according to the vertex coordinates, determining endpoint coordinates of edges of the commodity surfaces, according to the category information, dividing the commodity surfaces of which the category information represents that the commodity surfaces are front surfaces or back surfaces into a front-back group and dividing the commodity surfaces of which the category information represents that the commodity surfaces are end surfaces into an end surface group, and according to the endpoint coordinates, dividing the commodity surfaces of which the category information represents that the commodity surfaces are end surfaces into end surface groups; matching the commodity surfaces in the front and back face groups with the commodity surfaces in the end face group, and associating the commodity surfaces with overlapped edges; identifying the specification of the commodity surface, and recording the specification category and score value of the commodity surface; and comparing the score values of the commodity surfaces in the associated commodity surfaces, taking the specification category corresponding to the higher score value as a final identification result of the associated commodity surfaces, and taking the specification category obtained by specification identification of the unassociated commodity surfaces as a final identification result.

Description

Technical Field

[0001] The present application relates to the field of image recognition technology, and in particular to a method, device, equipment and medium for identifying commodities in a cluttered environment. Background Art

[0002] Currently, in shops that sell goods, merchants usually regularly count the inventory of goods in a cluttered environment. Although manual inventory counting can complete the task, it is time-consuming, labor-intensive and prone to errors. In order to improve inventory efficiency, many shops have begun to introduce inventory counting methods based on artificial intelligence algorithms. The current artificial intelligence inventory counting method mainly analyzes the photographed images of the goods, identifies their specifications from the end faces of the goods, and counts the results. However, the method of using only the end face for identification has defects such as the end face is not clear enough due to the merchant's shooting method, different angles, and loss of image information, and the end face features are difficult to accurately identify. These defects can easily cause problems such as missed identification or misidentification in product identification. Summary of the invention

[0003] The purpose of the present invention is to provide a method, device, equipment and medium for commodity recognition in a cluttered environment, aiming to improve the accuracy and robustness of commodity recognition.

[0004] In order to achieve the above-mentioned purpose, the first aspect of the embodiment of the present disclosure provides a method comprising: In response to turning on the commodity recognition function, a commodity image for commodity recognition is obtained, category information of all commodity faces in the commodity image and position information in the commodity image are detected, and according to the detected position information, the position information of each of the detected commodity faces is converted into vertex coordinates of four vertices of each of the commodity faces, and the category information and vertex coordinates of each of the commodity faces are output to obtain a commodity multi-face detection result, wherein the category information refers to the front side, the back side and the end side; According to the vertex coordinates of each of the product faces, the endpoint coordinates of each edge of each of the product faces are determined; according to the category information of each of the product faces, the product faces whose category information indicates that the product faces are the front side or the back side are divided into the front and back face group, and the product faces whose category information indicates that the product faces are the end face are divided into the end face group; according to the endpoint coordinates, the product faces in the front and back face group are matched with the product faces in the end face group, and the product faces in the front and back face group with overlapping edges are associated with the product faces in the end face group to obtain a product multi-face association result; According to the multi-faceted detection result of the commodity, specification recognition is performed on each detected commodity face, and the specification categories and score values ​​of all commodity faces are recorded to obtain the multi-faceted recognition result of the commodity; According to the multi-faceted association result of the product and the multi-faceted identification result of the product, the score values ​​of the product faces in each group of associated product faces are compared, and the specification category corresponding to the higher score value is taken as the final identification result of the associated product face; for the unassociated product face, the specification category obtained by the specification identification is directly taken as the final identification result.

[0005] In a possible implementation, matching the product faces in the front and back face groups with the product faces in the end face group according to the endpoint coordinates, associating the product faces in the front and back face groups with the product faces in the end face group having overlapping edges, and obtaining a product multi-face association result includes: According to the endpoint coordinates, calculating the Euclidean distances between the two endpoints of each side of the product surface in the front and back surface group and the two endpoints of each side of the product surface in the end surface group; According to the Euclidean distance, the product faces in the front and back face group are matched with the product faces in the end face group. If there is a product face in the front and back face group, and the Euclidean distances from the two end points of an edge of the product face to the two end points of an edge of any product face in the end face group are within a preset distance threshold range, then the edge of the product face in the front and back face group and the edge of the product face in the end face group are determined to be overlapping edges, and all the overlapping edges in the product faces in the front and back face group and the product faces in the end face group are matched, and the endpoint coordinates of all the product faces in the front and back face group and the product faces in the end face group with the overlapping edges and the Euclidean distances of the overlapping edges are recorded to obtain a matching result; According to the matching result, the product face of the front and back face groups having the overlapping edges and the product face of the end face are associated to obtain a product multi-face association result.

[0006] In a possible implementation, according to the matching result, associating the product face of the front and back face groups having the overlapping edges with the product face of the end face to obtain a product multi-face association result, including: According to the Euclidean distance of the overlapping edges, if multiple edges in the matching result form a group of the overlapping edges, two edges with the smallest Euclidean distance of the overlapping edges are taken as the overlapping edges, and two product faces corresponding to the two edges with the smallest Euclidean distance of the overlapping edges are associated to obtain a product association result; If the matching result includes two edges forming a group of overlapping edges, two commodity faces corresponding to the two edges forming a group of overlapping edges in the matching result are directly associated to obtain a commodity association result.

[0007] In a possible implementation, according to the matching result, associating the product face of the front and back face groups having the overlapping edges with the product face of the end face to obtain a product multi-face association result, including: According to the matching result, if the product surface of any of the products is partially blocked by the product placed in front of the product to form an overlapping edge causing a false association, then it is detected whether the blocked product surface of the product and the end surface of the product placed in front of the product have a partial overlap; If the obstructed product surface of the product partially overlaps with the end surface of the product placed in front of the product, calculate the ratio of the overlapping area of ​​the obstructed product surface and the end surface to the area of ​​the smaller surface between the obstructed product surface and the end surface; If the ratio is greater than a preset area ratio threshold, the blocked product surface and the end surface are determined as product surfaces that belong to two products but are incorrectly associated due to overlapping edges and are filtered out; If the ratio is less than or equal to the preset area ratio threshold, the blocked product surface and the end surface are determined to belong to the same product and are associated to obtain a product multi-surface association result.

[0008] In a possible implementation, calculating the ratio of the overlapping area of ​​the obstructed product surface and the end surface to the area of ​​the smaller surface between the obstructed product surface and the end surface includes: If the areas of the obstructed product surface and the end surface are not 0, then the ratio of the overlapping area of ​​the obstructed product surface and the end surface to the area of ​​the smaller surface of the two surfaces is calculated; If the area of ​​at least one of the obscured product surface and the end surface is 0, the ratio is directly set to 0.

[0009] In a possible implementation, according to the matching result, associating the product face of the front and back face groups having the overlapping edges with the product face of the end face to obtain a product multi-face association result, further comprising: According to the matching result, if the overlapping edges formed by the stacking of two products cause a false association, then according to the two adjacent edges of the overlapping edge in the first product surface and the two adjacent edges of the overlapping edge in the second product surface, the center point distance between each adjacent edge of the first product surface and each adjacent edge of the second product surface is calculated, and the two adjacent edges with the shorter center point distance are taken as a group of corresponding adjacent edges, and the remaining two adjacent edges are taken as another group of corresponding adjacent edges, to obtain the adjacent edge grouping result; According to the adjacent edge grouping result, calculating the angle between the two groups of corresponding adjacent edges; If the included angles of the two groups of corresponding adjacent edges are both within the preset angle threshold range, it is determined that the two groups of corresponding adjacent edges of the first product face and the second product face are on the same straight line, and the first product face and the second product face are determined as product faces that belong to two products but are mistakenly associated due to overlapping edges, and are filtered out; If at least one of the angles of the two groups of corresponding adjacent edges is outside the preset angle threshold range, it is determined that the corresponding adjacent edges of the first product surface and the second product surface are not on the same straight line, and the first product surface and the second product surface are determined to belong to the same product and associated to obtain a product multi-surface association result.

[0010] In a possible implementation, calculating the angle between the two groups of corresponding adjacent edges includes: According to the two groups of endpoint coordinates of the corresponding adjacent edges, grouping and calculating the coordinate difference of two adjacent edges in the corresponding adjacent edges; According to the coordinate difference, group and calculate the dot product and length of two adjacent edges of the corresponding adjacent edges; According to the dot product and the length, the cosine values ​​of two adjacent sides of the corresponding adjacent sides are calculated in groups, and the angle between the two groups of corresponding adjacent sides is obtained by calculating the arccosine function of the cosine value.

[0011] In a possible implementation, the step of performing specification recognition on each detected product facet according to the product multi-face detection result, recording the specification categories and score values ​​of all product faces, and obtaining the product multi-face recognition result includes: According to the detection results of the multiple sides of the commodity, input the surface area image of each detected surface of the commodity, and perform specification recognition on each surface area image; Output the specification category and score value of each product surface identified by the specification; The specification categories and score values ​​of all the product faces are recorded to obtain the product multi-faceted recognition result.

[0012] According to a second aspect of the embodiments of the present disclosure, there is provided a commodity identification device in a cluttered environment, the device comprising: The commodity multi-face detection module is configured to, in response to turning on the commodity recognition function, obtain a commodity image for commodity recognition, detect category information of all commodity faces in the commodity image and position information in the commodity image, convert the position information of each of the detected commodity faces into vertex coordinates of four vertices of each of the commodity faces according to the detected position information, output the category information and vertex coordinates of each of the commodity faces, and obtain a commodity multi-face detection result, wherein the category information refers to the front side, the back side, and the end side; The commodity multi-faceted association module is configured to determine the endpoint coordinates of each edge of each commodity face according to the vertex coordinates of each commodity face, and to classify the commodity face whose category information indicates that the commodity face is the front side or the back side into a front-back face group and to classify the commodity face whose category information indicates that the commodity face is the end face into an end face group according to the category information of each commodity face, and to match the commodity faces in the front-back face group with the commodity faces in the end face group according to the endpoint coordinates, and to associate the commodity faces in the front-back face group with the commodity faces in the end face group that have overlapping edges, so as to obtain a commodity multi-faceted association result; The commodity multi-faceted recognition module is configured to perform specification recognition on each detected commodity face according to the commodity multi-faceted detection result, record the specification category and score value of all commodity faces, and obtain the commodity multi-faceted recognition result; The comprehensive scoring module is configured to compare the score values ​​of the product faces in each group of associated product faces based on the multi-faceted association results of the product and the multi-faceted identification results of the product, and take the specification category corresponding to the higher score value as the final identification result of the associated product face; for unassociated product faces, directly take the specification category obtained by the specification identification as the final identification result.

[0013] According to a third aspect of an embodiment of the present disclosure, a computer-readable storage medium is provided, on which a computer program is stored, and when the program is executed by a processor, the steps of any one of the methods described in the first aspect are implemented.

[0014] According to a fourth aspect of the embodiments of the present disclosure, an electronic device is provided, including: a memory having a computer program stored thereon; A processor is used to execute the computer program in the memory to implement the steps of any one of the methods in the first aspect.

[0015] The present invention provides a method, device, equipment and medium for identifying commodities in a cluttered environment. Compared with the prior art, the present invention has the following beneficial effects: In response to turning on the product recognition function, a product image for product recognition is obtained, category information of all product faces in the product image and position information in the product image are detected, and according to the detected position information, the position information of each of the detected product faces is converted into vertex coordinates of four vertices of each of the product faces, and the category information and vertex coordinates of each of the product faces are output to obtain a product multi-face detection result, wherein the category information refers to the front side, the back side, and the end side; according to the vertex coordinates of each of the product faces, the endpoint coordinates of each edge of each of the product faces are determined, and according to the category information of each of the product faces, the product faces whose category information indicates whether the product face is the front side or the back side are divided into the front and back side groups, and the product faces whose category information indicates whether the product face is the front side or the back side are divided into the front and back side groups. The product faces of the end face are divided into the end face group, and according to the endpoint coordinates, the product faces in the front and back face group are matched with the product faces in the end face group, and the product faces in the front and back face group with overlapping edges are associated with the product faces in the end face group to obtain a product multi-face association result; according to the product multi-face detection result, specification recognition is performed on each detected product face, and the specification categories and score values ​​of all the product faces are recorded to obtain a product multi-face recognition result; according to the product multi-face association result and the product multi-face recognition result, the score values ​​of the product faces in each group of associated product faces are compared, and the specification category corresponding to the higher score value is taken as the final recognition result of the associated product face, and for the unassociated product face, the specification category obtained by the specification recognition is directly taken as the final recognition result. Product recognition in a cluttered environment can combine the information of multiple product surfaces for recognition, thereby improving the accuracy of the recognition results; the information of multiple product surfaces is more comprehensive, thereby improving the reliability of the recognition results; in this method, the front, back and end surfaces of the product can be associated and identified or identified separately, thereby improving the robustness of the method for product recognition under complex conditions such as different shooting angles and methods.

[0016] Other features and advantages of the present disclosure will be described in detail in the following detailed description. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] The accompanying drawings are used to provide a further understanding of the present disclosure and constitute a part of the specification. Together with the following specific embodiments, they are used to explain the present disclosure but do not constitute a limitation of the present disclosure. In the accompanying drawings: Figure 1 The present invention is a flow chart showing a method for identifying commodities in a cluttered environment according to an embodiment of the specification.

[0018] Figure 2 A block diagram of a commodity recognition device in a cluttered environment is shown according to an embodiment of the specification.

[0019] Figure 3 It is a block diagram of another commodity identification device in a cluttered environment according to an embodiment of the specification. DETAILED DESCRIPTION

[0020] In order to make the purpose, technical solution and advantages of the present application more clearly understood, the present application is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application.

[0021] This application provides a method for identifying commodities in a cluttered environment. Figure 1 This is a flow chart of a method for identifying commodities in a cluttered environment according to an embodiment. The method comprises: In step S11, in response to turning on the product recognition function, a product image for product recognition is obtained, category information of all product faces in the product image and position information in the product image are detected, and according to the detected position information, the position information of each of the detected product faces is converted into vertex coordinates of four vertices of each of the product faces, and the category information and vertex coordinates of each of the product faces are output to obtain a product multi-face detection result, wherein the category information refers to the front side, the back side and the end side; The category information refers to the packaging surface of the detected product surface of the product in the product image, including the front side, the back side and the end side. The position information refers to the position distribution of the detected product surface in the product image, including the actual position information actually detected and the predicted position information predicted by the target detection algorithm based on the actual position information. The vertex coordinates of the product surface can be expressed as: .

[0022] In an embodiment of the present disclosure, in response to turning on a product recognition function, a product image for product recognition is obtained, and a deep learning-based target detection algorithm is used to detect position information of all product faces in the product image. Based on the detected position information, a YOLOv8-pose deep learning model (YOLOv8 pose detection model) is used to convert the detected position information of each product face into vertex coordinates of four vertices of each product face, and the category information and vertex coordinates of each product face are output to obtain a multi-faceted product detection result.

[0023] In step S12, according to the vertex coordinates of each of the product faces, the endpoint coordinates of each edge of each of the product faces are determined; according to the category information of each of the product faces, the product faces whose category information indicates that the product faces are the front side or the back side are divided into the front and back face group, and the product faces whose category information indicates that the product faces are the end face are divided into the end face group; according to the endpoint coordinates, the product faces in the front and back face group are matched with the product faces in the end face group, and the product faces in the front and back face group with overlapping edges are associated with the product faces in the end face group to obtain a product multi-face association result; Among them, the endpoint coordinates of a product surface can be expressed as: , Overlapping edges refer to the Euclidean distance between the two endpoint coordinates of one of the four edges of the product surface in the matched front and back surface groups and the two endpoint coordinates of one of the four edges of the product surface in the end surface group, which is within a preset distance threshold range.

[0024] In the disclosed embodiment, the endpoint coordinates of each edge of each product face are determined according to the vertex coordinates of each product face, and according to the category information of each product face, the product face whose category information represents that the product face is the front side or the back side can be divided into the front and back face group, and the product face whose category information represents that the product face is the end face can be divided into the end face group through algorithms such as convolutional neural network (CNN). According to the endpoint coordinates, the product faces in the front and back face group and the product faces in the end face group can be matched through methods such as matching methods based on endpoint direction and distance constraints, and the product faces in the front and back face group and the product faces in the end face group with overlapping edges can be associated to obtain a product multi-face association result.

[0025] In step S13, according to the product multi-faceted detection result, specification recognition is performed on each detected product face, and the specification categories and score values ​​of all product faces are recorded to obtain the product multi-faceted recognition result; Among them, the specification category refers to the brand and specification of the product. For example, the specification category of a certain product is Brand A (fine product). The score value can represent the confidence level in the specification category. The specification category and the score value are one-to-one corresponding.

[0026] In the embodiments of the present disclosure, based on the multi-faceted detection results of the commodity, machine vision technology can be used to extract feature data of the detected commodity faces, such as edge contours, color features, area, etc., and match them with features in a preset template or database. Specifications of each detected commodity face are identified, and the specification categories and score values ​​of all the commodity faces are recorded to obtain multi-faceted commodity recognition results.

[0027] In step S14, based on the multi-faceted association result of the product and the multi-faceted identification result of the product, the score values ​​of the product faces in each group of associated product faces are compared, and the specification category corresponding to the higher score value is taken as the final identification result of the associated product face; for the unassociated product face, the specification category obtained by the specification identification is directly taken as the final identification result.

[0028] Among them, the related product faces refer to the product faces of the same product with overlapping edges.

[0029] For example, based on the multi-faceted association results of the product and the multi-faceted identification results of the product, the score values ​​of the product faces in each group of associated product faces are compared, where the specification category of one product face in a group of associated product faces is brand A (boutique) and the score value is 0.93, and the specification category of another product face in the associated product faces is brand B (hard) and the score value is 0.49. Then, the specification category corresponding to the higher score value, i.e., 0.93, i.e., brand A (boutique), is taken as the final identification result of the associated product face. For the unassociated product faces, the specification category obtained by the specification identification is directly taken as the final identification result.

[0030] The method in the above technical solution obtains a product image for product identification in response to opening a product identification function, detects the category information of all product faces in the product image and the position information in the product image, converts the position information of each of the detected product faces into vertex coordinates of four vertices of each of the product faces according to the detected position information, outputs the category information and vertex coordinates of each of the product faces, and obtains a product multi-face detection result, wherein the category information refers to the front side, the back side, and the end side; determines the endpoint coordinates of each edge of each of the product faces according to the vertex coordinates of each of the product faces, and divides the product faces whose category information indicates whether they are the front side or the back side into the front and back side groups according to the category information of each of the product faces, and divides the product faces whose category information indicates whether they are the front side or the back side into the front and back side groups, and divides the product faces whose category information indicates whether they are the front side or the back side into the front and back side groups. The product faces that are end faces are divided into end face groups, and according to the endpoint coordinates, the product faces in the front and back face groups are matched with the product faces in the end face groups, and the product faces in the front and back face groups with overlapping edges are associated with the product faces in the end face groups to obtain the product multi-face association results; according to the product multi-face detection results, the specifications of each detected product face are identified, and the specification categories and score values ​​of all the product faces are recorded to obtain the product multi-face recognition results; according to the product multi-face association results and the product multi-face recognition results, the score values ​​of the product faces in each group of associated product faces are compared, and the specification category corresponding to the higher score value is taken as the final recognition result of the associated product face, and the specification category obtained by the specification recognition is directly taken as the final recognition result for the unassociated product face. By identifying the product through multi-face association, more and more comprehensive product features can be identified, which can improve the accuracy of the product recognition results; and multi-face association can identify both the end face of the product and the front or back side, which can improve the robustness of product recognition.

[0031] In a possible implementation, in step S12, matching the product faces in the front and back face groups with the product faces in the end face groups according to the endpoint coordinates, associating the product faces in the front and back face groups with the product faces in the end face groups having overlapping edges, and obtaining a product multi-face association result includes: According to the endpoint coordinates, calculating the Euclidean distances between the two endpoints of each side of the product surface in the front and back surface group and the two endpoints of each side of the product surface in the end surface group; The Euclidean distance includes a forward distance and a reverse distance. When calculating the Euclidean distance, the distance in at least one direction is calculated.

[0032] In the embodiment of the present disclosure, according to the endpoint coordinates, the Euclidean distance between the two endpoints of each side of the commodity surface in the front and back surface group and the two endpoints of each side of the commodity surface in the end surface group is calculated, and the calculation formula of the forward distance can be: ; The calculation formula for the reverse distance can be: ; Among them, mathematical symbols It means to calculate the norm of the difference between two vectors, that is, the Euclidean distance between the two points.

[0033] According to the Euclidean distance, the product faces in the front and back face group are matched with the product faces in the end face group. If there is a product face in the front and back face group, and the Euclidean distances from the two end points of an edge of the product face to the two end points of an edge of any product face in the end face group are within a preset distance threshold range, then the edge of the product face in the front and back face group and the edge of the product face in the end face group are determined to be overlapping edges, and all the overlapping edges in the product faces in the front and back face group and the product faces in the end face group are matched, and the endpoint coordinates of all the product faces in the front and back face group and the product faces in the end face group with the overlapping edges and the Euclidean distances of the overlapping edges are recorded to obtain a matching result; In the disclosed embodiment, the product faces in the front and back face groups are matched with the product faces in the end face groups according to the Euclidean distance. During the matching, it is determined whether the Euclidean distances in the same direction are both within a preset distance threshold range. If there is a product face in the front and back face groups, and the Euclidean distances in the same direction from the two endpoints of an edge of the product face to the two endpoints of an edge of any product face in the end face group are both within the preset distance threshold range, then it is determined that the edge of the product face in the front and back face groups and the edge of the product face in the end face group are overlapping edges, and all the overlapping edges of the product faces in the front and back face groups and the product faces in the end face groups are matched, and the endpoint coordinates of all the product faces in the front and back face groups and the product faces in the end face groups with overlapping edges and the Euclidean distances of the overlapping edges are recorded. For example, according to the Euclidean distance, the product face in the front and back face group is matched with the product face in the end face group. When matching, the two positive distances of the four endpoints of the two edges should be determined. and Are they all within the preset distance threshold?

[0034] According to the matching result, the product face of the front and back face groups having the overlapping edges and the product face of the end face are associated to obtain a product multi-face association result.

[0035] In the embodiment of the present disclosure, based on the matching result, the product faces of the front and back face groups with the overlapping edges and the product faces of the end face are associated, and the associated product faces of the front and back face groups with the overlapping edges and the product faces of the end face are divided into a group of associated product faces, to obtain a multi-faceted product association result.

[0036] In the above technical solution, the multi-faceted association of goods is realized by judging the overlapping edges, and the product association result is obtained, which can provide multi-faceted product feature information for product identification and can identify the products more accurately.

[0037] In a possible implementation, in step S12, according to the matching result, the product face of the front and back face groups having the overlapping edges and the product face of the end face are associated to obtain a product multi-face association result, including: According to the Euclidean distance of the overlapping edges, if multiple edges in the matching result form a group of the overlapping edges, two edges with the smallest Euclidean distance of the overlapping edges are taken as the overlapping edges, and two product faces corresponding to the two edges with the smallest Euclidean distance of the overlapping edges are associated to obtain a product association result; Here, multiple edges refer to more than two edges.

[0038] In the disclosed embodiment, according to the Euclidean distance of the overlapping edges, multiple edges in the matching result are formed into a group of the overlapping edges. For example, if there are three edges in the matching result that form a group of the overlapping edges, namely edge 1, edge 2 and edge 3, the Euclidean distance between edge 1 and edge 2 is 1.001, the Euclidean distance between edge 1 and edge 3 is 1.005, and the Euclidean distance between edge 2 and edge 3 is 1.0055, only the two edges with the smallest Euclidean distance of the overlapping edges, namely edge 1 and edge 2, are taken as the overlapping edges, and the two product faces corresponding to edge 1 and edge 2 are associated to obtain a product association result; If the matching result includes two edges forming a group of overlapping edges, two commodity faces corresponding to the two edges forming a group of overlapping edges in the matching result are directly associated to obtain a commodity association result.

[0039] The above technical solution can eliminate the possibility that multiple product faces become associated product faces, eliminate the possibility of erroneous association caused by multiple product faces becoming associated product faces, and make the result of multi-face association more accurate.

[0040] In a possible implementation, in step S12, according to the matching result, the product face of the front and back face groups having the overlapping edges and the product face of the end face are associated to obtain a product multi-face association result, including: According to the matching result, if the product surface of any of the products is partially blocked by the product placed in front of the product to form an overlapping edge causing a false association, then it is detected whether the blocked product surface of the product and the end surface of the product placed in front of the product have a partial overlap; If the obstructed product surface of the product partially overlaps with the end surface of the product placed in front of the product, calculate the ratio of the overlapping area of ​​the obstructed product surface and the end surface to the area of ​​the smaller surface between the obstructed product surface and the end surface; If the ratio is greater than a preset area ratio threshold, the blocked product surface and the end surface are determined as product surfaces that belong to two products but are incorrectly associated due to overlapping edges and are filtered out; If the ratio is less than or equal to the preset area ratio threshold, the blocked product surface and the end surface are determined to belong to the same product and are associated to obtain a product multi-surface association result.

[0041] In the embodiment of the present disclosure, according to the matching result, it can be determined by an algorithm such as a convolutional neural network (CNN) that a product face of any of the products is partially blocked by a product placed in front of the product to form an overlapping edge that causes an erroneous association, and it can be detected by an algorithm model whether the blocked product face of the product and the end face of the product placed in front of the product have a partial overlap. If the blocked product face of the product and the end face of the product placed in front of the product have a partial overlap, the ratio of the overlapping area of ​​the blocked product face and the end face to the area of ​​the smaller face of the blocked product face and the end face is calculated; if the ratio is greater than a preset area ratio threshold, the blocked product face and the end face are determined as belonging to two products but with an overlapping edge that causes an erroneous association and are filtered; if the ratio is less than or equal to the preset area ratio threshold, the blocked product face and the end face are determined as belonging to the same product and are associated to obtain a product multi-face association result.

[0042] The above technical solution eliminates the erroneous association caused by the special placement of goods by calculating the ratio of the overlapping area of ​​the blocked product surface and the end surface to the area of ​​the smaller surface of the two surfaces, thereby improving the accuracy of the multi-surface association results of the goods.

[0043] In a possible implementation, in step S12, calculating the ratio of the overlapping area of ​​the obstructed product surface and the end surface to the area of ​​the smaller surface between the obstructed product surface and the end surface includes: If the areas of the obstructed product surface and the end surface are not 0, then the ratio of the overlapping area of ​​the obstructed product surface and the end surface to the area of ​​the smaller surface of the two surfaces is calculated; If the area of ​​at least one of the obscured product surface and the end surface is 0, the ratio is directly set to 0.

[0044] In the embodiment of the present disclosure, the calculation formula for the ratio of the overlapping area of ​​the blocked commodity surface and the end surface to the area of ​​the smaller surface of the two surfaces may be: .

[0045] The above technical solution determines whether it is necessary to calculate the area ratio by judging whether there is a product surface with an area of ​​0 among the blocked product surface and the end surface, thereby improving the calculation efficiency and facilitating the subsequent multi-faceted product recognition.

[0046] In a possible implementation, in step S12, according to the matching result, associating the product face of the front and back face groups having the overlapping edges with the product face of the end face to obtain a product multi-face association result, further comprising: According to the matching result, if the overlapping edges formed by the stacking of two products cause a false association, then according to the two adjacent edges of the overlapping edge in the first product surface and the two adjacent edges of the overlapping edge in the second product surface, the center point distance between each adjacent edge of the first product surface and each adjacent edge of the second product surface is calculated, and the two adjacent edges with the shorter center point distance are taken as a group of corresponding adjacent edges, and the remaining two adjacent edges are taken as another group of corresponding adjacent edges, to obtain the adjacent edge grouping result; Among them, the adjacent edge refers to the edge of the product surface that is adjacent to the overlapping edge, and the calculation formula of the center point distance can be: , Among them, C1 and C2 are the center points of edge 1 and edge 2.

[0047] In the disclosed embodiment, the false association caused by the overlapping edges formed by stacking two commodities can be determined by an algorithm such as a convolutional neural network (CNN), and two adjacent edges of the overlapping edge in the first commodity surface and two adjacent edges of the overlapping edge in the second commodity surface are found according to the overlapping edges of the two commodity surfaces, and the center point distance between each adjacent edge of the first commodity surface and each adjacent edge of the second commodity surface is calculated, and the two adjacent edges with shorter center point distance are taken as a group of corresponding adjacent edges, and the remaining two adjacent edges are taken as another group of corresponding adjacent edges, to obtain the adjacent edge grouping result; For example, side 4 of product face A and side 2 of product face B are a group of overlapping edges. Based on the overlapping edges, two edges adjacent to the overlapping edges are found in the two faces. For product face A, the adjacent edges are side 1 of product face A and side 3 of product face A; for product face B, the adjacent edges are side 1 of product face B and side 3 of product face B. The center distances of side 1 of product face A and side 1 of product face B and side 3 of product face B are calculated, and the center distances of side 3 of product face A and side 1 of product face B and side 3 of product face B are calculated. Assuming that the center distances of side 3 of product face A and side 3 of product face B are shorter, side 3 of product face A and side 3 of product face B are taken as a group of corresponding adjacent edges, and the remaining side 1 of product face A and side 1 of product face B are taken as another group of corresponding adjacent edges, and the adjacent edge grouping result is obtained.

[0048] According to the adjacent edge grouping result, calculating the angle between the two groups of corresponding adjacent edges; If the included angles of the two groups of corresponding adjacent edges are both within the preset angle threshold range, it is determined that the two groups of corresponding adjacent edges of the first product face and the second product face are on the same straight line, and the first product face and the second product face are determined as product faces that belong to two products but are mistakenly associated due to overlapping edges, and are filtered out; If at least one of the angles of the two groups of corresponding adjacent edges is outside the preset angle threshold range, it is determined that the corresponding adjacent edges of the first product surface and the second product surface are not on the same straight line, and the first product surface and the second product surface are determined to belong to the same product and associated to obtain a product multi-surface association result.

[0049] In an embodiment of the present disclosure, based on the adjacent edge grouping result, the included angle of the two groups of corresponding adjacent edges is calculated. If the included angle of the two groups of corresponding adjacent edges is within a preset angle threshold range, for example, the preset angle threshold range may be 0° to 1° and 179° to 180°. As long as the included angle satisfies any one of the two ranges, it is within the preset angle range. For example, if the included angles of the two groups of corresponding adjacent edges are 0.5° and 0.6° respectively, they are within the preset angle threshold range; if the included angles of the two groups of corresponding adjacent edges are 179.5° and 179.6° respectively, they are within the preset angle threshold range; if the included angles of the two groups of corresponding adjacent edges are 0.5° and 179.6° respectively, they are also within the preset angle threshold range.

[0050] The above technical solution ensures the correct division of adjacent edges by using the center distance to divide the corresponding adjacent edges, providing correct data for calculating the angle; by calculating the angle between the two groups of corresponding adjacent edges to determine whether it is a misassociation caused by special placement, some misassociations can be filtered out to provide correct association results for product identification.

[0051] In a possible implementation, in step S12, calculating the angle between the two groups of corresponding adjacent edges includes: According to the two groups of endpoint coordinates of the corresponding adjacent edges, grouping and calculating the coordinate difference of two adjacent edges in the corresponding adjacent edges; According to the coordinate difference, group and calculate the dot product and length of two adjacent edges of the corresponding adjacent edges; According to the dot product and the length, the cosine values ​​of two adjacent sides of the corresponding adjacent sides are calculated in groups, and the angle between the two groups of corresponding adjacent sides is obtained by calculating the arccosine function of the cosine value.

[0052] In the embodiment of the present disclosure, according to the endpoint coordinates of the two groups of corresponding adjacent edges, for example, one group of corresponding adjacent edges is edge 1 of commodity surface A and edge 1 of commodity surface B, the endpoint coordinates of edge 1 of commodity surface A are (x1, y1) and (x2, y2), and the endpoint coordinates of edge 1 of commodity surface B are (x3, y3) and (x4, y4). The coordinate differences of the corresponding adjacent edges edge 1 of commodity surface A and edge 1 of commodity surface B are calculated as (x2-x1), (x4-x3), (y2-y1), and (y4-y3). The dot product of the corresponding adjacent edges edge 1 of commodity surface A and edge 1 of commodity surface B is calculated as (x2-x1)(x4-x3)+(y2-y1)(y4-y3) and the length , , according to the dot product and the length, calculate the cosine values ​​of two adjacent edges of the corresponding adjacent edges , by calculating the arccosine function of the cosine value, that is Get the angle between the corresponding adjacent sides .

[0053] The above technical solution calculates the angle between the two groups of corresponding adjacent edges as the basis for misassociation, which can reasonably and effectively filter out misassociations caused by special placement of goods.

[0054] In a possible implementation, in step S13, according to the multi-faceted detection result of the commodity, specification recognition is performed on each detected commodity face, and the specification categories and score values ​​of all commodity faces are recorded to obtain the multi-faceted recognition result of the commodity, including: According to the detection results of the multiple sides of the commodity, input the surface area image of each detected surface of the commodity, and perform specification recognition on each surface area image; Output the specification category and score value of each product surface identified by the specification; The specification categories and score values ​​of all the product faces are recorded to obtain the product multi-faceted recognition result.

[0055] The surface area image refers to the image of the area covered by each product surface in the product image, including the surface area images of the front side, the back side and the end side.

[0056] In an embodiment of the present disclosure, based on the multi-faceted detection result of the commodity, a face area image of each detected commodity face is input, and the input commodity face area image is first preprocessed, including grayscale, binarization, noise removal, etc., and features are extracted from the preprocessed image using image processing technology or a deep learning model. These features may include color, texture, shape, etc. Based on the extracted features, a trained classification model (such as a convolutional neural network (CNN)) is used to perform specification recognition on each commodity face, and the specification category and score value of each commodity face identified by specification are output, and the specification categories and score values ​​of all the commodity faces are recorded to obtain the multi-faceted recognition result of the commodity.

[0057] The method of the above technical solution performs specification recognition on the detected surface area image of the product, and the surface area image includes the surface area images of the front side, the back side and the end side. Multi-faceted specification recognition can ensure the diversity and comprehensiveness of the product identification information.

[0058] It should be understood that although Figure 1 The steps in the flowchart are shown in sequence as indicated by the arrows, but these steps are not necessarily executed in the order indicated by the arrows. Unless otherwise specified in this document, there is no strict order restriction for the execution of these steps, and these steps can be executed in other orders. Moreover, Figure 1 At least part of the steps may include multiple sub-steps or multiple stages. These sub-steps or stages are not necessarily executed at the same time, but can be executed at different times. The execution order of these sub-steps or stages is not necessarily sequential, but can be executed in turn or alternately with other steps or at least part of the sub-steps or stages of other steps.

[0059] In one embodiment, Figure 2 As shown, a commodity identification device in a cluttered environment is provided, comprising: The commodity multi-face detection module 210 is configured to, in response to turning on the commodity recognition function, obtain a commodity image for commodity recognition, detect category information of all commodity faces in the commodity image and position information in the commodity image, convert the position information of each detected commodity face into vertex coordinates of four vertices of each commodity face according to the detected position information, output the category information and vertex coordinates of each commodity face, and obtain a commodity multi-face detection result, wherein the category information refers to the front face, the back face, and the end face; The commodity multi-faceted association module 220 is configured to determine the endpoint coordinates of each edge of each commodity face according to the vertex coordinates of each commodity face, and to classify the commodity face whose category information indicates that the commodity face is the front face or the back face into a front-back face group and to classify the commodity face whose category information indicates that the commodity face is the end face into an end face group according to the category information of each commodity face, and to match the commodity faces in the front-back face group with the commodity faces in the end face group according to the endpoint coordinates, and to associate the commodity faces in the front-back face group with the commodity faces in the end face group that have overlapping edges, so as to obtain a commodity multi-faceted association result; The commodity multi-faceted recognition module 230 is configured to perform specification recognition on each detected commodity face according to the commodity multi-faceted detection result, record the specification categories and score values ​​of all commodity faces, and obtain the commodity multi-faceted recognition result; The comprehensive scoring module 240 is configured to compare the score values ​​of the product faces in each group of associated product faces based on the multi-faceted association results of the product and the multi-faceted identification results of the product, and take the specification category corresponding to the higher score value as the final identification result of the associated product face; for unassociated product faces, directly take the specification category obtained by the specification identification as the final identification result.

[0060] In a possible implementation, the commodity multi-faceted association module 220 is configured to: Calculating the Euclidean distances between the endpoints of all edges of the product surface in the front and back surface group and the endpoints of all edges of the product surface in the end surface group according to the endpoint coordinates; According to the Euclidean distance, the product faces in the front and back face group are matched with the product faces in the end face group. If there is a product face in the front and back face group, and the Euclidean distances of the two end points of an edge of the product face to the two end points of an edge of a product face in the end face group are both within the preset distance threshold range, then the edge of the product face in the front and back face group and the edge of the product face in the end face group are determined to be overlapping edges, and all the overlapping edges in the product faces in the front and back face group and the product faces in the end face group are matched, and the endpoint coordinates of all the product faces in the front and back face group and the product faces in the end face group with overlapping edges and the Euclidean distances of the overlapping edges are recorded; According to the matching result, the product face of the front and back face groups having the overlapping edges and the product face of the end face are associated to obtain a product multi-face association result.

[0061] In a possible implementation, the commodity multi-faceted association module 220 is configured to: According to the Euclidean distance of the overlapping edges, for a plurality of edges in the matching result to be a group of the overlapping edges, only two edges with the smallest Euclidean distance of the overlapping edges are taken as the overlapping edges, and two product faces corresponding to the two edges with the smallest Euclidean distance of the overlapping edges are associated to obtain a product association result; If only two edges in the matching result form a group of overlapping edges, two product faces corresponding to the two edges in the matching result that form a group of overlapping edges are directly associated to obtain a product association result.

[0062] In a possible implementation, the commodity multi-faceted association module 220 is configured to: According to the matching result, the product face of the front and back face groups with the overlapping edges and the product face of the end face are associated, and the erroneous association caused by multiple product placements but with overlapping edges due to special placement is filtered out to obtain a multi-faceted product association result.

[0063] In a possible implementation, the commodity multi-faceted association module 220 is configured to: For the misassociation caused by the overlapping edge formed by the partial obstruction of the product surface of a certain product by another product placed in front of the certain product, the predicted position area of ​​the obstructed product surface of the certain product detected partially overlaps with the area of ​​the end surface of the other product, and the ratio of the overlapping area of ​​the obstructed product surface and the end surface to the area of ​​the smaller surface of the two surfaces is calculated; If the ratio is higher than the preset area ratio threshold, it is determined that the two faces belong to two commodities and are incorrectly associated, and are filtered out; If the ratio is lower than the preset area ratio threshold, it is determined that the two faces are from the same product and are correctly associated, so they are retained.

[0064] In a possible implementation, the commodity multi-faceted association module 220 is configured to: If the areas of the obstructed product surface and the end surface are not 0, then the ratio of the overlapping area of ​​the obstructed product surface and the end surface to the area of ​​the smaller surface of the two surfaces is calculated; If the area of ​​at least one of the obscured product surface and the end surface is 0, the ratio is directly set to 0.

[0065] In a possible implementation, the commodity multi-faceted association module 220 is configured to: In order to solve the misassociation caused by the overlapping edges formed by stacking two products, two adjacent edges of the overlapping edge in the first product surface and two adjacent edges of the overlapping edge in the second product surface are found, and the center point distance between each adjacent edge of the first product surface and each adjacent edge of the second product surface is calculated, and the two adjacent edges with shorter center point distance are taken as a group of corresponding adjacent edges, and the remaining two adjacent edges are taken as another group of corresponding adjacent edges to obtain the adjacent edge grouping result; According to the adjacent edge grouping result, calculating the angle between the two groups of corresponding adjacent edges; If the included angles of the two groups of corresponding adjacent edges are both within the preset angle threshold range, it is determined that the two groups of corresponding adjacent edges of the first product face and the second product face are on the same straight line, and the first product face and the second product face are determined as product faces that belong to two products but are mistakenly associated due to overlapping edges, and are filtered out; If at least one of the angles of the two groups of corresponding adjacent edges is outside the preset angle threshold range, it is determined that the corresponding adjacent edges of the first product surface and the second product surface are not on the same straight line, and the first product surface and the second product surface are determined to belong to the same product and associated to obtain a product multi-surface association result.

[0066] In a possible implementation, the commodity multi-faceted association module 220 is configured to: According to the two groups of endpoint coordinates of the corresponding adjacent edges, grouping and calculating the coordinate difference of two adjacent edges in the corresponding adjacent edges; According to the coordinate difference, group and calculate the dot product and length of two adjacent edges of the corresponding adjacent edges; According to the dot product and the length, the cosine values ​​of two adjacent sides of the corresponding adjacent sides are calculated in groups, and the angle between the two groups of corresponding adjacent sides is obtained by calculating the arccosine function of the cosine value.

[0067] In a possible implementation, the commodity multi-faceted identification module 230 is configured as follows: According to the detection results of the multiple sides of the commodity, input the surface area image of each detected surface of the commodity, and perform specification recognition on each surface area image; Output the specification category and score value of each product surface identified by the specification; The specification categories and score values ​​of all the product faces are recorded to obtain the product multi-faceted recognition result.

[0068] For the specific definition of a commodity identification device in a cluttered environment, please refer to the definition of a commodity identification method in a cluttered environment mentioned above, which will not be repeated here. Each module in the above-mentioned commodity identification device in a cluttered environment can be implemented in whole or in part by software, hardware and a combination thereof. The above-mentioned modules can be embedded in or independent of the processor in the computer device in the form of hardware, or can be stored in the memory of the computer device in the form of software, so that the processor can call and execute the operations corresponding to the above modules.

[0069] The present disclosure also provides a computer-readable storage medium on which a computer program is stored. When the program is executed by a processor, the steps of the method described in any one of the aforementioned embodiments are implemented.

[0070] The present disclosure also provides an electronic device, including: a memory having a computer program stored thereon; A processor is used to execute the computer program in the memory to implement the steps of the method in any one of the aforementioned embodiments.

[0071] Figure 3 The commodity identification device 100 shown in a cluttered environment includes: a processor 1001 and a memory 1003. The processor 1001 and the memory 1003 are connected, such as through a bus 1002. Optionally, the commodity identification device 100 in a cluttered environment may also include a communication component, which can be used for data interaction between the device 100 and other devices, such as data transmission or data reception. It should be noted that in actual scheduling, the communication component is not limited to one, and the structure of the commodity identification device 100 in a cluttered environment does not constitute a limitation on the embodiments of the present application.

[0072] Processor 1001 may be a CPU (Central Processing Unit), a general-purpose processor, a DSP (Digital Signal Processor), an ASIC (Application Specific Integrated Circuit), an FPGA (Field Programmable Gate Array) or other programmable logic devices, transistor logic devices, hardware components or any combination thereof. It may implement or execute various exemplary logic blocks, modules and circuits described in conjunction with the disclosure of this application. Processor 1001 may also be a combination that implements computing functions, such as a combination of one or more microprocessors, a combination of a DSP and a microprocessor, etc.

[0073] The bus 1002 may include a path to transmit information between the above components. The bus 1002 may be a PCI (Peripheral Component Interconnect) bus or an EISA (Extended Industry Standard Architecture) bus. The bus 1002 may be divided into an address bus, a data bus, a control bus, etc. For ease of representation, Figure 3 Only one thick line is used in the diagram, but this does not mean that there is only one bus or only one type of bus.

[0074] The memory 1003 may be a ROM (Read Only Memory) or other types of static storage devices that can store static information and instructions, a RAM (Random Access Memory) or other types of dynamic storage devices that can store information and instructions, or an EEPROM (Electrically Erasable Programmable Read Only Memory), a CD-ROM (Compact Disc Read Only Memory) or other optical disk storage, optical disk storage (including compressed optical disk, laser disk, optical disk, digital versatile disk, Blu-ray disk, etc.), magnetic disk storage medium, other magnetic storage devices, or any other medium that can be used to carry or store program code and can be read by a computer, without limitation herein.

[0075] The memory 1003 is used to store program codes for executing the embodiments of the present disclosure, and the execution is controlled by the processor 1001. The processor 1001 is used to execute the program codes stored in the memory 1003 to implement the steps shown in the embodiment of the commodity identification method in a cluttered environment.

[0076] The embodiment of the present disclosure also provides a computer-readable storage medium having program code stored thereon. When the program code is executed by a processor, the steps and corresponding contents of the aforementioned embodiment of a method for identifying goods in a cluttered environment can be implemented.

[0077] The preferred embodiments of the present disclosure are described in detail above in conjunction with the accompanying drawings; however, the present disclosure is not limited to the specific details in the above embodiments; within the technical concept of the present disclosure, various changes, modifications, substitutions and variations may be made to these embodiments, and these changes, modifications, substitutions and variations all fall within the protection scope of the present disclosure.

[0078] It should also be noted that the various specific technical features described in the above specific embodiments can be combined in any appropriate manner without contradiction, and they should also be regarded as the contents disclosed in this disclosure. In order to avoid unnecessary repetition, this disclosure will not further describe various possible combinations. The technical scope of this application is not limited to the contents in the specification, and its technical scope must be determined according to the scope of the claims.

Claims

1. A method for identifying commodities in a cluttered environment, characterized in that: The method comprises: In response to turning on the commodity recognition function, a commodity image for commodity recognition is obtained, category information of all commodity faces in the commodity image and position information in the commodity image are detected, and according to the detected position information, the position information of each of the detected commodity faces is converted into vertex coordinates of four vertices of each of the commodity faces, and the category information and vertex coordinates of each of the commodity faces are output to obtain a commodity multi-face detection result, wherein the category information refers to the front side, the back side and the end side; According to the vertex coordinates of each of the product faces, the endpoint coordinates of each edge of each of the product faces are determined; according to the category information of each of the product faces, the product faces whose category information indicates that the product faces are the front side or the back side are divided into the front and back face group, and the product faces whose category information indicates that the product faces are the end face are divided into the end face group; according to the endpoint coordinates, the product faces in the front and back face group are matched with the product faces in the end face group, and the product faces in the front and back face group with overlapping edges are associated with the product faces in the end face group to obtain a product multi-face association result; According to the multi-faceted detection result of the commodity, specification recognition is performed on each detected commodity face, and the specification categories and score values ​​of all commodity faces are recorded to obtain the multi-faceted recognition result of the commodity; According to the multi-faceted association result of the product and the multi-faceted identification result of the product, the score values ​​of the product faces in each group of associated product faces are compared, and the specification category corresponding to the higher score value is taken as the final identification result of the associated product face; for the unassociated product face, the specification category obtained by the specification identification is directly taken as the final identification result.

2. The method according to claim 1, characterized in that: The method of matching the product faces in the front and back face groups with the product faces in the end face groups according to the endpoint coordinates, and associating the product faces in the front and back face groups with the product faces in the end face groups having overlapping edges to obtain a product multi-face association result includes: According to the endpoint coordinates, calculating the Euclidean distances between the two endpoints of each side of the product surface in the front and back surface group and the two endpoints of each side of the product surface in the end surface group; According to the Euclidean distance, the product faces in the front and back face group are matched with the product faces in the end face group. If there is a product face in the front and back face group, and the Euclidean distances from the two end points of an edge of the product face to the two end points of an edge of any product face in the end face group are within a preset distance threshold range, then the edge of the product face in the front and back face group and the edge of the product face in the end face group are determined to be overlapping edges, and all the overlapping edges in the product faces in the front and back face group and the product faces in the end face group are matched, and the endpoint coordinates of all the product faces in the front and back face group and the product faces in the end face group with the overlapping edges and the Euclidean distances of the overlapping edges are recorded to obtain a matching result; According to the matching result, the product face of the front and back face groups having the overlapping edges and the product face of the end face are associated to obtain a product multi-face association result.

3. The method according to claim 2, characterized in that: According to the matching result, associating the product face of the front and back face groups having the overlapping edges with the product face of the end face to obtain a product multi-face association result, including: According to the Euclidean distance of the overlapping edges, if multiple edges in the matching result form a group of the overlapping edges, two edges with the smallest Euclidean distance of the overlapping edges are taken as the overlapping edges, and two product faces corresponding to the two edges with the smallest Euclidean distance of the overlapping edges are associated to obtain a product association result; If the matching result includes two edges forming a group of overlapping edges, two commodity faces corresponding to the two edges forming a group of overlapping edges in the matching result are directly associated to obtain a commodity association result.

4. The method according to claim 2, characterized in that: According to the matching result, associating the product face of the front and back face groups having the overlapping edges with the product face of the end face to obtain a product multi-face association result, including: According to the matching result, if the product surface of any of the products is partially blocked by the product placed in front of the product to form an overlapping edge causing a false association, then it is detected whether the blocked product surface of the product and the end surface of the product placed in front of the product have a partial overlap; If the obstructed product surface of the product partially overlaps with the end surface of the product placed in front of the product, calculate the ratio of the overlapping area of ​​the obstructed product surface and the end surface to the area of ​​the smaller surface between the obstructed product surface and the end surface; If the ratio is greater than a preset area ratio threshold, the blocked product surface and the end surface are determined as product surfaces that belong to two products but are incorrectly associated due to overlapping edges and are filtered out; If the ratio is less than or equal to the preset area ratio threshold, the blocked product surface and the end surface are determined to belong to the same product and are associated to obtain a product multi-surface association result.

5. The method according to claim 4, characterized in that: The calculating of the ratio of the overlapping area of ​​the obstructed product surface and the end surface to the area of ​​the smaller surface between the obstructed product surface and the end surface comprises: If the areas of the obstructed product surface and the end surface are not 0, then the ratio of the overlapping area of ​​the obstructed product surface and the end surface to the area of ​​the smaller surface of the two surfaces is calculated; If the area of ​​at least one of the obscured product surface and the end surface is 0, the ratio is directly set to 0.

6. The method according to claim 2, characterized in that: According to the matching result, associating the product face of the front and back face groups having the overlapping edges with the product face of the end face to obtain a product multi-face association result, further comprising: According to the matching result, if the overlapping edges formed by the stacking of two products cause a false association, then according to the two adjacent edges of the overlapping edge in the first product surface and the two adjacent edges of the overlapping edge in the second product surface, the center point distance between each adjacent edge of the first product surface and each adjacent edge of the second product surface is calculated, and the two adjacent edges with the shorter center point distance are taken as a group of corresponding adjacent edges, and the remaining two adjacent edges are taken as another group of corresponding adjacent edges, to obtain the adjacent edge grouping result; According to the adjacent edge grouping result, calculating the angle between the two groups of corresponding adjacent edges; According to the adjacent edge grouping result, calculating the angle between the two groups of corresponding adjacent edges; If the included angles of the two groups of corresponding adjacent edges are both within the preset angle threshold range, it is determined that the two groups of corresponding adjacent edges of the first product face and the second product face are on the same straight line, and the first product face and the second product face are determined as product faces that belong to two products but are mistakenly associated due to overlapping edges, and are filtered out; If at least one of the angles of the two groups of corresponding adjacent edges is outside the preset angle threshold range, it is determined that the corresponding adjacent edges of the first product surface and the second product surface are not on the same straight line, and the first product surface and the second product surface are determined to belong to the same product and associated to obtain a product multi-surface association result.

7. The method according to claim 6, characterized in that: The calculating the angle between the two groups of corresponding adjacent edges comprises: According to the two groups of endpoint coordinates of the corresponding adjacent edges, grouping and calculating the coordinate difference of two adjacent edges in the corresponding adjacent edges; According to the coordinate difference, group and calculate the dot product and length of two adjacent edges of the corresponding adjacent edges; According to the dot product and the length, the cosine values ​​of two adjacent sides of the corresponding adjacent sides are calculated in groups, and the angle between the two groups of corresponding adjacent sides is obtained by calculating the arccosine function of the cosine value.

8. The method according to claim 1, characterized in that: The step of performing specification recognition on each detected product surface according to the product multi-surface detection result, recording the specification categories and score values ​​of all product surfaces, and obtaining the product multi-surface recognition result includes: According to the detection results of the multiple sides of the commodity, input the surface area image of each detected surface of the commodity, and perform specification recognition on each surface area image; Output the specification category and score value of each product surface identified by the specification; The specification categories and score values ​​of all the product faces are recorded to obtain the product multi-faceted recognition result.

9. A commodity identification device in a cluttered environment, characterized in that: The device comprises: The commodity multi-face detection module is configured to, in response to turning on the commodity recognition function, obtain a commodity image for commodity recognition, detect category information of all commodity faces in the commodity image and position information in the commodity image, convert the position information of each of the detected commodity faces into vertex coordinates of four vertices of each of the commodity faces according to the detected position information, output the category information and vertex coordinates of each of the commodity faces, and obtain a commodity multi-face detection result, wherein the category information refers to the front side, the back side, and the end side; The commodity multi-faceted association module is configured to determine the endpoint coordinates of each edge of each commodity face according to the vertex coordinates of each commodity face, and to classify the commodity face whose category information indicates that the commodity face is the front side or the back side into a front-back face group and to classify the commodity face whose category information indicates that the commodity face is the end face into an end face group according to the category information of each commodity face, and to match the commodity faces in the front-back face group with the commodity faces in the end face group according to the endpoint coordinates, and to associate the commodity faces in the front-back face group with the commodity faces in the end face group that have overlapping edges, so as to obtain a commodity multi-faceted association result; The commodity multi-faceted recognition module is configured to perform specification recognition on each detected commodity face according to the commodity multi-faceted detection result, record the specification category and score value of all commodity faces, and obtain the commodity multi-faceted recognition result; The comprehensive scoring module is configured to compare the score values ​​of the product faces in each group of associated product faces based on the multi-faceted association results of the product and the multi-faceted identification results of the product, and take the specification category corresponding to the higher score value as the final identification result of the associated product face; for unassociated product faces, directly take the specification category obtained by the specification identification as the final identification result.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the program is executed by a processor, the steps of the method described in any one of claims 1 to 8 are implemented.

11. An electronic device, characterized in that: include: a memory having a computer program stored thereon; A processor, configured to execute the computer program in the memory to implement the steps of the method according to any one of claims 1 to 8.

Images