A method, apparatus, device, and medium for identifying commodities in a cluttered environment

Abstract

The present application relates to a method, apparatus, device and medium for identifying commodities in a cluttered environment, including detecting the category information and position information of commodity surfaces in a commodity image, converting the position information into vertex coordinates, and outputting the category information and vertex coordinates; determining the endpoint coordinates of the edges of the commodity surfaces according to the vertex coordinates, and according to the category information, dividing the commodity surfaces whose category information represents the front or back of the commodity surface into the front and back group and dividing the commodity surfaces whose category information represents the end surface of the commodity surface into the end surface group, matching the commodity surfaces in the front and back group and the commodity surfaces in the end surface group according to the endpoint coordinates, and associating the commodity surfaces with overlapping edges; performing specification identification on the commodity surfaces, recording the commodity surface specification category and the score value; comparing the score values of the commodity surfaces in the associated commodity surfaces, and taking the specification category corresponding to the higher score value as the final identification result of the associated commodity surfaces, and taking the specification category obtained by the specification identification of the unassociated commodity surfaces as the final identification result.

Description

Technical Field

[0001] This application relates to the field of image recognition technology, and in particular to a method, device, equipment and medium for identifying commodities in a cluttered environment. Background Art

[0002] Currently, in shops selling commodities, merchants usually regularly take inventory of the commodities in a cluttered environment. Although the manual inventory method can complete the task, it is time-consuming, laborious and error-prone. To improve the inventory efficiency, many shops have begun to introduce inventory methods based on artificial intelligence algorithms. The current artificial intelligence inventory methods mainly analyze the photographed images of commodities, identify their specifications from the end faces of the commodities and count the results. However, the method of only using the end faces for identification has defects such as unclear end faces and difficult-to-accurately identify end face features caused by different shooting methods and angles of merchants and loss of image information. These defects are likely to cause problems such as missed identification or misidentification in commodity identification. Summary of the Invention

[0003] The purpose of the present invention is to provide a method, device, equipment and medium for identifying commodities in a cluttered environment, aiming to improve the accuracy and robustness of commodity identification.

[0004] To achieve the above purpose, in the first aspect of the embodiments of the present disclosure, the method includes:

[0005] In response to opening the commodity identification function, obtain a commodity image for commodity identification, detect the category information of all commodity faces in the commodity image and the position information in the commodity image, and according to the detected position information, convert the position information of each detected commodity face into the vertex coordinates of the four vertices of each commodity face, and output the category information and the vertex coordinates of each commodity face to obtain a commodity multi-face detection result, where the category information refers to the front face, the back face and the end face;

[0006] According to the vertex coordinates of each commodity face, determine the endpoint coordinates of each edge of each commodity face, and according to the category information of each commodity face, divide the commodity faces whose category information represents the commodity face is the front face or the back face into the front and back face group, and divide the commodity faces whose category information represents the commodity face is the end face into the end face group. According to the endpoint coordinates, match the commodity faces in the front and back face group and the commodity faces in the end face group, and associate the commodity faces in the front and back face group and the commodity faces in the end face group that have overlapping edges to obtain a commodity multi-face association result;

[0007] Based on the multi - surface detection results of the commodity, identify the specifications for each detected commodity surface, record the specification categories and score values of all the commodity surfaces, and obtain the multi - surface identification results of the commodity;

[0008] Based on the multi - surface association results of the commodity and the multi - surface identification results of the commodity, compare the score values of the commodity surfaces in each group of associated commodity surfaces, and take the specification category corresponding to the higher score value as the final identification result of the associated commodity surfaces. For the unassociated commodity surfaces, directly take the specification category obtained from the specification identification as the final identification result.

[0009] In a possible implementation, the matching of the commodity surfaces in the front - and - back surface group and the commodity surfaces in the end - surface group according to the endpoint coordinates, and the association of the commodity surfaces in the front - and - back surface group and the commodity surfaces in the end - surface group with overlapping edges to obtain the multi - surface association results of the commodity, includes:

[0010] According to the endpoint coordinates, calculate the Euclidean distances between the two endpoints of each side of the commodity surfaces in the front - and - back surface group and the two endpoints of each side of the commodity surfaces in the end - surface group;

[0011] According to the Euclidean distances, match the commodity surfaces in the front - and - back surface group and the commodity surfaces in the end - surface group. If there is a commodity surface in the front - and - back surface group, and the Euclidean distances between the two endpoints of one side of this commodity surface and the two endpoints of one side of any commodity surface in the end - surface group are all within the preset distance threshold range, then determine that the two sides, namely the side of the commodity surface in the front - and - back surface group and the side of the commodity surface in the end - surface group, are overlapping edges. Match all the overlapping edges between the commodity surfaces in the front - and - back surface group and the commodity surfaces in the end - surface group, and record the endpoint coordinates of all the commodity surfaces in the front - and - back surface group and the commodity surfaces in the end - surface group with overlapping edges and the Euclidean distances of the overlapping edges to obtain the matching results;

[0012] According to the matching results, associate the commodity surfaces in the front - and - back surface group with overlapping edges and the commodity surfaces in the end - surface group to obtain the multi - surface association results of the commodity.

[0013] In a possible implementation, the association of the commodity surfaces in the front - and - back surface group with overlapping edges and the commodity surfaces in the end - surface group according to the matching results to obtain the multi - surface association results of the commodity, includes:

[0014] According to the Euclidean distances of the overlapping edges, if multiple edges in the matching results form a group of overlapping edges, then take the two edges with the smallest Euclidean distance of the overlapping edges as the overlapping edges, and associate the two commodity surfaces corresponding to the two edges with the smallest Euclidean distance of the overlapping edges to obtain the commodity association results;

[0015] If two edges in the matching result form a group of the overlapping edges, directly associate the two commodity faces corresponding to the two edges that form a group of the overlapping edges in the matching result to obtain a commodity association result.

[0016] In a possible implementation manner, the associating the commodity faces of the front and back face groups having the overlapping edges and the commodity faces of the end faces according to the matching result to obtain a multi-face commodity association result includes:

[0017] According to the matching result, if the overlapping edges formed by a commodity face of any commodity being blocked by a commodity part in front of the commodity cause mis-association, detect whether there is partial overlap between the blocked commodity face of the commodity and the end face of the commodity placed in front of the commodity;

[0018] If there is partial overlap between the blocked commodity face of the commodity and the end face of the commodity placed in front of the commodity, calculate the ratio of the overlapping area of the blocked commodity face and the end face to the area of the smaller face among the blocked commodity face and the end face;

[0019] If the ratio is greater than a preset area ratio threshold, determine the blocked commodity face and the end face as the commodity faces that belong to two commodities but are mis-associated due to overlapping edges for filtering;

[0020] If the ratio is less than or equal to the preset area ratio threshold, determine the blocked commodity face and the end face as belonging to the same commodity for association to obtain a multi-face commodity association result.

[0021] In a possible implementation manner, the calculating the ratio of the overlapping area of the blocked commodity face and the end face to the area of the smaller face among the blocked commodity face and the end face includes:

[0022] If the areas of both the blocked commodity face and the end face are not 0, calculate the ratio of the overlapping area of the blocked commodity face and the end face to the area of the smaller face of the two faces;

[0023] If the area of at least one commodity face among the blocked commodity face and the end face is 0, directly set the ratio to 0.

[0024] In a possible implementation manner, the associating the commodity faces of the front and back face groups having the overlapping edges and the commodity faces of the end faces according to the matching result to obtain a multi-face commodity association result further includes:

[0025] According to the matching result, if the overlapping edges formed by stacking two commodities cause mis-association, then based on the two adjacent edges of the overlapping edge in the first commodity surface and the two adjacent edges of the overlapping edge in the second commodity surface, calculate the center point distance between each of the adjacent edges of the first commodity surface and each of the adjacent edges of the second commodity surface, and take the two adjacent edges with the shorter center point distance as a group of corresponding adjacent edges, and the remaining two adjacent edges as another group of corresponding adjacent edges to obtain the adjacent edge grouping result;

[0026] According to the adjacent edge grouping result, calculate the included angle between the two groups of corresponding adjacent edges;

[0027] If both of the included angles between the two groups of corresponding adjacent edges are within the preset angle threshold range, then it is determined that the two groups of corresponding adjacent edges of the first commodity surface and the second commodity surface are on the same straight line, and then the first commodity surface and the second commodity surface are determined as commodity surfaces that belong to two commodities but have mis-association caused by overlapping edges for filtering;

[0028] If at least one of the included angles between the two groups of corresponding adjacent edges is not within the preset angle threshold range, then it is determined that the corresponding adjacent edges of the first commodity surface and the second commodity surface are not on the same straight line, and then the first commodity surface and the second commodity surface are determined to belong to the same commodity for association to obtain the commodity multi-surface association result.

[0029] In a possible implementation manner, the calculating the included angle between the two groups of corresponding adjacent edges includes:

[0030] According to the endpoint coordinates of the two groups of corresponding adjacent edges, group and calculate the coordinate differences of the two adjacent edges in the corresponding adjacent edges;

[0031] According to the coordinate differences, group and calculate the dot product and length of the two adjacent edges in the corresponding adjacent edges;

[0032] According to the dot product and the length, group and calculate the cosine values of the two adjacent edges in the corresponding adjacent edges, and obtain the included angle between the two groups of corresponding adjacent edges by calculating the inverse cosine function of the cosine values.

[0033] In a possible implementation manner, the according to the commodity multi-surface detection result, performing specification recognition on each detected commodity surface, and recording the specification categories and score values of all the commodity surfaces to obtain the commodity multi-surface recognition result includes:

[0034] According to the commodity multi-surface detection result, input the surface area image of each detected commodity surface, and perform specification recognition on each surface area image;

[0035] Output the specification category and score value of each commodity surface for which the specification is recognized;

[0036] Record the specification categories and score values of all the commodity faces to obtain the multi-face recognition result of the commodity.

[0037] In a second aspect of the embodiments of the present disclosure, there is provided a commodity recognition device for a cluttered environment, the device comprising:

[0038] A commodity multi-face detection module, configured to, in response to enabling the commodity recognition function, acquire a commodity image for commodity recognition, detect the category information of all the commodity faces in the commodity image and the position information in the commodity image, and according to the detected position information, convert the position information of each detected commodity face into the vertex coordinates of the four vertices of each commodity face, and output the category information and the vertex coordinates of each commodity face to obtain a commodity multi-face detection result, wherein the category information refers to the front face, the back face, and the end face;

[0039] A commodity multi-face association module, configured to determine the endpoint coordinates of each edge of each commodity face according to the vertex coordinates of each commodity face, divide the commodity faces whose category information represents that the commodity face is the front face or the back face into the front and back face group according to the category information of each commodity face, and divide the commodity faces whose category information represents that the commodity face is the end face into the end face group, match the commodity faces in the front and back face group and the commodity faces in the end face group according to the endpoint coordinates, and associate the commodity faces in the front and back face group and the commodity faces in the end face group having overlapping edges to obtain a commodity multi-face association result;

[0040] A commodity multi-face recognition module, configured to perform specification recognition on each detected commodity face according to the commodity multi-face detection result, record the specification categories and score values of all the commodity faces to obtain a commodity multi-face recognition result;

[0041] A comprehensive scoring module, configured to compare the score values of the commodity faces in each group of associated commodity faces according to the commodity multi-face association result and the commodity multi-face recognition result, and take the specification category corresponding to the higher score value as the final recognition result of the associated commodity face, and directly take the specification category obtained by the specification recognition of the unassociated commodity face as the final recognition result.

[0042] In a third aspect of the embodiments of the present disclosure, there is provided a computer-readable storage medium, on which a computer program is stored, and when the program is executed by a processor, the steps of the method according to any one of the first aspect are implemented.

[0043] In a fourth aspect of the embodiments of the present disclosure, there is provided an electronic device, comprising:

[0044] A memory, on which a computer program is stored;

[0045] A processor for executing the computer program in the memory to implement the steps of the method according to any one of the first aspects.

[0046] The present invention provides a method, apparatus, device and medium for identifying commodities in a cluttered environment. Compared with the prior art, the following beneficial effects are achieved:

[0047] In response to opening the commodity identification function, a commodity image for commodity identification is acquired, the category information of all commodity faces in the commodity image and the position information in the commodity image are detected, and according to the detected position information, the position information of each detected commodity face is converted into the vertex coordinates of the four vertices of each commodity face, and the category information and the vertex coordinates of each commodity face are output to obtain a commodity multi-face detection result, where the category information refers to the front face, the back face and the end face; according to the vertex coordinates of each commodity face, the endpoint coordinates of each edge of each commodity face are determined, and according to the category information of each commodity face, the commodity faces whose category information represents the front face or the back face are classified into the front and back face group, and the commodity faces whose category information represents the end face are classified into the end face group, and according to the endpoint coordinates, the commodity faces in the front and back face group and the commodity faces in the end face group are matched, and the commodity faces in the front and back face group and the commodity faces in the end face group with overlapping edges are associated to obtain a commodity multi-face association result; according to the commodity multi-face detection result, the specification of each detected commodity face is identified, and the specification categories and score values of all commodity faces are recorded to obtain a commodity multi-face identification result; according to the commodity multi-face association result and the commodity multi-face identification result, the score values of the commodity faces in each group of associated commodity faces are compared, and the specification category corresponding to the higher score value is taken as the final identification result of the associated commodity face, and for the unassociated commodity face, the specification category obtained by the specification identification is directly taken as the final identification result. The identification of commodities in a cluttered environment can combine the information of multiple commodity faces of a commodity for identification, improving the accuracy of the identification result; the information of multiple commodity faces is more comprehensive, improving the reliability of the identification result; in this method, the front face, the back face and the end face of the commodity can be associated and identified or individually identified, improving the robustness of the commodity identification method under complex conditions such as different shooting angles and methods.

[0048] Other features and advantages of the present disclosure will be described in detail in the subsequent specific implementation section. BRIEF DESCRIPTION OF THE DRAWINGS

[0049] The accompanying drawings are used to provide a further understanding of the present disclosure and form a part of the specification. Together with the following detailed description, they are used to explain the present disclosure, but do not limit the present disclosure. In the accompanying drawings:

[0050] Figure 1 It is a schematic flowchart showing a method for identifying commodities in a cluttered environment according to an embodiment of the specification.

[0051] Figure 2 A block diagram of a commodity identification device for a cluttered environment shown according to an embodiment of the specification.

[0052] Figure 3 It is a block diagram of another commodity identification device for a cluttered environment shown according to an embodiment of the specification. Detailed Description of the Embodiment

[0053] In order to make the objectives, technical solutions, and advantages of the present application more clear and understandable, the present application will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application.

[0054] The present application provides a method for identifying commodities in a cluttered environment. Figure 1 It is a flowchart of a method for identifying commodities in a cluttered environment shown according to an embodiment. The method includes:

[0055] In step S11, in response to the opening of the commodity identification function, a commodity image for commodity identification is acquired, the category information of all commodity surfaces in the commodity image and the position information in the commodity image are detected, and according to the detected position information, the position information of each detected commodity surface is converted into the vertex coordinates of the four vertices of each commodity surface, and the category information and the vertex coordinates of each commodity surface are output to obtain a commodity multi-surface detection result, where the category information refers to the front surface, the back surface, and the end surface;

[0056] Among them, the category information refers to the packaging surface of the detected commodity surface of the commodity in the commodity image, including the front surface, the back surface, and the end surface, and the position information refers to the position distribution of the detected commodity surface in the commodity image, including the actual detected actual position information and the predicted position information predicted by the target detection algorithm according to the actual position information. The vertex coordinates of the commodity surface can be expressed as:

[0057] .

[0058] In an embodiment of the present disclosure, in response to the opening of the commodity recognition function, a commodity image for commodity recognition is acquired, and a target detection algorithm based on deep learning is used to detect the position information of all commodity faces in the commodity image in the commodity image. According to the detected position information, the YOLOv8-pose deep learning model (YOLOv8 pose detection model) is used to convert the position information of each detected commodity face into the vertex coordinates of the four vertices of each commodity face, and the category information and the vertex coordinates of each commodity face are output to obtain the commodity multi-face detection result.

[0059] In step S12, according to the vertex coordinates of each commodity face, the endpoint coordinates of each edge of each commodity face are determined. According to the category information of each commodity face, the commodity faces whose category information represents the commodity face as the front or back are classified into the front and back group, and the commodity faces whose category information represents the commodity face as the end face are classified into the end face group. According to the endpoint coordinates, the commodity faces in the front and back group and the commodity faces in the end face group are matched, and the commodity faces in the front and back group and the commodity faces in the end face group with overlapping edges are associated to obtain the commodity multi-face association result.

[0060] Among them, the endpoint coordinates of a commodity face can be expressed as:

[0061] ,

[0062] The overlapping edge means that the Euclidean distance calculated from the two endpoint coordinates of one of the four edges of the commodity face in the matched front and back group and the two endpoint coordinates of one of the four edges of the commodity face in the end face group is within the preset distance threshold.

[0063] In an embodiment of the present disclosure, according to the vertex coordinates of each commodity face, the endpoint coordinates of each edge of each commodity face are determined. According to the category information of each commodity face, the commodity faces whose category information represents the commodity face as the front or back can be classified into the front and back group by algorithms such as convolutional neural network (CNN), and the commodity faces whose category information represents the commodity face as the end face can be classified into the end face group. According to the endpoint coordinates, the commodity faces in the front and back group and the commodity faces in the end face group can be matched by methods such as a matching method based on endpoint direction and distance constraints, and the commodity faces in the front and back group and the commodity faces in the end face group with overlapping edges are associated to obtain the commodity multi-face association result.

[0064] In step S13, according to the multi-sided detection result of the commodity, the specification of each detected commodity side is identified, the specification categories and score values of all the commodity sides are recorded, and a multi-sided recognition result of the commodity is obtained;

[0065] Among them, the specification category refers to the brand and specification of the commodity. For example, the specification category of a certain commodity side is brand A (premium), and the score value can represent the confidence in the specification category. The specification category and the score value are in one-to-one correspondence.

[0066] In the embodiment of the present disclosure, according to the multi-sided detection result of the commodity, machine vision technology can be used to extract the feature data of the detected commodity side, such as edge contour, color feature, area, etc., and match them with the features in the preset template or database, identify the specification of each detected commodity side, record the specification categories and score values of all the commodity sides, and obtain a multi-sided recognition result of the commodity.

[0067] In step S14, according to the multi-sided association result of the commodity and the multi-sided recognition result of the commodity, compare the score values of the commodity sides in each group of associated commodity sides, and take the specification category corresponding to the higher score value as the final recognition result of the associated commodity side. For the unassociated commodity side, directly take the specification category obtained by the specification recognition as the final recognition result.

[0068] Among them, the associated commodity side refers to the commodity sides of the same commodity with overlapping edges.

[0069] For example, according to the multi-sided association result of the commodity and the multi-sided recognition result of the commodity, compare the score values of the commodity sides in each group of associated commodity sides. Among them, the specification category of one commodity side in a group of associated commodity sides is brand A (premium) and the score value is 0.93, and the specification category of another commodity side in the associated commodity side is brand B (hard) and the score value is 0.49. Then take the specification category corresponding to the higher score value, that is, 0.93, which is brand A (premium), as the final recognition result of the associated commodity side. For the unassociated commodity side, directly take the specification category obtained by the specification recognition as the final recognition result.

[0070] In the method of the above technical solution, by responding to the opening of the commodity recognition function, a commodity image for commodity recognition is obtained, the category information of all commodity faces in the commodity image and the position information in the commodity image are detected, and according to the detected position information, the position information of each detected commodity face is converted into the vertex coordinates of the four vertices of each commodity face, and the category information and the vertex coordinates of each commodity face are output to obtain a multi-face detection result of the commodity. Among them, the category information refers to the front face, the back face, and the end face; according to the vertex coordinates of each commodity face, the endpoint coordinates of each side of each commodity face are determined, and according to the category information of each commodity face, the commodity faces whose category information represents the front or back face of the commodity face are classified into the front and back face group, and the commodity faces whose category information represents the end face of the commodity face are classified into the end face group. According to the endpoint coordinates, the commodity faces in the front and back face group and the commodity faces in the end face group are matched, and the commodity faces in the front and back face group and the commodity faces in the end face group with overlapping edges are associated to obtain a multi-face association result of the commodity; according to the multi-face detection result of the commodity, the specification of each detected commodity face is recognized, and the specification category and score value of all commodity faces are recorded to obtain a multi-face recognition result of the commodity; according to the multi-face association result and the multi-face recognition result of the commodity, the score values of the commodity faces in each group of associated commodity faces are compared, and the specification category corresponding to the higher score value is taken as the final recognition result of the associated commodity face, and for the unassociated commodity face, the specification category obtained by the specification recognition is directly taken as the final recognition result. By recognizing the commodity through multi-face association, more and more comprehensive commodity features can be recognized, and the accuracy of the commodity recognition result can be improved; and multi-face association can recognize both the end face of the commodity and the front or back face, which can improve the robustness of the commodity recognition.

[0071] In a possible implementation manner, in step S12, the matching of the commodity faces in the front and back face group and the commodity faces in the end face group according to the endpoint coordinates, and the association of the commodity faces in the front and back face group and the commodity faces in the end face group with overlapping edges to obtain a multi-face association result of the commodity includes:

[0072] According to the endpoint coordinates, calculate the Euclidean distance between the two endpoints of each side of the commodity faces in the front and back face group and the two endpoints of each side of the commodity faces in the end face group;

[0073] Among them, the Euclidean distance includes the forward distance and the reverse distance, and at least one direction of the distance is calculated when calculating the Euclidean distance.

[0074] In an embodiment of the present disclosure, according to the endpoint coordinates, calculate the Euclidean distance between the two endpoints of each side of the commodity surface in the front and back surface group and the two endpoints of each side of the commodity surface in the end surface group. The calculation formula for the forward distance can be:

[0075] ;

[0076] The calculation formula for the reverse distance can be:

[0077] ;

[0078] where the mathematical symbol represents the norm of the difference between two vectors, that is, the Euclidean distance between these two points.

[0079] According to the Euclidean distance, match the commodity surface in the front and back surface group and the commodity surface in the end surface group. If there is a commodity surface in the front and back surface group, and the Euclidean distances between the two endpoints of one side of this commodity surface and the two endpoints of one side of any commodity surface in the end surface group are all within the preset distance threshold range, then determine that the two sides, namely the side of the commodity surface in the front and back surface group and the side of the commodity surface in the end surface group, are coincident sides. Match all the coincident sides between the commodity surface in the front and back surface group and the commodity surface in the end surface group, and record the endpoint coordinates of the commodity surfaces in the front and back surface group and the commodity surfaces in the end surface group that have the coincident sides, as well as the Euclidean distance of the coincident sides, to obtain the matching result;

[0080] In an embodiment of the present disclosure, according to the Euclidean distance, match the commodity surface in the front and back surface group and the commodity surface in the end surface group. When matching, it should be judged whether the Euclidean distances in the same direction are all within the preset distance threshold range. If there is a commodity surface in the front and back surface group, and the Euclidean distances in the same direction between the two endpoints of one side of this commodity surface and the two endpoints of one side of any commodity surface in the end surface group are all within the preset distance threshold range, then determine that the two sides, namely the side of the commodity surface in the front and back surface group and the side of the commodity surface in the end surface group, are coincident sides. Match all the coincident sides between the commodity surface in the front and back surface group and the commodity surface in the end surface group, and record the endpoint coordinates of the commodity surfaces in the front and back surface group and the commodity surfaces in the end surface group that have the coincident sides, as well as the Euclidean distance of the coincident sides;

[0081] For example, according to the Euclidean distance, match the commodity surface in the front and back surface group and the commodity surface in the end surface group. When matching, it should be judged whether the two forward distances of the four endpoints of the two sides and Whether they are all within the preset distance threshold range.

[0082] According to the matching result, the commodity surfaces of the front and back surfaces group with the overlapping edge and the commodity surface of the end surface are associated to obtain a multi-surface association result of the commodity.

[0083] In the embodiment of the present disclosure, according to the matching result, the commodity surfaces of the front and back surfaces group with the overlapping edge and the commodity surface of the end surface are associated, and the commodity surfaces of the front and back surfaces group with the overlapping edge and the commodity surface of the end surface that are associated are divided into a group of associated commodity surfaces to obtain a multi-surface association result of the commodity.

[0084] In the above technical solution, the multi-surface association of the commodity is realized by judging the overlapping edge, and the commodity association result is obtained, which can provide multi-surface commodity feature information for commodity identification and can identify the commodity more accurately.

[0085] In a possible implementation manner, in step S12, the step of associating the commodity surfaces of the front and back surfaces group with the overlapping edge and the commodity surface of the end surface according to the matching result to obtain a multi-surface association result of the commodity includes:

[0086] According to the Euclidean distance of the overlapping edge, if multiple edges in the matching result form a group of overlapping edges, then the two edges with the smallest Euclidean distance among the overlapping edges are taken as the overlapping edges, and the two commodity surfaces corresponding to the two edges with the smallest Euclidean distance of the overlapping edge are associated to obtain a commodity association result;

[0087] Among them, multiple edges refer to more than two edges.

[0088] In the embodiment of the present disclosure, according to the Euclidean distance of the overlapping edge, for the case where multiple edges in the matching result form a group of overlapping edges, for example, if there are three edges in the matching result that form a group of overlapping edges, namely edge 1, edge 2, and edge 3, the Euclidean distance between edge 1 and edge 2 is 1.001, the Euclidean distance between edge 1 and edge 3 is 1.005, and the Euclidean distance between edge 2 and edge 3 is 1.0055, only the two edges with the smallest Euclidean distance among the overlapping edges, that is, edge 1 and edge 2, are taken as the overlapping edges, and the two commodity surfaces corresponding to edge 1 and edge 2 are associated to obtain a commodity association result;

[0089] If two edges in the matching result form a group of overlapping edges, then directly associate the two commodity surfaces corresponding to the two edges that form a group of overlapping edges in the matching result to obtain a commodity association result.

[0090] The above technical solution can exclude the possibility of multiple commodity surfaces becoming associated commodity surfaces, and exclude the incorrect association that may be caused by multiple commodity surfaces becoming associated commodity surfaces, making the result of the multi-surface association more accurate.

[0091] In a possible implementation, in step S12, the associating the product surface of the front and back surface group having the overlapping edge and the product surface of the end surface according to the matching result to obtain a product multi-surface association result includes:

[0092] According to the matching result, if the overlapping edge formed by the product surface of any product being blocked by the product part placed in front of the product causes mis-association, then detect whether there is partial overlap between the blocked product surface of the product and the end surface of the product placed in front of the product;

[0093] If there is partial overlap between the blocked product surface of the product and the end surface of the product placed in front of the product, calculate the ratio of the overlapping area of the blocked product surface and the end surface to the area of the smaller surface among the blocked product surface and the end surface;

[0094] If the ratio is greater than the preset area ratio threshold, then determine the blocked product surface and the end surface as the product surfaces of two products with mis-association caused by overlapping edges for filtering;

[0095] If the ratio is less than or equal to the preset area ratio threshold, then determine the blocked product surface and the end surface as belonging to the same product for association to obtain a product multi-surface association result.

[0096] In the embodiments of the present disclosure, according to the matching result, it can be determined by algorithms such as a convolutional neural network (CNN) that the overlapping edge formed by the product surface of any product being blocked by the product part placed in front of the product causes mis-association, and it can be detected by an algorithm model whether there is partial overlap between the blocked product surface of the product and the end surface of the product placed in front of the product. If there is partial overlap between the blocked product surface of the product and the end surface of the product placed in front of the product, calculate the ratio of the overlapping area of the blocked product surface and the end surface to the area of the smaller surface among the blocked product surface and the end surface; if the ratio is greater than the preset area ratio threshold, then determine the blocked product surface and the end surface as the product surfaces of two products with mis-association caused by overlapping edges for filtering; if the ratio is less than or equal to the preset area ratio threshold, then determine the blocked product surface and the end surface as belonging to the same product for association to obtain a product multi-surface association result.

[0097] The above technical solution excludes the mis-association caused by the special placement of the product by calculating the ratio of the overlapping area of the blocked product surface and the end surface to the area of the smaller surface of the two surfaces, improving the accuracy of the product multi-surface association result.

[0098] In a possible implementation, in step S12, calculating the ratio of the overlapping area of the occluded product surface and the end surface to the area of the smaller surface among the occluded product surface and the end surface includes:

[0099] If the areas of both the occluded product surface and the end surface are not zero, calculate the ratio of the overlapping area of the occluded product surface and the end surface to the area of the smaller surface of the two surfaces;

[0100] If the area of at least one of the occluded product surface and the end surface is zero, directly set the ratio to zero.

[0101] In the embodiments of the present disclosure, the calculation formula for the ratio of the overlapping area of the occluded product surface and the end surface to the area of the smaller surface of the two surfaces can be:

[0102] .

[0103] The above technical solution determines whether to calculate the area ratio by judging whether the area of the product surface in the occluded product surface and the end surface is zero, improving the calculation efficiency and facilitating subsequent multi-surface recognition of products.

[0104] In a possible implementation, in step S12, the associating the product surface of the front and back surface group having the coincident edge and the product surface of the end surface according to the matching result to obtain a product multi-surface association result further includes:

[0105] According to the matching result, if the coincident edge formed by stacking two products causes mis-association, then according to the two adjacent edges of the coincident edge in the first product surface and the two adjacent edges of the coincident edge in the second product surface, calculate the center point distance between each adjacent edge of the first product surface and each adjacent edge of the second product surface, and take the two adjacent edges with the shorter center point distance as a group of corresponding adjacent edges, and the remaining two adjacent edges as another group of corresponding adjacent edges, to obtain an adjacent edge grouping result;

[0106] Wherein, the adjacent edge refers to the edge adjacent to the coincident edge in the product surface, and the calculation formula for the center point distance can be:

[0107] ,

[0108] Where C1 and C2 are the center points of edge 1 and edge 2.

[0109] In the embodiments of the present disclosure, algorithms such as convolutional neural networks (CNNs) can be used to determine misassociations caused by overlapping edges formed by stacking two commodities. Based on the overlapping edges of the two commodity surfaces, find two adjacent edges of the overlapping edge in the first commodity surface and two adjacent edges of the overlapping edge in the second commodity surface, calculate the center point distances between each of the adjacent edges in the first commodity surface and each of the adjacent edges in the second commodity surface, and take the two adjacent edges with shorter center point distances as a set of corresponding adjacent edges, and the remaining two adjacent edges as another set of corresponding adjacent edges to obtain the adjacent edge grouping result;

[0110] For example, side 4 of commodity surface A and side 2 of commodity surface B are a set of overlapping edges. Based on the overlapping edges, find the two edges adjacent to the overlapping edge in each of the two surfaces. For commodity surface A, the adjacent edges are side 1 of commodity surface A and side 3 of commodity surface A; for commodity surface B, the adjacent edges are side 1 of commodity surface B and side 3 of commodity surface B. Calculate the center distances between side 1 of commodity surface A and side 1 and side 3 of commodity surface B, and calculate the center distances between side 3 of commodity surface A and side 1 and side 3 of commodity surface B. Assume that the center distance between side 3 of commodity surface A and side 3 of commodity surface B is shorter, then take side 3 of commodity surface A and side 3 of commodity surface B as a set of corresponding adjacent edges, and the remaining side 1 of commodity surface A and side 1 of commodity surface B as another set of corresponding adjacent edges to obtain the adjacent edge grouping result.

[0111] According to the adjacent edge grouping result, calculate the included angles between the two sets of corresponding adjacent edges;

[0112] If the included angles between the two sets of corresponding adjacent edges are both within the preset angle threshold range, it is determined that the two sets of corresponding adjacent edges of the first commodity surface and the second commodity surface are on the same straight line, and then the first commodity surface and the second commodity surface are determined as commodity surfaces that belong to two commodities but have misassociations caused by overlapping edges and are filtered;

[0113] If at least one of the included angles between the two sets of corresponding adjacent edges is not within the preset angle threshold range, it is determined that the corresponding adjacent edges of the first commodity surface and the second commodity surface are not on the same straight line, and then the first commodity surface and the second commodity surface are determined as belonging to the same commodity for association to obtain the commodity multi-surface association result.

[0114] In the embodiments of the present disclosure, according to the adjacent side grouping result, the included angles between two groups of the corresponding adjacent sides are calculated. If the included angles between two groups of the corresponding adjacent sides are all within a preset angle threshold range. For example, the preset angle threshold range can be two ranges: 0° to 1° and 179° to 180°. As long as the included angle satisfies any one of these two ranges, it is within the preset angle range. For example, if the included angles between two groups of the corresponding adjacent sides are 0.5° and 0.6° respectively, then it is within the preset angle threshold range; if the included angles between two groups of the corresponding adjacent sides are 179.5° and 179.6° respectively, then it is within the preset angle threshold range; if the included angles between two groups of the corresponding adjacent sides are 0.5° and 179.6° respectively, then it is also within the preset angle threshold range.

[0115] The above technical solution ensures the correct division of adjacent sides by using the center distance to divide the corresponding adjacent sides, providing correct data for calculating the included angle; by calculating the included angles between two groups of the corresponding adjacent sides to determine whether it is a mis-association caused by special placement, some mis-associations can be filtered out, providing a correct association result for commodity recognition.

[0116] In a possible implementation manner, in step S12, the calculating the included angles between two groups of the corresponding adjacent sides includes:

[0117] According to the endpoint coordinates of two groups of the corresponding adjacent sides, group and calculate the coordinate differences of two adjacent sides in the corresponding adjacent sides;

[0118] According to the coordinate differences, group and calculate the dot product and length of two adjacent sides in the corresponding adjacent sides;

[0119] According to the dot product and the length, group and calculate the cosine values of two adjacent sides in the corresponding adjacent sides, and obtain the included angles between two groups of the corresponding adjacent sides by calculating the inverse cosine function of the cosine values.

[0120] In the embodiments of the present disclosure, according to the endpoint coordinates of two groups of the corresponding adjacent sides. For example, one group of the corresponding adjacent sides is side 1 of commodity surface A and side 1 of commodity surface B. The endpoint coordinates of side 1 of commodity surface A are (x 1 ,y 1 ) and (x 2 ,y 2 ), and the endpoint coordinates of side 1 of commodity surface B are (x 3 ,y 3 ) and (x 4 ,y 4 ). Calculate the coordinate differences of the corresponding adjacent sides, side 1 of commodity surface A and side 1 of commodity surface B, as (x 2 -x 1 ), (x 4 -x 3 ), (y2 -y 1 ), (y 4 -y 3 ), calculate the dot product of the corresponding adjacent side commodity surface A side 1 and commodity surface B side 1 (x 2 -x 1 )(x 4 -x 3 ), + (y 2 -y 1 )(y 4 -y 3 ), and the length , , according to the dot product and the length, calculate the cosine value of two adjacent sides in the corresponding adjacent sides , by calculating the inverse cosine function of the cosine value, that is obtain the included angle of the corresponding adjacent sides .

[0121] The above technical solution uses the included angles of the two sets of corresponding adjacent sides as the basis for mis-association, and can reasonably and effectively filter out mis-associations caused by special placements of commodities.

[0122] In a possible implementation manner, in step S13, according to the commodity multi-surface detection result, perform specification recognition on each detected commodity surface, record the specification categories and score values of all the commodity surfaces, and obtain the commodity multi-surface recognition result, including:

[0123] According to the commodity multi-surface detection result, input the surface area image of each detected commodity surface, and perform specification recognition on each surface area image;

[0124] Output the specification category and score value of each commodity surface recognized by the specification;

[0125] Record the specification categories and score values of all the commodity surfaces, and obtain the commodity multi-surface recognition result.

[0126] Among them, the surface area image refers to the image of the area covered by each commodity surface in the commodity image, including the surface area images of the front, back, and end surfaces.

[0127] In the embodiments of the present disclosure, according to the multi-faceted detection result of the commodity, the surface area image of each detected commodity surface is input. First, the input commodity surface area image is preprocessed, including grayscale conversion, binarization, noise removal, etc. Image processing techniques or deep learning models are used to extract features from the preprocessed image. These features can include color, texture, shape, etc. According to the extracted features, a trained classification model (such as a convolutional neural network CNN) is used to identify the specifications of each commodity surface, and the specification category and score value of each detected commodity surface are output. The specification categories and score values of all commodity surfaces are recorded to obtain the multi-faceted recognition result of the commodity.

[0128] The method of the above technical solution performs specification recognition on the surface area image of the detected commodity. The surface area image includes the surface area images of the front, back, and end surfaces. The multi-faceted specification recognition can ensure the diversity and comprehensiveness of the commodity recognition information.

[0129] It should be understood that although Figure 1 the steps in the flowchart are shown in sequence according to the arrows, these steps are not necessarily executed in the order indicated by the arrows. Unless there is a clear indication in this article, the execution of these steps has no strict order limit, and these steps can be executed in other orders. Moreover, Figure 1 at least a part of the steps in

[0130] In one embodiment, as Figure 2 shown, a commodity recognition device in a cluttered environment is provided, including:

[0131] A commodity multi-faceted detection module 210, configured to, in response to opening the commodity recognition function, obtain a commodity image for commodity recognition, detect the category information of all commodity surfaces in the commodity image and the position information in the commodity image, and according to the detected position information, convert the position information of each detected commodity surface into the vertex coordinates of the four vertices of each commodity surface, and output the category information and the vertex coordinates of each commodity surface to obtain a commodity multi-faceted detection result, where the category information refers to the front, back, and end surfaces;

[0132] The multi - surface association module 220 of the commodity is configured to determine the endpoint coordinates of each edge of each commodity surface according to the vertex coordinates of each commodity surface, divide the commodity surfaces whose category information represents the commodity surface as the front or back surface into the front - back surface group and divide the commodity surfaces whose category information represents the commodity surface as the end surface into the end - surface group according to the category information of each commodity surface, match the commodity surfaces in the front - back surface group and the commodity surfaces in the end - surface group according to the endpoint coordinates, and associate the commodity surfaces in the front - back surface group and the commodity surfaces in the end - surface group with overlapping edges to obtain the multi - surface association result of the commodity;

[0133] The multi - surface recognition module 230 of the commodity is configured to perform specification recognition on each detected commodity surface according to the multi - surface detection result of the commodity, record the specification categories and score values of all the commodity surfaces, and obtain the multi - surface recognition result of the commodity;

[0134] The comprehensive scoring module 240 is configured to compare the score values of the commodity surfaces in each group of associated commodity surfaces according to the multi - surface association result and the multi - surface recognition result, take the specification category corresponding to the higher score value as the final recognition result of the associated commodity surfaces, and directly take the specification category obtained by the specification recognition as the final recognition result for the unassociated commodity surfaces.

[0135] In a possible implementation manner, the multi - surface association module 220 of the commodity is configured to:

[0136] Calculate the Euclidean distances between the endpoints of all the edges of the commodity surfaces in the front - back surface group and the endpoints of all the edges of the commodity surfaces in the end - surface group according to the endpoint coordinates;

[0137] Match the commodity surfaces in the front - back surface group and the commodity surfaces in the end - surface group according to the Euclidean distances. If there is a commodity surface in the front - back surface group and the Euclidean distances from the two endpoints of an edge of this commodity surface to the two endpoints of an edge of a certain commodity surface in the end - surface group are both within the preset distance threshold range, it is determined that the two edges, namely the edge of the commodity surface in the front - back surface group and the edge of the commodity surface in the end - surface group, are overlapping edges. Match all the overlapping edges among the commodity surfaces in the front - back surface group and the commodity surfaces in the end - surface group, and record the endpoint coordinates of all the commodity surfaces in the front - back surface group and the commodity surfaces in the end - surface group with overlapping edges and the Euclidean distances of the overlapping edges;

[0138] Associate the commodity surfaces in the front - back surface group with overlapping edges and the commodity surfaces in the end - surface group according to the matching result to obtain the multi - surface association result of the commodity.

[0139] In a possible implementation, the commodity multi - face association module 220 is configured to:

[0140] According to the Euclidean distance of the overlapping edges, for multiple edges in the matching result that form a group of overlapping edges, only take the two edges with the smallest Euclidean distance of the overlapping edges as the overlapping edges, and associate the two commodity faces corresponding to the two edges with the smallest Euclidean distance of the overlapping edges to obtain a commodity association result;

[0141] For the case where only two edges in the matching result form a group of overlapping edges, directly associate the two commodity faces corresponding to the two edges that form a group of overlapping edges in the matching result to obtain a commodity association result.

[0142] In a possible implementation, the commodity multi - face association module 220 is configured to:

[0143] According to the matching result, associate the commodity faces of the front - and - back face group with the overlapping edges and the commodity faces of the end faces, and filter out the mis - associations caused by special placements where multiple commodities are placed but there are overlapping edges, to obtain a commodity multi - face association result.

[0144] In a possible implementation, the commodity multi - face association module 220 is configured to:

[0145] For the mis - association caused by the overlapping edges formed by the partial occlusion of the commodity face of a certain commodity by another commodity placed in front of the certain commodity, when the predicted position area of the occluded commodity face of the certain commodity has partial overlap with the area of the end face of the other commodity, calculate the ratio of the overlapping area of the occluded commodity face and the end face to the area of the smaller face of the two faces;

[0146] If the ratio is higher than a preset area ratio threshold, it is determined that these two faces belong to two commodities and are mis - associations, and they are filtered out;

[0147] If the ratio is lower than the preset area ratio threshold, it is determined that these two faces come from the same commodity and are correct associations, and they are retained.

[0148] In a possible implementation, the commodity multi - face association module 220 is configured to:

[0149] If the areas of both the occluded commodity face and the end face are not zero, calculate the ratio of the overlapping area of the occluded commodity face and the end face to the area of the smaller face of the two faces;

[0150] If the area of at least one of the occluded commodity face and the end face is zero, directly set the ratio to zero.

[0151] In a possible implementation, the commodity multi - surface association module 220 is configured to:

[0152] Regarding the mis - association caused by the overlapping edges formed by stacking two commodities, find the two adjacent edges of the overlapping edge in the first commodity surface and the two adjacent edges of the overlapping edge in the second commodity surface, calculate the center - point distance between each adjacent edge of the first commodity surface and each adjacent edge of the second commodity surface, take the two adjacent edges with shorter center - point distances as a group of corresponding adjacent edges, and the remaining two adjacent edges as another group of corresponding adjacent edges, to obtain the adjacent - edge grouping result;

[0153] According to the adjacent - edge grouping result, calculate the included angle between the two groups of corresponding adjacent edges;

[0154] If both included angles of the two groups of corresponding adjacent edges are within the preset angle threshold range, it is determined that the two groups of corresponding adjacent edges of the first commodity surface and the second commodity surface are on the same straight line, and then the first commodity surface and the second commodity surface are determined as commodity surfaces that belong to two commodities but have mis - association caused by overlapping edges for filtering;

[0155] If at least one of the included angles of the two groups of corresponding adjacent edges is not within the preset angle threshold range, it is determined that the corresponding adjacent edges of the first commodity surface and the second commodity surface are not on the same straight line, and then the first commodity surface and the second commodity surface are determined as belonging to the same commodity for association, to obtain the commodity multi - surface association result.

[0156] In a possible implementation, the commodity multi - surface association module 220 is configured to:

[0157] According to the endpoint coordinates of the two groups of corresponding adjacent edges, group - calculate the coordinate differences of two adjacent edges in the corresponding adjacent edges;

[0158] According to the coordinate differences, group - calculate the dot product and length of two adjacent edges in the corresponding adjacent edges;

[0159] According to the dot product and the length, group - calculate the cosine values of two adjacent edges in the corresponding adjacent edges, and obtain the included angle between the two groups of corresponding adjacent edges by calculating the inverse cosine function of the cosine value.

[0160] In a possible implementation, the commodity multi - surface recognition module 230 is configured to:

[0161] According to the commodity multi - surface detection result, input the face - region image of each detected commodity surface, and perform specification recognition on each face - region image;

[0162] Output the specification category and score value of each of the commodity surfaces recognized by the specification;

[0163] Record the specification categories and score values of all the commodity surfaces to obtain the multi-surface recognition result of the commodity.

[0164] For the specific limitations of a commodity recognition device in a cluttered environment, reference can be made to the limitations of a commodity recognition method in a cluttered environment described above, which will not be elaborated here. Each module in the above-mentioned commodity recognition device in a cluttered environment can be implemented in whole or in part by software, hardware, and their combination. The above-mentioned modules can be embedded in the processor of the computer device in hardware form or be independent of it, or can be stored in the memory of the computer device in software form, so as to facilitate the processor to call and execute the operations corresponding to the above-mentioned modules.

[0165] The embodiments of the present disclosure also provide a computer-readable storage medium, on which a computer program is stored, and when the program is executed by a processor, the steps of the method described in any one of the foregoing embodiments are implemented.

[0166] The embodiments of the present disclosure also provide an electronic device, including:

[0167] A memory, on which a computer program is stored;

[0168] A processor, configured to execute the computer program in the memory to implement the steps of the method described in any one of the foregoing embodiments.

[0169] Figure 3 The commodity recognition device 100 in the shown cluttered environment includes: a processor 1001 and a memory 1003. Among them, the processor 1001 and the memory 1003 are connected, such as connected through a bus 1002. Optionally, the commodity recognition device 100 in the cluttered environment may further include a communication component, and the communication component can be used for data interaction between the device 100 and other devices, such as sending or receiving data, etc. It should be noted that in actual scheduling, the communication component is not limited to one, and the structure of the commodity recognition device 100 in the cluttered environment does not constitute a limitation to the embodiments of the present application.

[0170] The processor 1001 may be a CPU (Central Processing Unit), a general-purpose processor, a DSP (Digital Signal Processor), an ASIC (Application Specific Integrated Circuit), an FPGA (Field Programmable Gate Array), or other programmable logic devices, transistor logic devices, hardware components, or any combination thereof. It can implement or execute various exemplary logical blocks, modules, and circuits described in connection with the disclosure of this application. The processor 1001 may also be a combination that implements computing functions, such as a combination of one or more microprocessors, a combination of a DSP and a microprocessor, etc.

[0171] The bus 1002 may include a path for transmitting information between the above components. The bus 1002 may be a PCI (Peripheral Component Interconnect) bus or an EISA (Extended Industry Standard Architecture) bus, etc. The bus 1002 may be divided into an address bus, a data bus, a control bus, etc. For ease of representation, Figure 3 only a thick line is used to represent it herein, but it does not mean that there is only one bus or one type of bus.

[0172] The memory 1003 may be a ROM (Read Only Memory) or other type of static storage device that can store static information and instructions, a RAM (Random Access Memory) or other type of dynamic storage device that can store information and instructions, or it may also be an EEPROM (Electrically Erasable Programmable Read Only Memory), a CD-ROM (Compact Disc Read Only Memory), or other optical disc storage, optical disc storage (including compact discs, laser discs, optical discs, digital versatile discs, Blu-ray discs, etc.), magnetic disk storage media, other magnetic storage devices, or any other medium that can be used to carry or store program code and can be read by a computer, which is not limited herein.

[0173] The memory 1003 is used to store program codes for executing the embodiments of the present disclosure, and the execution is controlled by the processor 1001. The processor 1001 is used to execute the program codes stored in the memory 1003 to implement the steps shown in the embodiment of the commodity identification method in a cluttered environment.

[0174] The embodiment of the present disclosure also provides a computer-readable storage medium having program code stored thereon. When the program code is executed by a processor, the steps and corresponding contents of the aforementioned embodiment of a method for identifying goods in a cluttered environment can be implemented.

[0175] The preferred embodiments of the present disclosure are described in detail above in conjunction with the accompanying drawings; however, the present disclosure is not limited to the specific details in the above embodiments; within the technical concept of the present disclosure, various changes, modifications, substitutions and variations may be made to these embodiments, and these changes, modifications, substitutions and variations all fall within the protection scope of the present disclosure.

[0176] It should also be noted that the various specific technical features described in the above specific embodiments can be combined in any appropriate manner without contradiction, and they should also be regarded as the contents disclosed in this disclosure. In order to avoid unnecessary repetition, this disclosure will not further describe various possible combinations. The technical scope of this application is not limited to the contents in the specification, and its technical scope must be determined according to the scope of the claims.

Claims

1. A method for identifying commodities in a cluttered environment, characterized in that: The method comprises: In response to turning on the commodity recognition function, a commodity image for commodity recognition is obtained, category information of all commodity faces in the commodity image and position information in the commodity image are detected, and according to the detected position information, the position information of each of the detected commodity faces is converted into vertex coordinates of four vertices of each of the commodity faces, and the category information and vertex coordinates of each of the commodity faces are output to obtain a commodity multi-face detection result, wherein the category information refers to the front side, the back side and the end side; According to the vertex coordinates of each of the product faces, the endpoint coordinates of each edge of each of the product faces are determined; according to the category information of each of the product faces, the product faces whose category information indicates that the product faces are the front side or the back side are divided into the front and back face group, and the product faces whose category information indicates that the product faces are the end face are divided into the end face group; according to the endpoint coordinates, the product faces in the front and back face group are matched with the product faces in the end face group, and the product faces in the front and back face group with overlapping edges are associated with the product faces in the end face group to obtain a product multi-face association result; According to the multi-faceted detection result of the commodity, specification recognition is performed on each detected commodity face, and the specification categories and score values ​​of all commodity faces are recorded to obtain the multi-faceted recognition result of the commodity; According to the multi-faceted association result of the product and the multi-faceted identification result of the product, the score values ​​of the product faces in each group of associated product faces are compared, and the specification category corresponding to the higher score value is taken as the final identification result of the associated product face; for the unassociated product face, the specification category obtained by the specification identification is directly taken as the final identification result.

2. The method according to claim 1, characterized in that: The method of matching the product faces in the front and back face groups with the product faces in the end face groups according to the endpoint coordinates, and associating the product faces in the front and back face groups with the product faces in the end face groups having overlapping edges to obtain a product multi-face association result includes: According to the endpoint coordinates, calculating the Euclidean distances between the two endpoints of each side of the product surface in the front and back surface group and the two endpoints of each side of the product surface in the end surface group; According to the Euclidean distance, the product faces in the front and back face group are matched with the product faces in the end face group. If there is a product face in the front and back face group, and the Euclidean distances from the two end points of an edge of the product face to the two end points of an edge of any product face in the end face group are within a preset distance threshold range, then the edge of the product face in the front and back face group and the edge of the product face in the end face group are determined to be overlapping edges, and all the overlapping edges in the product faces in the front and back face group and the product faces in the end face group are matched, and the endpoint coordinates of all the product faces in the front and back face group and the product faces in the end face group with the overlapping edges and the Euclidean distances of the overlapping edges are recorded to obtain a matching result; According to the matching result, the product face of the front and back face groups having the overlapping edges and the product face of the end face are associated to obtain a product multi-face association result.

3. The method according to claim 2, characterized in that: According to the matching result, associating the product face of the front and back face groups having the overlapping edges with the product face of the end face to obtain a product multi-face association result, including: According to the Euclidean distance of the overlapping edges, if multiple edges in the matching result form a group of the overlapping edges, two edges with the smallest Euclidean distance of the overlapping edges are taken as the overlapping edges, and two product faces corresponding to the two edges with the smallest Euclidean distance of the overlapping edges are associated to obtain a product association result; If the matching result includes two edges forming a group of overlapping edges, two commodity faces corresponding to the two edges forming a group of overlapping edges in the matching result are directly associated to obtain a commodity association result.

4. The method according to claim 2, characterized in that: According to the matching result, associating the product face of the front and back face groups having the overlapping edges with the product face of the end face to obtain a product multi-face association result, including: According to the matching result, if the product surface of any of the products is partially blocked by the product placed in front of the product to form an overlapping edge causing a false association, then it is detected whether the blocked product surface of the product and the end surface of the product placed in front of the product have a partial overlap; If the obstructed product surface of the product partially overlaps with the end surface of the product placed in front of the product, calculate the ratio of the overlapping area of ​​the obstructed product surface and the end surface to the area of ​​the smaller surface between the obstructed product surface and the end surface; If the ratio is greater than a preset area ratio threshold, the blocked product surface and the end surface are determined as product surfaces that belong to two products but are incorrectly associated due to overlapping edges and are filtered out; If the ratio is less than or equal to the preset area ratio threshold, the blocked product surface and the end surface are determined to belong to the same product and are associated to obtain a product multi-surface association result.

5. The method according to claim 4, characterized in that: The calculating of the ratio of the overlapping area of ​​the obstructed product surface and the end surface to the area of ​​the smaller surface between the obstructed product surface and the end surface comprises: If the areas of the obstructed product surface and the end surface are not 0, then the ratio of the overlapping area of ​​the obstructed product surface and the end surface to the area of ​​the smaller surface of the two surfaces is calculated; If the area of ​​at least one of the obscured product surface and the end surface is 0, the ratio is directly set to 0.

6. The method according to claim 2, characterized in that: According to the matching result, associating the product face of the front and back face groups having the overlapping edges with the product face of the end face to obtain a product multi-face association result, further comprising: According to the matching result, if the overlapping edges formed by the stacking of two products cause a false association, then according to the two adjacent edges of the overlapping edge in the first product surface and the two adjacent edges of the overlapping edge in the second product surface, the center point distance between each adjacent edge of the first product surface and each adjacent edge of the second product surface is calculated, and the two adjacent edges with the shorter center point distance are taken as a group of corresponding adjacent edges, and the remaining two adjacent edges are taken as another group of corresponding adjacent edges, to obtain the adjacent edge grouping result; According to the adjacent edge grouping result, calculating the angle between the two groups of corresponding adjacent edges; If the included angles of the two groups of corresponding adjacent edges are both within the preset angle threshold range, it is determined that the two groups of corresponding adjacent edges of the first product face and the second product face are on the same straight line, and the first product face and the second product face are determined as product faces that belong to two products but are mistakenly associated due to overlapping edges, and are filtered out; If at least one of the angles of the two groups of corresponding adjacent edges is outside the preset angle threshold range, it is determined that the corresponding adjacent edges of the first product surface and the second product surface are not on the same straight line, and the first product surface and the second product surface are determined to belong to the same product and associated to obtain a product multi-surface association result.

7. The method according to claim 6, characterized in that: The calculating the angle between the two groups of corresponding adjacent edges comprises: According to the two groups of endpoint coordinates of the corresponding adjacent edges, grouping and calculating the coordinate difference of two adjacent edges in the corresponding adjacent edges; According to the coordinate difference, group and calculate the dot product and length of two adjacent edges of the corresponding adjacent edges; According to the dot product and the length, the cosine values ​​of two adjacent sides of the corresponding adjacent sides are calculated in groups, and the angle between the two groups of corresponding adjacent sides is obtained by calculating the arccosine function of the cosine value.

8. The method according to claim 1, characterized in that: The step of performing specification recognition on each detected product surface according to the product multi-surface detection result, recording the specification categories and score values ​​of all product surfaces, and obtaining the product multi-surface recognition result includes: According to the detection results of the multiple sides of the commodity, input the surface area image of each detected surface of the commodity, and perform specification recognition on each surface area image; Output the specification category and score value of each product surface identified by the specification; The specification categories and score values ​​of all the product faces are recorded to obtain the product multi-faceted recognition result.

9. A commodity identification device in a cluttered environment, characterized in that: The device comprises: The commodity multi-face detection module is configured to, in response to turning on the commodity recognition function, obtain a commodity image for commodity recognition, detect category information of all commodity faces in the commodity image and position information in the commodity image, convert the position information of each of the detected commodity faces into vertex coordinates of four vertices of each of the commodity faces according to the detected position information, output the category information and vertex coordinates of each of the commodity faces, and obtain a commodity multi-face detection result, wherein the category information refers to the front side, the back side, and the end side; The commodity multi-faceted association module is configured to determine the endpoint coordinates of each edge of each commodity face according to the vertex coordinates of each commodity face, and to classify the commodity face whose category information indicates that the commodity face is the front side or the back side into a front-back face group and to classify the commodity face whose category information indicates that the commodity face is the end face into an end face group according to the category information of each commodity face, and to match the commodity faces in the front-back face group with the commodity faces in the end face group according to the endpoint coordinates, and to associate the commodity faces in the front-back face group with the commodity faces in the end face group that have overlapping edges, so as to obtain a commodity multi-faceted association result; The commodity multi-faceted recognition module is configured to perform specification recognition on each detected commodity face according to the commodity multi-faceted detection result, record the specification category and score value of all commodity faces, and obtain the commodity multi-faceted recognition result; The comprehensive scoring module is configured to compare the score values ​​of the product faces in each group of associated product faces based on the multi-faceted association results of the product and the multi-faceted identification results of the product, and take the specification category corresponding to the higher score value as the final identification result of the associated product face; for unassociated product faces, directly take the specification category obtained by the specification identification as the final identification result.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the program is executed by a processor, the steps of the method described in any one of claims 1 to 8 are implemented.

11. An electronic device, characterized in that: include: a memory having a computer program stored thereon; A processor, configured to execute the computer program in the memory to implement the steps of the method according to any one of claims 1 to 8.

Images