Vending cabinet commodity visual identification system based on machine learning model

Abstract

The invention relates to the technical field of commodity visual identification, in particular to a vending cabinet commodity visual identification system based on a machine learning model, which comprises a feature extraction module used for determining position information of a target identification commodity, shooting a picture of the position of the target identification commodity to obtain a commodity image, and performing feature extraction on the commodity image to obtain a feature extraction result; constructing a target identification commodity feature network; the database construction module is used for collecting the feature information and the mutual relation of each commodity in the vending cabinet, processing the feature information and the mutual relation of each commodity, and storing the processed feature information and the mutual relation to a feature database; the matching identification module is used for establishing an association relationship between the image feature information association network and the standard feature information association network, determining feature information matched with the target commodity feature information from the feature database, and determining an identification result of the target commodity based on the commodity corresponding to the feature information, the method can achieve the accurate and efficient recognition of the commodities in the vending cabinet.

Description

Technical Field

[0001] The present invention relates to the technical field of commodity visual recognition, and in particular to a vending machine commodity visual recognition system based on a machine learning model. Background Art

[0002] With the rapid development of the retail industry, the visual recognition system of goods in vending machines has gradually become an important tool for improving shopping experience and optimizing inventory management. Traditional methods of product recognition mainly rely on barcodes or RFID tags, but these methods have problems such as easy damage to the tags and limited recognition distance. In recent years, product recognition systems based on computer vision and machine learning technology have gradually emerged. They realize label-free product recognition by capturing product images and extracting feature information.

[0003] However, when the existing visual recognition system for merchandise in vending machines is in use, it is difficult to accurately and efficiently identify the merchandise in the vending machines due to the wide variety of merchandise and different shapes, as well as the dense arrangement and mutual obstruction of the merchandise in the vending machines, which affects the recognition effect.

[0004] To this end, we propose a vending machine product visual recognition system based on a machine learning model to solve the above problems. Summary of the invention

[0005] The purpose of the present invention is to provide a vending machine commodity visual recognition system based on a machine learning model to solve the problems raised in the above background technology.

[0006] To achieve the above object, the present invention provides the following technical solution: a vending machine commodity visual recognition system based on a machine learning model, comprising: A feature extraction module is used to determine the location information of the target identification product, take a picture of the location of the target identification product to obtain a product image, extract features from the product image, and construct a target identification product feature network; A database construction module is used to collect characteristic information and mutual relationships of each commodity in the vending cabinet, process the characteristic information and mutual relationships of each commodity, and store them in a characteristic database; The matching recognition module is used to establish an association relationship between the image feature information association network and the standard feature information association network, and determine the feature information that matches the target product feature information from the feature database, and determine the recognition result of the target product based on the product corresponding to the feature information.

[0007] Preferably, the feature extraction module includes: A photographing unit, used to photograph the target identified commodity to obtain a commodity image; An extraction unit is used to pre-process the commodity image and extract image feature information of the commodity image, wherein the image feature information includes lines and turning points; a plurality of first feature points are set on the lines, second feature points are set corresponding to the turning points, and an image feature information association network is constructed based on the first feature points and the second feature points; The marking unit is used to mark the image feature information corresponding to the target identification commodity in the image feature information association network to obtain the target identification commodity feature network.

[0008] Preferably, the database construction module includes: A collection unit is used to capture the characteristic information of commodities in the vending cabinet, wherein the characteristic information of commodities includes the contour information of commodities, the pattern information of commodities, and the relative position relationship between the patterns; A processing unit, used to extract the contour information of each commodity and the commodity pattern information corresponding to each commodity, identify and mark lines and turning points on the contour information of the commodity and the commodity pattern information corresponding to each commodity, and set them as standard feature points; and construct a standard feature information association network of the commodity based on multiple standard feature points; The construction unit is used to obtain the basic information of the product, the product feature information and the standard feature association network and store them in the database.

[0009] Preferably, the step of constructing an image feature information association network based on the first feature point and the second feature point includes: obtaining historical first feature points and historical second feature points of multiple historical commodity images, and training an image feature information association network model based on the multiple historical first feature points and the multiple historical second feature points; inputting the first feature point and the second feature point into the image feature information association network model to obtain the image feature information association network.

[0010] Preferably, the processing unit comprises: An acquisition subunit is used to acquire the shooting angle of the product image, the contour information of each product, and the pattern information corresponding to each product; A shooting subunit, used for taking a shooting angle of the commodity image as a shooting angle for contour information of the commodity and pattern information corresponding to the commodity, and shooting the commodity in the vending cabinet based on the shooting angle to obtain an image of the vending cabinet; The processing subunit identifies and marks the contour information of the commodity and the lines and turning points of the pattern information corresponding to the commodity from the vending machine image, sets them as standard feature points, and sequentially connects multiple standard feature points to obtain a standard feature information association network of the commodity.

[0011] Preferably, the matching identification module includes: A matching unit, configured to compare the image feature information association network with the standard feature information association network, determine the position of the image feature information association network on the standard feature information association network to obtain at least one preliminary feature region, and select a region corresponding to the target identification commodity feature network from the preliminary feature region as a target region; The identification unit is used to associate the standard feature information with multiple standard feature points on the Internet and bind them to the corresponding commodities respectively, determine the standard feature points corresponding to the target area to obtain the target standard feature points, and use the commodities corresponding to the target standard feature points as the identification results of the target commodities.

[0012] Preferably, the matching unit comprises: The first acquisition subunit is used to obtain edge feature points of the target identification commodity feature network and determine the contour of the target identification commodity feature network based on the edge feature points; Divide the sub-units; divide the standard feature information association network into regions based on the outline of the target identification product feature network, and take the region with the highest number of feature points overlapping with the target identification product feature network as the feature region; The second collection subunit is used to collect feature point information of the target identification commodity feature network and feature point information in the feature area, wherein the feature point information includes the number of feature points, the positional relationship between the feature points, and the distance between the feature points; A calculation subunit calculates the similarity between the image feature information association network and the feature region based on the feature point information of the target recognition commodity feature network and the feature point information in the feature region; The matching subunit obtains a feature area whose similarity meets a preset condition as the position of the target identification product feature network on the standard feature information association network, and obtains at least one preliminary feature area; selects an area corresponding to the target identification product feature network from the preliminary feature area as the target area, and obtains the target area.

[0013] Preferably, the step of obtaining a feature region whose similarity meets a preset condition as the position of the target identification product feature network on the standard feature information association network to obtain a preliminary feature region; and selecting a region corresponding to the target identification product feature network from the feature region as the target region to obtain the target region comprises: The number of preliminary feature areas is counted. When the number of preliminary feature areas is greater than one, the image feature information association network is obtained, the position of the target identification product feature network is determined from the image feature information association network, and the range of the target identification product feature network is extended outward from the edge with the target identification product feature network as the center to obtain the second target identification product feature network, and the second target identification product feature network is continuously compared with the standard feature information association network until the similarity of only one preliminary feature area meets the preset conditions, and it is used as the target area.

[0014] Preferably, the step of dividing the standard feature information association network into regions based on the contour of the target identification commodity feature network and taking the region with the highest number of feature points overlapping with the target identification commodity feature network as the feature region comprises: The contour information of the target identification product feature network and multiple standard feature points in the standard feature information association network are obtained, and the target identification product feature network is traversed in the standard feature information association network. The contour of the target identification product feature network is used as the area range with the currently traversed feature point as the center; during the traversal process, the area with the highest number of overlaps with the feature points of the target identification product feature network is extracted as the feature area.

[0015] Compared with the prior art, the present invention has the following beneficial effects: 1. Obtain product images through the shooting unit, and use the extraction unit to preprocess and extract features of the product images to build a target recognition product feature network. In the feature extraction process, the present invention pays special attention to image feature information such as lines and turning points, sets multiple first feature points on the lines, sets second feature points corresponding to the turning points, and builds an image feature information association network based on these feature points, thereby improving the accuracy and stability of feature extraction.

[0016] 2. By collecting the characteristic information and mutual relationship of each commodity in the vending machine and constructing a standard characteristic information association network, a reliable data basis is provided for subsequent matching and identification. The matching and identification module quickly determines the identification result of the target commodity by comparing the association relationship between the image characteristic information association network and the standard characteristic information association network, effectively improving the efficiency and accuracy of matching and identification. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the accompanying drawings required for describing the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other accompanying drawings can be obtained based on these accompanying drawings without paying creative work.

[0018] Figure 1 It is a system structure block diagram of the present invention. DETAILED DESCRIPTION

[0019] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0020] Example: See Figure 1 The present invention provides a technical solution of a vending machine commodity visual recognition system based on a machine learning model: a vending machine commodity visual recognition system based on a machine learning model, comprising the following steps: A feature extraction module is used to determine the location information of the target identification product, take a picture of the location of the target identification product to obtain a product image, extract features from the product image, and construct a target identification product feature network; The feature extraction module includes: A photographing unit, used to photograph the target identified commodity to obtain a commodity image; An extraction unit is used to pre-process the commodity image and extract image feature information of the commodity image, wherein the image feature information includes lines and turning points; a plurality of first feature points are set on the lines, second feature points are set corresponding to the turning points, and an image feature information association network is constructed based on the first feature points and the second feature points; A marking unit, used to mark the image feature information corresponding to the target identification commodity in the image feature information association network to obtain the target identification commodity feature network; The step of constructing an image feature information association network based on the first feature point and the second feature point includes: obtaining a plurality of historical first feature points and a plurality of historical second feature points of the commodity images, training an image feature information association network model based on the plurality of historical first feature points and the plurality of historical second feature points; inputting the first feature point and the second feature point into the image feature information association network model to obtain the image feature information association network; Specifically, the product image is first grayscale preprocessed to convert the grayscale image into a binary image, that is, the pixels in the image are divided into two categories: foreground (white) and background (black). This step helps to more clearly identify the edges and contours of the product, and an edge detection algorithm (such as Canny edge detection) is used to identify the edges in the image. The edge includes at least the contour line of the product and the contour line of the pattern on the product. By connecting adjacent edge points, a continuous line is formed. The line is used to describe the contour of the product and the contour of the pattern on the product. On the extracted lines, a corner point detection algorithm (such as Harris corner point detection, Shi-Tomasi corner point detection, etc.) is used to identify the corner point. The corner point is a point on the line with a large change in curvature, usually corresponding to the inflection point or protruding part of the product. Multiple first feature points are set on the line evenly or according to specific rules. These feature points can be anywhere on the line. Usually, key points that can represent the shape of the line are selected, and second feature points are set corresponding to the corner points. These feature points directly correspond to the detected corner points. The first feature point and the second feature point are connected in sequence to form a feature information association network, which is used to describe the contour shape of the product and the positional relationship between the contour shape of the pattern and the corner points. The feature information association network can be used as an index or feature vector to quickly retrieve similar product images, which can improve the accuracy of subsequent product recognition; A database construction module is used to collect characteristic information and mutual relationships of each commodity in the vending cabinet, process the characteristic information and mutual relationships of each commodity, and store them in a characteristic database; The database building blocks include: A collection unit is used to capture the characteristic information of commodities in the vending cabinet, wherein the characteristic information of commodities includes the contour information of commodities, the pattern information of commodities, and the relative position relationship between the patterns; A processing unit, used to extract the contour information of each commodity and the commodity pattern information corresponding to each commodity, identify and mark lines and turning points on the contour information of the commodity and the commodity pattern information corresponding to each commodity, and set them as standard feature points; and construct a standard feature information association network of the commodity based on multiple standard feature points; A construction unit is used to obtain basic information of the product, product feature information and a standard feature association network and store them in a database; Specifically, the characteristic information of the goods in the vending machine is photographed to capture the contour information, pattern information and relative position relationship between the patterns of the goods. When photographing, multi-angle shooting or stereo shooting technology can be used to obtain more comprehensive characteristic information of the goods. The contour information of the goods is extracted by using image processing technology (such as edge detection, image segmentation, etc.). The pattern information on the goods, including the shape, color, texture, etc. of the pattern, is extracted by using image recognition technology (such as template matching, feature extraction, etc.). Lines and turning points are identified and marked on the contour information and pattern information of the goods, and these lines and turning points are set as standard characteristic points. The selection of standard characteristic points should have certain representativeness and stability to ensure the accuracy and reliability of subsequent product identification. Finally, a standard characteristic information association network of the goods is constructed based on multiple standard characteristic points. The association network describes the relative position and relationship between the characteristic points of the goods, and provides a basis for subsequent matching and identification. The basic information of the goods, such as the name, number, category, and price of the goods, is obtained from the goods management system, and the characteristic information and standard characteristic association network of the goods are obtained from the processing unit. This information is integrated into a complete product record. The product record is stored in the database. The database can be a relational database or a non-relational database, which can store the basic information, feature information and standard feature association network of each commodity in the vending cabinet, so as to facilitate the subsequent commodity identification; The processing unit includes: An acquisition subunit is used to acquire the shooting angle of the product image, the contour information of each product, and the pattern information corresponding to each product; A shooting subunit, used for taking a shooting angle of the commodity image as a shooting angle for contour information of the commodity and pattern information corresponding to the commodity, and shooting the commodity in the vending cabinet based on the shooting angle to obtain an image of the vending cabinet; The processing subunit identifies and marks the contour information of the commodity and the lines and turning points of the pattern information corresponding to the commodity from the vending machine image, sets them as standard feature points, and sequentially connects multiple standard feature points to obtain a standard feature information association network of the commodity; Specifically, the shooting angle of the product image, the contour information of each product, and the pattern information corresponding to each product are obtained. The placement angle and position information of the products in the vending cabinet are obtained through devices such as sensors or cameras, and the contour information and pattern information of the products are extracted using image processing technology. Example: Assuming that there is a beverage bottle with red packaging in the vending cabinet, the acquisition subunit can obtain the shooting angle (such as the front, side, etc.), contour information (such as the shape of the bottle body, the position of the bottle cap, etc.) and pattern information (such as the brand logo and pattern on the red packaging, etc.) of the beverage bottle. The shooting angle of the product image is used as the shooting angle for the contour information of the product and the pattern information corresponding to the product, and the products in the vending cabinet are photographed based on the shooting angle to obtain the vending cabinet image. According to the shooting angle information provided by the acquisition subunit, the shooting angle and position of the camera are adjusted to ensure that the overall appearance and feature information of the product can be clearly captured. Then, the product in the vending cabinet is photographed using the camera to obtain the vending cabinet image. Example: Continuing with the example of a beverage bottle with red packaging, the shooting subunit will adjust the shooting angle of the camera according to its shooting angle (such as the front) to ensure that the front image of the beverage bottle, including its contour information and pattern information, can be clearly captured. The contour information of the product and the lines and turning points of the pattern information corresponding to the product are identified and marked from the vending machine image, set as standard feature points, and multiple standard feature points are connected in sequence to obtain the standard feature information association network of the product. The image of the vending machine is extracted and identified using image processing technology and machine learning algorithms. First, the contour information and lines and turning points of the pattern information of the product are identified, and then these key points are set as standard feature points. Finally, according to the position and relationship of the standard feature points, they are connected in sequence to obtain the standard feature information association network of the product. For the beverage bottle with red packaging, the processing subunit will identify the lines of its contour (such as the curve of the bottle body, the straight line of the bottle cap, etc.) and turning points (such as the junction between the bottle body and the bottle cap, the turning point of the bottle bottom, etc.) from the vending machine image, and set the key points as standard feature points. Then, these standard feature points are connected in sequence to obtain a standard feature information association network that describes the shape and features of the beverage bottle. The shooting angle, contour information and pattern information of the goods in the vending machine can be accurately obtained, and the standard feature information association network of the goods can be generated based on this information. The accuracy and efficiency of product recognition can be improved; A matching recognition module is used to establish a correlation relationship between the image feature information correlation network and the standard feature information correlation network, and determine the feature information that matches the feature information of the target product from the feature database, and determine the recognition result of the target product based on the product corresponding to the feature information; The matching recognition module includes: A matching unit, configured to compare the image feature information association network with the standard feature information association network, determine the position of the image feature information association network on the standard feature information association network to obtain at least one preliminary feature region, and select a region corresponding to the target identification commodity feature network from the preliminary feature region as a target region; The identification unit is used to associate the standard feature information with multiple standard feature points on the network and bind them to the corresponding commodities respectively, determine the standard feature points corresponding to the target area to obtain the target standard feature points, and use the commodities corresponding to the target standard feature points as the identification results of the target commodities; The matching unit includes: The first acquisition subunit is used to obtain edge feature points of the target identification commodity feature network and determine the contour of the target identification commodity feature network based on the edge feature points; Divide the sub-units; divide the standard feature information association network into regions based on the outline of the target identification product feature network, and take the region with the highest number of feature points overlapping with the target identification product feature network as the feature region; The second collection subunit is used to collect feature point information of the target identification commodity feature network and feature point information in the feature area, wherein the feature point information includes the number of feature points, the positional relationship between the feature points, and the distance between the feature points; A calculation subunit calculates the similarity between the image feature information association network and the feature region based on the feature point information of the target recognition commodity feature network and the feature point information in the feature region; The matching subunit obtains a feature region whose similarity satisfies a preset condition as a position of the target identification product feature network on the standard feature information association network, and obtains at least one preliminary feature region; selects a region corresponding to the target identification product feature network from the preliminary feature region as a target region, and obtains the target region; It should be noted that the content of calculating the similarity between the image feature information association network and the feature area based on the feature point information of the target identification product feature network and the feature point information in the feature area is: Number of feature point matching pairs: First, use a matching algorithm (such as nearest neighbor search, etc.) to find matching pairs between the feature points of the two images. Then, calculate the number of matching pairs For each pair of matching feature points, calculate the difference in their positional relationship, such as the distance difference and angle difference , a similarity score can be constructed ,in The larger the value of, the more similar the two images are. For example, the following formula can be used: ,in, and is the weight coefficient, which is used to adjust the impact of distance difference and angle difference on the similarity score; Specifically, the edge feature points of the target identification commodity feature network are obtained, and the contour of the commodity is determined based on these edge feature points. The commodity image is processed using image processing technology (such as edge detection algorithm) to extract the edge feature points of the commodity. Then, the contour of the commodity is constructed according to the position and connection relationship of these feature points. The standard feature information association network (i.e., the standard image feature map of the shelf) is divided into regions based on the contour of the target identification commodity feature network, and the region with the highest number of feature point overlaps with the target identification commodity feature network is used as the feature region. The standard feature information association network is divided into multiple small regions. Then, for each small region, the number of feature point overlaps with the target identification commodity feature network is calculated. The region with the highest number of overlaps is determined as the feature region. The feature point information of the target identification commodity feature network and the feature point information in the feature region are collected. The feature point information includes the number of feature points, the positional relationship between feature points, and the distance between feature points. The target identification commodity feature network and the image in the feature region are processed using a feature extraction algorithm to extract feature point information. This information will be used in the subsequent similarity calculation and matching process. Based on the feature point information of the target identification product feature network and the feature point information in the feature area, the similarity between the image feature information association network (i.e., the overall feature information of the target identification product feature network) and the feature area is calculated. An appropriate similarity calculation method (such as feature point matching algorithm, shape context algorithm, etc.) is used to calculate the similarity between the image feature information association network and the feature area. The higher the similarity value, the higher the degree of match between the two. A similarity threshold is set as a preset condition. Then, the feature areas that meet the threshold conditions are selected from the similarity values ​​obtained by the calculation subunit as the preliminary feature areas. Finally, based on the position, size, shape and other information of the preliminary feature areas, combined with the feature information of the target identification product feature network, the area corresponding to the target identification product feature network is selected as the target area. The various subunits in the processing unit work together, from the acquisition of product images, feature extraction, area division, similarity calculation to matching positioning, which can improve the accuracy and efficiency of product identification; A feature region whose similarity meets a preset condition is obtained as the position of the target identification product feature network on the standard feature information association network to obtain a preliminary feature region; an area corresponding to the target identification product feature network is selected from the feature region as the target region, and the step of obtaining the target region includes: counting the number of preliminary feature regions, when the number of preliminary feature regions is greater than one, obtaining the image feature information association network, determining the position of the target identification product feature network from the image feature information association network, and taking the target identification product feature network as the center, extending the range of the target identification product feature network to the outside of the edge to obtain a second target identification product feature network, and continuing to compare the second target identification product feature network with the standard feature information association network until only one preliminary feature region has a similarity that meets the preset condition, and using it as the target region; Specifically, the target identification commodity feature network and the standard feature information association network are used to calculate the similarity, and the feature area whose similarity meets the preset condition is obtained as the preliminary feature area. The preset condition is a similarity threshold. Only when the similarity of the feature area exceeds the threshold, it is regarded as the preliminary feature area. The number of preliminary feature areas obtained is counted. If the number of preliminary feature areas is greater than one, it means that there are multiple possible matching positions and further screening is required. When there are multiple preliminary feature areas, the system determines the approximate position of the target identification commodity feature network from the image feature information association network. Then, with the target identification commodity feature network as the center, a certain range is extended to the outside of the edge to obtain the second target identification commodity feature network. This extended range can be adjusted according to actual conditions to ensure that possible matching areas can be covered. The second target identification commodity feature network is compared with the standard feature information association network, and the similarity is recalculated. If at this time, the similarity of multiple preliminary feature areas still meets the preset condition, the system continues to expand the range of the second target identification commodity feature network and repeats the above comparison and screening process. After multiple comparisons and screenings, when the similarity of only one preliminary feature area meets the preset condition, the system determines it as the final target area. At this time, the target area is the area that best matches the target identification commodity feature network, and its position on the standard feature information association network is also accurately determined. The final target area is selected from multiple preliminary feature areas, which improves the accuracy and efficiency of commodity identification. The steps of dividing the standard feature information association network into regions based on the contour of the target identification product feature network and taking the region with the highest number of feature points overlapping with the target identification product feature network as the feature region include: obtaining the contour information of the target identification product feature network and a plurality of standard feature points in the standard feature information association network, traversing the target identification product feature network in the standard feature information association network, taking the contour of the target identification product feature network as the region range with the feature point currently traversed as the center; in the traversal process, extracting the region with the highest number of feature points overlapping with the target identification product feature network as the feature region; Specifically, the contour information of the target identification commodity feature network is obtained, including the coordinates of the key points on the contour and their connection relationship. Multiple standard feature points in the standard feature information association network are obtained, and the feature points represent potential matching positions on the shelf. Each standard feature point in the standard feature information association network is traversed. During the traversal process, the contour of the target identification commodity feature network is taken as the potential matching area range with the feature point currently traversed as the center. For each traversed feature point, the number of overlaps with the feature points of the target identification commodity feature network is calculated. The calculation of the number of overlaps can be achieved by comparing the coordinates, direction, distance and other attributes of the feature points. After the traversal is completed, the number of overlaps between each potential matching area (i.e., the area centered on each standard feature point) and the feature points of the target identification commodity feature network is counted. The area with the highest number of overlaps is extracted as the feature area. This feature area is considered to be the area that best matches the target identification commodity feature network. The information of the feature area is output, including the center coordinates of the feature area, the range size, and the number of overlapped feature points with the target identification commodity feature network, etc., for subsequent commodity identification. It can realize fast and accurate identification and positioning of commodities on the shelf. The contour information of the target identification commodity feature network is used to divide the standard feature information association network into regions, and the feature region that best matches the target identification commodity feature network is extracted from it. This can improve the accuracy and efficiency of commodity identification; The present invention obtains commodity images through a shooting unit, and uses an extraction unit to preprocess and extract features of the commodity images to construct a target identification commodity feature network. In the feature extraction process, the present invention pays special attention to image feature information such as lines and turning points, sets multiple first feature points on the lines, sets second feature points corresponding to the turning points, and constructs an image feature information association network based on these feature points, thereby improving the accuracy and stability of feature extraction. By collecting the feature information and mutual relationship of each commodity in the vending machine, and constructing a standard feature information association network, a reliable data foundation is provided for subsequent matching and identification. The matching and identification module quickly determines the identification result of the target commodity by comparing the association relationship between the image feature information association network and the standard feature information association network, effectively improving the efficiency and accuracy of matching and identification.

[0021] In the description of this specification, the description with reference to the terms "one embodiment", "example", "specific example", etc. means that the specific features, structures, materials or characteristics described in conjunction with the embodiment or example are included in at least one embodiment or example of the present invention. In this specification, the schematic representation of the above terms does not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials or characteristics described can be combined in any one or more embodiments or examples in a suitable manner.

[0022] Although embodiments of the present invention have been shown and described, it will be appreciated by those skilled in the art that various changes, modifications, substitutions and variations may be made to the embodiments without departing from the principles and spirit of the present invention, and that the scope of the present invention is defined by the appended claims and their equivalents.

Claims

1. A visual recognition system for vending machine products based on a machine learning model, characterized in that: include: A feature extraction module is used to determine the location information of the target identification product, take a picture of the location of the target identification product to obtain a product image, extract features from the product image, and construct a target identification product feature network; A database construction module is used to collect characteristic information and mutual relationships of each commodity in the vending cabinet, process the characteristic information and mutual relationships of each commodity, and store them in a characteristic database; The matching recognition module is used to establish an association relationship between the image feature information association network and the standard feature information association network, and determine the feature information that matches the target product feature information from the feature database, and determine the recognition result of the target product based on the product corresponding to the feature information.

2. The vending machine commodity visual recognition system based on a machine learning model according to claim 1, characterized in that: The feature extraction module comprises: A photographing unit, used to photograph the target identified commodity to obtain a commodity image; An extraction unit is used to pre-process the commodity image and extract image feature information of the commodity image, wherein the image feature information includes lines and turning points; a plurality of first feature points are set on the lines, second feature points are set corresponding to the turning points, and an image feature information association network is constructed based on the first feature points and the second feature points; The marking unit is used to mark the image feature information corresponding to the target identification commodity in the image feature information association network to obtain the target identification commodity feature network.

3. The vending machine commodity visual recognition system based on a machine learning model according to claim 2, characterized in that: The step of constructing an image feature information association network based on the first feature point and the second feature point includes: obtaining historical first feature points and historical second feature points of multiple historical commodity images, training an image feature information association network model based on the multiple historical first feature points and multiple historical second feature points; inputting the first feature point and the second feature point into the image feature information association network model to obtain the image feature information association network.

4. The vending machine commodity visual recognition system based on a machine learning model according to claim 1, characterized in that: The database construction module includes: A collection unit is used to capture the characteristic information of commodities in the vending cabinet, wherein the characteristic information of commodities includes the contour information of commodities, the pattern information of commodities, and the relative position relationship between the patterns; A processing unit, used to extract the contour information of each commodity and the commodity pattern information corresponding to each commodity, identify and mark lines and turning points on the contour information of the commodity and the commodity pattern information corresponding to each commodity, and set them as standard feature points; and construct a standard feature information association network of the commodity based on multiple standard feature points; The construction unit is used to obtain the basic information of the product, the product feature information and the standard feature association network and store them in the database.

5. The vending machine commodity visual recognition system based on a machine learning model according to claim 4, characterized in that: The processing unit comprises: An acquisition subunit is used to acquire the shooting angle of the product image, the contour information of each product, and the pattern information corresponding to each product; A shooting subunit, used for taking a shooting angle of the commodity image as a shooting angle for contour information of the commodity and pattern information corresponding to the commodity, and shooting the commodity in the vending cabinet based on the shooting angle to obtain an image of the vending cabinet; The processing subunit identifies and marks the contour information of the commodity and the lines and turning points of the pattern information corresponding to the commodity from the vending machine image, sets them as standard feature points, and sequentially connects multiple standard feature points to obtain a standard feature information association network of the commodity.

6. The vending machine commodity visual recognition system based on a machine learning model according to claim 1, characterized in that: The matching identification module comprises: A matching unit, configured to compare the image feature information association network with the standard feature information association network, determine the position of the image feature information association network on the standard feature information association network to obtain at least one preliminary feature region, and select a region corresponding to the target identification commodity feature network from the preliminary feature region as a target region; The identification unit is used to associate the standard feature information with multiple standard feature points on the Internet and bind them to the corresponding commodities respectively, determine the standard feature points corresponding to the target area to obtain the target standard feature points, and use the commodities corresponding to the target standard feature points as the identification results of the target commodities.

7. The vending machine commodity visual recognition system based on a machine learning model according to claim 6, characterized in that: The matching unit comprises: The first acquisition subunit is used to obtain edge feature points of the target identification commodity feature network and determine the contour of the target identification commodity feature network based on the edge feature points; Divide the sub-units; divide the standard feature information association network into regions based on the outline of the target identification product feature network, and take the region with the highest number of feature points overlapping with the target identification product feature network as the feature region; The second collection subunit is used to collect feature point information of the target identification commodity feature network and feature point information in the feature area, wherein the feature point information includes the number of feature points, the positional relationship between the feature points, and the distance between the feature points; A calculation subunit calculates the similarity between the image feature information association network and the feature region based on the feature point information of the target recognition commodity feature network and the feature point information in the feature region; The matching subunit obtains a feature area whose similarity meets a preset condition as the position of the target identification product feature network on the standard feature information association network, and obtains at least one preliminary feature area; selects an area corresponding to the target identification product feature network from the preliminary feature area as the target area, and obtains the target area.

8. The vending machine commodity visual recognition system based on a machine learning model according to claim 7, characterized in that: The feature area whose similarity meets the preset conditions is obtained as the position of the target identification product feature network on the standard feature information association network to obtain the preliminary feature area; The step of selecting an area corresponding to the target identification commodity feature network from the feature area as the target area, and obtaining the target area includes: The number of preliminary feature areas is counted. When the number of preliminary feature areas is greater than one, the image feature information association network is obtained, the position of the target identification product feature network is determined from the image feature information association network, and the range of the target identification product feature network is extended outward from the edge with the target identification product feature network as the center to obtain the second target identification product feature network, and the second target identification product feature network is continuously compared with the standard feature information association network until the similarity of only one preliminary feature area meets the preset conditions, and it is used as the target area.

9. The vending machine commodity visual recognition system based on a machine learning model according to claim 7, characterized in that: The step of dividing the standard feature information association network into regions based on the outline of the target identification commodity feature network and taking the region with the highest number of feature points overlapping with the target identification commodity feature network as the feature region comprises: The contour information of the target identification product feature network and multiple standard feature points in the standard feature information association network are obtained, and the target identification product feature network is traversed in the standard feature information association network. The contour of the target identification product feature network is used as the area range with the currently traversed feature point as the center; during the traversal process, the area with the highest number of overlaps with the feature points of the target identification product feature network is extracted as the feature area.

Images