An unmanned retail commodity identification system

Abstract

The present invention proposes an unmanned retail commodity recognition system, which includes an object detection unit, an image acquisition unit, an image recognition unit, a sensor array, a data fusion processing unit, a recognition algorithm module and a commodity interaction database for data interaction; the object detection unit: monitors the unmanned retail cabinet in real time and detects the commodities taken by customers, determines the position and time of the commodities through a laser scanner and a motion detection sensor, and sends a trigger signal to the image recognition unit when reaching a preset area, triggering the image acquisition unit to capture the commodity image; the image recognition unit is built with several high-resolution cameras, when the object detection unit detects the movement of the commodity, the camera is activated and captures the image of the commodity, and uploads the captured image to the image recognition unit; the image recognition unit is used to process the image data captured by the image acquisition unit and use image processing and machine learning algorithms to analyze the image.

Description

Technical Field

[0001] The present invention belongs to the field of unmanned retail, and in particular relates to an unmanned retail commodity identification system. Background Art

[0002] The unmanned retail product recognition system is an intelligent system based on computer vision and artificial intelligence technology, which aims to realize the automatic recognition and management of products in unmanned retail scenarios. The system uses advanced image processing, object detection and machine learning algorithms, combined with sensor technology and database management, to realize functions such as automatic recognition, pricing, inventory management and user interaction of products, thereby improving the efficiency and intelligence level of retail business. The unmanned retail product recognition system is a new retail solution based on artificial intelligence and Internet of Things technology. It uses computer vision and deep learning algorithms to realize automatic recognition and measurement of products, thereby realizing an unattended retail experience. The emergence of this system not only improves the efficiency and convenience of the retail industry, but also brings a new shopping experience to consumers.

[0003] The technical background of unmanned retail product recognition systems mainly comes from the development of computer vision and deep learning technology. Computer vision is a technology that uses computers and cameras to simulate and realize human visual functions. It can convert information in images and videos into digital signals, thereby realizing the recognition and analysis of objects. Deep learning is a machine learning method based on artificial neural networks. It can realize the recognition and analysis of complex patterns through training with large amounts of data, thereby realizing the automated processing of complex tasks.

[0004] In the unmanned retail product recognition system, computer vision and deep learning technologies are used to identify and measure products. First, the system will take pictures of the products through cameras and other devices, and then use computer vision technology to process and analyze the images to extract the characteristic information of the products. Then, the system will use deep learning algorithms to learn and train these characteristic information to achieve automatic identification and measurement of products. Finally, the system will match the identified product information with inventory information to achieve automatic sales and management of products.

[0005] In addition to computer vision and deep learning technologies, the unmanned retail product recognition system also involves the application of Internet of Things technology. The Internet of Things is a technology that connects and communicates objects and devices through the Internet, which can achieve real-time monitoring and control of objects. In the unmanned retail product recognition system, the Internet of Things technology can connect and coordinate devices such as cameras, sensors and data processing equipment to achieve real-time monitoring and management of products.

[0006] The unmanned retail product recognition system is a new retail solution based on computer vision, deep learning and IoT technology. It uses advanced technical means to realize the automatic identification and measurement of goods, thereby improving the efficiency and convenience of the retail industry. With the continuous development of artificial intelligence and IoT technology, it is believed that the unmanned retail product recognition system will be more widely used in the future.

[0007] Therefore, there is an urgent need for an unmanned retail product recognition system. Summary of the invention

[0008] The present invention proposes an unmanned retail commodity identification system, which solves the problem in the prior art that the current unmanned retail commodity cannot be identified for some commodities that are not in stock, which leads to the problem that during unmanned retail, the commodity shelving efficiency is low and the commodity needs to be stored and retrieved according to the set area.

[0009] The technical solution of the present invention is implemented as follows: an unmanned retail commodity recognition system includes an object detection unit, an image acquisition unit, an image recognition unit, a sensor array, a data fusion processing unit, a recognition algorithm module and a commodity interaction database for data interaction;

[0010] The object detection unit: monitors the unmanned retail cabinet in real time and detects the goods taken by the customer, determines the location and time of the goods through the laser scanner and the motion detection sensor, and sends a trigger signal to the image recognition unit when reaching the preset area, triggering the image acquisition unit to capture the image of the goods;

[0011] The image recognition unit has several high-resolution cameras built in. When the object detection unit detects the movement of the commodity, the camera is activated and captures the image of the commodity, and uploads the captured image to the image recognition unit;

[0012] The image recognition unit is used to process the image data captured by the image acquisition unit, use image processing and machine learning algorithms to analyze the image, extract the characteristics of the product therefrom, and send the extracted product characteristics to the data fusion processing unit;

[0013] The sensor array includes a weight sensor, a temperature sensor, and a pressure sensor, which collects non-visual attributes of goods, performs auxiliary data collection on the movement of goods and the ambient temperature change data in the unmanned container, and sends the collected data to the data fusion processing unit;

[0014] The data fusion processing unit is responsible for integrating the data from the image recognition unit and the sensor array, integrating the visual and non-visual data through the fusion algorithm, and forming integrated data to be sent to the recognition algorithm module;

[0015] The recognition algorithm module uses deep learning and pattern recognition algorithms to identify the integrated data uploaded by the data fusion processing unit, compares the integrated data with the product information in the product interaction database to determine the identity of the product, and sends the product data with confirmed identity to the product interaction database for data update;

[0016] The commodity interaction database is a centralized repository of identifiable commodity information, including the commodity's visual features, non-visual attributes, price, and inventory information, and identifies and stores commodity features after receiving a newly identified commodity from the recognition algorithm module.

[0017] Current product identification systems usually use technologies such as barcodes, QR codes or RFID to identify products. These technologies require that products be marked before they are put on the shelves, and the information and location of the products need to be entered into the system. This approach has some disadvantages: the products must be entered into the system before they are placed in the designated location of the unmanned cabinet. This means that if new products are put on the shelves or the location of the products changes, the system needs to be manually updated, which increases management costs and workload. Products must be placed in designated locations to ensure that the system can correctly identify the products. This limits the display of products and the layout of cabinets, affecting the flexibility and efficiency of retail space. If the products are not correctly marked or entered into the system, or are placed in the wrong location, the system cannot accurately identify the products, resulting in misidentification or missed identification. Since the products need to be entered into the system in advance, manual intervention and management are required. This increases the complexity of the work and possible errors. For newly launched products, since they need to be entered into the system before being placed in the designated location, this brings certain difficulties and delays to the launch of new products. The existing commodity recognition system requires commodities to be entered into the system in advance and placed in a designated location, which has certain limitations and drawbacks. This requires a more intelligent unmanned retail commodity recognition system to solve these problems. This is why the new unmanned retail commodity recognition system needs to improve the accuracy and real-time performance of commodity recognition through technologies such as real-time monitoring, multi-camera image recognition, data fusion processing and deep learning algorithms, so as to solve the problems of the existing system.

[0018] The unmanned retail commodity recognition system in this application document has the following differences from the prior art: the object detection unit monitors the position and time of the commodity in real time through a laser scanner and a motion detection sensor, and sends a trigger signal to the image recognition unit when it reaches a preset area. This real-time trigger mechanism can improve the response speed and accuracy of the system, and can capture the image of the commodity more quickly than the traditional timed acquisition. The image recognition unit has multiple high-resolution cameras built in, which can capture multiple angles and details of the commodity, thereby improving the accuracy and stability of commodity recognition. Compared with a single-camera system, a multi-camera system has a stronger information collection capability. The sensor array collects non-visual attribute data of the commodity, such as weight, temperature, and pressure information. The data fusion processing unit is responsible for integrating data from the image recognition unit and the sensor array, and integrating visual and non-visual data through a fusion algorithm. This comprehensive processing can provide more comprehensive and accurate commodity information, which helps to improve the accuracy and reliability of commodity recognition. The recognition algorithm module uses deep learning and pattern recognition algorithms to identify commodities on the integrated data. Compared with traditional image processing algorithms, deep learning algorithms have stronger learning and recognition capabilities, which can improve the accuracy and adaptability of commodity recognition. The commodity interaction database is a centralized repository of identifiable commodity information, including the commodity's visual features, non-visual attributes, prices, inventory information, etc., which can achieve comprehensive management and updating of commodity information. This centralized storage and management method can improve the system's data management efficiency and accuracy. This unmanned retail commodity recognition system has obvious advantages and innovations in real-time monitoring, multi-camera image recognition, data fusion processing, deep learning algorithms, and commodity interaction databases. It can improve the system's recognition accuracy, response speed, and reliability, and bring more advanced technical solutions to the unmanned retail field.

[0019] As a preferred implementation, when combining visual and non-visual data, the data fusion module performs weighted fusion processing on the data, thereby improving the accuracy and robustness of recognition through the weighted fusion processing.

[0020] As a preferred implementation, when performing weighted fusion processing, the collected data is firstly subjected to comprehensive weighted averaging using a weighted averaging method, and then the data is subjected to real-time data processing using a Kalman filter.

[0021] As a preferred embodiment, when weighted averaging is performed by the weighted averaging method, data from different sensors or sources are weighted averaged to reduce noise and improve data accuracy. A weighting coefficient is obtained based on the accuracy and reliability of the sensor, and the data from different sources are weighted averaged using the weighting coefficient.

[0022] As a preferred implementation, the recognition algorithm module trains a deep learning model and uses any one of ResNet, VGG, and Inception to establish an algorithm model. The object in the image is recognized through the algorithm model, and the features in the image are extracted and the image data is matched with the product through feature matching.

[0023] After adopting the above technical solution, the beneficial effect of the present invention is that through multi-camera image recognition and deep learning algorithms, the system can more accurately identify goods, reduce misidentification and missed identification, thereby improving the reliability and accuracy of the entire system. The system can monitor the location and time of the goods in real time, and trigger image acquisition when reaching the preset area, which helps to capture the image of the goods in time and improves the response speed and real-time performance of the system. By integrating data from the image recognition unit and the sensor array, the system can provide more comprehensive and accurate product information, which helps to improve the accuracy and reliability of product identification. Since the system can quickly and accurately identify goods, users can more conveniently and quickly complete the selection and payment of goods during the shopping process, which improves the user's shopping experience. The commodity interaction database can centrally store and manage commodity information, realize comprehensive management and update of commodity information, and help improve the data management efficiency and accuracy of the system. This unmanned retail commodity recognition system can bring more efficient, convenient and reliable solutions to the unmanned retail industry by improving recognition accuracy, real-time monitoring and triggering mechanism, data fusion processing, user experience and data management and update, etc., which helps to improve the competitiveness and user satisfaction of the retail industry. BRIEF DESCRIPTION OF THE DRAWINGS

[0024] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative labor.

[0025] Figure 1 This is a system block diagram of the present invention. DETAILED DESCRIPTION

[0026] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0027] Example:

[0028] like Figure 1 As shown, an unmanned retail commodity recognition system includes an object detection unit, an image acquisition unit, an image recognition unit, a sensor array, a data fusion processing unit, a recognition algorithm module and a commodity interaction database for data interaction;

[0029] The object detection unit: monitors the unmanned retail cabinet in real time and detects the goods taken by the customer, determines the location and time of the goods through the laser scanner and the motion detection sensor, and sends a trigger signal to the image recognition unit when reaching the preset area, triggering the image acquisition unit to capture the image of the goods;

[0030] The image recognition unit has several high-resolution cameras built in. When the object detection unit detects the movement of the commodity, the camera is activated and captures the image of the commodity, and uploads the captured image to the image recognition unit;

[0031] The image recognition unit is used to process the image data captured by the image acquisition unit, use image processing and machine learning algorithms to analyze the image, extract the characteristics of the product therefrom, and send the extracted product characteristics to the data fusion processing unit;

[0032] The sensor array includes a weight sensor, a temperature sensor, and a pressure sensor, which collects non-visual attributes of goods, performs auxiliary data collection on the movement of goods and the ambient temperature change data in the unmanned container, and sends the collected data to the data fusion processing unit;

[0033] The data fusion processing unit is responsible for integrating the data from the image recognition unit and the sensor array, integrating the visual and non-visual data through the fusion algorithm, and forming integrated data to be sent to the recognition algorithm module;

[0034] The recognition algorithm module uses deep learning and pattern recognition algorithms to identify the integrated data uploaded by the data fusion processing unit, compares the integrated data with the product information in the product interaction database to determine the identity of the product, and sends the product data with confirmed identity to the product interaction database for data update;

[0035] The commodity interaction database is a centralized repository of identifiable commodity information, including the commodity's visual features, non-visual attributes, price, and inventory information, and identifies and stores commodity features after receiving a newly identified commodity from the recognition algorithm module.

[0036] The working principle and workflow of the unmanned retail product recognition system are as follows: The location of products in the unmanned retail cabinet and the time when customers take away products are monitored in real time through laser scanners and motion detection sensors. Once the product moves to the preset area, the object detection unit will trigger the image acquisition unit to capture the product image. When the object detection unit is triggered, the high-resolution camera built into the image recognition unit is activated and captures the image of the product, and then uploads the captured image to the image recognition unit. The image recognition unit uses image processing and machine learning algorithms to analyze the image, extract the characteristics of the product, and send the extracted product characteristics to the data fusion processing unit. The weight sensor, temperature sensor, and pressure sensor collect the non-visual attributes of the product and the ambient temperature change data in the unmanned cabinet, and then send the collected data to the data fusion processing unit. It is responsible for integrating the data from the image recognition unit and the sensor array, integrating the visual and non-visual data through the fusion algorithm, and forming integrated data to be sent to the recognition algorithm module. The integrated data uploaded by the data fusion processing unit is used for product recognition through deep learning and pattern recognition algorithms, and the comprehensive data is compared with the product information in the product interaction database to determine the identity of the product, and the product data with confirmed identity is sent to the product interaction library for data update. Centrally store identifiable product information, including visual features, non-visual attributes, prices, inventory information, etc. of the products, and receive and store newly identified product features from the recognition algorithm module.

[0037] The system of this application document monitors the location of goods in real time through laser scanners and motion detection sensors, and triggers image acquisition when reaching the preset area, which improves the response speed and real-time performance of the system and avoids the drawback of the need to enter product information in advance in traditional systems. The data of the image recognition unit and the sensor array are integrated, and the visual and non-visual data are integrated through a fusion algorithm to provide more comprehensive and accurate product information, which helps to improve the accuracy and reliability of product identification. The use of deep learning and pattern recognition algorithms for product identification has stronger learning and recognition capabilities, and improves the accuracy and adaptability of product identification. This unmanned retail product identification system solves the drawbacks of the need to enter product information and location in advance in traditional systems through the innovative application of technologies such as real-time monitoring, data fusion processing, and deep learning algorithms, improves the accuracy and real-time performance of product identification, and has a higher level of intelligence and automation.

[0038] In the data fusion module, when combining visual and non-visual data, the data is subjected to weighted fusion processing, and the accuracy and robustness of recognition are improved by weighted fusion processing. The weighted fusion processing of visual and non-visual data is performed to improve the accuracy and robustness of commodity recognition. The weighted fusion processing here refers to assigning different weights to different types of data and combining them to obtain more reliable results. In this case, the visual data may include image features of the commodity, while the non-visual data may include environmental and commodity attribute data collected by sensors such as weight, temperature and pressure. Through weighted fusion processing, the system can perform weighted processing on different data according to the importance and reliability of different data types to ensure that the actual situation of the commodity is more accurately reflected when various data are comprehensively considered. The image features of the commodity may be more intuitive and important, so they can be given a higher weight; and data such as ambient temperature and weight can also be used as auxiliary information, and appropriate weights are given according to their contribution to commodity recognition. Through this weighted fusion processing, the system can consider the contribution of different data more comprehensively, thereby improving the accuracy and robustness of commodity recognition. By performing weighted fusion processing on different types of data, the system can better utilize various information, improve the accuracy and reliability of product identification, and thus enhance the performance and efficiency of the entire unmanned retail product identification system.

[0039] When performing weighted fusion processing, the collected data is firstly subjected to comprehensive weighted averaging using the weighted averaging method, and then the data is processed in real time through the Kalman filter. The different types of collected data are subjected to comprehensive weighted averaging using the weighted averaging method. This means that the system assigns a weight to each data type, then multiplies each data type by its corresponding weight, and then adds them together to obtain a comprehensive weighted average. This approach ensures that different types of data are given appropriate attention when being integrated, and are weighted according to their importance and contribution.

[0040] Real-time data processing is performed on the data through the Kalman filter. The Kalman filter is a mathematical tool used to extract state information from incomplete and noisy data. It can eliminate noise in the data and improve the accuracy of the data by dynamically estimating and predicting the data. In this case, the Kalman filter can help the system process data in real time, improve the stability and robustness of the data, and better support the accuracy of the product recognition system. The use of weighted averaging for comprehensive weighted averaging can ensure that different types of data are given appropriate attention, while real-time data processing through the Kalman filter can improve the stability and accuracy of the data. This method can better integrate various information in the weighted fusion processing process, thereby improving the accuracy and robustness of the product recognition system.

[0041] When weighted averaging is performed by weighted averaging, the data from different sensors or sources are weighted averaged to reduce noise and improve data accuracy. The weighting coefficient is obtained according to the accuracy and reliability of the sensor, and the data from different sources are weighted averaged by the weighting coefficient. According to the accuracy and reliability of the sensor, the system assigns a weighting coefficient to each sensor or data source. Sensors with high accuracy and reliability will be given higher weights, while sensors with low accuracy and reliability will be given lower weights. This is based on the assumption that the data produced by sensors with high accuracy and reliability are closer to the true value and should therefore play a greater role in weighted averaging.

[0042] Data from different sources are weighted averaged using these weighting coefficients. Data with higher weights will have a larger share in the weighted average, while data with lower weights will have a smaller impact on the result. This ensures that data from different sensors or sources are given appropriate weight when combined, thereby improving the accuracy and reliability of the data.

[0043] This method can reduce noise and improve data accuracy by weighted averaging data from different sources and obtaining weighting coefficients based on the accuracy and reliability of the sensor, thereby ensuring that the system obtains more accurate and reliable results. This strategy is very important in data fusion processing, especially when multiple sensors or data sources need to be used in combination, which can effectively improve the performance of the system and the accuracy of the results.

[0044] The recognition algorithm module trains the deep learning model, adopts any one of ResNet, VGG, and Inception to establish the algorithm model, identifies the objects in the image through the algorithm model, and extracts the features in the image, and uses feature matching to match the image data with the goods. The deep learning model (such as ResNet, VGG, Inception, etc.) is a machine learning model based on an artificial neural network, which can learn the features and patterns in the image by training a large amount of image data. In this case, by training the deep learning model, the system can learn the visual features and patterns of various commodities, so that the system can more accurately identify the objects in the image. Once the deep learning model is trained, the system can use the model to identify the objects in the image. By extracting the features in the image, the system can compare and match these features with the pre-trained commodity features to determine the identity of the object in the image. This feature matching process can help the system accurately match the image data with the corresponding commodity, thereby realizing the identification and management of the commodity. By training deep learning models and using ResNet, VGG, Inception and other models to build algorithm models, the system can learn the visual features and patterns of products, so that it can more accurately identify objects in images and match image data with products through feature matching. This method can improve the accuracy and robustness of the system's recognition of products, providing more powerful recognition capabilities for unmanned retail product recognition systems.

[0045] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principle of the present invention should be included in the protection scope of the present invention.

Claims

1. An unmanned retail commodity identification system, characterized in that: It includes data interaction between an object detection unit, an image acquisition unit, an image recognition unit, a sensor array, a data fusion processing unit, a recognition algorithm module and a commodity interaction database; The object detection unit: monitors the unmanned retail cabinet in real time and detects the goods taken by the customer, determines the location and time of the goods through the laser scanner and the motion detection sensor, and sends a trigger signal to the image recognition unit when reaching the preset area, triggering the image acquisition unit to capture the image of the goods; The image recognition unit has several high-resolution cameras built in. When the object detection unit detects the movement of the commodity, the camera is activated and captures the image of the commodity, and uploads the captured image to the image recognition unit; The image recognition unit is used to process the image data captured by the image acquisition unit, use image processing and machine learning algorithms to analyze the image, extract the characteristics of the product therefrom, and send the extracted product characteristics to the data fusion processing unit; The sensor array includes a weight sensor, a temperature sensor, and a pressure sensor, which collects non-visual attributes of goods, performs auxiliary data collection on the movement of goods and the ambient temperature change data in the unmanned container, and sends the collected data to the data fusion processing unit; The data fusion processing unit is responsible for integrating the data from the image recognition unit and the sensor array, integrating the visual and non-visual data through the fusion algorithm, and forming integrated data to be sent to the recognition algorithm module; The recognition algorithm module uses deep learning and pattern recognition algorithms to identify the integrated data uploaded by the data fusion processing unit, compares the integrated data with the product information in the product interaction database to determine the identity of the product, and sends the product data with confirmed identity to the product interaction database for data update; The commodity interaction database is a centralized repository of commodity information available for identification, wherein the commodity information includes visual features and non-visual attributes, and identifies and stores the commodity features after receiving a newly identified commodity from the identification algorithm module.

2. The unmanned retail commodity identification system according to claim 1, characterized in that: The data fusion processing unit performs weighted fusion processing on the data when combining visual and non-visual data, thereby improving the accuracy and robustness of recognition through weighted fusion processing.

3. The unmanned retail commodity identification system according to claim 2, characterized in that: When performing weighted fusion processing, the collected data is firstly subjected to comprehensive weighted averaging using the weighted averaging method, and then the data is processed in real time using the Kalman filter.

4. The unmanned retail commodity identification system according to claim 3, characterized in that: When weighted averaging is performed using the weighted averaging method, data from different sensors or sources are weighted averaged to reduce noise and improve data accuracy. A weighting coefficient is obtained based on the accuracy and reliability of the sensor, and data from different sources are weighted averaged using the weighting coefficient.

5. The unmanned retail commodity identification system according to claim 3, characterized in that: The recognition algorithm module trains a deep learning model and uses any one of ResNet, VGG, and Inception to establish an algorithm model. The algorithm model is used to identify objects in the image, and features in the image are extracted and matched with the image data using feature matching.

Images