Interactive vending system based on AI large model

Abstract

The invention discloses an interactive vending system based on an AI large model, which comprises a vending cabinet main body, an AI large model processing module, an edge calculation module and a wireless network module, and is characterized in that a sensor group is arranged inside the vending cabinet main body, and a data acquisition unit is arranged outside the vending cabinet main body; the AI large model processing module is used for analyzing and processing the data information acquired by the data acquisition unit and making a response; the edge calculation module carries a pre-trained lightweight AI large model, is connected with the sensor group and the data acquisition unit through a communication interface, and carries out local processing and analysis on data acquired by the sensor group and the data acquisition unit; and the wireless network module is used for uploading the complex request which cannot be processed by the edge calculation module to the cloud large model, the cloud large model carries out deep reasoning, a processing result is returned, and the lightweight AI large model of the edge calculation module is updated. According to the invention, local intelligent reasoning is realized, cloud dependence is greatly reduced, and the response speed and data security of the system are improved.

Description

Technical Field

[0001] The present invention relates to the technical field of intelligent retail, and in particular to an interactive vending system based on an AI large model. Background Art

[0002] Mass production, mass consumption, and changes in consumption patterns and sales environments require the emergence of new circulation channels; on the contrary, with the emergence of new circulation channels such as supermarkets and department store shopping centers, labor costs have been rising continuously; coupled with the constraints of site limitations and shopping convenience and other factors, unmanned vending cabinets have emerged as a necessary machine. A vending cabinet is a machine that can automatically deliver goods according to the inserted coins. A vending cabinet is a commonly used device for commercial automation. It is not restricted by time and location, can save manpower, and facilitate transactions. It is a new form of commercial retail and is also known as a 24-hour mini supermarket.

[0003] The existing interaction methods of vending cabinets are single, only interacting through buttons or scanning codes, lacking the ability of active service, with a long user purchase decision-making time and low conversion rate; moreover, the large model solutions used in existing vending cabinets rely on cloud processing of data, posing risks of high latency, bandwidth consumption, and data privacy leakage. Summary of the Invention

[0004] In order to overcome the above problems existing in the prior art, the present invention proposes an interactive vending system based on an AI large model.

[0005] The technical solution adopted by the present invention to solve its technical problems is: an interactive vending system based on an AI large model, including a vending cabinet main body, an AI large model processing module, an edge computing module, and a wireless network module. A sensor group is arranged inside the vending cabinet main body, and a data acquisition unit is arranged outside. The AI large model processing module is used to analyze and process the data information collected by the data acquisition unit and make a response. The edge computing module is built into the vending cabinet main body, and the edge computing module is equipped with a pre-trained lightweight AI large model; the edge computing module is connected to the sensor group and the data acquisition unit of the vending cabinet main body through a communication interface, and locally processes and analyzes the data collected by the sensor group and the data acquisition unit. The wireless network module is used to upload complex requests that the edge computing module cannot process to the cloud large model. The cloud large model performs in-depth reasoning, returns the processing result, and updates the lightweight AI large model of the edge computing module.

[0006] In the above-mentioned interactive vending system based on an AI large model, an API interface is further arranged on the vending cabinet main body, and the API interface accesses external information through the wireless network module.

[0007] The above-mentioned interactive vending system based on the large AI model, wherein the data acquisition unit includes a camera for capturing the facial expressions and movements of consumers, a microphone array for collecting the voice commands of consumers, and an infrared sensor for sensing the approach of consumers to the main body of the vending cabinet; the sensor group includes a weight sensor for real-time monitoring of changes in the weight of goods and a temperature and humidity sensor for monitoring the temperature and humidity inside the vending cabinet by the user.

[0008] The above-mentioned interactive vending system based on the large AI model, wherein the large AI model is a machine learning model with large-scale parameters and a complex computing structure, including but not limited to DeepSeek-R1, OpenAI o1, Gemini-2Pro.

[0009] The specific process of loading the lightweight large AI model on the edge computing module in the above-mentioned interactive vending system based on the large AI model is as follows: Using the deep learning framework PyTorch, fine-tuning with the LoRA method, adopting an enhanced dataset in the retail field, reducing the inference computing requirements through INT8 quantization technology, and training the large AI model into a lightweight large AI model by means of knowledge distillation.

[0010] The above-mentioned interactive vending system based on the large AI model, wherein the main body of the vending cabinet is also provided with a touch display screen, which is used to display product information, interact with consumers, display the recommended content generated by the large AI model, and is connected to the edge computing module to achieve real-time information update and interaction.

[0011] The working process of the above-mentioned interactive vending system based on the large AI model specifically includes: The infrared sensor senses the distance between the customer and the main body of the vending cabinet. When the customer approaches the main body of the vending cabinet by a certain distance, a greeting is activated, and the voice information of the customer is collected through the microphone array. At the same time, the camera identifies the face, and it is judged whether the customer is a new customer through the edge computing module. The collected voice information and face information are fused into multi-modal data, and product recommendations are made for the customer in combination with the product knowledge graph and the user preference library; After the customer takes the product, product promotions are carried out according to the taken product.

[0012] The above-mentioned interactive vending system based on the large AI model, the specific process of commodity recommendation includes: sending the multi-modal data fusion result to the lightweight large AI model for real-time inference. If the lightweight large AI model cannot obtain the inference result, the data will be uploaded to the cloud large model. After filtering the inference result obtained by the lightweight large AI model or the cloud large model, a voice response will be output, and the recommended information will be displayed on the display screen. If the customer accepts the recommendation, it will be recorded in the user historical preference library to increase the recommendation weight of relevant commodities; if the recommendation is not accepted, the recommendation weight of relevant commodities will be decreased.

[0013] The above-mentioned interactive vending system based on the large AI model, the specific process of commodity promotion includes: after the user takes the commodity, the lightweight large AI model queries complementary commodities related to the commodity from the knowledge graph library, combines the customer's purchase records and browsing behaviors, analyzes the user's preference degree for different types of associated commodities, and combines the user's current purchase scenario and time, as well as the current promotion activities and inventory situation, to recommend associated commodities participating in the promotion; record whether the user purchases the recommended associated commodities. If the recommendation is accepted, the feedback on this recommendation will be recorded in the user historical preference library to increase the recommendation weight of relevant commodities; if the recommendation is not accepted, the recommendation weight of relevant commodities will be decreased.

[0014] The beneficial effect of the present invention is that the present invention provides an interactive vending cabinet based on the large AI model, which combines the freezer with the large AI model, and uses the large model to deeply mine and analyze multi-dimensional data such as the facial analysis, voice interaction, and historical preferences of consumers, reflecting powerful multi-modal fusion and inference capabilities, and realizing interactive functions such as active greeting, multi-round dialogue, commodity recommendation, and commodity promotion. The large AI model can more accurately understand the user's needs, is equipped with a touch display screen and voice interaction functions, enhances the interaction between the customer and the vending cabinet, and provides a more personalized and user-friendly service experience.

[0015] Through technologies such as large model quantization and distillation, the large model is trained into a lightweight version and deployed on the edge computing module. While retaining the core inference ability, the consumption of computing resources is greatly reduced, realizing local intelligent inference, greatly reducing the dependence on the cloud, protecting customer privacy, and improving the response speed and data security of the system. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] Figure 1 is a schematic diagram of the present invention; Figure 2 is a schematic diagram of the active greeting scenario process of the present invention; Figure 3 is a schematic diagram of the multi-round dialogue scenario process of the present invention; Figure 4 is a flowchart of the commodity recommendation of the present invention; Figure 5This is a schematic diagram of the product promotion process of the present invention. Detailed implementation manners

[0017] To enable those skilled in the art to better understand the technical solution of the present invention, the present invention will be described in detail below in conjunction with the accompanying drawings and specific implementation manners.

[0018] As Figure 1 shown, this embodiment discloses an interactive vending system based on an AI large model, including a vending cabinet main body, an AI large model processing module, an edge computing module, and a wireless network module. A sensor group is arranged inside the vending cabinet main body, and a data acquisition unit is arranged outside.

[0019] Inside the vending cabinet and on the cabinet door, a sensor group is installed. The infrared sensor senses the approach of consumers to the vending cabinet and automatically wakes up the interactive system for active greeting; the weight sensor monitors the change in the weight of the goods in real time, discovers the actions of the goods being picked up or put back, and provides data support for product recommendation and inventory management; the temperature and humidity sensor monitors the temperature and humidity environment inside the cabinet in real time to ensure that the goods are stored under suitable conditions and extend the shelf life.

[0020] The overall vending cabinet adopts a sturdy metal frame structure, and multiple shelves are arranged inside for placing various goods. The cabinet door is made of transparent material, which is convenient for consumers to view the goods. The data acquisition unit specifically includes: the cabinet body is equipped with a high-resolution wide-angle camera for capturing the facial expressions and actions of consumers, and at the same time is provided with a high-sensitivity microphone array for collecting the voice commands of consumers. In addition, the vending cabinet accesses data such as weather, holidays, and crowd heat maps through an external API.

[0021] The AI large model processing module is used to analyze and process the data information collected by the data acquisition unit and make a response; the AI large model is a machine learning model with large-scale parameters and complex computing structures, which can handle more complex tasks and data, such as DeepSeek-R1, OpenAI o1, Gemini-2Pro, etc. It is trained with a large amount of data and parameters to generate text similar to humans or answer natural language questions. It has a wide range of applications in the fields of natural language processing, text generation, and intelligent dialogue.

[0022] The edge computing module is built into the vending cabinet main body, and the edge computing module is equipped with a pre-trained lightweight AI large model; the edge computing module is connected to the sensor group and data acquisition unit of the vending cabinet main body through a communication interface, and locally processes and analyzes the data collected by the sensor group and data acquisition unit; the AI large model processing unit undergoes a large amount of data augmentation training in the retail field, as well as quantization and distillation optimization, to form a lightweight AI large model. While maintaining a high accuracy rate, this model significantly reduces the storage space and computing resource requirements, and can operate efficiently on the edge computing module.

[0023] The specific process of loading the lightweight AI large model on the edge computing module is as follows: The deep learning framework PyTorch is adopted, and the LoRA (Low-Rank Adaptation) method is used for fine-tuning. Its core is to add a trainable low-rank adaptation layer based on the pre-trained weights, freeze most of the parameters of the original model, only train a small part, introduce a low-rank matrix in key layers such as the Attention layer, approximate the modeling with a small number of parameters, and only train the LoRA adaptation layer without modifying the original model, thereby saving a large amount of video memory, loading the enhanced dataset in the retail field to train the model and save it.

[0024] When running on the edge computing module, it is necessary to adapt to the AI processor and acceleration unit, reduce the inference calculation requirements through INT8 quantization technology, and use knowledge distillation to train the large model into a lightweight version. Utilize RKNN to accelerate inference, convert the onnx model into an RKNN model, and finally deploy it on the Rockchip RK3588 board.

[0025] The wireless network module is used to upload complex requests that the edge computing module cannot handle to the cloud large model. The cloud large model performs in-depth inference, returns the processing results, and updates the lightweight AI large model of the edge computing module.

[0026] The vending cabinet main body is also provided with a touch display screen, which is used to display product information, interact with consumers, display recommended content generated by the AI large model, and is connected to the edge computing module to achieve real-time information update and interaction.

[0027] The working process of the interactive vending system includes: The infrared sensor senses the distance between the customer and the vending cabinet main body. When the customer approaches the vending cabinet main body by a certain distance, a greeting is initiated, and the voice information of the customer is collected through the microphone array. At the same time, the camera identifies the face, and the edge computing module determines whether it is a new customer. The collected voice information and face information are fused into multi-modal data, and combined with the product knowledge graph and user preference library to recommend products for the customer; After the customer picks up the product, product promotions are carried out according to the picked-up product.

[0028] Specifically, the interactive vending system in this embodiment can achieve active greeting, multi-round conversation, product recommendation, and product promotion. The following will be specifically described with reference to the accompanying drawings.

[0029] As Figure 2 shown, the process flow of the active greeting scenario is as follows: When a consumer approaches the vending cabinet, the infrared sensor triggers the sensing network, wakes up the interactive system of the vending cabinet, and sends out an active greeting. The high-resolution wide-angle camera then captures the consumer's facial image and quickly identifies their identity and emotional state. If it is a new customer, the vending cabinet will obtain external API environment data and in-cabinet product data, and through the touch display screen and high-sensitivity microphone array, directly recommend to the customer, such as: "Good noon! It's hot today. Do you need a bottle of iced Coke to relieve the heat?"; If it is an old customer, the large model algorithm combines the customer's historical preferences for personalized recommendation, for example: "Welcome back. Nice to see you. The new coffee you often buy has arrived. Would you like to have a taste?" As Figure 3 shown, the process flow of the multi-round conversation scenario is as follows: Consumers can communicate with the vending cabinet through voice or touch display screen. The microphone collects the voice signal and transmits it to the lightweight large model. The ASR (Automatic Speech Recognition) technology converts the text information into understandable natural language, generates a response text, and then synthesizes the voice through the TTS (Text-to-Speech) technology and plays it to the customer. The large model takes the complete conversation history between the user and the model as part of the input, and through the self-attention mechanism of Transformer, dynamically associates the context. When generating a response, the model will pay attention to the previous tokens, including the historical information in the current conversation, to achieve multi-round conversation.

[0030] The process flow of the product recommendation scenario is as Figure 4 shown, specifically including: Consumers can also interact with the vending cabinet through voice or touch display screen. For example, the consumer asks: "I want to buy some drinks. Do you have any recommendations?" The microphone array accurately collects the voice command and transmits it to the edge computing module.

[0031] After the large model performs natural language processing on voice commands and understands the needs of consumers, it combines the facial analysis results (such as age, gender, etc.), historical purchase preferences (if there are purchase records), and real-time data of in-cabinet products to perform multi-modal data fusion. It sends the result of multi-modal data fusion to a lightweight AI large model for real-time inference. If the request belongs to a complex request (that is, the lightweight AI large model cannot obtain an inference result), the data will be uploaded to the cloud large model; if the request does not belong to a complex request (that is, the lightweight AI large model can obtain an inference result), the lightweight AI large model will obtain the inference result; after filtering the inference result obtained by the lightweight AI large model or the cloud large model, it outputs a voice response and displays recommended information on the display screen. If the customer accepts the recommendation, it will be recorded in the user's historical preference library to increase the recommendation weight of related products; if the customer does not accept the recommendation, the recommendation weight of related products will be decreased.

[0032] The process of the product promotion scenario is as Figure 5 shown, specifically including: after the user picks up a product, the lightweight AI large model queries complementary products related to the product from the knowledge graph library, combines the customer's purchase records and browsing behaviors, and analyzes the user's preference degree for different types of associated products. For example, if the user often buys milk and bread to eat together, the system will give priority to recommending bread.

[0033] Combined with the user's current purchase scenario and time, such as recommending breakfast foods that match milk in the morning and ingredients suitable for dinner in the evening. And combined with the current promotion activities and inventory situation, recommend associated products participating in the promotion; record whether the user purchases the recommended associated products. If the recommendation is accepted, the feedback on this recommendation will be recorded in the user's historical preference library to increase the recommendation weight of related products; if the recommendation is not accepted, the recommendation weight of related products will be decreased.

[0034] The above embodiments are only exemplary embodiments of the present invention and are not used to limit the present invention. Those skilled in the art can make various modifications or equivalent replacements to the present invention within the essence and protection scope of the present invention, and such modifications or equivalent replacements should also be regarded as falling within the protection scope of the present invention.

Claims

1. An interactive vending system based on an AI large model, characterized in that, It includes a vending cabinet main body, an AI large model processing module, an edge computing module, and a wireless network module. Inside the vending cabinet main body, a sensor group is provided, and outside, a data collection unit is provided; The AI large model processing module is used to analyze and process the data information collected by the data collection unit and make a response; The edge computing module is built into the vending cabinet main body. The edge computing module is equipped with a pre-trained lightweight AI large model; the edge computing module is connected to the sensor group and the data collection unit of the vending cabinet main body through a communication interface, and locally processes and analyzes the data collected by the sensor group and the data collection unit; The wireless network module is used to upload complex requests that the edge computing module cannot process to the cloud large model. The cloud large model performs in-depth reasoning, returns the processing results, and updates the lightweight AI large model of the edge computing module.

2. The interactive vending system based on the AI large model according to claim 1, characterized in that, An API interface is also provided on the vending cabinet main body, and the API interface accesses external information through the wireless network module.

3. An interactive vending system based on an AI large model according to claim 1, characterized in that, The data collection unit includes a camera for capturing the facial expressions and movements of consumers, a microphone array for collecting the voice commands of consumers, and an infrared sensor for sensing the approach of consumers to the vending cabinet main body; the sensor group includes a weight sensor for real-time monitoring of changes in the weight of goods and a temperature and humidity sensor for monitoring the temperature and humidity inside the vending cabinet by users.

4. An interactive vending system based on an AI large model according to claim 1, characterized in that, The AI large model is a machine learning model with large-scale parameters and complex computing structures, including but not limited to DeepSeek-R1, OpenAI o1, Gemini-2Pro.

5. An interactive vending system based on an AI large model according to claim 1, characterized in that The specific process of loading the lightweight AI large model on the edge computing module is as follows: Using the deep learning framework PyTorch, fine-tuning with the LoRA method, using an enhanced dataset in the retail field, reducing the inference computing requirements through INT8 quantization technology, and training the AI large model into a lightweight AI large model with knowledge distillation.

6. An interactive vending system based on an AI large model according to claim 1, characterized in that, A touch display screen is also provided on the vending cabinet main body. The touch display screen is used to display product information, interact with consumers, display recommended content generated by the AI large model, and is connected to the edge computing module to achieve real-time information update and interaction.

7. An interactive vending system based on an AI large model according to claim 1, characterized in that, The working process of the interactive vending system specifically includes: The infrared sensor senses the distance between the customer and the vending cabinet main body. When the customer approaches the vending cabinet main body within a certain distance, a greeting is initiated, and the voice information of the customer is collected through the microphone array. At the same time, the camera identifies the face, and the edge computing module determines whether it is a new customer. The collected voice information and face information are fused into multi-modal data, and product recommendations are made for the customer in combination with the product knowledge graph and the user preference library; After the customer picks up the product, product promotions are carried out according to the picked-up product.

8. An interactive vending system based on an AI large model according to claim 7, characterized in that, The specific process of the commodity recommendation includes: sending the multi-modal data fusion result to the lightweight AI large model for real-time inference. If the lightweight AI large model cannot obtain the inference result, the data is uploaded to the cloud large model. After filtering the inference result obtained by the lightweight AI large model or the cloud large model, a voice response is output, and the recommendation information is displayed on the display screen. If the customer accepts the recommendation, it is recorded in the user historical preference library to increase the recommendation weight of relevant commodities; if the recommendation is not accepted, the recommendation weight of relevant commodities is decreased.

9. An interactive vending system based on an AI large model according to claim 7, characterized in that, The specific process of the commodity promotion includes: after the user picks up the commodity, the lightweight AI large model queries complementary commodities related to the commodity from the knowledge graph library, analyzes the user's preference degree for different types of associated commodities in combination with the customer's purchase records and browsing behaviors, and recommends associated commodities participating in the promotion in combination with the user's current purchase scenario, time, current promotion activities and inventory situation; records whether the user purchases the recommended associated commodity. If the recommendation is accepted, the feedback on this recommendation is recorded in the user historical preference library to increase the recommendation weight of relevant commodities; if the recommendation is not accepted, the recommendation weight of relevant commodities is decreased.

Images