Multi-dimensional data-based intelligent prediction and dynamic verbal skill adjustment system for automatic vending robot of train carriage
By integrating multi-dimensional data and using a dynamic script adjustment system, the problems of low sales efficiency and poor user experience in train carriages have been solved, resulting in high product conversion rates and improved user satisfaction, while adapting to the special environment of train carriages.
Patent Information
- Application Number
- CN202511180376.X
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2025-08-22
- Publication Date
- 2025-11-28
AI Technical Summary
Selling goods inside train carriages suffers from high labor costs, low efficiency, low product conversion rates, and poor user experience. Existing technologies struggle to integrate multi-dimensional data for accurate prediction and dynamic wording adjustments, and are unable to adapt to the unique environment of train carriages.
Employing multi-dimensional data fusion technology, the system collects passenger facial expressions, body movements, and voice data through 360-degree panoramic cameras, infrared cameras, and microphone arrays. This data is then combined with deep learning models for recognition and analysis to construct a product conversion rate prediction model and a dynamic dialogue adjustment system. This system includes multi-source data reception, distributed storage, feature engineering, prediction and analysis models, and a dynamic dialogue adjustment module, adapting to the vibration and noise environment of train carriages.
It improved product conversion rates to over 35%, reduced labor costs by 60%, and enhanced user experience ratings to 4.2, achieving stable operation and precise marketing in complex environments.
Smart Images

Figure FT_1 
Figure FT_2 
Figure FT_3
Abstract
Description
Technical Field
[0001] This invention relates to the field of intelligent retail and data analysis technology, and particularly to an intelligent prediction and dynamic sales script adjustment system for automated vending robots in the specific scenario of train carriages, based on multi-dimensional data. This system collects, analyzes, and processes multi-dimensional data such as passenger facial expressions, body movements, and voice feedback to accurately predict product conversion rates and dynamically adjusts sales scripts based on the prediction results, thereby improving the efficiency and user experience of automated vending in train carriages. This system integrates multiple technologies such as computer vision, natural language processing, and big data analysis, and is applicable to automated vending scenarios in various types of train carriages, including high-speed trains, bullet trains, and regular trains, providing technical support for the intelligent upgrading of railway passenger services. Background Technology
[0002] Currently, selling goods in train carriages mainly relies on manual methods, with train attendants pushing carts and hawking their wares. This method is not only labor-intensive but also inefficient and fails to accurately meet the needs of different passengers. Statistics show that an average ordinary train requires 3-5 full-time sales attendants, with daily labor costs exceeding 2,000 yuan. Furthermore, due to space limitations and passenger flow characteristics, the coverage rate for each sales session is less than 40%.
[0003] While some train carriages are equipped with vending machines, these machines mostly only offer basic product display and sales functions. They cannot intelligently predict passengers' purchasing intentions or dynamically adjust their sales pitches based on passenger feedback, resulting in low conversion rates and a poor user experience. The conversion rate of existing vending machines is generally below 15%, and due to the fixed sales pitches, approximately 30% of passengers reported refusing to purchase intended items because they dislike mechanical sales tactics.
[0004] In the field of smart retail, while some technologies exist for product recommendation and user behavior analysis, their application in the unique scenario of train carriages is limited. Train carriages experience high passenger turnover, limited stays (average 2-5 hours per trip), and a distinctive environment (with disturbances such as vibration, lighting changes, and noise). Passenger purchasing behavior and preferences are influenced by various factors, including train number, station, time of day, regional temperature, journey length, and passenger identity (business, tourism, student, etc.). For example, food sales on long-distance trains are 60% higher than on short-distance trains, and sales of snack foods increase by over 80% during holidays.
[0005] Existing technologies struggle to effectively integrate this multi-dimensional data for accurate prediction and analysis, and they also cannot dynamically adjust the script for interacting with passengers based on real-time conditions. Specifically, existing technologies have the following shortcomings: The data dimensions are limited: it mainly collects user purchase records and lacks the capture and analysis of real-time user interaction data (facial expressions, actions, voice).
[0006] Poor scene adaptability: The special environmental factors of train carriages (such as image blurring caused by vibration and noise interference with speech recognition) are not fully considered, resulting in low model prediction accuracy.
[0007] Rigid script adjustments: The script library is fixed and cannot be dynamically optimized according to real-time user reactions and changes in the scenario, resulting in low user acceptance.
[0008] Weak data fusion capability: It is difficult to effectively integrate external data such as train schedules and temperatures with user behavior data, resulting in poor generalization ability of the prediction model.
[0009] Therefore, there is an urgent need for an automated vending system that can adapt to the special environment of train carriages, integrate multi-dimensional data for intelligent prediction and dynamic wording adjustment, in order to solve the problems existing in the current technology. Summary of the Invention
[0010] (a) Purpose of the invention The purpose of this invention is to overcome the problems of high labor costs, low efficiency, low product conversion rate, and poor user experience in the sale of goods in train carriages in the existing technology, and to provide an intelligent prediction and dynamic script adjustment system for automated vending robots in train carriages based on multi-dimensional data.
[0011] Specific objectives include: It enables real-time collection and fusion of multi-dimensional data, covering user interaction data, historical behavior data, scene environment data, etc., thereby improving the comprehensiveness of data dimensions.
[0012] Establish a product conversion rate prediction model applicable to train carriage scenarios, with a prediction accuracy of over 85%.
[0013] Develop a dynamic script adjustment mechanism to optimize scripts in real time based on user feedback and changes in scenarios, thereby increasing product conversion rates to over 30%.
[0014] Enhance the system's adaptability to the special environment of train carriages and maintain stable operation under interference such as vibration and noise.
[0015] Labor costs are reduced; the labor cost for selling a single train carriage is reduced by more than 60%.
[0016] By achieving the above objectives, the system can accurately predict product conversion and intelligently optimize sales pitches through multi-dimensional data fusion analysis, effectively improving sales conversion rates while enhancing user interaction comfort and meeting the automated vending needs of specific scenarios such as train carriages.
[0017] (II) Technical Solution To achieve the above objectives, the present invention adopts the following technical solution: A system for intelligent prediction and dynamic script adjustment of automated vending robots in train carriages based on multi-dimensional data includes: User attention recognition module: Used to recognize passengers' facial expressions, body language, and voice feedback to determine their level of interest in products and potential purchase intentions. This module includes: The multimodal data acquisition unit consists of a 360-degree panoramic camera (1920×1080 resolution, 30fps), an infrared camera (adapted to low-light environments), and a microphone array (4 microphones, 3-meter pickup distance), used to simultaneously acquire passengers' image, motion, and voice data. The camera is mounted on the robot's top gimbal, allowing for 360-degree horizontal rotation and -30° to 60° vertical rotation, ensuring coverage of passengers within a 5-meter radius.
[0018] Data preprocessing unit: preprocesses the acquired multimodal data. For image data, it employs Gaussian filtering-based noise reduction, CLAHE algorithm-based contrast enhancement, and motion compensation-based image stabilization (for train vibration scenarios). For speech data, it uses adaptive filtering algorithms to remove environmental noise (such as train noise and passenger conversations) and extracts effective speech segments using VAD (voice activity detection) technology.
[0019] Multimodal recognition unit: A deep learning fusion model is used for data recognition. Specifically, facial expression recognition employs an improved VGG16 network, trained on a dataset containing over 100,000 facial expression samples from train scenarios, capable of recognizing eight facial expressions including smiling, frowning, surprise, and expressionlessness, with an accuracy rate of 92%. Body movement recognition uses a PoseNet model based on skeleton extraction, capable of recognizing 12 typical movements such as pointing, touching, waving, and moving away, with an accuracy rate of 90%. Voice emotion recognition uses a CNN-LSTM hybrid model to recognize six voice emotions, including "interested," "rejected," and "questioning," with an accuracy rate of 88%.
[0020] Attention Measurement Unit: This unit integrates facial expression, action, and speech recognition results, and uses a weighted voting method to calculate a passenger attention score (0-10 points). For example, smiling (+3 points) + pointing to a product (+4 points) + "How much is this?" (+2 points) = 9 points (high attention); frowning (-2 points) + moving away (-3 points) + "I don't need it" (-4 points) = 1 point (low attention). This score and the original recognition result are transmitted to the intelligent prediction and conversion analysis module via the communication module.
[0021] Intelligent Prediction and Conversion Analysis Module: This module combines interactive data, historical purchase behavior data, and information such as train schedules, stations, time periods, regional temperature, and travel itinerary from the user attention recognition module to generate product conversion rate and preference analysis reports. This module includes: Multi-source data receiving unit: Receives interactive data (attention score, facial expression / action / voice recognition results) transmitted by the user attention recognition module via the communication module; obtains historical purchase behavior data (including user ID, purchase time, product ID, quantity, payment amount, etc.) from the railway ticketing system via the data interface; obtains train information (train type, route, total duration, current location, remaining mileage, etc.) from the railway dispatching system; obtains station information (current stop, stop duration, number of passengers boarding and alighting, etc.) from the station management system; obtains time period information (accurate to the minute, including weekday / holiday markers) from the system clock module; obtains regional temperature information (current regional temperature, humidity, weather conditions, etc.) from the meteorological data interface; and extracts travel information (passenger departure station, arrival station, travel duration, etc.) from the ticketing data.
[0022] Distributed data storage unit: A distributed storage system is built using a Hadoop cluster (containing 3 data nodes). HDFS is used to store unstructured data such as raw images and audio; HBase is used to store structured data (such as purchase records and train information); and Redis is used to store real-time data (such as current popularity ratings and real-time temperature), ensuring data read / write latency is less than 100ms. Simultaneously, data encryption (AES-256 algorithm) and access control mechanisms are employed to protect passenger privacy data.
[0023] Feature Engineering Unit: This unit extracts and transforms features from the raw data. For user interaction data, it extracts features such as the average attention rating, duration of high attention, and frequency of actions. For historical purchase data, it extracts features such as user purchase frequency, preferred product categories, and average order value. For scenario data, it converts train type into discrete features (high-speed rail = 1, bullet train = 2, regular train = 3), time period into periodic features (using sine / cosine function encoding), and temperature into continuous features. A total of 128-dimensional feature vectors are extracted for model training.
[0024] Prediction and Analysis Model Unit: Includes product conversion rate prediction sub-model and user preference analysis sub-model.
[0025] The product conversion rate prediction sub-model uses the XGBoost algorithm to build the base model and incorporates an attention mechanism to focus on learning high-weight features (such as attention ratings, time periods, and temperatures). The model input is a 128-dimensional feature vector, and the output is the predicted conversion rate (0-100%) for each product. Model training uses a sliding window method (window size equal to 30 days of data), automatically updated daily to ensure model adaptability. On the test set, the model achieves a prediction accuracy of 89% and a mean absolute error (MAE) of less than 5%.
[0026] The user preference analysis sub-model uses K-means clustering to group users into six categories: business, family, student, and travel. Collaborative filtering is then used to analyze the product preferences of each group. For example, business users show a 70% preference for coffee and light meals, while student users show an 85% preference for snacks and beverages. Simultaneously, an LSTM network is used to analyze short-term user preference changes (e.g., during long-distance travel, users' food preferences increase over time).
[0027] Report Generation Unit: Based on the model output above, generate an analysis report containing the following: predicted conversion rate for each product (sorted in descending order), current passenger car user distribution and preferred products, recommended sales pitch types for high-conversion-rate products, and characteristics of users with high potential purchase intent. The report is in JSON format, updates every 30 seconds, and is pushed to the dynamic sales pitch adjustment module via the communication module.
[0028] Dynamic Sales Script Adjustment Module: Based on the analysis reports generated by the intelligent prediction and conversion analysis module, this module optimizes sales scripts to balance sales growth and user experience. This module includes: Multi-dimensional script library unit: Construct a script library containing four levels: basic scripts, scenario scripts, user segmentation scripts, and real-time adjustment scripts.
[0029] Basic sales pitch: Includes basic product information (name, price, specifications, features, etc.), such as "XX mineral water, 500ml, priced at 3 yuan, sourced from natural water sources".
[0030] Scenario-based sales pitch: Designed in conjunction with scenario factors such as train number, time of day, and temperature. For example, in a high-speed rail scenario: "This light meal uses high-speed rail cold chain technology to keep it warm and fresh, making it suitable for business travel"; during hot weather: "Iced cola, refreshing and cooling, now is a better deal to buy."
[0031] User segmentation scripts: Designed for different user groups, such as for families: "This children's snack is additive-free and suitable for children. There are parent-child discounts available now"; for travelers: "Local specialty snacks, easy to carry, and a great travel treat."
[0032] Adjust your sales pitch in real time: Design your pitch based on real-time user reactions. For example, if a user shows hesitation: "This product has sold over 100 units today, and many travelers like it." If a user inquires about the price: "Buy two now and enjoy a 10% discount, making it even more cost-effective."
[0033] The script library contains over 5,000 scripts, each tagged with applicable scenarios, target audiences, and expected conversion rate improvements for easy matching.
[0034] Script matching decision unit: adopts a hybrid decision-making mechanism of rule engine + reinforcement learning.
[0035] Rule Engine: Based on information such as product conversion rates and user groups in the analysis report, the engine initially filters the sales scripts. For example, products with a conversion rate >50% are prioritized for recommendation-based sales scripts; family-oriented users are prioritized for parent-child related sales scripts.
[0036] Reinforcement learning: The reward function is "user purchase rate + interaction satisfaction (judged by facial expressions and voice recognition)" to optimize the initially selected scripts. For example, when the interaction satisfaction of a script is below 60%, its matching weight is automatically reduced. Through continuous learning, the accuracy of script matching gradually improves, eventually reaching over 90%.
[0037] The script generation and optimization unit: For complex scenarios (such as users raising specific questions), the GPT-3.5-turbo model is used for real-time script generation. The generated scripts must meet the following requirements: consistent with current product information, suitable for the target user group, and friendly in tone (verified through a sentiment analysis model, with a friendliness score > 80). Simultaneously, the generated scripts undergo compliance checks (filtering sensitive words and ensuring accurate price descriptions) to avoid illegal content.
[0038] Multi-channel output unit: Outputs scripts through three channels: voice, text, and images.
[0039] Voice output: Employs TTS (Text-to-Speech) technology, supporting Mandarin, English, and 30+ dialects, with a naturalness score >4.5 out of 5. Speaker power automatically adjusts according to ambient noise (30-80dB) to ensure clear listening.
[0040] Text output: The script is displayed on the robot's screen (10.1-inch touchscreen), and the font size can be automatically adjusted according to the user's distance (24-36 points for the font size within 0.5-2 meters).
[0041] Image support: Use product images and promotional tags to complement the sales pitch and enhance product appeal. For example, show images of chilled beverages or discount signs during promotions.
[0042] Data storage module: Employing a distributed storage architecture, it consists of a primary storage node, backup nodes, and cache nodes. The primary storage node uses a 16TB enterprise-grade hard drive, supporting RAID5 redundancy to ensure data reliability. Backup nodes automatically perform a full backup daily at 2 AM, along with hourly incremental backups, retaining backup data for 30 days. Cache nodes use 128GB SSDs to store frequently accessed data from the last 24 hours, improving data read speed. This module supports data lifecycle management, automatically deleting non-critical data older than 180 days (such as interaction data from non-purchasing users), while retaining critical data (such as purchase records) long-term.
[0043] Communication module: Supports multi-protocol converged communication, including: 4G / 5G module: Used for communication with external systems (railway dispatching system, meteorological system, etc.), with a transmission rate of over 100Mbps to ensure real-time data updates.
[0044] Wi-Fi module (802.11ac): Used for communication between the robot and the local server inside the vehicle, transmitting large amounts of data such as images and voice.
[0045] Bluetooth module (BLE5.0): Used for close-range interaction with passengers' mobile phones (such as pushing product links), with a communication range of up to 10 meters.
[0046] Internal bus (CAN bus): Used for internal communication between various modules of the system, with a transmission delay of less than 50ms, ensuring that the modules work together.
[0047] The communication module adopts channel adaptive switching technology. When the signal of a certain communication method is weak (such as the 4G signal being below -100dBm), it automatically switches to other available methods to ensure communication continuity.
[0048] Environmental Adaptation Module: Designed for the unique environment of train carriages, this module provides the following functions: Vibration compensation: By installing a three-axis accelerometer (sampling rate 100Hz), the robot's vibration status is detected in real time, and the vibration data is fed back to the camera gimbal control unit to drive the motor for reverse compensation (compensation accuracy ±0.5°) to reduce image blur.
[0049] Light Adaptation: The camera is equipped with an automatic aperture (F1.8-F16) and an infrared fill light, which automatically adjusts according to the ambient light intensity (detected by a light sensor) to ensure clear image capture in different lighting scenarios such as daytime, nighttime, and tunnels.
[0050] Noise suppression: The microphone array uses beamforming technology for directional sound pickup (pickup angle ±30°), and combined with an adaptive noise cancellation algorithm, the suppression rate of train noise (80-100dB) reaches more than 30dB, improving the accuracy of speech recognition.
[0051] A method for intelligent prediction and dynamic script adjustment control of automated vending robots in train carriages based on multi-dimensional data includes the following steps: Step S1: Multi-dimensional data acquisition and preprocessing S1.1: The multimodal data acquisition unit of the user attention recognition module is activated. The camera acquires one frame of image every 0.5 seconds, and the microphone array continuously acquires voice data at a sampling rate of 16kHz.
[0052] S1.2: The data preprocessing unit processes the image data: first, Gaussian filtering (σ=1.0) is used to remove high-frequency noise, then the image contrast is enhanced by the CLAHE algorithm (cliplimit=2.0), and finally motion compensation is performed based on the vibration data of the accelerometer (image offset is corrected by affine transformation).
[0053] S1.3: Process the speech data: Use spectral subtraction to remove environmental noise, use VAD technology to detect valid speech segments (when the speech energy is greater than the threshold and the duration is greater than 0.5 seconds, it is considered valid), and extract Mel spectrum features.
[0054] S1.4: The multimodal recognition unit recognizes the preprocessed image and speech data and outputs the recognition results of facial expressions, actions, and speech emotions. The attention measurement unit calculates the attention score to form interactive data.
[0055] S1.5: The multi-source data receiving unit of the intelligent prediction and conversion analysis module synchronously acquires historical purchase behavior data, train information, station information, time period information, regional temperature information, travel information, etc., and transmits them to the data storage module.
[0056] Step S2: Data Analysis and Predictive Modeling S2.1: The feature engineering unit retrieves various types of data collected in step S1 from the data storage module and performs feature extraction: The interaction data extracted 15 features, including the average attention score, duration of high attention (duration > 7 points), action frequency (number of actions per minute), and voice sentiment (positive / negative / neutral).
[0057] Extract 20 features from historical purchase data, including the number of purchases made by users in the past 30 days, preferred product categories (using One-Hot coding), average order value, and distribution of purchase time periods.
[0058] The scene data is used to extract 30-dimensional features, including train type, current driving location (latitude and longitude), remaining mileage, current stop duration, time period code (sine / cosine value), temperature, humidity, and weather conditions (coded as 0-9).
[0059] Eight features, including travel duration and departure and arrival station type (first-tier city / second-tier city / other), are extracted from the travel information.
[0060] The above features are standardized (mean is 0, variance is 1), resulting in a 128-dimensional feature vector.
[0061] S2.2: The prediction and analysis model unit calls the product conversion rate prediction sub-model, inputs a 128-dimensional feature vector, and outputs the conversion rate prediction value of each product (accurate to 0.1%).
[0062] S2.3: The user preference analysis sub-model performs cluster analysis on the current carriage users to determine the distribution of user groups (e.g., business users account for 30%, students account for 25%, etc.), and outputs a list of preferred products for each group (sorted by preference).
[0063] S2.4: The report generation unit integrates the above results and generates an analysis report, including: the top 10 high-conversion products and their conversion rate values, user group distribution and preferred products, and suggestions on recommended sales script types, and pushes it to the dynamic sales script adjustment module.
[0064] Step S3: Dynamic script matching and output S3.1: The script matching decision unit of the dynamic script adjustment module receives the analysis report, and the rule engine filters the initial script pool based on the report: For the top 5 products in terms of conversion rate, we selected recommendation-type sales pitches (60%) and promotional-type sales pitches (40%).
[0065] For the preferred products of each user group, select corresponding segmented sales pitches (80%) and scenario-based sales pitches (20%).
[0066] S3.2: The reinforcement learning module optimizes the initial dialogue pool, calculates the expected reward value of each dialogue (based on historical interaction data), and selects the 3 dialogues with the highest reward value as candidate dialogues.
[0067] S3.3: The script generation and optimization unit checks the suitability of candidate scripts. If there are special user questions (such as "Is there a low-sugar option?"), it calls the GPT-3.5-turbo model to generate targeted scripts and performs compliance checks (sensitive word filtering, price accuracy verification).
[0068] S3.4: The multi-channel output unit outputs the final script simultaneously through voice, text, and images: the voice playback duration is controlled between 5 and 15 seconds, the text font size is automatically adjusted according to the user's distance, and the image display shows product pictures that match the script content.
[0069] Step S4: Real-time Feedback and Model Update S4.1: The user attention recognition module continuously collects user reaction data (facial expressions, actions, and voice) to the output script. The attention measurement unit updates the attention score and transmits it to the intelligent prediction and conversion analysis module.
[0070] S4.2: The intelligent prediction and conversion analysis module records the user's final purchase behavior (purchase / not purchase), compares the actual conversion rate with the predicted conversion rate, and calculates the error value.
[0071] S4.3: If the error value is greater than 10%, the incremental update of the product conversion rate prediction sub-model is triggered. The model is fine-tuned using the latest 100 interaction data points (learning rate 0.001), and the model parameters are updated.
[0072] S4.4: The dynamic script adjustment module records the interaction effects of each script (purchase rate, satisfaction rating), and the reinforcement learning module updates the reward value of the script based on the effect and adjusts the script matching weight.
[0073] S4.5: Update the user preference analysis sub-model every hour to incorporate the latest user interaction data and ensure the accuracy of user segmentation and preference analysis.
[0074] Step S5: Environmental Adaptation and System Monitoring S5.1: The vibration compensation unit of the environmental adaptation module receives acceleration sensor data in real time. When the vibration amplitude is greater than 0.5g, it drives the gimbal motor to perform reverse compensation and correct the camera angle.
[0075] S5.2: The light adaptation unit adjusts the camera parameters according to the light sensor data (0-10000 lux): when the light is >5000 lux, the aperture is adjusted to F8-F16 and the fill light is turned off; when the light is <1000 lux, the aperture is adjusted to F1.8-F4 and the infrared fill light is turned on.
[0076] S5.3: The system monitoring unit monitors the operating status of each module in real time (CPU utilization, memory usage, communication quality, etc.). When an abnormality occurs (such as CPU utilization > 90% for 10 seconds), an alarm is triggered and load balancing is automatically performed (such as transferring some data analysis tasks to a backup node).
[0077] S5.4: Every day at 3:00 AM, the system automatically performs a full data backup and log cleanup to ensure stable system operation.
[0078] (III) Beneficial Effects Compared with the prior art, the present invention has the following beneficial effects: Multi-dimensional data fusion improves prediction accuracy: By integrating user facial expressions, actions, voice and other interactive data, combined with scenario data such as train schedules and temperature, the product conversion rate prediction model has an accuracy of 89%, which is a significant improvement over existing technologies (accuracy <70%), providing data support for precision marketing.
[0079] Dynamic script adjustments improve product conversion rates: By employing a multi-level script library and reinforcement learning matching mechanism, the scripts are dynamically optimized based on real-time user responses and changes in the scenario, increasing the product conversion rate to over 35%, which is more than double that of existing automated vending machines (<15%).
[0080] Strong environmental adaptability: Designed for special environments such as train vibration, light changes, and noise, it features vibration compensation, light adaptation, and noise suppression functions to ensure stable operation of the system in complex environments, with data acquisition and recognition accuracy maintained at around 90%.
[0081] Significantly improved user experience: Through personalized sales script recommendations and multi-channel interaction, users' satisfaction rating for sales interaction reached 4.2 out of 5, a 50% improvement compared to traditional mechanical scripts (2.8 out of 5), reducing user aversion.
[0082] Significantly reduce operating costs: Equipping one automated vending robot per train carriage can replace the sales work of 3-5 attendants, saving approximately 200,000 yuan in labor costs annually. At the same time, through accurate forecasting and inventory optimization, the rate of unsold goods is reduced by 40%, further reducing operating costs.
[0083] Data security and privacy protection: We employ data encryption, access control, and lifecycle management mechanisms to ensure that passenger privacy data is not leaked, in compliance with the Personal Information Protection Law and other relevant regulations.
[0084] Highly scalable: The system adopts a modular design and can be expanded to include new data sources (such as passenger health data for recommending healthy foods) or new types of sales pitches according to actual needs, adapting to the future development needs of smart retail. Attached Figure Description
[0085] The present invention will be further described below with reference to the accompanying drawings: Figure 1 This is a flowchart illustrating the overall system architecture of the present invention. Figure 2 This is a flowchart of the intelligent prediction and transformation analysis process of the present invention; Figure 3 This is a flowchart of the dynamic script adjustment process of the present invention. Detailed Implementation
[0086] The present invention will now be described in further detail with reference to the accompanying drawings and specific embodiments.
[0087] Example 1: Scene from the business class carriage of high-speed train G100 (Beijing-Shanghai) System Deployment The system was deployed in the business class carriage (1 car) of the G100 high-speed train. The automated vending robot was placed in the middle of the carriage, and the camera pan-tilt unit covered the entire carriage (20 seats). The communication module was connected to the railway dispatching system and the meteorological system via the 4G network. The data storage module stored nearly 7 days of data locally and synchronized it to the cloud server daily.
[0088] Data acquisition and processing (corresponding to step S1) The multimodal data acquisition unit was activated at 8:00 AM. The camera captured images every 0.5 seconds, and the microphone array captured audio. At this time, the interior of the carriage was well-lit (5000 lux), the camera aperture was adjusted to F5.6, and the supplementary lighting was turned off; the ambient noise was 65 dB, and the microphone array was activated with beamforming for directional sound pickup.
[0089] The data preprocessing unit performs Gaussian filtering and CLAHE enhancement on the acquired images. Because the high-speed train runs smoothly (vibration < 0.1g), vibration compensation is not enabled. After spectral subtraction processing, effective voice segments of three passengers (such as "Do you have coffee?") are extracted from the voice data.
[0090] The multimodal recognition unit identified the following: Passenger A (window seat) smiled (85% confidence), leaned slightly forward (pointing to the coffee area), and asked "How much is the coffee?" (neutral emotion), with an attention score of 8; Passenger B showed no expression, looked down at his phone, and did not speak, with an attention score of 2; Passenger C frowned, shook his head, and said "I don't need it," with an attention score of 1.
[0091] The multi-source data receiving unit obtained the following: historical purchase data showed that the coffee purchase rate in the business class carriage reached 45% during this period; the train number was G100 (high-speed rail), currently traveling near Jinan West Station, with a remaining distance of 500 kilometers; the time period was 8:00 AM on Wednesday (weekday); the regional temperature was 25℃; and the passenger itinerary information showed that the travel time for passengers A and C was 5 hours (Beijing-Shanghai).
[0092] Data analysis and prediction (corresponding to step S2) The feature engineering unit extracts feature vectors, among which the high-weight features related to coffee include: attention score of 8 points (weight 0.25), business class (weight 0.2), morning time (weight 0.15), and journey duration of 5 hours (weight 0.1).
[0093] Product conversion rate prediction sub-model output: Coffee conversion rate prediction value 52%, mineral water 30%, light meals 40%.
[0094] Clustering results of the user preference analysis sub-model: Business users account for 70%, and their preferred products are coffee (75%) and light meals (60%).
[0095] The report generation unit generates a report recommending that coffee be prioritized for promotion, using business-oriented user language.
[0096] Dynamic script matching and output (corresponding to step S3) The script matching decision unit selects an initial script pool from the script library: "This Blue Mountain coffee is made with freshly ground coffee and is suitable for business refreshment" (business script), "Special offer on coffee in the morning, buy one get one free" (promotional script).
[0097] The reinforcement learning module calculates reward values. The historical purchase rate of "business-style sales pitch" is 48%, which is higher than that of "promotional sales pitch" at 35%, so the former is selected.
[0098] Multi-channel output: Voice playback "Sir, this Blue Mountain coffee is made with freshly ground coffee and is suitable for business refreshment. It is priced at 38 yuan" (naturalness rating 4.6). The display screen shows coffee pictures and prices, with a font size of 32 (user distance 1.5 meters).
[0099] Real-time feedback and model updates (corresponding to step S4) After hearing the message, passenger A nodded (action recognition) and said "a cup, please" (voice recognition), raising the attention score to 9 points, and ultimately purchased the coffee.
[0100] The intelligent prediction and conversion analysis module recorded an actual conversion rate of 100%, which differed from the predicted value of 52% by 48%, triggering an incremental update of the model and adding "the response weight of business users to coffee-related conversations during the morning session".
[0101] The dynamic script adjustment module records that the purchase rate of the script is 100%, updates its reward value, and increases the weight of subsequent matching.
[0102] Example 2: Nighttime scene of ordinary train K101 (long-distance) System Deployment The system was deployed in the hard-seat carriages of the K101 train (Beijing-Guangzhou, 24-hour journey). The robot was located at the carriage connection, and the camera covered 10 seats around it. The communication module prioritized the use of the 4G network, and automatically switched to Bluetooth to communicate with the local server in tunnel sections.
[0103] Data Acquisition and Processing At 11:00 PM, the lighting inside the carriage was relatively dim (500 lux), so the camera turned on its infrared fill light and the aperture was adjusted to F2.0; the ambient noise was 75 dB (including the sound of the train and passengers snoring), and the microphone array was set to strong noise reduction mode.
[0104] Passenger D (middle-aged male) was identified as yawning (facial expression), rubbing his eyes (action), and not speaking, with an attention score of 3 points; Passenger E (female with a child) was identified as paying attention to the snack area (action) and quietly asking the child "what do you want to eat?" (speech), with an attention score of 6 points.
[0105] Data obtained: Historically, the purchase rate of instant noodles and snacks on long-distance trains at night reached 60%; the current train is an ordinary train, traveling through Hunan Province, with 800 kilometers remaining; the time is 11 pm; the temperature is 28℃; and the travel time for all passengers is over 20 hours.
[0106] Data Analysis and Forecasting Feature engineering extracted high-weight features: nighttime (0.2), long-distance trip (0.18), and traveling with children (0.15).
[0107] Conversion rate prediction: instant noodles 58%, potato chips 62%, mineral water 40%.
[0108] User preference analysis: Family users account for 30%, and they prefer snacks and convenience foods.
[0109] Dynamic script matching and output Screening script: "These kids' potato chips are additive-free and suitable for children. Buy two get one free now" (family-oriented + promotional script).
[0110] Output: Voice playback (volume 60dB, to avoid disturbing other passengers), text font size 24 on the display screen (user distance 2 meters), and simultaneous display of potato chip packaging images.
[0111] Feedback and Updates Passenger E purchased 2 bags of potato chips, with an actual conversion rate of 100%. After recording the data, the model was fine-tuned to increase the weight of the potato chip conversion rate for the "nighttime + long-distance + family" feature combination.
[0112] As can be seen from the above embodiments, the present invention can adapt to the needs of different types of train carriages, and effectively improve product conversion rate and user experience through multi-dimensional data fusion and dynamic wording adjustment, thus having significant practical value.
Claims
1. A system for intelligent prediction and dynamic script adjustment of automated vending robots in train carriages based on multi-dimensional data, characterized in that: include: The user attention recognition module is used to identify passengers' facial expressions, body movements, and voice feedback in order to determine the degree of attention passengers pay to products and their potential purchase intentions. The intelligent prediction and conversion analysis module is connected to the user attention recognition module and is used to generate a product conversion rate and preference analysis report by combining the interactive data, historical purchase behavior data, train information, station information, time period information, regional temperature information, and travel information transmitted by the user attention recognition module. The dynamic sales script adjustment module is connected in communication with the intelligent prediction and conversion analysis module, and is used to optimize the sales script based on the analysis report generated by the intelligent prediction and conversion analysis module. The data storage module is connected to the user attention recognition module, the intelligent prediction and conversion analysis module, and the dynamic script adjustment module, respectively, and is used to store various types of data generated during system operation. The communication module is connected to the user attention recognition module, the intelligent prediction and conversion analysis module, the dynamic script adjustment module, and the data storage module, respectively, and is used to realize data transmission between the modules and between the system and external devices or systems. An environmental adaptation module, connected to the user attention recognition module, is used to provide environmental compensation for the user attention recognition module.
2. The system according to claim 1, characterized in that, The user attention recognition module includes: The multimodal data acquisition unit consists of a 360-degree panoramic camera, an infrared camera, and a microphone array, used to simultaneously acquire passengers' image, motion, and voice data; The data preprocessing unit is connected to the multimodal data acquisition unit and is used to preprocess the acquired multimodal data, including noise reduction, enhancement, and anti-shake processing of image data, and noise reduction and extraction of effective speech segments of speech data. A multimodal recognition unit, connected to the data preprocessing unit, uses a deep learning fusion model for data recognition, including facial expression recognition, body movement recognition, and voice emotion recognition. The attention measurement unit, connected to the multimodal recognition unit, is used to fuse the results of facial expression, action, and speech recognition to calculate the passenger attention score.
3. The system according to claim 2, characterized in that, In the multimodal recognition unit, the facial expression recognition uses an improved VGG16 network, which can recognize 8 facial expressions with an accuracy of 92%; the body movement recognition uses a PoseNet model based on skeleton extraction, which can recognize 12 typical movements with an accuracy of 90%; and the voice emotion recognition uses a CNN-LSTM hybrid model to recognize 6 voice emotions with an accuracy of 88%.
4. The system according to claim 1, characterized in that, The intelligent prediction and transformation analysis module includes: The multi-source data receiving unit is used to receive interactive data, historical purchase behavior data, train information, station information, time period information, regional temperature information, and travel information transmitted by the user attention recognition module; Distributed data storage unit, which uses a Hadoop cluster to build a distributed storage system for storing received data; The feature engineering unit, connected to the distributed data storage unit, is used to extract and transform features from the original data to generate a 128-dimensional feature vector. The prediction and analysis model unit, connected to the feature engineering unit, includes a product conversion rate prediction sub-model and a user preference analysis sub-model. The product conversion rate prediction sub-model is constructed using the XGBoost algorithm, and the user preference analysis model uses the K-means clustering algorithm and the collaborative filtering algorithm. The report generation unit, connected to the prediction and analysis model unit, is used to generate an analysis report based on the model output.
5. The system according to claim 4, characterized in that, The product conversion rate prediction sub-model achieved a prediction accuracy of 89% on the test set, with a MAE of less than 5%; the user preference analysis sub-model divides users into 6 groups and can analyze the product preferences of each group.
6. The system according to claim 1, characterized in that, The dynamic script adjustment module includes: The multi-dimensional script library unit is constructed to include four levels of scripts: basic scripts, scenario scripts, user segmentation scripts, and real-time adjustment scripts, totaling more than 5,000 scripts. The script matching decision unit uses a hybrid decision mechanism of rule engine + reinforcement learning to match appropriate scripts from the script library; The script generation and optimization unit is used to generate real-time scripts for complex scenarios and perform compliance checks. The multi-channel output unit is used to output scripts through three channels: voice, text, and images.
7. The system according to claim 6, characterized in that, In the multi-channel output unit, the voice output adopts TTS technology, supports Mandarin, English and 30+ dialects, and the naturalness score of the voice is >4.5; the text output is displayed on the robot's display screen, and the font size can be automatically adjusted according to the user's distance; the image output displays product pictures, discount signs, etc. in conjunction with the speech.
8. The system according to claim 1, characterized in that, The environmental adaptation module includes: The vibration compensation unit, by installing a three-axis accelerometer, detects the robot's vibration status in real time and drives the motor to perform reverse compensation; A light adaptation unit is used to automatically adjust camera parameters according to the intensity of ambient light. The noise suppression unit employs beamforming technology and an adaptive noise cancellation algorithm to improve speech recognition accuracy.
9. A method for intelligent prediction and dynamic script adjustment control of an automated vending robot in a train carriage based on multi-dimensional data, characterized in that, Includes the following steps: Step S1: Multi-dimensional data collection and preprocessing. The user attention recognition module collects and preprocesses passenger image, action and voice data, and the intelligent prediction and conversion analysis module obtains other relevant data. Step S2: Data analysis and predictive modeling, extracting features from the collected data, and calling the predictive and analysis model to generate a product conversion rate and preference analysis report; Step S3: Dynamic script matching and output, matching suitable scripts from the script library and outputting them through multiple channels; Step S4: Real-time feedback and model update, update model parameters and speech matching weights based on user feedback; Step S5: Environmental adaptation and system monitoring to ensure stable system operation in complex environments.
10. The method according to claim 9, characterized in that, In step S1, the image data is preprocessed using Gaussian filtering, CLAHE algorithm, and motion compensation; the speech data is preprocessed using spectral subtraction and VAD technology; in step S2, the extracted features include a total of 128-dimensional feature vectors extracted from interaction data, historical purchase data, scene data, and itinerary information.
