Intelligent shopping cart system based on deep reinforcement learning and implementation method thereof

Abstract

The invention discloses an intelligent shopping cart system based on deep reinforcement learning and an implementation method thereof. The method comprises the following steps: step 1, system deployment; step 2, data acquisition and preprocessing; step 3, commodity detection and classification; step 4, commodity tracking and state updating; step 5, attitude estimation and state judgment; step 6, performing anomaly detection and safety alarm; step 7, user interaction and recommendation; 8, settlement and payment are carried out; step 9, data synchronization and model updating; according to the invention, high-precision classification, tracking and attitude estimation of commodities are realized through multi-modal data acquisition and fusion in combination with an improved deep learning algorithm; deep reinforcement learning, self-supervised learning, meta-learning and federated learning technologies are introduced, so that the robustness, adaptability and real-time performance of the model are improved; a differential privacy and secure multi-party computing technology is adopted, so that the security and privacy of user data are protected; the operation cost is effectively reduced, the shopping efficiency is improved, and the user experience is improved.

Description

Technical Field

[0001] The present invention relates to the field of computer vision technology, and in particular to an intelligent shopping cart system based on deep reinforcement learning and an implementation method thereof. Background Art

[0002] With the digital transformation of the retail industry and the improvement of consumers' shopping experience, the research and application of smart shopping systems have become a hot topic. Existing smart shopping cart systems usually rely on RFID tags, weight sensors or other hardware devices to achieve product identification and settlement. This type of system has the following problems: First, it requires a large amount of hardware investment, such as RFID tags, readers, weight sensors, etc., which increases the operating costs of merchants; second, the system deployment requires the transformation of existing facilities, which is difficult to maintain and affects normal operations; third, it requires users to perform additional operations, such as scanning products and labeling, which increases the shopping burden. Some smart shopping cart systems use pure visual algorithms to achieve product recognition, but the existing visual algorithms have problems such as low recognition accuracy, poor real-time performance, and insufficient adaptability to new products in complex retail environments, so they cannot meet the needs of commercial applications. Summary of the invention

[0003] The purpose of the present invention is to provide an intelligent shopping cart system based on deep reinforcement learning and an implementation method thereof to solve the problems raised in the above background technology.

[0004] To achieve the above-mentioned purpose, the present invention provides the following technical solutions: an intelligent shopping cart system based on deep reinforcement learning, comprising a multimodal data acquisition module, a data preprocessing module, a multimodal fusion perception module, a product tracking module, a posture estimation algorithm module, an anomaly detection module, a learning module, an interaction module, a data security module and a server, wherein the data preprocessing module establishes data connections with the multimodal data acquisition module and the multimodal fusion perception module respectively, the multimodal fusion perception module establishes data connections with the product tracking module and the posture estimation algorithm module respectively, the anomaly detection module establishes data connections with the multimodal fusion perception module, the product tracking module and the posture estimation algorithm module respectively, the server data is connected with the interaction module and the data security module, and the learning module establishes data connections with the multimodal fusion perception module, the product tracking module, the posture estimation algorithm module and the anomaly detection module respectively.

[0005] Preferably, the data preprocessing module, multimodal fusion perception module, commodity tracking module, posture estimation algorithm module, anomaly detection module and learning module are all arranged on the interaction module.

[0006] Preferably, the multimodal data acquisition module includes a camera array, a depth sensor and a microphone array, the camera array uses a Sony IMX577 camera, and the depth sensor uses an Intel RealSense D435 depth camera.

[0007] Preferably, the multimodal fusion perception module includes a feature extraction unit, a feature fusion unit and a commodity detection and classification unit, the interaction module includes a touch screen display, a voice interaction device, a computing unit, a power supply unit and a communication unit, and the computing unit adopts an NVIDIA Jetson AGX Orin embedded computing device.

[0008] The implementation method of the smart shopping cart system based on deep reinforcement learning includes step one, system deployment; step two, data collection and preprocessing; step three, commodity detection and classification; step four, commodity tracking and status update; step five, posture estimation and state judgment; step six, anomaly detection and security alarm;

[0009] Step 7: User interaction and recommendation; Step 8: Settlement and payment; Step 9: Data synchronization and model update;

[0010] In the above step 1, a multimodal data acquisition module and an interaction module are deployed on the shopping cart, a data preprocessing module, a multimodal fusion perception module, a product tracking module, a posture estimation algorithm module, an anomaly detection module and a learning module are deployed on the interaction module, a data connection is established between the data preprocessing module and the multimodal data acquisition module, a data connection is established between the interaction module and the server, a data security module is constructed, and the data security module is used to perform data security protection in the data acquisition stage, the data transmission stage, the model training and aggregation stage, and the data storage stage;

[0011] In the above step 2, the multimodal data acquisition module is used to collect visual data, depth data and voice data, and transmit them to the data preprocessing module, which synchronizes, aligns and preprocesses the data to generate high-quality input data; synchronization and alignment are specifically: using timestamp t for data synchronization and using spatial variation matrix T for spatial alignment, the formula is as follows: D aligned =T·D raw ; The preprocessing is specifically: denoising, lighting correction and color correction of the visual data to obtain the preprocessed visual image I v , denoise, filter and convert the depth map to point cloud to obtain the preprocessed point cloud data I d , perform noise elimination and speech enhancement on the speech data to obtain the preprocessed speech signal I s ;

[0012] In the above step 3, the multimodal fusion perception module receives high-quality input data generated by the data preprocessing module, detects and classifies the goods, outputs the category, location and confidence of the goods, and is displayed by the interaction module;

[0013] In the above step 4, the product tracking module updates the trajectory and status information of the product according to the detection result of the multimodal fusion perception module;

[0014] In the above step 5, the posture estimation algorithm module calculates the 3D posture of the product according to the detection result of the multimodal fusion perception module, determines whether the product is put into the shopping cart, and updates the shopping list according to the judgment result;

[0015] In the above step 6, the anomaly detection module identifies potential abnormal behaviors based on the detection results of the multimodal fusion perception module, the product tracking module and the posture estimation algorithm module, and triggers the interaction module to issue a security alarm if abnormal behaviors are identified;

[0016] In the above step 7, the user views the shopping list, product information and personalized recommendations through the interactive module, specifically:

[0017] 1) User input: Users interact with the system through the interaction module;

[0018] 2) User feature extraction: The system collects the user’s shopping history, preferences, and behavior data to generate a user embedding vector e u ;

[0019] e u =f user (u)

[0020] 3) Recommendation algorithm generates recommendation results:

[0021] Product embedding vector calculation:

[0022] e i =f item (i)

[0023] Preference matching and scoring:

[0024]

[0025] Among them, W is the weight matrix and b is the bias term;

[0026] 4) Recommendation result display: The interactive module displays the recommended product list;

[0027] 5) User feedback: The system records user clicks, purchases, and reviews, updates user models, and optimizes recommendation algorithms;

[0028] In the above step 8, after the user completes shopping, he / she selects the self-service checkout and payment method through the interactive module;

[0029] In the above step nine, the shopping data and user behavior data are synchronized to the server through the interactive module to participate in the model update of the learning module.

[0030] Preferably, in step one, the data preprocessing module uses the ROS framework to manage the data acquisition and synchronization of multiple sensors, and the data preprocessing module uses CUDA accelerated image and point cloud processing algorithms to implement real-time preprocessing operations; the data security module uses differential privacy processing to protect data security in the data acquisition stage, uses the SSL / TLS protocol to encrypt data communication in the data transmission stage to prevent eavesdropping and data modification, performs encrypted calculations in the model training and aggregation stages to prevent data leakage, encrypts the stored data in the data storage stage, sets access permissions to prevent unauthorized access; differential privacy processing is specifically: in the process of data acquisition and model training, noise is added to protect user privacy, and the formula is:

[0031]

[0032] Among them, Δf is the global sensitivity and ε is the privacy budget.

[0033] Preferably, in the step 3, the multimodal fusion perception module performs classification based on Fusion CNN and GNN. The specific process of the multimodal fusion perception module for commodity detection and classification is as follows: the feature extraction unit performs multimodal feature extraction, the feature fusion unit uses the attention mechanism to fuse the extracted multimodal features, and the commodity detection and classification unit uses GNN to detect and classify commodities; in the multimodal feature extraction, the visual feature extraction adopts a convolutional neural network based on ResNet50, and the preprocessed visual image I is input. v , output visual feature vector F v ; Deep feature extraction based on PointNet++ point cloud processing network, input preprocessed point cloud data I d , output the deep feature vector F d ; Speech feature extraction uses a Bi-LSTM-based recurrent neural network, inputting the preprocessed speech signal I s , output speech feature vector F s ; Multimodal feature extraction is specifically as follows: for the visual feature vector F v , deep feature vector F d and speech feature vector F s , extracted through the corresponding feature extraction network:

[0034] F v =CNN vision (Iv );F d =CNN vision (I d );F s =CNN vision (I s );

[0035] The specific feature fusion is:

[0036] F fused =α v F v +α d F d +α s F s

[0037] Among them, α v +α d +α s =1, which is a learnable weight parameter;

[0038] The use of graph neural network classifiers for commodity detection and classification is as follows: construct a graph structure of commodity features, where nodes represent candidate commodity areas and edges represent the association between commodities. GNN is used to classify and predict the attributes of commodities. The formula is as follows:

[0039] H (l+1) =σ(AH (l) W (l) )

[0040] Among them, H (l) is the node feature of the lth layer, A is the adjacency matrix, W (l) is the weight matrix.

[0041] Preferably, in step 4, the commodity tracking module adopts a deep reinforcement learning algorithm based on the Actor-Critic architecture and a real-time decision-making tracking strategy. The environment simulation of the commodity tracking module adopts the OpenAI Gym framework to build a simulation environment for commodity tracking. The intelligent agent of the commodity tracking module includes a policy network and a value network, and both the policy network and the value network adopt MLP. The interaction process between the intelligent agent and the environment is as follows: the intelligent agent selects action a according to the strategy π t , the environment feedbacks the new state s t+1 and reward r t ; The learning update process is as follows: use the collected experience to calculate the advantage function A t , update the parameters θ of the policy network and the value network π and θ V ;

[0042] In the policy network, input the current state s t , output action at The probability distribution of is:

[0043] π(a t |s t θ π )=softmax(f(s t θ π ))

[0044] In the value network, input the current state s t , output estimated state value function V(s t θ V )for:

[0045] V(s t θ V )=f(s t θ V )

[0046] The loss function is:

[0047] L(θ π ,θ V )=Ε π [(r t +γV(s t+1 )-V(s t )) 2 -logπ(a t s t θ π )A t ]

[0048] Among them, A t is the advantage function, r t is the immediate reward, and γ is the discount factor.

[0049] Preferably, in step 5, the posture estimation algorithm module adopts self-supervised learning technology to learn the 3D posture of the product from unlabeled data, uses a Transformer-based posture estimation network to process sequence data, uses an unlabeled large-scale data set for pre-training, and uses the reprojection consistency loss to optimize the model; the algorithm training process of the posture estimation algorithm module is specifically as follows:

[0050] 1) Input image pairs: Get image pairs of adjacent frames from the unlabeled dataset. t and I t+1 ;

[0051] 2) Posture estimation network: Input I t and I t+1 , output relative posture transformation parameter T t→t+1 ;

[0052] 3) Depth estimation network: Input It , output depth map D t ;

[0053] 4) Disparity calculation and reprojection: using depth map D t and relative attitude transformation parameter T t→t+1 , will I t Reproject to I t+1 The reconstructed image is obtained from the viewing angle

[0054] 5) Calculate the reprojection consistency loss:

[0055]

[0056] 6) Model update: back propagation, update network parameters.

[0057] Preferably, in step nine, the learning module adopts a model-independent meta-learning algorithm and a federated average algorithm, and the federated learning adopts the TensorFlowFederated framework; the update process of the learning module is:

[0058] 1) Local task training: Each client performs task training on local data, using a model-independent meta-learning algorithm to obtain updated model parameters θ i ′;

[0059] 2) Parameter encryption upload: Encrypt the model parameters and upload them to the server;

[0060] 3) Parameter aggregation: The server receives the encrypted model parameters and aggregates them using the federated averaging algorithm to obtain the global model parameters θ;

[0061] 4) Model distribution: The server distributes the updated global model parameters θ to each client;

[0062] 5) Iteration: The client uses the new model parameters to perform the next round of local training;

[0063] The client is the interactive module on the shopping cart; the model-independent meta-learning algorithm is specifically:

[0064] 1) Meta-training process: For task A i , calculate the gradient:

[0065]

[0066] 2) Meta Update:

[0067]

[0068] Among them, α and β are learning rates;

[0069] Federated learning is specifically:

[0070] 1) Local update: Client k is trained on local data to obtain model parameters ω k ;

[0071] 2) Global aggregation: The server aggregates the model parameters of each client:

[0072]

[0073] Among them, n k is the data volume of client k,

[0074] Compared with the prior art, the beneficial effects of the present invention are as follows: the present invention realizes high-precision classification, tracking and posture estimation of goods through multimodal data collection and fusion, combined with an improved deep learning algorithm; introduces deep reinforcement learning, self-supervised learning, meta-learning and federated learning technologies to improve the robustness, adaptability and real-time performance of the model; adopts differential privacy and secure multi-party computing technologies to protect the security and privacy of user data; effectively reduces operating costs, improves shopping efficiency and enhances user experience. BRIEF DESCRIPTION OF THE DRAWINGS

[0075] Figure 1 is a system structure block diagram of the present invention;

[0076] Figure 2 It is a structural block diagram of the multimodal data acquisition module of the present invention;

[0077] Figure 3 This is a structural block diagram of the multimodal fusion perception module of the present invention;

[0078] Figure 4 It is a structural block diagram of the interaction module of the present invention;

[0079] Figure 5 is a flow chart of the method of the present invention;

[0080] Figure 6 It is a data collection and preprocessing flow chart of the present invention;

[0081] Figure 7 It is a flow chart of commodity detection and classification of the present invention;

[0082] Figure 8 This is a schematic diagram of the algorithm of the commodity tracking module of the present invention;

[0083] Fig. 9 It is a flow chart of posture estimation algorithm training of the present invention;

[0084] Fig.10 Update the flow chart for the learning module of the present invention;

[0085] Fig.11 It is the workflow diagram of the recommendation algorithm of the present invention;

[0086] Fig.12 It is a schematic diagram of the data security module protection mechanism of the present invention.

[0087] Data collection and preprocessing; Step three, product detection and classification; Step four, product tracking and status update; Step five, posture estimation and state judgment; Step six, anomaly detection and security alert; Step seven, user interaction and recommendation; Step eight, settlement and payment; Step nine, data synchronization and model update.

[0088] In the figure: 1. Multimodal data acquisition module; 11. Camera array; 12. Depth sensor; 13. Microphone array; 2. Data preprocessing module; 3. Multimodal fusion perception module; 31. Feature extraction unit; 32. Feature fusion unit; 33. Product detection and classification unit; 4. Product tracking module; 5. Posture estimation algorithm module; 6. Anomaly detection module; 7. Learning module; 8. Interaction module; 81. Touch screen display; 82. Voice interaction device; 83. Computing unit; 84. Power supply unit; 85. Communication unit; 9. Data security module; 10. Server. DETAILED DESCRIPTION

[0089] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0090] Please see attached Figure 1 -Attached Figure 4, an embodiment provided by the present invention: an intelligent shopping cart system based on deep reinforcement learning, comprising a multimodal data acquisition module 1, a data preprocessing module 2, a multimodal fusion perception module 3, a commodity tracking module 4, a posture estimation algorithm module 5, an anomaly detection module 6, a learning module 7, an interaction module 8, a data security module 9 and a server 10, the data preprocessing module 2 respectively establishes data connections with the multimodal data acquisition module 1 and the multimodal fusion perception module 3, the multimodal fusion perception module 3 respectively establishes data connections with the commodity tracking module 4 and the posture estimation algorithm module 5, the anomaly detection module 6 respectively establishes data connections with the multimodal fusion perception module 3, the commodity tracking module 4 and the posture estimation algorithm module 5, the server 10 is data-connected with the interaction module 8 and the data security module 9, the learning module 7 respectively establishes data connections with the multimodal fusion perception module 3, the commodity tracking module 4, the posture estimation algorithm module 5, The calculation method module 5 and the anomaly detection module 6 establish a data connection, the multimodal data acquisition module 1 is used to collect data, the data preprocessing module 2 is used for data preprocessing, the multimodal fusion perception module 3 is used for commodity detection and classification, the commodity tracking module 4 is used to update the trajectory and status information of the commodity, the posture estimation algorithm module 5 is used to calculate the 3D posture of the commodity, the anomaly detection module 6 is used to detect potential abnormal behavior, the learning module 7 is used to update the model, the interaction module 8 is used for human-computer interaction, and the data security module 9 is used to ensure data security and user privacy; the data preprocessing module 2, the multimodal fusion perception module 3, the commodity tracking module 4, the posture estimation algorithm module 5, the anomaly detection module 6 and the learning module 7 are all arranged on the interaction module 8; the multimodal data acquisition module 1 includes a camera array 11, a depth sensor 12 and a microphone array 13, the camera array 11 adopts a Sony IMX577 camera, the depth sensor 12 adopts an Intel RealSense D435 depth camera, and the multimodal data acquisition module 1 uses the camera array 11 to collect visual data, uses the depth sensor 12 to collect depth data, and uses the microphone array 13 to collect voice data;The multimodal fusion perception module 3 includes a feature extraction unit 31, a feature fusion unit 32 and a commodity detection and classification unit 33. The interaction module 8 includes a touch screen display 81, a voice interaction device 82, a computing unit 83, a power supply unit 84 and a communication unit 85. The computing unit 83 uses an NVIDIA Jetson AGX Orin embedded computing device. The multimodal fusion perception module 3 uses the feature extraction unit 31 to extract multimodal features, uses the feature fusion unit 32 to fuse the extracted multimodal features, inputs the fused features into the commodity detection and classification unit 33, and outputs the commodity classification results. The touch screen display 81 is used for data visualization and command input, the voice interaction device 82 is used for voice interaction, the computing unit 83 is used for data processing, the power supply unit 84 is used to power the interaction module 8 and the multimodal data acquisition module 1, and the communication unit 85 is used to establish a data connection with the multimodal data acquisition module 1 and the server 10. ;

[0091] See also Figure 5-12 , an embodiment provided by the present invention: a method for implementing an intelligent shopping cart system based on deep reinforcement learning, comprising step one, system deployment; step two, data collection and preprocessing; step three, commodity detection and classification; step four, commodity tracking and status update; step five, posture estimation and state judgment; step six, anomaly detection and security alarm; step seven, user interaction and recommendation;

[0092] Step 8: Settlement and payment; Step 9: Data synchronization and model update;

[0093] In the above step 1, a multimodal data acquisition module 1 and an interaction module 8 are deployed on the shopping cart, a data preprocessing module 2, a multimodal fusion perception module 3, a product tracking module 4, a posture estimation algorithm module 5, an anomaly detection module 6 and a learning module 7 are deployed on the interaction module 8, a data connection is established between the data preprocessing module 2 and the multimodal data acquisition module 1, a data connection is established between the interaction module 8 and the server 10, a data security module 9 is constructed, and the data security module 9 is used to perform data security protection on the data acquisition stage, the data transmission stage, the model training and aggregation stage, and the data storage stage; wherein the data preprocessing module 2 uses the ROS (Robot Operating System) framework to manage the data acquisition and synchronization of multiple sensors, and the data preprocessing module 2 adopts CUDA (Compute Unified Device Architecture, Architecture) to realize real-time preprocessing operations; Data security module 9 uses differential privacy processing to protect data security in the data acquisition stage, uses SSL / TLS protocol to encrypt data communication in the data transmission stage to prevent eavesdropping and data modification, performs encrypted calculations in the model training and aggregation stages to prevent data leakage, encrypts the stored data in the data storage stage, sets access rights to prevent unauthorized access; Differential privacy processing is specifically: in the process of data acquisition and model training, noise is added to protect user privacy, the formula is:

[0094]

[0095] Among them, Δf is the global sensitivity and ε is the privacy budget;

[0096] In the above step 2, the multimodal data acquisition module 1 is used to collect visual data, depth data and voice data, and transmit them to the data preprocessing module 2, which synchronizes, aligns and preprocesses the data to generate high-quality input data; the synchronization and alignment are specifically: using the timestamp t for data synchronization, and using the spatial variation matrix T for spatial alignment, the formula is as follows: D aligned =T·D raw ; The preprocessing is specifically: denoising, lighting correction and color correction of the visual data to obtain the preprocessed visual image I v , denoise, filter and convert the depth map to point cloud to obtain the preprocessed point cloud data I d , perform noise elimination and speech enhancement on the speech data to obtain the preprocessed speech signal I s ;

[0097] In the above step three, the multimodal fusion perception module 3 receives the high-quality input data generated by the data preprocessing module 2, detects and classifies the goods, outputs the category, location and confidence of the goods, and is displayed by the interactive module 8; wherein, the multimodal fusion perception module 3 is based on Fusion CNN (fusion convolutional neural network) and GNN (graph neural network, graph neural network) for classification, and the specific process of the multimodal fusion perception module 3 for detecting and classifying goods is as follows: the feature extraction unit 31 extracts multimodal features, the feature fusion unit 32 uses the attention mechanism to fuse the extracted multimodal features, and the commodity detection and classification unit 33 uses GNN to detect and classify the goods; in the multimodal feature extraction, the visual feature extraction adopts the convolutional neural network based on ResNet50, and the preprocessed visual image I is input. v , output visual feature vector F v ; Deep feature extraction based on PointNet++ point cloud processing network, input preprocessed point cloud data I d , output the deep feature vector F d ; Speech feature extraction uses a recurrent neural network based on Bi-LSTM (Bidirectional Long Short-Term Memory) and inputs the preprocessed speech signal I s , output speech feature vector F s ; Multimodal feature extraction is specifically as follows: for the visual feature vector F v , deep feature vector F d and speech feature vector F s , extracted through the corresponding feature extraction network:

[0098] F v =CNN vision (I v );F d =CNN vision (I d );F s =CNN vision (I s );

[0099] The specific feature fusion is:

[0100] F fused =α v F v +α d F d +α s F s

[0101] Among them, α v +αd +α s =1, which is a learnable weight parameter;

[0102] The use of graph neural network classifiers for commodity detection and classification is as follows: construct a graph structure of commodity features, where nodes represent candidate commodity areas and edges represent the association between commodities. GNN is used to classify and predict the attributes of commodities. The formula is as follows:

[0103] H (l+1) =σ(AH (l) W (l) )

[0104] Among them, H( l ) is the node feature of the lth layer, A is the adjacency matrix, W( l ) is the weight matrix;

[0105] In the above step 4, the product tracking module 4 updates the trajectory and status information of the product according to the detection results of the multimodal fusion perception module 3; the product tracking module 4 adopts a deep reinforcement learning algorithm based on the Actor-Critic architecture and a real-time decision-making tracking strategy. The environment simulation of the product tracking module 4 adopts the OpenAI Gym framework to build a simulation environment for product tracking. The intelligent agent of the product tracking module 4 includes a policy network and a value network, and both the policy network and the value network adopt MLP (Multilayer Perceptron); the interaction process between the intelligent agent and the environment is: the intelligent agent selects action a according to the strategy π t , the environment feedbacks the new state s t+1 and reward r t ; The learning update process is as follows: use the collected experience to calculate the advantage function A t , update the parameters θ of the policy network and the value network π and θ V ;

[0106] In the policy network, input the current state s t , output action a t The probability distribution of is:

[0107] π(at|st;θ π )=softmax(f(st;θ π ))

[0108] In the value network, input the current state s t , output estimated state value function V(s t θ V )for:

[0109] V(s t θV )=f(s t θ V )

[0110] The loss function is:

[0111] L(θ π ,θ V )=Ε π [(r t +γV(s t+1 )-V(s t )) 2 -logπ(a t s t θ π )A t ]

[0112] Among them, A t is the advantage function, r t is the immediate reward, γ is the discount factor;

[0113] In the above step 5, the posture estimation algorithm module 5 calculates the 3D posture of the product according to the detection result of the multimodal fusion perception module 3, determines whether the product is put into the shopping cart, and updates the shopping list according to the judgment result; wherein, the posture estimation algorithm module 5 adopts self-supervised learning technology to learn the 3D posture of the product from unlabeled data, uses a Transformer-based posture estimation network to process sequence data, uses unlabeled large-scale data sets for pre-training, and uses reprojection consistency loss to optimize the model; the algorithm training process of the posture estimation algorithm module 5 is specifically as follows:

[0114] 1) Input image pairs: Get image pairs of adjacent frames from the unlabeled dataset. t and I t+1 ;

[0115] 2) Posture estimation network: Input I t and I t+1 , output relative posture transformation parameter T t→t+1 ;

[0116] 3) Depth estimation network: Input I t , output depth map D t ;

[0117] 4) Disparity calculation and reprojection: using depth map D t and relative attitude transformation parameter T t→t+1 , will I t Reproject to I t+1 The reconstructed image is obtained from the viewing angle

[0118] 5) Calculate the reprojection consistency loss:

[0119]

[0120] 6) Model update: back propagation, update network parameters;

[0121] In the above step 6, the abnormality detection module 6 identifies potential abnormal behaviors according to the detection results of the multimodal fusion perception module 3, the commodity tracking module 4 and the posture estimation algorithm module 5, and triggers the interaction module 8 to issue a security alarm if abnormal behaviors are identified;

[0122] In the above step 7, the user views the shopping list, product information and personalized recommendations through the interactive module 8, specifically:

[0123] 1) User input: The user interacts with the system through the interaction module 8;

[0124] 2) User feature extraction: The system collects the user’s shopping history, preferences, and behavior data to generate a user embedding vector e u ;

[0125] e u =f user (u)

[0126] 3) Recommendation algorithm generates recommendation results:

[0127] Product embedding vector calculation:

[0128] e i =f item (i)

[0129] Preference matching and scoring:

[0130]

[0131] Among them, W is the weight matrix and b is the bias term;

[0132] 4) Recommendation result display: the interactive module 8 displays the recommended product list;

[0133] 5) User feedback: The system records user clicks, purchases, and reviews, updates user models, and optimizes recommendation algorithms;

[0134] In the above step 8, after the user completes shopping, he / she selects the self-service checkout and payment method through the interactive module 8;

[0135] In the above step 9, the shopping data and user behavior data are synchronized to the server 10 through the interactive module 8 to participate in the model update of the learning module 7; wherein, the learning module 7 adopts the model-independent meta-learning algorithm and the federated average algorithm, and the federated learning adopts the TensorFlow Federated framework; the update process of the learning module 7 is:

[0136] 1) Local task training: Each client performs task training on local data, using a model-independent meta-learning algorithm to obtain updated model parameters θ i ′;

[0137] 2) Parameter encryption upload: Encrypt the model parameters and upload them to the server 10;

[0138] 3) Parameter aggregation: The server 10 receives the encrypted model parameters and aggregates them using the federated averaging algorithm to obtain the global model parameters θ;

[0139] 4) Model distribution: The server 10 distributes the updated global model parameters θ to each client;

[0140] 5) Iteration: The client uses the new model parameters to perform the next round of local training;

[0141] The client is the interactive module 8 on the shopping cart; the model-independent meta-learning algorithm is specifically:

[0142] 1) Meta-training process: For task A i , calculate the gradient:

[0143]

[0144] 2) Meta Update:

[0145]

[0146] Among them, α and β are learning rates;

[0147] Federated learning is specifically:

[0148] 1) Local update: Client k is trained on local data to obtain model parameters ω k ;

[0149] 2) Global aggregation: The server 10 aggregates the model parameters of each client:

[0150]

[0151] Among them, n k is the amount of data from client k,

[0152] Based on the above, the advantage of the present invention is that when the present invention is used, firstly, the multimodal data acquisition module 1 is used to collect data, the camera array 11 is used to collect visual data, the depth sensor 12 is used to collect depth data, and the microphone array 13 is used to collect voice data, and the data is transmitted to the data preprocessing module 2. The data preprocessing module 2 synchronizes, aligns and preprocesses the data to generate high-quality input data, and sends the data to the multimodal fusion perception module 3. The multimodal fusion perception module 3 uses the feature extraction unit 31 to extract multimodal features, and uses the feature fusion unit 32 to fuse the extracted multimodal features, and inputs the fused features into the product detection and classification unit 33, and outputs the product classification result. The multimodal fusion perception module 3 sends the detection result to the product tracking module 4 and the posture estimation algorithm module 5. The product tracking module 4 updates the trajectory and status information of the product, and the posture estimation algorithm module 5 calculates the 3D posture of the product to determine whether the product is placed in the shopping cart. The shopping list is updated according to the judgment result. At the same time, the detection results of the multimodal fusion perception module 3, the product tracking module 4 and the posture estimation algorithm module 5 are sent to the anomaly detection module 6. The anomaly detection module 6 identifies potential abnormal behaviors and issues an alarm reminder. The user views the shopping list, product information and personalized recommendations through the interactive module 8. After the user completes shopping, the user selects the self-service settlement and payment method through the interactive module 8. The interactive module 8 synchronizes the shopping data and user behavior data to the server 10. These data participate in the model update of the learning module 7. During the operation of the system, the data security module 9 ensures data security and user privacy. Among them, the touch screen display 81 is used for data visualization and command input, the voice interaction device 82 is used for voice interaction, the computing unit 83 is used for data processing, the power supply unit 84 is used to supply power to the interactive module 8 and the multimodal data acquisition module 1, and the communication unit 85 is used to establish a data connection with the multimodal data acquisition module 1 and the server 10.

[0153] It will be apparent to those skilled in the art that the invention is not limited to the details of the exemplary embodiments described above and that the invention can be implemented in other specific forms without departing from the spirit or essential features of the invention. Therefore, the embodiments should be considered exemplary and non-limiting in all respects, and the scope of the invention is defined by the appended claims rather than the foregoing description, and it is intended that all variations falling within the meaning and scope of the equivalent elements of the claims be included in the invention. Any reference numeral in a claim should not be considered as limiting the claim to which it relates.

Claims

1. An intelligent shopping cart system based on deep reinforcement learning, comprising a multimodal data acquisition module (1), a data preprocessing module (2), a multimodal fusion perception module (3), a product tracking module (4), a posture estimation algorithm module (5), an anomaly detection module (6), a learning module (7), an interaction module (8), a data security module (9) and a server (10), Features: The data preprocessing module (2) establishes data connections with the multimodal data acquisition module (1) and the multimodal fusion perception module (3), respectively; the multimodal fusion perception module (3) establishes data connections with the commodity tracking module (4) and the posture estimation algorithm module (5), respectively; the anomaly detection module (6) establishes data connections with the multimodal fusion perception module (3), the commodity tracking module (4) and the posture estimation algorithm module (5), respectively; the server (10) is data-connected with the interaction module (8) and the data security module (9), and the learning module (7) establishes data connections with the multimodal fusion perception module (3), the commodity tracking module (4), the posture estimation algorithm module (5) and the anomaly detection module (6), respectively.

2. According to claim 1, the smart shopping cart system based on deep reinforcement learning, Features: The data preprocessing module (2), the multimodal fusion perception module (3), the commodity tracking module (4), the posture estimation algorithm module (5), the anomaly detection module (6) and the learning module (7) are all arranged on the interaction module (8).

3. The smart shopping cart system based on deep reinforcement learning according to claim 1, Features: The multimodal data acquisition module (1) comprises a camera array (11), a depth sensor (12) and a microphone array (13); the camera array (11) adopts a Sony IMX577 camera, and the depth sensor (12) adopts an Intel RealSense D435 depth camera.

4. The smart shopping cart system based on deep reinforcement learning according to claim 2, Features: The multimodal fusion perception module (3) comprises a feature extraction unit (31), a feature fusion unit (32) and a commodity detection and classification unit (33); the interaction module (8) comprises a touch screen display (81), a voice interaction device (82), a computing unit (83), a power supply unit (84) and a communication unit (85); the computing unit (83) adopts an NVIDIA Jetson AGX Orin embedded computing device.

5. A method for implementing a smart shopping cart system based on deep reinforcement learning, including step 1, system deployment; step 2, data collection and preprocessing; step 3, commodity detection and classification; step 4, commodity tracking and status update; step 5, posture estimation and state judgment; step 6, anomaly detection and security alarm; step 7, user interaction and recommendation; step 8, settlement and payment; step 9, data synchronization and model update; Features: In the above step 1, a multimodal data acquisition module (1) and an interaction module (8) are deployed on the shopping cart, a data preprocessing module (2), a multimodal fusion perception module (3), a product tracking module (4), a posture estimation algorithm module (5), an anomaly detection module (6) and a learning module (7) are deployed on the interaction module (8), a data connection is established between the data preprocessing module (2) and the multimodal data acquisition module (1), a data connection is established between the interaction module (8) and the server (10), a data security module (9) is constructed, and the data security module (9) is used to perform data security protection on the data acquisition stage, the data transmission stage, the model training and aggregation stage, and the data storage stage; In the above step 2, the multimodal data acquisition module (1) is used to collect visual data, depth data and voice data, and transmits them to the data preprocessing module (2). The data preprocessing module (2) synchronizes, aligns and preprocesses the data to generate high-quality input data; the synchronization and alignment are specifically: using the timestamp t for data synchronization and using the spatial variation matrix T for spatial alignment, the formula is as follows: D aligned =T·D raw ; The preprocessing is specifically: denoising, lighting correction and color correction of the visual data to obtain the preprocessed visual image I v , denoise, filter and convert the depth map to point cloud to obtain the preprocessed point cloud data I d , perform noise elimination and speech enhancement on the speech data to obtain the preprocessed speech signal I s ; In the above step 3, the multimodal fusion perception module (3) receives high-quality input data generated by the data preprocessing module (2), detects and classifies the goods, outputs the category, location and confidence of the goods, and is displayed by the interaction module (8); In the above step 4, the commodity tracking module (4) updates the trajectory and status information of the commodity according to the detection result of the multimodal fusion perception module (3); In the above step 5, the posture estimation algorithm module (5) calculates the 3D posture of the product according to the detection result of the multimodal fusion perception module (3), determines whether the product is put into the shopping cart, and updates the shopping list according to the determination result; In the above step 6, the abnormality detection module (6) identifies potential abnormal behavior based on the detection results of the multimodal fusion perception module (3), the commodity tracking module (4) and the posture estimation algorithm module (5), and triggers the interaction module (8) to issue a security alarm if abnormal behavior is identified; In the above step 7, the user views the shopping list, product information and personalized recommendations through the interactive module (8), specifically: 1) User input: The user interacts with the system through the interaction module (8); 2) User feature extraction: The system collects the user’s shopping history, preferences, and behavior data to generate a user embedding vector e u ; have been u =f user (u) 3) Recommendation algorithm generates recommendation results: Product embedding vector calculation: have been i =f item (i) Preference matching and scoring: Among them, W is the weight matrix and b is the bias term; 4) Recommendation result display: The interactive module (8) displays the recommended product list; 5) User feedback: The system records user clicks, purchases, and reviews, updates user models, and optimizes recommendation algorithms; In the above step eight, after the user completes shopping, he / she selects the self-service checkout and payment method through the interactive module (8); In the above step nine, the shopping data and user behavior data are synchronized to the server (10) through the interactive module (8), and the model update of the participating learning module (7) is carried out.

6. According to the method for implementing the smart shopping cart system based on deep reinforcement learning according to claim 5, Features: In the step 1, the data preprocessing module (2) uses the ROS framework to manage the data collection and synchronization of multiple sensors, and the data preprocessing module (2) uses the CUDA accelerated image and point cloud processing algorithm to implement real-time preprocessing operations; the data security module (9) uses differential privacy processing to protect data security in the data collection stage, uses the SSL / TLS protocol to encrypt data communication in the data transmission stage to prevent eavesdropping and data modification, performs encrypted calculations in the model training and aggregation stages to prevent data leakage, encrypts the stored data in the data storage stage, sets access permissions to prevent unauthorized access; the differential privacy processing is specifically: in the data collection and model training process, noise is added to protect user privacy, and the formula is: Among them, Δf is the global sensitivity and ε is the privacy budget.

7. The method for implementing the smart shopping cart system based on deep reinforcement learning according to claim 5, Features: In the step 3, the multimodal fusion perception module (3) performs classification based on Fusion CNN and GNN. The specific process of the multimodal fusion perception module (3) performing commodity detection and classification is as follows: the feature extraction unit (31) performs multimodal feature extraction, the feature fusion unit (32) uses the attention mechanism to fuse the extracted multimodal features, and the commodity detection and classification unit (33) uses GNN to perform commodity detection and classification; in the multimodal feature extraction, the visual feature extraction uses a convolutional neural network based on ResNet50, and the pre-processed visual image I is input. v , output visual feature vector F v ; Deep feature extraction based on PointNet++ point cloud processing network, input preprocessed point cloud data I d , output the deep feature vector F d ; Speech feature extraction uses a recurrent neural network based on Bi-LSTM, and inputs the preprocessed speech signal I s , output speech feature vector F s ; Multimodal feature extraction is specifically as follows: for the visual feature vector F v , deep feature vector F d and speech feature vector F s , extracted through the corresponding feature extraction network: F v =CNN vision (I v );F d =CNN vision (I d );F s =CNN vision (I s ); The specific feature fusion is: F fused =a v F v +a d F d +a s F s Among them, α v +α d +α s =1, which is a learnable weight parameter; The use of graph neural network classifiers for commodity detection and classification is as follows: construct a graph structure of commodity features, where nodes represent candidate commodity areas and edges represent the association between commodities. GNN is used to classify and predict the attributes of commodities. The formula is as follows: H (l+1) =σ(AH (l) W (l) ) Among them, H( l ) is the node feature of the lth layer, A is the adjacency matrix, W( l ) is the weight matrix.

8. According to claim 5, the method for implementing the smart shopping cart system based on deep reinforcement learning, Features: In the step 4, the product tracking module (4) adopts a deep reinforcement learning algorithm based on the Actor-Critic architecture and a real-time decision-making tracking strategy. The environment simulation of the product tracking module (4) adopts the OpenAI Gym framework to build a simulation environment for product tracking. The intelligent agent of the product tracking module (4) includes a policy network and a value network, and both the policy network and the value network adopt MLP. The interaction process between the intelligent agent and the environment is as follows: the intelligent agent selects action a according to the strategy π t , the environment feedbacks the new state s t+1 and reward r t ; The learning update process is as follows: use the collected experience to calculate the advantage function A t , update the parameters θ of the policy network and the value network π and θ V ; In the policy network, input the current state s t , output action a t The probability distribution of is: π(at|st;θ π )=softmax(f(st;θ π )) In the value network, input the current state s t , output estimated state value function V(s t θ V )for: V(s t ;θ V )=f(s t ;θ V ) The loss function is: L(θ π ,i V )=E π [(r t +γV(s t+1 )-V(s t )) 2 -logπ(a t s t ;θ π )A t ] Among them, A t is the advantage function, r t is the immediate reward, and γ is the discount factor.

9. The method for implementing the smart shopping cart system based on deep reinforcement learning according to claim 5, Features: In the step 5, the posture estimation algorithm module (5) adopts self-supervised learning technology to learn the 3D posture of the product from unlabeled data, uses a Transformer-based posture estimation network to process sequence data, uses an unlabeled large-scale data set for pre-training, and uses the reprojection consistency loss to optimize the model; the algorithm training process of the posture estimation algorithm module (5) is specifically as follows: 1) Input image pairs: Get image pairs of adjacent frames from the unlabeled dataset. t and I t+1 ; 2) Posture estimation network: Input I t and I t+1 , output relative posture transformation parameter T t→t+1 ; 3) Depth estimation network: Input I t , output depth map D t ; 4) Disparity calculation and reprojection: using depth map D t and relative attitude transformation parameter T t→t+1 , will I t Reproject to I t+1 The reconstructed image is obtained from the viewing angle 5) Calculate the reprojection consistency loss: 6) Model update: back propagation, update network parameters.

10. The method for implementing the smart shopping cart system based on deep reinforcement learning according to claim 5, Features: In step nine, the learning module (7) adopts a model-independent meta-learning algorithm and a federated averaging algorithm, and the federated learning adopts the TensorFlow Federated framework; the update process of the learning module (7) is as follows: 1) Local task training: Each client performs task training on local data, using a model-independent meta-learning algorithm to obtain updated model parameters θ i ′; 2) Parameter encryption upload: Encrypt the model parameters and upload them to the server (10); 3) Parameter aggregation: The server (10) receives the encrypted model parameters and aggregates them using the federated averaging algorithm to obtain the global model parameters θ; 4) Model distribution: The server (10) distributes the updated global model parameters θ to each client; 5) Iteration: The client uses the new model parameters to perform the next round of local training; The client is the interactive module (8) on the shopping cart; the model-independent meta-learning algorithm is specifically: 1) Meta-training process: For task A i , calculate the gradient: 2) Meta Update: Among them, α and β are learning rates; Federated learning is specifically: 1) Local update: Client k is trained on local data to obtain model parameters ω k ; 2) Global aggregation: The server (10) aggregates the model parameters of each client: Among them, n k is the data volume of client k,

Images