Artificial intelligence offline retail system and method

Abstract

The invention discloses an artificial intelligence offline retail system and method, and relates to the technical field of artificial intelligence. According to the invention, through the integrated artificial intelligence offline retail system, the key technical problem of accurate recognition and tracking of customer behavior tracks by a computer vision technology in an entity retail scene is effectively solved; the customer behavior recognition and tracking capability in a complex shopping mall environment is improved, the system adapts to the diversity of customer behavior modes, and the system can obtain the position and moving track information of customers in real time and construct a preliminary portrait in combination with the appearance characteristics of the customers; in addition, the graph neural network and the knowledge graph technology are used for correlation analysis of multi-modal data, a comprehensive and accurate customer portrait is constructed, semantic user behavior understanding is realized, personalized service and accurate marketing capabilities are remarkably improved, and retailers can better understand and meet customer demands.

Description

Technical Field

[0001] The present invention relates to the field of artificial intelligence technology, and specifically to an artificial intelligence offline retail system and method. Background Art

[0002] In the physical retail scenario, how to accurately identify and track the behavior trajectories of customers using computer vision technology is a key technical problem. Due to the complex and ever-changing shopping mall environment and the diverse behavior patterns of customers, traditional rule-based tracking algorithms are difficult to adapt to this dynamic scenario. In addition, the massive customer behavior data poses high requirements on the storage and computing capabilities of the system. How to design efficient data processing processes and algorithms to achieve real-time behavior analysis and decision optimization is also an urgent problem to be solved. There are still many technical problems to be overcome to truly apply computer vision technology to the physical retail scenario. Summary of the Invention

[0003] The purpose of the present invention is to provide an artificial intelligence offline retail system and method to improve the ability to identify and track customer behavior in the physical retail scenario and enhance personalized services and precision marketing.

[0004] The purpose of the present invention can be achieved through the following technical solutions:

[0005] The present application provides an artificial intelligence offline retail system, including a positioning and tracking module, a behavior analysis and inference module, a multi-modal data correlation analysis module, a data protection and collaboration module, a dynamic update and optimization module, and a decision optimization module;

[0006] The positioning and tracking module uses a deep learning-based object detection algorithm to perform real-time positioning and tracking of customers, obtain information such as the positions and movement trajectories of customers, and combine the appearance features of customers to construct a preliminary portrait of customers;

[0007] The behavior analysis and inference module analyzes the body postures and movements of customers through a pose estimation algorithm according to the preliminary portrait of customers, judges the behavior states of customers, and combines the interaction information between customers and goods to infer the shopping intentions and preferences of customers;

[0008] The multi-modal data correlation analysis module performs correlation analysis on the multi-modal data of customer videos, images, and trajectories through a graph neural network algorithm to construct a more comprehensive and accurate customer portrait, and uses knowledge graph technology to associate the customer portrait with product and scenario information to form a semantic understanding of user behavior;

[0009] The data protection and collaboration module performs distributed storage and computing in the form of federated learning based on semantically understood user behaviors, shares and collaboratively analyzes data among multiple shopping malls, and then introduces a differential privacy protection mechanism to desensitize and add noise to customers' sensitive information.

[0010] The dynamic update and optimization module constructs a neural network model based on deep learning using an incremental learning strategy according to the dynamic changes in customer behaviors, updates its weights and biases by continuously learning new data, and performs real-time updates and optimizations on the model to adapt to new behavior patterns and data distributions. Then, it adopts an active learning strategy to collect and label key samples.

[0011] The decision optimization module optimizes the model through model compression and pruning techniques for real-time behavior analysis and decision optimization, and conducts visual analysis on the decision-making process of the model through a model interpretation mechanism.

[0012] Furthermore, the positioning and tracking module includes:

[0013] By deploying cameras inside the shopping mall, real-time video image data of customers in the shopping mall is collected, and the image data is transmitted to the background server for processing. A pre-trained YOLO or Faster R-CNN deep learning object detection model is used to process the real-time video images, identify customer targets in the images, and obtain the position coordinate information of the customers.

[0014] Based on the position coordinate information of the customers and combined with the floor plan of the shopping mall, the specific location where the customers are located is judged to determine the spatial location information of the customers.

[0015] For consecutive multiple frames of images of the same customer, the position of the customer is tracked through an object tracking algorithm to obtain the movement trajectory information of the customer.

[0016] During the customer detection and tracking process, through the analysis of the appearance features of the customers, the feature information of the customers is extracted, a preliminary customer portrait is constructed, and the position, trajectory, and portrait information of the customers are fused to generate structured customer data, which is stored in the database.

[0017] The collected customer data is analyzed in real time, and combined with the customer portrait information, personalized recommendations and precision marketing are carried out.

[0018] Furthermore, the behavior analysis and inference module includes: obtaining the surveillance video stream in the shopping mall, performing human body detection and tracking on the video frames to obtain the movement trajectory of each customer, and for each customer, using the OpenPose pose estimation algorithm to analyze their body postures and action sequences to judge the current behavior state of the customer.

[0019] Identify products in the video through object detection algorithms, and judge the interaction between the customer and the product, which is used as a measure of the customer's interest in the product. Based on the customer's historical shopping records, build a customer profile, and use recommendation algorithms such as collaborative filtering to obtain products that the customer may be interested in; comprehensively consider the customer's behavior status, product interaction situation, and shopping preferences, and use a rule engine or machine learning model to infer the customer's shopping intention and make personalized product recommendations;

[0020] For products with a long stop time or repeated picking, actively recommend relevant products to the customer through voice interaction or intelligent push.

[0021] Further, the multi-modal data association analysis module includes: obtaining the video, image, and trajectory multi-modal data of the customer in the mall, preprocessing the data, extracting key features, and then using the graph neural network algorithm to construct a customer behavior graph, where the nodes represent the key behaviors of the customer and the edges represent the associations between the behaviors;

[0022] Through the message passing mechanism of the graph neural network, mine the implicit connections between customer behavior data, learn the customer behavior pattern, and fuse the learned customer behavior pattern with the pre-constructed customer attribute knowledge graph to generate a more comprehensive and accurate customer profile;

[0023] Use knowledge graph reasoning technology to semantically associate the customer profile with information such as products and scenarios to obtain the customer's preferences for specific products and scenarios. According to the customer's preferences, use the collaborative filtering algorithm to generate a personalized product recommendation list; continuously collect customer feedback data and dynamically optimize the recommendation strategy through the reinforcement learning algorithm.

[0024] Further, the data protection and collaboration module includes: extracting and annotating the user behavior data of multiple malls according to semantic analysis technology to obtain structured user behavior characteristics; using the federated learning framework to distribute and store the user behavior characteristic data of each mall, and realizing secure data transmission between different mall nodes through an encrypted communication protocol;

[0025] Then use the differential privacy mechanism to desensitize the original user data and hide sensitive information by adding random noise; based on the desensitized user behavior characteristics, use distributed machine learning algorithms such as distributed gradient descent for model training to obtain a global user profile model;

[0026] Use the trained user profile model to predict and recommend the target marketing population of the mall, and output the top-k recommendation results; match the recommendation results with the promotional offers of the mall, and convey the marketing content to the target users through personalized push.

[0027] Collect user feedback data on pushed content to optimize the personalized recommendation model.

[0028] Furthermore, the dynamic update and optimization module includes: acquiring customer behavior data, preprocessing the data, extracting key features, constructing a training data set and a test data set, using a deep neural network model, initially training the model according to the training data set, and obtaining initial model parameters;

[0029] During the operation of the online system, new customer behavior data is continuously obtained and used as incremental data for incremental learning of the model. The model parameters are updated using the incremental data through the small batch gradient descent algorithm to adapt the model to the new behavior pattern.

[0030] Adopt active learning strategies, select samples that have the greatest effect on improving the model based on the prediction uncertainty of the model, actively ask users or experts for annotations, obtain high-quality key samples, add them to the training data set, evaluate the model regularly, and calculate performance indicators on the test data set;

[0031] Furthermore, the performance indicators on the test data set are judged. When the performance indicators are lower than the preset threshold, the model retraining is triggered, and the model is comprehensively updated using all accumulated training data. When the model is inferred, the user's real-time behavior data is predicted, and the user's behavior preferences or purchase intention results are output; when the prediction confidence is lower than the preset threshold, the active learning process is triggered to request user feedback or expert annotation.

[0032] Furthermore, the decision optimization module includes: acquiring an original behavior stream formed by collecting continuous user operations at the device end, extracting features of a single behavior in the original behavior stream according to a pre-established behavior feature library, and obtaining a pre-processed behavior stream;

[0033] Extract behavior data from the preprocessed behavior stream and input it into the trained behavior recognition model to obtain a preliminary classification result. When the confidence of a behavior in the preliminary classification result is lower than a preset threshold, the behavior category is determined to be ambiguous. The activation values ​​of each neuron in the behavior recognition model calculated for the input behavior data are used to obtain several neurons with high contribution to the classification result.

[0034] Based on the neurons with high contribution, we trace back to the input layer to obtain the highly sensitive feature areas. Then, we use the pruning algorithm to prune the neurons below the activation threshold in the behavior recognition model to obtain a streamlined model.

[0035] Further, after obtaining the simplified model, the method further includes: quantifying the network parameters of each layer in the simplified model through a compression algorithm to determine the compressed model, judging the behavior category according to the classification result output by the compressed model; when the behavior category belongs to a preset high-risk category, starting an early warning mechanism, calculating the highly sensitive feature area by using the compressed model, obtaining the classification result output by the compressed model, and getting a high-risk behavior warning; triggering corresponding security policies according to the high-risk behavior warning, analyzing the user behavior according to the high-risk behavior warning output by the compressed model; and then judging the user behavior pattern according to the historical behavior data to obtain the user behavior risk level.

[0036] The present invention provides an artificial intelligence offline retail method, which is applied to an artificial intelligence offline retail system, and includes the following steps:

[0037] Adopting a deep learning-based object detection algorithm to perform real-time positioning and tracking on customers, obtaining information such as the positions and movement trajectories of customers, and combining the appearance features of customers to construct a preliminary portrait of the customers;

[0038] According to the preliminary portrait of the customers, analyzing the body postures and actions of the customers through a pose estimation algorithm, judging the behavior states of the customers, and combining the interaction information between the customers and the commodities to infer the shopping intentions and preferences of the customers;

[0039] Performing correlation analysis on the multi-modal data of the videos, images and trajectories of customers through a graph neural network algorithm to construct a more comprehensive and accurate customer portrait, and using knowledge graph technology to associate the customer portrait with commodity and scene information to form a semantic understanding of user behavior;

[0040] According to the semantic understanding of user behavior, adopting the method of federated learning for distributed storage and calculation, data sharing and collaborative analysis among multiple shopping malls, and then introducing a differential privacy protection mechanism to desensitize and denoise the sensitive information of customers;

[0041] According to the dynamic changes of customer behavior, adopting an incremental learning strategy to construct a deep learning-based neural network model, updating its weights and biases by continuously learning new data, performing real-time update and optimization on the model, and then adopting an active learning strategy to collect and label key samples targeted;

[0042] Optimizing the model through model compression and pruning technologies for real-time behavior analysis and decision optimization, and then performing visual analysis on the decision-making process of the model through a model interpretation mechanism.

[0043] The beneficial effects of the present invention are:

[0044] Through the integrated artificial intelligence offline retail system, the present invention effectively solves the key technical problems of accurately identifying and tracking the customer behavior trajectory in the physical retail scenario. By adopting the object detection algorithm and pose estimation algorithm based on deep learning, the ability to identify and track customer behavior in a complex shopping mall environment is improved, adapting to the diversity of customer behavior patterns. Through these technologies, the system can obtain the location and movement trajectory information of customers in real time, and construct a preliminary portrait by combining the appearance features of customers. In addition, the graph neural network and knowledge graph technologies are used to analyze and correlate multi-modal data, construct a comprehensive and accurate customer portrait, realize semantic understanding of user behavior, and significantly improve the ability of personalized service and precision marketing, enabling retailers to better understand and meet customer needs.

[0045] The present invention optimizes the data processing flow and algorithms, realizes efficient real-time behavior analysis and decision optimization. By adopting the federated learning and differential privacy protection mechanisms, it not only realizes the distributed storage and calculation of data, but also protects customer privacy. The dynamic update and optimization module enables the model to be updated in real time and adapt to new behavior patterns through incremental learning and active learning strategies, maintaining the accuracy and timeliness of the model. The decision optimization module reduces the computational complexity and storage requirements of the model through model compression and pruning technologies, and improves the interpretability and credibility of the system through the model explanation mechanism, not only improving the operation efficiency of the system, but also ensuring the security of data processing and privacy protection, providing a highly efficient and reliable decision support system for retailers. BRIEF DESCRIPTION OF THE DRAWINGS

[0046] For better understanding and implementation, the technical solutions of the present application will be described in detail below with reference to the drawings.

[0047] Figure 1 It is a schematic structural diagram of an artificial intelligence offline retail system provided in Embodiment 1 of the present application;

[0048] Figure 2 It is a schematic flow diagram of the positioning and tracking module of an artificial intelligence offline retail system provided in Embodiment 1 of the present application;

[0049] Figure 3 It is a schematic flow diagram of an artificial intelligence offline retail method provided in Embodiment 2 of the present application. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0050] To further elaborate on the technical means and effects adopted by the present invention to achieve the predetermined invention purpose, exemplary embodiments will be described in detail herein, and the examples are shown in the accompanying drawings. When the following description refers to the accompanying drawings, unless otherwise indicated, the same numbers in different drawings represent the same or similar elements. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with the present application. On the contrary, they are merely examples of methods and systems consistent with some aspects of the present application as detailed in the appended claims.

[0051] The terms used in this application are for the purpose of describing specific embodiments only and are not intended to limit the present application. The singular forms "a", "the", and "said" used in this application and the appended claims are also intended to include the plural forms unless the context clearly indicates otherwise. It should also be understood that the term "and / or" used herein refers to and includes any or all possible combinations of one or more of the associated listed items.

[0052] The following will, in conjunction with the accompanying drawings and preferred embodiments, elaborate in detail on the specific implementation manners, features, and effects of the present invention.

[0053] Embodiment 1

[0054] Please refer to Figure 1 - Figure 2 , this embodiment provides an artificial intelligence offline retail system, including a positioning and tracking module, a behavior analysis and inference module, a multi-modal data association analysis module, a data protection and collaboration module, a dynamic update and optimization module, and a decision optimization module;

[0055] The positioning and tracking module, aiming at the complex and changeable environment of the shopping mall, adopts a deep learning-based object detection algorithm, such as YOLO or Faster R-CNN, to perform real-time positioning and tracking on customers, obtain information such as the positions and movement trajectories of customers, and construct a preliminary portrait of customers in combination with the appearance features of customers, such as clothing and body postures.

[0056] Further, the positioning and tracking module includes:

[0057] S11. By deploying cameras inside the shopping mall, collect real-time video image data of customers in the shopping mall, transmit the image data to the background server for processing, and use a pre-trained deep learning object detection model such as YOLO or Faster R-CNN to process the real-time video image, identify the customer objects in the image, and obtain the position coordinate information of the customers;

[0058] S12. According to the position coordinate information of the customers, in combination with the floor plan of the shopping mall, judge the specific position where the customers are located, such as near a certain area or a certain store, and determine the spatial position information of the customers;

[0059] S13. For consecutive multiple frames of images of the same customer, track the position of the customer through a target tracking algorithm, such as Kalman filtering or particle filtering, to obtain the customer's movement trajectory information;

[0060] S14. During the customer detection and tracking process, analyze the customer's appearance features, such as clothing color, style, body posture, etc., extract the customer's feature information, construct a preliminary customer portrait, fuse information such as the customer's position, trajectory, and portrait, generate structured customer data, and store it in the database to provide data support for subsequent applications such as passenger flow analysis and shopping behavior analysis.

[0061] S15. Conduct real-time analysis on the collected customer data, such as customer heat maps and regional passenger flow, to provide a decision-making basis for optimizing mall operations. At the same time, combine the customer portrait information to perform personalized recommendations and precision marketing.

[0062] Specifically, by deploying cameras to collect real-time video image data of customers in the mall and using pre-trained deep learning object detection models such as YOLO or Faster R-CNN for processing, it is possible to locate and track customers in real time, obtain their position coordinates and movement trajectory information. Combining with the mall layout map, the specific position of the customer can be accurately judged. Using target tracking algorithms such as Kalman filtering or particle filtering, it is possible to continuously track the same customer and construct their movement trajectory. In addition, by analyzing the customer's appearance features, key information can be extracted and a preliminary customer portrait can be constructed. The fusion and structured storage of these data not only provide a solid data foundation for passenger flow analysis and shopping behavior analysis, but also provide decision-making support for optimizing mall operations through real-time analysis, such as customer heat maps and regional passenger flow. At the same time, combining the customer portrait information can achieve personalized recommendations and precision marketing, thereby enhancing the customer experience and mall efficiency.

[0063] The behavior analysis and inference module, according to the preliminary portrait of the customer, analyzes the body posture and actions of the customer through a pose estimation algorithm, such as OpenPose, judges the customer's behavior state, such as walking, standing still, picking up items, etc., and combines the interaction information between the customer and the commodity, such as gazing, touching, etc., to infer the customer's shopping intention and preference.

[0064] Furthermore, the behavior analysis and inference module includes: obtaining the monitoring video stream in the mall, performing human body detection and tracking on the video frames, obtaining the movement trajectory of each customer, and for each customer, using a pose estimation algorithm such as OpenPose to analyze their body posture and action sequence, and judge the current behavior state of the customer, such as walking, standing still, picking up items, etc.;

[0065] Identify products in the video through the object detection algorithm, and judge the interaction between the customer and the product, such as the fixation time, the number of touches, etc., as a measure of the customer's interest in the product. According to the customer's historical shopping records, build a customer profile, including attributes such as age, gender, purchasing power, etc., and through recommendation algorithms such as collaborative filtering, obtain the products that the customer may be interested in; comprehensively consider the customer's behavior status, product interaction situation, and shopping preferences, and use a rule engine or a machine learning model to infer the customer's shopping intention and make personalized product recommendations;

[0066] For products with a long standing time or repeated picking, actively recommend relevant products to the customer through voice interaction or intelligent push, provide shopping guide consultation and other services, improve the customer experience, collect customer feedback, conduct statistical analysis on the purchase conversion rate of the recommended products, etc., and continuously optimize the personalized recommendation strategy and service quality to improve customer satisfaction and sales performance.

[0067] Specifically, it can accurately capture the customer's behavior patterns in the mall, including actions such as walking, standing still, and picking up items, as well as the interaction details with the products, such as fixation and touch. Combining the customer's historical shopping data and collaborative filtering recommendation algorithm, the system can infer the customer's shopping intention and preferences, realize personalized product recommendations, and improve the customer experience through intelligent push and voice interaction. Finally, continuously optimize the recommendation strategy by collecting feedback and analyzing the conversion rate, aiming to improve customer satisfaction and sales performance.

[0068] The multi-modal data correlation analysis module conducts correlation analysis on the customer's multi-modal data such as videos, images, and trajectories through the graph neural network algorithm, mines the potential connections between the data, constructs a more comprehensive and accurate customer profile, and uses knowledge graph technology to associate the customer profile with information such as products and scenarios to form a semantic understanding of user behavior.

[0069] Furthermore, the multi-modal data correlation analysis module includes: obtaining the customer's multi-modal data such as videos, images, and trajectories in the mall, preprocessing the data, extracting key features, and then using the graph neural network algorithm to construct a customer behavior graph, where the nodes represent the customer's key behaviors and the edges represent the associations between the behaviors;

[0070] Through the message passing mechanism of the graph neural network, mine the implicit connections between the customer behavior data, learn the customer behavior patterns, and fuse the learned customer behavior patterns with the pre-constructed customer attribute knowledge graph to generate a more comprehensive and accurate customer profile;

[0071] Use knowledge graph reasoning technology to semantically associate the customer profile with information such as products and scenarios, obtain the customer's preferences for specific products and scenarios, and generate a personalized product recommendation list according to the customer's preferences using the collaborative filtering algorithm to improve the customer's shopping experience;

[0072] Continuously collect customer feedback data, and dynamically optimize the recommendation strategy through reinforcement learning algorithms to improve the accuracy and timeliness of recommendations.

[0073] Specifically, through graph neural network algorithms, deeply analyze data such as customers' videos, images, and trajectories to construct a detailed customer behavior map, thereby uncovering deep patterns in customer behavior. Combining knowledge graph technology, this module can deeply semantically associate customer portraits with information such as products and scenarios, accurately capture customer preferences, and use collaborative filtering algorithms to provide personalized product recommendations to enhance the customer shopping experience. At the same time, by continuously collecting customer feedback and applying reinforcement learning algorithms, this module can dynamically adjust the recommendation strategy to ensure the accuracy and timeliness of the recommended content, thereby continuously improving customer satisfaction and shopping efficiency.

[0074] The data protection and collaboration module, based on the semantic understanding of user behavior, adopts the method of federated learning for distributed storage and calculation, data sharing and collaborative analysis among multiple shopping malls, and then introduces a differential privacy protection mechanism to desensitize and denoise customers' sensitive information.

[0075] Furthermore, the data protection and collaboration module includes: extracting and annotating the user behavior data of multiple shopping malls according to semantic analysis technology to obtain structured user behavior characteristics; adopting a federated learning framework to distribute the storage of the user behavior characteristic data of each shopping mall, and realizing secure data transmission between different shopping mall nodes through an encrypted communication protocol;

[0076] Then use the differential privacy mechanism to desensitize the original user data, and hide sensitive information by adding random noise; based on the desensitized user behavior characteristics, use distributed machine learning algorithms such as distributed gradient descent for model training to obtain a global user portrait model;

[0077] Use the trained user portrait model to predict and recommend the target marketing population of the shopping mall, and output the top-k recommendation results; match the recommendation results with the promotional offers of the shopping mall, and accurately reach the target users with the marketing content through personalized push.

[0078] Collect the feedback data of users on the pushed content to optimize the personalized recommendation model and continuously improve the conversion effect of marketing.

[0079] Specifically, data sharing and collaborative analysis among multiple shopping malls are achieved through federated learning while protecting customer privacy. This module first uses semantic analysis technology to extract and label user behavior features, and then distributes and stores these feature data through the federated learning framework, ensuring the security of data transmission. The introduction of differential privacy mechanism effectively hides sensitive information and enhances data privacy protection by adding random noise to the original user data. Based on the desensitized data, distributed machine learning algorithms such as distributed gradient descent are used to train the global user portrait model, which helps to achieve precise marketing and recommendations. Finally, by matching the recommendation results with the shopping mall promotion information, personalized push content can accurately reach the target users, and the collected user feedback data is further used to optimize the recommendation model and improve the marketing conversion effect.

[0080] The dynamic update and optimization module adopts an incremental learning strategy to construct a deep learning-based neural network model according to the dynamic changes of customer behavior. By continuously learning new data to update its weights and biases, the model is updated and optimized in real time to adapt to new behavior patterns and data distributions. Then, an active learning strategy is adopted to selectively collect and label key samples to improve the learning efficiency and generalization ability of the model.

[0081] Furthermore, the dynamic update and optimization module includes: obtaining customer behavior data, preprocessing the data, extracting key features, constructing training and test data sets, and using a deep neural network model such as Long Short-Term Memory (LSTM) or Convolutional Neural Network (CNN) to initially train the model according to the training data set to obtain initial model parameters.

[0082] During the operation of the online system, continuously obtain new customer behavior data, use it as incremental data for incremental learning of the model, and update the model parameters using the incremental data through the mini-batch gradient descent algorithm to make the model adapt to new behavior patterns.

[0083] Adopt an active learning strategy, select the samples that have the greatest impact on improving the model according to the prediction uncertainty of the model, actively ask users or experts for annotations to obtain high-quality key samples, supplement them to the training data set, and regularly evaluate the model to calculate performance metrics on the test data set, such as accuracy, recall, etc.

[0084] When the performance metric is lower than the preset threshold, trigger model retraining, use all the accumulated training data to comprehensively update the model, and during model inference, predict the real-time behavior data of users and output results such as user behavior preferences or purchase intentions.

[0085] When the prediction confidence is lower than the preset threshold, trigger the active learning process and request user feedback or expert annotation.

[0086] Continue to iterate the above process, continuously accumulate training data, optimize model parameters, improve the generalization and adaptability of the model, and provide high-quality user behavior prediction services for the business.

[0087] Specifically, through incremental learning strategies and active learning strategies, the deep learning-based neural network model is continuously optimized and updated in real time. The customer behavior data is preprocessed and feature extracted, and then the deep neural network model such as LSTM or CNN is used for initial training. During the operation of the system, new behavior data is continuously integrated into the model, and the model parameters are updated through the small batch gradient descent algorithm to adapt to the new behavior pattern. At the same time, the module actively collects key samples based on the uncertainty of the model prediction, and regularly evaluates the model performance to ensure that indicators such as model accuracy and recall rate remain above the preset threshold. If the performance degrades or the prediction confidence is insufficient, the module will trigger retraining or request user feedback, so as to continuously iterate and optimize, improve the generalization ability and adaptability of the model, and ultimately provide accurate user behavior prediction services for the business.

[0088] The decision optimization module optimizes the model through model compression and pruning technology to perform real-time behavior analysis and decision optimization, reducing the model's computational complexity and storage overhead. The model's decision-making process is visualized and analyzed through the model interpretation mechanism to improve the system's interpretability and credibility.

[0089] Furthermore, the decision optimization module includes: acquiring an original behavior stream formed by collecting continuous user operations at the device end, extracting features of a single behavior in the original behavior stream according to a pre-established behavior feature library, and obtaining a pre-processed behavior stream;

[0090] Extract behavior data from the preprocessed behavior stream and input it into the trained behavior recognition model to obtain a preliminary classification result. When the confidence of a behavior in the preliminary classification result is lower than a preset threshold, the behavior category is determined to be ambiguous. The activation values ​​of each neuron in the behavior recognition model calculated for the input behavior data are used to obtain several neurons with high contribution to the classification result.

[0091] Based on the neurons with high contribution, we trace back to the input layer to obtain the highly sensitive feature areas. Then, we use the pruning algorithm to prune the neurons below the activation threshold in the behavior recognition model to obtain a streamlined model.

[0092] Further, after obtaining the simplified model, it further includes: quantifying the network parameters of each layer in the simplified model through a compression algorithm to determine the compressed model, judging the behavior category according to the classification result output by the compressed model; when the behavior category belongs to a preset high-risk category, the warning mechanism is activated, and the compressed model is used to calculate the highly sensitive feature area, obtain the classification result output by the compressed model, and obtain a high-risk behavior warning; according to the high-risk behavior warning, the corresponding security policy is triggered, and the user behavior is analyzed according to the high-risk behavior warning output by the compressed model; then, according to the historical behavior data, the user behavior pattern is judged to obtain the user behavior risk level.

[0093] Specifically, the model compression and pruning techniques improve the running efficiency of the model, reduce the computing and storage costs. At the same time, the model interpretation mechanism enhances the transparency and credibility of the model decision-making. This module can extract key features from the user behavior stream, identify behavior patterns, and optimize the model by analyzing neuron activation values when the confidence is insufficient. The simplified model can classify behaviors quickly and accurately, can give early warnings for high-risk behaviors in a timely manner and trigger security policies, and combine the user behavior historical data to evaluate the risk level, so as to realize real-time behavior analysis and precise decision-making optimization.

[0094] Embodiment 2

[0095] Please refer to Figure 3 , this embodiment provides an artificial intelligence offline retail method, which is applied to an artificial intelligence offline retail system, and includes the following steps:

[0096] S1. For the complex and changeable environment of the shopping mall, a deep learning-based object detection algorithm, such as YOLO or Faster R-CNN, is used to perform real-time positioning and tracking of customers, obtain information such as the positions and movement trajectories of customers, and combine the appearance features of customers, such as clothing and body postures, to construct a preliminary portrait of the customers;

[0097] S2. According to the preliminary portrait of the customers, through a pose estimation algorithm, such as OpenPose, analyze the body postures and actions of the customers, judge the behavior states of the customers, such as walking, standing still, picking up items, etc., and combine the interaction information between the customers and the goods, such as gazing and touching, to infer the shopping intentions and preferences of the customers;

[0098] S3. Through a graph neural network algorithm, perform correlation analysis on multi-modal data such as videos, images, and trajectories of customers, mine the potential connections between the data, construct a more comprehensive and accurate customer portrait, and use knowledge graph technology to associate the customer portrait with information such as goods and scenarios to form a semantic understanding of user behavior;

[0099] S4. Based on the semantic understanding of user behavior, adopt the method of federated learning for distributed storage and computing, data sharing and collaborative analysis among multiple shopping malls, and then introduce the differential privacy protection mechanism to desensitize and denoise customers' sensitive information;

[0100] S5. According to the dynamic changes of customer behavior, adopt the incremental learning strategy to build a neural network model based on deep learning, update its weights and biases by continuously learning new data, update and optimize the model in real time, adapt to new behavior patterns and data distributions, and then adopt the active learning strategy to collect and label key samples targeted, so as to improve the learning efficiency and generalization ability of the model;

[0101] S6. Optimize the model through model compression and pruning techniques for real-time behavior analysis and decision optimization, and then conduct visual analysis on the decision-making process of the model through the model interpretation mechanism to improve the interpretability and credibility of the system.

[0102] The above are only the preferred embodiments of the present invention, and do not impose any form of limitation on the present invention. Although the present invention has been disclosed above with the preferred embodiments, it is not intended to limit the present invention. Any person skilled in the art can make some changes or modifications into equivalent embodiments by using the technical content disclosed above within the scope of the technical solution of the present invention. However, as long as it does not depart from the content of the technical solution of the present invention, any brief modification, equivalent change and modification made to the above embodiments according to the technical essence of the present invention still fall within the scope of the technical solution of the present invention.

Claims

1. An artificial intelligence offline retail system, characterized by: It includes positioning and tracking module, behavior analysis and inference module, multimodal data association analysis module, data protection and collaboration module, dynamic update and optimization module and decision optimization module; The positioning and tracking module uses a deep learning-based target detection algorithm to locate and track customers in real time, obtain the customer's location and movement trajectory information, and build a preliminary profile of the customer based on the customer's appearance characteristics; The behavior analysis and inference module analyzes the customer's body posture and movements based on the customer's preliminary portrait through a posture estimation algorithm, determines the customer's behavior state, and infers the customer's shopping intention and preference based on the interaction information between the customer and the product; The multimodal data association analysis module uses a graph neural network algorithm to perform association analysis on the customer's video, image and trajectory multimodal data to build a more comprehensive and accurate customer portrait, and uses knowledge graph technology to associate the customer portrait with product and scene information to form a semantic understanding of user behavior; The data protection and collaboration module uses federated learning to perform distributed storage and computing based on semantic understanding of user behavior, data sharing and collaborative analysis among multiple shopping malls, and then introduces a differential privacy protection mechanism to desensitize and add noise to customers' sensitive information; The dynamic update and optimization module adopts an incremental learning strategy to build a neural network model based on deep learning according to the dynamic changes in customer behavior, updates its weights and biases by continuously learning new data, updates and optimizes the model in real time, adapts to new behavior patterns and data distribution, and then adopts an active learning strategy to collect and annotate key samples; The decision optimization module optimizes the model through model compression and pruning technology to perform real-time behavior analysis and decision optimization, and visualizes the decision-making process of the model through the model interpretation mechanism.

2. The artificial intelligence offline retail system according to claim 1, characterized in that: The positioning and tracking module includes: By deploying cameras in the mall, real-time video image data of customers in the mall is collected, and the image data is transmitted to the backend server for processing. The pre-trained YOLO or Faster R-CNN deep learning target detection model is used to process the real-time video images, identify the customer targets in the image, and obtain the customer's location coordinate information; According to the customer's location coordinate information, combined with the floor plan of the shopping mall, the customer's location is judged and the customer's spatial location information is determined; For the continuous multi-frame images of the same customer, the customer's position is tracked through the target tracking algorithm to obtain the customer's movement trajectory information; In the process of customer detection and tracking, by analyzing the customer's appearance characteristics, extracting the customer's characteristic information, constructing a preliminary customer portrait, integrating the customer's location, trajectory, and portrait information, generating structured customer data, and storing it in the database; Conduct real-time analysis on collected customer data, and then combine it with customer portrait information to provide personalized recommendations and precision marketing.

3. The artificial intelligence offline retail system according to claim 1, characterized in that: The behavior analysis and inference module includes: obtaining the surveillance video stream in the mall, performing human body detection and tracking on the video frames, obtaining the motion trajectory of each customer, and using the OpenPose posture estimation algorithm for each customer to analyze their body posture and action sequence to determine the customer's current behavior state; The target detection algorithm is used to identify the products in the video and judge the interaction between the customer and the product as a measure of the customer's interest in the product. The customer profile is established based on the customer's historical shopping records, and the products that the customer is interested in are obtained through the collaborative filtering recommendation algorithm. The customer's behavior status, product interaction and shopping preferences are comprehensively considered, and the rule engine or machine learning model is used to infer the customer's shopping intention and make personalized product recommendations. For products that customers spend a long time looking at or pick up repeatedly, we proactively recommend related products to customers through voice interaction or smart push notifications.

4. The artificial intelligence offline retail system according to claim 1, characterized in that: The multimodal data association analysis module includes: obtaining multimodal data of videos, images, and trajectories of customers in the mall, preprocessing the data, extracting key features, and then using a graph neural network algorithm to construct a customer behavior graph, where nodes represent key behaviors of customers and edges represent associations between behaviors; Through the message passing mechanism of graph neural networks, we can mine the implicit connections between customer behavior data, learn customer behavior patterns, and integrate the learned customer behavior patterns with the pre-built customer attribute knowledge graph to generate a more comprehensive and accurate customer portrait. Using knowledge graph reasoning technology, customer portraits are semantically associated with product and scene information to obtain customer preferences for specific products and scenes. Based on customer preferences, a collaborative filtering algorithm is used to generate a personalized product recommendation list; customer feedback data is continuously collected, and the recommendation strategy is dynamically optimized through a reinforcement learning algorithm.

5. The artificial intelligence offline retail system according to claim 1, characterized in that: The data protection and collaboration module includes: extracting and annotating user behavior data of multiple shopping malls based on semantic analysis technology to obtain structured user behavior features; using a federated learning framework to distribute and store user behavior feature data of each shopping mall, and realizing secure data transmission between nodes of different shopping malls through an encrypted communication protocol; The original user data is then desensitized using the differential privacy mechanism, and sensitive information is hidden by adding random noise. Based on the desensitized user behavior characteristics, a distributed machine learning algorithm such as distributed gradient descent is used for model training to obtain a global user portrait model. Use the trained user portrait model to predict and recommend the mall's target marketing population, and output the top-k recommendation results; match the recommendation results with the mall's promotional information, and deliver the marketing content to the target users through personalized push; Collect user feedback data on pushed content to optimize the personalized recommendation model.

6. The artificial intelligence offline retail system according to claim 1, characterized in that: The dynamic update and optimization module includes: obtaining customer behavior data, preprocessing the data, extracting key features, constructing a training data set and a test data set, using a deep neural network model, performing initial training on the model according to the training data set, and obtaining initial model parameters; During the operation of the online system, new customer behavior data is continuously obtained and used as incremental data for incremental learning of the model. The model parameters are updated using the incremental data through the small batch gradient descent algorithm to adapt the model to the new behavior pattern. An active learning strategy is adopted. According to the prediction uncertainty of the model, samples with great effect on improving the model are selected. Users or experts are actively asked to label them to obtain high-quality key samples, which are added to the training data set. The model is evaluated regularly and the performance indicators on the test data set are calculated.

7. An artificial intelligence offline retail system according to claim 6, characterized in that: Determine the performance indicators on the test data set. When the performance indicators are lower than the preset threshold, the model retraining is triggered, and all accumulated training data is used to comprehensively update the model. When the model is inferred, the user's real-time behavior data is predicted, and the user's behavior preference or purchase intention result is output; when the prediction confidence is lower than the preset threshold, the active learning process is triggered to request user feedback or expert annotation.

8. The artificial intelligence offline retail system according to claim 1, characterized in that: The decision optimization module includes: obtaining the original behavior stream formed by collecting continuous user operations at the device end, extracting features of a single behavior in the original behavior stream according to a pre-established behavior feature library, and obtaining a pre-processed behavior stream; Extract behavior data from the preprocessed behavior stream and input it into the trained behavior recognition model to obtain a preliminary classification result. When the confidence of a behavior in the preliminary classification result is lower than a preset threshold, the behavior category is determined to be ambiguous. The activation values ​​of each neuron in the behavior recognition model calculated for the input behavior data are used to obtain several neurons with high contribution to the classification result. Based on the neurons with high contribution, we trace back to the input layer to obtain the highly sensitive feature areas. Then, we use the pruning algorithm to prune the neurons below the activation threshold in the behavior recognition model to obtain a streamlined model.

9. An artificial intelligence offline retail system according to claim 8, characterized in that: After obtaining the streamlined model, it also includes: quantifying the network parameters of each layer in the streamlined model through a compression algorithm, determining the compressed model, and judging the behavior category according to the classification result output by the compressed model; when the behavior category belongs to the preset high-risk category, the early warning mechanism is activated, and the compressed model is used to calculate the high-sensitive feature area, and the classification result output by the compressed model is obtained to obtain a high-risk behavior warning; according to the high-risk behavior warning, the corresponding security strategy is triggered, and the user behavior is analyzed according to the high-risk behavior warning output by the compressed model; and then the user behavior pattern is judged according to the historical behavior data to obtain the user behavior risk level.

10. An artificial intelligence offline retail method, used in an artificial intelligence offline retail system as claimed in any one of claims 1 to 9, characterized in that: The steps include: Adopting a deep learning-based target detection algorithm to locate and track customers in real time, obtain their location and movement trajectory information, and build a preliminary profile of them based on their appearance characteristics; Based on the customer's preliminary portrait, the posture estimation algorithm analyzes the customer's body posture and movements, determines the customer's behavior status, and combines the interaction information between the customer and the product to infer the customer's shopping intention and preference; Through the graph neural network algorithm, the customer's video, image and trajectory multimodal data are correlated and analyzed to build a more comprehensive and accurate customer portrait. The knowledge graph technology is used to associate the customer portrait with the product and scene information to form a semantic understanding of user behavior. Based on the semantic understanding of user behavior, we use federated learning to perform distributed storage and computing, data sharing and collaborative analysis among multiple shopping malls, and introduce differential privacy protection mechanisms to desensitize and add noise to customers' sensitive information. According to the dynamic changes in customer behavior, an incremental learning strategy is used to build a neural network model based on deep learning. The weights and biases are updated by continuously learning new data, and the model is updated and optimized in real time. Then, an active learning strategy is used to collect and annotate key samples in a targeted manner. The model is optimized through model compression and pruning technology to perform real-time behavior analysis and decision optimization, and the decision-making process of the model is visualized through the model interpretation mechanism.

Images