Information technology consultation method based on big data and related equipment thereof

Abstract

The invention belongs to the technical field of package data analysis, and particularly provides an information technology consultation method based on big data and related equipment thereof, and the method mainly comprises the steps: dividing a store into a plurality of commodity regions; acquiring video data, identifying a specified customer from the video data, acquiring a behavior record of the specified customer in each commodity area, and acquiring a historical shopping record of the specified customer; identifying the behavior of checking the commodity by the customer, and recording the duration; analyzing a shopping record of a specified customer in the commodity area in history, and calculating a shopping stability factor; quantifying the interest value of the specified customer on the commodity area; and replying the consultation content of the specified customer according to the interest value. According to the method, the store is divided into the different commodity areas, the monitoring videos in the store are analyzed, the interest values of the customer for the different commodity areas are evaluated, and the consultation content is replied according to the interest values, so that the personalized requirements of the customer can be met, and the service quality and competitiveness of the store can be improved.

Description

Technical Field

[0001] The present invention belongs to the technical field of information technology consulting, and in particular relates to an information technology consulting method based on big data and related equipment. Background Art

[0002] Information consulting technology is a combination of information technology and professional knowledge to provide information services to customers. Information is collected through search engines and database retrieval; analysis uses statistics and machine learning algorithms to mine value and predict trends; processing and storage rely on cloud computing and big data technology; display and transform information using data visualization tools. It also covers consulting service technologies such as demand analysis and project management to help various industries make scientific decisions.

[0003] Nowadays, in order to optimize customers' shopping experience, many large stores have set up consulting equipment. These equipment can provide customers with more convenient services when customers want to know relevant information about the products in the store. However, it cannot be ignored that these consulting equipment have obvious shortcomings. They lack customer information, which makes it difficult to provide targeted responses to consulting content based on customers' past consumption preferences. Summary of the invention

[0004] This application provides an information technology consulting method based on big data and related equipment, which effectively solves the problem in the prior art that the consulting equipment in large stores lacks customer information, making it difficult to provide targeted responses to consulting content based on customers' past consumption preferences. By dividing the store into different commodity areas and analyzing the surveillance videos in the store, the customer's interest value in different commodity areas is evaluated, and the consulting content is responded to based on the interest value, the personalized needs of customers are met, which helps the store improve its service quality and competitiveness.

[0005] In order to achieve the above object, the present invention adopts the following technical solutions: In a first aspect, the present application provides an information technology consulting method based on big data, comprising: According to the types of goods, the store is divided into multiple product areas; video data is obtained, a designated customer is identified from the video data, the behavior record of the designated customer in each product area is obtained, and the historical shopping record of the designated customer is obtained; for each product area: the behavior of the designated customer viewing the goods is identified from the behavior record, and the duration is recorded; according to the historical shopping record of the designated customer, the historical shopping record of the designated customer in the product area is analyzed, and the shopping stability factor is calculated; according to the duration and the stability factor, the interest value of the designated customer in the product area is quantified; and the consultation content of the designated customer is responded to according to the interest value.

[0006] Furthermore, the store is divided into multiple commodity areas, including: extracting key frame images from video data; using data annotation to determine the range of each commodity area in the key frame image, and defining a commodity category label for each commodity area; obtaining video stream data, and using the trained improved target detection model to process the video stream data, identifying each commodity area in the image in the video stream data, and outputting the corresponding commodity category label.

[0007] Furthermore, on the basis of the YOLOv5 model, a video repetitive region recognition module and an improved multimodal fusion module are added to obtain the improved target detection model, which includes: an input layer, which is used to receive video stream data and scale the image in the video frame; a video repetitive region recognition module, which is used to extract image features using MobileNet, match them by calculating the cosine similarity between feature vectors, and generate a repetitive region mask; a backbone network, which extracts the basic features of the image with repetitive region mask information and outputs multi-scale basic features; an attention mechanism module, which is used to screen and weight the multi-scale basic features, and use channel and spatial attention mechanisms to generate enhanced aggregation focus feature; the neck network includes SPP and PAN, SPP is used to perform multi-scale pooling operation on the enhanced focus feature, fuse the feature information at multiple scales, and generate scale fusion features, PAN is used to receive scale fusion features and multi-scale basic features, pass the underlying feature detail information from bottom to top, pass the high-level feature semantic information from top to bottom, and fuse the two to output path aggregation features; the multimodal fusion module is used to receive infrared sensor features, and weightedly fuse the path aggregation features, repeated area masks and infrared sensor features to generate comprehensive multimodal features; the output layer is used to generate a bounding box with coordinates, target confidence and category probability of the supermarket product area based on the comprehensive feature information.

[0008] Furthermore, the behavior record of the designated customer in each commodity area is obtained, including: using a human key point detection algorithm to detect the human key points of the designated customer, extracting the key point coordinates of the feet of the designated customer, selecting one of the key points of the foot and marking it as a reference point; determining a bounding box where the key point is located, and obtaining the commodity area where the designated customer is located; and classifying the video data according to the commodity area where the designated customer is located, and obtaining a sub-video of the designated customer in each commodity area.

[0009] Furthermore, the behavior of the designated customer viewing the product is identified from the behavior record, and the duration is recorded, including: dividing the sub-video into a continuous video frame sequence, and preprocessing the video frame sequence; marking a behavior label for each video frame, and the behavior label includes: viewing the product, not viewing the product; marking the start frame and the end frame of the viewing product behavior in the sub-video; dividing the continuous video frames into a training set and a test set, and using the training set and the test set to train the 3D convolutional neural network; using the trained 3D convolutional neural network to process the preprocessed video frame sequence, and outputting the behavior label of each video frame; checking each video frame in the order of the video frames, marking the start frame and the end frame of the video frame labeled as viewing the product, and recording the duration; summarizing the recorded duration to obtain the time period for the designated customer to view the product.

[0010] Furthermore, based on the historical shopping records of the designated customer, the customer's historical shopping records in the commodity area are analyzed to calculate a shopping stability factor, including: for each commodity area, based on the historical shopping records, calculating the designated customer's shopping frequency within a certain time period, the average number of commodity types purchased per shopping, and the volatility of the consumption amount; standardizing the shopping frequency, the average number of commodity types purchased per shopping, and the volatility of the consumption amount; assigning weights to the difference between the shopping frequency, the average number of commodity types purchased per shopping, and one minus the volatility of the consumption amount, and performing weighted summation, and marking the result of the weighted summation as the stability factor.

[0011] Furthermore, based on the duration and stability factor, the interest value of the specified customer in the commodity area is quantified, including: normalizing the data of the duration and stability factor; using the entropy weight method to assign weights to the duration and stability factor after data normalization and taking the weighted sum, and marking the calculation result as the interest value.

[0012] Furthermore, replying to the consultation content of the designated customer is performed according to the interest value, including: determining the designated customer who is consulting based on the video data and marking him as a consulting customer; obtaining the consulting content of the consulting customer; determining the keywords of each product and extracting the keywords from the consulting content using natural language processing technology; determining the interest value of the consulting customer based on the keywords; selecting a matching reply template from a preset response strategy library based on the interest value; and generating and outputting the consultation reply content based on the reply template.

[0013] Furthermore, determining the interest value of the consulting customer based on the keyword includes: constructing a mapping table of product areas and keywords, including: associating and storing each product area with the keywords of the products in the product area; recording the interest value of each designated customer for each product area, and generating a correspondence table including the designated customer, product area, and interest value; determining the product area consulted by the consulting customer from the keyword mapping table based on the keyword; and determining the interest value of the consulting customer for the product area consulted by the consulting customer from the correspondence table based on the consulted product area.

[0014] Furthermore, the reply template includes the following generation rules: setting a first threshold, a second threshold and a third threshold; when the interest value is higher than the first threshold, triggering an active recommendation mechanism and inserting real-time inventory data; when the interest value is in the second threshold range, generating a product parameter comparison table and a historical purchase analysis report; when the interest value is lower than the third threshold, providing basic product information and attaching interest-guiding words.

[0015] In the second aspect, the present application provides an information technology consulting system based on big data, which adopts an information technology consulting method based on big data as described in the first aspect, including: a data acquisition module, an intermediate data calculation module, an interest value generation module and a reply module.

[0016] The data acquisition module is used to divide the store into multiple product areas according to the types of goods; obtain video data, identify specified customers from the video data, obtain the behavior records of the specified customers in each product area, and obtain the historical shopping records of the specified customers.

[0017] The intermediate data calculation module is used for each commodity area: identifying the behavior of the designated customer viewing commodities from the behavior record and recording the duration; analyzing the designated customer's historical shopping records in the commodity area based on the designated customer's historical shopping records, and calculating the shopping stability factor.

[0018] The interest value generating module is used to quantify the interest value of a specified customer to the commodity area according to the duration and the stability factor.

[0019] The reply module is used to reply to the consultation content of the designated customer according to the interest value.

[0020] In a third aspect, the present application provides a device comprising a memory and a processor; the memory is used to store a computer program; the processor is used to implement the steps of the big data-based information technology consulting method as described in the first aspect when executing the computer program.

[0021] In a fourth aspect, the present application provides a readable storage medium, in which computer program instructions are stored. When the computer program instructions are read and executed by a processor, the steps of the big data-based information technology consulting method as described in the first aspect are executed.

[0022] Beneficial effects of the present invention: This application divides the store into different merchandise areas, analyzes surveillance videos in the store, evaluates customers' interest in different merchandise areas, and responds to inquiries based on the interest values. This effectively solves the problem in the prior art that the consulting equipment in large stores lacks customer information, making it difficult to provide targeted responses to inquiries based on customers' past consumption preferences. This application can meet customers' personalized needs and help stores improve their service quality and competitiveness.

[0023] Other features and advantages of the present invention will be described in the following description, and partly become obvious from the description, or be understood by implementing the present invention. The purpose and other advantages of the present invention can be realized and obtained by the structures pointed out in the description and the drawings. BRIEF DESCRIPTION OF THE DRAWINGS

[0024] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0025] Figure 1 A schematic diagram of a process of an information technology consulting method based on big data in Embodiment 1 of the present invention is shown; Figure 2 A schematic diagram of a process of replying to the consultation content of the designated customer according to the interest value in the first embodiment of the present invention is shown; Figure 3 A module diagram of an information technology consulting system based on big data in the second embodiment of the present invention is shown. DETAILED DESCRIPTION

[0026] In order to solve the problems raised by the background technology, this application divides the store into different product areas, analyzes the surveillance videos in the store, evaluates the customer's interest in different product areas, and responds to inquiries based on the interest value, thereby meeting the customer's personalized needs and helping the store improve its service quality and competitiveness.

[0027] In order to make the purpose, technical solution and advantages of the embodiments of the present invention clearer, the technical solution in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0028] Embodiment 1: like Figure 1 - Figure 2 As shown, this embodiment provides an information technology consulting method based on big data, including: S100. Divide the store into multiple commodity areas according to the types of commodities; obtain video data, identify a specified customer from the video data, obtain the behavior record of the specified customer in each commodity area, and obtain the historical shopping record of the specified customer.

[0029] S200. For each commodity area: identify the behavior of a designated customer viewing commodities from the behavior record and record the duration; analyze the designated customer's historical shopping record in the commodity area based on the designated customer's historical shopping record and calculate the shopping stability factor.

[0030] S300. Evaluate and quantify the interest value of a specified customer in a product area according to the duration and stability factor.

[0031] S400. Reply to the inquiry content of the designated customer according to the interest value.

[0032] If the store is a supermarket, it can be divided into fresh food area, daily necessities area, etc. The video data comes from the customer data collected by various cameras in the store. The image recognition technology is used to accurately identify the specified customer from these data, and then obtain their behavior records in various commodity areas, such as the trajectory of their stay and movement in the clothing area. The customer's historical shopping records can be obtained from the store sales database.

[0033] For each product area, identify the customer's product viewing behavior from the behavior records, such as where the customer's eyes linger, pick up the product, and other actions, and record their duration. Based on the historical shopping records, analyze the customer's purchase frequency and interval length in a certain product area, calculate the shopping stability factor, and quantitatively evaluate the customer's interest in the product area based on the duration and stability factor. For example, a high interest value indicates that the customer is more interested in the products in this area.

[0034] Finally, the system gives recommendations and answers that suit their preferences based on the interest value, thereby effectively improving the customer consultation experience.

[0035] In S100, the store is divided into multiple merchandise areas, including: S110. Extract key frame images from video data.

[0036] S120. Determine the range of each commodity area in the key frame image by using data annotation, and define a commodity category label for each commodity area.

[0037] S130. Obtain video stream data, and use the trained improved target detection model to process the video stream data, identify each commodity area in the image in the video stream data, and output the corresponding commodity category label.

[0038] The cameras in the store are used to continuously collect video data under different time periods, different lighting conditions, and different customer flows.

[0039] Use professional image annotation tools such as LabelImg to annotate product areas in video frames, mark each product area (such as fresh food area, food area, daily necessities area, etc.) with a rectangular box, and annotate each box with the corresponding area category label. In addition, annotate repeated areas in the field of view of different cameras and use special labels to distinguish them for model learning.

[0040] The labeled data can be divided into training set, validation set and test set in the ratio of 70%, 15% and 15%, which are used for model training, parameter adjustment and final performance evaluation respectively.

[0041] Based on the YOLOv5 model, a video repetitive region recognition module and an improved multimodal fusion module are added to solve the problem of repetitive region recognition in videos from different cameras. The model includes the input layer, backbone network, attention mechanism module, neck network, output layer of the basic YOLOv5 model, as well as the newly added video repetitive region recognition module and the extended multimodal fusion module. The video repetitive region recognition module is located after the input layer, and the extended multimodal fusion module is located after the neck network.

[0042] The input layer is used to receive video stream data and resize the images in the video frames.

[0043] The input layer includes an adaptive scale adjustment module, which is used to analyze the video frames input from different cameras and dynamically adjust the image scaling according to the size and distribution of the product areas in the video frames. For example, if most of the product areas in the image are small, the module will appropriately enlarge the image so that the model can capture the features of the small area more clearly; conversely, if the area is large, the image will be appropriately reduced to increase the detection speed. This ensures that product areas of different sizes can be detected by the model at an appropriate scale, improving the model's adaptability to different scenarios.

[0044] The video repeated region recognition module is used to extract image features using MobileNet, match them by calculating the cosine similarity between feature vectors, and generate repeated region masks.

[0045] The video repeated region recognition module is mainly used to process repeated regions in videos shot by different cameras. It includes: feature extraction layer, feature matching layer and repeated region determination layer. The feature extraction layer uses a lightweight MobileNet network to extract features from scaled video frames. The feature matching layer is used to divide the extracted feature map into multiple small blocks, and find possible repeated regions by calculating the cosine similarity of the corresponding small block feature vectors in different video frames. The repeated region determination layer is used to determine the boundaries of the repeated region according to the set similarity threshold, and use the non-maximum suppression algorithm to remove the repeated region candidate frames with too high overlap, and finally generate the mask M of the repeated region.

[0046] The backbone network extracts the basic features of the image with repeated region mask information and outputs multi-scale basic features.

[0047] The backbone network includes a channel attention module and a spatial attention module. The channel attention module learns the importance of each channel feature, weights the channel features, enhances the expression of key channel features, and suppresses unimportant channels. The spatial attention module focuses on the spatial position of the target in the feature map, highlights the features of the target area, and ignores irrelevant information such as the background. This helps the model focus on the product area more accurately, such as highlighting the features of the products on the shelf, and improves the ability to recognize different areas in complex backgrounds.

[0048] The neck network includes SPP and PAN. SPP is used to perform multi-scale pooling operations on the enhanced focus features, fuse the feature information at multiple scales, and generate scale fusion features. PAN is used to receive scale fusion features and multi-scale basic features, pass the underlying feature detail information from bottom to top, pass the high-level feature semantic information from top to bottom, and fuse the two to output path aggregation features.

[0049] The neck network includes the SPP (spatial pyramid pooling) module and the PAN (path aggregation network). The SPP module integrates multi-scale features through pooling operations at different scales. For example, if the product areas are of different sizes, the SPP module can capture feature information at different scales and enhance the model's ability to detect targets of different sizes. PAN can fuse the upper and lower paths of features, fusing the underlying features transmitted from the bottom to the top of the backbone network with the high-level features transmitted from the top to the bottom, enhancing feature transmission and further improving the ability to detect small targets.

[0050] The multimodal fusion module is used to receive infrared sensor features and perform weighted fusion of path aggregation features, repeated area masks and infrared sensor features to generate comprehensive multimodal features.

[0051] The multimodal fusion module is used to transform the image features output by the backbone network , multimodal data (such as infrared sensor data ) The processed features and the mask M output by the video repeated area recognition module are fused. The specific structure and workflow are as follows: Feature concatenation layer: concatenates the above three features to form a higher-dimensional feature vector and integrates multi-source information.

[0052] Fully connected fusion layer: The concatenated features are fused through the fully connected layer, high-dimensional features are mapped to low-dimensional space, and more representative features are extracted to provide richer and more accurate information for subsequent target detection and classification.

[0053] If the fused feature is ,but , where α, β, and γ represent learnable weight coefficients, and their optimal values ​​are determined through training to balance the contribution of different modal data. The distribution characteristics of representatives in different areas of the supermarket, such as the density of people in the fresh food area, food area, etc.

[0054] The output layer is used to generate a bounding box with coordinates, target confidence, and category probability of the supermarket product area based on the comprehensive feature information.

[0055] The output layer is used to output the prediction results, including the bounding box coordinates, target confidence, and category probability of the supermarket product area. The bounding box coordinates are used to accurately mark the location and range of the product area on the image; the target confidence reflects the reliability of the detection result, and a threshold (such as 0.5) can be set to filter out low-confidence detection results; the category probability is used to determine the product category to which the area belongs (such as fresh food area, food area, daily necessities area, etc.).

[0056] Stochastic gradient descent can be used as the optimization algorithm, combined with the cosine annealing learning rate adjustment strategy. The cosine annealing learning rate adjustment strategy can use a larger learning rate in the early stage of training to speed up the convergence of the model. As the training progresses, the learning rate is gradually reduced, so that the model can adjust parameters more finely and improve the training effect.

[0057] The training steps include: S131. Install Python, PyTorch deep learning framework and related dependent libraries to ensure that the environment can support model training and operation.

[0058] S132. Initialize the model, including: loading the pre-trained YOLOv5 weights, and randomly initializing the parameters of the newly added modules (adaptive scale adjustment module, video repeated area recognition module, multimodal fusion module, etc.). The pre-trained weights can provide a good initial state for the model and accelerate the convergence speed of the model.

[0059] S133. Set training parameters, such as: determine the number of training rounds to be 300, the initial learning rate to be 0.01, and the batch size to be 16. These parameters can be adjusted according to actual conditions to achieve the best training effect.

[0060] S134. A batch of supermarket video frames from different cameras, corresponding infrared sensor data and annotation information are read from the training set. The video frames are first scaled by the adaptive scale adjustment module. The adjusted video frames are input into the video repeated area recognition module to obtain repeated area masks. The video frames are extracted from the supermarket image features by the backbone network. The feature expression is enhanced by the attention mechanism module after the partial convolution layer. The neck network performs multi-scale fusion of the features. The supermarket image features, infrared sensor data features and repeated area masks are input into the multimodal fusion module for fusion. The fused features are input into the output layer for target detection and classification prediction. The loss between the prediction result and the true value of the annotation is calculated. The loss function provided by YOLOv5 is used, including bounding box loss, confidence loss and classification loss.

[0061] After every 10 rounds of training, the performance of the model is evaluated on the validation set, the changes in indicators such as loss value and mean average precision (mAP) are observed, and the parameters are adjusted according to the evaluation results.

[0062] After training is completed, the video stream data is input into the model. After processing, the model can output the various product areas in each frame image and the corresponding product category labels.

[0063] Get the behavior records of the specified customer in each product area, including: S140. Use a human body key point detection algorithm to detect the human body key points of the designated customer, extract the key point coordinates of the designated customer's feet, select one of the key points of the foot, and mark it as a reference point.

[0064] S150. Determine the bounding box where the key point is located, and obtain the product area where the specified customer is located.

[0065] S160. Classify the video data according to the commodity area where the designated customer is located, and obtain a sub-video of the designated customer in each commodity area.

[0066] The human key point detection algorithm is used to detect the human key points of the designated customer. Specifically, a pre-trained human key point detection model can be used, such as the OpenPose model based on a convolutional neural network. The video stream data output by the improved target detection model is decoded into continuous image frames, and the image frames are pre-processed. The pre-processed image frames are input into the model in sequence, and finally the probability map of each key point of the human body is output. For each output probability map, for the key points of both feet, the position with the largest probability value is found.

[0067] From the extracted coordinates of the key points of the two feet, any key point of the foot is selected as a reference point and displayed on the image frame to determine which bounding box it is located in, thereby determining the specific product area.

[0068] After the commodity areas where the designated customers are located are identified, the video data are classified according to these areas. By traversing all the video data, the video clips containing the designated customer behaviors in the same commodity area are screened out to form a sub-video of the area.

[0069] In S200, the behavior of a designated customer viewing a product is identified from the behavior record, and the duration is recorded, including: Sa210. Split the sub-video into continuous video frame sequences and pre-process the video frame sequences.

[0070] The acquired sub-videos of each product area are divided into a continuous sequence of video frames. Each video frame represents the behavior of the customer in that area at a certain moment. The video frame sequence is preprocessed, including resizing to the size required by the model and normalizing the pixel values.

[0071] Sa220. Mark each video frame with a viewing behavior label, where the behavior labels include: viewing a product, not viewing a product; and marking the start frame and end frame of the viewing product behavior in the sub-video.

[0072] Sa230. Divide the continuous video frames into training set and test set, and use the training set and test set to train the 3D convolutional neural network.

[0073] The continuous video frames are divided into training sets and test sets. The training set is used to train the 3D convolutional neural network, allowing the model to learn the characteristic patterns of customers' behavior of viewing products in video frames; the test set is used to evaluate the performance of the trained model and determine whether the model can accurately identify behaviors in unseen video frames. Through a large amount of training data, the model gradually grasps the video frame features corresponding to different behaviors, such as the performance features of customers' hands reaching for products and their eyes focusing on products in video frames.

[0074] Sa240. uses a trained 3D convolutional neural network to process the preprocessed video frame sequence and outputs the behavior label of each video frame.

[0075] The trained 3D convolutional neural network is used to process the preprocessed video frame sequence. The model analyzes each video frame and outputs the behavior label corresponding to each video frame based on the learned feature pattern to determine whether the customer is viewing the product in the video frame.

[0076] Sa250. Check each video frame in the order of the video frames, mark the start frame and the end frame of the video frame labeled as viewing the product, and record the duration.

[0077] Check each video frame in sequence. Once a video frame with the label "view product" is identified, mark its start frame. When the label no longer appears, mark the corresponding end frame. By recording the position of the start frame and the end frame in the video frame sequence and combining the video frame rate, the duration of this product viewing behavior can be calculated.

[0078] Sa260. Summarize the recorded durations to obtain the time period when the specified customer viewed the product.

[0079] Summarize the duration of each previously recorded product viewing behavior segment. Add up the duration of these segments to finally get the complete time period that the specified customer viewed the product in the product area.

[0080] In S200, according to the historical shopping record of the designated customer, the customer's historical shopping record in the commodity area is analyzed to calculate the shopping stability factor, including: Sb210. For each commodity area, based on historical shopping records, calculate the shopping frequency of a specified customer within a certain period of time, the average number of commodity types purchased per shopping, and the volatility of the consumption amount.

[0081] The camera can be used to identify the designated customer at checkout, and the identity of the designated customer can be associated with the corresponding shopping record to establish a mapping relationship, which is convenient for subsequent data analysis.

[0082] For each product area, let N be the number of times a given customer makes purchases within a certain time period T, then the shopping frequency f is: ; Shopping frequency can be used to measure how frequently customers purchase products in this product area during that time period.

[0083] If within the time period T, the number of types of goods purchased by customers are , the average number of commodity types m purchased each time is: The average number of commodity types purchased each time shopping reflects the richness of commodity choices made by customers each time they shop in this area.

[0084] Suppose that a customer shops N times in the product area within a time period T, and the amount of money spent on each shopping trip is , average consumption amount for: ; Then calculate the standard deviation of the consumption amount according to the standard deviation formula for: ; The larger the standard deviation, the greater the fluctuation of the consumption amount, that is, the consumption amount of customers in this product area is unstable.

[0085] Sb220. Standardize the volatility of shopping frequency, average number of items purchased per shopping trip, and spending amount.

[0086] Sb230. Assign weights to the shopping frequency, the average number of types of goods purchased per shopping trip, and the difference between one and the volatility of the consumption amount, and perform the weighted summation, and mark the result of the weighted summation as the stability factor.

[0087] The stability factor S is: ;in, , , is the weight, and can be , , , Represent f, m, After standardization, the stability factor comprehensively considers the stability of shopping frequency, the average number of items purchased per shopping trip, and the amount of consumption.

[0088] In S300, the interest value of the designated customer in the commodity area is evaluated and quantified according to the duration and stability factor, including: S310. Perform data normalization on duration and stability factors.

[0089] S320. Use the entropy weight method to assign weights to the duration and stability factors after data normalization and take weighted sum, and mark the calculation results as interest values.

[0090] The weight is determined by calculating the information entropy of duration and stability factor S. The calculation formula is: , where , n represents the number of customers, , The normalized value of the jth indicator (j is 1 for duration and 2 for stability factor) of the i-th customer is used to obtain the information entropy of the duration indicator. Information entropy of the stability factor , and then get the weight of duration according to the entropy weight calculation formula and the weight of the stability factor , the interest value can be obtained after weighted summation. The interest value comprehensively considers the customer's behavior duration and shopping stability in the product area, and can more objectively quantify the customer's interest in the product area.

[0091] In S400, responding to the inquiry content of the designated customer according to the interest value includes: S410. Determine the designated customer being consulted based on the video data and mark the customer as a consulting customer.

[0092] S420. Obtain the consulting content of the consulting client.

[0093] S430. Determine the keywords for each product and extract the keywords from the consultation content using natural language processing technology.

[0094] S440. Determine the interest value of the consulting customer based on the keywords.

[0095] S450. Select a matching reply template from a preset response strategy library according to the interest value.

[0096] S460. Generate and output the consultation response content according to the response template.

[0097] When a customer initiates a consultation, the identity of the customer currently being consulted must first be clarified. The customer's image can be obtained through a camera, and the customer's identity can be identified through image recognition technology.

[0098] For each product in the store, pre-set keywords that can represent its core features. For example, for a smartphone, keywords may include "brand name", "model", "processor performance", "camera pixels", etc. Use the text analysis algorithm in natural language processing technology to process the consulting content of consulting customers. Common technologies include lexical analysis, syntactic analysis, and semantic analysis. Through these technologies, keywords related to the product are extracted from the consulting content. For example, if the consulting content is "How is the camera effect of a new brand of mobile phone", keywords such as "a certain brand", "mobile phone", and "photographic effect" can be extracted.

[0099] Based on the previously calculated customer interest values ​​in different product areas, combined with the extracted keywords, the consulting customer's interest value in the product area involved in this consultation is determined. For example, if the extracted keywords are mainly related to the electronic product area, the system will look up the customer's interest value in the electronic product area.

[0100] The preset response strategy library stores response templates for different interest value ranges and keyword combinations. These templates are pre-formulated based on a large amount of historical consultation data and customer feedback, and are designed to provide the most appropriate responses that best meet customer needs. The system matches the determined interest value and extracted keywords in the response strategy library to find the corresponding response template.

[0101] After finding a matching reply template, the system will fill in the relevant information in the consultation content, such as the specific product name and model mentioned by the customer, into the corresponding position in the reply template. After the information is filled in, a complete consultation reply is generated and presented to the consulting customer through the output interface of the consulting device, such as screen display or voice broadcast. The customer can get a personalized reply for his or her consultation content and taking into account his or her interests and preferences, thereby improving the customer's consultation experience and satisfaction.

[0102] In S440, determining the interest value of the consulting customer according to the keyword includes: S441. Constructing a mapping table of commodity areas and keywords, including: associating and storing each commodity area with the keywords of the commodities in the commodity area; recording the interest value of each designated customer for each commodity area, and generating a corresponding relationship table including designated customers, commodity areas, and interest values; S442. Determine the product area consulted by the consulting customer from the keyword mapping table according to the keyword; S443. Determine the interest value of the consulting customer in the commodity area he / she is consulting from the corresponding relationship table according to the commodity area he / she is consulting.

[0103] For each product area in the store, such as the electronics area, clothing area, food area, etc., the keywords of all products in the area are summarized and sorted. For example, in the electronics area, products such as smart phones have keywords such as "brand name", "model", "processor performance", "camera pixels", etc.; computer products have keywords such as "CPU model", "memory capacity", "graphics card type", etc. These product areas are associated with the corresponding keywords and stored to form a clear mapping relationship.

[0104] After obtaining each customer's interest value in different product areas, organize this information into a corresponding relationship table.

[0105] After extracting keywords from the consulting customer's consulting content, the product area consulted by the consulting customer is determined based on the constructed product area and keyword mapping table. After determining the product area consulted by the consulting customer, further query the correspondence table containing the specified customer, product area, and interest value, so as to quickly determine the customer's interest value.

[0106] The reply template contains the following generation rules: A first threshold, a second threshold, and a third threshold are set.

[0107] When the interest value is above a first threshold, the active recommendation mechanism is triggered and real-time inventory data is inserted.

[0108] When the interest value is within the second threshold range, a product parameter comparison table and a historical purchase analysis report are generated.

[0109] When the interest value is lower than the third threshold, basic product information is provided and interest-guiding words are attached.

[0110] The first threshold can be set to a relatively high value, for example, assuming it is 0.8. When the system determines that the consulting customer's interest value in the inquired product area is higher than the first threshold, it indicates that the customer has shown a very high interest in the products in this area; the second threshold interval: for example, it can be set to 0.4-0.6. When the interest value is in this interval, it means that the customer has a certain interest in the product, but needs more information to assist in decision-making; the third threshold can be set to a relatively low value, such as 0.2. When the interest value is lower than the third threshold, it means that the customer has little interest in the product area.

[0111] The specific values ​​of the first threshold, the second threshold and the third threshold can be determined based on actual business data and analysis.

[0112] When the interest value is higher than the first threshold, the active recommendation mechanism is triggered and real-time inventory data is inserted. For such customers who show great interest in the product area, active recommendations of products that match their preferences can accurately meet their potential needs and save customers time and energy in searching for products on their own. The provision of real-time inventory data allows customers to grasp the purchase availability of products in a timely manner, improve the timeliness of purchase decisions, and greatly improve the conversion rate from consultation to purchase.

[0113] For customers whose interest values ​​are in the second threshold range, a product parameter comparison table and a historical purchase analysis report are generated. The product parameter comparison table can help customers clearly understand the differences between different products and assist them in making more rational purchase decisions. The historical purchase analysis report provides customers with more comprehensive product information from multiple aspects such as time dimension and customer group characteristics, meeting their needs for in-depth understanding of products and further enhancing customers' understanding of products and willingness to purchase.

[0114] For customers whose interest value is lower than the third threshold, basic product information is provided and interest-guiding words are attached. The basic product information concisely and clearly answers the customer's initial questions, while the interest-guiding words try to tap into the customer's potential needs, guide the customer to pay attention to more related products, and gradually increase the customer's interest in the product area, laying the foundation for possible subsequent purchase behavior.

[0115] Embodiment 2: like Figure 3 As shown, this embodiment provides an information technology consulting system based on big data, including: a data acquisition module, an intermediate data calculation module, an interest value generation module and a reply module.

[0116] The data acquisition module is used to divide the store into multiple commodity areas according to the types of commodities; obtain video data, identify the specified customer from the video data, obtain the behavior record of the specified customer in each commodity area, and obtain the historical shopping record of the specified customer. The intermediate data calculation module is used for each commodity area: identifying the behavior of the specified customer viewing the commodity from the behavior record and recording the duration; analyzing the historical shopping record of the specified customer in the commodity area according to the historical shopping record of the specified customer, and calculating the shopping stability factor. The interest value generation module is used to evaluate and quantify the interest value of the specified customer in the commodity area according to the duration and stability factor. The reply module is used to reply to the consultation content of the specified customer according to the interest value.

[0117] This embodiment has all the advantages of the information technology consulting method based on big data in the first embodiment, and can automatically execute the steps of the information technology consulting method based on big data.

[0118] Embodiment three: This embodiment provides a device, which includes a memory and a processor; the memory is used to store a computer program; the processor is used to implement the steps of the information technology consulting method based on big data in the first embodiment when executing the computer program.

[0119] Embodiment 4: This embodiment provides a readable storage medium, in which computer program instructions are stored. When the computer program instructions are read and executed by a processor, the steps of the information technology consulting method based on big data in the first embodiment are executed.

[0120] Among them, any reference to memory, storage, database or other media used in the embodiments provided by the present invention may include non-volatile and / or volatile memory. Non-volatile memory may include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM) or flash memory. Volatile memory may include random access memory (RAM) or external cache memory.

[0121] It should be noted that, in this article, relational terms such as "first" and "second" are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or elements inherent to such process, method, article or device. In the absence of further restrictions, the elements defined by the sentence "comprise a ..." do not exclude the existence of other identical elements in the process, method, article or device including the elements.

[0122] Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent substitutions for some of the technical features therein; and these modifications or substitutions do not make the essence of the corresponding technical solutions deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. An information technology consulting method based on big data, characterized in that: include: Divide the store into multiple merchandise areas based on the types of merchandise; Obtain video data, identify a specified customer from the video data, obtain the behavior record of the specified customer in each product area, and obtain the historical shopping record of the specified customer; For each commodity area: identifying the behavior of the designated customer viewing commodities from the behavior record, and recording the duration; According to the historical shopping records of the designated customer, the historical shopping records of the designated customer in the commodity area are analyzed to calculate the shopping stability factor; quantifying the interest value of the specified customer in the commodity area according to the duration and stability factor; The inquiry content of the designated customer is replied according to the interest value.

2. The method according to claim 1, characterized in that According to the types of goods, the store is divided into multiple product areas, including: Extract key frame images from video data; Data annotation is used to determine the range of each commodity area in the key frame image, and a commodity category label is defined for each commodity area; Obtain video stream data, and use the trained improved target detection model to process the video stream data, identify the various product areas in the image in the video stream data, and output the corresponding product category labels.

3. The method according to claim 2, characterized in that The improved target detection model includes: The input layer is used to receive video stream data and resize the images in the video frames; The video repeated region recognition module is used to extract image features using MobileNet, match them by calculating the cosine similarity between feature vectors, and generate repeated region masks; The backbone network extracts the basic features of the image with repeated region mask information and outputs multi-scale basic features; The attention mechanism module is used to filter and weight multi-scale basic features, and use channel and spatial attention mechanisms to generate enhanced focus features; The neck network includes SPP and PAN. SPP is used to perform multi-scale pooling operations on the enhanced focus features, fuse feature information at multiple scales, and generate scale fusion features. PAN is used to receive scale fusion features and multi-scale basic features, pass the underlying feature details from bottom to top, pass the high-level feature semantic information from top to bottom, and fuse the two to output path aggregation features. A multimodal fusion module is used to receive infrared sensor features and perform weighted fusion of path aggregation features, repeated area masks and infrared sensor features to generate comprehensive multimodal features; The output layer is used to generate a bounding box with coordinates, target confidence, and category probability of the supermarket product area based on the comprehensive feature information.

4. The method according to claim 3, characterized in that Get the behavior records of the specified customer in each product area, including: Using a human body key point detection algorithm to detect the human body key points of the designated customer, extracting the key point coordinates of the designated customer's feet, selecting one of the key points of the foot and marking it as a reference point; Determine the bounding box where the key points are located and obtain the product area where the specified customer is located; The video data is classified according to the commodity area where the designated customer is located, and a sub-video of the designated customer in each commodity area is obtained.

5. The method according to claim 4, characterized in that Identifying the behavior of the designated customer viewing the product from the behavior record and recording the duration, including: Splitting the sub-video into a continuous video frame sequence, and preprocessing the video frame sequence; Mark each video frame with a behavior tag, where the behavior tags include: viewing a product, not viewing a product; and mark the start and end frames of the product viewing behavior in the sub-video; The continuous video frames are divided into a training set and a test set, and the 3D convolutional neural network is trained using the training set and the test set; Use the trained 3D convolutional neural network to process the preprocessed video frame sequence and output the behavior label of each video frame; Check each video frame in order, mark the start and end frames of the video frame labeled as viewing the product, and record the duration; Summarize the recorded durations to obtain the time period when the specified customer viewed the product.

6. The method according to claim 5, characterized in that Analyze the customer's historical shopping records in the product area and calculate the shopping stability factor, including: For each product area, calculate the shopping frequency of a specified customer within a certain period of time, the average number of product types purchased per shopping trip, and the volatility of the spending amount based on historical shopping records; Standardize the volatility of shopping frequency, average number of items purchased per shopping trip, and amount spent; Assign weights to the shopping frequency, the average number of types of goods purchased per shopping trip, and the difference between one and the volatility of the spending amount, and perform a weighted sum, and mark the result of the weighted sum as a stability factor.

7. The method according to claim 1, characterized in that Responding to the inquiry content of the designated customer according to the interest value includes: Determine the designated customer being consulted based on the video data and mark the customer as a consulting customer; Obtain consulting content from consulting clients; Determine the keywords for each product and use natural language processing technology to extract keywords from the consultation content; Determining the interest value of the consulting customer according to the keywords; Selecting a matching reply template from a preset answer strategy library according to the interest value; Generate and output consultation reply content according to the reply template; The reply template contains the following generation rules: Setting a first threshold, a second threshold interval, and a third threshold; When the interest value is higher than a first threshold, the active recommendation mechanism is triggered and real-time inventory data is inserted; When the interest value is within the second threshold range, a product parameter comparison table and a historical purchase analysis report are generated; When the interest value is lower than the third threshold, basic product information is provided and interest-guiding words are attached.

8. An information technology consulting system based on big data, characterized in that: It includes: A data acquisition module, which is used to divide the store into multiple commodity areas according to the types of commodities; Obtain video data, identify a specified customer from the video data, obtain the behavior record of the specified customer in each product area, and obtain the historical shopping record of the specified customer; An intermediate data calculation module is used for: for each commodity area: identifying the behavior of the designated customer viewing the commodity from the behavior record, and recording the duration; According to the historical shopping records of the designated customer, the historical shopping records of the designated customer in the commodity area are analyzed to calculate the shopping stability factor; An interest value generating module, which is used to quantify the interest value of a specified customer in the commodity area according to the duration and stability factor; A reply module is used to reply to the consultation content of the designated customer according to the interest value.

9. An information technology consulting device based on big data, characterized in that: It includes a memory and a processor; the memory is used to store computer programs; the processor is used to implement the steps of the big data-based information technology consulting method described in any one of claims 1 to 7 when executing the computer program.

10. A readable storage medium, characterized in that: The readable storage medium stores computer program instructions, and when the computer program instructions are read and executed by a processor, the steps of the big data-based information technology consulting method described in any one of claims 1 to 7 are executed.

Images