An information technology consulting method based on big data and its related devices

Abstract

The present invention belongs to the technical field of packaging data analysis, and specifically provides an information technology consulting method based on big data and related devices. The method mainly includes: dividing a store into multiple commodity areas; obtaining video data, identifying a specified customer from the video data, obtaining the behavior records of the specified customer in each commodity area, and obtaining the historical shopping records of the specified customer; identifying the behavior of the customer viewing the commodity and recording the duration; analyzing the shopping records of the specified customer in this commodity area historically, and calculating the shopping stability factor; quantifying the interest value of the specified customer in the commodity area; and replying to the consulting content of the specified customer according to the interest value. By dividing the store into different commodity areas, analyzing the surveillance videos in the store, evaluating the interest values of customers in different commodity areas, and replying to the consulting content based on the interest values, the present application can meet the personalized needs of customers and help the store improve service quality and competitiveness.

Description

Technical Field

[0001] The present invention belongs to the technical field of information technology consulting, and particularly relates to an information technology consulting method based on big data and related devices. Background Art

[0002] Information consulting technology is a technology combination that integrates information technology and professional knowledge to provide information services to customers. Information collection relies on search engines, database retrieval, etc.; analysis uses statistical and machine learning algorithms to mine value and predict trends; processing and storage rely on cloud computing and big data technologies; display uses data visualization tools to transform information. It also covers consulting service technologies such as requirements analysis and project management to help industries make scientific decisions.

[0003] Nowadays, many large stores have set up consulting devices to optimize the shopping experience of customers. These devices can provide relatively convenient services for customers. When customers want to know information about the products in the store, it cannot be ignored that these consulting devices have obvious shortcomings. They lack customer information, resulting in difficulty in providing targeted responses to consulting content based on customers' past consumption preferences. Summary of the Invention

[0004] The present application effectively solves the problem in the prior art that the consulting devices in large stores lack customer information, resulting in difficulty in providing targeted responses to consulting content based on customers' past consumption preferences. By dividing the store into different product areas, analyzing the surveillance videos in the store, evaluating the interest values of customers in different product areas, and replying to the consulting content based on the interest values, it meets the personalized needs of customers and helps the store improve service quality and competitiveness.

[0005] To achieve the above object, the present invention adopts the following technical solutions:

[0006] In a first aspect, the present application provides an information technology consulting method based on big data, including:

[0007] Dividing the store into multiple product areas according to the types of products; obtaining video data, identifying a specified customer from the video data, obtaining the behavior records of the specified customer in each product area, and obtaining the historical shopping records of the specified customer; for each product area: identifying the behavior of the specified customer viewing products from the behavior records and recording the duration; analyzing the shopping records of the specified customer in this product area historically according to the historical shopping records of the specified customer, and calculating the shopping stability factor; quantifying the interest value of the specified customer in the product area according to the duration and the stability factor; and replying to the consulting content of the specified customer according to the interest value.

[0008] Further, the store is divided into multiple commodity areas, including: extracting key-frame images from video data; using data annotation to determine the scope of each commodity area in the key-frame images and defining commodity category labels for each commodity area; obtaining video stream data, and using the trained improved object detection model to process the video stream data, identifying each commodity area in the images of the video stream data, and outputting the corresponding commodity category labels.

[0009] Further, based on the YOLOv5 model, a video repeated area recognition module and an improved multi-modal fusion module are added to obtain the improved object detection model. The improved object detection model includes: an input layer, which is used to receive video stream data and perform scale adjustment on the images in the video frames; a video repeated area recognition module, which is used to extract image features by using MobileNet, perform matching by calculating the cosine similarity between feature vectors, and generate a repeated area mask; a backbone network, which extracts the basic features of the images with repeated area mask information and outputs multi-scale basic features; an attention mechanism module, which is used to screen and weight the multi-scale basic features, and generate enhanced focused features by using channel and spatial attention mechanisms; a neck network, which includes SPP and PAN. SPP is used to perform multi-scale pooling operations on the enhanced focused features, fuse the feature information at multiple scales, and generate scale fusion features. PAN is used to receive the scale fusion features and the multi-scale basic features, transmit the underlying feature details from bottom to top, transmit the high-level feature semantics from top to bottom, and fuse the two to output path aggregation features; a multi-modal fusion module, which is used to receive infrared sensor features, and perform weighted fusion on the path aggregation features, the repeated area mask and the infrared sensor features to generate comprehensive multi-modal features; an output layer, which is used to generate the bounding boxes with coordinates, object confidence levels and category probabilities of the supermarket commodity areas according to the comprehensive feature information.

[0010] Further, obtain the behavior records of the specified customer in each commodity area, including: using a human key-point detection algorithm to detect the human key points of the specified customer, extracting the key-point coordinates of the two feet of the specified customer, selecting one of the key points of the feet and marking it as a reference point; determining the bounding box where the key point is located to obtain the commodity area where the specified customer is located; classifying the video data according to the commodity area where the specified customer is located to obtain the sub-videos of the specified customer in each commodity area.

[0011] Further, identify the behavior of the specified customer viewing the product from the behavior record and record the duration, including: splitting the sub-video into a sequence of consecutive video frames and preprocessing the video frame sequence; viewing and annotating behavior labels for each video frame, where the behavior labels include: viewing the product, not viewing the product; annotating the start frame and end frame of the product viewing behavior in the sub-video; dividing the consecutive video frames into a training set and a test set, and training a 3D convolutional neural network using the training set and the test set; processing the preprocessed video frame sequence using the trained 3D convolutional neural network to output the behavior label of each video frame; checking each video frame in the order of the video frames, marking the start frame and end frame of the video frames with the label of viewing the product, and recording the duration; summarizing the recorded durations to obtain the time period during which the specified customer views the product.

[0012] Further, according to the historical shopping records of the specified customer, analyze the customer's historical shopping records in the product area and calculate the shopping stability factor, including: for each product area, calculate the shopping frequency, the average number of product types purchased per shopping, and the volatility of the consumption amount of the specified customer within a certain time period based on the historical shopping records; perform normalization processing on the shopping frequency, the average number of product types purchased per shopping, and the volatility of the consumption amount; assign weights to the shopping frequency, the average number of product types purchased per shopping, and the difference between one and the volatility of the consumption amount, and perform weighted summation, and mark the result of the weighted summation as the stability factor.

[0013] Further, quantify the interest value of the specified customer in the product area according to the duration and the stability factor, including: performing data normalization on the duration and the stability factor; using the entropy weight method to assign weights to the duration and the stability factor after data normalization and perform weighted summation, and mark the calculation result as the interest value.

[0014] Further, reply to the consultation content of the specified customer according to the interest value, including: determining the specified customer who is consulting based on the video data and marking as the consulting customer; obtaining the consultation content of the consulting customer; determining the keywords of each product, and using natural language processing technology to extract the keywords from the consultation content; determining the interest value of the consulting customer according to the keywords; selecting a matching reply template from the preset reply strategy library according to the interest value; generating and outputting the consultation reply content according to the reply template.

[0015] Further, determining the interest value of the consulting customer according to the keyword includes: constructing a mapping table between commodity areas and keywords, including: associating and storing each commodity area with the keywords of the commodities within the commodity area; recording the interest value of each designated customer for each commodity area, and generating a correspondence table including the designated customer, commodity area, and interest value; determining the commodity area consulted by the consulting customer from the keyword mapping table according to the keyword; and determining the interest value of the consulting customer for the commodity area consulted by the consulting customer from the correspondence table according to the consulted commodity area.

[0016] Further, the reply template includes the following generation rules: setting a first threshold, a second threshold, and a third threshold; when the interest value is higher than the first threshold, triggering an active recommendation mechanism and inserting real-time inventory data; when the interest value is in the second threshold range, generating a product parameter comparison table and a historical purchase analysis report; when the interest value is lower than the third threshold, providing basic commodity information and attaching interest guiding remarks.

[0017] In a second aspect, the present application provides an information technology consulting system based on big data, which adopts an information technology consulting method based on big data as described in the first aspect, including: a data acquisition module, an intermediate data calculation module, an interest value generation module, and a reply module.

[0018] The data acquisition module is used to divide the store into multiple commodity areas according to the types of commodities; acquire video data, identify designated customers from the video data, acquire the behavior records of the designated customers in each commodity area, and acquire the historical shopping records of the designated customers.

[0019] The intermediate data calculation module is used for each commodity area: identifying the behavior of the designated customer viewing the commodity from the behavior record and recording the duration; analyzing the shopping record of the designated customer in this commodity area historically according to the historical shopping record of the designated customer, and calculating the shopping stability factor.

[0020] The interest value generation module is used to quantify the interest value of the designated customer for the commodity area according to the duration and the stability factor.

[0021] The reply module is used to reply to the consulting content of the designated customer according to the interest value.

[0022] In a third aspect, the present application provides a device, which includes a memory and a processor; the memory is used to store a computer program; the processor is used to implement the steps of the information technology consulting method based on big data as described in the first aspect when executing the computer program.

[0023] Fourthly, the present application provides a readable storage medium storing computer program instructions, which, when read and run by a processor, execute the steps of the information technology consulting method based on big data as described in the first aspect.

[0024] Advantages of the present invention:

[0025] By dividing the store into different commodity areas, analyzing the surveillance videos in the store, evaluating the interest values of customers in different commodity areas, and replying to the consultation content based on the interest values, the present application effectively solves the problem in the prior art that it is difficult to reply to the consultation content targeted according to the past consumption preferences of customers due to the lack of customer information in the consultation devices of large stores, can meet the personalized needs of customers, and helps the store improve the service quality and competitiveness.

[0026] Other features and advantages of the present invention will be described in the following specification, and, in part, will be obvious from the specification, or will be understood by implementing the present invention. The objectives and other advantages of the present invention can be realized and obtained by the structures pointed out in the specification and the drawings. Description of the Drawings

[0027] In order to more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the following will briefly introduce the drawings required for use in the description of the embodiments or the prior art. Obviously, the following drawings are some embodiments of the present invention. For those of ordinary skill in the art, other drawings can be obtained based on these drawings without creative efforts.

[0028] Figure 1 Shows a schematic flowchart of an information technology consulting method based on big data in Embodiment 1 of the present invention;

[0029] Figure 2 Shows a schematic flowchart of replying to the consultation content of the specified customer according to the interest value in Embodiment 1 of the present invention;

[0030] Figure 3 Shows a schematic module diagram of an information technology consulting system based on big data in Embodiment 2 of the present invention. Detailed Embodiments

[0031] To solve the problems raised in the background art, the present application divides the store into different commodity areas, analyzes the surveillance videos in the store, evaluates the interest values of customers in different commodity areas, and replies to the consultation content based on the interest values, so as to meet the personalized needs of customers and help the store improve the service quality and competitiveness.

[0032] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the following will clearly and completely describe the technical solutions in the embodiments of the present invention with reference to the accompanying drawings in the embodiments of the present invention. Apparently, the described embodiments are some, but not all, of the embodiments of the present invention. All other embodiments obtained by those of ordinary skill in the art based on the embodiments of the present invention without creative efforts shall fall within the protection scope of the present invention.

[0033] Embodiment 1:

[0034] As Figure 1 - Figure 2 shown, this embodiment provides an information technology consulting method based on big data, including:

[0035] S100. According to the types of commodities, divide the store into multiple commodity areas; obtain video data, identify a specified customer from the video data, obtain the behavior records of the specified customer in each commodity area, and obtain the historical shopping records of the specified customer.

[0036] S200. For each commodity area: identify the behavior of the specified customer viewing the commodity from the behavior records and record the duration; according to the historical shopping records of the specified customer, analyze the shopping records of the specified customer in this commodity area historically, and calculate the shopping stability factor.

[0037] S300. Evaluate and quantify the interest value of the specified customer in the commodity area according to the duration and the stability factor.

[0038] S400. Reply to the consultation content of the specified customer according to the interest value.

[0039] If the store is a supermarket, the supermarket can be divided into a fresh food area, a daily necessities area, etc. The video data comes from the customer data collected by each camera in the store. The specified customer is accurately identified from these data through image recognition technology, and then the behavior records of the customer in each commodity area are obtained, such as the staying and walking trajectories in the clothing area. The historical shopping records of this customer can be obtained from the store sales database.

[0040] For each commodity area, identify the behavior of the customer viewing the commodity from the behavior records, such as actions like staying of the eyes, picking up the commodity, etc., record the duration, and based on the historical shopping records, analyze the purchase frequency, interval duration, etc. of the customer in a certain commodity area, calculate the shopping stability factor, and quantitatively evaluate the interest value of the customer in the commodity area according to the duration and the stability factor. If the interest value is high, it indicates that the customer is more concerned about the commodities in this area.

[0041] Finally, the system gives recommendations and answers that match the customer's preferences based on the interest value, thereby effectively improving the customer consultation experience.

[0042] In S100, the store is divided into multiple commodity areas, including:

[0043] S110. Extract key-frame images from the video data.

[0044] S120. Use data annotation to determine the range of each commodity area in the key-frame images and define commodity category labels for each commodity area.

[0045] S130. Obtain the video stream data and use the trained improved object detection model to process the video stream data, identify each commodity area in the images of the video stream data, and output the corresponding commodity category labels.

[0046] The cameras in the store are used to continuously collect video data at different time periods, under different lighting conditions, and with different customer flows.

[0047] Use a professional image annotation tool such as LabelImg to annotate the commodity areas in the video frames, mark each commodity area with a rectangular box (such as the fresh food area, food area, daily necessities area, etc.), and annotate the corresponding area category label for each box. Additionally, annotate the overlapping areas existing in the fields of view of different cameras and use special labels to distinguish them for the model to learn.

[0048] The labeled data can be divided into a training set, a validation set, and a test set according to the ratio of 70%, 15%, and 15% respectively, which are used for model training, parameter adjustment, and final performance evaluation.

[0049] Based on the YOLOv5 model, a video overlapping area recognition module and an improved multi-modal fusion module are added to solve the problem of identifying video overlapping areas from different cameras. The model includes the input layer, backbone network, attention mechanism module, neck network, output layer of the basic YOLOv5 model, as well as the newly added video overlapping area recognition module and the extended multi-modal fusion module. The video overlapping area recognition module is located after the input layer, and the extended multi-modal fusion module is located after the neck network.

[0050] The input layer is used to receive the video stream data and perform scale adjustment on the images in the video frames.

[0051] The input layer includes an adaptive scale adjustment module, which is used to analyze the video frames of different cameras input, and dynamically adjust the scaling ratio of the images according to the size and distribution of the commodity areas in the video frames. For example, if most of the commodity areas in the image are small, this module will appropriately enlarge the image so that the model can capture the features of the small areas more clearly; on the contrary, if the areas are large, the image will be appropriately reduced to improve the detection speed. This can ensure that commodity areas of different sizes can be detected by the model at an appropriate scale, improving the adaptability of the model to different scenarios.

[0052] A video duplicate region recognition module, which is used to extract image features by using MobileNet, perform matching by calculating the cosine similarity between feature vectors, and generate a duplicate region mask.

[0053] The video duplicate region recognition module is mainly used to process duplicate regions in videos captured by different cameras. It includes: a feature extraction layer, a feature matching layer, and a duplicate region determination layer. The feature extraction layer uses the lightweight MobileNet network to extract features from the video frames after scale adjustment. The feature matching layer is used to divide the extracted feature maps into multiple small blocks, find possible duplicate regions by calculating the cosine similarity of the corresponding small block feature vectors in different video frames. The duplicate region determination layer is used to determine the boundaries of the duplicate regions according to the set similarity threshold, use the non-maximum suppression algorithm to remove the duplicate region candidate boxes with too high overlap, and finally generate the mask M of the duplicate region.

[0054] A backbone network, which extracts the basic features of the image with duplicate region mask information and outputs multi-scale basic features.

[0055] The backbone network includes a channel attention module and a spatial attention module. The channel attention module weights the channel features by learning the importance of each channel feature, enhances the expression of key channel features, and suppresses unimportant channels. The spatial attention module focuses on the spatial position where the target is located in the feature map, highlights the features of the target area, and ignores irrelevant information such as the background. This helps the model to more accurately focus on the commodity area, for example, highlighting the features of the commodities on the commodity shelf, and improving the recognition ability of different regions under complex backgrounds.

[0056] A neck network, which includes SPP and PAN. SPP is used to perform multi-scale pooling operations on the enhanced focused features, fuse the feature information at multiple scales, and generate scale-fused features. PAN is used to receive the scale-fused features and multi-scale basic features, transmit the underlying feature details from bottom to top, transmit the high-level feature semantics from top to bottom, and fuse the two to output path-aggregated features.

[0057] The neck network includes an SPP (Spatial Pyramid Pooling) module and a PAN (Path Aggregation Network). The SPP module fuses multi-scale features through pooling operations at different scales. Since the sizes of commodity areas are different, the SPP module can capture feature information at different scales and enhance the model's detection ability for targets of different sizes. PAN can perform up-down path fusion of features, fuse the underlying features transmitted from bottom to top by the backbone network and the high-level features transmitted from top to bottom, enhance feature transmission, and further improve the detection ability of small targets.

[0058] A multimodal fusion module, which is used to receive infrared sensor features and perform weighted fusion on path aggregation features, duplicate region masks, and infrared sensor features to generate comprehensive multimodal features.

[0059] The multimodal fusion module is used to fuse the image features output by the backbone network , the features after processing multimodal data (such as infrared sensor data ), and the mask M output by the video duplicate region recognition module. The specific structure and working process are as follows:

[0060] Feature concatenation layer: Concatenate the above three features to form a higher-dimensional feature vector, integrating multi-source information.

[0061] Fully connected fusion layer: Fuse the concatenated features through a fully connected layer, map the high-dimensional features to a low-dimensional space, extract more representative features, and provide richer and more accurate information for subsequent object detection and classification.

[0062] If the fused feature is , then , where α, β, and γ represent learnable weight coefficients, and their optimal values are determined through training to balance the contributions of different modal data. represents the distribution characteristics of people in different areas of the supermarket, such as the personnel density in the fresh food area, food area, etc.

[0063] Output layer, which is used to generate a bounding box with coordinates, object confidence, and class probability for the supermarket commodity area according to the comprehensive feature information.

[0064] The output layer is used to output the prediction results, including the bounding box coordinates, object confidence, and class probability of the supermarket commodity area. The bounding box coordinates are used to accurately mark the position and range of the commodity area on the image; the object confidence reflects the reliability of the detection result, and a threshold (such as 0.5) can be set to filter out low-confidence detection results; the class probability is used to determine the commodity category to which the area belongs (such as fresh food area, food area, daily necessities area, etc.).

[0065] Stochastic gradient descent can be used as the optimization algorithm, combined with the cosine annealing learning rate adjustment strategy. The cosine annealing learning rate adjustment strategy can use a larger learning rate at the beginning of training to accelerate the model convergence speed. As training progresses, the learning rate is gradually decreased, enabling the model to more finely adjust the parameters and improve the training effect.

[0066] The training steps include:

[0067] S131. Install Python, the PyTorch deep learning framework, and related dependency libraries to ensure that the environment can support the training and operation of the model.

[0068] S132. Initialize the model, including: loading the pre-trained YOLOv5 weights and randomly initializing the parameters of the newly added modules (adaptive scale adjustment module, video duplicate region recognition module, multi-modal fusion module, etc.). The pre-trained weights can provide a good initial state for the model and accelerate the convergence speed of the model.

[0069] S133. Set the training parameters, such as: determining the number of training epochs to be 300, the initial learning rate to be 0.01, and the batch size to be 16. These parameters can be adjusted according to the actual situation to achieve the best training effect.

[0070] S134. Read a batch of supermarket video frames from different cameras, corresponding infrared sensor data, and annotation information from the training set. The video frames are first scaled by the adaptive scale adjustment module, and the adjusted video frames are input into the video duplicate region recognition module to obtain the duplicate region mask. The video frames are used to extract supermarket image features through the backbone network, and the feature expressions are enhanced through the attention mechanism module after some convolutional layers. The neck network performs multi-scale fusion on the features. The supermarket image features, infrared sensor data features, and duplicate region mask are input into the multi-modal fusion module for fusion, and the fused features are input into the output layer for object detection and classification prediction. Calculate the loss between the prediction result and the annotation ground truth, and use the built-in loss function of YOLOv5, including bounding box loss, confidence loss, and classification loss.

[0071] Every 10 epochs of training, evaluate the performance of the model on the validation set, observe the changes in indicators such as the loss value and the mean average precision (mAP), and adjust the parameters according to the evaluation results.

[0072] After the training is completed, input the video stream data into the model, and the model can output the various commodity regions of each frame of the image and the corresponding commodity category labels after processing.

[0073] Obtain the behavior records of the specified customer in each commodity region, including:

[0074] S140. Use the human keypoint detection algorithm to detect the human keypoints of the specified customer, extract the keypoint coordinates of the two feet of the specified customer, and select one of the keypoints of the foot and mark it as the reference point.

[0075] S150. Determine the bounding box where the keypoint is located to obtain the commodity region where the specified customer is located.

[0076] S160. Classify the video data according to the commodity region where the specified customer is located to obtain the sub-videos of the specified customer in each commodity region.

[0077] Use a human key point detection algorithm to detect the human key points of a specified customer. Specifically, a pre-trained human key point detection model can be used, such as the OpenPose model based on a convolutional neural network. Decode the video stream data output by the improved object detection model into continuous image frames, preprocess the image frames, and sequentially input the preprocessed image frames into the model to finally output the probability maps of each human key point. For each output probability map, for the key points of the two feet, find the position with the maximum probability value.

[0078] From the coordinates of the key points of the two feet extracted, select a key point of one foot as a reference point and display it on the image frame, then it can be determined which bounding box it is located in, thereby determining the specific commodity area.

[0079] After clarifying the commodity area where the specified customer is located, classify the video data according to these areas. By traversing all the video data, filter out the video segments containing the behaviors of the specified customer in the same commodity area to form a sub-video of this area.

[0080] In S200, identify the behavior of the specified customer viewing the commodity from the behavior records and record the duration, including:

[0081] Sa210. Split the sub-video into a continuous sequence of video frames and preprocess the video frame sequence.

[0082] Split the sub-videos of each commodity area obtained into a continuous sequence of video frames. Each video frame represents the behavior picture of the customer in that area at a certain moment. Preprocess the video frame sequence, including resizing to the size required by the model and normalizing the pixel values.

[0083] Sa220. View and label the behavior tags for each video frame. The behavior tags include: viewing the commodity, not viewing the commodity; label the start frame and end frame of the behavior of viewing the commodity in the sub-video.

[0084] Sa230. Divide the continuous video frames into a training set and a test set, and use the training set and the test set to train a 3D convolutional neural network.

[0085] Divide the continuous video frames into a training set and a test set. The training set is used to train the 3D convolutional neural network to let the model learn the characteristic patterns of the customer's behavior of viewing the commodity in the video frames; the test set is used to evaluate the performance of the trained model to judge whether the model can accurately identify the behaviors in the unseen video frames. Through a large amount of training data, the model gradually masters the video frame characteristics corresponding to different behaviors, such as the performance characteristics of actions such as the customer reaching for the commodity with the hand and focusing on the commodity with the eyes in the video frames.

[0086] Sa240. Process the pre - processed video frame sequence using the trained 3D convolutional neural network and output the behavior label for each video frame.

[0087] Use the trained 3D convolutional neural network to process the pre - processed video frame sequence. The model will analyze each video frame and, based on the learned feature patterns, output the corresponding behavior label for each video frame to determine whether the customer in the video frame is viewing the product.

[0088] Sa250. Check each video frame in the order of the video frames, mark the start frame and end frame of the video frames with the label of viewing the product, and record the duration.

[0089] Check each video frame in sequence. Once a video frame with the label "viewing the product" is identified, mark its start frame. When this label no longer appears subsequently, mark the corresponding end frame. By recording the positions of the start frame and end frame in the video frame sequence and combining with the video frame rate, the duration of this period of viewing the product behavior can be calculated.

[0090] Sa260. Aggregate the recorded durations to obtain the time period during which the specified customer views the product.

[0091] Aggregate the durations of each recorded segment of the behavior of viewing the product. Add up the durations of these segments to finally obtain the complete time period during which the specified customer views the product in this product area.

[0092] In S200, according to the historical shopping records of the specified customer, analyze the shopping records of the customer in this product area in history and calculate the shopping stability factor, including:

[0093] Sb210. For each product area, calculate the shopping frequency, the average number of product types purchased per shopping, and the volatility of the consumption amount of the specified customer within a certain time period according to the historical shopping records.

[0094] The specified customer who checks out can be identified through the camera, and the identity identifier of the specified customer is associated with the corresponding shopping record to establish a mapping relationship for subsequent data analysis.

[0095] For each product area, assume that the number of shopping times of the specified customer within a certain time period T is N, then the shopping frequency f is: ; The shopping frequency can measure how frequently the customer purchases the products in this product area during this time period.

[0096] If within the time period T, the number of product types purchased by the customer are respectively , the average number of product types purchased per shopping m is: , the average number of product categories purchased per shopping reflects the richness of product selection when customers shop in this area each time.

[0097] Suppose that within the time period T, a customer shops N times in this product area, and the consumption amounts for each shopping are respectively , the average consumption amount is: ; then calculate the standard deviation of the consumption amount according to the standard deviation formula is: ; the larger the standard deviation, the greater the fluctuation of the consumption amount, that is, the consumption amount of the customer in this product area is unstable.

[0098] Sb220. Standardize the shopping frequency, the average number of product categories purchased per shopping, and the volatility of the consumption amount.

[0099] Sb230. Assign weights to the shopping frequency, the average number of product categories purchased per shopping, and the difference between one and the volatility of the consumption amount, and perform weighted summation. Mark the result after weighted summation as the stability factor.

[0100] The stability factor S is: ; where , , are weights, and each can take , , , respectively represent the normalized results of f, m, The stability factor comprehensively considers the shopping frequency, the average number of product categories purchased per shopping, and the stability of the consumption amount.

[0101] In S300, evaluate and quantify the interest value of a specified customer in the product area according to the duration and the stability factor, including:

[0102] S310. Perform data normalization on the duration and the stability factor.

[0103] S320. Use the entropy weight method to assign weights to the duration and the stability factor after data normalization and perform weighted summation. Mark the calculation result as the interest value.

[0104] Determine their weights by calculating the information entropy of the duration and the stability factor S. The information entropy The calculation formula is: , in the formula, , n represents the number of customers, , Represents the normalized value of the j-th indicator of the i-th customer (when j takes the value of 1, it represents the duration, and when j takes the value of 2, it represents the stability factor). The information entropy of the duration indicator and the information entropy of the stability factor are obtained respectively. The information entropy of the stability factor Then, according to the entropy weight calculation formula, the weight of the duration and the weight of the stability factor are obtained. After weighted summation, the interest value can be obtained. The interest value comprehensively considers the behavior duration and shopping stability of the customer in the commodity area, and can objectively quantify the customer's interest degree in the commodity area.

[0105] In S400, according to the interest value, the consultation content of the specified customer is replied, including:

[0106] S410. Determine the specified customer who is consulting according to the video data and mark them as the consulting customer.

[0107] S420. Obtain the consultation content of the consulting customer.

[0108] S430. Determine the keywords of each commodity, and use natural language processing technology to extract keywords from the consultation content.

[0109] S440. Determine the interest value of the consulting customer according to the keywords.

[0110] S450. Select a matching reply template from the preset reply strategy library according to the interest value.

[0111] S460. Generate the consultation reply content according to the reply template and output it.

[0112] When a customer initiates a consultation, first, it is necessary to clarify the identity of the customer who is currently consulting. The customer image can be obtained through a camera, and the identity of the customer can be recognized through image recognition technology.

[0113] For each commodity in the store, keywords that can represent its core features are preset in advance. For example, for a smart phone, the keywords may include "brand name", "model", "processor performance", "camera pixel", etc. Use the text analysis algorithm in natural language processing technology to process the consultation content of the consulting customer. Common technologies include lexical analysis, syntactic analysis, and semantic analysis, etc. Through these technologies, keywords related to the commodity are extracted from the consultation content. For example, if the consultation content is "How about the photo-taking effect of a new mobile phone of a certain brand", keywords such as "a certain brand", "mobile phone", "photo-taking effect" can be extracted.

[0114] Based on the interest values of the customer in different product areas calculated previously, combined with the extracted keywords, determine the interest value of the consulting customer in the product area involved in the current consultation content. For example, if the extracted keywords are mainly related to the electronics product area, the system will look up the interest value of this customer in the electronics product area.

[0115] In the preset response strategy library, response templates for different interest value ranges and keyword combinations are stored. These templates are pre-developed based on a large amount of historical consultation data and customer feedback, aiming to provide the most appropriate and customer-demand-compliant responses. The system matches according to the determined interest value and the extracted keywords in the response strategy library to find the corresponding response template.

[0116] After finding the matching response template, the system fills relevant information in the consultation content, such as the specific product names, models, etc. mentioned by the customer, into the corresponding positions in the response template. After information filling, a complete consultation response content is generated, and through the output interface of the consultation device, such as screen display or voice broadcast, etc., the response content is presented to the consulting customer, and the customer can obtain a personalized response for their consultation content and taking into account their interest preferences, improving the customer's consultation experience and satisfaction.

[0117] In S440, determining the interest value of the consulting customer according to the keywords includes:

[0118] S441. Construct a mapping table between product areas and keywords, including: associating and storing each product area with the keywords of the products within that product area; recording the interest value of each specified customer in each product area, and generating a correspondence table including the specified customer, product area, and interest value;

[0119] S442. Determine the product area consulted by the consulting customer from the keyword mapping table according to the keywords;

[0120] S443. Determine the interest value of the consulting customer in the product area they consulted from the correspondence table according to the consulted product area.

[0121] For each product area in the store, such as the electronics area, clothing area, food area, etc., summarize and organize the keywords of all products within that area. For example, in the electronics area, for products like smartphones, its keywords are "brand name", "model", "processor performance", "camera pixel", etc.; for computer products, there are keywords such as "CPU model", "memory capacity", "graphics card type", etc. Associate and store these product areas with the corresponding keywords to form a clear mapping relationship.

[0122] After obtaining the interest values of each customer in different product areas, organize this information into a correspondence table.

[0123] After extracting keywords from the consultation content of consulting customers, determine the commodity area consulted by the consulting customers according to the constructed mapping table of commodity area and keywords. After determining the commodity area consulted by the consulting customers, further query the corresponding relationship table containing the specified customers, commodity areas, and interest values, so as to quickly determine the interest values of the customers.

[0124] The reply template includes the following generation rules:

[0125] Set the first threshold, the second threshold, and the third threshold.

[0126] When the interest value is higher than the first threshold, trigger the active recommendation mechanism and insert real-time inventory data.

[0127] When the interest value is in the second threshold range, generate a product parameter comparison table and a historical purchase analysis report.

[0128] When the interest value is lower than the third threshold, provide basic commodity information and append interest guiding words.

[0129] The first threshold can be set to a relatively high value. For example, assume the value is 0.8. When the system determines that the interest value of the consulting customer in the consulted commodity area is higher than the first threshold, it indicates that the customer shows extremely high interest in the commodities in this area; the second threshold range: for example, it can be set to 0.4 - 0.6. When the interest value is in this range, it means that the customer has a certain interest in the commodities, but more information is needed to assist in decision-making; the third threshold can be set to a relatively low value, such as 0.2. When the interest value is lower than the third threshold, it means that the customer has a relatively weak interest in this commodity area.

[0130] The specific values of the first threshold, the second threshold, and the third threshold can be determined according to actual business data and analysis.

[0131] When the interest value is higher than the first threshold, trigger the active recommendation mechanism and insert real-time inventory data. For such customers who show extremely high interest in the commodity area, actively recommending commodities that match their preferences can accurately meet their potential needs and save the time and effort of customers to search for commodities by themselves. The provision of real-time inventory data enables customers to promptly grasp the purchasability of commodities, enhances the timeliness of purchase decisions, and greatly improves the conversion rate from consultation to purchase of customers.

[0132] For customers whose interest values are in the second threshold range, generate a product parameter comparison table and a historical purchase analysis report. The product parameter comparison table can help customers clearly understand the differences between different commodities and assist them in making more rational purchase decisions. The historical purchase analysis report provides more comprehensive commodity information for customers from multiple aspects such as the time dimension and customer group characteristics, meets their needs for in-depth understanding of commodities, and further enhances customers' awareness and purchase intention of commodities.

[0133] For customers whose interest value is lower than the third threshold, basic product information is provided and interest-guiding words are attached. The basic product information concisely and clearly answers the customer's initial questions, while the interest-guiding words try to tap into the customer's potential needs, guide the customer to pay attention to more related products, and gradually increase the customer's interest in the product area, laying the foundation for possible subsequent purchase behavior.

[0134] Embodiment 2:

[0135] like Figure 3 As shown, this embodiment provides an information technology consulting system based on big data, including: a data acquisition module, an intermediate data calculation module, an interest value generation module and a reply module.

[0136] The data acquisition module is used to divide the store into multiple commodity areas according to the types of commodities; obtain video data, identify the specified customer from the video data, obtain the behavior record of the specified customer in each commodity area, and obtain the historical shopping record of the specified customer. The intermediate data calculation module is used for each commodity area: identifying the behavior of the specified customer viewing the commodity from the behavior record and recording the duration; analyzing the historical shopping record of the specified customer in the commodity area according to the historical shopping record of the specified customer, and calculating the shopping stability factor. The interest value generation module is used to evaluate and quantify the interest value of the specified customer in the commodity area according to the duration and stability factor. The reply module is used to reply to the consultation content of the specified customer according to the interest value.

[0137] This embodiment has all the advantages of the information technology consulting method based on big data in the first embodiment, and can automatically execute the steps of the information technology consulting method based on big data.

[0138] Embodiment three:

[0139] This embodiment provides a device, which includes a memory and a processor; the memory is used to store a computer program; the processor is used to implement the steps of the information technology consulting method based on big data in the first embodiment when executing the computer program.

[0140] Embodiment 4:

[0141] This embodiment provides a readable storage medium, in which computer program instructions are stored. When the computer program instructions are read and executed by a processor, the steps of the information technology consulting method based on big data in the first embodiment are executed.

[0142] Among them, any reference to a memory, storage, database, or other medium used in the embodiments provided by the present invention may include non-volatile and / or volatile memories. The non-volatile memory may include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM), or flash memory. The volatile memory may include random access memory (RAM) or external cache memory.

[0143] It should be noted that, in this document, relational terms such as "first" and "second" are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any actual relationship or order between these entities or operations. Moreover, the terms "comprise," "include," or any other variant thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or device that comprises a series of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such process, method, article, or device. Without further limitation, an element defined by the phrase "comprising a..." does not exclude the presence of additional identical elements in the process, method, article, or device that comprises the element.

[0144] Although the present invention has been described in detail with reference to the foregoing embodiments, those of ordinary skill in the art should understand that they can still modify the technical solutions described in the foregoing embodiments, or perform equivalent replacements for some of the technical features; and these modifications or replacements do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. An information technology consulting method based on big data, characterized in that: include: Divide the store into multiple merchandise areas based on the types of merchandise; Obtain video data, identify a specified customer from the video data, obtain the behavior record of the specified customer in each product area, and obtain the historical shopping record of the specified customer; For each commodity area: identifying the behavior of the designated customer viewing commodities from the behavior record, and recording the duration; According to the historical shopping records of the designated customer, the historical shopping records of the designated customer in the commodity area are analyzed to calculate the shopping stability factor; quantifying the interest value of the specified customer in the commodity area according to the duration and stability factor; Responding to the inquiry content of the designated customer according to the interest value; Among them, according to the types of goods, the store is divided into multiple product areas, including: Extract key frame images from video data; Data annotation is used to determine the range of each commodity area in the key frame image, and a commodity category label is defined for each commodity area; Obtain video stream data, and use the trained improved object detection model to process the video stream data, identify each product area in the image in the video stream data, and output the corresponding product category label; The improved target detection model includes: The input layer is used to receive video stream data and resize the images in the video frames; The video repeated region recognition module is used to extract image features using MobileNet, match them by calculating the cosine similarity between feature vectors, and generate repeated region masks; The backbone network extracts the basic features of the image with repeated region mask information and outputs multi-scale basic features; The attention mechanism module is used to filter and weight multi-scale basic features, and use channel and spatial attention mechanisms to generate enhanced focus features; The neck network includes SPP and PAN. SPP is used to perform multi-scale pooling operations on the enhanced focus features, fuse feature information at multiple scales, and generate scale fusion features. PAN is used to receive scale fusion features and multi-scale basic features, pass the underlying feature details from bottom to top, pass the high-level feature semantic information from top to bottom, and fuse the two to output path aggregation features. A multimodal fusion module is used to receive infrared sensor features and perform weighted fusion of path aggregation features, repeated area masks and infrared sensor features to generate comprehensive multimodal features; The output layer is used to generate a bounding box with coordinates, target confidence, and category probability of the supermarket product area based on the comprehensive feature information.

2. The method according to claim 1, characterized in that Get the behavior records of the specified customer in each product area, including: Using a human body key point detection algorithm to detect the human body key points of the designated customer, extracting the key point coordinates of the designated customer's feet, selecting one of the key points of the foot and marking it as a reference point; Determine the bounding box where the key points are located and obtain the product area where the specified customer is located; The video data is classified according to the commodity area where the designated customer is located, and a sub-video of the designated customer in each commodity area is obtained.

3. The method according to claim 2, characterized in that Identifying the behavior of the designated customer viewing the product from the behavior record and recording the duration, including: Splitting the sub-video into a continuous video frame sequence, and preprocessing the video frame sequence; Mark each video frame with a behavior tag, where the behavior tags include: viewing a product, not viewing a product; and mark the start and end frames of the product viewing behavior in the sub-video; The continuous video frames are divided into a training set and a test set, and the 3D convolutional neural network is trained using the training set and the test set; Use the trained 3D convolutional neural network to process the preprocessed video frame sequence and output the behavior label of each video frame; Check each video frame in order, mark the start and end frames of the video frame labeled as viewing the product, and record the duration; Summarize the recorded durations to obtain the time period when the specified customer viewed the product.

4. The method according to claim 3, characterized in that: Analyze the customer's historical shopping records in the product area and calculate the shopping stability factor, including: For each product area, calculate the shopping frequency of a specified customer within a certain period of time, the average number of product types purchased per shopping trip, and the volatility of the spending amount based on historical shopping records; Standardize the volatility of shopping frequency, average number of items purchased per shopping trip, and amount spent; Assign weights to the shopping frequency, the average number of types of goods purchased per shopping trip, and the difference between one and the volatility of the spending amount, and perform a weighted sum, and mark the result of the weighted sum as a stability factor.

5. The method according to claim 1, characterized in that Responding to the inquiry content of the designated customer according to the interest value includes: Determine the designated customer being consulted based on the video data and mark the customer as a consulting customer; Obtain consulting content from consulting clients; Determine the keywords for each product and use natural language processing technology to extract keywords from the consultation content; Determining the interest value of the consulting customer according to the keywords; Selecting a matching reply template from a preset answer strategy library according to the interest value; Generate and output the consultation reply content according to the reply template; The reply template contains the following generation rules: Setting a first threshold, a second threshold interval, and a third threshold; When the interest value is higher than a first threshold, the active recommendation mechanism is triggered and real-time inventory data is inserted; When the interest value is within the second threshold range, a product parameter comparison table and a historical purchase analysis report are generated; When the interest value is lower than the third threshold, basic product information is provided and interest-guiding words are attached.

6. An information technology consulting system based on big data, characterized in that: It includes: A data acquisition module, which is used to divide the store into multiple commodity areas according to the types of commodities; Obtain video data, identify a specified customer from the video data, obtain the behavior record of the specified customer in each product area, and obtain the historical shopping record of the specified customer; An intermediate data calculation module is used for: for each commodity area: identifying the behavior of the designated customer viewing the commodity from the behavior record, and recording the duration; According to the historical shopping records of the designated customer, the historical shopping records of the designated customer in the commodity area are analyzed to calculate the shopping stability factor; An interest value generating module, which is used to quantify the interest value of a specified customer in the commodity area according to the duration and stability factor; A reply module, which is used to reply to the consultation content of the designated customer according to the interest value; Among them, according to the types of goods, the store is divided into multiple product areas, including: Extract key frame images from video data; Data annotation is used to determine the range of each commodity area in the key frame image, and a commodity category label is defined for each commodity area; Obtain video stream data, and use the trained improved object detection model to process the video stream data, identify each product area in the image in the video stream data, and output the corresponding product category label; The improved target detection model includes: The input layer is used to receive video stream data and resize the images in the video frames; The video repeated region recognition module is used to extract image features using MobileNet, match them by calculating the cosine similarity between feature vectors, and generate repeated region masks; The backbone network extracts the basic features of the image with repeated region mask information and outputs multi-scale basic features; The attention mechanism module is used to filter and weight multi-scale basic features, and use channel and spatial attention mechanisms to generate enhanced focus features; The neck network includes SPP and PAN. SPP is used to perform multi-scale pooling operations on the enhanced focus features, fuse feature information at multiple scales, and generate scale fusion features. PAN is used to receive scale fusion features and multi-scale basic features, pass the underlying feature details from bottom to top, pass the high-level feature semantic information from top to bottom, and fuse the two to output path aggregation features. A multimodal fusion module is used to receive infrared sensor features and perform weighted fusion of path aggregation features, repeated area masks and infrared sensor features to generate comprehensive multimodal features; The output layer is used to generate a bounding box with coordinates, target confidence, and category probability of the supermarket product area based on the comprehensive feature information.

7. An information technology consulting device based on big data, characterized in that: It includes a memory and a processor; the memory is used to store computer programs; the processor is used to implement the steps of the big data-based information technology consulting method described in any one of claims 1 to 5 when executing the computer program.

8. A readable storage medium, characterized in that: The readable storage medium stores computer program instructions, and when the computer program instructions are read and executed by a processor, the steps of the big data-based information technology consulting method described in any one of claims 1 to 5 are executed.

Images