Offline consumption behavior analysis and marketing decision method fused with image recognition

Abstract

The present application relates to the technical field of behavior characteristic analysis, in particular to an offline consumption behavior analysis and marketing decision method fusing image recognition, which detects customers and goods through image recognition to generate behavior trajectories, locates stay areas and extracts customer micro behaviors, generates a consumption behavior characteristic set after labeling in time sequence; builds a cause-and-effect node time sequence data, generates a cause-and-effect graph through cause-and-effect structure learning, divides mixed variables in layers and blocks their bidirectional interference links to obtain a real cause-and-effect graph of behavior and conversion; discriminates positive cause-and-effect, reverse cause-and-effect and false correlation behavior through counterfactual scenario simulation comparison, and finally generates a differentiated marketing decision. The present application can accurately determine the real cause-and-effect value of consumption behavior, and solve the problems of easy judgment deviation in traditional analysis relying on correlation and low-efficiency marketing decision.

Description

Technical Field

[0001] This invention relates to the field of behavioral feature analysis technology, and in particular to a method for analyzing offline consumer behavior and making marketing decisions by integrating image recognition. Background Technology

[0002] In existing offline retail scenarios, consumer behavior analysis and marketing decisions primarily rely on image recognition to extract visual data such as customer dwell time, trajectory, and browsing behavior. This data is then combined with store sales data to derive the correlation between consumer behavior and conversion rates. In real-world scenarios, customer dwell time is often positively correlated with purchase rate—the longer the dwell time, the higher the purchase rate. However, the causal relationship between dwell time and purchase rate can vary. It could be due to factors such as whether the product attracts customers to browse for extended periods and ultimately leads to a purchase, or whether factors like holidays or other periods of peak customer traffic contribute to the increased purchase rate. Simply statistically analyzing the superficial correlation between dwell time and conversion rate can establish a causal relationship, but this method fails to identify whether customer interaction is voluntary or driven by actual personal needs, thus failing to infer the user's purchase intention and ultimately leading to flawed marketing strategies.

[0003] Therefore, it is necessary to propose a method for analyzing offline consumer behavior and making marketing decisions by integrating image recognition to solve the above problems. Summary of the Invention

[0004] The purpose of this invention is to provide a method for analyzing offline consumer behavior and making marketing decisions by integrating image recognition, so as to solve the problem of being unable to infer users' purchase intentions, which ultimately leads to incorrect marketing strategies.

[0005] To achieve the above objectives, the present invention provides the following technical solution: The offline consumer behavior analysis and marketing decision-making method integrating image recognition includes the following steps: Acquire store video data, consumption conversion result data, and environmental confounding variable data. Use image recognition to detect customers and products in store images to obtain customer behavior trajectories. Based on the behavior trajectories, locate the customer dwell area and use image recognition to extract micro-behaviors related to consumption decisions. Record the order of occurrence of micro-behaviors and string them together. Label the micro-behaviors and integrate all the labeled information to generate a feature set with consumption behavior characteristics. Set causal nodes, obtain the time series data corresponding to the causal nodes, input the time series data into the causal structure learning algorithm, traverse the causal pointing relationship between each causal node, and generate a causal graph; Based on the causal graph, confounding variables are hierarchically divided, the preceding interference paths of confounding variables in the causal graph are located, and according to the preceding interference paths, the bidirectional transmission link of confounding variables acting simultaneously on consumer behavior characteristics and consumption conversion results is blocked, thus obtaining the true causal graph of consumer behavior and consumption conversion results. Based on each type of consumer behavior in the feature set, and combined with behavioral data and consumption conversion results in real scenarios, a corresponding counterfactual scenario is constructed. The counterfactual scenario is simulated based on the real cause-effect graph to obtain the simulation results. By inputting real-world behavioral data, consumption conversion results, and simulation results into a real causal graph, the direction of the influence of consumption behavior on consumption conversion results can be obtained, thus identifying positive causal behavior, negative causal behavior, and falsely related behavior. Based on positive causal behavior, negative causal behavior, and false relevance behavior, corresponding marketing decisions are generated.

[0006] Preferably, based on behavioral trajectories, image recognition is used to identify various micro-behaviors related to consumption decisions in the customer's dwell area. These micro-behaviors include dwell time, product interaction, visual attention, and social interaction. Dwell time indicates the duration of time spent in the dwell area, product interaction indicates the act of picking up, comparing, or touching products, visual attention indicates the duration and angle of gazing at products, and social interaction indicates interactive behaviors between customers pointing towards products.

[0007] Preferably, the moment a single customer enters the store is taken as the customer's personal timeline. Based on the timeline, micro-behaviors are concatenated in chronological order to obtain a behavioral temporal feature vector, and then labeled with information. The annotation information includes the time period when the behavior occurred, the type of behavior, and the duration of the behavior.

[0008] Preferably, the causal nodes include consumer behavior feature nodes, environmental confounding variable nodes, and consumer conversion result nodes. The time-series data corresponding to each causal node is obtained based on the feature set of consumer behavior features, environmental confounding variable data, and consumer conversion result data.

[0009] Preferably, the step of inputting time-series data into a causal structure learning algorithm, traversing the causal pointing relationships between each causal node, and generating a causal graph includes: The time-series data of each causal node that has been acquired are time-series normalized according to the timeline of a single customer to form a time-series dataset; The causal structure learning algorithm is used to traverse the time series dataset and perform independence verification on any two causal nodes to determine whether there is a direct causal dependency between the two causal nodes and generate verification results. Based on the temporal sequence logic of the time series dataset and combined with the verification results, the directed causal relationship between two causal nodes is determined. Based on the directed causal relationship, causal nodes that have causal effects are connected by directed edges to obtain a causal graph.

[0010] Preferably, the hierarchical classification of confounding variables specifically includes three layers: the first layer is environmental confounding variables, the second layer is human intervention confounding variables, and the third layer is user-inherent attribute confounding variables.

[0011] Preferably, the steps of hierarchically dividing confounding variables based on a causal graph, locating the preceding interference paths of confounding variables in the causal graph, and blocking the bidirectional transmission link of confounding variables simultaneously acting on consumer behavior characteristics and consumption conversion results according to the preceding interference paths, to obtain the true causal graph of consumer behavior and consumption conversion results, include: Identify the leading source nodes of confounding variables in the causal graph, locate the paths between confounding variables and consumer behavior characteristics, and the paths between confounding variables and consumption conversion results, in order to obtain the leading interference paths. By blocking the bidirectional transmission links of the confounding variables based on the preceding interference path, the true causal graph is obtained.

[0012] Preferably, the step of blocking the bidirectional transmission link of the profane variables based on the preceding interference path to obtain the true causal graph includes: For each preceding interference path, the bidirectional connection between confounding variables and consumer behavior characteristics and consumption conversion results is severed. For the blocked causal graph, the true causal graph is obtained by removing interfering links based on the hierarchical confounding variables.

[0013] Preferably, the positive causal behavior is the behavior that directly leads to consumption conversion; the negative causal behavior is the consumption behavior generated when the customer has a prior purchase intention; and the spurious correlation behavior is the consumption behavior that is driven only by confounding variables and has no direct causal relationship with consumption conversion.

[0014] Preferably, the generation of the corresponding marketing decision includes: Generate shelf-optimized marketing decisions based on positive causal behavior; Targeting reverse causal behavior, generate non-intrusive information service marketing strategies; To address false related behaviors, generate marketing strategies that do not implement marketing interventions.

[0015] The technical effects and advantages of the present invention in the above technical solution are as follows: This invention utilizes store video data, consumer conversion result data, and environmental confounding variable data, along with image recognition detection, to obtain customer behavior trajectories and extract micro-behaviors related to consumption decisions, generating a consumer behavior feature set. Then, based on causal node time-series data, a causal graph is generated through causal structure learning. By hierarchically dividing confounding variables and locating and blocking bidirectional interference links that simultaneously affect consumer behavior features and consumption conversion results, a true causal graph is obtained. Combined with counterfactual scenario construction and simulation, positive causal behavior, negative causal behavior, and spurious correlation behavior are identified, ultimately generating corresponding marketing decisions. This effectively eliminates spurious correlation interference caused by environmental, human, and user attribute confounding factors, improving the accuracy of determining the impact of consumer behavior on consumption conversion. It achieves a deep integration of image recognition and causal inference, forming a solution from behavior extraction to marketing, solving the problems of traditional analysis relying solely on correlation, which easily leads to judgment bias and inefficient marketing decisions. Attached Figure Description

[0016] Figure 1 This is a flowchart of the offline consumer behavior analysis and marketing decision-making method that integrates image recognition, as described in this invention. Detailed Implementation

[0017] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0018] like Figure 1 As shown, this embodiment provides a method for offline consumer behavior analysis and marketing decision-making that integrates image recognition, including the following steps: S1: Acquire store video data, consumption conversion result data, and environmental confounding variable data. Use image recognition to detect customers and products in store images to obtain customer behavior trajectories. Based on the behavior trajectories, locate the customer dwell area and use image recognition to extract micro-behaviors related to consumption decisions. Record the order of occurrence of micro-behaviors and string them together. Label the micro-behaviors and integrate all the labeled information to generate a feature set with consumption behavior characteristics. S2: Set causal nodes, obtain the time series data corresponding to the causal nodes, input the time series data into the causal structure learning algorithm, traverse the causal pointing relationship between each causal node, and generate a causal graph; S3: Based on the causal graph, the confounding variables are hierarchically divided, the preceding interference paths of the confounding variables in the causal graph are located, and the bidirectional transmission link of the confounding variables acting on both consumer behavior characteristics and consumption conversion results is blocked according to the preceding interference paths, so as to obtain the true causal graph of consumer behavior and consumption conversion results. S4: Based on each type of consumer behavior in the feature set, and combined with behavioral data and consumption conversion results in real scenarios, construct corresponding counterfactual scenarios, simulate the counterfactual scenarios based on the real cause-effect graph, and obtain simulation results; S5: Input real-world behavioral data, consumption conversion results, and simulation results into the real causal graph to obtain the direction of the influence of consumption behavior on consumption conversion results, thereby obtaining positive causal behavior, negative causal behavior, and falsely related behavior; S6: Generate corresponding marketing decisions based on positive causal behavior, negative causal behavior, and false relevance behavior.

[0019] In one embodiment of the present invention, based on behavioral trajectories, image recognition is used to identify various micro-behaviors related to consumption decisions in the customer's dwell area. The micro-behaviors include dwelling type, product interaction type, visual attention type, and social interaction type. The dwelling type is used to indicate the duration of dwelling in the area. The product interaction type is used to indicate the behavior of picking up, comparing, and touching products. The visual attention type is used to indicate the duration and angle of product gaze. The social interaction type is used to indicate the interactive behavior between customers pointing to products.

[0020] Using the moment a single customer enters the store as their personal timeline, micro-behaviors are concatenated according to their chronological order to obtain a behavioral temporal feature vector, which is then labeled with information. The annotation information includes the time period when the behavior occurred, the type of behavior, and the duration of the behavior.

[0021] In this embodiment of the invention, for store video data, high-definition network cameras and intelligent vision terminals are deployed in various consumption areas of the store to collect video data including customer bodies, product entities, store areas, and interactive micro-behaviors; consumption conversion result data is used to characterize the business result data of the customer's final consumption behavior to determine whether the consumption behavior truly leads to a purchase, and can be obtained from the store's POS cashier system and self-checkout terminal; environmental confounding variable data is independent of consumption behavior, but can simultaneously affect customer behavior and consumption conversion results through third-party interference data. To acquire customer behavior trajectories, after preprocessing store videos frame by frame, target detection is used to simultaneously locate customers and products. Multi-target tracking is employed, a unique ID is assigned to each customer and cross-frame coordinates are recorded. After spatial coordinate mapping, the coordinates are concatenated in time sequence to generate the physical behavior trajectory of customers within the store. Based on behavioral trajectories, image recognition is used to identify various micro-behaviors related to consumer decisions in customer dwell areas. Specifically, based on the generated continuous customer behavior trajectories, a set of points with stable coordinates and continuous dwelling within a fixed small range is extracted. Invalid points that are only briefly passed through or paused are filtered out by a preset dwell time threshold (e.g., ≥2 seconds). Areas that meet the criteria are matched to physical areas such as store shelves, promotional areas, and display areas. Behavioral areas related to consumer decisions are selected from the trajectories. In order to transform dwelling behavior into behavioral signals that can be used to represent purchase intentions, a local image of the corresponding video frame is cropped with the dwelling area as the center. Within the dwelling area image, image recognition algorithms are used to extract micro-behaviors related to consumer decisions, specifically including the following micro-behaviors: Dwelling-related behaviors: Calculate the continuous dwell time of customers in this area; Product interaction behaviors: Detect customer hand movements to determine whether actions such as picking up products, touching products, comparing multiple products, or putting products back occur; Visual attention behavior: By detecting facial key points and gaze, calculate the duration and angle of a customer's gaze at a product; Social interaction behaviors: Detect the presence of multiple customers in the same group, and identify interactive actions such as pointing at products, body language, and face-to-face communication; Based on the timeline, which is the timeline starting from the moment a single customer enters the store, the micro-behaviors are connected in sequence to obtain the behavior time sequence feature vector, and the information is labeled. The label information includes the time period of the behavior, the behavior type, and the duration of the behavior. The time period of the behavior is used to label the start and end time of the micro-behavior on the personal timeline. The behavior type is used to label the corresponding type among the stay type, product exchange type, visual attention type, and social interaction type. The duration of the behavior is used to label the continuous time length of the micro-behavior from start to end. For the process of extracting micro-behaviors related to consumption decisions through image recognition, specifically, for the extraction of dwelling-type micro-behaviors, the YOLOv8 target detection algorithm is used to track customer targets in the dwelling area and record the customer's coordinates and continuous dwelling status. According to the number of frames, the dwelling time of customers in the dwelling area is counted to obtain the start and end times of the dwelling and the total dwelling time. The YOLOv8 object detection algorithm is used to analyze and calculate the start and end times of customers' stay in the designated area, as well as the total dwell time. Specifically, this is achieved through the following method: Load the pre-trained YOLOv8n human detection model, initialize the ByteTrack tracker, assign a unique ID to each customer using the tracker to avoid confusion, and input store parameters, including the coordinates of the dwell area, video frame rate, and pixel displacement threshold. Set to 5 pixels, effective dwell time threshold Set to 2 seconds; After decoding the video stream frame by frame, the data is input into the YOLOv8 model for inference. The model outputs the bounding box coordinates of customers in the frame. Based on their IDs, a tracker is used to associate the same customer in consecutive frames and calculate the geometric center coordinates of the customer's bounding box. Then its geometric center coordinates are ,in, , , and Let x and y represent the horizontal x-coordinate of the top-left vertex, the horizontal x-coordinate of the bottom-right vertex, the vertical y-coordinate of the top-left vertex, and the vertical y-coordinate of the bottom-right vertex of the customer's bounding box, respectively. Combined with the coordinates of the dwell area, the coordinates of the geometric center are determined. Whether a target falls within the dwelling area is determined in this embodiment; targets within the dwelling area are only counted. For consecutive frames, the Euclidean displacement D between the center coordinates of the current frame and the previous frame is used to determine whether the customer is stationary. Specifically, the calculation method is as follows: ; in, , Indicates the geometric center coordinates of the current frame. , Indicates the coordinates of the geometric center of the previous frame; Based on the above calculation results, if At the same time as marking it as stationary, the number of stationary frames begins to accumulate; Based on the video frame rate, calculate the duration of the still frames. If this duration is less than the effective dwell time threshold... This is determined to be a pre-staying state; If the duration is greater than or equal to the effective stay duration threshold Marked as a valid stay status; When a customer first switches from a pre-stay status to a valid stay status, the timestamp of that frame is marked as the start time of the stay. ; If the consecutive frame displacement D ≥ displacement threshold The system determines when a customer begins to move. When the geometric center coordinates of the customer's border leave the stopping area, the stop is considered to have ended, and the corresponding frame timestamp of the state transition is marked as the end of the stop. ; Based on the start time and end time The total dwell time is calculated, that is, the total dwell time = - .

[0022] For the extraction of micro-behaviors related to product interaction, a human pose estimation algorithm and a hand key point detection algorithm are used. Video frames containing customer hands and products in the area where the product is displayed are used as input. By detecting hand key points and product bounding boxes, the spatial overlap, contact state, and hand pose changes of the hand and product are determined, thereby recognizing actions such as picking up, touching, multi-product comparison, and product return. Customer movements are identified using human pose estimation and hand keypoint detection algorithms, specifically through the following methods: The system acquires a cropped sequence of video frames within the customer's dwell area, along with the store's camera intrinsics and region mask. Using the HRNet human pose model, the video frame sequence is input into the model to calculate the customer's shoulder position. elbow and wrist The coordinates of the attitude key points; Construct elbow and shoulder vectors Elbow and wrist vectors arm lifting angle The calculation is performed using the vector dot product, and the formula is as follows: ; When the arm is raised at an angle When the arm raise angle is greater than or equal to the threshold for the lifting action, it is determined as an arm raise; otherwise, it is determined as an arm lowering. The wrist region is extracted from the video frame as the region of interest for the hand, and the palm point is obtained using the MediaPipe Hands hand keypoint model. Calculate the coordinates of the palm point ; Perform product detection on video frames and output product bounding boxes. ,in, This represents the bounding box of the m-th item, where m is the item number. This represents the coordinates of the top-left corner of the bounding box of the m-th item. Represent the coordinates of the bottom right corner of the bounding box of the m-th product, and calculate the coordinates of the product center. Product area mask , specifically, , ; Based on the above results, calculate the coordinates of the palm center point. Coordinates of the product center distance The calculation formula is: ; Calculate the circumscribed rectangle of the hand With product frame The intersection-union ratio (IUR) is calculated. If the IUR is greater than 0, it indicates that the hand overlaps with the product space. For judging the action of touching the product, if If the distance between the hand and the product is less than or equal to the threshold, and the crossover ratio is 0, and the arm is not raised, it is determined to be a touching action on the product. If the cross-union ratio is greater than 0 within 3 consecutive frames and the arm is raised, it is determined to be an action of picking up the goods; In a frame sequence, if the arm changes from raising to lowering and the crossover ratio changes from greater than 0 to 0, it is determined as a product return action; Based on the total dwell time calculated above, and combined with the video frame rate, the duration of the action is calculated. In this embodiment, only actions whose duration is greater than or equal to the effective duration threshold are retained.

[0023] For the extraction of visual attention-related micro-behaviors, by inputting video frames of the customer's facial area, key points such as the eyes and face are located, and by calculating the gaze direction, gaze target location, gaze duration and gaze angle, the gaze start and end time, gaze angle and gaze target are obtained. The calculation of the gaze start and end time, gaze angle, and gaze target is specifically achieved through the following method: Continuous video frames of the facial region, cropped to the area where the customer lingers, are input into the MediaPipe Face Mesh model. For the gaze angle, which includes both horizontal and vertical angles, calculations are performed using the dot product and modulus operation of eye geometric vectors. Specifically, for the horizontal gaze angle, the vertical offset is calculated by extracting the vertical coordinates of the pupil and the coordinates of the upper and lower edges of the eye, and then the average value is taken. The calculation formula is as follows: ; in, Represents the horizontal reference vector for the left eye. This represents the left eye gaze vector. Represents the horizontal reference vector for the right eye. Represents the right eye gaze vector; When vertical offset >0 indicates a gaze to the right, with a vertical offset of [value missing]. <0 indicates gazing to the left, when the vertical offset is less than 0. =0 indicates that you are looking straight ahead; For the vertical gaze angle, the vertical offset is calculated by extracting the vertical coordinates of the pupil and the coordinates of the upper and lower edges of the eye, and then the average value is taken. The calculation formula is as follows: ; in, , These represent the vertical pixel coordinates of the centers of the left and right pupils, respectively. , These represent the vertical pixel coordinates of the upper eyelids of the left and right eyes, respectively. , These represent the vertical pixel coordinates of the lower eyelids of the left and right eyes, respectively; According to the calculation results, if >0.5 indicates gazing downwards. <0.5 indicates gazing upwards; if =0.5 indicates eye level; For the calculation of gaze target localization, the gaze angle is projected as the image gaze point. By combining product coordinates with the gaze target, the center coordinates of all products within the video frame are traversed. Calculate the Euclidean distance between the gaze point and the center of the goods using the following formula: ; According to the calculation results, if If the distance to the gaze target is less than or equal to the gaze target matching threshold, the product is determined to be the current gaze target. When consecutive frames satisfy the condition that the gaze target is fixed and the cumulative duration is greater than or equal to the effective gaze duration threshold, the timestamp of the first frame that meets the condition is recorded as the gaze start time; when the gaze angle changes abruptly, the gaze target changes or the face leaves the frame, the timestamp of the frame that changes the state is recorded as the gaze end time. The gaze duration is calculated based on the gaze start time and gaze end time.

[0024] For the extraction of social interaction micro-behaviors, the SlowFast video behavior recognition algorithm and multi-human target detection are used. The input data is a video clip of multiple people in the area where they stay. The number of customers in the same group, human posture and body movements are detected. The interaction type, participating customer ID and interaction duration are obtained by using the recognized interactive behaviors such as pointing at products, face-to-face communication and body gestures. The calculation of interaction type, participating customer ID, and interaction duration is achieved through the following method: For participating customer IDs, a unique ID is assigned to each customer using the ByteTrack tracker described above. Whether two customers are face-to-face is determined by calculating the Euclidean distance between their centers. The calculation method is as follows: ; in, , , and These are the coordinates of two customers. Using the SlowFast video behavior recognition algorithm, spatiotemporal features of the video are extracted, and finger pointing and body gestures are identified. Combined with spatial relationships, the following judgments are made: If the key point of the hand points to the product area, and the distance between the two customers is less than the human body distance threshold, it is determined that the finger is pointing to the product. If the distance between two customers is less than the human body distance threshold and the angle between their faces is less than the angle between their faces, it is determined to be face-to-face communication. If the body movements match the standard and the interaction continues, it is judged as a body gesture. The interaction duration is calculated by taking the timestamp of the first frame that satisfies the above interaction type as the interaction start time and the timestamp of the interaction end frame as the interaction end time.

[0025] In one embodiment of the present invention, the causal nodes include consumer behavior feature nodes, environmental confounding variable nodes, and consumer conversion result nodes. Time-series data corresponding to each causal node are obtained based on the feature set of consumer behavior features, environmental confounding variable data, and consumer conversion result data.

[0026] The steps of inputting time-series data into a causal structure learning algorithm, traversing the causal relationships between each causal node, and generating a causal graph include: S21: The time-series data of each causal node that has been acquired are time-series normalized according to the timeline of a single customer to form a time-series dataset; S22: Use the causal structure learning algorithm to traverse the time series dataset, perform independence verification on any two causal nodes to determine whether there is a direct causal dependency between the two causal nodes, and generate verification results; S23: Based on the temporal sequence logic of the time series dataset and combined with the verification results, determine the directed causal relationship between two causal nodes; S24: Based on the directed causal relationship, connect the causal nodes that have causal effects with directed edges to obtain the causal graph.

[0027] In this embodiment of the invention, as shown in steps S21 to S24 above, for the generation of verification results, the time series dataset is input into the PC algorithm to determine the causal node set, wherein the causal nodes include consumer behavior feature nodes, environmental confounding variable nodes, and consumer conversion result nodes. Conditional independence verification rules are set, and all causal nodes are incorporated into the graph structure as vertices. Node pairs are set, and for each pair of nodes, conditional independence verification is performed sequentially to determine whether there is a direct correlation between each pair of nodes. The node pairs include environmental confounding variable node-consumer behavior feature node, consumer behavior feature node-consumer conversion result node, and environmental confounding variable node-consumer conversion result node. When performing conditional independence checks, the appropriate check method needs to be selected based on the node data type. For discrete nodes (e.g., whether a promotion was launched or whether a transaction was completed), a chi-square independence test is used. Specifically, the values ​​of the two discrete nodes to be tested are cross-grouped, and the actual observation frequency of each group in the time series data is counted. To generate a two-dimensional contingency table, the total rows, total columns, and total number of samples in the contingency table are plotted, and the theoretical expected frequency for each group is calculated. The calculation formula is as follows: ; The chi-square statistic is calculated based on the deviation between the observed frequency and the expected frequency. The formula is as follows: ; This statistic quantifies the degree of deviation between the actual distribution of the two nodes and the independent theoretical distribution. Calculate the degrees of freedom df, i.e. Set the significance level and look up the corresponding critical value in the chi-square distribution table. ; If calculated If the conditions of two nodes are not independent, a direct causal relationship exists; otherwise, if the conditions of two nodes are independent, a direct causal relationship exists. For continuous nodes (such as dwell time, gaze duration, and average order value), the Fisher-Z test is used. Specifically, the two continuous nodes to be tested are... The time series data of the condition variable set Z are aligned with the frames and time axis. After eliminating the interference of the condition variable Z, the partial correlation coefficient P between X and Y is calculated. The partial correlation coefficient P is transformed using the Fisher-Z transform to become a Z-value that conforms to a normal distribution. The formula for its calculation is as follows: ; Let the total sample size be n and the number of condition variables be k. Using the standard error SE in the Fisher-Z test, i.e. The test statistic was calculated. ; Set the significance level, look up the two-sided critical value in the standard normal distribution table. If the test statistic is greater than the critical value, the two nodes are determined to be conditionally independent and have a direct association; if the test statistic is less than or equal to the critical value, the two nodes are determined to be conditionally independent and have no direct association. During the verification process, other relevant causal nodes are also introduced as control variables to determine whether the nodes in the node pair can be independent of each other when other relevant causal nodes exist.

[0028] After the verification is completed, if the nodes in the node pair are determined to be independent, it means that there is no direct causal relationship between the two nodes, only an indirect relationship or no relationship. The algorithm marks the node pair as having no direct dependency and does not retain the edge. If the nodes in the node pair are not conditionally independent, it means that there is a direct causal dependency between the two nodes. The algorithm marks the node pair as having a direct dependency, retains the node pair, and adds an edge to the undirected graph to connect them, which is used to determine the directed causal direction later. After traversal, delete all node pairs marked as having no direct dependency, and only retain node pairs and edges with direct causal dependency, to obtain the conditional independence verification result set and the undirected causal skeleton graph.

[0029] In one embodiment of the present invention, the hierarchical division of confounding variables specifically includes three layers: the first layer is environmental confounding variables, the second layer is human intervention confounding variables, and the third layer is user-inherent attribute confounding variables.

[0030] The steps of hierarchically segmenting confounding variables based on a causal graph, locating the preceding interference paths of confounding variables in the causal graph, and blocking the bidirectional transmission link between confounding variables and consumption conversion results based on the preceding interference paths to obtain the true causal graph of consumption behavior and consumption conversion results include: S31: Determine the preceding source node of the confounding variable in the causal graph, locate the path between the confounding variable and the consumer behavior characteristics, and the path between the confounding variable and the consumption conversion result, so as to obtain the preceding interference path. S32: Based on the preceding interference path, block the bidirectional transmission link of the mixed variables to obtain the true causal graph.

[0031] The step of blocking the bidirectional transmission link of profane variables based on the preceding interference path to obtain the true causal graph includes: S321: For each preceding interference path, sever the bidirectional connection between the confounding variables and the consumption behavior characteristics and consumption conversion results. S322: For the blocked causal graph, based on the hierarchical confounding variables, the interfering links are removed to obtain the true causal graph.

[0032] In this embodiment of the invention, as shown in steps S31 to S32 above, the hierarchical division of confounding variables is based on the source of interference, the timing of action, and the scope of influence. The confounding variable nodes in the causal graph are divided into three layers to achieve the structured classification of interference factors. The first layer of environmental confounding variables are external objective factors, including customer flow density, holidays, store promotions, business district activities, weather and sunlight, etc. The second layer of human intervention confounding variables are on-site active guidance factors, including sales guide active promotion, peer interaction and communication, and group conformity influence, etc. The third layer of user inherent attribute confounding variables are the stable characteristics of the customer themselves, including consumption level, age group, single person, and group consumption habits. In the causal graph, the effects of confounding variables precede those of consumer behavior characteristics and consumption conversion results. Therefore, all confounding variable nodes are identified as upstream source nodes, located at the beginning of the causal chain. By traversing the output chain of each confounding variable node, we can locate the two transmission paths that exist simultaneously for that confounding variable node. One path is from the confounding variable node to the consumer behavior characteristic node, such as behaviors like interfering with lingering, staring, or product interaction. The other path is from the confounding variable node to the consumption conversion result node. The two aforementioned related transmission paths are uniformly marked as pre-interference paths to complete the location of pre-interference paths for confounding variables. For each located pre-interference path, a blocking operation is performed. Specifically, the two directed edges from the confounding variable node to the consumer behavior feature node and from the confounding variable node to the consumer conversion result node are disconnected respectively. Following the division order of the three layers of confounding variables, i.e., first removing all interference links corresponding to environmental confounding variables, then removing all interference links corresponding to human intervention confounding variables, and finally removing all interference links corresponding to user inherent attribute confounding variables, interference is eliminated from the causal graph in sequence. The remaining nodes and effective links are then restructured to form a true causal graph.

[0033] To construct counterfactual scenarios, we traverse each type of consumer behavior in the consumer behavior feature set, including dwell time, product interaction, visual attention, and social interaction. We construct the scenario by comparing it with real consumer scenarios. We only reverse the target consumer behavior to be analyzed, reconstructing the state that the behavior actually occurred as if it did not occur. Based on the individual customer's personal behavior timeline, we rearrange the behavior sequence under the counterfactual state. After removing the target behavior, we keep the order and duration of the remaining behaviors unchanged, thus forming a counterfactual scenario with the target behavior as an independent controlled variable. The counterfactual scenario is input into the real cause-effect graph to obtain simulation results, which include quantitative conversion indicators such as the probability of transaction, purchase likelihood, and expected average order value under the counterfactual scenario. By aligning and comparing real-world behavior occurrences and actual consumption conversion results with counterfactual simulation results of no behavior occurrence and simulated consumption conversion results on the same customer timeline, we can determine the true direction of the behavior's impact on conversion results. The direction of impact includes positive improvement, no impact, and false association.

[0034] In one embodiment of the present invention, the positive causal behavior is the behavior that directly leads to consumption conversion; the negative causal behavior is the consumption behavior generated when the customer has a prior purchase intention; and the spurious correlation behavior is the consumption behavior that is driven only by confounding variables and has no direct causal relationship with consumption conversion.

[0035] The generation of corresponding marketing decisions includes: Generate shelf-optimized marketing decisions based on positive causal behavior; Targeting reverse causal behavior, generate non-intrusive information service marketing strategies; To address false related behaviors, generate marketing strategies that do not implement marketing interventions.

[0036] In this embodiment of the invention, based on the direction of influence and the timing of the behavior, positive causal behavior, negative causal behavior, and falsely related behavior are identified. Positive causal behavior indicates that the conversion result in the real scenario is significantly higher than the result in the counterfactual simulation, indicating that the behavior directly increases the probability of a transaction and the average order value. It is a driving behavior that actively promotes consumption conversion. Negative causal behavior indicates that there is no significant difference in the conversion results between the real scenario and the counterfactual scenario. This behavior occurs after the customer's purchase intention is formed, indicating that the customer has a purchase intention before the behavior occurs. The behavior is an external manifestation of the conversion result. Falsely related behavior indicates that there is no direct causal relationship between the behavior and the consumption conversion. Stores generate targeted marketing decisions based on the causal attributes of the consumption behavior.

[0037] The foregoing has shown and described the basic principles, main features, and advantages of the present invention. Those skilled in the art should understand that the present invention is not limited to the above embodiments. The embodiments and descriptions in the specification are merely principles of the invention. Various changes and modifications can be made to the invention without departing from its spirit and scope, and all such changes and modifications fall within the scope of the claimed invention. The scope of protection claimed by the appended claims and their equivalents is defined.

Claims

1. A method for analyzing offline consumer behavior and making marketing decisions by integrating image recognition, characterized in that: Includes the following steps: Acquire store video data, consumption conversion result data, and environmental confounding variable data. Use image recognition to detect customers and products in store images to obtain customer behavior trajectories. Based on the behavior trajectories, locate the customer dwell area and use image recognition to extract micro-behaviors related to consumption decisions. Record the order of occurrence of micro-behaviors and string them together. Label the micro-behaviors and integrate all the labeled information to generate a feature set with consumption behavior characteristics. Set causal nodes, obtain the time series data corresponding to the causal nodes, input the time series data into the causal structure learning algorithm, traverse the causal pointing relationship between each causal node, and generate a causal graph; Based on the causal graph, confounding variables are hierarchically divided, the preceding interference paths of confounding variables in the causal graph are located, and according to the preceding interference paths, the bidirectional transmission link of confounding variables acting simultaneously on consumer behavior characteristics and consumption conversion results is blocked, thus obtaining the true causal graph of consumer behavior and consumption conversion results. Based on each type of consumer behavior in the feature set, and combined with behavioral data and consumption conversion results in real scenarios, a corresponding counterfactual scenario is constructed. The counterfactual scenario is simulated based on the real cause-effect graph to obtain the simulation results. By inputting real-world behavioral data, consumption conversion results, and simulation results into a real causal graph, the direction of the influence of consumption behavior on consumption conversion results can be obtained, thus identifying positive causal behavior, negative causal behavior, and falsely related behavior. Based on positive causal behavior, negative causal behavior, and false relevance behavior, corresponding marketing decisions are generated.

2. The offline consumer behavior analysis and marketing decision-making method based on fused image recognition as described in claim 1, characterized in that: Based on behavioral trajectories, image recognition is used to identify various micro-behaviors related to consumer decisions in customer dwell areas. These micro-behaviors include dwell time, product interaction, visual attention, and social interaction. Dwell time indicates the duration of time spent in the dwell area; product interaction indicates picking up, comparing, or touching products; visual attention indicates the duration and angle of product gaze; and social interaction indicates interactions between customers pointing towards products.

3. The offline consumer behavior analysis and marketing decision-making method based on image recognition as described in claim 1, characterized in that: Using the moment a single customer enters the store as their personal timeline, micro-behaviors are concatenated according to their chronological order to obtain a behavioral temporal feature vector, which is then labeled with information. The annotation information includes the time period when the behavior occurred, the type of behavior, and the duration of the behavior.

4. The offline consumer behavior analysis and marketing decision-making method based on fused image recognition according to claim 1, characterized in that: The causal nodes include consumer behavior feature nodes, environmental confounding variable nodes, and consumer conversion result nodes. Based on the feature set of consumer behavior features, environmental confounding variable data, and consumer conversion result data, the time series data corresponding to each causal node is obtained.

5. The offline consumer behavior analysis and marketing decision-making method based on fused image recognition according to claim 1, characterized in that, The steps of inputting time-series data into a causal structure learning algorithm, traversing the causal relationships between each causal node, and generating a causal graph include: The time-series data of each causal node that has been acquired are time-series normalized according to the timeline of a single customer to form a time-series dataset; The causal structure learning algorithm is used to traverse the time series dataset and perform independence verification on any two causal nodes to determine whether there is a direct causal dependency between the two causal nodes and generate verification results. Based on the temporal sequence logic of the time series dataset and combined with the verification results, the directed causal relationship between two causal nodes is determined. Based on the directed causal relationship, causal nodes that have causal effects are connected by directed edges to obtain a causal graph.

6. The offline consumer behavior analysis and marketing decision-making method based on fused image recognition according to claim 1, characterized in that: The hierarchical classification of confounding variables specifically includes three layers: the first layer is environmental confounding variables, the second layer is human intervention confounding variables, and the third layer is user-inherent attribute confounding variables.

7. The offline consumer behavior analysis and marketing decision-making method based on fused image recognition according to claim 1, characterized in that, The steps of hierarchically segmenting confounding variables based on a causal graph, locating the preceding interference paths of confounding variables in the causal graph, and blocking the bidirectional transmission link between confounding variables and consumption conversion results based on the preceding interference paths to obtain the true causal graph of consumption behavior and consumption conversion results include: Identify the leading source nodes of confounding variables in the causal graph, locate the paths between confounding variables and consumer behavior characteristics, and the paths between confounding variables and consumption conversion results, in order to obtain the leading interference paths. By blocking the bidirectional transmission links of the confounding variables based on the preceding interference path, the true causal graph is obtained.

8. The offline consumer behavior analysis and marketing decision-making method based on fused image recognition according to claim 7, characterized in that, The step of blocking the bidirectional transmission link of profane variables based on the preceding interference path to obtain the true causal graph includes: For each preceding interference path, the bidirectional connection between confounding variables and consumer behavior characteristics and consumption conversion results is severed. For the blocked causal graph, the true causal graph is obtained by removing interfering links based on the hierarchical confounding variables.

9. The offline consumer behavior analysis and marketing decision-making method based on fused image recognition according to claim 1, characterized in that: The positive causal behavior refers to the behavior that directly leads to consumption conversion; the negative causal behavior refers to the consumption behavior generated when the customer has a prior purchase intention; the spurious correlation behavior refers to the consumption behavior that is driven only by confounding variables and has no direct causal relationship with consumption conversion.

10. The offline consumer behavior analysis and marketing decision-making method based on fused image recognition according to claim 1, characterized in that, The generation of corresponding marketing decisions includes: Generate shelf-optimized marketing decisions based on positive causal behavior; Targeting reverse causal behavior, generate non-intrusive information service marketing strategies; To address false related behaviors, generate marketing strategies that do not implement marketing interventions.

Images