Retail data processing method and system for retail commodity display evaluation

Abstract

The invention provides a retail data processing method and system for retail commodity display evaluation. According to the method, customer interaction behavior data and shelf three-dimensional space data are synchronously acquired through a monitoring device deployed in a commodity display area, and a commodity displacement track and human body action characteristics are extracted from a video stream by using an image recognition technology. And establishing a space-time correlation model to match the behavior data with spatial distribution, and generating an evaluation parameter group including a space occupancy density index, a visual focus distribution rate and a contact behavior delay value. A two-factor optimization mechanism is innovatively introduced, vertical direction optimization factors are generated according to the ratio relation between the space density and the visual focus, dynamic adjustment of the goods shelf height direction distribution density and the horizontal display orientation is achieved, and a closed-loop feedback type display decision-making system is constructed. According to the technical scheme provided by the invention, adaptive optimization of the commodity display layout is realized, the commodity display efficiency is effectively improved, and the customer interaction experience is optimized.

Description

Technical Field

[0001] This application relates to the technical fields of computer vision, spatial computing, and data-driven decision-making, and particularly relates to a retail data processing method and system for retail commodity display evaluation. Background Art

[0002] In the new retail scenario of omnichannel integration, consumers' behavior trajectories are scattered across multiple platform touchpoints. It is necessary to evaluate the impact of commodity display layouts on consumers' decision-making paths through cross-channel data integration and correlation analysis. The core requirements include: preliminary collection and coarse-grained correlation of cross-channel behavior data; evaluation of display effects based on static statistical analysis, such as mining the simple correlation between display parameters and sales indicators through historical data; and support for periodic adjustment of display strategies.

[0003] Currently, some technical solutions adopt data association and offline statistical analysis based on membership systems. This solution realizes cross-channel behavior data collection by forcing users to log in to an online-offline integrated membership account. Offline, it obtains the rough movement trajectories of users in the store through POS transaction data and simple regional heat maps. Matches commodity display parameters with static indicators such as historical sales data and member consumption frequencies to generate periodic display optimization suggestions.

[0004] However, the above solutions have some limitations: relying on the forced login membership system results in the complete loss of behavior data of unlogged users, and the cross-channel data coverage rate is not high; the analysis model can only capture linear correlations and cannot identify the non-linear associations between display parameters and consumers' complex behaviors, with a high risk of misjudgment; both data collection and analysis are in an offline batch processing mode, unable to perceive changes in consumers' behaviors in real time, resulting in a serious lag in the adjustment of display strategies behind market dynamics. Summary of the Invention

[0005] This application provides a retail data processing method and system for retail commodity display evaluation to solve the problem of low cross-channel data coverage rate in the prior art.

[0006] In a first aspect, this application provides a retail data processing method for retail commodity display evaluation, including:

[0007] Obtaining an interaction behavior dataset of customers and commodities through monitoring devices set in the commodity display area, and simultaneously obtaining three-dimensional spatial distribution data of the commodities displayed on the shelves;

[0008] Performing image recognition processing on the continuous video stream collected by the monitoring device to extract a dynamic feature set including commodity displacement trajectories and human body movement forms;

[0009] Perform spatio-temporal correlation matching on the interactive behavior dataset and the three-dimensional space distribution data, and generate an evaluation parameter group for display effects including a space occupancy density index, a visual focus distribution rate, and a contact behavior delay value according to the matching result and the change parameters corresponding to the product displacement trajectories in the dynamic feature set;

[0010] Determine a first optimization factor according to the ratio relationship between the space occupancy density index and the visual focus distribution rate in the evaluation parameter group for display effects, and at the same time determine a second optimization factor according to the association rule between the contact behavior delay value and a preset product category;

[0011] Adjust the distribution density of products in the shelf height direction based on the first optimization factor, and adjust the display orientation of products on the horizontal display surface based on the second optimization factor.

[0012] Optionally, perform time-space correspondence association on the behavior event time stamps of customers touching products in the interactive behavior dataset and the product display position coordinates in the three-dimensional space distribution data to establish a correspondence table between the customer positions with time stamps and the product areas as the matching result;

[0013] According to the total number of times the product is moved and the length of the customer stay period corresponding to each movement in the matching result, divide the product of the total number and the average value of the stay period lengths by the space occupancy volume of the corresponding shelf partition to generate a space occupancy density index representing the degree of product aggregation;

[0014] Extract a continuous image group with continuous changes in the customer's head direction from the human motion forms in the dynamic feature set, and combine the length of the customer stay period in the matching result to generate a visual focus distribution rate representing the visual attention intensity;

[0015] According to the time difference between the time stamp of the first behavior event when the customer enters the shelf partition and the start time of the product displacement trajectory in the matching result, calculate the median value of the first contact time differences of all customers in the same partition to generate a contact behavior delay value reflecting the product accessibility efficiency;

[0016] Combine and associate the space occupancy density index, the visual focus distribution rate, and the contact behavior delay value according to three-dimensional space coordinates to form an evaluation parameter group for display effects.

[0017] Optionally, extract the behavior event time stamps of customers touching products from the interactive behavior dataset, and at the same time obtain the product display position coordinates of the customers from the three-dimensional space distribution data, and unify the behavior event time stamps and the product display position coordinates to the same time axis based on the system time axis;

[0018] Convert the coordinates of the product display positions according to the coordinate system of the three-dimensional space distribution data to determine the shelf area identifier where the customer is located under each time stamp. The shelf area identifier is generated by combining the layer number, column number, and depth distance of the product display position;

[0019] Generate an initial record table including customer identifiers, shelf area identifiers, and product displacement status based on the shelf area identifier;

[0020] When the start time of the movement of the product display position coordinates is monitored, retrieve the time stamps of the customer's behavior events of contacting the product on the same time axis in the initial record table, and synchronously append the change relationship between the duration of the product movement process in the time stamps of the customer's behavior events of contacting the product and the change of the product shelf area identifier to the initial record table to form a corresponding relationship table of the customer's position and the product area with time stamps as the matching result.

[0021] Optionally, calculate the real-time ratio of the space occupancy density index to the visual focus distribution rate. When the real-time ratio is greater than the preset high ratio range, convert the real-time ratio into a positive adjustment coefficient in the shelf height direction. When the real-time ratio is less than the preset low ratio range, convert it into a negative adjustment coefficient in the shelf height direction;

[0022] Generate a first optimization factor according to the product relationship between the cumulative number of times of the positive adjustment coefficient or negative adjustment coefficient in a continuous time period and the number of layers in the shelf height direction;

[0023] Compare the contact behavior delay value with the associated rule standard delay range corresponding to the preset product category, and generate an azimuth matching difference for the same type of product according to the degree to which the contact behavior delay value exceeds or is lower than the standard delay range;

[0024] Generate a second optimization factor based on the proportional relationship between the azimuth matching difference and the historical contact position distribution density of the preset product category on the horizontal display surface.

[0025] Optionally, divide the first optimization factor evenly according to the number of layers in the shelf height direction to obtain the unit adjustment amount corresponding to each layer;

[0026] Determine the direction of increasing or decreasing the product spacing of the corresponding layer according to the positive or negative sign of the unit adjustment amount, and update the distribution density of the products in the corresponding layer in the shelf height direction according to the product of the absolute value of the unit adjustment amount and the preset spacing base value;

[0027] Determine the movement direction identifier of the horizontal display surface according to the positive or negative sign of the second optimization factor, where the positive identifier indicates moving towards the area with a high density of historical contact positions, and the negative identifier indicates moving towards the area with a low density;

[0028] Perform a proportional conversion of the absolute value of the second optimization factor and the side length of the regional division grid of the horizontal display surface to obtain the horizontal movement distance, and adjust the display orientation of the commodity in the corresponding grid according to the movement direction identifier and the horizontal movement distance.

[0029] Optionally, extract the maximum allowed delay value and the minimum allowed delay value defined in the preset commodity category, and divide the difference interval between the maximum allowed delay value and the minimum allowed delay value into at least three equal sub-intervals;

[0030] Mark the position of the contact behavior delay value according to the difference interval. If the contact behavior delay value is in the right interval of the maximum allowed delay value, calculate the excess amount exceeding the maximum allowed delay value as the positive difference amount. If it is in the left interval of the minimum allowed delay value, calculate the shortage amount less than the minimum allowed delay value as the negative difference amount;

[0031] Calculate the ratio of the positive difference amount or the negative difference amount to the width of the equal sub-interval to generate an orientation matching difference.

[0032] Optionally, perform moving object segmentation on the difference regions between adjacent frames in the continuous video stream collected by the monitoring device, mark the regions that meet the commodity size threshold in the segmented moving objects as candidate commodity regions, and mark the remaining regions as human body candidate regions;

[0033] Perform direction continuity verification on the position coordinates of the candidate commodity regions in consecutive frames. If the coordinate change direction of the same commodity in adjacent frames is consistent with the movement direction trend, record its movement path points and connect them as the commodity displacement trajectory;

[0034] Lock the positions of the torso and limb key points in the human body candidate regions, generate an attitude change sequence according to the position change direction of the key points in adjacent frames, and mark the continuous direction combinations that meet the preset action forms in the attitude change sequence as human action forms;

[0035] Associate the start position of the commodity displacement trajectory and the corresponding human action form within the same time period, and merge them to form a dynamic feature set including the commodity displacement trajectory and the human action form.

[0036] In a second aspect, the present application provides a retail data processing system for retail commodity display evaluation, including:

[0037] An acquisition module that acquires a dataset of customer-product interaction behaviors through monitoring devices set in the product display area, and synchronously acquires three-dimensional spatial distribution data of the products displayed on the shelves;

[0038] A processing module that performs image recognition processing on the continuous video stream collected by the monitoring device, and extracts a set of dynamic features including product displacement trajectories and human motion forms;

[0039] A generation module that performs spatio-temporal correlation matching on the interaction behavior dataset and the three-dimensional spatial distribution data, and generates an evaluation parameter group of display effects including a space occupancy density index, a visual focus distribution rate, and a contact behavior delay value according to the matching result and the change parameters corresponding to the product displacement trajectories in the set of dynamic features;

[0040] A determination module that determines a first optimization factor according to the ratio relationship between the space occupancy density index and the visual focus distribution rate in the display effect evaluation parameter group, and simultaneously determines a second optimization factor according to the association rule between the contact behavior delay value and a preset product category;

[0041] An adjustment module that adjusts the distribution density of products in the vertical direction of the shelves based on the first optimization factor, and adjusts the display orientation of products on the horizontal display surface based on the second optimization factor.

[0042] In a third aspect, an embodiment of the present application provides a computing device, including a processing component and a storage component; the storage component stores one or more computer instructions; the one or more computer instructions are used to be called and executed by the processing component to implement a retail data processing method for retail product display evaluation as described in the first aspect above.

[0043] In a fourth aspect, an embodiment of the present application provides a computer storage medium storing a computer program, and when the computer program is executed by a computer, it implements a retail data processing method for retail product display evaluation as described in the first aspect.

[0044] In the embodiment of the present application, an interaction behavior dataset of customers and products is obtained through a monitoring device arranged in the product display area, and three-dimensional spatial distribution data of the products displayed on the shelves is synchronously obtained; image recognition processing is performed on the continuous video stream collected by the monitoring device to extract a set of dynamic features including product displacement trajectories and human motion forms; the interaction behavior dataset is subjected to spatio-temporal correlation matching with the three-dimensional spatial distribution data, and according to the matching result and the change parameters corresponding to the product displacement trajectories in the set of dynamic features, a set of display effect evaluation parameters including a space occupancy density index, a visual focus distribution rate, and a contact behavior delay value is generated; a first optimization factor is determined according to the ratio relationship between the space occupancy density index and the visual focus distribution rate in the set of display effect evaluation parameters, and at the same time, a second optimization factor is determined according to the association rule between the contact behavior delay value and a preset product category; based on the first optimization factor, the distribution density of products in the vertical direction of the shelves is adjusted, and based on the second optimization factor, the display orientation of products on the horizontal display surface is adjusted.

[0045] The technical solution of the present application has the following beneficial effects:

[0046] By performing spatio-temporal correlation modeling to dynamically bind customer contact behaviors, visual focus distributions, and product spatial coordinates, multi-dimensional parameters (aggregation degree, attention degree, reach efficiency) are generated to realize the transformation from raw data to decision-making indicators; based on parameter logic, a vertical density optimization factor (balancing space occupancy and visual attractiveness) and a horizontal orientation optimization factor (matching product attributes and reach efficiency) are derived, ultimately driving the three-dimensional intelligent adjustment of the shelf layout, solving the problems of the separation between behavior and space and the invalidation of static rules in traditional display evaluation, and improving product display efficiency and customer interaction experience.

[0047] Further, by correlating the time stamps of customer contact behaviors with the three-dimensional coordinates of products through spatio-temporal correlation, a time-stamped customer-product position mapping relationship is established; the space occupancy density index is calculated based on the number of product movements, customer stay duration, and space volume to quantify the product aggregation degree; the visual focus distribution rate is generated by combining the head direction change images and the stay duration to reflect the visual attention intensity; the contact behavior delay value is calculated through the median value of the first contact time difference to evaluate the reach efficiency of products; finally, the above parameters are correlated according to the three-dimensional coordinates to form a multi-dimensional evaluation system. By quantifying the dynamic association between customer behaviors and product spatial distributions, high-concern product areas and inefficient display blind spots are accurately identified, providing data support for optimizing the vertical density distribution and horizontal orientation layout of the shelves, realizing dynamic adjustment of the display based on real interaction behaviors, effectively improving product exposure efficiency, optimizing space resource allocation, and shortening the customer decision-making path, forming a closed-loop intelligent display optimization mechanism.

[0048] These aspects or other aspects of the present application will be more clearly understood in the following description of the embodiments. BRIEF DESCRIPTION OF THE DRAWINGS

[0049] In order to more clearly illustrate the technical solutions in the embodiments of the present application or the prior art, the following will briefly introduce the drawings required for the description of the embodiments or the prior art. Obviously, the drawings in the following description are some embodiments of the present application. For those of ordinary skill in the art, without creative efforts, other drawings can also be obtained based on these drawings.

[0050] Figure 1 Shows a flowchart of a retail data processing method for retail commodity display evaluation provided by the present application;

[0051] Figure 2 Shows a schematic structural diagram of a retail data processing system for retail commodity display evaluation provided by the present application;

[0052] Figure 3 Shows a schematic structural diagram of a computing device provided by the present application. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0053] In order to enable those skilled in the art to better understand the solutions of the present application, the following will clearly and completely describe the technical solutions in the embodiments of the present application with reference to the drawings in the embodiments of the present application.

[0054] In some of the processes described in the specification, claims and above-mentioned drawings of the present application, a plurality of operations that appear in a specific order are included. However, it should be clearly understood that these operations may not be executed in the order in which they appear in this article or may be executed in parallel. The serial numbers of the operations, such as 101, 102, etc., are only used to distinguish the different operations, and the serial numbers themselves do not represent any execution order. In addition, these processes may include more or fewer operations, and these operations may be executed in sequence or in parallel. It should be noted that the descriptions such as "first" and "second" in this article are used to distinguish different messages, devices, modules, etc., and do not represent a sequence, nor do they limit that "first" and "second" are of different types.

[0055] Researchers have found that traditional retail display evaluation relies on manual observation or static data analysis, which has problems such as being unable to capture the dynamic behavior of customers in real time and the disconnection between the spatial distribution of goods and interaction behaviors, resulting in a lag in display adjustment and a lack of quantitative basis. Based on this, an intelligent display optimization method based on multi-source data fusion is provided. This method can synchronously collect customer interaction behaviors and three-dimensional data of the shelf through monitoring devices, extract the displacement trajectories of goods and human motion features through image recognition, construct a spatio-temporal correlation model to generate multi-dimensional evaluation parameters (space occupancy density, visual focus distribution, contact delay), and dynamically generate optimization factors based on the logical relationships between the parameters and the rules of commodity attributes to achieve adaptive adjustment of the vertical density and horizontal orientation of the shelf.

[0056] The technical solution of this application can be applied to the real-time optimization of commodity display in scenarios such as supermarket shelves and retail stores, especially in refined operation scenarios that require improving the exposure rate of high-value commodities and shortening the customer decision-making path.

[0057] Next, the technical solutions in the embodiments of this application will be clearly and completely described in conjunction with the accompanying drawings in the embodiments of this application. Obviously, the described embodiments are only a part of the embodiments of this application, rather than all the embodiments. Based on the embodiments in this application, all other embodiments obtained by those skilled in the art without creative efforts shall fall within the protection scope of this application.

[0058] Figure 1 The following is a flowchart of a retail data processing method for retail commodity display evaluation provided for the embodiments of this application, as Figure 1 shown, the method includes:

[0059] 101. Obtain the interaction behavior data set of customers and commodities through the monitoring devices set in the commodity display area, and synchronously obtain the three-dimensional space distribution data of the commodities displayed on the shelf;

[0060] In this step, the monitoring device refers to the video acquisition equipment and sensor network deployed in the commodity display area, including hardware devices such as high-definition cameras, depth sensors, and RFID readers, which are used to collect the interaction behavior data of customers and commodities in real time.

[0061] The interaction behavior data set refers to the structured data set obtained through the monitoring device, including the record of the number of times of reaching hand action triggers, the record of the entity contact holding time, and the record of the visual attention duration in the continuous time dimension.

[0062] The three-dimensional space distribution data refers to the shelf commodity space layout data generated through lidar scanning or computer vision three-dimensional reconstruction technology, including the position coordinates of each commodity unit in the three-dimensional space and the adjacent commodity spacing information.

[0063] In the embodiments of the present application, first, the infrared sensors and cameras in the commodity display area are used to collect the interaction behavior data of customers and commodities in real time (for example, customer A stays in front of the shelf for 30 seconds and touches the commodity 3 times), and at the same time, a three-dimensional laser scanner is used to obtain the three-dimensional spatial distribution data of the commodities displayed on the shelf (for example, commodity B is located on the second layer of the shelf, and the coordinates (x, y, z) = (1.2m, 0.5m, 1.8m)). Secondly, the two types of data are aligned through a timestamp synchronization mechanism to generate an original data set with spatio-temporal tags. Finally, the data is stored in a distributed database (such as MongoDB) to provide input for step 102.

[0064] In the intelligent display optimization project for daily chemical products in a large supermarket, supermarket staff installed multiple groups of high-definition cameras and depth sensors above the shelves. These monitoring devices record the interaction process between customers and commodities all day long. Whenever a customer picks up or touches commodities such as shampoo and conditioner on the shelf, the system will accurately record the time, location, and commodity information when the action occurs. At the same time, the three-dimensional modeling of the shelf is carried out by a laser scanner to obtain the specific placement position and spatial coordinates of each commodity on the shelf. After these data are synchronized, a complete data set containing timestamps, commodity information, and three-dimensional coordinates is formed, laying a foundation for subsequent analysis.

[0065] 102. Perform image recognition processing on the continuous video stream collected by the monitoring device, and extract a dynamic feature set including the displacement trajectory of the commodity and the body movement form.

[0066] In this step, the continuous video stream refers to the uncompressed original video data continuously collected by the monitoring device, which records the customer behavior at a rate of 30 frames per second.

[0067] The dynamic feature set refers to the quantified features extracted from the video stream through computer vision algorithms.

[0068] The displacement trajectory of the commodity is the commodity movement path data generated by target detection and tracking algorithms, including motion parameters such as displacement, speed, and acceleration.

[0069] The body movement form is the coordinates of the key points of the customer's limbs extracted by the pose estimation algorithm, especially the behavior features such as the hand grasping action and the body orientation angle.

[0070] In the embodiments of the present application, first, the continuous video stream in step 101 is frame - decomposed using the OpenCV image - processing library, and the customer's actions (such as "reach for an item" and "stop to watch") and the displacement trajectory of the commodity (such as commodity C moving from the shelf to the customer's hand) are recognized through the YOLOv5 object - detection algorithm. Secondly, based on the optical flow method, the moving speed and direction of the commodity are calculated (such as a displacement speed of 0.3 m / s and a direction angle of 45°), and a dynamic feature set including trajectory coordinates, speed, and action categories is generated. Finally, structured feature data (such as {"commodity C": {"displacement trajectory": [(x1, y1, t1), (x2, y2, t2)], "action type": "fetch"}}) is output, providing a matching basis for step 103.

[0071] When a customer stays in front of the shelf, the system will perform real - time analysis on the video images captured by the camera. Through advanced image - recognition algorithms, the system can accurately recognize the action of the customer picking up a certain bottle of shampoo and track the complete moving trajectory of the commodity from the shelf to the shopping basket. At the same time, the system will also analyze the customer's body posture and hand movements, such as whether the customer carefully checks the label after picking up the commodity, or puts the commodity back on the shelf after hesitation and other behavioral characteristics. These dynamic data are processed to form a detailed record including the commodity displacement trajectory and the customer's behavioral characteristics, helping the supermarket understand the real shopping habits of customers.

[0072] 103. Perform spatio - temporal correlation matching between the interaction behavior data set and the three - dimensional space distribution data, and generate an array of display effect evaluation parameters including a space occupancy density index, a visual focus distribution rate, and a contact behavior delay value according to the matching result and the change parameters corresponding to the commodity displacement trajectory in the dynamic feature set;

[0073] In this step, spatio - temporal correlation matching refers to the mapping relationship established by aligning the interaction behavior data with the three - dimensional space data through timestamp alignment and coordinate transformation.

[0074] The space occupancy density index is the ratio of the number of interacted commodities to the total capacity per unit shelf area, which is obtained by calculating the adjacent commodity spacing information and position coordinates.

[0075] The visual focus distribution rate is the proportion of the residence time of the customer's fixation points in each partition of the shelf generated by the eye - tracking algorithm, which is generated by associating the visual attention duration record with the commodity displacement trajectory.

[0076] The contact behavior delay value refers to the time interval from when the customer first fixes their eyes on the commodity to actual contact, and this value is calculated through time - series analysis.

[0077] In the embodiments of the present application, first, the three-dimensional space distribution data in step 101 is associated with the dynamic feature set in step 102 according to the time-space label to match the customer behavior with the product location (for example, customer D touches product E at time t1, and the location of product E is (1.5m, 0.6m, 1.2m)). Secondly, the evaluation parameters are calculated through the following formula: Space occupancy density index: The number of customer stays per unit area (for example, staying 10 times within the shelf area of 2m 2 → index 5 times / m 2 ); Visual focus distribution rate: The proportion of the total duration that customers fixate on products (for example, the total duration is 120 seconds, and product F is fixated for 60 seconds → distribution rate 50%); Contact behavior delay value: The average time difference from when a customer fixates to actual contact (for example, the average delay is 2.5 seconds). Finally, an evaluation parameter group is generated (for example, {"space density index": 5, "focus distribution rate": 50%, "contact delay": 2.5s}), providing a basis for calculating the optimization factor in step 104.

[0078] The system matches and analyzes the customer behavior data collected with the three-dimensional model of the shelves. By comparing the data in different time periods, the degree of attention and the actual purchase conversion rate of each product area can be calculated. For example, the data shows that although a certain brand of shampoo is often picked up and viewed by customers, the proportion finally put into the shopping cart is not high. The system will also analyze the time interval from when a customer notices a product to finally deciding to purchase. These indicators together constitute a complete parameter system for evaluating the product display effect, providing data support for subsequent optimization and adjustment.

[0079] 104. Determine the first optimization factor according to the ratio relationship between the space occupancy density index and the visual focus distribution rate in the display effect evaluation parameter group, and at the same time determine the second optimization factor according to the association rule between the contact behavior delay value and the preset product category;

[0080] In this step, the display effect evaluation parameter group includes three core indicators: space occupancy density index, visual focus distribution rate, and contact behavior delay value.

[0081] The first optimization factor refers to the normalized ratio of the space occupancy density index to the visual focus distribution rate, which is used to measure the space efficiency of product display.

[0082] The second optimization factor is an adjustment coefficient generated based on a preset rule engine, which associates and analyzes the contact behavior delay value with product category characteristics (such as shelf life, price band).

[0083] The association rule of the preset product category refers to a decision tree model obtained through machine learning training, which defines the optimal delay threshold for different categories of products according to historical sales data.

[0084] In the embodiment of the present application, first, according to the space occupancy density index (5 times / m 2 ) and the visual focus distribution rate (50%) in step 103, calculate the first optimization factor (formula: the first factor = focus distribution rate / space density index = 50% / 5 = 10%·m 2 / time). Secondly, based on the Apriori algorithm for association rule mining, analyze the relationship between the contact behavior delay value (2.5 s) in step 103 and the product category (such as "food category", "daily necessities category") to determine the second optimization factor (for example, the delay threshold for the food category is 2 s, and the actual delay is 2.5 s → factor = 2.5 - 2 = 0.5 s). Finally, integrate the first optimization factor and the second optimization factor and output an optimization factor list (such as {"the first factor": 10%, "the second factor": 0.5 s}) to provide display adjustment parameters for step 105.

[0085] Based on the previous data analysis results, the system will automatically generate optimization suggestions. For example, it is found that although some high-end shampoos are placed in prominent positions, due to price factors, the customer decision-making time is relatively long, and the system will recommend adjusting their display positions. At the same time, for the hot-selling products that are often quickly picked up, the system will recommend increasing their display density. These optimization suggestions are obtained by analyzing a large amount of actual sales data, taking into account both the characteristics of the products and fully respecting the shopping habits of customers.

[0086] 105. Adjust the distribution density of products in the vertical direction of the shelf based on the first optimization factor, and adjust the display orientation of products on the horizontal display surface based on the second optimization factor.

[0087] In this step, the distribution density in the vertical direction of the shelf refers to the degree of density of the spatial arrangement of products on the vertical display layer, which is achieved by adjusting the shelf board spacing or the number of products per layer.

[0088] The display orientation on the horizontal display surface refers to the placement angle and orientation of products on the single-layer plane of the shelf, specifically including parameters such as the rotation angle of the product label surface and the included angle with the aisle.

[0089] The implementation of the first optimization factor is reflected in the control instruction for automatically adjusting the shelf lifting mechanism, while the second optimization factor is converted into a display orientation adjustment plan guided by AR glasses.

[0090] In the embodiment of the present application, first, based on the first optimization factor (10%·m 2 / time) in step 104, adjust the distribution density in the vertical direction of the shelf proportionally (for example, the larger the factor value, the lower the density of products on the high shelf layer. The original density is 5 pieces / m 2 is adjusted to 4 pieces / m 2) Second, adjust the product display orientation on the horizontal display surface according to the second optimization factor (0.5s) (for example, if the delay of food products exceeds the standard, move the products 0.3m forward on the shelf). Finally, execute the layout update through the automated shelf adjustment device, and monitor the adjusted parameters in real time to complete the closed-loop optimization.

[0091] Finally, supermarket staff will adjust the shelf display according to the optimization plan provided by the system. This may include re-planning the hierarchical placement of products, adjusting some products to a more ergonomic height; or changing the placement angle of products to make product labels more visible to customers. In some supermarkets with higher levels of intelligence, it is even possible to optimize product display in real time through automatically adjustable shelves. These adjustments are all based on in-depth analysis of customer shopping behavior, aiming to improve the customer shopping experience and also help the supermarket increase sales performance.

[0092] In summary, through steps 101 to 105, the dynamic and precise control of product display is achieved. Through multi-source data fusion and intelligent analysis technology, a deep association is established between customer behavior characteristics and product spatial distribution, forming a quantifiable display effect evaluation system. The system can automatically identify the differences in product attractiveness and customer decision-making patterns, and intelligently generate targeted display optimization plans to achieve the optimal allocation of shelf space utilization rate and product exposure. Finally, a closed-loop optimization mechanism with self-learning ability is constructed, significantly improving the product display effect and customer shopping experience, and providing data-driven intelligent display decision support for the retail scenario.

[0093] To solve the problem of the lack of an objective and quantitative evaluation standard for product display effects in the retail scenario, an intelligent display evaluation system based on multi-source data fusion is developed. By performing spatio-temporal correlation matching on the customer interaction behavior data set and the three-dimensional space distribution data, the dynamic and parametric analysis of the display effect is achieved. It provides data-driven decision support for retail display optimization and effectively solves the problem that traditional display evaluation relies on subjective experience.

[0094] In some embodiments, in step 103, the spatio-temporal correlation matching of the interaction behavior data set and the three-dimensional space distribution data is performed, and according to the matching result and the change parameters corresponding to the product displacement trajectory in the dynamic feature set, a display effect evaluation parameter group including a space occupancy density index, a visual focus distribution rate, and a contact behavior delay value is generated, including:

[0095] 201. Perform spatio-temporal correspondence association on the time stamps of the behavior events when customers contact products in the interaction behavior data set and the product display position coordinates in the three-dimensional space distribution data, so as to establish a correspondence table between the customer positions with time stamps and the product areas as the matching result;

[0096] In step 201, the interactive behavior dataset refers to the set of records of the customer's interactive behaviors with the products during the shopping process, including actions such as touching, picking up, and putting down, as well as the time information when these actions occur. The three-dimensional spatial distribution data is the coordinate data of the product display positions collected through spatial modeling or sensors, which is used to accurately describe the physical distribution of the products on the shelves or in the exhibition areas. The time-space correspondence association is to match the time stamps of the customer's behaviors with the spatial coordinates of the products, so as to establish the association relationship between the position where the customer is located at a specific time and the product area. The correspondence table is the finally generated tabular data, which contains fields such as the customer behavior time, the product area coordinates, and the behavior type, and is used for subsequent analysis of the customer's shopping path and product contact situation.

[0097] In the embodiment of the present application, first, the time stamps of the customer contact product behavior events in the interactive behavior dataset (such as customer A contacting product B at time t1) are aligned with the product display position coordinates in the three-dimensional spatial distribution data of step 101 (such as the coordinates of product B in shelf area X (1.2m, 0.5m, 1.8m)) through the time stamp synchronization technology; secondly, a key-value pair mapping is established using the hash table data structure (Key = time stamp + area ID, Value = customer position and product coordinates); then, the customer positions and product area coordinates within the same time period are aggregated through a spatial clustering algorithm (such as DBSCAN) to generate a time-stamped correspondence table (such as {"t1":{"area X":{"customer position":(1.1m, 0.6m),"product coordinates":[(1.2m, 0.5m, 1.8m)]}}}); finally, the matching result table is output to provide a spatio-temporal association basis for subsequent parameter calculations.

[0098] 202. According to the total number of times the product is moved and the length of the customer stay period corresponding to each movement in the matching result, divide the product of the total number of times and the average value of the stay period lengths by the spatial occupancy volume of the corresponding shelf partition to generate a spatial occupancy density index characterizing the product aggregation degree;

[0099] In step 202, according to the definition of the shelf partition in the three-dimensional spatial distribution data, the product positions in the matching result are mapped to the shelf partitions. The total number of times the product is moved refers to the cumulative number of times a certain product is picked up or moved by the customer as counted in the correspondence table. The length of the customer stay period refers to the duration from the start of the interaction to the end when the customer contacts the product, which reflects the degree of attention of the customer to the product. The spatial occupancy density index is an index calculated by dividing the product of the total number of movements and the average stay duration by the volume of the partition where the product is located, and is used to measure the attraction intensity of the product in the unit space. The higher the value, the more likely the product is to aggregate customer behaviors during the display.

[0100] In the embodiments of the present application, first, based on the matching result table of step 201, the total number of times a product is moved (for example, product B in area X is moved 5 times) and the length of the customer's stay period corresponding to each movement (for example, the stay durations are 10s, 15s, 20s respectively, and the average duration is 15s) are counted; second, the space occupancy volume of the corresponding shelf partition is extracted (for example, the volume of area X is 2m 3 ); then, calculate according to the formula space occupancy density index = total number of times × average stay duration / space volume (for example, 5 × 15 / 2 = 37.5 times·second / m 3 ); finally, integrate all the space occupancy density indices and output a density index list for each partition, providing an input for step 205.

[0101] 203. Extract a continuous image group with continuous changes in the customer's head direction from the human motion forms in the dynamic feature set, and combine the customer's stay period length in the matching result to generate a visual focus distribution rate representing the visual attention intensity;

[0102] In step 203, the human motion form refers to the customer's posture features extracted from the surveillance video through computer vision technology, such as head angle, body orientation, etc. The continuous image group refers to an image sequence of the customer's head direction change arranged in chronological order, which is used to analyze the customer's line of sight trajectory. The visual focus distribution rate is an index calculated based on the customer's line of sight stay duration and the commodity area coverage, reflecting the visual attention intensity of the customer to a specific commodity. The higher the value, the more likely the commodity is to attract the customer's attention in the display.

[0103] In the embodiments of the present application, first, extract a continuous image group of the customer's head direction change from the dynamic feature set of step 102 (for example, the head deflection angle of customer A increases from 0° to 30° within 10 seconds); second, calculate the proportion of the duration of the continuous head direction in the total stay duration through the OpenCV direction tracking algorithm (for example, the duration of the head facing product B is 8 seconds / total stay duration of 10 seconds = 80%); then, combine the customer's stay period length of step 201 (for example, 10 seconds) to generate a visual focus distribution rate (for example, 80%); finally, output a focus distribution rate list for each commodity (for example, {"product B": "focus distribution rate 80%"}) to provide a correlation parameter for step 205.

[0104] 204. Calculate the median value of the first contact time difference of all customers in the same partition according to the time difference between the time mark of the first behavior event when the customer enters the shelf partition and the start time of the product displacement trajectory movement in the matching result, so as to generate a contact behavior delay value reflecting the product reachability efficiency;

[0105] In step 204, the first behavior event time mark refers to the time point when a customer first contacts a product after entering a certain shelf partition. The start time of the product displacement trajectory movement refers to the starting moment when the product is picked up or moved by the customer. The median value of the first contact time difference is the median of the time differences of all customers' first contacts with the product within the same partition, and is used to measure the delay of customers from entering the partition to actually contacting the product. The contact behavior delay value is a standardized expression of this median time difference, and is used to evaluate the accessibility of the product display. The lower the value, the easier it is for the product to be discovered and interacted with by customers.

[0106] In the embodiment of the present application, first, extract the first behavior event time mark when the customer enters the shelf partition (such as the time t1 = 10:00:00 when customer A enters area X) and the start time of the product displacement trajectory movement (such as the displacement start time t2 = 10:00:02 of product B) from the matching result table in step 201; secondly, calculate the first contact time difference (such as t2 - t1 = 2 seconds), and count the delay values of all customers in the same partition (such as the delay value list [2s, 3s, 1s] in area X); then, generate the delay value through the median calculation algorithm (such as the median value 2s after sorting); finally, output the contact behavior delay values of each partition (such as {"area X": "delay value 2s"}), providing an efficiency evaluation parameter for step 205.

[0107] 205. Combine and associate the space occupancy density index, the visual focus distribution rate, and the contact behavior delay value according to three-dimensional space coordinates to form an evaluation parameter group for the display effect.

[0108] In step 205, the evaluation parameter group for the display effect is a comprehensive evaluation system composed of three indicators: the space occupancy density index, the visual focus distribution rate, and the contact behavior delay value. Each indicator respectively reflects the performance of the product in terms of behavior aggregation, visual attraction, and accessibility. By associating these indicators with the three-dimensional space coordinates of the product, a multi-dimensional evaluation of the shelf display effect can be formed, providing data support for optimizing the product placement.

[0109] In the embodiment of the present application, first, based on the space occupancy density index in step 202 (such as the index of area X is 37.5), the visual focus distribution rate in step 203 (such as the distribution rate of product B is 80%), and the contact behavior delay value in step 204 (such as the delay in area X is 2s), perform data association according to the three-dimensional space coordinates (such as the coordinate range of area X is x[1.0m, 1.5m]); secondly, generate the parameter group through the JSON structured encapsulation technology (such as {"area X": {"space density index": 37.5, "focus distribution rate": 80%, "contact delay": 2s}}); finally, output the evaluation parameter group for the display effect in the three-dimensional space dimension, providing a quantitative basis for the display optimization strategy.

[0110] The following is a specific example:

[0111] In the actual application of the supermarket intelligent shopping cart system, the system first collects the behavior data of customer A picking up a bottled shampoo on shelf H1-3 (coordinates X = 2.1m, Y = 1.5m) at 10:03:21 through in-vehicle cameras and shelf RFID sensors, associates the timestamp with the product coordinates to generate a corresponding relationship table, and simultaneously records the touch behavior of customer B on the same product at 10:05:44; subsequently, based on the data that the products in the H1-3 area were moved 28 times, the average customer stay was 12 seconds, and the partition volume was 0.8m 3 ³, the space occupancy density index is calculated to be 420, reflecting the concentration of product attraction; then, by analyzing the image group of customer A continuously looking at the shampoo in the head direction for 7 seconds and combining the stay duration, the visual focus distribution rate of 58.3% is obtained, indicating that the packaging design effectively attracts attention; again, according to the time difference between the first contact times of customer A and B with the product (3.2 seconds and 5.8 seconds), the median contact behavior delay value of 4.5 seconds is calculated, showing that the product display position is easily accessible; finally, the space occupancy density index (420), the visual focus distribution rate (58.3%), and the contact behavior delay value (4.5 seconds) are associated according to three-dimensional coordinates to form a complete set of shelf display effect evaluation parameters, providing data support for optimizing product placement.

[0112] In summary, through steps 201 to 205, the quantitative evaluation and precise optimization of the product display effect are realized. By establishing a spatio-temporal correlation model between customer behavior and product location, the system can intelligently identify the differences in product attraction and customer decision-making characteristics, and generate a three-dimensional evaluation system including spatial aggregation, visual attention, and contact convenience. This technical solution breaks through the experience-dependent mode of traditional display optimization, realizes the intelligent transformation from actual customer behavior data to scientific display decisions, provides a quantifiable and verifiable display optimization solution for the retail scenario, and significantly improves the product display effect and customer shopping experience.

[0113] To solve the problem of insufficient matching accuracy of customer behavior and spatial location data in product display analysis, an intelligent display analysis system based on spatio-temporal synchronization is developed, which realizes the precise correlation and matching of customer contact behavior and product displacement status, and provides a high-precision spatio-temporal data basis for display effect evaluation. In some embodiments, the time-space correspondence association of the behavior event time markers of customers contacting products in the interaction behavior data set with the product display position coordinates in the three-dimensional space distribution data in step 201, and establishing a corresponding relationship table between the customer position with time markers and the product area as the matching result includes:

[0114] 301. Extract the time stamps of the customer's behavior events of contacting the commodity from the interaction behavior dataset, and at the same time obtain the commodity display position coordinates of the customer from the three-dimensional space distribution data, and unify the time stamps of the behavior events and the commodity display position coordinates to the same time axis based on the system time axis;

[0115] In step 301, the interaction behavior dataset refers to a time series data set recording the interaction behaviors of customers with commodities during the shopping process, including specific actions such as touching, picking up, and putting down and the accurate time stamps of their occurrences. The three-dimensional space distribution data is the commodity display position coordinate information collected through spatial modeling technology or sensor devices, which is used to accurately describe the physical distribution of commodities in the shelf space. The system time axis is a time coordinate system used as a unified benchmark to synchronize the time stamps of customer behavior events with the commodity display position coordinates to ensure the consistency of the time dimension in subsequent analyses.

[0116] In the embodiment of the present application, first, extract the time stamps of the customer's behavior events of contacting the commodity (such as customer A contacting commodity B at t1 = 10:00:00) from the interaction behavior dataset in step 101, and at the same time obtain the commodity display position coordinates at the corresponding moment (such as the coordinates of commodity B (x, y, z) = (1.2m, 0.5m, 1.8m)) from the three-dimensional space distribution data in step 101; second, unify the timestamps of the two types of data to the same benchmark (such as the calibrated time error ≤ 1ms) through the system time axis alignment technology (such as NTP protocol synchronization); then, use the time window sliding algorithm to aggregate the behavior events and coordinate data according to the timestamps to generate a raw record table with spatio-temporal alignment (such as {"t1": {"customer A": "commodity B", "coordinates": (1.2, 0.5, 1.8)}}), which provides input for step 302.

[0117] 302. Convert the commodity display position coordinates according to the coordinate system of the three-dimensional space distribution data to determine the shelf partition identifier where the customer is located at each time stamp, and the shelf partition identifier is generated by combining the layer number, column number, and depth distance of the commodity display position;

[0118] In step 302, the commodity display position coordinates refer to the specific position data of the commodity in three-dimensional space, including the coordinate values of the X-axis (column number), Y-axis (layer number), and Z-axis (depth distance) in three dimensions. The coordinate system conversion refers to the process of standardizing these original coordinate data according to the physical layout of the actual shelf. The shelf partition identifier is a unique position code generated by combining the three dimensions of the layer number, column number, and depth distance, which is used to accurately locate the shelf area where the customer or commodity is located. For example, the coding format of "A-3-2" is used to represent the area 2 meters deep in the 3rd layer of column A.

[0119] In the embodiments of the present application, first, according to the predefined rules of the three-dimensional space distribution data (for example, the number of layers is divided by the Z-axis height: 0 ≤ z < 1m is the first layer, 1 ≤ z < 2m is the second layer; the number of columns is one column every 0.5 meters along the X-axis; the depth is one area every 0.3 meters along the Y-axis), the commodity coordinate system is converted into a structured shelf partition identifier. For example, the coordinate (1.2m, 0.5m, 1.8m) of commodity B is mapped to the layer 2 (L2), column 3 (C3), and depth area 2 (Y2), generating the identifier "L2-C3-Y2". Secondly, all commodity coordinates are batch processed through the space grid mapping algorithm to generate a list of shelf partition identifiers. Finally, the identifier is associated with the time stamp to form a spatio-temporal data set with partition information, providing a standardized position label for the subsequent construction of the record table.

[0120] 303. Generate an initial record table containing customer identifiers, shelf partition identifiers, and commodity displacement status based on the shelf partition identifier;

[0121] In step 303, the shelf partition identifier is a standardized position code generated after coordinate conversion processing. The customer identifier refers to a digital or character code used to uniquely identify the customer's identity, usually automatically generated by the monitoring system or sensor data. The commodity displacement status refers to the dynamic information recording whether the commodity has moved, including states such as picked up, moving, and put back. The initial record table is a structured data storage table, containing core fields such as customer identifiers, shelf partition identifiers, and commodity displacement status, and is used to initially record the basic information of the interaction between the customer and the commodity.

[0122] In the embodiments of the present application, first, based on the data after spatio-temporal alignment, customer identifiers (such as customer A), shelf partition identifiers (such as L2-C3-Y2), and commodity displacement status (such as "stationary" or "moving") are extracted. Secondly, an initial record table is constructed through the key-value pair storage technology, and each record contains a time stamp, customer identification, partition identification, and commodity status. For example, when customer A is located in partition L2-C3-Y2 at 10:00:00 and commodity B is in a stationary state, the record is: time 10:00:00, customer A, partition L2-C3-Y2, commodity status: stationary. Finally, the initial record table is stored in the time series database to support efficient time range retrieval and dynamic update.

[0123] 304. When the start time of the movement of the commodity display position coordinate is monitored, retrieve the time mark of the customer's behavior event of contacting the commodity on the same time axis in the initial record table, and synchronously append the relationship between the duration of the commodity movement process and the change of the commodity shelf partition identifier in the time mark of the customer's behavior event of contacting the commodity to the initial record table to form a corresponding relationship table of the customer's position and the commodity area with time marks as the matching result.

[0124] In step 304, the start time of the movement of the product display position coordinates refers to the starting moment when the product is picked up or moved by the customer detected by the sensor. Time axis synchronization refers to the process of retrieving customer behavior events that match the product movement time in the initial record table. The duration of the product movement process refers to the complete time period experienced from the moment the product is picked up until it is finally put back or moved to another position. The change relationship of the shelf area identifier refers to the change of the shelf area position involved in the product movement process. The correspondence table is the final generated complete matching result data table, which contains complete information fields such as customer identifier, precise time mark, shelf area identifier, product displacement status, and movement process duration.

[0125] In the embodiment of the present application, when the start time of the movement of the product displacement trajectory is detected (for example, product B starts to move at 10:00:02), first retrieve the customer behavior event under the same time axis from the initial record table (for example, customer A touches product B at 10:00:02). Secondly, calculate the duration of the product movement process (for example, from 10:00:02 to 10:00:05, the duration is 3 seconds), and track the change of the partition identifier caused by the product displacement (such as moving from L2-C3-Y2 to L1-C2-Y1). Then, insert the movement duration and partition change relationship into the initial record table through the data append algorithm to generate a matching result table with complete spatio-temporal association. For example, the updated record is: time 10:00:02, customer A, partition L2-C3-Y2 → L1-C2-Y1, product status: moving, movement duration: 3 seconds. Finally, output a matching result table containing spatio-temporal behavior association to provide accurate data support for display effect analysis.

[0126] The following is a specific example:

[0127] In the actual application of the supermarket intelligent analysis system, the system first aligns the behavior event of customer C picking up the boxed chocolate on shelf G2-5 (coordinates X = 3.4m, Y = 1.2m, Z = 1.5m) at 14:22:36 with the three-dimensional space data through a unified time axis, and confirms that this behavior occurs on the transition path from the fresh food area to the snack area; then converts the original coordinates into the standard shelf area identifier G2-5-3N (the 3rd layer / the 5th column / the proximal display area), and establishes an initial record table containing customer ID, time stamp, area identifier, and product status (picked up - brand A); when the system detects that the product coordinates start to move at 14:22:38, it automatically associates the behavior time mark of customer C, and adds the product movement process duration (2 seconds) and the partition change relationship (moved from G2-5-3N to the shopping cart) to the record table. Finally, a complete spatio-temporal correspondence table is formed, in which the contact behavior of customer D in the same partition at 14:23:11 is also synchronously recorded, providing accurate data support for subsequent analysis of product contact timeliness and display position optimization.

[0128] In summary, through steps 301 to 304, the precise spatio-temporal mapping of product display and customer behavior is achieved. By establishing a unified time axis benchmark, the system synchronizes discrete customer contact events with product spatial coordinates at the millisecond level, and constructs a three-dimensional partition identification system including the number of floors, columns, and depth distance. Based on the real-time collected behavior data, the system automatically generates a dynamic association table with spatio-temporal tags, completely recording the state changes of the whole process from the start to the end of product displacement. This technical solution breaks through the limitation of the separation of spatio-temporal data in traditional retail analysis, realizes the digital twin modeling of customer movement routes and product display, and provides a precise spatio-temporal association basis for the subsequent quantitative evaluation of display effects.

[0129] To solve the problem that the dynamic optimization of product display lacks multi-dimensional decision-making basis, an intelligent display optimization system driven by dual factors is developed. By analyzing the ratio relationship between space occupancy density and visual focus, the first optimization factor for adjusting the shelf height is generated, and by combining the association rule between contact delay and product category, the second optimization factor for optimizing the horizontal orientation is generated, realizing an intelligent adjustment scheme for display space based on the vertical-horizontal two dimensions, and significantly improving the scientificity and accuracy of display optimization decisions.

[0130] In some embodiments, determining the first optimization factor according to the ratio relationship between the space occupancy density index and the visual focus distribution rate in the display effect evaluation parameter group in step 104, and at the same time determining the second optimization factor according to the association rule between the contact behavior delay value and the preset product category, includes:

[0131] 401. Calculate the real-time ratio of the space occupancy density index to the visual focus distribution rate. When the real-time ratio is greater than the preset high ratio range, convert the real-time ratio into a positive adjustment coefficient in the shelf height direction. When the real-time ratio is less than the preset low ratio range, convert it into a negative adjustment coefficient in the shelf height direction;

[0132] In step 401, the space occupancy density index is a comprehensive index reflecting the frequency of customer contact and the staying duration of goods in a unit space. The higher the value of this index, the stronger the attraction of the goods. The visual focus distribution rate is a parameter measuring the distribution of customer visual attention on goods, reflecting the degree of attraction of goods to customers' eyes. The real-time ratio refers to the dynamic proportional relationship between the space occupancy density index and the visual focus distribution rate, and is used to determine whether the actual attraction of goods matches the visual attraction. The high ratio range is a preset upper threshold. When the real-time ratio exceeds this range, it indicates that the actual contact frequency of the goods is higher than the visual attraction. The low ratio range is a preset lower threshold. When the real-time ratio is lower than this range, it indicates that the visual attraction of the goods is higher than the actual contact frequency. The positive adjustment coefficient is a shelf height upward adjustment parameter generated according to the high ratio situation, and is used to increase the display height of goods. The reverse adjustment coefficient is a shelf height downward adjustment parameter generated according to the low ratio situation, and is used to reduce the display height of goods.

[0133] In the embodiment of the present application, first, according to the space occupancy density index (such as the shelf area X index of 37.5 times·second / m 3 ) generated in step 205 and the visual focus distribution rate (such as 80%), calculate the real-time ratio (formula: real-time ratio = space occupancy density index / visual focus distribution rate, example: 37.5 / 80 ≈ 0.469). Second, preset the high ratio range (such as ≥0.6) and the low ratio range (such as ≤0.3). If the real-time ratio > 0.6, convert it into a positive adjustment coefficient (such as coefficient = real-time ratio - 0.6 = 0.469 - 0.6 = -0.131, since it does not exceed the threshold, the positive adjustment is not triggered); if the real-time ratio < 0.3, convert it into a reverse adjustment coefficient (such as coefficient = 0.3 - real-time ratio = 0.3 - 0.469 = -0.169, since it does not fall below the threshold, the reverse adjustment is not triggered). Finally, dynamically adjust the coefficient according to the threshold determination result to provide parameter input for step 402.

[0134] 402. Generate a first optimization factor according to the product relationship between the cumulative number of times of the positive adjustment coefficient or the reverse adjustment coefficient within a continuous time period and the number of layers in the shelf height direction;

[0135] In step 402, the positive adjustment coefficient and the reverse adjustment coefficient are parameters reflecting the adjustment direction and amplitude required for the display height of goods. The continuous time period refers to a specific time period for data collection and analysis. The cumulative number of times refers to the total number of coefficients that need to be adjusted during this time period. The number of layers in the shelf height direction refers to the total number of layers of the shelf in the vertical direction. The first optimization factor is a parameter calculated through the product relationship between the cumulative number of times and the number of layers, and this factor is used to quantify the overall degree of adjustment required for goods in the shelf height direction.

[0136] In the embodiment of the present application, first, count the cumulative trigger times (e.g., 2 forward adjustment triggers and 1 reverse adjustment trigger) of the forward adjustment coefficient or the reverse adjustment coefficient in step 401 within a continuous time period (such as the past 1 hour); secondly, obtain the number of layers in the height direction of the shelf (e.g., the number of layers in area X is 3); then, calculate according to the formula the first optimization factor = cumulative times × number of layers (e.g., forward factor = 2 × 3 = 6, reverse factor = 1 × 3 = 3); finally, output the first optimization factor list (e.g., {"area X": {"forward factor": 6, "reverse factor": 3}}) to provide a quantitative basis for adjusting the shelf height density.

[0137] 403. Compare the contact behavior delay value with the associated rule standard delay range corresponding to the preset commodity category, and generate an azimuth matching difference for the same type of commodity according to the degree to which the contact behavior delay value exceeds or is lower than the standard delay range.

[0138] In step 403, the contact behavior delay value is an index reflecting the time required for a customer to actually contact a commodity from entering the shelf partition. The preset commodity category refers to a classification system pre-divided according to commodity characteristics. The associated rule standard delay range is a reasonable contact time range preset for each type of commodity. The azimuth matching difference is a parameter calculated according to the deviation degree between the actual contact behavior delay value and the standard range, and is used to measure the difference degree between the current display position and the ideal position of the commodity. This difference includes two cases: a positive difference exceeding the standard range and a negative difference lower than the standard range.

[0139] In the embodiment of the present application, first, based on the contact behavior delay value in step 204 (e.g., 2 seconds delay in area X) and the standard delay range of the preset commodity category (e.g., food category standard delay [1.5s, 2.5s]), determine whether the delay value exceeds or is lower than the standard range (e.g., 2 seconds is within the range, no difference is triggered); if the delay value (e.g., 2.6 seconds), calculate the positive azimuth matching difference (difference = actual delay - upper limit value = 2.6 - 2.5 = 0.1 second); if it is lower than the lower limit (e.g., 1.3 seconds), calculate the negative azimuth matching difference (difference = lower limit value - actual delay = 1.5 - 1.3 = 0.2 second). Finally, output the difference list (e.g., {"food category": "difference + 0.1s"}) to provide a horizontal display adjustment parameter for step 404.

[0140] 404. Generate a second optimization factor based on the proportional relationship between the azimuth matching difference and the historical contact position distribution density of the preset commodity category on the horizontal display surface.

[0141] In step 404, the horizontal display surface refers to the display area of the shelf in the horizontal direction. The historical contact position distribution density refers to the frequency distribution of the contact of the same type of goods by customers at various positions on the horizontal display surface. The second optimization factor is a parameter calculated based on the proportional relationship between the azimuth matching difference and the historical contact position distribution density. This factor is used to guide the direction and amplitude of the position adjustment of goods on the horizontal display surface. This factor comprehensively considers the matching degree between the current display effect and the historical contact habits, providing a quantitative basis for the optimization of the horizontal position of goods.

[0142] In the embodiment of the present application, first, based on the azimuth matching difference in step 403 (such as +0.1 seconds), the historical contact position distribution density of the same type of goods on the horizontal display surface is extracted (such as the density of food products at the front of area X is 5 times / m 2 ); secondly, calculate according to the formula second optimization factor = azimuth matching difference / historical density (such as factor = 0.1 / 5 = 0.02); then, if the difference is positive, the factor indicates that the display density needs to be increased (such as factor 0.02 corresponding to a density increase of 0.1 times / m 2 ); if it is negative, the density needs to be reduced. Finally, output the second optimization factor (such as {"food products": "factor +0.02"}), driving the position adjustment of goods on the horizontal display surface.

[0143] The following is a specific example:

[0144] During the real-time operation of the supermarket intelligent display optimization system, the system first calculates the real-time ratio of the space occupancy density index to the visual focus distribution rate of the current shelf partition. When this ratio exceeds the preset high threshold (such as >1.5), a positive shelf height adjustment coefficient (+0.3 / layer) is automatically generated. When it is lower than the low threshold (such as <0.8), it is converted into a negative adjustment coefficient (-0.2 / layer), and based on the situation that there are 5 positive adjustments within 3 consecutive hours and the shelf has 4 layers, the first optimization factor = 5×4 = 20 is generated; at the same time, the system compares the contact behavior delay value of 4.5 seconds in the shampoo partition with the standard delay range of the daily chemical category (3 - 6 seconds). Since it is within the normal range, no azimuth matching difference is generated, but for the adjacent hair conditioner partition with a delay value of 7.2 seconds (exceeding the standard by 1.2 seconds), an azimuth matching difference = 1.2 seconds is generated. Combining the historical horizontal distribution density of this category of 12 times / ㎡, the second optimization factor = 1.2×12 = 14.4 is calculated. Finally, through the dual-factor collaborative mechanism, the adaptive adjustment of the shelf layer height and the horizontal display surface is driven.

[0145] In summary, the dynamic adaptive adjustment of product display parameters is achieved through steps 401 to 404. The system intelligently generates an optimization coefficient in the height direction of the shelf by real-time monitoring the change in the ratio of space density to visual focus. When the product aggregation degree is too high, an upward movement instruction is automatically triggered, and when it is too low, a downward movement strategy is initiated. At the same time, based on the matching analysis of product contact delay data and category characteristics, the system can automatically calculate the optimal display orientation and generate a personalized horizontal display plan for each category of products. This technical solution innovatively constructs a two-dimensional optimization mechanism of "vertical density adaptation and horizontal orientation personalization", realizing the intelligent upgrade from static display to dynamic optimization, and keeping the product display in the best state all the time.

[0146] To solve the problem that there is a lack of quantitative standards for the implementation of the display optimization plan, an intelligent display system for space parameterization execution is developed. By converting the first optimization factor into the spacing adjustment amount of each layer of the shelf, vertical density optimization is achieved. At the same time, the second optimization factor is analyzed into the moving direction and distance of the horizontal grid, and a two-dimensional execution model of "height spacing and horizontal displacement" is established, realizing the accurate conversion from the optimization parameter to the space adjustment. In some embodiments, in step 105, adjusting the distribution density of products in the height direction of the shelf based on the first optimization factor and adjusting the display orientation of products on the horizontal display surface based on the second optimization factor includes:

[0147] 501. Divide the first optimization factor evenly according to the number of layers in the height direction of the shelf to obtain the corresponding unit adjustment amount for each layer;

[0148] In step 501, the first optimization factor is a comprehensive parameter reflecting the degree of adjustment required by products in the height direction of the shelf. This factor is generated by the product relationship between the cumulative number calculated in the early stage and the number of layers. The number of layers in the height direction of the shelf refers to the total number of levels into which the shelf is divided in the vertical dimension, and each layer represents a specific height interval. The unit adjustment amount is the value obtained by evenly distributing the first optimization factor to each level, indicating the quantitative index that each specific level needs to be adjusted. This value contains information on the adjustment direction and amplitude. The equal division process ensures that the adjustment amount is reasonably distributed in the height direction of the shelf, avoiding the situation of excessive or insufficient local adjustment.

[0149] In the embodiment of the present application, first, extract the first optimization factor generated in step 402 (such as the positive factor 6 and negative factor 3 in region X), and divide it evenly according to the number of layers in the height direction of the shelf (such as 3 layers). The calculation formula is: unit adjustment amount = first optimization factor / number of layers (such as positive unit adjustment amount = 6 / 3 = 2, negative unit adjustment amount = 3 / 3 = 1). Secondly, store the results according to the layer numbers (such as the positive adjustment amount of the first layer +2, the negative adjustment amount of the second layer -1), generate the unit adjustment amount table for each layer, and provide the layer density adjustment parameters for step 502.

[0150] 502. Determine the increasing or decreasing direction of the product spacing of the corresponding layer according to the positive or negative sign of the unit adjustment amount, and update the distribution density of the products on the corresponding layer in the vertical direction of the shelf according to the product of the absolute value of the unit adjustment amount and the preset spacing base value;

[0151] In step 502, the positive or negative sign of the unit adjustment amount indicates the direction in which the product spacing needs to be adjusted. A positive value corresponds to the increasing direction of the spacing, and a negative value corresponds to the decreasing direction of the spacing. The preset spacing base value is a preset reference value for the standard product spacing and serves as a basic parameter for adjustment calculation. The distribution density of the products in the vertical direction of the shelf is a value updated by multiplying the absolute value of the unit adjustment amount by the preset spacing base value. This density value directly determines the display density of the products at a specific level. The updated distribution density will guide the rearrangement of the products in the vertical direction to optimize space utilization.

[0152] In the embodiment of the present application, first, determine the product spacing adjustment direction according to the positive or negative sign of the unit adjustment amount in step 501 (for example, +2 means reducing the spacing to increase the density, and -1 means increasing the spacing to reduce the density); second, extract the preset spacing base value (for example, the basic spacing of the first layer is 0.5 meters), and calculate according to the formula new spacing = basic spacing × (1 - absolute value of the unit adjustment amount × 0.1) (for example, when the adjustment amount is +2, the new spacing = 0.5 × (1 - 2 × 0.1) = 0.4 meters, and the density increases by 25%); finally, update the distribution density of the products on each layer (for example, the density of the first layer increases from 5 pieces / m 2 to 6.25 pieces / m 2 ), and output the updated density configuration table.

[0153] 503. Determine the movement direction identifier of the horizontal display surface according to the positive or negative sign of the second optimization factor, where the positive identifier indicates moving towards the area with a high distribution density at the historical contact position, and the negative identifier indicates moving towards the area with a low density;

[0154] In step 503, the second optimization factor is a comprehensive parameter reflecting the degree of adjustment required for the products on the horizontal display surface. This factor is generated from the proportional relationship between the azimuth matching difference and the distribution density at the historical contact position. The movement direction identifier of the horizontal display surface is an azimuth indication parameter determined according to the positive or negative sign of the second optimization factor. The positive identifier points to the advantageous area with a high historical contact frequency, and the negative identifier points to the non-advantageous area with a low contact frequency. The area with a high distribution density at the historical contact position represents the hot spot position where customers are used to contacting, and the area with a low density represents the relatively unpopular position. This identifier ensures that the product movement direction conforms to the customer behavior habit.

[0155] In the embodiment of the present application, first, based on the positive and negative signs of the second optimization factor in step 404 (such as +0.02), determine the movement direction identifier of the horizontal display surface: positive identifier (+): move towards the area with a high distribution density of historical contact positions (such as the density at the front end of food products is 5 times / m 2 → move towards this area); negative identifier (-): move towards the area with a low density (such as the density at the rear end of daily necessities is 2 times / m 2 → move towards this area). Secondly, bind the identifier to the commodity category (such as the identifier for food products is "+1") to provide a basis for the movement direction in step 504.

[0156] 504. Perform a proportional conversion of the absolute value of the second optimization factor and the side length of the area division grid of the horizontal display surface to obtain the horizontal movement distance, and adjust the display orientation of the commodity in the corresponding grid according to the movement direction identifier and the horizontal movement distance.

[0157] In step 504, the area division grid of the horizontal display surface is a coordinate system obtained by dividing the display surface according to certain specifications. The side length of the grid is the reference dimension parameter of a single grid unit. The horizontal movement distance is a specific value obtained by the proportional conversion of the absolute value of the second optimization factor and the side length of the grid, which determines the spatial span that the commodity needs to move. The adjustment of the display orientation is a specific value calculated according to the direction indicated by the movement direction identifier and the horizontal movement distance, which relocates the display position of the commodity in the grid, and finally realizes the optimal display layout of the commodity in the horizontal dimension.

[0158] In the embodiment of the present application, first, according to the movement direction identifier in step 503 (such as +1) and the absolute value of the second optimization factor in step 404 (such as 0.02), calculate according to the formula horizontal movement distance = absolute value of the factor × side length of the grid (such as the side length of the grid is 1 meter → movement distance = 0.02 × 1 = 0.02 meters); secondly, move the display orientation of the commodity according to the direction identifier (such as move forward by +0.02 meters to the high-density area at the front end); finally, execute position update through the automated shelf adjustment device and verify the density after movement (such as the density at the front end of food products increases from 5 times / m 2 to 5.1 times / m 2 ), and complete the closed loop of horizontal display optimization.

[0159] The following is a specific example:

[0160] During the implementation of the dynamic adjustment system of supermarket smart shelves, the system first divides the first optimization factor 20 equally among the four shelves to obtain a unit adjustment amount of +5 (positive) for each layer. According to the preset base value of 0.5 cm, it is calculated that the distance between commodities on each layer should be reduced by 2.5 cm (5×0.5), so that the distribution density of shampoo in the third layer of the golden line of sight of 1.2 m-1.7 m is increased. At the same time, the second optimization factor +14.4 of the conditioner partition is analyzed to generate an identifier for moving to the historical high contact density area on the east side (18 interactions per square meter). According to the 0.3 m grid side length, a movement of 4.32 m (14.4×0.3) is required. The system automatically adjusts the category from the original 3B grid to the 5D grid, and updates the electronic price tag position in a linked manner, ultimately achieving vertical density optimization and efficient matching of the horizontal position, thereby improving the contact efficiency of human traffic at the conditioner display stand.

[0161] In summary, through steps 501 to 504, refined dynamic control of commodity display parameters is achieved. The system decomposes the optimization factors into executable instructions, intelligently adjusts the distribution density of commodities on each layer in the vertical dimension, and automatically increases or decreases the interlayer distance according to the real-time data analysis results; in the horizontal dimension, based on the commodity contact heat map, it accurately calculates the optimal display position and moving distance to achieve intelligent displacement of commodities on the display surface. This technical solution innovatively constructs a dual-dimensional execution mechanism of "vertical layered density adjustment and horizontal grid displacement", and realizes seamless connection from data analysis to physical display by converting abstract optimization parameters into specific space adjustment instructions, ensuring that commodities are always displayed in the best position.

[0162] In order to solve the quantitative association problem between contact delay analysis and product display orientation optimization, an intelligent optimization system based on dynamic division of delay intervals has been developed. The evaluation intervals are divided equally by presetting the allowable delay range of the product category, and the directional difference is calculated according to the interval positioning of the actual delay value, and converted into a standardized orientation matching difference, thereby realizing the precise quantitative conversion from contact behavior efficiency to display orientation adjustment. In some embodiments, in step 403, the contact behavior delay value is compared with the standard delay range of the association rule corresponding to the preset product category, and the orientation matching difference of the same type of products is generated according to the degree to which the contact behavior delay value exceeds or falls below the standard delay range, including:

[0163] 601. Extract the maximum allowable delay value and the minimum allowable delay value defined in the preset commodity category, and divide the difference interval between the maximum allowable delay value and the minimum allowable delay value into at least three equal intervals;

[0164] In step 601, the preset commodity category refers to a classification system pre-divided according to commodity characteristics and sales strategies. The allowable maximum delay value is the upper limit threshold of the acceptable contact time set for each category of commodities. Exceeding this value indicates poor accessibility of the commodity. The allowable minimum delay value is the lower limit threshold of the acceptable contact time set for each category of commodities. Being lower than this value indicates that the accessibility of the commodity is too high, which may affect the display of other commodities. The difference interval is the time range interval obtained by subtracting the allowable minimum delay value from the allowable maximum delay value. This interval is evenly divided into at least three sub-intervals, and each sub-interval represents a different level of contact behavior delay performance.

[0165] In the embodiment of the present application, first, the allowable maximum delay value and the allowable minimum delay value defined in the preset commodity category are extracted, and the difference interval between the allowable maximum delay value and the allowable minimum delay value is divided into at least three equal sub-intervals. This process provides a basis for accurately calculating the position of the contact behavior delay value in the subsequent steps by carefully dividing the delay value interval. Then, the width of each equal sub-interval is determined to ensure that the subsequent calculations can accurately reflect the position relationship of the contact behavior delay value in different intervals. Finally, through the careful division of the delay value interval, more accurate data support is provided for optimizing the display orientation of commodities.

[0166] 602. Mark the position of the contact behavior delay value according to the difference interval. If the contact behavior delay value is in the right interval of the allowable maximum delay value, calculate the excess amount exceeding the allowable maximum delay value as the positive difference amount. If it is in the left interval of the allowable minimum delay value, calculate the shortage amount less than the allowable minimum delay value as the negative difference amount.

[0167] In step 602, the contact behavior delay value is the time actually measured for a customer to reach the commodity from entering the shelf partition. Position marking refers to the process of classifying and labeling according to the specific position of the contact behavior delay value in the difference interval. The positive difference amount is the numerical value of the excess part calculated when the contact behavior delay value exceeds the allowable maximum delay value, reflecting the degree to which the accessibility of the commodity is lower than the expected standard. The negative difference amount is the numerical value of the shortage part calculated when the contact behavior delay value is lower than the allowable minimum delay value, reflecting the degree to which the accessibility of the commodity is higher than the expected standard. The right interval refers to the range in the difference interval that is greater than the allowable maximum delay value, and the left interval refers to the range in the difference interval that is less than the allowable minimum delay value.

[0168] In the embodiments of the present application, first, the contact behavior delay value is position - marked according to the difference interval, specifically analyzing the position of each contact behavior delay value relative to the allowed maximum and minimum delay values. If the contact behavior delay value is in the right - hand interval of the allowed maximum delay value, it indicates that the time when the customer contacts the commodity is significantly later than the ideal situation. At this time, calculate the specific excess amount exceeding the allowed maximum delay value as the positive difference amount. If the contact behavior delay value is in the left - hand interval of the allowed minimum delay value, it means that the time when the customer contacts the commodity is too early or too rapid. At this time, calculate the specific shortage amount less than the allowed minimum delay value as the negative difference amount. Then, during the calculation process, ensure that each contact behavior delay value can be accurately classified into the corresponding interval, and calculate the corresponding difference amount accordingly. Secondly, through this precise difference amount calculation method, it is possible to more clearly identify which commodities need to be preferentially adjusted in the display orientation to improve the customer experience.

[0169] 603. Calculate the ratio of the positive difference amount or the negative difference amount to the width of the equal - division interval to generate an orientation - matching difference.

[0170] In step 603, the width of the equal - division interval refers to the time - span value of each sub - interval, which is obtained by dividing the total difference interval by the number of divisions. The orientation - matching difference is a standardized parameter generated by calculating the ratio of the positive difference amount or the negative difference amount to the width of the equal - division interval. This parameter quantifies the deviation degree of the actual contact behavior delay from the expected standard. The ratio calculation process converts the absolute time difference into a relative proportion value, facilitating horizontal comparison and unified evaluation among different commodity categories. The generated orientation - matching difference will be used as an important reference parameter for subsequent display optimization and adjustment.

[0171] In the embodiments of the present application, first, extract the allowed maximum delay value (for example, the allowed maximum for food is 2.5 seconds) and the allowed minimum delay value (for example, 1.5 seconds) from the association rules of the preset commodity categories, and calculate the difference interval (2.5 - 1.5 = 1.0 second). Secondly, divide the difference interval into three equal - division intervals (each approximately 0.33 seconds), and the division results are as follows: left - hand interval: 1.5 seconds ≤ delay value < 1.83 seconds; middle interval: 1.83 seconds ≤ delay value ≤ 2.16 seconds; right - hand interval: 2.16 seconds < delay value ≤ 2.5 seconds. Finally, output the range of the equal - division interval, providing a basis for determining the position of the delay value for step 602.

[0172] The following is a specific example:

[0173] In the delay analysis module of the supermarket intelligent shelf optimization system, the system first extracts the preset delay threshold for the shampoo category (the minimum allowable delay value is 3 seconds, and the maximum allowable delay value is 6 seconds), and divides the 3-second difference interval into three equal sub-intervals (3 - 4 seconds, 4 - 5 seconds, 5 - 6 seconds); when the detected actual contact behavior delay value in a certain area is 7.2 seconds, the system determines that it is in the right interval of the maximum allowable value of 6 seconds, calculates a positive difference of 1.2 seconds (7.2 - 6 = 1.2), and then calculates through the ratio with the width of the equal sub-interval of 1 second ((6 - 3) / 3 = 1) to generate an orientation matching difference of +1.2 (1.2 / 1). This value will be used as the quantitative basis for subsequent horizontal display adjustment. When the difference is positive, it triggers an optimization strategy to move towards the high-passenger-flow area on the east side. At the same time, when the system monitors a delay value of 2.8 seconds in another area (to the left of the minimum allowable value), it generates a negative difference of -0.2 (3 - 2.8 = 0.2) and calculates an orientation matching difference of -0.2, automatically starting a defensive displacement adjustment towards the low-density area on the west side. Through this two-way difference mechanism, precise spatial adaptation of product display is achieved.

[0174] In summary, through steps 601 to 603, intelligent hierarchical evaluation and precise regulation of the product display delay parameters are achieved. The system establishes a delay tolerance interval model for product categories, conducts multi-level comparison and analysis of actual contact delay data with preset standards, and precisely quantifies the degree of delay deviation. When it detects that the delay exceeds the maximum tolerance threshold, the system automatically calculates the positive deviation amount and generates a display optimization plan; when the delay is lower than the minimum expected value, it calculates the negative deviation amount and triggers corresponding adjustments. This technical solution innovatively constructs a dynamic evaluation mechanism based on the delay tolerance interval. By converting abstract delay data into operable orientation matching differences, it realizes precise quantification and intelligent response of product display sensitivity, ensuring that all types of products are always in the optimal display state.

[0175] To solve the problem of precise identification of the dynamic interaction between customers and products, we have developed an intelligent video feature extraction system. Through a three-step analysis method of moving object segmentation, displacement trajectory verification, and human motion association, the system matches the product movement path and triggering behavior in space and time, constructs a dynamic feature set containing causal relationships, and realizes the reconstruction of the complete event chain of interaction behavior. In some embodiments, the image recognition process of the continuous video stream collected by the monitoring device in step 102 to extract the dynamic feature set including the product displacement trajectory and human motion form includes:

[0176] 701. Segment moving objects in the difference regions between adjacent frames of the continuous video stream collected by the monitoring device, mark the regions that meet the product size threshold in the segmented moving objects as candidate product regions, and mark the remaining regions as candidate human regions;

[0177] In step 701, the continuous video stream collected by the monitoring device refers to the real-time monitoring video data obtained through a fixed camera, which contains continuous image frames arranged in chronological order. The difference region between adjacent frames is the contour region of the moving object detected by comparing the pixel changes of the two consecutive frames. Moving object segmentation is a processing process that uses computer vision algorithms to separate moving objects from the static background. The commodity size threshold is a pixel range standard corresponding to the preset physical size of the commodity, which is used to screen the moving regions that meet the commodity size. The candidate commodity region is the region of the moving object that is initially identified as a possible commodity after size screening. The human body candidate region is the remaining region of the moving object that is identified as a possible human body except for the candidate commodity region.

[0178] In the embodiment of the present application, first, the background subtraction algorithm (such as the ViBe algorithm) is used to process the difference region between adjacent frames in the continuous video stream collected by the monitoring device to segment the contour of the moving object; secondly, according to the preset commodity size threshold (such as the length and width range [10 cm, 50 cm]), the moving objects that meet the size are marked as candidate commodity regions (such as the moving region of the shelf commodity) through morphological filtering (such as opening operation for denoising); the remaining regions (such as the customer's torso and arms) are marked as human body candidate regions. Finally, the classification result of the candidate region (such as {"frame t1": {"candidate commodity region": [rectangle frame coordinates], "human body candidate region": [keypoint coordinates]}}) is output, providing input for the subsequent steps.

[0179] 702. Perform direction continuity verification on the position coordinates of the candidate commodity region in consecutive frames. If the direction of coordinate change of the same commodity in adjacent frames is consistent with the moving direction trend, record its moving path points and connect them into a commodity displacement trajectory;

[0180] In step 702, the direction continuity verification is a process of logically verifying the movement trajectory of the candidate commodity region in consecutive video frames. The direction of coordinate change refers to the moving azimuth angle of the center point of the commodity region in adjacent frames. The moving direction trend is the main movement path direction deduced by analyzing the position changes of the commodity in multiple consecutive frames. The moving path point is the specific position coordinate data point that records the commodity in each frame of the picture. The commodity displacement trajectory is a complete movement path line formed by connecting each moving path point, reflecting the actual movement process of the commodity in space.

[0181] In the embodiment of the present application, first, the position coordinates of the candidate commodity area in consecutive frames (such as the coordinates (x1, y1) in frame t1 and the coordinates (x2, y2) in frame t2) are subjected to direction continuity verification: Direction trend calculation: Calculate the moving direction angle through the coordinate difference between adjacent frames (such as θ = arctan((y2 - y1) / (x2 - x1))); Continuity determination: If the change in the direction angle within 3 consecutive frames is less than the threshold (such as ±10°), it is determined that the directions are consistent; Secondly, the coordinate points that pass the verification are connected into a commodity displacement trajectory (such as {"Commodity A": "Trajectory points [(x1, y1, t1), (x2, y2, t2)]"}), and jitter noise is filtered. Finally, a trajectory data table is output, providing an event association basis for step 704.

[0182] 703. Lock the positions of the trunk and limb key points in the candidate human area, generate a posture change sequence according to the position change direction of the key points in adjacent frames, and mark the continuous direction combinations that conform to the preset action form in the posture change sequence as human action forms;

[0183] In step 703, the trunk and limb key points are the coordinate points of the main body joint parts identified by the human posture estimation algorithm. Position locking refers to the process of continuously tracking the coordinates of specific key points in consecutive frames. The posture change sequence is a set of key point position change data arranged in chronological order. The preset action form is a predefined standard human action mode library, which contains the direction combination characteristics of typical shopping actions. The human action form is a specific behavior mode identified by matching the posture change sequence with the preset action form.

[0184] In the embodiment of the present application, first, use the OpenPose key point detection algorithm to lock the trunk and limb key points (such as the shoulder, elbow, and wrist joint coordinates) in the candidate human area; Secondly, calculate the displacement direction of the key points in adjacent frames (such as the direction angle θ = 30° of the wrist from (x1, y1) to (x2, y2)), and generate a posture change sequence (such as ["Reaching out: θ = 30°", "Retracting the hand: θ = 210°"]); Then, match the preset action form (such as the standard sequence of the "Fetching" action) through the dynamic time warping algorithm (DTW). If the matching similarity > 90%, it is marked as a valid action form. Finally, output a list of human action forms (such as {"Customer A": "Action form: Fetching"}), providing a behavior event label for step 704.

[0185] 704. Perform event association between the starting position of the commodity displacement trajectory and the corresponding human action form within the same time period, and merge them to form a dynamic feature set including the commodity displacement trajectory and the human action form.

[0186] In step 704, event association is a process of matching the causal relationship between the commodity displacement trajectory and the human body movement that causes the displacement. The starting position refers to the initial coordinate point when the commodity begins to move. The dynamic feature set is a composite data set that integrates the physical trajectory of commodity displacement and the behavioral characteristics of relevant human body movements, and this set completely records the spatio-temporal dynamic information in the process of "human-object" interaction. Merging and forming refers to the processing process of aligning the two types of feature data according to the time stamp and establishing a corresponding relationship.

[0187] In the embodiment of the present application, first, based on the unified time axis, align the starting time of the commodity displacement trajectory in step 702 (such as commodity A starts to move at t1) with the occurrence time of the human body movement form in step 703 (such as customer A starts the "taking object" action at t1); secondly, through the event association engine, merge the trajectory and the action within the same time window (such as from t1 to t1 + 3 seconds) to generate a dynamic feature set (such as {"t1": {"commodity A displacement trajectory": [coordinate sequence], "customer A action": "taking object"}}); finally, output the dynamic feature set with spatio-temporal tags to provide multi-dimensional data support for the analysis of the display effect.

[0188] The following is a specific example:

[0189] In the real-time analysis process of the intelligent retail monitoring system, the system first performs inter-frame difference processing on the continuous video stream collected by the camera, segments the moving objects through the background subtraction algorithm, marks the areas with length and width in the range of 5 - 50 cm as candidate commodities (such as the shampoo bottle on the shelf), and marks the remaining areas as the human body candidate areas; then, track the movement trajectory of the candidate commodity area. When it is detected that a certain bottled beverage keeps moving downward to the right (coordinate change Δx>0 and Δy>0) in 5 consecutive frames (0.2 seconds interval), the system records its moving path points and generates a complete commodity displacement trajectory; at the same time, perform 17-point key point detection on the human body candidate area through the OpenPose algorithm. When the action sequence of "the angle of the right elbow joint decreases → the wrist joint extends forward → the fingers close" appears in 3 consecutive frames, it is marked as a "grasping action"; finally, the system performs spatio-temporal association on the displacement trajectory of the beverage bottle in the H3 area of the shelf at 14:05:23 (starting coordinate X = 2.1m, Y = 1.4m) and the grasping action of customer E, and forms a dynamic feature set including the commodity movement path and the customer's grasping posture. This set shows that customer E completes the entire interaction process from gazing to picking up the goods in 0.8 seconds, providing a basis for fine-grained behavior analysis for subsequent display optimization.

[0190] In summary, through steps 701 to 704, the accurate identification of products and customer behaviors and the extraction of dynamic features are achieved. The system, through intelligent video analysis technology, first precisely segments the moving objects in the monitoring screen to automatically distinguish the product area from the human body area; then constructs the product displacement trajectory based on the analysis of motion continuity, and at the same time captures the customer action features through human pose recognition technology; finally, spatio-temporally correlates the product movement path with the customer behavior actions to form a complete interactive behavior feature map. This technical solution innovatively integrates computer vision and behavior analysis algorithms, realizing the intelligent transformation from the original video stream to structured behavior features, providing high-precision behavior data support for subsequent display optimization decisions.

[0191] Figure 2 FIG. is a schematic structural diagram of a retail data processing system provided by an embodiment of the present application for retail product display evaluation, as Figure 2 shown, the system includes:

[0192] An acquisition module 21, which acquires an interactive behavior data set of customers and products through monitoring devices arranged in the product display area, and synchronously acquires three-dimensional spatial distribution data of the products displayed on the shelves;

[0193] A processing module 22, which performs image recognition processing on the continuous video stream collected by the monitoring device to extract a dynamic feature set including product displacement trajectories and human body movement forms;

[0194] A generation module 23, which performs spatio-temporal correlation matching on the interactive behavior data set and the three-dimensional spatial distribution data, and generates an array of display effect evaluation parameters including a space occupancy density index, a visual focus distribution rate, and a contact behavior delay value according to the matching result and the change parameters corresponding to the product displacement trajectories in the dynamic feature set;

[0195] A determination module 24, which determines a first optimization factor according to the ratio relationship between the space occupancy density index and the visual focus distribution rate in the array of display effect evaluation parameters, and at the same time determines a second optimization factor according to the association rule between the contact behavior delay value and a preset product category;

[0196] An adjustment module 25, which adjusts the distribution density of products in the vertical direction of the shelves based on the first optimization factor, and adjusts the display orientation of products on the horizontal display surface based on the second optimization factor.

[0197] Figure 2 The described retail data processing system for retail product display evaluation can execute Figure 1A retail data processing method for retail commodity display evaluation described in the illustrated embodiment, the implementation principle and technical effects of which will not be elaborated further. For the retail data processing system for retail commodity display evaluation in the above embodiment, the specific manners in which each module and unit perform operations have been described in detail in the embodiments related to the method, and will not be elaborated here.

[0198] In a possible design, Figure 2 A retail data processing system for retail commodity display evaluation in the illustrated embodiment can be implemented as a computing device, such as Figure 3 as shown, the computing device may include a storage component 31 and a processing component 32;

[0199] The storage component 31 stores one or more computer instructions, wherein the one or more computer instructions are called and executed by the processing component 32.

[0200] The processing component 32 is used for the above Figure 1 retail data processing method for retail commodity display evaluation in the illustrated embodiment.

[0201] Among them, the processing component 32 may include one or more processors to execute computer instructions to complete all or part of the steps in the above method. Of course, the processing component may also be implemented by one or more application-specific integrated circuits (ASICs), digital signal processors (DSPs), digital signal processing devices (DSPDs), programmable logic devices (PLDs), field programmable gate arrays (FPGAs), controllers, microcontrollers, microprocessors or other electronic components for executing the above method.

[0202] The storage component 31 is configured to store various types of data to support operations on the terminal. The storage component may be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic memory, flash memory, magnetic disk or optical disk.

[0203] Of course, the computing device may also necessarily include other components, such as input / output interfaces, display components, communication components, etc.

[0204] The input / output interface provides an interface between the processing component and the peripheral interface module, and the above peripheral interface module may be an output device, an input device, etc.

[0205] The communication component is configured to facilitate wired or wireless communication between the computing device and other devices, etc.

[0206] Among them, the computing device can be a physical device or an elastic computing host provided by a cloud computing platform. In this case, the computing device can refer to a cloud server. The above-mentioned processing components, storage components, etc. can be basic server resources leased or purchased from a cloud computing platform.

[0207] The embodiments of the present application also provide a computer storage medium storing a computer program, and when the computer program is executed by a computer, it can implement the above-mentioned Figure 1 XX method shown in the embodiments.

[0208] Those skilled in the art can clearly understand that for the convenience and brevity of description, the specific working processes of the above-described system, device, and unit can refer to the corresponding processes in the foregoing method embodiments, and will not be described herein again.

[0209] The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place, or may be distributed to multiple network units. Some or all of the modules can be selected according to actual needs to achieve the purpose of the solution of this embodiment. Those of ordinary skill in the art can understand and implement it without creative efforts.

[0210] Through the description of the above embodiments, those skilled in the art can clearly understand that each embodiment can be implemented by means of software plus a necessary general hardware platform, and of course, it can also be implemented by hardware. Based on such an understanding, the essence of the above technical solution, or the part that contributes to the prior art, can be embodied in the form of a software product. The computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, magnetic disk, optical disk, etc., including several instructions for causing a computer device (which can be a personal computer, a server, or a network device, etc.) to execute the methods described in each embodiment or some parts of the embodiments.

[0211] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present application, and are not intended to limit them. Although the present application has been described in detail with reference to the foregoing embodiments, those of ordinary skill in the art should understand that they can still modify the technical solutions described in the foregoing embodiments, or perform equivalent replacements for some of the technical features; and these modifications or replacements do not make the essence of the corresponding technical solutions deviate from the spirit and scope of the technical solutions of the embodiments of the present application.

Claims

1. A retail data processing method for retail commodity display evaluation, characterized in that Including: Obtaining an interaction behavior dataset of customers and products through a monitoring device set in the product display area, and simultaneously obtaining three-dimensional spatial distribution data of the products displayed on the shelves; Performing image recognition processing on the continuous video stream collected by the monitoring device, and extracting a dynamic feature set including product displacement trajectories and human action forms; Performing spatio-temporal correlation matching on the interaction behavior dataset and the three-dimensional spatial distribution data, and generating an evaluation parameter group of display effects including a space occupancy density index, a visual focus distribution rate, and a contact behavior delay value according to the matching result and the change parameters corresponding to the product displacement trajectories in the dynamic feature set; Determining a first optimization factor according to the ratio relationship between the space occupancy density index and the visual focus distribution rate in the evaluation parameter group of display effects, and simultaneously determining a second optimization factor according to the association rule between the contact behavior delay value and a preset product category; Adjusting the distribution density of products in the vertical direction of the shelves based on the first optimization factor, and adjusting the display orientation of products on the horizontal display surface based on the second optimization factor.

2. The method according to claim 1, characterized in that, Performing spatio-temporal correlation matching on the interaction behavior dataset and the three-dimensional spatial distribution data, and generating an evaluation parameter group of display effects including a space occupancy density index, a visual focus distribution rate, and a contact behavior delay value, including: Performing time-space correspondence association on the time stamps of the behavior events of customers touching products in the interaction behavior dataset and the product display position coordinates in the three-dimensional spatial distribution data to establish a correspondence table between the customer positions with time stamps and the product areas as the matching result; According to the total number of times the products are moved and the length of each customer stay period corresponding to each move in the matching result, dividing the product of the total number and the average value of the stay period lengths by the space occupancy volume of the corresponding shelf partition to generate a space occupancy density index representing the degree of product aggregation; Extracting a continuous image group with continuous changes in the customer's head direction from the human action forms in the dynamic feature set, and generating a visual focus distribution rate representing the visual attention intensity in combination with the length of the customer stay period in the matching result; Calculating the median value of the first contact time differences of all customers in the same partition according to the time difference between the time stamp of the first behavior event when the customer enters the shelf partition and the start time of the product displacement trajectory movement in the matching result to generate a contact behavior delay value reflecting the product accessibility efficiency; Combining and correlating the space occupancy density index, the visual focus distribution rate, and the contact behavior delay value according to three-dimensional spatial coordinates to form an evaluation parameter group of display effects.

3. The method according to claim 2, wherein Performing time-space correspondence association on the time stamps of the behavior events of customers touching products in the interaction behavior dataset and the product display position coordinates in the three-dimensional spatial distribution data, and establishing a correspondence table between the customer positions with time stamps and the product areas as the matching result, including: Extract the time stamps of the behavior events when customers contact the goods from the interaction behavior dataset, and at the same time obtain the coordinates of the goods display positions of the customers from the three-dimensional space distribution data. Align the time stamps of the behavior events and the coordinates of the goods display positions to the same time axis based on the system time axis; Convert the coordinates of the goods display positions according to the coordinate system of the three-dimensional space distribution data to determine the shelf partition identifiers where the customers are located under each time stamp. The shelf partition identifiers are generated by combining the number of layers, columns, and depth distances of the goods display positions; Generate an initial record table containing customer identifiers, shelf partition identifiers, and the displacement status of the goods based on the shelf partition identifiers; When the start time of the movement of the coordinates of the goods display position is monitored, retrieve the time stamps of the behavior events when the customers contact the goods on the same time axis in the initial record table, and synchronously append the relationship between the duration of the goods movement process in the time stamps of the behavior events when the customers contact the goods and the change of the shelf partition identifiers of the goods to the initial record table to form a corresponding relationship table between the customer positions and the goods areas with time stamps as the matching result.

4. The method according to claim 1, wherein Determine the first optimization factor according to the ratio relationship between the space occupancy density index and the visual focus distribution rate in the display effect evaluation parameter group, and at the same time determine the second optimization factor according to the association rule between the contact behavior delay value and the preset commodity category, including: Calculate the real-time ratio of the space occupancy density index to the visual focus distribution rate. When the real-time ratio is greater than the preset high ratio range, convert the real-time ratio into a positive adjustment coefficient in the shelf height direction. When the real-time ratio is less than the preset low ratio range, convert it into a negative adjustment coefficient in the shelf height direction; Generate the first optimization factor according to the product relationship between the cumulative number of times of the positive adjustment coefficient or the negative adjustment coefficient in a continuous time period and the number of layers in the shelf height direction; Compare the contact behavior delay value with the standard delay range of the association rule corresponding to the preset commodity category, and generate the azimuth matching difference of the same type of goods according to the degree to which the contact behavior delay value exceeds or is lower than the standard delay range; Generate the second optimization factor based on the proportional relationship between the azimuth matching difference and the historical contact position distribution density of the preset commodity category on the horizontal display surface.

5. The method according to claim 1, characterized in that, Adjust the distribution density of the goods in the shelf height direction based on the first optimization factor, and adjust the display azimuth of the goods on the horizontal display surface based on the second optimization factor, including: Divide the first optimization factor evenly according to the number of layers in the shelf height direction to obtain the unit adjustment amount corresponding to each layer; Determine the direction of increasing or decreasing the goods spacing of the corresponding layer according to the positive or negative sign of the unit adjustment amount, and update the distribution density of the goods in the corresponding layer in the shelf height direction according to the product of the absolute value of the unit adjustment amount and the preset spacing base value; Determine the movement direction identifier of the horizontal display surface according to the positive or negative sign of the second optimization factor, where the positive identifier indicates moving towards the area with a high distribution density of historical contact positions, and the negative identifier indicates moving towards the area with a low density; Perform a proportional conversion of the absolute value of the second optimization factor and the side length of the regional division grid of the horizontal display surface to obtain the horizontal movement distance, and adjust the display orientation of the commodity in the corresponding grid according to the movement direction identifier and the horizontal movement distance.

6. The method according to claim 4, wherein Compare the contact behavior delay value with the associated rule standard delay range corresponding to the preset commodity category, and generate an orientation matching difference for the same category of commodities according to the degree to which the contact behavior delay value exceeds or is lower than the standard delay range, including: Extract the allowed maximum delay value and the allowed minimum delay value defined in the preset commodity category, and divide the difference interval between the allowed maximum delay value and the allowed minimum delay value into at least three equal sub-intervals; Mark the position of the contact behavior delay value according to the sub-intervals. If the contact behavior delay value is in the right sub-interval of the allowed maximum delay value, calculate the excess amount exceeding the allowed maximum delay value as the positive difference amount. If it is in the left sub-interval of the allowed minimum delay value, calculate the shortage amount less than the allowed minimum delay value as the negative difference amount; Calculate the ratio of the positive difference amount or the negative difference amount to the width of the equal sub-intervals to generate the orientation matching difference.

7. The method according to claim 1, characterized in that Perform image recognition processing on the continuous video stream collected by the monitoring device, and extract a dynamic feature set including the commodity displacement trajectory and the human body movement form, including: Perform moving object segmentation on the difference regions between adjacent frames in the continuous video stream collected by the monitoring device, mark the regions that meet the commodity size threshold in the segmented moving objects as candidate commodity regions, and mark the remaining regions as human body candidate regions; Perform direction continuity verification on the position coordinates of the candidate commodity regions in consecutive frames. If the coordinate change direction of the same commodity in adjacent frames is consistent with the movement direction trend, record its movement path points and connect them into a commodity displacement trajectory; Lock the positions of the trunk and limb key points in the human body candidate regions, generate a posture change sequence according to the position change direction of the key points in adjacent frames, and mark the continuous direction combinations that meet the preset movement forms in the posture change sequence as human body movement forms; Perform event association on the starting position of the commodity displacement trajectory and the corresponding human body movement form within the same time period, and merge them to form a dynamic feature set including the commodity displacement trajectory and the human body movement form.

8. A retail data processing system for retail commodity display evaluation, characterized in that, Including: An acquisition module that acquires the interaction behavior data set of customers and commodities through a monitoring device set in the commodity display area, and synchronously acquires the three-dimensional space distribution data of the commodities displayed on the shelves; A processing module that performs image recognition processing on the continuous video stream collected by the monitoring device and extracts a dynamic feature set including the commodity displacement trajectory and the human body movement form; A generation module that performs spatio-temporal correlation matching on the interaction behavior data set and the three-dimensional space distribution data, and generates an evaluation parameter group of display effects including a space occupancy density index, a visual focus distribution rate, and a contact behavior delay value according to the matching result and the change parameters corresponding to the commodity displacement trajectories in the dynamic feature set; A determination module that determines a first optimization factor according to the ratio relationship between the space occupancy density index and the visual focus distribution rate in the evaluation parameter group of display effects, and simultaneously determines a second optimization factor according to the association rule between the contact behavior delay value and a preset commodity category; An adjustment module that adjusts the distribution density of commodities in the vertical direction of the shelf based on the first optimization factor, and adjusts the display orientation of commodities on the horizontal display surface based on the second optimization factor.

9. A computing device, characterized in that, It includes a processing component and a storage component; the storage component stores one or more computer instructions; the one or more computer instructions are used to be called and executed by the processing component to implement a retail data processing method for retail commodity display evaluation as described in any one of claims 1 to 7.

10. A computer storage medium, characterized in that, A computer program is stored, and when the computer program is executed by a computer, it implements a retail data processing method for retail commodity display evaluation as described in any one of claims 1 to 7.

Images