A retail data processing method and system for retail product display evaluation

Abstract

The present application provides a retail data processing method and system for retail merchandise display evaluation. Among them, customer interaction behavior data and shelf three-dimensional space data are synchronously collected by monitoring devices deployed in the merchandise display area, and image recognition technology is used to extract merchandise displacement trajectories and human motion features from the video stream. A spatiotemporal correlation model is established to match the behavior data with the spatial distribution, and an evaluation parameter group including the space occupancy density index, the visual focus distribution rate, and the contact behavior delay value is generated. A dual-factor optimization mechanism is innovatively introduced: a vertical optimization factor is generated based on the ratio relationship between the spatial density and the visual focus, to achieve dynamic adjustment of the shelf height distribution density and the horizontal display orientation, and to construct a closed-loop feedback display decision system. The technical solution provided by this application realizes the adaptive optimization of the merchandise display layout, effectively improves the merchandise display efficiency and optimizes the customer interaction experience.

Description

Technical Field

[0001] The present application relates to the fields of computer vision, spatial computing, and data-driven decision-making technology, and in particular to a retail data processing method and system for retail product display evaluation. Background Art

[0002] In the new retail landscape of omnichannel integration, consumer behavior is scattered across multiple platforms and touchpoints. Cross-channel data integration and correlation analysis are essential to assess the impact of product display layout on consumer decision-making processes. Core requirements include: preliminary collection and coarse-grained correlation of cross-channel behavioral data; evaluation of display effectiveness based on static statistical analysis, such as mining simple correlations between display parameters and sales indicators through historical data; and support for periodic adjustments to display strategies.

[0003] Currently, some technical solutions utilize data correlation and offline statistical analysis based on membership systems. This solution collects cross-channel behavioral data by forcing users to log into integrated online and offline membership accounts. Offline, users' in-store movements are captured using POS transaction data and simple regional heat maps. Product display parameters are then matched with static indicators such as historical sales data and member purchase frequency to generate periodic display optimization recommendations.

[0004] However, the above solution has some limitations: it relies on a mandatory login membership system, resulting in the complete loss of behavioral data of non-logged-in users and low cross-channel data coverage; the analytical model can only capture linear correlations and cannot identify the nonlinear associations between display parameters and complex consumer behaviors, resulting in a high risk of misjudgment; data collection and analysis are all in offline batch processing mode, which cannot perceive changes in consumer behavior in real time, resulting in display strategy adjustments seriously lagging behind market dynamics. Summary of the Invention

[0005] The present application provides a retail data processing method and system for retail product display evaluation, which is used to solve the problem of low cross-channel data coverage in the prior art.

[0006] In a first aspect, the present application provides a retail data processing method for retail product display evaluation, comprising:

[0007] The monitoring devices installed in the product display area are used to obtain the interaction behavior data set between customers and products, and simultaneously obtain the three-dimensional spatial distribution data of the products displayed on the shelves;

[0008] Performing image recognition processing on the continuous video stream collected by the monitoring device to extract a dynamic feature set including the displacement trajectory of the product and the movement form of the human body;

[0009] Performing spatiotemporal correlation matching on the interactive behavior dataset and the three-dimensional spatial distribution data, and generating a display effect evaluation parameter group including a spatial occupancy density index, a visual focus distribution rate, and a contact behavior delay value based on the matching results and the change parameters corresponding to the product displacement trajectory in the dynamic feature set;

[0010] Determining a first optimization factor based on a ratio between a space occupancy density index and a visual focus distribution rate in the display effect evaluation parameter group, and determining a second optimization factor based on an association rule between the contact behavior delay value and a preset product category;

[0011] The distribution density of the goods in the height direction of the shelf is adjusted based on the first optimization factor, and the display orientation of the goods on the horizontal display surface is adjusted based on the second optimization factor.

[0012] Optionally, a time stamp of a customer's behavior event of contacting a product in the interactive behavior dataset is temporally and spatially associated with the coordinates of the product display location in the three-dimensional spatial distribution data to establish a correspondence table between customer locations and product areas with time stamps as a matching result;

[0013] Based on the total number of times the products were moved in the matching results and the length of the customer's stay period corresponding to each move, the product of the total number of times and the average length of the stay period is divided by the space occupied by the corresponding shelf partition to generate a space occupancy density index representing the degree of product aggregation;

[0014] Extracting a continuous image group in which the customer's head direction continuously changes from the human motion form of the dynamic feature set, and combining the customer's stay period length in the matching result to generate a visual focus distribution rate representing the visual attention intensity;

[0015] Based on the time difference between the first behavioral event time stamp of the customer entering the shelf partition and the start time of the product displacement trajectory in the matching result, the median value of the first contact time difference of all customers in the same partition is calculated to generate a contact behavior delay value reflecting the accessibility efficiency of the product;

[0016] The space occupancy density index, the visual focus distribution rate and the contact behavior delay value are combined and associated according to three-dimensional space coordinates to form a display effect evaluation parameter group.

[0017] Optionally, extracting a time stamp of a customer's behavior event of contacting a product from the interactive behavior dataset, and simultaneously obtaining the coordinates of the customer's product display location from the three-dimensional spatial distribution data, and unifying the behavior event time stamp and the product display location coordinates onto the same time axis based on the system time axis;

[0018] Converting the coordinates of the product display location according to the coordinate system of the three-dimensional spatial distribution data to determine the shelf partition identifier of the customer at each time mark, wherein the shelf partition identifier is generated by combining the number of layers, the number of columns, and the depth distance of the product display location;

[0019] Based on the shelf partition identification, generating an initial record table including a customer identifier, a shelf partition identification and a product displacement status;

[0020] When the start time of the movement of the coordinates of the product display position is monitored, the time mark of the customer's behavior event of contacting the product under the same time axis is retrieved from the initial record table, and the change relationship between the product movement process duration and the product shelf partition mark in the time mark of the customer's behavior event of contacting the product is synchronously appended to the initial record table to form a correspondence table between the customer position and the product area with a time mark as a matching result.

[0021] Optionally, a real-time ratio of the space occupancy density index to the visual focus distribution rate is calculated; when the real-time ratio is greater than a preset high ratio range, the real-time ratio is converted into a positive adjustment coefficient in the direction of the shelf height; and when the real-time ratio is less than a preset low ratio range, the real-time ratio is converted into a negative adjustment coefficient in the direction of the shelf height;

[0022] Generate a first optimization factor according to the product relationship between the cumulative number of the positive adjustment coefficient or the reverse adjustment coefficient in a continuous time period and the number of layers in the shelf height direction;

[0023] Comparing the contact behavior delay value with a preset association rule standard delay range corresponding to a product category, and generating a position matching difference value for similar products based on the degree to which the contact behavior delay value exceeds or falls below the standard delay range;

[0024] A second optimization factor is generated based on a proportional relationship between the orientation matching difference and the historical contact position distribution density of the preset commodity category on the horizontal display surface.

[0025] Optionally, the first optimization factor is evenly divided according to the number of layers in the shelf height direction to obtain a unit adjustment amount corresponding to each layer;

[0026] Determine the direction of increase or decrease of the spacing between commodities on the corresponding layer according to the positive or negative sign of the unit adjustment amount, and update the distribution density of commodities on the corresponding layer in the height direction of the shelf according to the product of the absolute value of the unit adjustment amount and the preset spacing base value;

[0027] Determine a movement direction identifier of the horizontal display surface according to the positive or negative sign of the second optimization factor, wherein a positive identifier indicates movement toward an area with a high density of historical contact positions, and a negative identifier indicates movement toward an area with a low density;

[0028] The absolute value of the second optimization factor is proportionally converted to the side length of the area division grid of the horizontal display surface to obtain a horizontal movement distance, and the display orientation of the product in the corresponding grid is adjusted according to the movement direction identifier and the horizontal movement distance.

[0029] Optionally, extracting the maximum allowable delay value and the minimum allowable delay value defined in the preset product category, and dividing the difference interval between the maximum allowable delay value and the minimum allowable delay value into at least three equal intervals;

[0030] Marking the contact behavior delay value according to the difference interval; if the contact behavior delay value is in the right interval of the maximum allowable delay value, calculating the excess of the contact behavior delay value over the maximum allowable delay value as a positive difference; if the contact behavior delay value is in the left interval of the minimum allowable delay value, calculating the shortage of the contact behavior delay value from the minimum allowable delay value as a negative difference;

[0031] The positive difference or the negative difference is calculated as a ratio to the width of the equal partitions to generate an orientation matching difference.

[0032] Optionally, performing moving object segmentation on the difference areas of adjacent frames in the continuous video stream collected by the monitoring device, marking the areas of the segmented moving objects that meet the product size threshold as candidate product areas, and marking the remaining areas as candidate human body areas;

[0033] Performing a directional continuity check on the position coordinates of the candidate product area in consecutive frames. If the coordinate change direction and movement direction trend of the same product in adjacent frames are consistent, then record its movement path points and connect them to form a product displacement trajectory.

[0034] Position locking is performed on the trunk and limb key points in the candidate human body region, a posture change sequence is generated according to the position change direction of the key points in adjacent frames, and a continuous direction combination in the posture change sequence that conforms to a preset action form is marked as a human body action form;

[0035] The starting position of the commodity displacement trajectory within the same time period is associated with the corresponding human motion form, and the dynamic feature set including the commodity displacement trajectory and the human motion form is formed.

[0036] In a second aspect, the present application provides a retail data processing system for retail product display evaluation, comprising:

[0037] The acquisition module acquires the interactive behavior data set between customers and products through the monitoring device installed in the product display area, and simultaneously acquires the three-dimensional spatial distribution data of the products displayed on the shelves;

[0038] a processing module that performs image recognition processing on the continuous video stream collected by the monitoring device to extract a dynamic feature set including the displacement trajectory of the product and the movement form of the human body;

[0039] a generation module that performs spatiotemporal correlation matching on the interactive behavior dataset and the three-dimensional spatial distribution data, and generates a display effect evaluation parameter group including a spatial occupancy density index, a visual focus distribution rate, and a contact behavior delay value based on the matching results and the change parameters corresponding to the product displacement trajectory in the dynamic feature set;

[0040] a determination module for determining a first optimization factor based on a ratio of a space occupancy density index to the visual focus distribution rate in the display effect evaluation parameter group, and determining a second optimization factor based on an association rule between the contact behavior delay value and a preset product category;

[0041] An adjustment module adjusts the distribution density of the goods in the height direction of the shelf based on the first optimization factor, and adjusts the display orientation of the goods on the horizontal display surface based on the second optimization factor.

[0042] In a third aspect, an embodiment of the present application provides a computing device comprising a processing component and a storage component; the storage component stores one or more computer instructions; the one or more computer instructions are used to be called and executed by the processing component to implement a retail data processing method for retail product display evaluation as described in the first aspect above.

[0043] In a fourth aspect, an embodiment of the present application provides a computer storage medium storing a computer program. When the computer program is executed by a computer, the computer program implements a retail data processing method for retail product display evaluation as described in the first aspect.

[0044] In an embodiment of the present application, a monitoring device set in a product display area obtains a data set of interactive behavior between customers and products, and simultaneously obtains three-dimensional spatial distribution data of products displayed on shelves; image recognition processing is performed on the continuous video stream collected by the monitoring device to extract a dynamic feature set including product displacement trajectories and human body movement forms; the interactive behavior data set is temporally and spatially correlated with the three-dimensional spatial distribution data, and a display effect evaluation parameter group including a space occupancy density index, a visual focus distribution rate and a contact behavior delay value is generated based on the matching results and the change parameters corresponding to the product displacement trajectories in the dynamic feature set; a first optimization factor is determined based on the ratio of the space occupancy density index to the visual focus distribution rate in the display effect evaluation parameter group, and a second optimization factor is determined based on the association rule between the contact behavior delay value and a preset product category; the distribution density of the products in the height direction of the shelf is adjusted based on the first optimization factor, and the display orientation of the products on the horizontal display surface is adjusted based on the second optimization factor.

[0045] The technical solution of this application has the following beneficial effects:

[0046] Through spatiotemporal correlation modeling, customer contact behavior, visual focus distribution and product spatial coordinates are dynamically bound to generate multi-dimensional parameters (aggregation, attention, and reach efficiency), realizing the transformation from raw data to decision-making indicators; based on parameter logic, vertical density optimization factors (balancing space occupancy and visual appeal) and horizontal orientation optimization factors (matching product attributes and reach efficiency) are derived, ultimately driving the three-dimensional intelligent adjustment of shelf layout, solving the problems of behavioral and spatial separation and the failure of static rules in traditional display evaluation, and improving product display efficiency and customer interaction experience.

[0047] Furthermore, through spatiotemporal correlation, the time stamps of customer contact behaviors are matched to the three-dimensional coordinates of the products, establishing a timestamped customer-product location mapping relationship. A spatial occupancy density index is calculated based on the number of product moves, customer dwell time, and spatial volume to quantify product concentration. A visual focus distribution rate is generated by combining head orientation change images with dwell time to reflect visual attention intensity. The contact behavior delay value is calculated using the median value of the first contact time difference to assess product reachability. Finally, the above parameters are correlated according to three-dimensional coordinates to form a multi-dimensional evaluation system. By quantifying the dynamic correlation between customer behavior and product spatial distribution, high-attention product areas and inefficient display blind spots can be accurately identified, providing data support for optimizing the vertical density distribution and horizontal layout of shelves. This allows for dynamic display adjustments based on real-world interactive behaviors, effectively improving product exposure efficiency, optimizing spatial resource allocation, and shortening customer decision-making processes, forming a closed-loop intelligent display optimization mechanism.

[0048] These and other aspects of the present application will become more readily apparent from the description of the following embodiments. BRIEF DESCRIPTION OF THE DRAWINGS

[0049] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, a brief introduction will be given below to the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without any creative work.

[0050] Figure 1 A flowchart of a retail data processing method for retail product display evaluation provided by the present application is shown;

[0051] Figure 2 A schematic diagram of the structure of a retail data processing system for retail product display evaluation provided by the present application is shown;

[0052] Figure 3 A schematic structural diagram of a computing device provided by the present application is shown. DETAILED DESCRIPTION

[0053] In order to enable those skilled in the art to better understand the solution of the present application, the technical solution in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application.

[0054] In some of the processes described in the specification and claims of this application and the above-mentioned figures, multiple operations that appear in a specific order are included, but it should be clearly understood that these operations may not be executed in the order in which they appear in this document or may be executed in parallel. The serial numbers of the operations, such as 101, 102, etc., are only used to distinguish between different operations, and the serial numbers themselves do not represent any order of execution. In addition, these processes may include more or fewer operations, and these operations may be executed in sequence or in parallel. It should be noted that the descriptions of "first", "second", etc. in this document are used to distinguish different messages, devices, modules, etc., and do not represent a sequential order, nor do they limit "first" and "second" to being different types.

[0055] Researchers have found that traditional retail display evaluation relies on manual observation or static data analysis, which is unable to capture customer behavior dynamics in real time and disconnects product spatial distribution from interactive behavior. This leads to delayed display adjustments and a lack of quantitative basis. Based on this, an intelligent display optimization method based on multi-source data fusion is proposed. This method can synchronously collect customer interactive behavior and three-dimensional shelf data through monitoring devices, combine image recognition to extract product displacement trajectories and human motion characteristics, construct a spatiotemporal correlation model to generate multidimensional evaluation parameters (spatial occupancy density, visual focus distribution, contact delay), and dynamically generate optimization factors based on the logical relationship between parameters and product attribute rules to achieve adaptive adjustment of the vertical density and horizontal orientation of the shelves.

[0056] The technical solution of this application can be applied to the real-time optimization of merchandise display in scenarios such as supermarket shelves and retail stores, and is particularly suitable for refined operational scenarios that require increasing the exposure of high-value goods and shortening the customer decision path.

[0057] The following will be combined with the drawings in the embodiments of this application to clearly and completely describe the technical solutions in the embodiments of this application. Obviously, the embodiments described are only part of the embodiments of this application, not all of the embodiments. Based on the embodiments in this application, all other embodiments obtained by those skilled in the art without making creative efforts are within the scope of protection of this application.

[0058] Figure 1 A flowchart of a retail data processing method for retail product display evaluation is provided for an embodiment of the present application. Figure 1 As shown, the method includes:

[0059] 101. Obtain a data set of customer-product interaction behaviors through monitoring devices installed in the product display area, and simultaneously obtain three-dimensional spatial distribution data of the products displayed on the shelves;

[0060] In this step, the monitoring device refers to the video acquisition equipment and sensor network deployed in the product display area, including hardware devices such as high-definition cameras, depth sensors and RFID readers, which are used to collect real-time interaction behavior data between customers and products.

[0061] The interactive behavior dataset refers to a structured data set obtained through a monitoring device, which includes records of the number of hand-reaching action triggers, the duration of physical contact, and the duration of visual attention in a continuous time dimension.

[0062] Three-dimensional spatial distribution data refers to the shelf merchandise spatial layout data generated by lidar scanning or computer vision three-dimensional reconstruction technology, which includes the position coordinates of each merchandise unit in three-dimensional space and the distance between adjacent merchandise.

[0063] In an embodiment of the present application, first, the infrared sensors and cameras in the product display area are used to collect the interactive behavior data between customers and products in real time (such as customer A staying in front of the shelf for 30 seconds and touching the product 3 times), and at the same time, a three-dimensional laser scanner is used to obtain the three-dimensional spatial distribution data of the products displayed on the shelf (such as product B is located on the second layer of the shelf, with coordinates (x, y, z) = (1.2m, 0.5m, 1.8m)). Secondly, the two types of data are aligned through a timestamp synchronization mechanism to generate an original data set with spatiotemporal labels. Finally, the data is stored in a distributed database (such as MongoDB) to provide input for step 102.

[0064] In a smart display optimization project for the daily necessities section of a large supermarket, supermarket staff installed multiple sets of high-definition cameras and depth sensors above the shelves. These monitoring devices record customer interactions with products around the clock. Every time a customer picks up or touches items like shampoo and conditioner on the shelf, the system accurately records the time, location, and product information. Simultaneously, laser scanners create a 3D model of the shelves, capturing the specific placement and spatial coordinates of each item. This data is synchronized and processed to form a complete dataset containing timestamps, product information, and 3D coordinates, laying the foundation for subsequent analysis.

[0065] 102. Perform image recognition processing on the continuous video stream collected by the monitoring device to extract a dynamic feature set including the displacement trajectory of the product and the movement form of the human body;

[0066] In this step, the continuous video stream refers to the uncompressed raw video data continuously collected by the monitoring device, which records customer behavior at a rate of 30 frames per second.

[0067] Dynamic feature sets refer to quantitative features extracted from video streams through computer vision algorithms.

[0068] The product displacement trajectory is the product movement path data generated by the target detection and tracking algorithm, including motion parameters such as displacement, speed and acceleration.

[0069] Human motion morphology is the coordinates of key points of the customer's limbs extracted by the posture estimation algorithm, especially behavioral features such as hand grasping movements and body orientation angles.

[0070] In the embodiment of the present application, first, the continuous video stream of step 101 is frame decomposed using the OpenCV image processing library, and the customer's action (such as "reaching out for something" and "stopping to watch") and the product displacement trajectory (such as product C is displaced from the shelf to the customer's hand) are identified by the YOLOv5 target detection algorithm. Secondly, the product movement speed and direction (such as displacement speed 0.3m / s and direction angle 45°) are calculated based on the optical flow method (Optical Flow), and a dynamic feature set including trajectory coordinates, speed and action category is generated. Finally, structured feature data (such as {"Product C":{"Displacement trajectory":[(x1,y1,t1),(x2,y2,t2)],"Action type":"Get something"}}) is output to provide a matching basis for step 103.

[0071] When a customer pauses in front of a shelf, the system analyzes the video footage captured by the camera in real time. Using advanced image recognition algorithms, the system accurately identifies the customer's movements as they pick up a bottle of shampoo and tracks the product's complete movement from the shelf to their shopping basket. The system also analyzes the customer's body posture and hand movements, such as whether they carefully check the label after picking up the product or hesitate before returning it to the shelf. This dynamic data is processed to create a detailed record of product movement and customer behavior, helping supermarkets understand their customers' true shopping habits.

[0072] 103. Perform spatiotemporal correlation matching on the interactive behavior dataset and the three-dimensional spatial distribution data, and generate a display effect evaluation parameter group including a spatial occupancy density index, a visual focus distribution rate, and a contact behavior delay value based on the matching results and the change parameters corresponding to the product displacement trajectory in the dynamic feature set;

[0073] In this step, spatiotemporal correlation matching refers to establishing a mapping relationship between interactive behavior data and three-dimensional space data through timestamp alignment and coordinate transformation.

[0074] The space occupancy density index is the ratio of the number of interacted products to the total capacity within a unit shelf area, which is calculated based on the spacing information and location coordinates of adjacent products.

[0075] The visual focus distribution rate is the proportion of the time that customers' gazes stay in each section of the shelf, generated by the eye tracking algorithm. It is generated by associating the visual attention duration record with the product displacement trajectory.

[0076] The contact behavior delay value refers to the time interval from the first time a customer looks at the product to the actual contact with it. This value is calculated through time series analysis.

[0077] In the embodiment of the present application, first, the three-dimensional spatial distribution data of step 101 is associated with the dynamic feature set of step 102 according to the time-space label to match customer behavior with product location (for example, customer D contacts product E at time t1, and the location of product E is (1.5m, 0.6m, 1.2m)). Secondly, the evaluation parameters are calculated by the following formula: Space occupancy density index: the number of customer stops per unit area (for example, the number of times a customer stays per unit area in a 2m shelf area). 2 Stay 10 times → Index 5 times / m 2 ); Visual Focus Distribution Rate: the percentage of the total time a customer gazes at a product (e.g., if the total duration is 120 seconds and product F is gazed at for 60 seconds, the distribution rate is 50%); Contact Behavior Delay: the average time difference between a customer's gaze and actual contact (e.g., an average delay of 2.5 seconds). Finally, an evaluation parameter set (e.g., {"spatial density index": 5,""focus distribution rate": 50%,""contact delay": 2.5s}) is generated to provide a basis for calculating the optimization factor in step 104.

[0078] The system matches and analyzes collected customer behavior data with a three-dimensional model of the shelf. By comparing data from different time periods, it can calculate the level of attention and actual purchase conversion rate for each product area. For example, data shows that while a certain brand of shampoo is frequently picked up and inspected by customers, a low percentage of them ultimately add it to their shopping carts. The system also analyzes the time between a customer's awareness of a product and their final purchase decision. These metrics together form a comprehensive parameter system for evaluating product display effectiveness, providing data support for subsequent optimization and adjustment.

[0079] 104. Determine a first optimization factor based on a ratio of a space occupancy density index to the visual focus distribution rate in the display effect evaluation parameter group, and simultaneously determine a second optimization factor based on an association rule between the contact behavior delay value and a preset product category;

[0080] In this step, the display effect evaluation parameter group includes three core indicators: space occupancy density index, visual focus distribution rate and contact behavior delay value.

[0081] The first optimization factor refers to the normalized ratio of the space occupancy density index to the visual focus distribution rate, which is used to measure the spatial efficiency of product display.

[0082] The second optimization factor is an adjustment coefficient generated based on a preset rule engine, which correlates the contact behavior delay value with product category characteristics (such as shelf life and price range).

[0083] The association rules for preset product categories refer to a decision tree model trained through machine learning. This model defines the optimal delay threshold for different product categories based on historical sales data.

[0084] In the embodiment of the present application, first, according to the space occupancy density index (5 times / m 2 ) and the visual focus distribution rate (50%), calculate the first optimization factor (formula: first factor = focus distribution rate / spatial density index = 50% / 5 = 10% m 2 / times). Next, using an association rule mining algorithm (Aprior i), the relationship between the contact behavior delay value (2.5s) in step 103 and the product category (e.g., "food," "daily necessities") is analyzed to determine a second optimization factor (e.g., food category delay threshold 2s, actual delay 2.5s → factor = 2.5 - 2 = 0.5s). Finally, the first and second optimization factors are combined to output an optimization factor list (e.g., {"first factor": 10%, "second factor": 0.5s}), providing display adjustment parameters for step 105.

[0085] Based on the results of previous data analysis, the system automatically generates optimization suggestions. For example, if it discovers that certain high-end shampoos, while prominently displayed, take longer to decide due to price, the system will recommend adjusting their display position. Furthermore, for popular items that are frequently and quickly purchased, the system will recommend increasing their display density. These optimization suggestions are derived from analyzing extensive amounts of actual sales data, taking into account both the product's characteristics and customer shopping habits.

[0086] 105. Adjust the distribution density of the products in the height direction of the shelf based on the first optimization factor, and adjust the display orientation of the products on the horizontal display surface based on the second optimization factor.

[0087] In this step, the distribution density in the height direction of the shelf refers to the spatial arrangement density of the goods on the vertical display level, which is achieved by adjusting the distance between the shelves or the number of goods on each layer.

[0088] The display orientation of the horizontal display surface refers to the placement angle and direction of the product within a single-layer plane of the shelf, specifically including parameters such as the rotation angle of the product label surface and the angle with the aisle.

[0089] The execution of the first optimization factor is reflected in the control instructions for automatically adjusting the shelf lifting mechanism, while the second optimization factor is converted into a display orientation adjustment plan guided by AR glasses.

[0090] In the embodiment of the present application, first, based on the first optimization factor (10%·m 2 / times), adjust the distribution density of the shelf height direction in proportion (for example, the larger the factor value, the lower the density of goods in the high shelf layer, the original density is 5 pieces / m 2 Adjusted to 4 pieces / m 2Secondly, the product display orientation on the horizontal display surface is adjusted based on the second optimization factor (0.5s) (e.g., if food products are delayed beyond the limit, the products are moved 0.3m to the front of the shelf). Finally, the layout is updated through the automated shelf adjustment device, and the adjusted parameters are monitored in real time to complete the closed-loop optimization.

[0091] Finally, supermarket staff will adjust shelf displays based on the optimization solutions provided by the system. This may include reorganizing the product tiers, adjusting certain items to a more ergonomic height, or changing the product display angles to make product labels more visible to customers. In some more intelligent supermarkets, automatic shelf adjustments can even optimize product displays in real time. These adjustments are based on in-depth analysis of customer shopping behavior and are designed to enhance the customer shopping experience while also helping supermarkets improve sales performance.

[0092] In summary, steps 101 to 105 achieve dynamic and precise control of product displays. Through multi-source data fusion and intelligent analysis, customer behavioral characteristics are deeply correlated with product spatial distribution, forming a quantifiable display effectiveness evaluation system. The system automatically identifies differences in product attractiveness and customer decision-making patterns, intelligently generating targeted display optimization plans to achieve optimal configuration of shelf space utilization and product exposure. Ultimately, a closed-loop optimization mechanism with self-learning capabilities is established, significantly improving product display effectiveness and the customer shopping experience, providing data-driven intelligent display decision support for retail scenarios.

[0093] To address the lack of objective, quantitative evaluation criteria for merchandise display effectiveness in retail scenarios, we developed an intelligent display evaluation system based on multi-source data fusion. By performing spatiotemporal correlation matching of customer interaction behavior datasets with three-dimensional spatial distribution data, we enable dynamic, parametric analysis of display effectiveness. This system provides data-driven decision support for retail display optimization, effectively addressing the traditional reliance on subjective experience in display evaluation.

[0094] In some embodiments, step 103 involves performing spatiotemporal correlation matching on the interactive behavior dataset and the three-dimensional spatial distribution data, and generating a display effect evaluation parameter group including a spatial occupancy density index, a visual focus distribution rate, and a contact behavior delay value based on the matching results and the change parameters corresponding to the product displacement trajectory in the dynamic feature set, including:

[0095] 201. Performing temporal-spatial correspondence between the time stamps of the customer's behavior events of contacting the product in the interactive behavior dataset and the coordinates of the product display locations in the three-dimensional spatial distribution data, so as to establish a correspondence table between the customer's location and the product area with the time stamp as a matching result;

[0096] In step 201, the interactive behavior dataset refers to a collection of customer interactions with products during the shopping process, including actions such as touching, picking up, and putting down, as well as the time information when they occur. Three-dimensional spatial distribution data refers to the coordinate data of product display locations collected through spatial modeling or sensors, and is used to accurately describe the physical distribution of products on shelves or display areas. Time-space correspondence is the process of matching the time stamps of customer behavior with the spatial coordinates of products, thereby establishing an association between the customer's location at a specific time and the product area. The correspondence table is the final generated tabular data, containing fields such as customer behavior time, product area coordinates, and behavior type, and is used for subsequent analysis of customer shopping paths and product engagement.

[0097] In an embodiment of the present application, first, the time stamp of the customer contact behavior event in the interactive behavior data set (such as customer A contacts product B at time t1) is aligned with the product display position coordinates in the three-dimensional spatial distribution data of step 101 (such as the coordinates of product B located in shelf area X (1.2m, 0.5m, 1.8m)) through timestamp synchronization technology; secondly, a hash table data structure is used to establish a key-value pair mapping (Key = timestamp + area ID, Value = customer position and product coordinates); then, the customer position and product area coordinates in the same time period are aggregated through a spatial clustering algorithm (such as DBSCAN) to generate a corresponding relationship table with time stamps (such as {"t1":{"area X":{"customer position":(1.1m,0.6m),"product coordinates":[(1.2m,0.5m,1.8m)]}}}); finally, a matching result table is output to provide a spatiotemporal association basis for subsequent parameter calculation.

[0098] 202. Based on the total number of times that products are moved in the matching results and the length of customer stay periods corresponding to each move, divide the product of the total number of times and the average length of stay periods by the spatial volume of the corresponding shelf partition to generate a spatial occupancy density index representing the degree of product aggregation;

[0099] In step 202, the product locations in the matching results are mapped to the shelf partitions according to the shelf partition definitions in the three-dimensional spatial distribution data. The total number of times a product is moved refers to the cumulative number of times a certain product is picked up or moved by customers, as counted in the correspondence table. The length of a customer's stay period refers to the duration from the start to the end of the interaction when the customer comes into contact with the product, reflecting the customer's level of attention to the product. The space occupancy density index is an indicator calculated by multiplying the total number of moves and the average stay time by the volume of the partition where the product is located. It is used to measure the attractiveness of a product in a unit space. The higher the value, the more likely the product is to attract customer behavior in the display.

[0100] In the embodiment of the present application, first, based on the matching result table of step 201, the total number of times the product is moved (for example, product B is moved 5 times in area X) and the length of the customer's stay period corresponding to each move (for example, the stay time is 10s, 15s, and 20s respectively, with an average stay of 15s); secondly, the space occupied by the corresponding shelf partition is extracted (for example, the volume of area X is 2m 3 ); Then, calculate according to the formula space occupancy density index = total number of times × average stay time / space volume (such as 5 × 15 / 2 = 37.5 times · seconds / m 3 ); Finally, all space occupancy density indices are integrated to output a density index list of each partition to provide input for step 205.

[0101] 203. Extracting a continuous image group in which the customer's head direction continuously changes from the human motion form of the dynamic feature set, and combining the customer's stay period length in the matching result to generate a visual focus distribution rate representing the visual attention intensity;

[0102] In step 203, human motion morphology refers to customer posture features, such as head angle and body orientation, extracted from surveillance video using computer vision technology. A continuous image group is a chronological sequence of images showing changes in the customer's head orientation, used to analyze the customer's gaze trajectory. The visual focus distribution rate is an indicator calculated based on the duration of a customer's gaze and the area covered by the product. It reflects the intensity of a customer's visual attention to a specific product. A higher value indicates that the product is more likely to attract customers' attention within the display.

[0103] In an embodiment of the present application, first, a continuous image group of changes in the customer's head direction is extracted from the dynamic feature set of step 102 (such as the head deflection angle of customer A increases from 0° to 30° within 10 seconds); secondly, the proportion of the duration of the head direction to the total stay time is calculated by the OpenCV direction tracking algorithm (such as the time when the head is facing product B for 8 seconds / the total stay time of 10 seconds = 80%); then, combined with the length of the customer's stay period in step 201 (such as 10 seconds), the visual focus distribution rate is generated (such as 80%); finally, a focus distribution rate list of each product is output (such as {"product B":"focus distribution rate 80%"}) to provide associated parameters for step 205.

[0104] 204. Calculate the median of the first contact time differences of all customers in the same shelf partition based on the time difference between the first behavioral event time of the customer entering the shelf partition and the start time of the product displacement trajectory in the matching result, so as to generate a contact behavior delay value reflecting the product accessibility efficiency;

[0105] In step 204, the first behavioral event timestamp refers to the time when a customer first touches a product after entering a shelf section. The product displacement trajectory start time refers to the moment when the product is picked up or moved by the customer. The median first contact time difference is the median of the time differences between the first contact with the product for all customers in the same section. It is used to measure the delay from entering the section to the actual contact with the product. The contact behavior delay value is a standardized expression of this median time difference and is used to assess the accessibility of product displays. The lower the value, the easier it is for customers to discover and interact with the product.

[0106] In an embodiment of the present application, first, the time mark of the first behavior event of the customer entering the shelf partition (such as the time t1 = 10:00:00 when customer A enters area X) and the start time of the product displacement trajectory (such as the start time t2 = 10:00:02 when product B moves) are extracted from the matching result table of step 201; secondly, the first contact time difference is calculated (such as t2-t1 = 2 seconds), and the delay values ​​of all customers in the same partition are counted (such as the delay value list of area X [2s, 3s, 1s]); then, the delay value is generated by the median calculation algorithm (such as the middle value 2s after sorting); finally, the contact behavior delay value of each partition is output (such as {"area X":"delay value 2s"}) to provide efficiency evaluation parameters for step 205.

[0107] 205. Combine and associate the space occupancy density index, the visual focus distribution rate, and the contact behavior delay value according to three-dimensional spatial coordinates to form a display effect evaluation parameter group.

[0108] In step 205, the display effectiveness evaluation parameter set is a comprehensive evaluation system composed of three indicators: spatial occupancy density index, visual focus distribution rate, and contact behavior delay value. Each indicator reflects the product's performance in terms of behavioral aggregation, visual appeal, and accessibility. By linking these indicators with the product's three-dimensional spatial coordinates, a multi-dimensional evaluation of shelf display effectiveness can be formed, providing data support for optimizing product placement.

[0109] In the embodiment of the present application, first, based on the spatial occupancy density index of step 202 (such as the area X index of 37.5), the visual focus distribution rate of step 203 (such as the product B distribution rate of 80%), and the contact behavior delay value of step 204 (such as the area X delay of 2s), data association is performed according to the three-dimensional spatial coordinates (such as the area X coordinate range x[1.0m,1.5m]); secondly, a parameter group is generated through JSON structured encapsulation technology (such as {"area X":{"spatial density index":37.5,"focus distribution rate":80%,"contact delay":2s}}); finally, a display effect evaluation parameter group of the three-dimensional spatial dimension is output to provide a quantitative basis for the display optimization strategy.

[0110] Here's a specific example:

[0111] In the actual application of the supermarket smart shopping cart system, the system first uses the on-board camera and shelf RFID sensor to collect the behavioral data of customer A picking up a bottle of shampoo on shelf H1-3 (coordinates X = 2.1m, Y = 1.5m) at 10:03:21. The system then associates the timestamp with the product coordinates to generate a correspondence table. At the same time, it records customer B's touch behavior of the same product at 10:05:44. Then, based on the H1-3 partition, the product was moved 28 times that day, the average customer stay was 12 seconds, and the partition volume was 0.8m. 3 The data was used to calculate the space occupancy density index of 420, reflecting the concentrated attraction of the product; then, by analyzing the image group of customer A's head direction staring at the shampoo for 7 seconds, combined with the length of stay, the visual focus distribution rate was obtained to be 58.3%, indicating that the packaging design effectively attracted attention; then, based on the time difference between customers A and B's first contact with the product (3.2 seconds and 5.8 seconds), the median contact behavior delay value of 4.5 seconds was calculated, indicating that the product display location is easy to reach; finally, the space occupancy density index (420), visual focus distribution rate (58.3%) and contact behavior delay value (4.5 seconds) were associated according to three-dimensional coordinates to form a complete shelf display effect evaluation parameter group, providing data support for optimizing product placement.

[0112] In summary, steps 201 to 205 achieve quantitative evaluation and precise optimization of product display effectiveness. By establishing a spatiotemporal correlation model between customer behavior and product location, the system intelligently identifies differences in product attractiveness and customer decision-making characteristics, generating a three-dimensional evaluation system encompassing spatial concentration, visual attention, and ease of access. This technical solution breaks through the traditional empirical model of display optimization, achieving an intelligent transformation from actual customer behavior data to scientific display decisions. It provides a quantifiable and verifiable display optimization solution for retail scenarios, significantly improving product display effectiveness and the customer shopping experience.

[0113] To address the issue of insufficient accuracy in matching customer behavior and spatial location data in product display analysis, an intelligent display analysis system based on spatiotemporal synchronization has been developed. This system achieves precise correlation and matching between customer contact behavior and product displacement, providing a high-precision spatiotemporal data foundation for display effectiveness evaluation. In some embodiments, step 201 involves performing a temporal and spatial correlation between the time stamps of customer contact events in the interactive behavior dataset and the product display location coordinates in the three-dimensional spatial distribution data, and establishing a time-stamped correspondence table between customer locations and product areas as the matching result, including:

[0114] 301. Extract the time stamp of the customer's behavior event of contacting the product from the interactive behavior dataset, and simultaneously obtain the coordinates of the customer's product display location from the three-dimensional spatial distribution data, and unify the behavior event time stamp and the product display location coordinates onto the same time axis based on the system time axis;

[0115] In step 301, the interactive behavior dataset refers to a time-series data set that records customer interactions with products during the shopping process, including specific actions such as touching, picking up, and putting down, along with their precise time stamps. Three-dimensional spatial distribution data is product display location coordinate information collected through spatial modeling techniques or sensor devices, used to accurately describe the physical distribution of products on the shelf. The system timeline is a unified time coordinate system used to synchronize the time stamps of customer behavior events with the product display location coordinates, ensuring temporal consistency in subsequent analysis.

[0116] In an embodiment of the present application, first, the time mark of the customer's behavioral event of contacting the product is extracted from the interactive behavior data set of step 101 (such as customer A contacts product B at t1 = 10:00:00), and at the same time, the coordinates of the product display position at the corresponding time are obtained from the three-dimensional spatial distribution data of step 101 (such as the coordinates of product B (x, y, z) = (1.2m, 0.5m, 1.8m)); secondly, the timestamps of the two types of data are unified to the same benchmark (such as the time error after calibration ≤ 1ms) through system time axis alignment technology (such as NTP protocol synchronization); then, the time window sliding algorithm is used to aggregate the behavioral events and coordinate data according to the timestamps to generate a time-space aligned original record table (such as {"t1":{"customer A":"product B","coordinates":(1.2,0.5,1.8)}}), which provides input for step 302.

[0117] 302. Convert the coordinates of the product display location according to the coordinate system of the three-dimensional spatial distribution data to determine the shelf partition identifier of the customer at each time mark, wherein the shelf partition identifier is generated by combining the number of layers, the number of columns, and the depth of the product display location;

[0118] In step 302, the product display location coordinates refer to the specific location data of the product in three-dimensional space, including coordinate values ​​of the three dimensions of the X-axis (number of columns), Y-axis (number of layers), and Z-axis (depth distance). Coordinate system conversion refers to the process of standardizing these raw coordinate data according to the physical layout of the actual shelf. The shelf partition identifier is a unique location code generated by combining the three dimensions of layer number, column number, and depth distance. It is used to accurately locate the shelf area where customers or products are located. For example, a code format such as "A-3-2" is used to represent the area of ​​​​the third layer of column A with a depth of 2 meters.

[0119] In an embodiment of the present application, first, according to the predefined rules of the three-dimensional spatial distribution data (such as the number of layers is divided according to the Z-axis height: 0≤z<1m is the first layer, 1≤z<2m is the second layer; the number of columns is one column every 0.5 meters on the X-axis; the depth is one zone every 0.3 meters on the Y-axis), the product coordinate system is converted into a structured shelf partition identifier. For example, the coordinates of product B (1.2m, 0.5m, 1.8m) are mapped to the number of layers 2 (L2), the number of columns 3 (C3), and the depth zone 2 (Y2), and the identifier "L2-C3-Y2" is generated. Secondly, all product coordinates are batch processed by the spatial grid mapping algorithm to generate a list of shelf partition identifiers. Finally, the identifier is associated with the time stamp to form a spatiotemporal data set with partition information, providing standardized location labels for the construction of subsequent record tables.

[0120] 303. Based on the shelf partition identifier, generate an initial record table including a customer identifier, a shelf partition identifier, and a product displacement status;

[0121] In step 303, the shelf partition identifier is a standardized location code generated after coordinate conversion. The customer identifier is a numeric or character code used to uniquely identify the customer, typically automatically generated by a monitoring system or sensor data. Product movement status is dynamic information recording whether the product has been moved, including states such as being picked up, moving, and put back. The initial record table is a structured data storage table containing core fields such as the customer identifier, shelf partition identifier, and product movement status. It is used to initially record basic information about customer interactions with products.

[0122] In an embodiment of the present application, first, based on the data after time and space alignment, the customer identifier (such as customer A), shelf partition identifier (such as L2-C3-Y2) and product displacement status (such as "stationary" or "moving") are extracted. Secondly, an initial record table is constructed through key-value pair storage technology, and each record contains a timestamp, customer identifier, partition identifier and product status. For example, when customer A is located in partition L2-C3-Y2 at 10:00:00 and product B is in a stationary state, the record is: time 10:00:00, customer A, partition L2-C3-Y2, product status: stationary. Finally, the initial record table is stored in a time series database to support efficient time range retrieval and dynamic update.

[0123] 304. When the movement start time of the coordinates of the product display position is monitored, the time mark of the customer's behavior event of contacting the product under the same time axis is retrieved in the initial record table, and the change relationship between the product movement process duration and the product shelf partition identifier in the time mark of the customer's behavior event of contacting the product is synchronously appended to the initial record table to form a correspondence table between the customer position and the product area with a time mark as a matching result.

[0124] In step 304, the start time of movement of the product display location coordinates refers to the starting moment detected by the sensor when the product is picked up or moved by the customer. Timeline synchronization refers to the process of retrieving customer behavior events that match the product movement time in the initial record table. The duration of the product movement process refers to the complete time period from the time the product is picked up to the time it is finally put back or moved to another location. The change relationship of the shelf partition identification refers to the change in the position of the shelf area involved in the product movement process. The correspondence table is the final generated complete matching result data table, which contains complete information fields such as customer identifier, precise time stamp, shelf partition identification, product displacement status and movement process duration.

[0125] In an embodiment of the present application, when the movement start time of the product displacement trajectory is monitored (such as product B starts to move at 10:00:02), the customer behavior event under the same time axis is first retrieved from the initial record table (such as customer A touches product B at 10:00:02). Secondly, the duration of the product movement process is calculated (for example, from 10:00:02 to 10:00:05, the duration is 3 seconds), and the partition identification change caused by the product displacement is tracked (such as moving from L2-C3-Y2 to L1-C2-Y1). Then, the movement duration and partition change relationship are inserted into the initial record table through the data append algorithm to generate a matching result table with complete spatiotemporal association. For example, the updated record is: time 10:00:02, customer A, partition L2-C3-Y2→L1-C2-Y1, product status: moving, moving duration: 3 seconds. Finally, the matching result table containing spatiotemporal behavior association is output to provide accurate data support for display effect analysis.

[0126] Here's a specific example:

[0127] In the actual application of the supermarket intelligent analysis system, the system first aligns the behavioral event of customer C picking up a box of chocolates on shelf G2-5 (coordinates X = 3.4m, Y = 1.2m, Z = 1.5m) at 14:22:36 with the three-dimensional spatial data using a unified timeline, confirming that this behavior occurred on the transition path from the fresh food area to the snack area. The original coordinates are then converted to the standard shelf partition identifier G2-5-3N (3rd layer / 5th column / near-end display area) and an initial record table is created containing the customer ID, timestamp, partition identifier, and product status (picked - brand A). When the system detects that the product coordinates began to move at 14:22:38, it automatically associates customer C's behavior with the time stamp and appends the product movement duration (2 seconds) and partition change relationship (from G2-5-3N to the shopping cart) to the record table, ultimately forming a complete spatiotemporal correspondence table. Customer D's contact with the same partition at 14:23:11 is also synchronously recorded, providing accurate data support for subsequent analysis of product contact timeliness and display location optimization.

[0128] In summary, steps 301 to 304 achieve a precise spatiotemporal mapping of product display and customer behavior. By establishing a unified timeline benchmark, the system synchronizes discrete customer contact events with product spatial coordinates at the millisecond level, building a three-dimensional partition identification system that includes the number of layers, columns, and depth distances. Based on real-time collected behavioral data, the system automatically generates a dynamic association table with spatiotemporal markers, fully recording the state changes from the beginning to the end of the product displacement process. This technical solution breaks through the limitations of the spatiotemporal data separation in traditional retail analysis, realizes digital twin modeling of customer movement routes and product displays, and provides a precise spatiotemporal association foundation for the subsequent quantitative evaluation of display effects.

[0129] To address the lack of a multi-dimensional decision-making basis for dynamic optimization of merchandise displays, a dual-factor-driven intelligent display optimization system was developed. The system generates a first optimization factor for shelf height adjustment by analyzing the ratio of space occupancy density to visual focus. The second optimization factor for horizontal orientation optimization is generated by combining the association rules between contact delay and product category. This system implements an intelligent adjustment solution for display space based on both vertical and horizontal dimensions, significantly improving the scientific nature and accuracy of display optimization decisions.

[0130] In some embodiments, the first optimization factor is determined in step 104 based on the ratio of the space occupancy density index in the display effect evaluation parameter group to the visual focus distribution rate, and the second optimization factor is determined based on the association rule between the contact behavior delay value and the preset product category, including:

[0131] 401. Calculate a real-time ratio of the space occupancy density index to the visual focus distribution rate. When the real-time ratio is greater than a preset high ratio range, convert the real-time ratio into a positive adjustment coefficient in the direction of shelf height. When the real-time ratio is less than a preset low ratio range, convert the real-time ratio into a negative adjustment coefficient in the direction of shelf height.

[0132] In step 401, the spatial occupancy density index is a comprehensive indicator reflecting the frequency of customer contact and duration of customer interaction within a unit space. A higher value indicates a more attractive product. The visual focus distribution rate measures the distribution of customer visual attention on a product, reflecting the degree to which the product attracts the customer's attention. The real-time ratio is the dynamic proportional relationship between the spatial occupancy density index and the visual focus distribution rate, used to determine whether the actual attractiveness of a product matches its visual appeal. The high ratio range is a pre-set upper threshold. When the real-time ratio exceeds this range, it indicates that the actual frequency of contact with the product exceeds its visual appeal. The low ratio range is a pre-set lower threshold. When the real-time ratio falls below this range, it indicates that the visual appeal of the product exceeds its actual frequency of contact. The positive adjustment coefficient is a parameter generated based on a high ratio, used to adjust the shelf height upward and increase the product display height. The negative adjustment coefficient is a parameter generated based on a low ratio, used to adjust the shelf height downward and decrease the product display height.

[0133] In the embodiment of the present application, first, according to the space occupancy density index generated in step 205 (such as the shelf area X index 37.5 times·second / m 3 ) and the visual focus distribution rate (such as 80%), calculate the real-time ratio (formula: real-time ratio = space occupancy density index / visual focus distribution rate, example: 37.5 / 80≈0.469). Secondly, preset a high ratio range (such as ≥0.6) and a low ratio range (such as ≤0.3). If the real-time ratio is >0.6, convert it into a positive adjustment coefficient (such as coefficient = real-time ratio - 0.6 = 0.469 - 0.6 = -0.131, because it does not exceed the threshold, the positive adjustment is not triggered); if the real-time ratio is <0.3, convert it into a negative adjustment coefficient (such as coefficient = 0.3 - real-time ratio = 0.3 - 0.469 = -0.169, because it does not fall below the threshold, the negative adjustment is not triggered). Finally, the coefficient is dynamically adjusted according to the threshold judgment result to provide parameter input for step 402.

[0134] 402. Generate a first optimization factor based on the product relationship between the cumulative number of the positive adjustment coefficient or the reverse adjustment coefficient in a continuous time period and the number of layers in the shelf height direction;

[0135] In step 402, the forward adjustment coefficient and the reverse adjustment coefficient are parameters that reflect the direction and magnitude of adjustments required for the product display height. A continuous time period refers to a specific time period during which data collection and analysis are performed. The cumulative number of times a coefficient requires adjustment occurs within that time period. The number of layers in the shelf height direction refers to the total number of layers in the vertical direction of the shelf. The first optimization factor is a parameter calculated by multiplying the cumulative number of times by the number of layers. This factor is used to quantify the overall degree to which the product requires adjustment in the shelf height direction.

[0136] In an embodiment of the present application, first, the cumulative number of triggering of the forward adjustment coefficient or the reverse adjustment coefficient of step 401 within a continuous time period (such as the past 1 hour) is counted (such as 2 times for the forward adjustment and 1 time for the reverse adjustment); secondly, the number of layers in the shelf height direction is obtained (such as 3 layers in area X); then, the first optimization factor is calculated according to the formula: cumulative number of times × number of layers (such as forward factor = 2×3=6, reverse factor = 1×3=3); finally, a first optimization factor list is output (such as {"area X":{"forward factor":6,"reverse factor":3}}) to provide a quantitative basis for shelf height density adjustment.

[0137] 403. Compare the contact behavior delay value with a preset association rule standard delay range corresponding to a product category, and generate a position matching difference value for similar products based on the degree to which the contact behavior delay value exceeds or falls below the standard delay range;

[0138] In step 403, the contact behavior delay value is an indicator that reflects the time it takes for a customer to enter a shelf section and actually touch a product. Preset product categories refer to a classification system pre-defined based on product characteristics. The association rule standard delay range is a pre-set reasonable contact time range for each product category. The orientation matching difference is a parameter calculated based on the degree of deviation between the actual contact behavior delay value and the standard range. It is used to measure the degree of difference between the current product display position and the ideal position. This difference value includes both positive differences exceeding the standard range and negative differences falling below the standard range.

[0139] In the embodiment of the present application, first, based on the contact behavior delay value of step 204 (such as a 2-second delay in area X) and the standard delay range of the preset product category (such as the standard delay of the food category [1.5s, 2.5s]), it is determined whether the delay value exceeds or falls below the standard range (such as 2 seconds is within the range, and the difference is not triggered); if the delay value is (such as 2.6 seconds), the positive orientation matching difference is calculated (difference = actual delay - upper limit = 2.6-2.5 = 0.1 seconds); if it is below the lower limit (such as 1.3 seconds), the negative orientation matching difference is calculated (difference = lower limit - actual delay = 1.5-1.3 = 0.2 seconds). Finally, a difference list is output (such as {"food category":"difference + 0.1s"}) to provide horizontal display adjustment parameters for step 404.

[0140] 404. Generate a second optimization factor based on a proportional relationship between the orientation matching difference and the historical contact position distribution density of the preset product category on the horizontal display surface.

[0141] In step 404, the horizontal display surface refers to the horizontal display area of ​​the shelf. The historical contact position distribution density refers to the frequency distribution of customer contact for similar products at various locations on the horizontal display surface. The second optimization factor is a parameter calculated from the proportional relationship between the orientation matching difference and the historical contact position distribution density. This factor is used to guide the direction and magnitude of product position adjustment on the horizontal display surface. This factor comprehensively considers the degree of match between the current display effect and historical contact habits, providing a quantitative basis for optimizing the horizontal position of products.

[0142] In the embodiment of the present application, first, based on the orientation matching difference (e.g., +0.1 seconds) of step 403, the historical contact position distribution density of similar products on the horizontal display surface is extracted (e.g., the density of food at the front end of area X is 5 times / m 2 ); Secondly, the second optimization factor is calculated according to the formula = orientation matching difference / historical density (e.g. factor = 0.1 / 5 = 0.02); Then, if the difference is positive, the factor indicates that the display density needs to be increased (e.g. factor 0.02 corresponds to a density increase of 0.1 times / m 2 ); if negative, the density needs to be reduced. Finally, the second optimization factor (e.g., {"food":"factor +0.02"}) is output to drive the adjustment of product positions on the horizontal display surface.

[0143] Here's a specific example:

[0144] During the real-time operation of the supermarket intelligent display optimization system, the system first calculates the real-time ratio of the spatial occupancy density index of the current shelf partition to the visual focus distribution rate. When this ratio exceeds a preset upper threshold (e.g., >1.5), a positive adjustment coefficient (+0.3 / layer) is automatically generated. When it falls below a lower threshold (e.g., <0.8), it is converted to a negative adjustment coefficient (-0.2 / layer). Based on the occurrence of five positive adjustments within three consecutive hours and the four-layer shelf structure, a first optimization factor of 5 × 4 = 20 is generated. Simultaneously, the system compares the contact behavior delay value of 4.5 seconds for the shampoo partition with the standard delay range for daily chemical products (3-6 seconds). Since it is within the normal range, no orientation matching difference is generated. However, the delay value of 7.2 seconds for the adjacent conditioner partition (exceeding the standard by 1.2 seconds) generates an orientation matching difference of 1.2 seconds. Combined with the historical horizontal distribution density of 12 times / ㎡ for this category, a second optimization factor of 1.2 × 12 = 14.4 is calculated. Ultimately, this dual-factor synergy drives the adaptive adjustment of shelf height and horizontal display surface.

[0145] In summary, steps 401 to 404 enable dynamic adaptive adjustment of product display parameters. By monitoring the ratio of spatial density to visual focus in real time, the system intelligently generates an optimization coefficient for shelf height. When product concentration is too high, an upward movement command is automatically triggered, while when it is too low, a downward movement strategy is initiated. Furthermore, based on a matching analysis of product contact delay data and category characteristics, the system automatically calculates the optimal display orientation and generates a personalized horizontal display plan for each product category. This technical solution innovatively constructs a dual-dimensional optimization mechanism of "vertical density adaptation and horizontal orientation personalization," achieving an intelligent upgrade from static display to dynamic optimization, ensuring that product display is always in optimal condition.

[0146] To address the lack of quantitative standards for the implementation of display optimization solutions, a smart display system with spatial parameterization has been developed. This system optimizes vertical density by converting a first optimization factor into the spacing adjustment between shelf layers. Simultaneously, the second optimization factor is parsed into the movement direction and distance of the horizontal grid, establishing a two-dimensional execution model of "height spacing and horizontal displacement" to achieve a precise conversion of optimization parameters into spatial adjustments. In some embodiments, the steps of adjusting the product distribution density along the shelf height based on the first optimization factor and adjusting the product display orientation on the horizontal display surface based on the second optimization factor in step 105 include:

[0147] 501. Divide the first optimization factor evenly according to the number of layers in the shelf height direction to obtain a unit adjustment amount corresponding to each layer;

[0148] In step 501, the first optimization factor is a comprehensive parameter that reflects the degree to which the product needs to be adjusted in the direction of shelf height. This factor is generated by the product of the cumulative number of times calculated in the previous stage and the number of layers. The number of layers in the direction of shelf height refers to the total number of levels into which the shelf is divided in the vertical dimension, and each layer represents a specific height range. The unit adjustment amount is a value obtained by evenly dividing the first optimization factor to each level, indicating the quantitative index that needs to be adjusted for each specific level. This value contains information on the adjustment direction and amplitude. The equal distribution process ensures that the adjustment amount is reasonably distributed in the direction of shelf height to avoid excessive or insufficient local adjustments.

[0149] In the embodiment of the present application, first, the first optimization factor generated in step 402 is extracted (e.g., the forward factor of 6 and the reverse factor of 3 for region X) and evenly divided according to the number of layers in the shelf height direction (e.g., 3 layers). The calculation formula is: unit adjustment amount = first optimization factor / number of layers (e.g., forward unit adjustment amount = 6 / 3 = 2, reverse unit adjustment amount = 3 / 3 = 1). Second, the results are stored by layer number (e.g., forward adjustment amount +2 for layer 1, reverse adjustment amount -1 for layer 2), and a unit adjustment amount table for each layer is generated to provide the layer density adjustment parameter for step 502.

[0150] 502. Determine the direction of increase or decrease of the spacing between products on the corresponding layer based on the positive or negative sign of the unit adjustment amount, and update the distribution density of the products on the corresponding layer in the height direction of the shelf based on the product of the absolute value of the unit adjustment amount and the preset spacing base value;

[0151] In step 502, the sign of the unit adjustment indicates the direction in which the product spacing needs to be adjusted: a positive value corresponds to an increase in spacing, and a negative value corresponds to a decrease in spacing. The preset spacing base value is a pre-set reference value for standard product spacing and serves as the basis for the adjustment calculation. The product distribution density along the shelf height is updated by multiplying the absolute value of the unit adjustment by the preset spacing base value. This density directly determines the density of product display at a specific level. The updated distribution density guides the vertical rearrangement of products, optimizing space utilization.

[0152] In the embodiment of the present application, first, the direction of the product spacing adjustment is determined according to the positive and negative signs of the unit adjustment amount in step 501 (e.g., +2 means reducing the spacing to increase the density, and -1 means increasing the spacing to reduce the density); secondly, the preset spacing base value is extracted (e.g., the first layer basic spacing is 0.5 meters), and the new spacing is calculated according to the formula: new spacing = basic spacing × (1-absolute value of unit adjustment amount × 0.1) (e.g., adjustment amount + 2 corresponds to new spacing = 0.5 × (1-2 × 0.1) = 0.4 meters, and the density is increased by 25%); finally, the distribution density of the products on each layer is updated (e.g., the density of the first layer is increased from 5 pieces / m 2 Increased to 6.25 pieces / m 2 ), output the updated density configuration table.

[0153] 503. Determine a movement direction identifier of the horizontal display surface according to the positive or negative sign of the second optimization factor, wherein a positive direction identifier indicates movement toward an area with a high density of historical contact positions, and a negative direction identifier indicates movement toward an area with a low density;

[0154] In step 503, the second optimization factor is a comprehensive parameter reflecting the degree of adjustment required for the product on the horizontal display surface. This factor is generated by the proportional relationship between the orientation matching difference and the historical contact location distribution density. The horizontal display surface movement direction identifier is a direction indicator parameter determined by the positive or negative sign of the second optimization factor. A positive identifier points to a favorable area with a high historical contact frequency, while a negative identifier points to a less favorable area with a low contact frequency. Areas with a high historical contact location distribution density represent hot spots where customers habitually interact, while areas with a low historical contact location distribution density represent relatively unpopular locations. This identifier ensures that the product movement direction aligns with customer behavior.

[0155] In the embodiment of the present application, first, based on the positive and negative signs of the second optimization factor in step 404 (e.g., +0.02), the moving direction identifier of the horizontal display surface is determined: positive identifier (+): moving to the area with high density of historical contact positions (e.g., the density of the front end of the food is 5 times / m 2 →Move to this area); Reverse identifier (-): Move to the area with low density (such as the density of daily necessities back end is 2 times / m 2 →Move to the area). Secondly, the identifier is bound to the commodity category (such as the food category identifier "+1") to provide a basis for the moving direction in step 504.

[0156] 504. Convert the absolute value of the second optimization factor to the side length of the area division grid of the horizontal display surface to obtain a horizontal movement distance, and adjust the display position of the product in the corresponding grid according to the movement direction identifier and the horizontal movement distance.

[0157] In step 504, the horizontal display surface's area division grid is a coordinate system that divides the display surface into specific dimensions. The grid side length is the baseline dimension parameter for a single grid cell. The horizontal movement distance is a specific value calculated by converting the absolute value of the second optimization factor into the ratio of the grid side length, and determines the spatial span over which the product must be moved. Display orientation adjustment involves repositioning the product within the grid based on the direction indicated by the movement direction identifier and the specific value calculated using the horizontal movement distance, ultimately achieving the optimal horizontal display layout for the product.

[0158] In the embodiment of the present application, first, according to the moving direction identifier (such as +1) of step 503 and the absolute value of the second optimization factor (such as 0.02) of step 404, the horizontal moving distance is calculated according to the formula: absolute value of factor × grid side length (such as grid side length 1 meter → moving distance = 0.02 × 1 = 0.02 meters); secondly, the product display position is moved according to the direction identifier (such as moving forward +0.02 meters to the front high-density area); finally, the position is updated through the automated shelf adjustment device, and the density after the move is verified (such as the front density of food from 5 times / m 2 Increased to 5.1 times / m 2 ), completing the horizontal display optimization closed loop.

[0159] Here's a specific example:

[0160] During the implementation of the supermarket smart shelf dynamic adjustment system, the system first divides the first optimization factor of 20 equally among the four shelves, obtaining a unit adjustment of +5 (positive) per layer. Based on the preset base value of 0.5cm, it is calculated that the distance between products on each layer should be reduced by 2.5cm (5×0.5), so that the distribution density of shampoo in the third layer with a golden line of sight of 1.2m-1.7m is increased; at the same time, the second optimization factor of +14.4 for the conditioner partition is analyzed to generate an identifier moving to the historically high contact density area on the east side (18 interactions per square meter). According to the 0.3m grid side length, the required movement is 4.32m (14.4×0.3). The system automatically adjusts the category from the original 3B grid to the 5D grid, and updates the electronic price tag position in a linked manner, ultimately achieving vertical density optimization and efficient matching of horizontal orientation, thereby improving the efficiency of human contact at the conditioner display.

[0161] In summary, steps 501 to 504 enable refined dynamic control of product display parameters. By decomposing optimization factors into executable instructions, the system intelligently adjusts the distribution density of products on each layer in the vertical dimension, automatically increasing or decreasing the distance between layers based on real-time data analysis. Horizontally, based on product contact heat maps, it precisely calculates the optimal display position and movement distance, enabling intelligent displacement of products on the display surface. This technical solution innovatively constructs a dual-dimensional execution mechanism of "vertical layered density adjustment and horizontal grid-based displacement." By converting abstract optimization parameters into specific spatial adjustment instructions, it achieves a seamless transition from data analysis to physical display, ensuring that products are always displayed in the optimal location.

[0162] To address the quantitative correlation between contact delay analysis and product display orientation optimization, an intelligent optimization system based on dynamic delay interval partitioning has been developed. This system constructs equally divided evaluation intervals based on the allowable delay ranges of preset product categories. The directional difference is calculated based on the interval positioning of the actual delay value and converted into a standardized orientation matching difference, achieving a precise quantitative conversion from contact behavior efficiency to display orientation adjustment. In some embodiments, step 403 compares the contact behavior delay value with the standard delay range of the association rule corresponding to the preset product category, and generates orientation matching differences for similar products based on the degree to which the contact behavior delay value exceeds or falls below the standard delay range, including:

[0163] 601. Extract the maximum allowable delay value and the minimum allowable delay value defined in the preset product category, and divide the difference interval between the maximum allowable delay value and the minimum allowable delay value into at least three equal intervals;

[0164] In step 601, the preset product category refers to a classification system pre-divided according to product characteristics and sales strategies. The maximum allowable delay value is the upper threshold of the acceptable contact time set for each category of products. Exceeding this value indicates that the product is poorly accessible. The minimum allowable delay value is the lower threshold of the acceptable contact time set for each category of products. Below this value indicates that the product is too accessible and may affect the display of other products. The difference interval is the time range interval obtained by subtracting the minimum allowable delay value from the maximum allowable delay value. This interval is equally divided into at least three sub-intervals, each sub-interval representing a different level of contact behavior delay performance.

[0165] In an embodiment of the present application, the maximum allowable delay value and the minimum allowable delay value defined in the preset product category are first extracted, and the difference interval between the maximum allowable delay value and the minimum allowable delay value is divided into at least three equal intervals. This process provides a basis for the subsequent accurate calculation of the position of the contact behavior delay value by carefully dividing the delay value interval. Next, the width of each equal partition is determined to ensure that subsequent calculations can accurately reflect the positional relationship of the contact behavior delay value in different intervals. Finally, by carefully dividing the delay value interval, more accurate data support is provided for optimizing the product display orientation.

[0166] 602. Mark the contact behavior delay value according to the difference interval. If the contact behavior delay value is in the right interval of the maximum allowable delay value, calculate the excess of the contact behavior delay value over the maximum allowable delay value as the positive difference. If the contact behavior delay value is in the left interval of the minimum allowable delay value, calculate the shortage of the contact behavior delay value from the minimum allowable delay value as the negative difference.

[0167] In step 602, the contact behavior delay value is the actual measured time required for a customer to enter the shelf partition and contact the product. Position marking refers to the process of classifying and marking the contact behavior delay value according to its specific position in the difference interval. The positive difference value is the numerical value of the excess calculated when the contact behavior delay value exceeds the maximum allowable delay value, reflecting the extent to which the accessibility of the product is lower than the expected standard. The negative difference value is the numerical value of the shortfall calculated when the contact behavior delay value is lower than the minimum allowable delay value, reflecting the extent to which the accessibility of the product is higher than the expected standard. The right interval refers to the range in the difference interval that is greater than the maximum allowable delay value, and the left interval refers to the range in the difference interval that is less than the minimum allowable delay value.

[0168] In an embodiment of the present application, the contact behavior delay value is firstly marked according to the difference interval, and the position of each contact behavior delay value relative to the maximum and minimum delay values ​​allowed is specifically analyzed. If the contact behavior delay value is located in the right interval of the maximum delay value allowed, it indicates that the customer comes into contact with the product significantly later than the ideal situation. In this case, the specific amount by which it exceeds the maximum delay value allowed is calculated as the positive difference amount; if the contact behavior delay value is located in the left interval of the minimum delay value allowed, it indicates that the customer comes into contact with the product too early or too quickly. In this case, the specific amount by which it is less than the minimum delay value allowed is calculated as the reverse difference amount. Then, during the calculation process, it is ensured that each contact behavior delay value can be accurately classified into the corresponding interval, and the corresponding difference amount is calculated accordingly. Secondly, through this precise difference amount calculation method, it is possible to more clearly identify which products need to be adjusted in their display position in priority to improve the customer experience.

[0169] 603. Calculate the ratio of the forward difference or the reverse difference to the width of the equal partitions to generate an orientation matching difference.

[0170] In step 603, the width of the equally spaced intervals refers to the time span of each subinterval, calculated by dividing the total difference interval by the number of partitions. The orientation matching difference is a standardized parameter generated by calculating the ratio of the positive or negative difference to the width of the equally spaced interval. This parameter quantifies the degree to which the actual contact delay deviates from the expected standard. The ratio calculation process converts the absolute time difference into a relative ratio, facilitating horizontal comparison and unified evaluation across different product categories. The resulting orientation matching difference will serve as an important parameter for subsequent display optimization and adjustment.

[0171] In the embodiment of the present application, first, the maximum allowable delay value (e.g., the maximum allowable delay for food is 2.5 seconds) and the minimum allowable delay value (e.g., 1.5 seconds) are extracted from the association rules of the preset commodity categories, and the difference interval (2.5-1.5=1.0 seconds) is calculated; secondly, the difference interval is divided into three equal intervals (each equal interval is ≈0.33 seconds), and the division results are as follows: left interval: 1.5 seconds ≤ delay value <1.83 seconds; middle interval: 1.83 seconds ≤ delay value ≤2.16 seconds; right interval: 2.16 seconds < delay value ≤2.5 seconds. Finally, the range of the equal intervals is output to provide a delay value position determination benchmark for step 602.

[0172] Here's a specific example:

[0173] In the delay analysis module of the supermarket smart shelf optimization system, the system first extracts the preset delay threshold for the shampoo category (the minimum allowed delay value is 3 seconds and the maximum delay value is 6 seconds), and divides the 3-second difference interval into three equal intervals (3-4 seconds, 4-5 seconds, and 5-6 seconds). When the actual contact behavior delay value of a certain partition is detected to be 7.2 seconds, the system determines that it is in the interval to the right of the maximum allowed value of 6 seconds, calculates the positive difference of 1.2 seconds (7.2-6=1.2), and then compares it with the width of the equal interval of 1 second ((6-3) / 3=1 ) is calculated to generate an azimuth matching difference of +1.2 (1.2 / 1). This value will serve as a quantitative basis for subsequent horizontal display adjustments. When the difference is positive, the optimization strategy of moving to the high-traffic area on the east side is triggered. At the same time, when the system detects a delay value of 2.8 seconds in another partition (to the left of the minimum allowable value), a reverse difference of -0.2 (3-2.8=0.2) is generated and an azimuth matching difference of -0.2 is calculated, automatically starting a defensive displacement adjustment to the low-density area on the west side. Through this two-way difference mechanism, precise spatial adaptation of product display is achieved.

[0174] In summary, through steps 601 to 603, intelligent hierarchical evaluation and precise control of product display delay parameters are achieved. By establishing a delay tolerance interval model for product categories, the system conducts multi-level comparison analysis of actual contact delay data with preset standards to accurately quantify the degree of delay deviation. When it is detected that the delay exceeds the maximum tolerance threshold, the system automatically calculates the positive deviation and generates a display optimization plan; when the delay is lower than the minimum expected value, the negative deviation is calculated and corresponding adjustments are triggered. This technical solution innovatively constructs a dynamic evaluation mechanism based on delay tolerance intervals. By converting abstract delay data into actionable orientation matching differences, it achieves precise quantification and intelligent response of product display sensitivity, ensuring that all types of products are always in the optimal display state.

[0175] To accurately identify the dynamic interactions between customers and products, we have developed an intelligent video feature extraction system. Using a three-step analysis method consisting of moving object segmentation, displacement trajectory verification, and human motion correlation, we perform spatiotemporal matching of product movement paths and triggering behaviors, constructing a dynamic feature set encompassing causal relationships and enabling reconstruction of the complete event chain of the interaction. In some embodiments, step 102 involves performing image recognition processing on the continuous video stream captured by the monitoring device to extract a dynamic feature set encompassing product displacement trajectories and human motion patterns, including:

[0176] 701. Perform moving object segmentation on the difference areas of adjacent frames in the continuous video stream collected by the monitoring device, mark the areas of the segmented moving objects that meet the product size threshold as candidate product areas, and mark the remaining areas as candidate human body areas;

[0177] In step 701, the continuous video stream collected by the monitoring device refers to real-time monitoring video data obtained by a fixed camera, which contains continuous image frames arranged in chronological order. The difference area between adjacent frames is the contour area of ​​the moving object detected by comparing the pixel changes between the two frames. Moving object segmentation is the process of separating the moving object from the static background using a computer vision algorithm. The product size threshold is a pre-set pixel range standard corresponding to the physical size of the product, which is used to screen moving areas that meet the size of the product. The candidate product area is the moving object area that is preliminarily identified as a possible product after size screening. The human candidate area is the remaining moving object area that is identified as a possible human body in addition to the candidate product area.

[0178] In this embodiment, a background subtraction algorithm (e.g., the ViBe algorithm) is first applied to the difference regions of adjacent frames in the continuous video stream captured by the monitoring device to segment the contours of moving objects. Secondly, based on a preset product size threshold (e.g., a length and width range of [10cm, 50cm]), morphological screening (e.g., open operation denoising) is performed to mark moving objects that meet the size criteria as candidate product regions (e.g., product movement regions on a shelf). The remaining regions (e.g., a customer's torso or arm) are marked as candidate human regions. Finally, the candidate region classification results (e.g., {"frame t1":{"candidate product region":[rectangular frame coordinates],"candidate human region":[key point coordinates]}}) are output to provide input for subsequent steps.

[0179] 702. Perform a directional continuity check on the position coordinates of the candidate product area in consecutive frames. If the coordinate change direction and movement direction trend of the same product in adjacent frames are consistent, record the movement path points and connect them to form a product displacement trajectory.

[0180] In step 702, directional continuity verification involves logically verifying the motion trajectory of the candidate product area across consecutive video frames. The coordinate change direction refers to the azimuth angle of movement of the product area's center point across consecutive frames. The movement trend is the primary direction of the motion path derived by analyzing the product's position changes across multiple consecutive frames. A movement path point is a data point recording the specific coordinate location of the product within each frame. The product's displacement trajectory is a complete motion path line formed by connecting each movement path point, reflecting the actual movement of the product in space.

[0181] In the embodiment of the present application, first, the position coordinates of the candidate product area in consecutive frames (such as the coordinates (x1, y1) of frame t1 and the coordinates (x2, y2) of frame t2) are checked for directional continuity: Direction trend calculation: The moving direction angle is calculated by the difference in coordinates of adjacent frames (such as θ = arctan ((y2-y1) / (x2-x1))); Continuity judgment: If the change in the direction angle within three consecutive frames is less than a threshold (such as ±10°), it is judged to be consistent in direction; secondly, the coordinate points that pass the verification are connected into a product displacement trajectory (such as {"Product A":"Trajectory points [(x1, y1, t1), (x2, y2, t2)]"}), and the jitter noise is filtered. Finally, the trajectory data table is output to provide an event association basis for step 704.

[0182] 703. Positionally lock the trunk and limb key points in the candidate human body region, generate a posture change sequence based on the position change directions of the key points in adjacent frames, and mark the continuous direction combination in the posture change sequence that conforms to the preset action form as a human body action form;

[0183] In step 703, the trunk and limb key points are the coordinates of the main body joints identified by the human posture estimation algorithm. Position locking refers to the process of continuously tracking the coordinates of specific key points in consecutive frames. The posture change sequence is a collection of key point position change data arranged in chronological order. The preset action form is a library of predefined standard human action patterns that contains directional combination features of typical shopping actions. The human action form is a specific behavior pattern identified by matching the posture change sequence with the preset action form.

[0184] In the embodiment of the present application, first, the OpenPose key point detection algorithm is used to lock the trunk and limb key points (such as the coordinates of the shoulder, elbow, and wrist joints) of the candidate human body area; secondly, the displacement direction of the key points in adjacent frames is calculated (such as the wrist direction angle θ = 30° from (x1, y1) to (x2, y2)), and a posture change sequence is generated (such as ["extending hand: θ = 30°", "retracting hand: θ = 210°"]); then, the dynamic time warping algorithm (DTW) is used to match the preset action form (such as the standard sequence of the "picking up" action). If the matching similarity is greater than 90%, it is marked as a valid action form. Finally, a list of human action forms is output (such as {"Customer A":"Action form: picking up"}) to provide a behavior event label for step 704.

[0185] 704. Perform event association on the starting position of the product displacement trajectory within the same time period and the corresponding human motion form, and combine them to form a dynamic feature set including the product displacement trajectory and the human motion form.

[0186] In step 704, event correlation is the process of causally matching the product's displacement trajectory with the human motion that triggered it. The starting position refers to the initial coordinate point at which the product began to move. The dynamic feature set is a composite data set that integrates the physical trajectory of the product's displacement and the associated human motion and behavioral characteristics. This set fully records the spatiotemporal dynamics of the "human-object" interaction process. Merging is the process of aligning the two types of feature data by timestamp and establishing a corresponding relationship.

[0187] In the embodiment of the present application, first, the starting time of the product displacement trajectory in step 702 (such as product A starts to move at t1) and the occurrence time of the human action form in step 703 (such as customer A starts the "take the object" action at t1) are aligned based on the unified time axis; secondly, the trajectory and action within the same time window (such as t1 to t1+3 seconds) are merged through the event association engine to generate a dynamic feature set (such as {"t1":{"Product A displacement trajectory":[coordinate sequence],"Customer A action":"take the object"}}); finally, the dynamic feature set with spatiotemporal labels is output to provide multi-dimensional data support for display effect analysis.

[0188] Here's a specific example:

[0189] In the real-time analysis process of the intelligent retail monitoring system, the system first performs inter-frame difference processing on the continuous video stream captured by the camera, and segments the moving objects through the background subtraction algorithm. The area with a length and width of 5-50 cm is marked as a candidate product (such as a shampoo bottle on the shelf), and the rest of the area is marked as a candidate human body area. Then, the motion trajectory of the candidate product area is tracked. When a bottled beverage is detected to keep moving to the lower right (coordinate change Δx>0 and Δy>0) in 5 consecutive frames (0.2 second intervals), the system records its moving path points and generates a complete product displacement trajectory. At the same time, OpenPos The algorithm detects 17 key points in the candidate human body area. When it identifies the action sequence of "right elbow angle decreases → wrist extends → fingers close" in three consecutive frames, it marks it as a "grasping action." Finally, the system spatially and temporally correlates the displacement trajectory of the beverage bottle in shelf H3 (starting point coordinates X = 2.1m, Y = 1.4m) during the period 14:05:23 with customer E's grasping action, forming a dynamic feature set that includes the product movement path and the customer's grasping posture. This feature set shows that customer E completed the entire interaction process from looking at the product to picking up the product in 0.8 seconds, providing a basis for fine-grained behavioral analysis for subsequent display optimization.

[0190] In summary, steps 701 to 704 achieve accurate identification of products and customer behavior, as well as dynamic feature extraction. Using intelligent video analysis technology, the system first precisely segments moving objects in the surveillance footage, automatically distinguishing between product areas and human areas. It then constructs product displacement trajectories based on motion continuity analysis, while simultaneously capturing customer motion characteristics through human posture recognition. Finally, it spatially and temporally correlates product movement paths with customer actions, forming a complete interactive behavioral feature map. This technical solution innovatively integrates computer vision and behavioral analysis algorithms, achieving intelligent conversion from raw video streams to structured behavioral features, providing highly accurate behavioral data support for subsequent display optimization decisions.

[0191] Figure 2 The present invention provides a schematic diagram of a retail data processing system for evaluating retail product displays. Figure 2 As shown, the system includes:

[0192] Acquisition module 21, which acquires a data set of customer-product interaction behaviors through monitoring devices installed in the product display area, and simultaneously acquires three-dimensional spatial distribution data of the products displayed on the shelves;

[0193] The processing module 22 performs image recognition processing on the continuous video stream collected by the monitoring device to extract a dynamic feature set including the displacement trajectory of the product and the movement form of the human body;

[0194] A generation module 23 performs spatiotemporal correlation matching on the interaction behavior dataset and the three-dimensional spatial distribution data, and generates a display effect evaluation parameter set including a spatial occupancy density index, a visual focus distribution rate, and a contact behavior delay value based on the matching results and the change parameters corresponding to the product displacement trajectory in the dynamic feature set;

[0195] A determination module 24 determines a first optimization factor based on a ratio of a space occupancy density index to the visual focus distribution rate in the display effect evaluation parameter group, and determines a second optimization factor based on an association rule between the contact behavior delay value and a preset product category;

[0196] The adjustment module 25 adjusts the distribution density of the products in the height direction of the shelf based on the first optimization factor, and adjusts the display orientation of the products on the horizontal display surface based on the second optimization factor.

[0197] Figure 2 The retail data processing system for retail product display evaluation can execute Figure 1The implementation principles and technical effects of the retail data processing method for retail product display evaluation described in the illustrated embodiment are not further elaborated. The specific manner in which the various modules and units in the retail data processing system for retail product display evaluation in the aforementioned embodiment perform operations has been described in detail in the related embodiments of the method and will not be further elaborated here.

[0198] In one possible design, Figure 2 A retail data processing system for retail merchandise display evaluation of the illustrated embodiment may be implemented as a computing device, such as Figure 3 As shown, the computing device may include a storage component 31 and a processing component 32;

[0199] The storage component 31 stores one or more computer instructions, wherein the one or more computer instructions are called and executed by the processing component 32 .

[0200] The processing component 32 is used for the above Figure 1 The embodiment provides a retail data processing method for retail commodity display evaluation.

[0201] The processing component 32 may include one or more processors to execute computer instructions to perform all or part of the steps in the above method. Of course, the processing component may also be implemented as one or more application-specific integrated circuits (ASICs), digital signal processors (DSPs), digital signal processing devices (DSPDs), programmable logic devices (PLDs), field programmable gate arrays (FPGAs), controllers, microcontrollers, microprocessors, or other electronic components to perform the above method.

[0202] The storage component 31 is configured to store various types of data to support operations at the terminal. The storage component can be implemented by any type of volatile or non-volatile memory device, or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic memory, flash memory, magnetic disk, or optical disk.

[0203] Of course, a computing device may also include other components, such as input / output interfaces, display components, communication components, etc.

[0204] The input / output interface provides an interface between the processing component and the peripheral interface module, which can be an output device, an input device, etc.

[0205] The communication component is configured to facilitate, among other things, wired or wireless communications between the computing device and other devices.

[0206] Among them, the computing device can be a physical device or an elastic computing host provided by a cloud computing platform, etc. In this case, the computing device can refer to a cloud server, and the above-mentioned processing components, storage components, etc. can be basic server resources rented or purchased from the cloud computing platform.

[0207] The present application also provides a computer storage medium storing a computer program, wherein the computer program can achieve the above-mentioned Figure 1 XX method of the illustrated embodiment.

[0208] Those skilled in the art will clearly understand that, for the convenience and brevity of description, the specific working processes of the systems, devices and units described above can refer to the corresponding processes in the aforementioned method embodiments and will not be repeated here.

[0209] The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate, and the components shown as units may or may not be physical units, i.e., they may be located in one location or distributed across multiple network units. Some or all of the modules may be selected based on actual needs to achieve the objectives of the present embodiment. Persons of ordinary skill in the art will be able to understand and implement the present invention without inventive effort.

[0210] Through the description of the above embodiments, those skilled in the art can clearly understand that each embodiment can be implemented by means of software plus a necessary general hardware platform, or of course, by hardware. Based on this understanding, the essence of the above technical solution or the part that contributes to the existing technology can be embodied in the form of a software product. The computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, a magnetic disk, an optical disk, etc., and includes a number of instructions for enabling a computer device (which can be a personal computer, a server, or a network device, etc.) to execute the methods described in each embodiment or certain parts of the embodiments.

[0211] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present application, rather than to limit them. Although the present application has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present application.

Claims

1. A retail data processing method for retail product display evaluation, characterized in that: include: The monitoring devices installed in the product display area are used to obtain the interaction behavior data set between customers and products, and simultaneously obtain the three-dimensional spatial distribution data of the products displayed on the shelves; Performing image recognition processing on the continuous video stream collected by the monitoring device to extract a dynamic feature set including the displacement trajectory of the product and the movement form of the human body; Performing spatiotemporal correlation matching on the interactive behavior dataset and the three-dimensional spatial distribution data, and generating a display effect evaluation parameter group including a spatial occupancy density index, a visual focus distribution rate, and a contact behavior delay value based on the matching results and the change parameters corresponding to the product displacement trajectory in the dynamic feature set; Determining a first optimization factor based on a ratio between a space occupancy density index and a visual focus distribution rate in the display effect evaluation parameter group, and determining a second optimization factor based on an association rule between the contact behavior delay value and a preset product category; Adjusting the distribution density of the products in the height direction of the shelf based on the first optimization factor, and adjusting the display orientation of the products on the horizontal display surface based on the second optimization factor; Performing spatiotemporal correlation matching on the interactive behavior dataset and the three-dimensional spatial distribution data, and generating a display effect evaluation parameter group including a space occupancy density index, a visual focus distribution rate, and a contact behavior delay value based on the matching results and the change parameters corresponding to the product displacement trajectory in the dynamic feature set, including: Performing a temporal-spatial correspondence between the time stamps of the customer's behavior events of contacting the product in the interactive behavior dataset and the coordinates of the product display locations in the three-dimensional spatial distribution data, so as to establish a correspondence table between the customer's location and the product area with the time stamp as a matching result; Based on the total number of times the products were moved in the matching results and the length of the customer's stay period corresponding to each move, the product of the total number of times and the average length of the stay period is divided by the space occupied by the corresponding shelf partition to generate a space occupancy density index representing the degree of product aggregation; Extracting a continuous image group in which the customer's head direction continuously changes from the human motion form of the dynamic feature set, and combining the customer's stay period length in the matching result to generate a visual focus distribution rate representing the visual attention intensity; Based on the time difference between the first behavioral event time stamp of the customer entering the shelf partition and the start time of the product displacement trajectory in the matching result, the median value of the first contact time difference of all customers in the same partition is calculated to generate a contact behavior delay value reflecting the accessibility efficiency of the product; The space occupancy density index, the visual focus distribution rate and the contact behavior delay value are combined and associated according to three-dimensional space coordinates to form a display effect evaluation parameter group.

2. The method according to claim 1, characterized in that The time stamps of the customer's behavior events of contacting the product in the interactive behavior dataset are temporally and spatially associated with the product display location coordinates in the three-dimensional spatial distribution data, and a correspondence table between the customer location and the product area with the time stamp is established as a matching result, including: Extracting the time stamp of the customer's behavior event of contacting the product from the interactive behavior dataset, and obtaining the coordinates of the customer's product display location from the three-dimensional spatial distribution data, and unifying the behavior event time stamp and the product display location coordinates to the same time axis based on the system time axis; Converting the coordinates of the product display location according to the coordinate system of the three-dimensional spatial distribution data to determine the shelf partition identifier of the customer at each time mark, wherein the shelf partition identifier is generated by combining the number of layers, the number of columns, and the depth distance of the product display location; Based on the shelf partition identification, generating an initial record table including a customer identifier, a shelf partition identification and a product displacement status; When the start time of the movement of the coordinates of the product display position is monitored, the time mark of the customer's behavior event of contacting the product under the same time axis is retrieved from the initial record table, and the change relationship between the product movement process duration and the product shelf partition mark in the time mark of the customer's behavior event of contacting the product is synchronously appended to the initial record table to form a correspondence table between the customer position and the product area with a time mark as a matching result.

3. The method according to claim 1, characterized in that The first optimization factor is determined based on the ratio of the space occupancy density index in the display effect evaluation parameter group to the visual focus distribution rate, and the second optimization factor is determined based on the association rule between the contact behavior delay value and the preset product category, including: Calculating a real-time ratio of the space occupancy density index to the visual focus distribution rate; when the real-time ratio is greater than a preset high ratio range, converting the real-time ratio into a positive adjustment coefficient in the direction of shelf height; and when the real-time ratio is less than a preset low ratio range, converting the real-time ratio into a negative adjustment coefficient in the direction of shelf height; Generate a first optimization factor according to the product relationship between the cumulative number of the positive adjustment coefficient or the reverse adjustment coefficient in a continuous time period and the number of layers in the shelf height direction; Comparing the contact behavior delay value with a preset association rule standard delay range corresponding to a product category, and generating a position matching difference value for similar products based on the degree to which the contact behavior delay value exceeds or falls below the standard delay range; A second optimization factor is generated based on a proportional relationship between the orientation matching difference and the historical contact position distribution density of the preset commodity category on the horizontal display surface.

4. The method according to claim 1, wherein Adjusting the distribution density of the products in the height direction of the shelf based on the first optimization factor, and adjusting the display orientation of the products on the horizontal display surface based on the second optimization factor, including: Divide the first optimization factor equally according to the number of layers in the shelf height direction to obtain a unit adjustment amount corresponding to each layer; Determine the direction of increase or decrease of the spacing between commodities on the corresponding layer according to the positive or negative sign of the unit adjustment amount, and update the distribution density of commodities on the corresponding layer in the height direction of the shelf according to the product of the absolute value of the unit adjustment amount and the preset spacing base value; Determine a movement direction identifier of the horizontal display surface according to the positive or negative sign of the second optimization factor, wherein a positive identifier indicates movement toward an area with a high density of historical contact positions, and a negative identifier indicates movement toward an area with a low density; The absolute value of the second optimization factor is proportionally converted to the side length of the area division grid of the horizontal display surface to obtain a horizontal movement distance, and the display orientation of the product in the corresponding grid is adjusted according to the movement direction identifier and the horizontal movement distance.

5. The method according to claim 3, characterized in that Comparing the contact behavior delay value with a preset association rule standard delay range corresponding to a product category, and generating a position matching difference value for similar products based on the degree to which the contact behavior delay value exceeds or falls below the standard delay range, including: Extracting the maximum allowable delay value and the minimum allowable delay value defined in the preset product category, and dividing the difference interval between the maximum allowable delay value and the minimum allowable delay value into at least three equal intervals; Marking the contact behavior delay value according to the difference interval; if the contact behavior delay value is in the right interval of the maximum allowable delay value, calculating the excess of the contact behavior delay value over the maximum allowable delay value as a positive difference; if the contact behavior delay value is in the left interval of the minimum allowable delay value, calculating the shortage of the contact behavior delay value from the minimum allowable delay value as a negative difference; The positive difference or the negative difference is calculated as a ratio to the width of the equal partitions to generate an orientation matching difference.

6. The method according to claim 1, characterized in that Performing image recognition processing on the continuous video stream collected by the monitoring device to extract a dynamic feature set including the product displacement trajectory and human body movement form, including: Performing moving object segmentation on the difference areas of adjacent frames in the continuous video stream collected by the monitoring device, marking the areas of the segmented moving objects that meet the product size threshold as candidate product areas, and marking the remaining areas as candidate human body areas; Performing a directional continuity check on the position coordinates of the candidate product area in consecutive frames. If the coordinate change direction and movement direction trend of the same product in adjacent frames are consistent, then record its movement path points and connect them to form a product displacement trajectory. Position locking is performed on the trunk and limb key points in the candidate human body region, a posture change sequence is generated according to the position change direction of the key points in adjacent frames, and a continuous direction combination in the posture change sequence that conforms to a preset action form is marked as a human body action form; The starting position of the commodity displacement trajectory within the same time period is associated with the corresponding human motion form, and the dynamic feature set including the commodity displacement trajectory and the human motion form is formed.

7. A retail data processing system for retail product display evaluation, configured to execute the retail data processing method for retail product display evaluation according to any one of claims 1 to 6, characterized in that: include: The acquisition module acquires the interactive behavior data set between customers and products through the monitoring device installed in the product display area, and simultaneously acquires the three-dimensional spatial distribution data of the products displayed on the shelves; a processing module that performs image recognition processing on the continuous video stream collected by the monitoring device to extract a dynamic feature set including the displacement trajectory of the product and the movement form of the human body; a generation module that performs spatiotemporal correlation matching on the interactive behavior dataset and the three-dimensional spatial distribution data, and generates a display effect evaluation parameter group including a spatial occupancy density index, a visual focus distribution rate, and a contact behavior delay value based on the matching results and the change parameters corresponding to the product displacement trajectory in the dynamic feature set; a determination module for determining a first optimization factor based on a ratio of a space occupancy density index to the visual focus distribution rate in the display effect evaluation parameter group, and determining a second optimization factor based on an association rule between the contact behavior delay value and a preset product category; An adjustment module adjusts the distribution density of the goods in the height direction of the shelf based on the first optimization factor, and adjusts the display orientation of the goods on the horizontal display surface based on the second optimization factor.

8. A computing device, characterized in that The method comprises a processing component and a storage component; the storage component stores one or more computer instructions; the one or more computer instructions are used to be called and executed by the processing component to implement a retail data processing method for retail product display evaluation as described in any one of claims 1 to 6.

9. A computer storage medium, characterized in that A computer program is stored, and when the computer program is executed by a computer, the retail data processing method for retail commodity display evaluation according to any one of claims 1 to 6 is implemented.

Images