Commodity interaction behavior analysis method and device, equipment and storage medium

Abstract

The invention discloses a commodity interaction behavior analysis method and device, equipment and a storage medium. In the scheme, when the commodity interaction behaviors are identified, the commodity interaction behaviors such as a commodity touch behavior and a commodity deep experience behavior need to be identified according to the region overlapping condition of a user hand region and each commodity region detected in a video picture and the position change condition of each commodity region; moreover, for the behavior analysis result of each commodity, commodity interaction data taking the SKU information as the dimension can be generated. Visibly, accurate analysis of the commodity touch behavior and the commodity deep experience behavior can be realized only through the video picture, the identification accuracy of the commodity interaction behavior is improved, the deployment cost is reduced, and the expandability is improved; moreover, the commodity interaction data of the SKU level can be subjected to refined statistics, the data acquisition capability of refined behaviors is realized, and the commodity interaction data of the SKU level can effectively support refined marketing and operation decisions.

Description

Technical Field

[0001] This application relates to the field of interactive behavior analysis, and in particular to a method, apparatus, device, and storage medium for analyzing product interactive behavior. Background Technology

[0002] In existing retail scenarios, consumer-product interactions refer to the behaviors consumers engage with, explore, and exchange information with products during the purchase decision-making process. These behaviors are crucial steps in consumers' perception and evaluation of products, ultimately leading to their purchase decision.

[0003] Traditional solutions can identify consumer stopping or approaching behavior, but cannot distinguish between looking and touching behavior, leading to an underestimation of the actual level of product interaction and affecting the accuracy of display effect evaluation. Furthermore, traditional solutions only count foot traffic on a regional basis, making it difficult to obtain precise interaction data for specific products, such as the frequency of touch and the number of times a product is picked up, which cannot support single-product-level marketing decisions. If more refined interaction data is required, it is necessary to rely on solutions using dedicated sensors or RFID (Radio Frequency Identification) tags, which are costly and difficult to scale up in ordinary shelf environments.

[0004] Therefore, how to accurately and effectively identify the interaction behavior between consumers and products, and reduce deployment costs and difficulties, is a problem that needs to be solved by those skilled in the art. Summary of the Invention

[0005] This application provides a method, apparatus, device, and storage medium for analyzing product interaction behavior, so as to accurately and effectively identify the interaction behavior between consumers and products, and reduce deployment costs and difficulties.

[0006] Firstly, this application provides a method for analyzing product interaction behavior, including:

[0007] Acquire the video frame to be tested;

[0008] The user's hand area and the product areas of each item are detected from the video footage;

[0009] Based on the overlap between the hand area and each product area, as well as the positional changes of each product area, product interaction behaviors are identified; wherein, the product interaction behaviors include at least one of product touch behavior and product depth experience behavior;

[0010] Analyze the interaction behavior of each product to generate product interaction data based on the product's SKU information.

[0011] Optionally, the step of identifying product interaction behavior based on the overlap between the hand area and each product area, and the positional changes of each product area, includes:

[0012] If the hand area overlaps with the product area, then it is detected whether the overlap time between the hand area and the product area exceeds a predetermined threshold.

[0013] If the overlap time does not exceed the predetermined threshold, it is determined that no product interaction behavior has been identified; if the overlap time exceeds the predetermined threshold, it is determined that product touch behavior has been identified.

[0014] Determine whether the position of the product area overlapping with the hand area has changed;

[0015] If the position changes and the hand posture is in the predetermined posture, then the product depth experience behavior is identified.

[0016] Optionally, the interaction behavior of each product is analyzed to generate product interaction data based on the product's SKU information, including:

[0017] Determine the product information corresponding to each product interaction behavior, and use the product information to determine the corresponding SKU information;

[0018] The interaction behavior of products belonging to the same SKU is analyzed and statistically analyzed to generate product interaction data corresponding to each SKU; wherein, the product interaction data includes: number of touches, touch rate, in-depth experience time and conversion rate.

[0019] Optionally, after generating product interaction data corresponding to each SKU, the process also includes:

[0020] Identify abnormal SKUs based on product interaction data and product sales data for each SKU;

[0021] An early warning message is generated based on the abnormal SKU.

[0022] Optionally, determining the product information corresponding to each product interaction behavior includes:

[0023] Determine the first image and the second image corresponding to the product interaction behavior; wherein, the first image is an image of a predetermined number of consecutive frames before the product interaction behavior occurs, and the second image is an image of a predetermined number of consecutive frames after the product interaction behavior occurs;

[0024] The hand movement trajectory of the product interaction behavior is determined by the second image, and the hand movement trajectory is associated with each product area in the first image to determine the associated product area;

[0025] The corresponding product information is determined by identifying the associated product areas.

[0026] Optionally, after generating product interaction data based on SKU information, the process also includes:

[0027] Using the product interaction data of each SKU information in the target shelf, the product touch heat corresponding to different SKU information in the target shelf is determined and displayed through a touch heat map;

[0028] Based on the number of touches on each SKU in the target shelf, filter the target SKU information and display the product interaction data of the target SKU information through an interaction data list.

[0029] Optionally, after identifying product interaction behavior, the process also includes:

[0030] The system collects and analyzes product interaction behavior in different analysis areas to generate product interaction data based on the analysis area. The analysis area includes: display area or shelf area.

[0031] Acquire passenger flow data for each analysis area; wherein, the passenger flow data is generated after passenger flow identification in the passenger flow detection area of ​​each analysis area;

[0032] The system displays product interaction data, customer flow data, and sales data from different analysis areas.

[0033] Secondly, this application provides an analysis device for product interaction behavior, comprising:

[0034] The acquisition module is used to acquire the video frame to be detected;

[0035] The detection module is used to detect the user's hand area and the product area of ​​each item from the video frame;

[0036] The recognition module is used to identify product interaction behaviors based on the overlap between the hand area and each product area, as well as the positional changes of each product area; wherein, the product interaction behaviors include at least one of product touch behavior and product depth experience behavior;

[0037] The analysis module is used to analyze the interaction behavior of each product and generate product interaction data based on the product's SKU information.

[0038] Thirdly, this application provides an electronic device, comprising:

[0039] Memory, used to store computer programs;

[0040] A processor is used to implement the steps of the above-described analysis method when executing the computer program.

[0041] Fourthly, this application also provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements the steps of the above-described analysis method.

[0042] Compared with the prior art, the technical solution provided in this application has the following advantages: This application discloses a method, apparatus, device, and storage medium for analyzing product interaction behavior. In this solution, when identifying product interaction behavior, it is necessary to identify product touch behavior and product depth experience behavior based on the overlap between the user's hand area and each product area detected in the video frame, as well as the positional changes of each product area. Furthermore, based on the behavior analysis results of each product, product interaction data can be generated using SKU information as the dimension. Therefore, this application can achieve accurate analysis of product touch behavior and product depth experience behavior solely through video frames, improving the accuracy of product interaction behavior identification, reducing deployment costs, and enhancing scalability. Moreover, this application can also perform refined statistical analysis of SKU-level product interaction data, achieving refined behavior data collection capabilities. SKU-level product interaction data can effectively support refined marketing and operational decisions. Attached Figure Description

[0043] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments consistent with the invention and, together with the description, serve to explain the principles of the invention.

[0044] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, for those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0045] One or more embodiments are illustrated by way of example with reference numerals in the accompanying drawings. These illustrations do not constitute a limitation on the embodiments. Elements with the same reference numerals in the drawings are denoted as similar elements. Unless otherwise stated, the figures in the drawings are not to be limited by scale.

[0046] Figure 1 A schematic flowchart illustrating a product interaction behavior analysis method provided in this application embodiment;

[0047] Figure 2 This is a schematic diagram of a data display page provided in an embodiment of this application;

[0048] Figure 3 This is a schematic diagram of the region division provided for an embodiment of this application;

[0049] Figure 4 This application provides a schematic diagram of passenger flow display based on a floor plan.

[0050] Figure 5 This is a schematic diagram of customer attributes within a store area provided in an embodiment of this application;

[0051] Figure 6 This is a schematic diagram illustrating the AI ​​capability configuration provided in an embodiment of this application.

[0052] Figure 7 This is a schematic diagram of the area settings provided in an embodiment of this application;

[0053] Figure 8 Add a page illustration to the area provided in the embodiments of this application;

[0054] Figure 9 This is a schematic diagram of the user interface for analyzing the user experience based on the touch heat of the user, provided in an embodiment of this application.

[0055] Figure 10 Complete passenger flow trajectory data provided for embodiments of this application;

[0056] Figure 11 This is a schematic diagram illustrating the playback video interface provided in an embodiment of this application;

[0057] Figure 12 This is a schematic diagram of the precise passenger flow area display interface provided in the embodiments of this application;

[0058] Figure 13 A schematic diagram of a device for analyzing product interaction behavior provided in an embodiment of this application;

[0059] Figure 14 This is a schematic diagram of an electronic device structure provided in an embodiment of this application. Detailed Implementation

[0060] This specific embodiment is merely an explanation of this application and is not intended to limit it. After reading this specification, those skilled in the art can make modifications to this embodiment without contributing any inventive step, but such modifications are protected by patent law as long as they are within the scope of this application.

[0061] It should be noted that, in the optional embodiments of this application, the data related to object information, when applied to specific products or technologies, requires the permission or consent of the object. Furthermore, the collection, use, and processing of this data must comply with the relevant laws, regulations, and standards of the relevant countries and regions. In other words, if the embodiments of this application involve data related to an object, it must be obtained with the object's authorization and consent, the authorization and consent of relevant departments, and in accordance with the relevant laws, regulations, and standards of the country and region. If the embodiments involve personal information, the acquisition of all personal information requires the individual's consent. If sensitive information is involved, the separate consent of the information subject is required. The embodiments also need to be implemented with the object's authorization and consent.

[0062] To make the objectives, technical solutions, and advantages of the embodiments of this application clearer, the technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0063] Furthermore, the term "and / or" in this article is merely a description of the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A existing alone, A and B existing simultaneously, or B existing alone. Additionally, the character " / " in this article, unless otherwise specified, generally indicates that the preceding and following related objects have an "or" relationship.

[0064] In traditional solutions, consumer-product interactions in retail scenarios suffer from problems such as difficulty in quantification, fragmented behavioral pathways, and severe data silos, specifically including:

[0065] 1. Traditional methods cannot effectively capture fine-grained touch behavior: Existing customer flow systems can only count stopping or approaching behavior, and cannot distinguish between "looking" and "touching", which leads to an underestimation of the actual level of interaction with the products and affects the accuracy of display effect evaluation;

[0066] 2. Lack of SKU (Stock Keeping Unit) level refined behavioral data collection capability: Most systems count pedestrian traffic by region, which is difficult to pinpoint the touch frequency and number of times a specific product is picked up, and cannot support single-item level marketing decisions;

[0067] The behavioral chain is broken, making it difficult to provide a basis for conversion attribution: the steps of entering the store, browsing, touching, and purchasing are recorded by different systems, lacking a unified data connection mechanism, making it difficult to analyze the impact of touching behavior on the final transaction.

[0068] 3. Lack of automatic identification mechanism for high-attractiveness / high-churn products: There is a lack of effective identification and early warning methods for products that are frequently touched but rarely purchased (potentially problematic products) or products that are rarely touched but have stable sales (potential best-sellers);

[0069] 4. Limited data application, making it difficult to guide actual operations: Existing behavioral data mostly remains at the report level and has not formed a closed loop linkage with operational actions such as replenishment, display adjustment, and promotional placement;

[0070] 5. High deployment cost and poor scalability: Solutions that rely on dedicated sensors or RFID tags are expensive and difficult to scale up to ordinary shelving environments.

[0071] Therefore, this application provides a method, apparatus, device, and storage medium for analyzing product interaction behavior, so as to accurately and effectively identify the interaction behavior between consumers and products, and reduce deployment costs and difficulties.

[0072] See Figure 1 The above is a flowchart illustrating a method for analyzing product interaction behavior provided in an embodiment of this application, including:

[0073] S101. Obtain the video frame to be detected;

[0074] This application can reuse existing cameras to acquire video footage for detection when identifying product interaction behavior, so as to identify product interaction behavior from the video footage. This method requires no additional hardware investment, has low deployment costs and a short cycle, is applicable to various retail scenarios, and has the capability for large-scale replication and promotion.

[0075] The video footage to be detected may include video footage captured by cameras in different locations, including footage of products being displayed on shelves, footage of consumers browsing products, footage of consumers touching products, footage of consumers experiencing products, etc., without being specifically limited here.

[0076] S102. Detect the user's hand area and the product area of ​​each item from the video image;

[0077] S103. Based on the overlap between the hand area and each product area, and the positional changes of each product area, identify product interaction behaviors; wherein, product interaction behaviors include at least one of product touch behavior and product depth experience behavior;

[0078] In this application, a touch detection algorithm can be used to identify product interaction behavior. When applying this solution to analyze product interaction behavior, an electronic device equipped with the touch detection algorithm needs to be prepared in advance. This electronic device can be a camera providing video feed, or a computer or server that performs analysis functions independently, etc., without specific limitations. Furthermore, after deploying the touch detection algorithm, it is also necessary to configure relevant parameters, including but not limited to configuring deployment time, confidence level, and detection frequency. Deployment time refers to the algorithm's detection time; for example, analyzing and statistically analyzing product interaction behavior results only during business hours (e.g., 10:00-22:00). Confidence level is the threshold for the algorithm to determine "a touch behavior has occurred," used to filter low-quality or suspected false detection results. Detection frequency is the frequency at which touch behavior analysis is performed once per second or every several frames.

[0079] This touch detection algorithm can identify the user's hand area and the product areas in the video frame being detected. The hand area is the display area of ​​the user's hand in the video frame, and the product area is the display area of ​​the product in the video frame, such as the area displaying each cup or the area displaying each bag of potato chips.

[0080] After detecting the user's hand area and the product areas of each item, this application can identify product interaction behavior based on the overlap between the hand area and the product areas, as well as the positional changes of the product areas. Specifically, if the hand area overlaps with the product areas, it indicates that the user may be touching the product. However, simply judging by whether the areas overlap cannot accurately distinguish between product touching behavior and in-depth product experience behavior. Therefore, this application also needs to consider the positional changes of the product areas. If the position of the product areas changes, it indicates that the user is picking up the product to experience it, i.e., in-depth product experience behavior has occurred. If the position of the product areas does not change, it indicates that the user is only touching the product with their hand and has not picked it up to experience it, thus only product touching behavior has occurred.

[0081] In another embodiment of this application, product interaction behavior is identified based on the overlap between the hand area and each product area, and the positional changes of each product area. This includes: if the hand area overlaps with a product area, detecting whether the overlap time exceeds a predetermined threshold; if the overlap time does not exceed the predetermined threshold, determining that no product interaction behavior has been identified; if the overlap time exceeds the predetermined threshold, determining that a product touch behavior has been identified; determining whether the position of the product area overlapping with the hand area has changed; if the position has changed and the hand posture is a predetermined posture, determining that a product depth experience behavior has been identified.

[0082] Specifically, to improve recognition accuracy, this application uses multiple video frames taken from different angles to comprehensively judge whether the hand area overlaps with the product area when determining whether the hand area overlaps with the product area based on multiple video frames. Only when the overlap is determined across multiple video frames can a decision be made that the hand area overlaps with the product area. For example, if the overlap between the user's hand area and the water cup area is detected in two video frames taken from different angles, then it is determined that the user has touched the water cup. Furthermore, after detecting the overlap, this application also needs to determine whether the overlap time exceeds a predetermined threshold. If it exceeds the predetermined threshold, then a product touch behavior is recognized; if it does not exceed the predetermined threshold, then a product interaction behavior is not recognized. By detecting the overlap time, this application can effectively distinguish between intentional touches, accidental touches, and noise interference, thereby improving recognition accuracy.

[0083] It should be noted that this application divides product interaction behavior into product touch behavior and product deep experience behavior. Product touch behavior refers to the behavior of a user touching a product but not experiencing it. That is, if the overlap between the user's hand area and the product area is detected and the overlap lasts for a certain period of time, it is considered that the user has touched the product, and product touch behavior has occurred. On the other hand, product deep experience behavior refers to the behavior of a user picking up and putting down the product after touching it. That is, if the coordinates of the product area change and the user's hand area overlaps with the product area, it is considered that the user has picked up the product for deep experience. It can be seen that product touch behavior must have occurred before product deep experience behavior occurs, but product touch behavior does not necessarily lead to product deep experience behavior.

[0084] This application can determine whether there is a deep product experience behavior by detecting changes in the position of the product area and the user's hand posture. If the position of the product area overlapping with the hand area changes and the hand posture is a predetermined posture, then a deep product experience behavior is identified. The predetermined posture can be a fist posture, a pinching posture, or other postures related to the experience behavior.

[0085] Specifically, the touch detection algorithm described in this application employs a product touch recognition mechanism based on modeling the interaction relationship between gestures and objects. This is a multimodal fusion algorithm, which integrates human keypoint detection, hand pose estimation, and target object spatial relationship analysis. Human keypoint detection includes detecting key points of the human body in the video frame, such as hands, shoulders, elbows, wrists, hips, and knees. Through human keypoint detection, the user's hand area can be determined from the video frame. Hand pose estimation refers to specific hand poses, such as open, clenched fist, or pinching. When recognizing product interaction behavior, referencing hand poses can generate more accurate recognition results. For example, an open hand pose indicates that the user is about to touch the product, while a clenched fist or pinching pose can serve as direct evidence of whether a deep product interaction behavior has occurred. For instance, after recognizing a deep product interaction behavior, the algorithm detects whether the user's hand pose is a specific pose (e.g., clenched fist or pinching). If so, it is determined to be a deep product interaction behavior. This method improves recognition accuracy.

[0086] The touch detection algorithm described in this application can accurately determine whether a customer has performed a series of actions such as "reaching out → touching → picking up → putting back," overcoming the limitations of traditional methods that rely solely on distance or dwell time to infer interactive behavior. Furthermore, the touch detection algorithm described in this application can also employ attention mechanisms and multi-task learning mechanisms. Specifically, the attention mechanism allows the touch detection algorithm to dynamically focus on potentially touched areas and keyframes when recognizing behavior, suppressing the influence of irrelevant people or occluded areas. The multi-task learning mechanism can simultaneously predict gestures, object categories, and contact states, sharing feature representations and improving generalization ability. Therefore, this solution can maintain high-precision recognition even in complex environments (such as changes in lighting, occlusion, and multi-person interaction), significantly improving the accuracy and robustness of touch behavior recognition.

[0087] S104. Analyze the interaction behavior of each product and generate product interaction data with the product's SKU information as the dimension.

[0088] In this application, SKU information refers to the smallest unit used to uniquely identify and manage product inventory in retail, e-commerce, and supply chain management. Specifically, in this solution, it refers to a specific product; for example, SKU information could be for a T-shirt, gift box, water bottle, etc. It should be noted that when performing product interaction behavior analysis, this application needs to identify the interaction behavior of each product. The SKU information of these products can be the same or different. However, when statistically analyzing product interaction data, this application needs to generate product interaction data based on SKU information. For example, if the SKU information of 10 water bottles on a shelf is all "water bottle," and the identified product interaction behaviors include: interaction behavior with the first water bottle, interaction behavior with the third water bottle, and interaction behavior with the seventh water bottle, then when statistically analyzing product interaction data, the product interaction data of the above three product interaction behaviors is counted and used as the product interaction data for the SKU information (water bottle).

[0089] In another embodiment of this application, analyzing the interaction behavior of each product to generate product interaction data based on the SKU information of the product includes: determining the product information corresponding to each product interaction behavior, and using the product information to determine the corresponding SKU information; analyzing and statistically analyzing the interaction behavior of products belonging to the same SKU information to generate product interaction data corresponding to each SKU information; wherein, the product interaction data includes: number of touches, touch rate, in-depth experience time, and conversion rate.

[0090] Specifically, when implementing SKU-level product touch analysis, this application does not require drawing a bounding box for touch detection, but it does require pre-entering the SKU information of each product. This entry method can be achieved through product information recognition by a self-developed robot or other means of entering product information; no specific limitation is made here. When analyzing the interaction behavior of each product, this application first needs to determine the product information corresponding to each product interaction behavior, and then, based on this product information, determine the pre-entered SKU information to generate SKU-level product interaction data. For example, by analyzing the video footage of the product interaction behavior, the product information can be determined as: blue, cup shape, product logo color, product logo content, etc. By querying, the corresponding SKU information can be determined as: blue water cup.

[0091] The product interaction data in this application includes: touch count, touch rate, in-depth experience time, conversion rate, etc. In the statistical analysis of product interaction data based on SKU information, touch count refers to the total number of times each product corresponding to the SKU information is touched. Touch rate refers to the attractiveness of each product corresponding to the SKU information relative to other products on the same shelf, i.e., the number of touches of each product corresponding to the same SKU information / the total number of touches of all products on the shelf. In-depth experience time refers to the average time it takes for each product corresponding to the same SKU information to be picked up through in-depth experience behavior. For example, if each product corresponding to the same SKU information is picked up 5 times, totaling 10 minutes, then the average in-depth experience time is 2 minutes. Conversion rate is the number of times in-depth experience behavior occurs for each product corresponding to the same SKU information / the number of times product touch behavior occurs.

[0092] The final product data generated includes the number of touches, touch rate, in-depth experience time, and conversion rate corresponding to each SKU. This data is the product interaction data based on SKU information.

[0093] It should be noted that, without drawing the bounding box of the area to be detected, this application can detect product interaction behavior by actively identifying product areas, achieving fine-grained measurement of product interaction behavior at the SKU level. Furthermore, this application also features a flexible area labeling function, that is, it supports flexible labeling of individual SKUs or small areas in video footage, actively drawing detection areas, and accurately calculating metrics such as the number of touches, touch duration, and conversion rate for each SKU through the aforementioned touch detection algorithm. This solution provides micro-data support for single-product-level marketing decisions, filling the gap in existing systems that cannot accurately analyze behavior down to the specific product level.

[0094] It should be noted that when applying this solution to real-world scenarios, existing resources can be directly reused, and it is entirely based on the existing cameras in the store. No additional sensors or tag devices need to be deployed, which significantly reduces implementation costs. Furthermore, this solution can be rapidly deployed at scale, has the ability to be quickly replicated and promoted in chain stores, is suitable for various retail scenarios, and has good versatility and scalability.

[0095] In summary, this application can achieve precise analysis of product touch behavior and in-depth product experience behavior solely through video footage, improving the accuracy of product interaction behavior recognition, reducing deployment costs, and enhancing scalability. Furthermore, this application can also perform refined statistical analysis of SKU-level product interaction data, enabling refined data collection capabilities for behavior. SKU-level product interaction data can effectively support refined marketing and operational decisions.

[0096] In another embodiment of this application, determining the product information corresponding to each product interaction behavior includes:

[0097] Determine the first image and the second image corresponding to the product interaction behavior; wherein, the first image is an image of a predetermined number of consecutive frames before the product interaction behavior occurs, and the second image is an image of a predetermined number of consecutive frames after the product interaction behavior occurs;

[0098] The hand movement trajectory of the product interaction behavior is determined by the second image, and the hand movement trajectory is associated with each product area of ​​the first image to determine the associated product area; the corresponding product information is determined by identifying the associated product area.

[0099] In this application, when determining the product information corresponding to each product interaction behavior, if the product information may be obscured or unclear due to factors such as people / hands when the user takes the product from the shelf, the product information can be identified by sharing multi-angle image data between NVRs (Network Video Recorders). For example, if one of the video feeds of the NVR is obscured, the product information can be identified from the video feed of the other NVR.

[0100] Furthermore, this application can also identify product information through multiple consecutive frames of images, that is: acquiring a first image of a predetermined number of consecutive frames before the product interaction occurs, and a second image of a predetermined number of consecutive frames after the interaction occurs. The predetermined number of frames can be a number of frames between a few seconds before the product interaction occurs and a few seconds after the interaction occurs, such as a number of frames between 2 seconds before the interaction occurs and 1 second after the interaction occurs. In the first image before the product interaction occurs, the product is still on the shelf and is not obscured, so the position and appearance features of the product can be obtained in advance. After the product interaction occurs, the hand movement trajectory of the product interaction can be determined by optical flow or target tracking, and then the hand movement trajectory can be combined with the movement trajectory to associate the hand area with the original shelf position, and finally associate it with the product area at a certain position on the original shelf, which is the associated product area. Then, the product information can be determined through the associated product area.

[0101] Furthermore, this application can also extract the information of the retrieved product by detecting out-of-stock items in previous and subsequent frames. This process includes: after the product interaction occurs, comparing the shelf status before and after the retrieval; if a vacancy appears in a certain position, and the shape and size of the vacancy match the product corresponding to a certain SKU, and at the same time, a touch / retrieval behavior occurs during that period, then it can be inferred with high confidence that the missing product is the product that was retrieved, and then the SKU can be determined through the product information.

[0102] In summary, this application can accurately identify product information and thus accurately determine the SKU by using multi-angle image capture, multi-frame continuous images, and out-of-stock detection methods. This method can also be used to accurately collect product interaction data.

[0103] In another embodiment of this application, after generating product interaction data with SKU information as the dimension, the method further includes:

[0104] By utilizing the product interaction data of each SKU in the target shelf, the touch intensity of products corresponding to different SKU information in the target shelf is determined and displayed through a touch intensity map; based on the touch frequency of each SKU information in the target shelf, the target SKU information is filtered, and the product interaction data of the target SKU information is displayed through an interaction data list.

[0105] In this application, after generating product interaction data, the corresponding touch heatmap and interaction data list can be displayed using the product interaction data. The touch heatmap is used to display the touch heat of products corresponding to different SKU information. This product touch heat can be generated by the number of touches or the touch rate. The target shelf of the touch heatmap includes different levels of the target shelf, and products corresponding to each SKU information are placed in different levels. When displaying product touch heat, it can be displayed through areas of different colors and sizes, and the display location can be the placement position of the product corresponding to each SKU information on the target shelf.

[0106] The target SKU information filtered in this application can also be filtered according to actual needs. If a user wants to view the product interaction data of the SKUs with the highest number of touches, the SKUs with the highest number of touches will be selected as the target SKUs. Conversely, if a user wants to view the product interaction data of the SKUs with the lowest number of touches, the SKUs with the lowest number of touches will be selected as the target SKUs. Furthermore, this application can display the generated touch heatmap and interaction data list on the data display page, making it convenient for users to understand the product interaction data of different SKUs.

[0107] See Figure 2 This is a schematic diagram of the data display page provided in the embodiment of this application. The left side of the data display page displays a touch heat map. Different products are distinguished by circular areas of different colors and sizes. The right side displays a touch statistics list and a list of popular products. The touch statistics list displays the total number of touches, the total number of in-depth experiences, and the conversion rate of the left shelf. The list of popular products displays the interaction data of the three products with the most touches on the shelf.

[0108] In another embodiment of this application, after generating product interaction data corresponding to each SKU information, the method further includes: identifying abnormal SKUs based on the product interaction data and product sales data of each SKU information; and generating early warning information based on the abnormal SKUs.

[0109] After generating product interaction data in this application, the product interaction data and product sales data of each SKU can be analyzed to identify abnormal SKUs. Abnormal SKUs can be: SKUs with high touch counts but low sales volume, or SKUs with low touch counts but high sales volume. For abnormal products of abnormal SKUs, early warning information can be provided, including: display optimization suggestions, replenishment warnings, and promotional strategy recommendations, etc., which are not specifically limited here.

[0110] For example, SKUs with high touch counts but low sales volume: Consumers frequently touch and experience these products, but the actual purchase conversion rate is low. This usually indicates that the product is attractive (generating interest), but there are obstacles in the decision-making process. In this case, the generated alert messages would be: optimize the display location to prime visibility areas; add key selling points or compare sales guide information to strengthen the reasons for purchase; check if popular specifications are out of stock or if display models are damaged, and trigger restocking or maintenance alerts; simultaneously recommend limited-time trial discounts, bundled promotions, or display real-time sales / positive reviews and other social media evidence to lower the decision-making threshold and effectively convert high attention into sales.

[0111] As can be seen, this application can automatically generate touch heatmaps at the shelf level, intuitively displaying the distribution of product interaction density in different locations, and assisting in optimizing the allocation of prime display positions. Furthermore, after generating product interaction data, it can also identify abnormal SKUs with high touch counts but low sales volume, and abnormal SKUs with low touch counts but high sales volume, by combining product sales data, and automatically generate early warning information, including display optimization suggestions, replenishment warnings, and promotional strategy recommendations, thereby achieving a data-driven intelligent operation closed loop.

[0112] In another embodiment of this application, after identifying the product interaction behavior, the method further includes:

[0113] The system collects and analyzes product interaction behavior in different analysis areas to generate product interaction data based on the analysis area. The analysis area includes: display area or shelf area.

[0114] Acquire passenger flow data for each analysis area; wherein, the passenger flow data is generated after passenger flow identification in the passenger flow detection area of ​​each analysis area; display the product interaction data, passenger flow data and sales data of different analysis areas.

[0115] In this application, after identifying product interaction behavior, this solution can not only achieve SKU-level product touch analysis, but also achieve product touch analysis in a preset analysis area. This analysis area refers to the display area or shelf area that the user pre-selects and draws on the camera's captured image. The display area refers to the area marked on the captured image of the display area. For example, based on the image captured by the camera from above the product display area, the new mobile phone area, the historical mobile phone models, and other accessories areas are selected as the display areas respectively. The shelf area refers to the area marked on the captured image of each shelf.

[0116] Therefore, by statistically analyzing the product interaction behavior in different booth or shelf areas, product interaction data can be generated at the booth or shelf area level. This data includes the number of touches, the number of deep experiences, and the duration of deep experiences in different shelf areas. Furthermore, this application, in addition to defining detection areas, identifying touch behaviors occurring within those areas, and capturing product interaction data, can further capture the dwell time and touch behaviors of customers entering the store in corresponding customer flow detection areas. Therefore, when labeling each analysis area, this application also needs to simultaneously label the corresponding customer flow detection areas and perform customer flow identification in these areas to generate customer flow data for each analysis area. This data includes the number of people staying, the duration of stay, etc. This application also needs to obtain sales data for each analysis area, including the number of items sold and the number of returned items. Finally, the product interaction data, customer flow data, and sales data from different analysis areas are displayed, including touch heatmaps and / or interaction data lists.

[0117] See Figure 3 The figure shows a schematic diagram of the area division provided in this application embodiment. As shown in the figure, this application divides the area into a customer flow statistics area and a touch statistics area. The customer flow statistics area is used to count customer flow data, including the number of customers, the number of people stopping, the number of people with purchasing intentions, etc. The touch statistics area is used to count product interaction data related to touch, including the number of touches, etc. As shown in the figure, the number of customers, the number of touches, and the number of transactions in a certain area are in the shape of an inverted pyramid, with the number of customers being the largest and the number of transactions being the smallest.

[0118] The touch heat map includes all analysis areas set in different locations, and displays the regional touch heat of each analysis area in the corresponding location. This touch heat can be determined by the number of deep experiences of product interaction data. The interaction data list can display all or part of the product interaction data, customer flow data, and sales data of the partitioned areas.

[0119] For example: If the designated area is a booth, then collect product interaction data (including touch and in-depth experience behaviors) for all products in each booth area, as well as customer flow and sales data. Generate an interaction data list from this data and display it on the floor plan corresponding to the product booth. The list fields include: number of people stopping, number of people engaging in in-depth experiences, and number of items sold, etc. The touch intensity of different booth areas also needs to be displayed on the floor plan. Similarly, if the designated area is a shelf, collect product interaction data (including touch and in-depth experience behaviors) for all products in each shelf area, as well as customer flow and sales data. Generate an interaction data list from this data and display it on the floor plan corresponding to the product shelf. The list fields include: shelf type, number of people stopping, number of people engaging in in-depth experiences, and number of items sold. The touch intensity of different shelf areas also needs to be displayed on the floor plan.

[0120] It should be noted that in this application, after the user defines the customer flow detection area, a precise customer flow algorithm and / or a browsing analysis algorithm can be used to perform precise customer flow detection. The generated customer flow data can perform functions such as deduplication of customers entering the store and browsing analysis. Based on the precise customer flow, the system can track customers entering the store across the entire area and display the data on the report in combination with shelf data and product interaction data.

[0121] See Figure 4 This is a schematic diagram of customer flow display based on a floor plan provided in an embodiment of this application. As shown in the figure, the left side displays a heat map with a blue-to-red gradient based on the store's floor plan, and the right side displays specific customer flow data for area x1 in the figure. See also Figure 5 The figure shows a schematic diagram of customer attributes in a store area provided in this application embodiment. The figure includes customer flow data and touch count in different areas of the store. In addition to the number of customers, number of visits, number of people stopping, stopping rate, and dwell time mentioned in the figure, the customer flow data also includes the number of men, women, teenagers, young people, middle-aged people, middle-aged and elderly people, and elderly people, etc., which are not shown in the figure one by one.

[0122] In summary, the product interaction behavior analysis method described in this application, by flexibly delineating detection areas in video footage and combining multimodal AI vision algorithms based on human posture estimation, hand motion recognition, and object interaction relationship modeling, accurately identifies continuous touch interaction behaviors such as "reaching out—touching—picking up—putting back" between customers and touched objects (product level or area level). The algorithm output data is then combined with application scenarios or other algorithms to demonstrate data and effects. This solution not only fills the gap in existing technologies for recognizing consumer physical interaction behaviors but also achieves a leap from "seeing people" to "understanding behavior" through multi-source data fusion and deep coupling with business scenarios, providing the retail industry with a new, refined operational tool. The technical effects achieved by this solution include, but are not limited to, the following:

[0123] 1. Achieve precise recognition of fine-grained touch behavior: By integrating human posture and hand action recognition algorithms, it effectively distinguishes between "looking" and "touching" behaviors, and realizes automatic detection of actions such as "touching" and "picking up", significantly improving the accuracy of product interaction behavior recognition.

[0124] 2. Supports SKU-level refined behavior measurement: It can accurately count the frequency of touch, duration of touch and number of times picked up a single product, realizing the collection of behavior data from the regional level to the single product level, supporting refined marketing and operational decisions.

[0125] 3. Streamline the entire behavioral chain and support conversion attribution analysis: Integrate customer flow and POS data (sales data) to build a complete path of "entering the store → browsing → touching → purchasing", forming a quantifiable conversion funnel, clarifying the impact of touch behavior on transactions, and improving attribution analysis capabilities.

[0126] 4. Automatic identification of high-attractiveness and high-churn products: Based on touch frequency and conversion rate metrics, automatically identify abnormal products such as "high touch, low conversion" and "low touch, high conversion", provide early warning prompts, and assist in product optimization and strategy adjustment.

[0127] 5. Promote data-driven closed-loop operation applications: Link touch data with operational actions such as display optimization, replenishment scheduling, promotional placement, and sales guide configuration to achieve closed-loop management from data insight to business execution.

[0128] 6. Reduce deployment costs and improve scalability: It fully reuses existing cameras, requires no additional hardware investment, has low deployment costs and short cycles, is suitable for various retail scenarios, and has the ability to be replicated and promoted on a large scale.

[0129] 7. Enhanced real-time performance and visualization capabilities: Supports second-level data updates, heat map display, timeline backtracking, and multi-store comparison, providing intuitive and dynamic visualization analysis tools to facilitate rapid response and decision-making by managers.

[0130] Here, we will use cloud-monitored shelves as a practical application scenario to explain the process of customer flow identification and touch detection in this solution. This embodiment needs to record the browsing and touching of customers entering the store, as well as the sales status of the shelves.

[0131] In this embodiment, one AINVR (Artificial Intelligence Network VideoRecorder) can operate 7 streams, with 1 stream entering the store, the remaining 3 streams used for customer flow detection, and the remaining 3 streams used for analysis areas linked to the merchandise area. Here, "streams" refers to the number of video streams the NVR can simultaneously perform real-time intelligent analysis. See also... Figure 6The figure shows a schematic diagram of the AI ​​capability configuration provided in the embodiment of this application. As shown in the figure, three cameras have been set up to detect gestures touching the shelf.

[0132] Furthermore, this application also requires drawing the analysis area and the passenger flow detection area in the video frame. In this embodiment, the analysis area refers to the shelf area. See [link to relevant documentation]. Figure 7 This is a schematic diagram of the area setting provided in this application embodiment. In the video footage, the instant noodle shelf area is drawn with a pink box, and the customer flow detection area of ​​the instant noodle shelf area is drawn with a green box. It should be noted that if only touch behavior is detected, it is impossible to track who touched it and how many times. Therefore, in this application, it is necessary to associate the hand ID and the personnel ID to track the consumer's picking behavior. If it is necessary to associate it with the sales situation of the shelf, the shelf and the customer flow detection area need to be bound when the shelf is created.

[0133] See Figure 8 The following is a schematic diagram of the area added to the embodiment of this application. As shown in the figure, after adding the store's floor plan, a customer flow detection area (green area) can be added to the floor plan. The left side is the customer flow detection area for the potato chip and jelly shelf, the middle side is the customer flow detection area for the instant noodle shelf, and the right side is the customer flow detection area for the freezer. See also Figure 9 This is a schematic diagram of the customer flow touch heat experience analysis display interface provided in this application embodiment. The left side of the interface displays a store floor plan, and the touch heat of different areas is displayed within the customer flow detection area. The right side of the interface displays the customer flow trajectory area and the shelf interaction area. It can be seen that the potato chip and jelly shelf has the highest number of touches (136), thus having the highest heat and the darkest color. The instant noodle shelf and the refrigerated display case have the highest number of touches (4 and 7 respectively), resulting in lower heat and lighter colors. The customer flow trajectory area only displays the latest 5 customer browsing data. Clicking... Figure 9 You can view detailed data for today by clicking on the customer flow trajectory area and the shelf interaction area. Here, we will only use clicking on the customer flow trajectory area as an example for explanation.

[0134] See Figure 10 The image displays complete customer flow data after a user clicks on the customer flow trajectory. It shows the trajectory data of multiple customers who entered the store at different times. Customer 41, highlighted in the image, spent varying amounts of time in the instant noodle area, potato chip and jelly area, and refrigerated display area after entering the store. Furthermore, in a cloud-based monitoring scenario, a single visit to the store may result in a purchase. Therefore, clicking on the time of the entry event allows the system to trace the entire customer's entry process via the store's video equipment, enabling playback of recordings and viewing of customer browsing behavior without deduplicating customer data. See also... Figure 11This is a schematic diagram of the playback recording interface provided in this embodiment. As shown in the figure, the entire interface includes multiple monitoring screens from different angles. The event list on the right is used to display each entry time. Users can click on different entry events to switch monitoring screens. The monitoring screen control area at the bottom can control functions such as pausing, playing, speeding up, and taking screenshots of the monitoring screen.

[0135] Furthermore, this application can also see the correlation data between the analysis areas of touch and goods in the precise customer flow area data, see [link to relevant documentation]. Figure 12 The diagram below is a schematic of the precise customer flow area display interface provided in this application embodiment. It can be seen that the precise customer flow area data includes the number of touches. Therefore, the relationship between the number of touches and different products can be viewed through the precise customer flow area.

[0136] In summary, this solution, by drawing customer flow detection areas and shelf areas under the video screen, can use algorithms to statistically analyze the product interaction data of the areas in real time, and view the interaction between customers and products in real-time video playback and replay, thus constructing an end-to-end consumer behavior conversion link and realizing the visualization and accurate quantification of consumer-product interaction behavior;

[0137] This solution supports SKU-level refined behavioral analysis, accurately tracking product interaction data such as touch frequency, touch rate, and depth of experience for individual products. It fills the technological gap in existing systems that cannot achieve single-product-level interaction measurement, providing micro-data support for refined operations. Utilizing SKU-level product interaction data, this solution can generate shelf-level touch heatmaps, visually presenting the distribution of product touch density. This helps optimize prime display locations, increasing the exposure and conversion opportunities of high-potential products, and improving product display and space utilization efficiency. Furthermore, on the right side of the touch heatmap, a list of interaction data shows the total touch statistics and the top three most touched products, allowing users to quickly understand the interaction status of popular products. In addition, this application links product interaction data with regional customer flow data (customer source, customer profile, dwell time, etc.), promotional placement, and sales guide resource allocation to form a closed-loop management mechanism of "perception—analysis—decision—execution," improving the intelligence level of store operations and promoting the implementation of data-driven operational loops.

[0138] The following describes the product interaction behavior analysis device provided in the embodiments of this application. The product interaction behavior analysis device described below and the product interaction behavior analysis method described above can be referred to in correspondence.

[0139] See Figure 13 , Figure 13 This application provides a schematic diagram of a device for analyzing product interaction behavior, which specifically includes:

[0140] Acquisition module 11 is used to acquire the video frame to be detected;

[0141] The detection module 12 is used to detect the user's hand area and the product area of ​​each product from the video image;

[0142] The recognition module 13 is used to identify product interaction behavior based on the overlap between the hand area and each product area, as well as the positional changes of each product area; wherein, the product interaction behavior includes at least one of product touch behavior and product depth experience behavior;

[0143] Analysis module 14 is used to analyze the interaction behavior of each product and generate product interaction data with the product's SKU information as the dimension.

[0144] As an optional embodiment, the identification module includes:

[0145] The detection unit is used to detect whether the overlap time between the hand area and the product area exceeds a predetermined threshold when the hand area overlaps with the product area; if the overlap time does not exceed the predetermined threshold, it is determined that no product interaction behavior has been identified; if the overlap time exceeds the predetermined threshold, it is determined that product touch behavior has been identified.

[0146] The judgment unit is used to determine whether the position of the product area overlapping with the hand area has changed; if the position has changed and the hand posture is a predetermined posture, then the product depth experience behavior is identified.

[0147] As an optional embodiment, the analysis module includes:

[0148] The first determining unit is used to determine the product information corresponding to each product interaction behavior;

[0149] The second determining unit is used to determine the corresponding SKU information using the product information;

[0150] The analysis unit is used to analyze and statistically analyze the interactive behaviors of products belonging to the same SKU information, and generate product interaction data corresponding to each SKU information; wherein, the product interaction data includes: number of touches, touch rate, depth of experience time and conversion rate.

[0151] As an optional embodiment, the analysis module further includes:

[0152] The alert unit is used to identify abnormal SKUs based on the product interaction data and product sales data of each SKU; and to generate early warning alert information based on the abnormal SKUs.

[0153] As an optional embodiment, the first determining unit is specifically used for:

[0154] Determine the first image and the second image corresponding to the product interaction behavior; wherein, the first image is an image of a predetermined number of consecutive frames before the product interaction behavior occurs, and the second image is an image of a predetermined number of consecutive frames after the product interaction behavior occurs;

[0155] The hand movement trajectory of the product interaction behavior is determined by the second image, and the hand movement trajectory is associated with each product area in the first image to determine the associated product area;

[0156] The corresponding product information is determined by identifying the associated product areas.

[0157] As an optional embodiment, the device further includes:

[0158] The first display module is used to determine the touch heat of products corresponding to different SKU information in the target shelf by using the product interaction data of each SKU information in the target shelf, and display it through a touch heat map;

[0159] The second display module is used to filter target SKU information based on the number of times each SKU information in the target shelf is touched, and to display the product interaction data of the target SKU information through an interaction data list.

[0160] As an optional embodiment, the device further includes:

[0161] The statistics module is used to collect statistics on product interaction behavior in different analysis areas, analyze the product interaction behavior in different analysis areas, and generate product interaction data with the analysis area as the dimension; wherein, the analysis area includes: display area or shelf area;

[0162] The third display module is used to acquire customer flow data from each analysis area and display the product interaction data, customer flow data, and sales data from different analysis areas; wherein, the customer flow data is generated after customer flow identification in the customer flow detection area of ​​each analysis area.

[0163] Figure 14 A structural diagram of an electronic device provided in an embodiment of the present invention is shown in the figure, comprising:

[0164] Memory 20 is used to store computer programs;

[0165] The processor 21 is configured to implement the steps of the analysis method as described in the above embodiments when executing a computer program.

[0166] The electronic devices provided in this embodiment may include, but are not limited to, smartphones, tablets, laptops, or desktop computers.

[0167] The processor 21 may include one or more processing cores, such as a quad-core processor or an octa-core processor. The processor 21 may be implemented using at least one hardware form selected from Digital Signal Processing (DSP), Field-Programmable Gate Array (FPGA), and Programmable Logic Array (PLA). The processor 21 may also include a main processor and a coprocessor. The main processor, also known as the Central Processing Unit (CPU), is used to process data in the wake-up state; the coprocessor is a low-power processor used to process data in the standby state. In some embodiments, the processor 21 may integrate a Graphics Processing Unit (GPU), which is responsible for rendering and drawing the content to be displayed on the screen. In some embodiments, the processor 21 may also include an Artificial Intelligence (AI) processor, which handles computational operations related to machine learning.

[0168] The memory 20 may include one or more computer-readable storage media, which may be non-transitory. The memory 20 may also include high-speed random access memory and non-volatile memory, such as one or more disk storage devices or flash memory devices. In this embodiment, the memory 20 is used to store at least the following computer program 201, which, after being loaded and executed by the processor 21, is capable of implementing the relevant steps of the analysis method disclosed in any of the foregoing embodiments. In addition, the resources stored in the memory 20 may also include an operating system 202 and data 203, and the storage method may be temporary storage or permanent storage. The operating system 202 may include Windows, Unix, Linux, etc.

[0169] In some embodiments, the electronic device may further include a display screen 22, an input / output interface 23, a communication interface 24, a power supply 25, and a communication bus 26.

[0170] Those skilled in the art will understand that Figure 14 The structures shown do not constitute a limitation on electronic devices and may include more or fewer components than those shown.

[0171] In another exemplary embodiment, a computer storage medium is also provided, wherein the program instructions, when executed by a processor, implement the steps of the data deduplication method described in any of the above method embodiments.

[0172] It is understood that if the analysis methods in the above embodiments are implemented as software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, in essence, or the part that contributes to the current technology, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and executes all or part of the steps of the methods in the various embodiments of the present invention. The aforementioned storage medium includes: USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), electrically erasable programmable ROM, registers, hard disks, removable disks, CD-ROMs, magnetic disks, or optical disks, and other media capable of storing program code.

[0173] The various embodiments described in this specification are presented in a progressive manner, with each embodiment focusing on its differences from other embodiments. Similar or identical parts between embodiments can be referred to interchangeably. It should be understood that the terminology used herein is for the purpose of describing particular exemplary embodiments only and is not intended to be limiting. Unless the context clearly indicates otherwise, the singular forms “a,” “an,” and “described” used herein may also mean the plural forms. The terms “comprising,” “including,” “containing,” and “having” are inclusive and therefore indicate the presence of the stated features, steps, operations, elements, and / or components, but do not exclude the presence or addition of one or more other features, steps, operations, elements, components, and / or combinations thereof. The method steps, processes, and operations described herein are not construed as requiring them to be performed in a specific order described or illustrated, unless the order of performance is explicitly indicated. It should also be understood that additional or alternative steps may be used.

[0174] It should be understood that although the steps in the flowcharts of the accompanying figures are shown sequentially as indicated by the arrows, these steps are not necessarily executed in the order indicated by the arrows. Unless explicitly stated herein, there is no strict order restriction on the execution of these steps, and they can be executed in other orders. Moreover, at least some steps in the flowcharts of the accompanying figures may include multiple sub-steps or multiple stages. These sub-steps or stages are not necessarily completed at the same time, but can be executed at different times, and their execution order is not necessarily sequential, but can be performed alternately or in turn with other steps or at least some of the sub-steps or stages of other steps.

[0175] The above are only some embodiments of this application. It should be noted that for those skilled in the art, several improvements and modifications can be made without departing from the principle of this application, and these improvements and modifications should also be considered within the scope of protection of this application.

Claims

1. A method of analyzing a merchandise interaction behavior, characterized by, The method comprises the following steps: acquiring a video picture to be detected; detecting a hand region of a user and a commodity region of each commodity from the video picture; recognizing a commodity interaction behavior according to a region overlap between the hand region and each commodity region and a position change of each commodity region, wherein the commodity interaction behavior comprises at least one of a commodity touch behavior and a commodity deep experience behavior; analyzing each commodity interaction behavior to generate commodity interaction data with SKU information of commodities as a dimension.

2. The analysis method according to claim 1, characterized in that, The step of recognizing the commodity interaction behavior according to the region overlap between the hand region and each commodity region and the position change of each commodity region comprises the following steps: if the hand region overlaps with the commodity region, detecting whether an overlap time of the hand region and the commodity region exceeds a predetermined threshold; if the overlap time does not exceed the predetermined threshold, determining that the commodity interaction behavior is not recognized; if the overlap time exceeds the predetermined threshold, determining that the commodity touch behavior is recognized; judging whether a position of the commodity region overlapping with the hand region changes; if the position changes and a hand posture is a predetermined posture, determining that the commodity deep experience behavior is recognized.

3. The analysis method according to claim 2, characterized in that, The step of analyzing each commodity interaction behavior to generate the commodity interaction data with the SKU information of the commodities as the dimension comprises the following steps: determining commodity information corresponding to each commodity interaction behavior and determining corresponding SKU information by using the commodity information; analyzing and counting commodity interaction behaviors belonging to the same SKU information to generate commodity interaction data corresponding to each SKU information, wherein the commodity interaction data comprises a touch frequency, a touch rate, a deep experience time and a conversion rate.

4. The analysis method according to claim 3, characterized in that, After the commodity interaction data corresponding to each SKU information is generated, the following steps are further included: recognizing an abnormal SKU according to the commodity interaction data of each SKU information and commodity sales data; generating a warning prompt information according to the abnormal SKU.

5. The analysis method according to claim 3, characterized in that, The step of determining the commodity information corresponding to each commodity interaction behavior comprises the following steps: determining a first image and a second image corresponding to the commodity interaction behavior, wherein the first image is a continuous predetermined number of frames of images before the commodity interaction behavior occurs, and the second image is a continuous predetermined number of frames of images after the commodity interaction behavior occurs; determining a hand motion trajectory of the commodity interaction behavior by using the second image, associating the hand motion trajectory with each commodity region of the first image, and determining an associated commodity region; determining corresponding commodity information by recognizing the associated commodity region.

6. The analysis method of claim 1, wherein, After the commodity interaction data with the SKU information as the dimension is generated, the following steps are further included: determining commodity touch heat corresponding to different SKU information in a target shelf by using commodity interaction data of each SKU information in the target shelf, and displaying by a touch heat map; screening a target SKU information according to a touch frequency of each SKU information in the target shelf, and displaying commodity interaction data of the target SKU information by an interaction data list.

7. The analysis method according to any one of claims 1 to 6, characterized in that, After the commodity interaction behavior is recognized, the following steps are further included: The commodity interaction behaviors of different analysis regions are counted and analyzed to generate commodity interaction data with the analysis region as a dimension; the analysis region includes an exhibition stand region or a shelf region; Obtain the passenger flow data of each analysis region; the passenger flow data is generated after passenger flow detection regions of each analysis region are identified; The commodity interaction data, passenger flow data and sales data of different analysis regions are displayed.

8. An analysis device of a merchandise interaction behavior, characterized by, Comprise: An acquisition module, configured to acquire a video picture to be detected; A detection module, configured to detect a hand region of a user and a commodity region of each commodity from the video picture; An identification module, configured to identify a commodity interaction behavior according to a region overlap between the hand region and each commodity region and a position change of each commodity region; the commodity interaction behavior includes at least one of a commodity touching behavior and a commodity deep experience behavior; An analysis module, configured to analyze each commodity interaction behavior to generate commodity interaction data with SKU information of a commodity as a dimension.

9. An electronic device, comprising: Comprise: A memory, configured to store a computer program; A processor, configured to execute the computer program to implement the steps of the analysis method in any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that, The computer program is stored on the computer readable storage medium and is executed by the processor to implement the steps of the analysis method in any one of claims 1 to 7.

Images