Video image analysis method, device and system

Abstract

The invention discloses a video image analysis method, device and system, and particularly relates to the technical field of video image analysis. The method comprises the following steps: extracting associated data from store video image data to establish an associated database, establishing a quadrilateral comparison model to mark passenger flow weakness dimensions, generating regional popularity ranks and popularity shelf numbers, converting abstract video image data into a visual result, integrating the information to generate an evaluation report, defining an optimization direction, and improving the evaluation efficiency. A complete closed loop is formed, and the problems of single analysis dimension and insufficient depth of video image data in the prior art are solved.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of video image analysis, and more particularly, to a video image analysis method, device and system. BACKGROUND

[0002] In the process of digital upgrading of retail store operation, video image analysis technology has become the core support of data-driven decision-making.

[0003] However, the video image analysis method in the prior art still has the following shortcomings in actual application process:

[0004] The traditional analysis method has weak data correlation fusion capability, and multi-source data such as video stream data, commodity SKU information, shelf position data and transaction data are stored in a scattered manner, and an effective correlation mechanism is not established, so that the interaction relationship between customer behavior and commodity and space is difficult to quantify;

[0005] In addition, the analysis dimension of video image data is single and insufficient, and the existing scheme focuses on the surface indicators such as the number of people entering the store, lacks systematic disassembly of core dimensions such as store entry rate, conversion rate, regional heat, and shelf interaction effect, and cannot accurately mine operation short boards, and it is also difficult to realize multi-level disassembly analysis from the whole store to the region and then to the shelf, so as to accurately identify high-value areas and low-efficiency areas, popular shelves and unsalable shelves, resulting in blind allocation of operation resources;

[0006] Finally, the result presentation and application are disconnected, the analysis result is output in the form of pure numerical value, lacks visual and intuitive short board positioning tools, and a complete closed loop from data collection, analysis to decision-making suggestion is not formed, so that the operator is difficult to quickly land optimization measures.

[0007] Therefore, a video image analysis method, device and system are proposed. SUMMARY

[0008] In order to overcome the above-mentioned defects of the prior art, the embodiments of the present application provide a video image analysis method, device and system.

[0009] In order to achieve the above-mentioned purpose, the present application provides the following technical scheme:

[0010] A video image analysis method, comprising:

[0011] S1: collecting video data of a store monitoring area, and generating a globally unique, random anonymous identity identifier for each customer detected in the video data;

[0012] S2: based on the anonymized video stream data, performing a track tracking task, and based on the anonymous identity identifier, generating a continuous moving track of the customer in the store from the store entrance;

[0013] S3: Establish a digital map of the store space, and associate the product information, trajectory information, and shelf location with the location in the digital map. Based on the associated fused data, establish an association database;

[0014] S4: Extract the associated data within the current set time zone from the store association database for aggregation and mining, generate the customer flow performance coefficient of the store and the hot zone performance coefficient of each divided area of the store;

[0015] S5: Based on the customer flow performance coefficient of the store and the hot zone performance coefficient of each divided area of the store, combined with historical association data and customer behavior data in video images for comprehensive analysis, generate an evaluation report of the store within the current set time zone.

[0016] Specifically, the calculation logic of the customer flow performance coefficient in S4 step;

[0017] Extract the store entry rate, conversion rate, average in-store time, and customer flow performance within the set time zone from the store association data;

[0018] Take the store entry rate, conversion rate, average in-store time, and customer flow performance extracted from the association data within the current set time zone as input, and output the customer flow performance coefficient after processing by the weighted calculation logic.

[0019] Specifically, the acquisition logic of the store entry rate, conversion rate, average in-store time, and customer flow performance;

[0020] Store entry rate: Count the total number of people flowing into the entrance area and the number of customers entering the store that meet the preset conditions, and calculate the proportion of the two;

[0021] Conversion rate: Match the anonymous IDs of the customers entering the store and the customers consuming, and count the proportion of the customers consuming to the customers entering the store;

[0022] Average in-store time: Extract the in-store time of all customers entering the store, and calculate the average value;

[0023] Customer flow performance: Take the average of the customers entering the store in the past X time zones as a benchmark, and calculate the ratio of the current number of customers entering the store.

[0024] Specifically, the calculation logic of the hot zone performance coefficient of each divided area in S4 step;

[0025] Get the partition visit rate, partition stay proportion, and partition conversion coefficient of each divided area;

[0026] Take the partition visit rate, partition stay proportion, and partition conversion coefficient of each divided area within the current set time zone as input, and output the hot zone performance coefficient of each divided area after processing by the weighted calculation logic.

[0027] Specifically, the acquisition logic of the partition visit rate, the partition stay proportion, and the partition conversion coefficient;

[0028] The partition visit rate: the number of customers in each region is divided by the total number of customers in the store to obtain the partition visit rate;

[0029] The partition stay proportion: the average stay time of customers in each region is divided by the average in-store time of the store to obtain the partition stay proportion.

[0030] The partition conversion coefficient: the contribution of the number of transactions and the contribution of the amount of money in each region are calculated and processed to obtain the partition conversion coefficient.

[0031] Specifically, the calculation logic of the contribution of the number of transactions and the contribution of the amount of money;

[0032] The contribution of the number of transactions is calculated by dividing the number of purchases of the divided region by the total number of transactions of the store;

[0033] The contribution of the amount of money is calculated by dividing the contribution of the divided region by the total turnover of the store.

[0034] Specifically, the generation logic of the evaluation report in the S5 step;

[0035] The customer flow improvement degree or the customer flow comparison model, the regional heat ranking, and the heat shelf number of the store in the current setting time zone are filled into the pre-constructed report template to generate an evaluation report of the store;

[0036] The acquisition logic of the customer flow improvement degree or the customer flow comparison model;

[0037] The customer flow performance coefficient of the store in the previous setting time zone is extracted from the historical correlation data as a reference performance coefficient;

[0038] The customer flow performance coefficient of the current setting time zone is compared with the reference performance coefficient. If the customer flow performance coefficient is higher than the reference performance coefficient, the ratio between them is calculated and recorded as the customer flow improvement degree;

[0039] If the customer flow performance coefficient is lower than the reference performance coefficient;

[0040] The customer flow performance coefficient of the current setting time zone is analyzed, and four line segments are emitted from the origin at equal angles, and the lengths of the four line segments correspond to the in-store rate, the conversion rate, the average in-store time, and the customer flow performance standardized value, respectively;

[0041] The emitting end points of the four line segments are connected in turn to form a quadrilateral, which is recorded as a performance graph;

[0042] Similarly, the quadrangle of the passenger flow performance coefficient of the previous set time zone is constructed as a comparison graph;

[0043] The origins of the comparison graph and the performance graph are aligned. After alignment, the line segment in the performance graph that is shorter than the line segment in the comparison graph is identified as a weak line segment. After the weak line segment is marked with a preset color, the final passenger flow comparison model is obtained.

[0044] Specifically, the logic for obtaining the regional heat ranking and the heat shelf number;

[0045] The regional heat ranking is generated by sorting the heat performance coefficients;

[0046] Extracting continuous frame images of each region video, obtaining shelf interaction data, including stay count, take count, and contribution turnover;

[0047] The stay count counts the number of customers whose bodies are facing the shelf and reach the reference time length. The proportion of the stay count of different shelf numbers in the total stay count of the belonging division area is calculated to obtain the attention proportion;

[0048] The take count counts the number of times the hand key point enters the commodity area. The proportion of the take count of different shelf numbers in the total take count of the belonging division area is calculated to obtain the contact proportion;

[0049] The contribution turnover of different shelf numbers is counted in the contribution turnover of the belonging division area as the contribution proportion;

[0050] The shelf heat coefficient is obtained by processing the attention proportion, the contact proportion, and the contribution proportion. The shelf with the highest heat coefficient in each region is marked as the heat shelf number.

[0051] Specifically, a video image analysis device comprises:

[0052] High-definition network camera: deployed at each node of the store to collect global video stream;

[0053] Solid state disk: stores anonymized video stream, structured trajectory data, and associated database;

[0054] WiFi6: realizes data transmission between devices and data linkage with the store POS system.

[0055] Specifically, a video image analysis system comprises:

[0056] Anonymized perception module: collects store video stream and generates unique anonymous ID;

[0057] Multi-source data acquisition module: based on the anonymous ID, extracts continuous moving trajectory, regional stay information, and product interaction micro-behavior;

[0058] Three-element association fusion module: construct a digital map of the store, bind the SKU of the goods and the position of the shelf, map the anonymous customer track, and establish an association database;

[0059] Multi-source data analysis: extract the association data in the current set time zone from the association database of the store for aggregation and mining, generate the customer flow performance coefficient of the store and the hot area performance coefficient of each divided area of the store;

[0060] Analysis report generation module: extract the shelf interaction data, calculate the attention, contact and contribution proportion for comprehensive processing to obtain the shelf heat coefficient, mark the heat shelf number, compare the current and historical customer flow performance coefficients, generate the customer flow improvement degree or comparison model, combine the area heat ranking, and generate the store evaluation report.

[0061] Technical effects and advantages of the present application:

[0062] (1) By extracting association data from store video image data to establish an association database, and establishing a quadrilateral comparison model to mark the weak customer flow dimension, generate the area heat ranking and heat shelf number, convert abstract video image data into visual results, and integrate these information to generate an evaluation report, the optimization direction is clear, a complete closed loop is formed, and the problem of single analysis dimension and insufficient depth of existing video image data is solved.

[0063] (2) By constructing a digital map of the store, binding the SKU of the goods and the position of the shelf, mapping the continuous track of anonymous customers, and establishing an association database, multi-level analysis from the store to the area and the shelf can be realized based on the database, the association between customer track, stay time and product interaction can be accurately traced, and the problem that existing video image analysis can only count surface indicators and cannot mine the core logic of operation is solved.

[0064] (3) By using the anonymous ID of the random hash value, only matching the de-identified contour and posture features across cameras, ensuring that the ID is unique and has no personal association information, and performing pixel blur and shielding processing on sensitive features such as faces, discarding the original biological data in real time, and only retaining the anonymous contour and environmental picture, the pain point of balancing data collection and compliance is solved while collecting complete video data. BRIEF DESCRIPTION OF DRAWINGS

[0065] Figure 1 A flowchart of a video image analysis method of the present application;

[0066] Figure 2 A schematic diagram of a video image analysis system of the present application. DETAILED DESCRIPTION

[0067] With reference to the drawings and embodiments of the present application, the technical solutions in the embodiments of the present application will be described clearly and completely. Obviously, the described embodiments are only a part of the embodiments of the present application, rather than all the embodiments of the present application. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative work fall within the scope of the present application.

[0068] Embodiment 1

[0069] As Figure 1 shown, a video image analysis method comprises:

[0070] Anonymization awareness: a camera network deployed at key nodes of a store (such as an entrance, a main aisle, a shelf area, and a cash register) synchronously collects video streams of the store. First, an anonymization tracking algorithm is run to generate a globally unique and random anonymous identifier for each customer detected.

[0071] Running the anonymization tracking algorithm:

[0072] Cross-frame tracking is performed on the human body target detected to establish intra-frame and inter-frame association.

[0073] For each customer whose tracking is successful, a globally unique and random anonymous ID (such as “ANON_8f7d29”, which is composed of a 32-bit random hash value and does not have any personal identity association information) is generated.

[0074] When switching between cameras, matching is performed through human appearance features (de-identified contours and posture features, without biological identification information) to ensure that the anonymous ID of the same customer is unique within the entire store.

[0075] For the human face, fingerprint, and other identifiable biological features in the human body area, “pixel blur + occlusion” processing (blur kernel size ≥ 5x5, and the occlusion area covers the entire range of the human face) is adopted.

[0076] The original video stream only retains the human body contours and the environment pictures after anonymization, and the original face frames and biological feature data are discarded in real time, without storing any information that can identify the personal identity reversely.

[0077] Multi-source data collection: based on the anonymized video stream data, a track tracking task is performed, and based on the anonymous identity identifier, a continuous moving track of the customer entering the store at the store entrance is generated.

[0078] The physical coordinates of each anonymous ID are recorded according to the timestamp (interval 0.5 seconds) to generate a continuous moving track.

[0079] Triadic association fusion: build a digital three-dimensional space map of the store ("field"), bind the SKU information of the goods with the physical shelf location ("goods"), map the trajectory of anonymous customers ("people") to the map, establish an association database, and record the continuous movement trajectory of each anonymous identifier;

[0080] Multi-source data analysis: extract the association data in the current set time zone (for example, a week) from the store's association database for aggregation and mining, generate the customer flow performance coefficient of the store and the hot zone performance coefficient of each divided area of the store;

[0081] Calculation logic of customer flow performance coefficient:

[0082] Extract the store entry rate, conversion rate, average in-store time, and customer flow performance in the set time zone from the store association data;

[0083] Predefine the entrance area of the store, analyze the video stream data of the entrance area, detect all anonymous identifiers and extract continuous movement trajectories;

[0084] Supplementary note, deduplication rule: the same temporary ID trajectory appears in the entrance area within 10 minutes, only counted once (avoiding repeated counting of passing through);

[0085] Statistical method: within the set period (such as 9:00-21:00 on the same day), count the number of all temporary IDs that meet the conditions, which is the "total number of people passing through the entrance area".

[0086] Set the division time, which can be set to 10s, if the continuous movement trajectory of an anonymous identifier in the entrance area reaches the division time in any divided area of the store, it is counted as an in-store customer;

[0087] For example, "at least 80% of the image trajectory points fall within the store divided area within 10 seconds", to avoid the case of short-term crossing and immediately leaving as a false judgment of entering the store;

[0088] At the same time, it is clear that abnormal in-store customers need to be excluded (such as "store employees, suppliers, and other non-consumer groups, which are filtered out through clothing features and fixed activity areas"; "customers with in-store time <1 minute and no area stay record, judged as false entry, not counted in in-store customers and average in-store time calculation").

[0089] Statistical in-store customer number in the set time zone of the store, the number of anonymous identifiers detected by the entrance area video stream data as the total number of flows;

[0090] Calculate the proportion of the number of in-store customers in the total number of flows to get the in-store rate of the store in the set time zone;

[0091] Extract the anonymous identity identifier of each customer in the number of in-store customers, and match it with the anonymous identity identifier of the purchase behavior in the set time zone, and record the number of matched identifiers as the consumption customers; (such as "associate POS system transaction record anonymous ID, and transaction time in the set time zone");

[0092] Calculate the proportion of the number of consumption customers in the number of in-store customers to get the conversion rate of the store in the set time zone;

[0093] Extract the in-store time of each customer in the number of in-store customers; the in-store time starts timing from the generation of the anonymous identity identifier when the customer enters the entrance area, and ends timing when the customer leaves the entrance area and is not in any divided area of the store;

[0094] And perform average value calculation to get the average in-store time of the store in the set time zone;

[0095] Take the current set time zone as the starting point, extract the number of in-store customers of the starting point X set time zones, and perform mean value calculation as the historical passenger flow; Where X is set by the management personnel, and X>3;

[0096] Take the number of in-store customers in the current set time zone as the numerator and the historical passenger flow as the denominator to perform ratio calculation to get the customer flow performance of the store in the set time zone;

[0097] Take the in-store rate, conversion rate, average in-store time and customer flow performance of the store in the current set time zone extracted from the associated data as input, and output the passenger flow performance coefficient after using the weighted calculation logic processing;

[0098] That is, the in-store rate, conversion rate, average in-store time and customer flow performance are respectively marked as 、 、 、 ; According to the formula + + Calculate the passenger flow performance coefficient; Where 、 and respectively represent the reference in-store rate, reference conversion rate and reference average in-store time;

[0099] Extract the in-store rate of the starting point X set time zones, and perform mean value calculation as the reference in-store rate;

[0100] Extract the conversion rate of the starting point X set time zones, and perform mean value calculation as the reference conversion rate;

[0101] Extract the average in-store time of the starting point X set time zones, and perform mean value calculation as the reference average in-store time;

[0102] 、 、 、 as the set weight coefficient.

[0103] The calculation logic of the hot area performance coefficient;

[0104] Clearly define the specific range of each divided area in the store (such as the snack area, fresh food area, clothing area, and cashier area), mark the independent pixel coordinates and physical boundaries of each area in the digital map, and ensure that there is no overlap and no blind area;

[0105] Associate area attribute information (such as area size, belonging category, and shelf number);

[0106] In the set time zone, count the number of anonymous identity identifiers of the trajectory points falling in each divided area (such as the snack area, fresh food area, and clothing area) of the store as the partition customer quantity;

[0107] Calculate the ratio of the partition customer quantity to the number of customers in the store to obtain the partition visit rate of each divided area in the set time zone; reflect the proportion of the divided area attracting customers into the store;

[0108] For the partition customer quantity in the divided area, respectively count the cumulative stay time (summed if multiple entries) in the divided area, and obtain the partition visit average time after average value calculation;

[0109] Calculate the ratio of the partition visit average time to the average in-store time to obtain the partition stay proportion of each divided area in the set time zone; reflect the time allocation proportion of the area in the total access time of the store;

[0110] Classify the anonymous identity identifiers that occur in the set time zone and have a purchase behavior according to the divided area of the goods to obtain the number of goods purchases of each divided area, and based on the amount of goods sold in each divided area, perform summation calculation to obtain the contribution turnover of each divided area;

[0111] Calculate the ratio of the number of goods purchases of the divided area to the total number of transactions in the store in the set time zone to obtain the contribution pen number proportion of the goods of each divided area;

[0112] Calculate the ratio of the contribution turnover of the divided area to the total turnover of the store in the set time zone to obtain the contribution amount proportion of the goods of each divided area;

[0113] Process the contribution pen number proportion and the contribution amount proportion of the goods using the weighted calculation logic to obtain the partition conversion coefficient in the set time zone;

[0114] That is, the weight coefficients of the commodity contribution pen number proportion and the commodity contribution amount proportion are set respectively, the commodity contribution pen number proportion and the commodity contribution amount proportion are multiplied by the corresponding set weight coefficients respectively, and then the sum is obtained to obtain the partition conversion coefficient.

[0115] The partition visit rate, the partition stay proportion, and the partition conversion coefficient of each division area in the current set time zone are taken as inputs, and the weighted calculation logic is used to process and output the hot area performance coefficient of each division area.

[0116] That is, the partition visit rate, the partition stay proportion, and the partition conversion coefficient are marked as , , respectively. The hot area performance coefficient is calculated according to the formula + ; wherein , and represent the reference partition visit rate, the reference partition stay proportion, and the reference partition conversion coefficient respectively.

[0117] The average calculation result of the partition visit rate of each division area is taken as the reference partition visit rate.

[0118] The average calculation result of the partition stay proportion of each division area is taken as the reference partition stay proportion.

[0119] The average calculation result of the partition conversion coefficient of each division area is taken as the reference partition conversion coefficient.

[0120] , , are the set weight coefficients.

[0121] Analysis report generation: based on the customer flow performance coefficient of the store and the hot area performance coefficient of each division area of the store, combined with historical correlation data and customer behavior data in video images, a comprehensive analysis is performed to generate an evaluation report of the store in the current set time zone.

[0122] Specifically:

[0123] The customer flow performance coefficient of the store in the last set time zone is extracted from the historical correlation data as the reference performance coefficient.

[0124] The customer flow performance coefficient of the current set time zone is compared with the reference performance coefficient. If the customer flow performance coefficient is higher than the reference performance coefficient, the ratio between the two is calculated and recorded as the customer flow improvement degree.

[0125] If the customer flow performance coefficient is lower than the reference performance coefficient.

[0126] Then the current set time zone passenger flow performance coefficient is analyzed, and four line segments are emitted from the origin at equal angles, and the lengths of the four line segments correspond to the store entry rate, conversion rate, average in-store time, and customer flow performance standardized values respectively;

[0127] The emitting end points of the four line segments are connected in turn to form a quadrilateral, which is referred to as a performance graph;

[0128] Similarly, the quadrilateral of the passenger flow performance coefficient of the previous set time zone is constructed as a comparison graph;

[0129] Align the origin points of the comparison graph and the performance graph. After alignment, identify the line segment in the performance graph that is shorter than the comparison graph as a weak line segment. After marking the weak line segment with a preset color, it is used as the final passenger flow comparison model. The preset color can be set to red;

[0130] It is supplemented that the passenger flow performance coefficient is decomposed into four core dimensions (four sides of the quadrilateral): store entry rate, conversion rate, average in-store time, and customer flow performance;

[0131] Compare the lengths of the quadrilateral line segments of the current and reference time zones, and directly mark the "weak line segment shorter than the reference" (red mark);

[0132] It realizes the precise penetration of "coefficient decline → positioning specific dimension → locking problem source".

[0133] Example: If the current coefficient is lower than the reference, and only the "conversion rate corresponding line segment" is a red weak line segment, it can be directly determined that "the core reason for the decline in passenger flow performance is low conversion efficiency, not the problem of flow or stay time", avoiding blind optimization.

[0134] Extract the performance coefficients of the hot zones of each divided area of the store, and sort them from large to small to obtain the regional heat ranking of the store in the current set time zone;

[0135] Extract the video data of each divided area, and extract the continuous frame images from the video data. Extract the shelf interaction data of each divided area from the continuous frame images. The shelf interaction data includes the stay count, take count, and contribution turnover of different shelf numbers in the divided area;

[0136] The stay count is obtained by counting the number of customers whose bodies are aligned with the shelf area in different shelf numbers in the continuous frame images, and the duration meets the reference duration;

[0137] Calculate the proportion of the stay count of different shelf numbers in the total stay count of the corresponding divided area to obtain the attention proportion;

[0138] Using a 2D / 3D human pose estimation model (such as OpenPose, AlphaPose, MMPose), real-time detection and output of key point coordinates of each human contour in each frame (such as head, shoulder, elbow, wrist, hip, knee, ankle, etc. 17-25 points), using a head pose estimation model or a simple line of sight inference model based on human key points, estimate the orientation vector of the head (yaw angle, pitch angle), note: here no eye tracking, only use rough head orientation as a proxy for line of sight, to protect privacy; Output: {key point coordinate set, head orientation vector} corresponding to each anonymous ID in each frame; When the ray of the head orientation vector of a certain anonymous ID intersects with the 3D bounding box of a certain shelf area in space, and lasts more than a threshold (such as 0.5 seconds), it is recorded as a "gaze event".

[0139] Pick-up count is obtained by counting the number of times the customer's hand enters the shelf product area in the continuous frame image;

[0140] The contact proportion is obtained by calculating the proportion of the pick-up count of different shelf numbers in the total pick-up count of the corresponding division area.

[0141] When the coordinates of the wrist or hand key points enter the 3D bounding box of a certain shelf product display area, and the hand opening and closing state is judged according to the relative position change of the hand key points (wrist, palm, fingertip), it is determined as a pick-up event.

[0142] The contribution proportion is obtained by calculating the proportion of the sales contributed by different shelf numbers in the sales contributed by the corresponding division area.

[0143] The attention proportion, contact proportion and contribution proportion of each shelf number in the current time zone in different division areas are comprehensively processed to obtain the shelf heat coefficient of each shelf number in different division areas.

[0144] That is, by setting the attention weight, contact weight and contribution weight of the attention proportion, contact proportion and contribution proportion, the attention proportion, contact proportion and contribution proportion of each shelf number are multiplied by the corresponding set weight respectively, and then summed to obtain the shelf heat coefficient.

[0145] The shelf number with the highest shelf heat coefficient in each division area is marked as the heat shelf number.

[0146] The customer flow improvement degree or customer flow comparison model of the store in the current time zone, the regional heat ranking and the heat shelf number are filled into the pre-constructed report template to generate an evaluation report of the store.

[0147] The above formulas are all dimensionless values calculated by taking the numerical value, and specific dimensionless can be standardized and other means, which will not be described here. The formula is obtained by collecting a large amount of data to simulate the most real situation. The preset parameters in the formula are set by the person skilled in the art according to the actual situation.

[0148] Embodiment 2

[0149] Please refer to Figure 2 As shown in the video image analysis method provided by Embodiment 1 of the present application, Embodiment 2 of the present application proposes a video image analysis system. Embodiment 2 is only a preferred way of Embodiment 1, and the implementation of Embodiment 2 will not affect the separate implementation of Embodiment 1.

[0150] Specifically, the video image analysis system provided by Embodiment 2 of the present application comprises:

[0151] Anonymization perception module: collecting store video stream, generating unique anonymous ID;

[0152] Multi-source data acquisition module: based on the anonymous ID, extracting continuous moving track, regional stay information and commodity interaction micro-behavior;

[0153] Ternary association fusion module: constructing a digital map of the store, binding the SKU of the commodity and the shelf position, mapping the anonymous customer track, and establishing an association database;

[0154] Multi-source data analysis: extracting the association data in the current set time zone from the association database of the store for aggregation and mining, generating the customer flow performance coefficient of the store and the hot zone performance coefficient of each divided region of the store;

[0155] Analysis report generation module: extracting shelf interaction data, calculating attention, contact and contribution proportion for comprehensive processing to obtain shelf heat coefficient, marking heat shelf number, comparing current and historical customer flow performance coefficient, generating customer flow improvement degree or comparison model, combining regional heat ranking, and generating store evaluation report;

[0156] Device composition:

[0157] High-definition network camera (1080P and above resolution, frame rate ≥25fps), infrared auxiliary camera (adapted to weak light environment), lens module (wide-angle lens covers the entrance / main channel, directional lens focuses on the shelf area / cashier desk);

[0158] Function: deployed at key nodes of the store (entrance, main channel, shelf area, cashier desk), synchronously collecting global video stream, supporting NTP time synchronization (timestamp error ≤10ms), ensuring cross-camera data time sequence consistency.

[0159] Edge computing chip, image processor;

[0160] Function: Run anonymization tracking algorithm, track human targets across frames in video streams, generate anonymous IDs (32-bit random hash values); perform "pixel blur + occlusion" processing on biological features such as faces and fingerprints (blur kernel ≥ 5x5), discard original biological feature data in real time, and only keep anonymized human contours and environmental pictures.

[0161] Data cache unit (RAM ≥ 8GB), feature extraction chip;

[0162] Function: Based on anonymous ID, collect continuous moving trajectories (record physical coordinates at 0.5 second intervals), extract human contours, postures, and other non-biological features; denoise trajectory data and behavior data (remove drift trajectory points), format standardization processing, and provide structured data for subsequent association and fusion.

[0163] Solid state drive (SSD ≥ 512GB), cloud storage interface module.

[0164] Function: Local storage of anonymized video streams, structured trajectory data, and association databases (person-goods-place association data), supports data retention according to compliance requirements (original video ≤ 24 hours, anonymized statistical data can be retained as needed); realize data synchronization backup and remote calling through cloud storage interface.

[0165] Core components: 5G / Gigabit Ethernet module, WiFi6 module, Bluetooth communication unit.

[0166] Function: Realize data transmission between devices (cross-camera trajectory synchronization), data linkage with store POS system / ERP system, push analysis results and evaluation reports to cloud management platform, support real-time data transmission delay ≤ 1 second.

[0167] The above-described embodiments can be implemented in part or in whole through software, hardware, firmware or any combination thereof. When implemented in software, the above-described embodiments can be implemented using one or more computer programs written in any suitable programming language. Such programs can be stored in one or more storage media or memory devices (e.g., a computer readable medium) associated with the computer or other suitable devices. The memory devices can include, but are not limited to, RAM, ROM, EEPROM, flash memory or other suitable memory devices. The computer programs can be loaded and / or executed on the computer or other suitable devices to produce a computer implemented process, such that the actions specified in the computer programs are performed. The computer programs can be executed on a single computer or on multiple computers.

[0168] It should be understood that the sequence of the above processes is not intended to mean the execution order of the processes, and the execution order of the processes should be determined according to the functions and inherent logic of the processes, and should not constitute any limitation on the implementation process of the embodiments of the present application.

[0169] Those skilled in the art can understand that the units and algorithm steps of the examples described in combination with the embodiments disclosed herein can be realized by electronic hardware or a combination of computer software and electronic hardware. Whether the functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of the present application.

[0170] In several embodiments provided in the present application, it should be understood that the disclosed system, device and method can be implemented in other ways. For example, the above-described device embodiments are merely illustrative, for example, the division of units is merely a logical function division, and actual implementation can have another division manner, for example, a plurality of units or components can be combined or integrated into another system, or some features can be ignored or not executed. In addition, the coupling or direct coupling or communication connection between the shown or discussed units can be indirect coupling or communication connection through some interfaces, devices or units, and can be electrical, mechanical or other forms.

[0171] The units described as separate components may or may not be physically separate, and the components displayed as units may or may not be physical units, which may be located in one place or distributed on multiple network units. Part or all of the units can be selected according to actual needs to achieve the purpose of the embodiment scheme.

[0172] In addition, the functional units in each embodiment of the present application can be integrated in one processing unit, or each unit can be physically present alone, or two or more units can be integrated in one unit.

[0173] The functions, if realized in the form of software functional units and sold or used as independent products, can be stored in a computer readable storage medium. Based on such understanding, the technical solutions of the present application or the part of the present application that essentially contributes to the prior art or the part of the technical solutions can be embodied in the form of a software product. The computer software product is stored in a storage medium and includes a plurality of instructions for causing a computer device (which can be a personal computer, a server, or a network device, etc.) to execute all or part of the steps of the method described in each embodiment of the present application. The aforementioned storage medium includes: a U disk, a mobile ATA hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk, and various program code storage media.

[0174] The above is only a specific implementation of the present application, but the protection scope of the present application is not limited thereto. Any person skilled in the art can easily think of changes or replacements within the technical scope disclosed in the present application, which should be covered within the protection scope of the present application. Therefore, the protection scope of the present application should be subject to the protection scope of the claims.

Claims

1. A method of video image analysis, characterized by, Comprise: S1: Collecting video data of the monitoring area of the store, generating a globally unique, random anonymous identity identifier for each customer detected in the video data; S2: Based on the anonymized video stream data, perform trajectory tracking tasks, based on the anonymous identity identifier, generate its continuous moving trajectory in the store entrance into the store; S3: Establish a digital map of the store space, and associate the product information, trajectory information and shelf location with the position in the digital map. Based on the associated fusion data, an association database is established; S4: Extract the associated data in the current setting time zone from the store association database for aggregation and mining, generate the customer flow performance coefficient of the store and the hot zone performance coefficient of each divided area of the store; S5: Based on the customer flow performance coefficient of the store and the hot zone performance coefficient of each divided area of the store, combined with historical association data and customer behavior data in video images, comprehensive analysis is carried out to generate an evaluation report of the store in the current setting time zone.

2. The video image analysis method of claim 1, wherein: The calculation logic of the customer flow performance coefficient in step S4; Extract the store rate, conversion rate, average time in store and customer flow performance in the setting time zone from the store association data; The store rate, conversion rate, average time in store and customer flow performance extracted from the association data in the current setting time zone are used as input, and the weighted calculation logic is used to process and output the customer flow performance coefficient.

3. The video image analysis method of claim 2, wherein: The logic for obtaining the store rate, conversion rate, average time in store and customer flow performance; Store rate: count the total number of people flowing into the entrance area and the number of customers entering the store meeting the preset conditions, and calculate the proportion of the two; Conversion rate: match the anonymous IDs of the customers entering the store and the customers consuming, and calculate the proportion of the customers consuming to the customers entering the store; Average time in store: extract the time in store of all customers entering the store, and calculate the average value; Customer flow performance: based on the historical X time zone average of customers entering the store, calculate the ratio of the current number of customers entering the store to the average.

4. The video image analysis method of claim 3, wherein: The calculation logic of the hot zone performance coefficient of each divided area in step S4; Obtain the partition visit rate, partition stay proportion and partition conversion coefficient of each divided area; The partition visit rate, partition stay proportion and partition conversion coefficient of each divided area in the current setting time zone are used as input, and the weighted calculation logic is used to process and output the hot zone performance coefficient of each divided area.

5. The video image analysis method of claim 4, wherein: The logic for obtaining the partition visit rate, partition stay proportion and partition conversion coefficient; Partition visit rate: count the number of customers in each area partition, divide by the total number of customers entering the store to get the partition visit rate; Partition stay proportion: calculate the average stay time of customers in each area, and divide by the average time in store of the store to get the partition stay proportion. Partition conversion coefficient: calculate the proportion of the number of pens contributed by each area and the proportion of the amount of money contributed, and then process to get the partition conversion coefficient.

6. The video image analysis method of claim 5, wherein the calculation logic of the proportion of the number of transactions contributed by the commodity and the proportion of the amount of money contributed by the commodity; The proportion of the number of transactions contributed by the commodity is calculated by taking the number of transactions of the commodity in the divided area as the numerator and the total number of transactions of the store as the denominator. The proportion of the amount of money contributed by the commodity is calculated by taking the contribution turnover of the divided area as the numerator and the total turnover of the store as the denominator.

7. The video image analysis method of claim 1, wherein the generation logic of the evaluation report in step S5; The evaluation report of the store is generated by filling the customer flow improvement degree or customer flow comparison model, regional heat ranking, and heat shelf number into the pre-constructed report template. The acquisition logic of the customer flow improvement degree or customer flow comparison model; The customer flow performance coefficient of the store in the previous setting time zone is extracted from the historical correlation data as the reference performance coefficient. The customer flow performance coefficient of the current setting time zone is compared with the reference performance coefficient. If the customer flow performance coefficient is higher than the reference performance coefficient, the ratio between the two is calculated and recorded as the customer flow improvement degree. If the customer flow performance coefficient is lower than the reference performance coefficient; The customer flow performance coefficient of the current setting time zone is analyzed. From the origin, four line segments are emitted at equal angles, and the lengths of the four line segments correspond to the in-store rate, conversion rate, average in-store time, and customer flow performance standardized values, respectively. The emitting end points of the four line segments are connected in turn to form a quadrilateral, which is recorded as the performance graph. Similarly, the quadrilateral of the customer flow performance coefficient of the previous setting time zone is constructed as the comparison graph. The origin of the comparison graph and the performance graph are aligned. The line segment in the performance graph that is shorter than the comparison graph is identified as the weak line segment. The weak line segment is marked with a pre-set color as the final customer flow comparison model.

8. The video image analysis method of claim 7, wherein the acquisition logic of the regional heat ranking and the heat shelf number; The regional heat ranking is generated by sorting the heat zone performance coefficients. The video continuous frame images of each region are extracted to obtain shelf interaction data, including stay count, take count, and contribution turnover. The stay count is the number of customers whose bodies are facing the shelf for a reference duration. The stay count of different shelf numbers in the total stay count of the corresponding divided area is calculated to obtain the attention proportion. The take count is the number of times the hand key point enters the commodity area.

9. A video image analysis apparatus for use in a video image analysis method according to any one of the preceding claims 1-8, characterized by The take count of different shelf numbers in the total take count of the corresponding divided area is calculated to obtain the contact proportion. The contribution proportion of the turnover contributed by different shelf numbers in the contribution turnover of the corresponding divided area is calculated. The shelf heat coefficient is obtained by processing the attention proportion, contact proportion, and contribution proportion. The shelf with the highest heat coefficient in each region is marked as the heat shelf number.

10. A video image analysis system for use in a video image analysis method according to any one of the preceding claims 1-8, characterized by Including: High-definition network camera: deployed at each node of the store to collect global video stream; Solid state drive: stores anonymized video stream, structured trajectory data, and correlation database; WiFi6: realizes data transmission between devices and data linkage with the store POS system. Including: Anonymization perception module: collect store video stream, generate unique anonymous ID; Multi-source data acquisition module: based on anonymous ID, extract continuous moving track, regional stay information and commodity interaction micro behavior; Three-element association fusion module: construct store digital map, bind commodity SKU and shelf location, map anonymous customer track, establish association database; Multi-source data analysis: extract association data in current setting time zone from store association database for aggregation and mining, generate store customer flow performance coefficient and store each divided region's hot zone performance coefficient; Analysis report generation module: extract shelf interaction data, calculate attention, contact and contribution proportion for comprehensive processing to obtain shelf heat coefficient, mark heat shelf number, compare current and historical customer flow performance coefficient, generate customer flow improvement degree or comparison model, combine regional heat ranking to generate store evaluation report.

Images