Statistical method for moving direction of entity object, electronic device and storage medium

Abstract

The application provides a method for counting the moving direction of an entity object, an electronic device and a storage medium. According to the embodiment of the application, the video stream collected by the camera deployed in the target area is subjected to entity object trajectory detection to generate the first action trajectory of each entity object in the camera coordinate system, the first action trajectory is converted to the second action trajectory in the electronic map coordinate system of the target area, the moving direction and the heat of each entity object are counted according to the distribution of the second action trajectory in the multiple grids evenly divided in the electronic map, and the heat map representing the moving direction of the multiple entity objects in the target area is generated and output according to the moving direction and the heat of the multiple entity objects, so that the moving analysis of the entity object can be performed by referring to the heat map. The application realizes the alignment of the trajectory from the local perspective of the camera to the physical space of the target area, and provides the spatially uniform data basis for the subsequent grid direction counting and heat map generation.

Description

Technical Field

[0001] This application relates to the field of data analysis technology, and in particular to a statistical method for determining the direction of movement of an entity, an electronic device, a storage medium, and a computer program product. Background Technology

[0002] With the rapid development of new retail and smart stores, businesses increasingly need a refined understanding of customer behavior within their stores to support optimized product layout, guided shopping routes, and adjustments to marketing strategies. Analyzing the activity trajectories and movement paths of customers within a given space helps identify high-traffic areas, determine customer flow relationships between shelves, optimize customer flow design, and improve marketing conversion efficiency. Among numerous application scenarios, customer flow analysis is one of the most typical and direct needs.

[0003] To meet the above needs, existing technologies have developed a variety of customer flow analysis solutions. One solution uses infrared sensors, Wi-Fi probes, or single-camera counting to count the number of people entering and leaving the area. Although this solution can count the number of customers, it cannot depict the complete movement path and directional preferences of customers, making it difficult to support refined operational decisions. Another solution can introduce visual technology to detect the direction of customer movement through image recognition of a single camera. However, due to the limited coverage of a single camera, it cannot achieve comprehensive analysis of larger areas such as stores. Summary of the Invention

[0004] This application provides a statistical method, electronic device, storage medium, and computer program product for determining the movement direction of an entity object, in order to solve one or more of the aforementioned technical problems.

[0005] In a first aspect, embodiments of this application provide a method for statistically analyzing the movement direction of entity objects, comprising: performing entity object trajectory detection on video streams captured by cameras deployed in a target area, generating a first movement trajectory of each entity object in the target area under the coordinate system of the cameras; converting the first movement trajectory to a second movement trajectory of the entity objects under the coordinate system of an electronic map of the target area, wherein the electronic map is uniformly divided into multiple grids; statistically analyzing the movement direction and heat of multiple entity objects in the electronic map based on the distribution of the second movement trajectory in the grids; wherein the movement direction and heat of multiple entity objects in the grids are determined based on the frequency of occurrence of different movement directions; generating a heat map characterizing the movement direction of the multiple entity objects in the target area based on the movement direction and heat of multiple entity objects in the electronic map and outputting and displaying it, so as to perform entity object movement analysis by referring to the heat map.

[0006] Secondly, embodiments of this application provide an electronic device, including a memory, a processor, and a computer program stored in the memory, wherein the processor implements the above-described method when executing the computer program.

[0007] Thirdly, embodiments of this application provide a computer-readable storage medium storing a computer program that, when executed by a processor, implements the above-described method.

[0008] Fourthly, embodiments of this application provide a computer program product, wherein the computer program product includes a computer program that, when executed by a processor, implements the above-described method.

[0009] According to the embodiments of this application, entity object trajectory detection is performed on the video streams collected by cameras deployed in the target area to generate a first movement trajectory for each entity object in the camera coordinate system. The first movement trajectory is then converted to the electronic map coordinate system of the target area to obtain a second movement trajectory. Based on the distribution of the second movement trajectory in multiple grids evenly divided in the electronic map, the movement direction and heat of multiple entity objects are statistically analyzed. The movement direction and heat of multiple entity objects in the grid are determined by the frequency of occurrence of different movement directions. Then, a heat map of movement direction is generated and output based on the movement direction and heat of multiple entity objects, so as to perform entity object movement analysis by referring to the heat map. It can be seen that the embodiments of this application first convert the movement trajectory detected under the camera's field of view into the movement trajectory in the electronic map coordinate system by registering between pixel coordinates and store map coordinates. This achieves trajectory alignment from the local viewpoint of the camera to the physical space of the target area, providing a unified data foundation for subsequent gridded direction statistics and heat map generation, overcoming the problem of the analysis results being disconnected from the physical spatial layout in traditional single-camera solutions. Secondly, the electronic map is evenly divided into multiple grids. Based on the distribution of the second movement trajectory in each grid, the movement direction of each entity object in the electronic map is statistically analyzed. A directional heat map is generated based on the movement direction data of all grids, providing data-driven visualization decision support for the transformation of the target area's traffic flow, layout optimization, and the setting of marketing points.

[0010] Furthermore, this application's embodiments extract the appearance features of entities from video streams captured by multiple cameras, identify the same entity under different cameras based on the similarity of appearance features, extract the second movement trajectory corresponding to the entity under each camera after conversion, and stitch the extracted trajectory segments together to reconstruct the entity's complete movement trajectory across cameras. This solves the problem of trajectory breakpoints and data fragmentation caused by identity switching between multi-camera coverage areas, providing a continuous and complete trajectory data foundation for full-site movement direction statistics and path analysis. In addition, this application's solution only relies on existing security or surveillance cameras in the store, without the need for additional deployment of dedicated hardware such as infrared or Wi-Fi probes, and has the ability to be low-cost, easy to promote, and replicable across stores.

[0011] The above description is only an overview of the technical solution of this application. In order to better understand the technical means of this application, it can be implemented according to the contents of the specification. In order to make the above and other objects, features and advantages of this application more obvious and understandable, specific embodiments of this application are given below. Attached Figure Description

[0012] In the accompanying drawings, unless otherwise specified, the same reference numerals throughout the various drawings denote the same or similar parts or elements. These drawings are not necessarily drawn to scale. It should be understood that these drawings depict only some embodiments according to this application and should not be construed as limiting the scope of this application.

[0013] Figure 1 A schematic diagram of a heat map is shown in one example of an embodiment of this application; Figure 2 A schematic diagram illustrating the determination of the direction of movement is shown in an example of an embodiment of this application; Figure 3 A flowchart is shown for a statistical method for the movement direction of an entity object provided in an embodiment of this application; Figure 4 A flowchart of a customer flow analysis method in a physical store provided in an embodiment of this application is shown; Figure 5 This paper shows a structural block diagram of a statistical device for the movement direction of an entity object provided in an embodiment of this application; Figure 6 This application illustrates a structural block diagram of a customer movement analysis device for physical stores, as provided in an embodiment of this application; and Figure 7 A block diagram of an electronic device used to implement embodiments of this application is shown. Detailed Implementation

[0014] In the following description, only certain exemplary embodiments are briefly described. As those skilled in the art will recognize, the described embodiments can be modified in various ways without departing from the concept or scope of this application. Therefore, the drawings and description are considered to be exemplary in nature and not restrictive.

[0015] To facilitate understanding of the technical solutions of the embodiments of this application, the relevant technologies of the embodiments of this application are described below. The following relevant technologies are optional solutions and can be combined with the technical solutions of the embodiments of this application in any way, and all of them fall within the protection scope of the embodiments of this application.

[0016] This application addresses the need to statistically analyze the movement direction of physical objects in offline areas, designing a scheme for this purpose. The offline area can be any physical space, including stores, shopping malls, exhibition halls, schools, farms, warehouses, factory workshops, airports, and subway stations, and is not limited to commercial retail scenarios. It can be broadly extended to any location requiring directional analysis and path tracking of moving objects. The physical objects can be human objects such as pedestrians, exhibition participants, and students in stores or shopping malls, or livestock in farms, and handling robots, automated guided vehicles, unmanned delivery vehicles, or mobile devices in warehouses or picking areas. This scheme enables unified modeling and statistical analysis of the movement direction of different types of physical objects within a space, meeting the direction perception and decision support needs in diverse application scenarios such as smart retail, smart security, smart agriculture, industrial logistics, and public safety. Examples include analyzing customer movement to optimize shelf layout, monitoring robot paths to improve handling efficiency, and statistically analyzing crowd evacuation directions to assess safety risks.

[0017] One or more cameras are deployed in the target area, each responsible for capturing real-time video streams within its field of view. By performing object trajectory detection on the video streams captured by these cameras, the movement trajectory of each object in its respective camera coordinate system, i.e., the first movement trajectory, can be generated. The generated first movement trajectory consists of a series of continuous pixel coordinate points (e.g., the coordinates of the object's feet or center point in each frame) arranged in chronological order, completely recording the movement process of the object within the camera's coverage area. At this point, this trajectory data is still in the camera's own pixel space coordinate system, with its coordinate unit being pixels, and has not yet established a correspondence with the actual physical space coordinates of the target area.

[0018] Specifically, the process begins by calling an object detection model to detect entities in the video stream captured by the deployed camera frame by frame. This step aims to locate the positions of all entities in each frame, typically outputting the bounding box coordinates (e.g., the positions of the top-left and bottom-right corners of the bounding box in pixel coordinates) and their category confidence for each entity. This detection step converts the pixel information in the raw video stream into structured entity location data, providing input for subsequent tracking. Then, a tracking algorithm is invoked to track the trajectories of the detected entities in each frame. Detection results belonging to the same entity are correlated and matched in the time dimension to generate the first movement trajectory of that entity within the corresponding camera's field of view. The tracking algorithm utilizes the positional change patterns of entities between adjacent frames to concatenate discrete detection results into a continuous movement path, assigning each entity a temporary identity under that camera's view.

[0019] In one implementation, a single-stage object detection model (such as YOLO v8) can be used to perform pedestrian detection frame-by-frame on the video stream captured by the camera. YOLO v8 is a convolutional neural network model based on single-stage object detection. Its core principle is to divide the input image into multiple grids, each grid being responsible for predicting whether an object exists within its region, as well as the object's bounding box location and class probability. Through a single forward inference, the object's location and classification can be completed simultaneously, enabling fast and accurate identification of the location of entities in each frame, providing reliable detection input for subsequent trajectory tracking. YOLOv8 is suitable for real-time processing of multiple video streams, offering fast detection speed and high detection accuracy. It maintains good recognition performance even in complex backgrounds and occluded scenes. Furthermore, its lightweight model facilitates deployment on edge computing devices or servers, making it suitable for scenarios requiring low-cost, high-real-time deployment, such as retail stores.

[0020] After detecting the bounding boxes of entity objects in each frame, DeepSORT (Deep Simple Online and Realtime Tracking) is further used to generate the movement trajectory of the entity objects within the field of view of a single camera. Specifically, DeepSORT first uses Kalman filtering to predict the state of each known trajectory in the current frame. Then, it performs a concatenated matching between the bounding boxes of entity objects detected by YOLOv8 in the current frame and the predicted positions. During the matching process, both the intersection-union ratio (IUU) between bounding boxes and the cosine distance of the entity object's appearance feature vectors are considered, thus associating the detection results with existing trajectories. By extracting the appearance features of each entity object (such as color and body shape), it can maintain the continuity of identity recognition even in complex situations such as mutual occlusion or reappearance after briefly leaving the camera's field of view, avoiding trajectory breaks and identity switching. Using the DeepSORT tracking algorithm, through joint matching of appearance features and motion information, can significantly improve the tracking robustness in occluded and intersecting scenes. In addition, during trajectory generation, the bottom center point of the pedestrian bounding box in each frame can be taken as the foot point (i.e. the contact point between the pedestrian and the ground). This point is more geometrically stable in perspective transformation than other positions in the bounding box (such as the center point or the top point), and is more suitable for subsequent homography mapping from pixel coordinates to physical coordinates. This can effectively reduce position fluctuations caused by changes in pedestrian posture and improve the accuracy of coordinate transformation.

[0021] The first trajectory is transformed into the physical coordinate system of the electronic map of the target area through coordinate mapping, generating the second trajectory. This involves transforming the sequence of position points of the entity object in the camera's pixel coordinate system using a pre-calibrated mapping matrix to calculate the physical coordinates of each point on the electronic map. After this transformation, the entire trajectory is migrated from pixel space to physical space, forming the second trajectory. Each coordinate point in this trajectory corresponds to its actual position in the target area, achieving trajectory alignment from the camera's local viewpoint to the physical space of the target area. This provides a unified spatial reference data foundation for subsequent gridded direction statistics and heatmap generation.

[0022] Mapping from the camera coordinate system to the target area's electronic map coordinate system can be achieved using a mapping transformation matrix (such as a homography transformation matrix). This matrix describes the projection mapping relationship between the image pixel coordinate system and the target area's electronic map physical coordinate system. To achieve this mapping, each camera needs to be spatially calibrated beforehand to establish the correspondence between the camera image coordinate system and the target area's electronic map coordinate system. A standard object with a known physical location can be selected first, and used as a spatial reference point to solve for the coordinate mapping relationship between the camera image and the electronic map. By acquiring test images of the standard object captured by the camera, and based on the standard object's pixel coordinates in the test image and its physical coordinates in the store's electronic map coordinate system at the same time, the mapping transformation matrix between the camera coordinate system and the electronic map coordinate system can be calculated. In one implementation, a self-moving standard object (such as an inspection robot or automated guided vehicle within the area) can be set up, which can report its physical coordinates in the map coordinate system in real time. By acquiring test images of the standard object from each camera, the pixel coordinates of the standard object in the image are identified. Combined with the physical coordinates reported by the standard object at the same time, the mapping transformation matrix between the camera coordinate system and the target area's electronic map coordinate system can be calculated. Alternatively, at least four non-collinear reference points (e.g., QR code labels or dedicated calibration boards) can be selected on the ground in the target area. The physical coordinates of these reference points in the target area's electronic map are pre-measured and known. Then, test images containing these reference points are acquired by each camera, and the coordinates of each reference point in the pixel coordinate system are marked in the images. For each camera, the pixel coordinates of the same reference point are paired with the known physical coordinates to form a corresponding point pair. Using at least four non-collinear point pairs, the mapping transformation matrix between the camera coordinate system and the target area's electronic map coordinate system is calculated by solving the homography transformation matrix.

[0023] The electronic map of the target area is uniformly divided into multiple grids, with each grid serving as an independent statistical unit. The purpose of this is to discretize the continuous map space into a finite number of grid cells, allowing the movement trajectory of each entity to be decomposed and statistically analyzed according to the grids it traverses. This transforms the trajectory analysis problem into a directional statistical problem within the grid. Taking a store as an example, the map can be divided into several uniformly sized cells using a 1m x 1m grid. The 1m x 1m grid size matches the stride of a human walk, ensuring the rationality of directional determination, avoiding the loss of directional information due to excessively large grids, and preventing excessive computational redundancy due to overly fine grids.

[0024] Based on the distribution of the second movement trajectory in the grid, the movement direction and popularity of each entity object in the electronic map can be statistically analyzed. Specifically, for each entity object, the movement direction from the previous grid to the next grid is determined based on the second movement trajectory. Specifically, the direction of movement from one grid to another can be determined based on adjacent points on the second movement trajectory. For example, moving from grid A to adjacent grid B can be calculated based on the direction of the vector of the line connecting the grid center points or the trajectory segment.

[0025] After tracking the movement directions of each entity object from the previous grid to the next, for each grid, which is then designated as the current grid, the movement directions of multiple entities from the current grid to the next grid are tracked. Since the movement directions of different entities from the current grid to the next grid are not entirely consistent, movement direction data of all entities passing through that grid can be collected. Then, based on the frequency of occurrence of different movement directions, a target movement direction is selected as the movement direction of multiple entities within that current grid. In one example, statistical methods such as direction voting or direction histograms can be used to determine the dominant movement direction within that grid. For instance, by statistically analyzing the angular distribution of all trajectory directions, the direction with the highest frequency is selected as the dominant direction. After determining the movement directions of multiple entities within the current grid, the frequency of occurrence of different movement directions is used to determine the "popularity" of multiple entities within the current grid. Popularity is positively correlated with the frequency of occurrence; that is, the higher the frequency of occurrence, the more entities choose that movement direction, and the higher the popularity of that movement direction. Conversely, the lower the frequency of occurrence, the lower the popularity. The specific correspondence between frequency and heat can be determined based on a pre-defined numerical mapping table or calculated using a set formula; this application does not impose any restrictions on this. Through the above statistics, this scheme converts continuous trajectory data into discrete directional information and heat on each grid, providing a data foundation for subsequently generating a full-field movement direction heatmap. The heatmap is then output and displayed, allowing for the analysis of entity movement by referring to it. This gridded directional statistical method can intuitively reflect the mainstream flow direction at different locations within the target area, helping to identify group movement, bottleneck areas, and abnormal movement patterns.

[0026] Specifically, when determining the movement direction of an entity object from one grid cell to the next based on the second movement trajectory, the relative positional relationship between the position point falling into the previous grid cell and the position point in the next grid cell can be determined for each entity object according to the timestamp information of the position points in the second movement trajectory. Specifically, the relative positional relationship can be constructed using the direction vector from the previous grid cell to the next grid cell, representing the relative positional relationship as a direction vector from the previous grid cell to the next grid cell. Specifically, first, determine the grid cell that a position point on the trajectory falls into, denoted as the previous grid cell. Then, determine the grid cell that the next position point after that position point falls into, denoted as the next grid cell. Next, based on the position of the previous grid cell (e.g., its center point coordinates) and the position of the next grid cell, calculate the direction vector from the previous grid cell to the next grid cell. This direction vector is the movement direction of the entity object between the previous and next grid cells. Based on the determined relative positional relationship, the movement direction of the next grid cell relative to the previous grid cell can be determined, such as one of the four main directions: up, down, left, or right. By repeating the above process for each pair of adjacent grid jumps in the trajectory, the entire trajectory can be decomposed into a series of directional movement sequences between grids, providing basic data for subsequent directional statistics within each grid.

[0027] Figure 2 This diagram illustrates an example of determining the direction of movement in an embodiment of this application. Assume a customer's second movement trajectory within a store passes through grid A (center coordinates (0,0)) and grid B (coordinates (1,1)). First, the landing order of the position points is determined to be first in grid A, then in grid B. Then, based on the center coordinates of grids A and B, the direction vector from the previous grid to the next is calculated as (1-0, 1-0) = (1,1), corresponding to the upper right direction. Therefore, the customer's movement direction from grid A to grid B is determined to be upper right. If the customer continues to move to grid C (coordinates (1,0)) at the next moment, the direction vector from grid B to grid C is (0,-1), corresponding to the downward direction. In this way, the entire trajectory is decomposed into a series of directional movement sequences, such as grid A → grid B (upper right) and grid B → grid C (downward), for subsequent direction statistics within each grid. For example, at grid B, record one "entry from the upper right direction of grid A" and one "departure from the lower direction towards grid C" to determine the dominant movement direction of that grid.

[0028] For each grid, the movement direction of each entity from the current grid to the next grid is counted. For example, the direction from grid A to grid B is upper right, and the direction from grid A to grid C is right, etc. Then, a target movement direction is selected based on the frequency of occurrence of different movement directions. That is, the frequency of each movement direction within the grid is counted, and the target movement direction is selected as the movement direction of multiple entities within the current grid. Finally, based on the frequency of occurrence of different movement directions, the movement direction with the most occurrences is selected as the target movement direction for that grid. This is a direction voting mechanism to determine the dominant movement direction within each grid. Specifically, for each grid, the movement direction records of all entities from that grid to adjacent grids are counted. Each direction is counted as one vote, and the direction with the most votes is the target movement direction for that grid, representing the dominant flow direction of multiple entities within that grid. Through this direction voting mechanism, this scheme can aggregate grid-level dominant direction information from the discrete movement behavior of individuals, providing a statistical basis for generating a full-field movement direction heatmap. For example, if grid A records 10 movement directions, with the upper right direction appearing 6 times, the rightward direction appearing 3 times, and the upward direction appearing once, then the upper right direction is selected as the movement direction for grid A. Through the above statistical method, this scheme can aggregate discrete individual movement directions into grid-level dominant direction information, providing a data foundation for subsequently generating a full-field movement direction heatmap.

[0029] In determining the direction of movement from the current grid to the next grid, a Cartesian coordinate system can be constructed with the center of the current grid as the origin. A straight line connecting the center points of the current grid and the next grid is formed, with the direction of the line from the current grid to the next grid. In one definition, the direction of the line falling into the first quadrant is the upper right direction, falling into the second quadrant is the upper left direction, falling into the third quadrant is the lower left direction, and falling into the fourth quadrant is the lower right direction. In another definition, only four directions (up, down, left, and right) are used: the direction between the angle bisectors of the first and second quadrants is the upward direction, the direction between the angle bisectors of the second and third quadrants is the left direction, the direction between the angle bisectors of the third and fourth quadrants is the downward direction, and the direction between the angle bisectors of the first and fourth quadrants is the right direction. In practice, more detailed divisions can be made, and this application does not impose any restrictions on this.

[0030] When generating a heatmap representing the movement direction of multiple entities in a target area based on their movement direction and popularity on an electronic map, for each grid, the direction of the arrow representing the movement direction in the heatmap is determined based on the movement direction of the multiple entities on the electronic map. The size of the arrow representing the movement direction is determined based on the popularity of the entity on the electronic map. The arrow size is positively correlated with the popularity, that is, the greater the popularity, the larger the arrow, and the smaller the popularity, the smaller the arrow. Further, based on the direction and size of the arrow, a heatmap representing the movement direction of multiple entities in the target area can be generated. Through this popularity mapping mechanism, the heatmap can intuitively reflect the mainstream movement direction and its flow intensity at different locations within the target area, helping operators quickly identify high-flow channels and low-flow areas.

[0031] like Figure 1 The image shows a heatmap illustration illustrating the internal layout of a fresh food supermarket or retail store. The main areas marked on the map include: vegetable section, fruit section, seafood section, standard product section, flower section, catering section, dining area, bakery section, customer service area, checkout area, and processing areas such as fruit processing area, seafood processing area, catering processing area, refrigerated pre-baked fresh (raw) food processing area, refrigerated daily delivery area, and large frozen area. This layout allows stores to analyze customer flow distribution in each area, such as tracking customer traffic and movement directions in areas like the vegetable, fruit, seafood, catering, and checkout areas. The arrows in the image indicate the main movement direction within each grid cell, and the size or width of the arrow indicates the frequency of movement in that direction. For example, a thicker or larger arrow indicates a higher number of customers passing through that grid cell or a higher frequency of directional movement. Through this visualization, operators can intuitively identify the main flow channels, bottleneck areas, return flow areas, and cold areas, providing data support for optimizing customer flow, adjusting shelf space, and setting up marketing points.

[0032] Based on the dominant movement directions within each grid obtained through statistics, a movement direction heatmap is generated on an electronic map in a visual manner. Specifically, the heatmap is drawn in grid units, with each grid using graphic elements such as arrows, color blocks, or directional lines to represent the dominant movement direction. The frequency or intensity of movement in that direction can be indicated by color intensity or arrow thickness; for example, grids with more frequent movement have darker colors or thicker arrows. The heatmap generated in this way can intuitively display the distribution of mainstream movement directions at different locations within the entire target area, such as which areas are the main channels for passenger flow, and which areas exhibit backflow or directional dispersion. This heatmap can be output to display devices or a management backend, allowing operations personnel to quickly identify passenger flow patterns and providing visualized data-driven decision-making support for spatial layout optimization, traffic flow design adjustments, and marketing strategy formulation.

[0033] In one embodiment, multiple cameras are deployed in the target area. In a multi-camera deployment scenario, the trajectory of the same entity can be stitched together across cameras. Specifically, when multiple cameras are deployed in the target area, each camera detects and generates a first motion trajectory within its own field of view. By using pedestrian re-identification technology to determine whether the entity under different cameras is the same entity, the second motion trajectory segments of these entities under each camera, after coordinate transformation, can be stitched together in chronological order to form a continuous and complete second motion trajectory of the entity in the entire target area. Through this trajectory stitching mechanism, this solution solves the problem of trajectory fragmentation in multi-camera coverage scenarios, ensuring the integrity of the entity's trajectory throughout the entire field.

[0034] Specifically, visual features of entities can be extracted from video streams captured by multiple cameras, such as color, texture, shape, clothing style, and carried items. Then, the similarity between the appearance features of entities under different cameras is calculated. When the similarity exceeds a preset threshold, the entities under these cameras are identified as belonging to the same entity, and a globally unique identifier is assigned to it. Next, the second movement trajectory of the entity under each camera, after coordinate transformation, is extracted. Finally, these trajectory segments are stitched together in chronological order, merging adjacent and temporally continuous position points to form a continuous and complete second movement trajectory of the entity within the target area. Through the above appearance feature matching and trajectory stitching mechanism, this solution achieves cross-camera identity consistency association and full-field trajectory reconstruction.

[0035] In the initial trajectory generation stage, for each camera's video stream, each identified entity is assigned a first identity identifier (e.g., ID-01 in camera A, ID-02 in camera B, etc.) valid within that camera's field of view (i.e., unique within a single camera's range). This identifier is used for trajectory tracking and association within that camera's range. However, the first identity identifiers from different cameras are independent and cannot directly correspond to the same entity. In the subsequent trajectory stitching stage, after identifying entities from different cameras as belonging to the same entity through appearance feature similarity, a globally unique second identity identifier is reassigned to that entity (i.e., unique within multiple cameras' ranges). This second identifier is uniformly valid across cameras and is used to identify the entity's complete identity throughout the entire target area. Through these two identity identifier allocation mechanisms, both the efficiency of trajectory tracking within a single camera and the unified association of identities across cameras and the complete reconstruction of the entire trajectory are achieved.

[0036] In one embodiment, the trajectory stitching of the second movement trajectory can be performed at the camera switching point, which refers to the location where a pedestrian walks out of the field of view of one camera and enters the field of view of the next camera. Specifically, during the matching process, for each camera, firstly, entity objects located at the edge of the current camera's coverage area at different times are identified (denoted as first candidate entity objects). This means detecting entity objects that have moved to the edge of their coverage area within a single camera. Based on the movement direction of these entity objects, the neighboring camera associated with the current camera is determined, i.e., the next camera that is spatially adjacent and directionally related. Then, using the moment when the first candidate entity object is at the edge as the current moment, the next moment (which can be set as needed, e.g., the current moment is 15 minutes and 20 seconds, the next moment is 21 seconds) is used to identify one or more entity objects located at the edge of the coverage area of ​​the neighboring camera as second candidate entity objects. These second candidate entity objects are considered as options that may be the same entity object as the first candidate entity object after crossing camera boundaries. Further, pedestrian re-identification (ReID) technology is used to calculate the similarity of the appearance features of the first and second candidate entity objects before and after the comparison. If the similarity exceeds a preset threshold, they are determined to be the same pedestrian. After a successful match, the pedestrian's movement trajectory segments, which have been converted to map coordinates, are stitched together from the two cameras to form a continuous and complete trajectory across the cameras.

[0037] After identifying the first and second candidate entity objects, their appearance features are extracted, and the similarity between these features is calculated (e.g., cosine similarity or Euclidean distance). Based on this similarity, the second candidate entity object, corresponding to the first candidate entity object, is determined. For example, the calculated similarity value can be compared with a preset similarity threshold. When the similarity exceeds this threshold, the second candidate entity object and the first candidate entity object are determined to be the same entity object.

[0038] The execution entity in this application embodiment can be an application, service, instance, functional module in software form, virtual machine (VM), container, or cloud server, or hardware device with data processing function (such as server or terminal device) or hardware chip (such as CPU, GPU, FPGA, NPU, AI accelerator card, or DPU). The device for providing the service can be deployed on the computing device of the application providing the corresponding service or on a cloud computing platform providing computing power, storage, and network resources. The cloud computing platform can provide services in the following modes: IaaS (Infrastructure as a Service), PaaS (Platform as a Service), SaaS (Software as a Service), or DaaS (Data as a Service). Taking the platform providing SaaS (Software as a Service) as an example, the cloud computing platform can utilize its own computing resources to provide the execution of one or more of the above steps, and the specific application architecture can be built according to service requirements.

[0039] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, data stored, data displayed, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties. Furthermore, the collection, use and processing of the relevant data must comply with the relevant laws, regulations and standards of the relevant countries and regions, and corresponding operation entry points are provided for users to choose to authorize or refuse.

[0040] The technical solution of this application and how it solves the aforementioned technical problems are described in detail below with specific embodiments. The listed specific embodiments can be combined with each other, and the same or similar concepts or processes may not be described again in some embodiments. The embodiments of this application will be described in detail below with reference to the accompanying drawings.

[0041] This application provides a statistical method for the movement direction of an entity object, such as... Figure 3The diagram shows a flowchart of a statistical method 300 for determining the movement direction of entities according to an embodiment of this application. This method 300 may include: in step S301, performing entity trajectory detection on video streams captured by cameras deployed in a target area to generate a first movement trajectory of each entity in the target area under the coordinate system of the cameras; in step S302, converting the first movement trajectories to a second movement trajectory of the entity in the coordinate system of an electronic map of the target area, wherein the electronic map is uniformly divided into multiple grids; in step S303, statistically analyzing the movement direction and heat of multiple entities in the electronic map based on the distribution of the second movement trajectories in the grids; wherein the movement direction and heat of multiple entities in the grids are determined based on the frequency of occurrence of different movement directions; in step S304, generating and outputting a heat map characterizing the movement direction of the multiple entities in the target area based on the movement direction and heat of the multiple entities in the electronic map, so as to perform entity movement analysis by referring to the heat map.

[0042] In one embodiment, the cameras deployed in the target area include multiple cameras, and the method further includes: stitching together the second movement trajectories corresponding to the first movement trajectories of the same entity object under different cameras to obtain the continuous second movement trajectories of the entity object in the target area.

[0043] In one embodiment, the step of stitching together the second action trajectory corresponding to the first action trajectory of the same entity object under different cameras includes: extracting the appearance features of the entity object from the video streams captured by multiple cameras, identifying the same entity object in the video streams of different cameras based on the similarity of the appearance features of the entity object; extracting the second action trajectory corresponding to the same entity object after conversion, and stitching together the extracted second action trajectory.

[0044] In one embodiment, after generating the first movement trajectory of each entity object in the target area under the coordinate system of the camera, the method further includes: assigning a first identity identifier unique to each individual camera range to the identified entity object for each camera's video stream; after extracting the second movement trajectory corresponding to the same entity object after transformation, the method further includes: reassigning a second identity identifier unique to multiple camera ranges to the entity object, so that the same entity object corresponds to the same second identity identifier.

[0045] In one embodiment, before extracting the appearance features of the entity object from the video streams captured by multiple cameras, the step of stitching together the second movement trajectory corresponding to the first movement trajectory of the same entity object under different cameras further includes: for each camera, finding a first candidate entity object located at the edge of the shooting range covered by the current camera at different times; determining the neighboring camera associated with the current camera based on the first movement trajectory of the first candidate entity object; finding at least one second candidate entity object located at the edge of the shooting range covered by the neighboring camera at the next time after the first candidate entity object is located at the edge; the step of extracting the appearance features of the entity object from the video streams captured by multiple cameras and identifying the same entity object in the video streams of different cameras based on the similarity of the appearance features of the entity object includes: extracting the appearance features of the first candidate entity object and the second candidate entity object respectively; determining the second candidate entity object corresponding to the same entity object as the first candidate entity object based on the similarity of the appearance features.

[0046] In one embodiment, the step of performing entity object trajectory detection on the video stream captured by the camera deployed in the target area and generating the first movement trajectory of each entity object in the target area under the coordinate system of the camera includes: calling the target detection model to detect entity objects frame by frame in the video stream captured by the camera; calling the target tracking algorithm to track the trajectory of the entity objects detected in each frame and generating the first movement trajectory of the entity objects under the camera.

[0047] In one embodiment, the step of calculating the movement direction and popularity of each entity object in the electronic map based on the distribution of the second movement trajectory in the grid includes: for each entity object, determining the movement direction of the entity object from the previous grid to the next grid based on the second movement trajectory; for each grid, calculating the movement direction of multiple entity objects from the current grid to the next grid; selecting a target movement direction based on the frequency of occurrence of different movement directions as the movement direction of multiple entity objects in the current grid; and determining the popularity of the multiple entity objects in the current grid based on the frequency of occurrence of different movement directions.

[0048] In one embodiment, determining the movement direction of each entity object from the previous grid to the next grid based on the second movement trajectory includes: for each entity object, determining the relative positional relationship between the position point falling into the previous grid and the position point in the next grid based on the timestamp information of the position point in the second movement trajectory, wherein the relative positional relationship is represented as a direction vector from the previous grid to the next grid; and determining the movement direction of the entity object from the previous grid to the next grid based on the determined relative positional relationship.

[0049] In one embodiment, generating a heatmap representing the movement direction of multiple entity objects in a target area based on their movement direction and heat intensity in the electronic map includes: for each grid cell, determining the direction of the arrow representing the movement direction in the heatmap based on the movement direction of the multiple entity objects in the electronic map; determining the size of the arrow representing the movement direction based on the heat intensity of the multiple entity objects in the electronic map, wherein the size of the arrow is positively correlated with the heat intensity; and generating a heatmap representing the movement direction of the multiple entity objects in the target area based on the determined arrow direction and arrow size.

[0050] In one embodiment, the method further includes: acquiring a test image of a standard object captured by the camera; and calculating a mapping transformation matrix between the camera coordinate system and the coordinate system of the electronic map based on the pixel coordinates of the standard object in the test image and the physical coordinates of the standard object in the store's electronic map coordinate system at the same time.

[0051] According to the embodiments of this application, entity object trajectory detection is performed on the video streams collected by cameras deployed in the target area to generate a first movement trajectory for each entity object in the camera coordinate system. The first movement trajectory is then converted to the electronic map coordinate system of the target area to obtain a second movement trajectory. Based on the distribution of the second movement trajectory in multiple grids evenly divided in the electronic map, the movement direction and heat of multiple entity objects are statistically analyzed. The movement direction and heat of multiple entity objects in the grid are determined by the frequency of occurrence of different movement directions. Then, a movement direction heatmap is generated and output based on the movement direction and heat of multiple entity objects, so as to perform entity object movement analysis by referring to the heatmap. It can be seen that the embodiments of this application first convert the movement trajectory detected under the camera's field of view into the movement trajectory in the electronic map coordinate system by registering between pixel coordinates and store map coordinates, thus overcoming the problem of the analysis results of traditional single-camera solutions being disconnected from the physical spatial layout. Secondly, the electronic map is evenly divided into multiple grids. Based on the distribution of the second movement trajectory in each grid, the movement direction of each entity object in the electronic map is statistically analyzed. A directional heat map is generated based on the movement direction data of all grids, providing data-driven visualization decision support for the transformation of the target area's traffic flow, layout optimization, and the setting of marketing points.

[0052] Furthermore, this application's embodiments extract the appearance features of entities from video streams captured by multiple cameras, identify the same entity under different cameras based on the similarity of appearance features, extract the second movement trajectory corresponding to the entity under each camera after conversion, and stitch the extracted trajectory segments together to reconstruct the entity's complete movement trajectory across cameras. This solves the problem of trajectory breakpoints and data fragmentation caused by identity switching between multi-camera coverage areas, providing a continuous and complete trajectory data foundation for full-site movement direction statistics and path analysis. In addition, this application's solution only relies on existing security or surveillance cameras in the store, without the need for additional deployment of dedicated hardware such as infrared or Wi-Fi probes, and has the ability to be low-cost, easy to promote, and replicable across stores.

[0053] This application provides a method for analyzing customer movement in physical stores, such as... Figure 4 The diagram shows a flowchart of a customer movement analysis method 400 in a physical store according to an embodiment of this application. The method 400 may include: in step S401, performing customer trajectory detection on video streams captured by cameras deployed in the physical store to generate a first movement trajectory for each customer in the physical store under the coordinate system of the cameras; in step S402, converting the first movement trajectory to a second movement trajectory for the customers under the coordinate system of an electronic map of the physical store, wherein the electronic map is uniformly divided into multiple grids; in step S403, statistically analyzing the movement direction and popularity of multiple customers in the electronic map based on the distribution of the second movement trajectory in the grids; wherein the movement direction and popularity of multiple entities in the grids are determined based on the frequency of occurrence of different movement directions; in step S404, generating and outputting a heat map representing the movement direction of the multiple customers in the physical store based on the movement direction and popularity of the multiple customers in the electronic map, so as to perform customer movement analysis by referring to the heat map.

[0054] Corresponding to the examples and method embodiments provided in this application, this application also provides a statistical device for the movement direction of an entity object. For example... Figure 5The diagram shows a structural block diagram of a statistical device 500 for tracking the movement direction of entities according to an embodiment of this application. The device 500 may include: a trajectory detection module 501, used to perform entity trajectory detection on video streams captured by cameras deployed in a target area, generating a first movement trajectory of each entity in the target area under the coordinate system of the cameras; a trajectory conversion module 502, used to convert the first movement trajectory to a second movement trajectory of the entity in the coordinate system of an electronic map of the target area, wherein the electronic map is uniformly divided into multiple grids; a statistics module 503, used to statistically analyze the movement direction and heat of multiple entities in the electronic map based on the distribution of the second movement trajectory in the grids; wherein the movement direction and heat of multiple entities in the grids are determined based on the frequency of occurrence of different movement directions; and a heat map generation module 504, used to generate and output a heat map representing the movement direction of the multiple entities in the target area based on the movement direction and heat of the multiple entities in the electronic map, so as to perform entity movement analysis by referring to the heat map.

[0055] In one embodiment, the camera deployed in the target area includes multiple cameras, and the device further includes: a trajectory stitching module, used to stitch together the second movement trajectory corresponding to the first movement trajectory of the same entity object under different cameras, so as to obtain the continuous second movement trajectory of the entity object in the target area.

[0056] In one embodiment, the trajectory stitching module includes: an appearance similarity recognition submodule, used to extract the appearance features of the entity object from video streams captured by multiple cameras, and to identify the same entity object in the video streams of different cameras based on the similarity of the appearance features of the entity object; and a trajectory stitching submodule, used to extract the second motion trajectory corresponding to the same entity object after conversion, and to stitch the extracted second motion trajectory together.

[0057] In one embodiment, the apparatus further includes: a first identifier allocation module, configured to, after generating the first movement trajectory of each entity object in the target area under the coordinate system of the camera, assign a first identifier uniquely existing within the range of a single camera to the identified entity object for the video stream of each camera; the apparatus further includes: a second identifier allocation module, configured to, after extracting the second movement trajectory corresponding to the same entity object after transformation, reassign a second identifier uniquely existing within the range of multiple cameras to the entity object, so that the same entity object corresponds to the same second identifier.

[0058] In one embodiment, the trajectory stitching module further includes: a first object search submodule, configured to, before extracting the appearance features of the entity object from the video streams acquired from multiple cameras, search for first candidate entity objects located at the edge of the shooting range corresponding to the current camera at different times for each camera; a camera determination submodule, configured to determine the neighboring camera associated with the current camera based on the first movement trajectory of the first candidate entity object; a second object search submodule, configured to search for at least one second candidate entity object located at the edge of the shooting range corresponding to the neighboring camera at the next time after the first candidate entity object is located at the edge; and an appearance similarity recognition submodule, specifically configured to extract the appearance features of the first candidate entity object and the second candidate entity object respectively; and determine the second candidate entity object corresponding to the same entity object as the first candidate entity object based on the similarity of the appearance features.

[0059] In one embodiment, the trajectory detection module is specifically used to call the target detection model to detect entity objects frame by frame in the video stream captured by the deployed camera; and to call the target tracking algorithm to track the trajectory of the entity objects detected in each frame, thereby generating the first movement trajectory of the entity objects under the camera.

[0060] In one embodiment, the statistics module includes: a first movement direction statistics submodule, used to determine the movement direction of each entity object from the previous grid to the next grid based on the second movement trajectory; a second movement direction statistics submodule, used to count the movement directions of multiple entity objects from the current grid to the next grid for each grid; a movement direction selection submodule, used to select a target movement direction as the movement direction of multiple entity objects in the current grid based on the frequency of occurrence of different movement directions; and a heat determination submodule, used to determine the heat of the multiple entity objects in the current grid based on the frequency of occurrence of different movement directions.

[0061] In one embodiment, the first movement direction statistics submodule is specifically used to determine, for each entity object, the relative positional relationship between the position point falling into the previous grid and the position point of the next grid based on the timestamp information of the position point in the second movement trajectory, wherein the relative positional relationship is represented as a direction vector from the previous grid to the next grid; and to determine the movement direction of the entity object from the previous grid to the next grid based on the determined relative positional relationship.

[0062] In one embodiment, the heatmap generation module is specifically used to, for each grid, determine the direction of the arrows representing the movement direction in the heatmap based on the movement direction of multiple entity objects in the electronic map; determine the size of the arrows representing the movement direction based on the heat level of the multiple entity objects in the electronic map, wherein the size of the arrows is positively correlated with the heat level; and generate a heatmap representing the movement direction of the multiple entity objects in the target area based on the determined arrow direction and arrow size.

[0063] In one embodiment, the apparatus further includes: a test image acquisition module for acquiring a test image of a standard object captured by the camera; and a mapping matrix calculation module for calculating a mapping transformation matrix between the camera coordinate system and the coordinate system of the electronic map based on the pixel coordinates of the standard object in the test image and the physical coordinates of the standard object in the store's electronic map coordinate system at the same time.

[0064] According to the embodiments of this application, entity object trajectory detection is performed on the video streams collected by cameras deployed in the target area to generate a first movement trajectory for each entity object in the camera coordinate system. The first movement trajectory is then converted to the electronic map coordinate system of the target area to obtain a second movement trajectory. Based on the distribution of the second movement trajectory in multiple grids evenly divided in the electronic map, the movement direction and heat of multiple entity objects are statistically analyzed. The movement direction and heat of multiple entity objects in the grid are determined by the frequency of occurrence of different movement directions. Then, a movement direction heatmap is generated and output based on the movement direction and heat of multiple entity objects, so as to perform entity object movement analysis by referring to the heatmap. It can be seen that the embodiments of this application first convert the movement trajectory detected under the camera's field of view into the movement trajectory in the electronic map coordinate system by registering between pixel coordinates and store map coordinates, thus overcoming the problem of the analysis results of traditional single-camera solutions being disconnected from the physical spatial layout. Secondly, the electronic map is evenly divided into multiple grids. Based on the distribution of the second movement trajectory in each grid, the movement direction of each entity object in the electronic map is statistically analyzed. A directional heat map is generated based on the movement direction data of all grids, providing data-driven visualization decision support for the transformation of the target area's traffic flow, layout optimization, and the setting of marketing points.

[0065] Furthermore, this application's embodiments extract the appearance features of entities from video streams captured by multiple cameras, identify the same entity under different cameras based on the similarity of appearance features, extract the second movement trajectory corresponding to the entity under each camera after conversion, and stitch the extracted trajectory segments together to reconstruct the entity's complete movement trajectory across cameras. This solves the problem of trajectory breakpoints and data fragmentation caused by identity switching between multi-camera coverage areas, providing a continuous and complete trajectory data foundation for full-site movement direction statistics and path analysis. In addition, this application's solution only relies on existing security or surveillance cameras in the store, without the need for additional deployment of dedicated hardware such as infrared or Wi-Fi probes, and has the ability to be low-cost, easy to promote, and replicable across stores.

[0066] Corresponding to the examples and method embodiments provided in this application, this application also provides a customer flow analysis device for physical stores. For example... Figure 6 The diagram shows a structural block diagram of a customer movement analysis device 600 in a physical store according to an embodiment of this application. The device 600 may include: a trajectory detection module 601, used to perform customer trajectory detection on video streams captured by cameras deployed in the physical store, generating a first movement trajectory for each customer in the physical store under the coordinate system of the cameras; a trajectory conversion module 602, used to convert the first movement trajectory to a second movement trajectory of the customer under the coordinate system of an electronic map of the physical store, the electronic map being uniformly divided into multiple grids; a statistics module 603, used to statistically analyze the movement direction and popularity of multiple customers in the electronic map based on the distribution of the second movement trajectory in the grids; wherein the movement direction and popularity of multiple entities in the grids are determined based on the frequency of occurrence of different movement directions; and a heatmap generation module 604, used to generate and output a heatmap representing the movement direction of the multiple customers in the physical store based on the movement direction and popularity of the multiple customers in the electronic map, so as to perform customer movement analysis by referring to the heatmap.

[0067] The functions of each module in each device in the embodiments of this application can be found in the corresponding description in the above method, and they have corresponding beneficial effects, which will not be repeated here.

[0068] Figure 7 This is a block diagram of an electronic device used to implement embodiments of this application. For example... Figure 7 As shown, the electronic device includes a memory 701 and a processor 702. The memory 701 stores a computer program that can run on the processor 702. When the processor 702 executes the computer program, it implements the method described in the above embodiments. The number of memories 701 and processors 702 can be one or more.

[0069] The electronic device also includes: The communication interface 703 is used to communicate with external devices and perform data exchange and transmission.

[0070] If the memory 701, processor 702, and communication interface 703 are implemented independently, they can be interconnected via a bus to communicate with each other. This bus can be an Industry Standard Architecture (ISA) bus, a Peripheral Component Interconnect (PCI) bus, or an Extended Industry Standard Architecture (EISA) bus, etc. This bus can be divided into address bus, data bus, control bus, etc. For ease of representation, Figure 7 The bus is represented by a single thick line, but this does not mean that there is only one bus or one type of bus.

[0071] Optionally, in a specific implementation, if the memory 701, processor 702, and communication interface 703 are integrated on a single chip, then the memory 701, processor 702, and communication interface 703 can communicate with each other through an internal interface.

[0072] This application provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements the method provided in this application.

[0073] This application also provides a computer program product, which includes a computer program that, when executed by a processor, implements the methods provided in any embodiment of this application.

[0074] This application also provides a chip including a processor for calling and executing instructions stored in a memory, causing a communication device with the chip installed to perform the method provided in this application.

[0075] This application also provides a chip, including: an input interface, an output interface, a processor, and a memory. The input interface, output interface, processor, and memory are connected through an internal connection path. The processor is used to execute code in the memory. When the code is executed, the processor is used to execute the method provided in this application.

[0076] It should be understood that the aforementioned processor can be a Central Processing Unit (CPU), or other general-purpose processors, Digital Signal Processors (DSPs), Application Specific Integrated Circuits (ASICs), Field-Programmable Gate Arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. General-purpose processors can be microprocessors or any conventional processor. It is worth noting that the processor can be a processor supporting Advanced Reduced Instruction Set Machines (ARM) architecture.

[0077] Further, optionally, the aforementioned memory may include read-only memory and random access memory. The memory may be volatile memory or non-volatile memory, or may include both. Non-volatile memory may include read-only memory (ROM), programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), or flash memory. Volatile memory may include random access memory (RAM), which serves as an external cache. By way of example, but not limitation, many forms of RAM are available. Examples include Static Random Access Memory (SRAM), Dynamic Random Access Memory (DRAM), Synchronous DRAM (SDRAM), Double Data Rate SDRAM (DDR SDRAM), Enhanced Synchronous DRAM (ESDRAM), Sync Link DRAM (SLDRAM), and Direct Rambus RAM (DR RAM).

[0078] In the above embodiments, implementation can be achieved, in whole or in part, through software, hardware, firmware, or any combination thereof. When implemented in software, it can be implemented, in whole or in part, as a computer program product. A computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, all or part of the processes or functions according to this application are generated. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions can be stored in a computer-readable storage medium or transferred from one computer-readable storage medium to another.

[0079] In the description of this specification, the references to terms such as "one embodiment," "some embodiments," "example," "specific example," or "some examples," etc., indicate that a specific feature, structure, material, or characteristic described in connection with that embodiment or example is included in at least one embodiment or example of this application. Furthermore, the specific features, structures, materials, or characteristics described may be combined in any suitable manner in one or more embodiments or examples. Moreover, without contradiction, those skilled in the art can combine and integrate the different embodiments or examples described in this specification, as well as the features of those different embodiments or examples.

[0080] Furthermore, the terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of technical features indicated. Thus, a feature defined as "first" or "second" may explicitly or implicitly include at least one of that feature. In the description of this application, "a plurality of" means two or more, unless otherwise explicitly specified.

[0081] Any process or method described in the flowchart or otherwise herein can be understood as representing a module, segment, or portion of code comprising one or more executable instructions for implementing a particular logical function or process. Furthermore, the scope of the preferred embodiments of this application includes additional implementations in which functions may be performed not in the order shown or discussed, including substantially simultaneously or in reverse order depending on the functionality involved.

[0082] The logic and / or steps described in the flowchart or otherwise herein, for example, can be considered as a sequenced list of executable instructions for implementing logical functions, and can be embodied in any computer-readable medium for use by, or in conjunction with, an instruction execution system, apparatus or device (such as a computer-based system, a processor-included system or other system that can fetch and execute instructions from, an instruction execution system, apparatus or device).

[0083] It should be understood that various parts of this application can be implemented using hardware, software, firmware, or a combination thereof. In the above embodiments, multiple steps or methods can be implemented using software or firmware stored in memory and executed by a suitable instruction execution system. All or part of the steps of the methods in the above embodiments can be implemented by a program instructing related hardware, the program being stored in a computer-readable storage medium, which, when executed, includes one or a combination of the steps of the method embodiments.

[0084] Furthermore, the functional units in the various embodiments of this application can be integrated into a processing module, or each unit can exist physically separately, or two or more units can be integrated into a module. The integrated module can be implemented in hardware or as a software functional module. If the integrated module is implemented as a software functional module and sold or used as an independent product, it can also be stored in a computer-readable storage medium. This storage medium can be a read-only memory, a disk, or an optical disk, etc.

[0085] The above description is merely an exemplary embodiment of this application, but the scope of protection of this application is not limited thereto. Any person skilled in the art can easily conceive of various variations or substitutions within the technical scope described in this application, and these should all be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

Claims

1. A statistical method for determining the movement direction of an entity object, comprising: Perform entity object trajectory detection on the video stream captured by the cameras deployed in the target area, and generate the first movement trajectory of each entity object in the target area in the coordinate system of the camera; The first movement trajectory is converted to the second movement trajectory of the entity object in the coordinate system of the electronic map of the target area, wherein the electronic map is evenly divided into multiple grids; Based on the distribution of the second movement trajectory in the grid, the movement direction and popularity of multiple entity objects in the electronic map are statistically analyzed; wherein, the movement direction and popularity of multiple entity objects in the grid are determined according to the frequency of occurrence of different movement directions; Based on the movement direction and heat of multiple entity objects in the electronic map, a heat map representing the movement direction of the multiple entity objects in the target area is generated and output for display, so as to perform entity object movement analysis by referring to the heat map.

2. The method according to claim 1, wherein, The method further includes the deployment of multiple cameras in the target area: By stitching together the second motion trajectory corresponding to the first motion trajectory of the same entity under different cameras, a continuous second motion trajectory of the entity in the target area can be obtained.

3. The method according to claim 2, wherein, The process of stitching together the second motion trajectory corresponding to the first motion trajectory of the same entity under different cameras includes: Extract the appearance features of the entity object from video streams captured by multiple cameras, and identify the same entity object in video streams from different cameras based on the similarity of the appearance features of the entity object; Extract the second action trajectory corresponding to the same entity object after transformation, and stitch the extracted second action trajectory together.

4. The method according to claim 3, wherein, After generating the first movement trajectory of each entity object in the target area in the coordinate system of the camera, the method further includes: For each camera's video stream, a unique first identity identifier is assigned to each identified entity within the range of a single camera. After extracting the second action trajectory corresponding to the transformed same entity object, the method further includes: The entity object is reassigned a second identity identifier that is unique to each of the multiple cameras, so that the same entity object corresponds to the same second identity identifier.

5. The method according to claim 3, wherein, Before extracting the appearance features of the entity object from the video streams captured by multiple cameras, the process of stitching together the second motion trajectory corresponding to the first motion trajectory of the same entity object under different cameras further includes: For each camera, find the first candidate entity object located at the edge of the shooting range covered by the current camera at different times; Based on the first movement trajectory of the first candidate entity, determine the neighboring cameras associated with the current camera; Find at least one second candidate entity object located at the edge of the shooting range covered by the adjacent camera at the next moment after the first candidate entity object is located at the current moment when it is located at the edge; Extracting the appearance features of the entity object from video streams captured by multiple cameras, and identifying the same entity object in video streams from different cameras based on the similarity of the appearance features of the entity object, includes: Extract the appearance features of the first candidate entity object and the second candidate entity object respectively; Based on the similarity of the appearance features, a second candidate entity object corresponding to the first candidate entity object is determined.

6. The method according to claim 1, wherein, The step of performing entity object trajectory detection on the video stream captured by the cameras deployed in the target area, and generating the first movement trajectory of each entity object in the target area in the coordinate system of the cameras, includes: The target detection model is invoked to detect entity objects frame by frame in the video stream captured by the deployed camera; The target tracking algorithm is invoked to track the trajectory of the entity objects detected in each frame, generating the first movement trajectory of the entity objects under the camera.

7. The method according to claim 1, wherein, The step of calculating the movement direction and popularity of multiple entity objects in the electronic map based on the distribution of the second movement trajectory in the grid includes: For each entity object, the movement direction of the entity object from the previous grid to the next grid is determined according to the second movement trajectory; For each grid, count the movement direction of multiple entity objects from the current grid to the next grid; The target movement direction is selected based on the frequency of occurrence of different movement directions, and is used as the movement direction of multiple entity objects in the current grid; The heat of the multiple entity objects in the current grid is determined based on the number of times they appear in different movement directions.

8. The method according to claim 7, wherein, Determining the movement direction of each entity object from the previous grid to the next grid based on the second movement trajectory includes: For each entity object, the relative positional relationship between the position point falling into the previous grid and the position point in the next grid is determined based on the timestamp information of the position point in the second movement trajectory. The relative positional relationship is represented by the direction vector from the previous grid to the next grid. Based on the determined relative positional relationship, the direction of movement of the entity object from the previous grid to the next grid is determined.

9. The method according to claim 1, wherein, The step of generating a heat map representing the movement direction of multiple entity objects in the target area based on their movement direction and heat intensity in the electronic map includes: For each grid, based on the movement direction of multiple entity objects in the electronic map, the direction of the arrow representing the movement direction in the heat map is determined; The size of the arrow representing the direction of movement is determined based on the heat of multiple entity objects in the electronic map, and the size of the arrow is positively correlated with the magnitude of the heat. Based on the determined direction and size of the arrows, a heat map is generated that characterizes the movement direction of the plurality of entity objects in the target area.

10. The method according to claim 1, wherein, Also includes: Acquire test images of a standard object captured by the camera; Based on the pixel coordinates of the standard object in the test image and the physical coordinates of the standard object in the store's electronic map coordinate system at the same time, calculate the mapping transformation matrix between the camera coordinate system and the electronic map coordinate system.

11. A method for analyzing customer movement in physical stores, comprising: Perform customer trajectory detection on the video streams captured by cameras deployed in physical stores, and generate the first movement trajectory of each customer in the physical store in the coordinate system of the cameras; The first movement trajectory is converted into the customer's second movement trajectory in the coordinate system of the electronic map of the physical store, wherein the electronic map is evenly divided into multiple grids; Based on the distribution of the second movement trajectory in the grid, the movement direction and popularity of multiple customers in the electronic map are statistically analyzed; wherein, the movement direction and popularity of multiple entity objects in the grid are determined according to the frequency of occurrence of different movement directions; Based on the movement direction and heat of multiple customers in the electronic map, a heat map representing the movement direction of the multiple customers in the physical store is generated and output for display, so as to conduct customer movement analysis by referring to the heat map.

12. An electronic device comprising a memory, a processor, and a computer program stored in the memory, wherein the processor, when executing the computer program, implements the method of any one of claims 1-11.

13. A computer-readable storage medium storing a computer program that, when executed by a processor, implements the method of any one of claims 1-11.

14. A computer program product, wherein, The computer program product includes a computer program that, when executed by a processor, implements the method described in any one of claims 1-11.

Images