Vehicle tracking data processing method and device, storage medium and program product

Abstract

The invention provides a vehicle tracking data processing method and device, a storage medium and a program product. The method comprises the steps of obtaining to-be-processed vehicle tracking data under a plurality of lenses in a target area; mapping the to-be-processed vehicle tracking data to a preset map space to obtain mapped vehicle tracking data; according to the mapped vehicle tracking data, merging different tracking data belonging to the same vehicle under the same lens to generate single-lens merging data under the same lens; and according to the single-lens merging data corresponding to the plurality of lenses, merging the tracking data belonging to the same vehicle under the plurality of lenses, and generating unique tracking data of the same vehicle under the plurality of lenses. According to the method and the device, the continuity and the accuracy of vehicle tracking in the multi-lens monitoring system are improved, the possibility of tracking interruption or misrecognition caused by different multi-lens shooting viewing angles is reduced, and the accuracy and the efficiency of the monitoring system are remarkably improved.

Description

Technical Field

[0001] The present application relates to the field of information processing technology, and in particular to a vehicle tracking data processing method, device, storage medium and program product. Background Art

[0002] Deploying surveillance cameras in car stores and using machine vision-based artificial intelligence technology to analyze customers' visits and test drives of exhibition vehicles is one of the effective ways to accurately grasp the popularity of the models launched.

[0003] Automotive stores are typically large, requiring numerous surveillance cameras to fully cover the entire space. Some vehicles appear only in a few cameras, while others are captured across multiple cameras simultaneously. To accurately capture the popularity of each model, it's necessary to accurately correlate vehicles captured across multiple cameras and merge and de-duplicate vehicle identities to avoid duplicate counting.

[0004] Currently, the process of merging and deduplicating vehicle identities requires prior spatial calibration of each lens to map the lens's imaging coordinates to map coordinates. Map space calibration typically involves the annotator finding corresponding key points captured by the lens on a building floor plan map, thereby establishing a mapping relationship between the map plane coordinates and the pixel coordinates of the lens's ground points. However, in practice, due to the limited number of available key points on the floor plan map and the fact that reliable key points may not be visible under a specific lens, the annotator can only perform a rough calibration, resulting in large final calibration errors. Furthermore, due to the large width of indoor surveillance cameras, the nonlinear spatial distortion of photographic imaging amplifies mapping errors at long distances and at the edges of the image. This can lead to significant deviations in the position of the same vehicle in map space when projected under different lenses. Summary of the Invention

[0005] The main purpose of the embodiments of the present application is to provide a vehicle tracking data processing method, device, storage medium and program product, which not only improves the continuity and accuracy of vehicle tracking in a multi-lens monitoring system, but also reduces the possibility of tracking interruption or misidentification caused by different shooting angles of multiple lenses, thereby significantly improving the accuracy and efficiency of the monitoring system.

[0006] In a first aspect, an embodiment of the present application provides a vehicle tracking data processing method, comprising: obtaining to-be-processed vehicle tracking data under multiple lenses in a target area; mapping the to-be-processed vehicle tracking data to a preset map space to obtain mapped vehicle tracking data; merging different tracking data belonging to the same vehicle under the same lens based on the mapped vehicle tracking data to generate single-lens merged data under the same lens; and merging the tracking data belonging to the same vehicle under the multiple lenses based on the single-lens merged data corresponding to the multiple lenses to generate unique tracking data for the same vehicle under the multiple lenses.

[0007] In one embodiment, obtaining the vehicle tracking data to be processed under multiple lenses in the target area includes: receiving the vehicle tracking data to be processed sent by the edge computing device corresponding to the multiple lenses in the target area, where the vehicle tracking data to be processed is obtained by the edge computing device through vehicle tracking detection based on the monitoring data under the corresponding lenses.

[0008] In one embodiment, the preset map space is the map space of the real scene covered by the target area; mapping the vehicle tracking data to be processed to the preset map space to obtain mapped vehicle tracking data includes: determining the image coordinates of the vehicle in the monitoring image based on the vehicle tracking data to be processed; determining the real space map of the real scene covered by the target area; mapping the image coordinates to the real space map based on the camera calibration data of the multiple lenses to obtain the map coordinates of the vehicle on the real space map, and the mapped vehicle tracking data includes the map coordinates of the vehicle under the multiple lenses.

[0009] In one embodiment, merging different tracking data belonging to the same vehicle under the same shot based on the mapped vehicle tracking data to generate single-shot merged data under the same shot includes: determining time information and position information corresponding to each vehicle tracking data in the mapped vehicle tracking data; under the same shot, filtering candidate vehicle tracking data with non-overlapping time from the mapped vehicle tracking data based on the time information; and merging different vehicle tracking data with positional overlap areas greater than a first threshold in the candidate vehicle tracking data under the same shot based on the position information to generate the single-shot merged data under the same shot.

[0010] In one embodiment, merging different tracking data belonging to the same vehicle under the same lens based on the mapped vehicle tracking data to generate single-shot merged data under the same lens further includes: merging different vehicle tracking data belonging to the same vehicle under the same lens, having a position overlap area greater than a second threshold and less than the first threshold and having the same vehicle attribute information, based on the position information, to generate the single-shot merged data under the same lens.

[0011] In one embodiment, before merging different tracking data belonging to the same vehicle under the same lens based on the mapped vehicle tracking data to generate single-lens merged data under the same lens, the method further includes: performing data cleaning on the mapped vehicle tracking data to eliminate redundant data, where the redundant data includes one or more of the following data: vehicle tracking data within a specified time period, tracking data with a number of vehicle key points less than a preset number, and tracking data with an area of ​​the vehicle in the preset map space less than a preset area threshold.

[0012] In one embodiment, merging the tracking data belonging to the same vehicle under the multiple lenses based on the single-shot merged data corresponding to the multiple lenses to generate unique tracking data for the same vehicle under the multiple lenses includes: determining the position information and / or vehicle attribute information corresponding to each vehicle tracking data in the single-shot merged data; performing a one-to-one matching between the single-shot merged data between the multiple lenses based on the position information and / or the vehicle attribute information to obtain matching pairs of vehicle tracking data between different lenses; and aggregating the matching pairs under the multiple lenses to generate unique tracking data for the same vehicle under the multiple lenses.

[0013] In one embodiment, based on the position information and / or the vehicle attribute information, one-to-one matching is performed on the single-shot merged data between the multiple shots to obtain matching pairs of vehicle tracking data between different shots, including: for any two shots, based on the position information and / or the vehicle attribute information, respectively calculating similarity scores between the different vehicle tracking data under the any two shots; using the similarity scores as matching scores, with the goal of maximizing the matching scores, performing bipartite graph assignment matching on the different vehicle tracking data under the any two shots, to obtain the matching pairs corresponding to the different vehicle tracking data under the any two shots.

[0014] In one embodiment, aggregating the matching pairs under the multiple shots in pairs to generate unique tracking data of the same vehicle under the multiple shots includes: traversing the matching pairs under the multiple shots, and if both vehicle tracking data in the current matching pair have not been aggregated, aggregating the two vehicle tracking data in the current matching pair into tracking data of the same vehicle; if the first vehicle tracking data in the current matching pair has been aggregated into a first set, the second vehicle tracking data in the current matching pair has not been aggregated, and the first set does not contain vehicle tracking data under the shot to which the second vehicle tracking data belongs, aggregating the second vehicle tracking data into the first set; if the current matching pair has been aggregated into two different sets, and the two different sets do not contain vehicle tracking data under the same shot, merging the two different sets into a single set; if the current matching pair has been aggregated into two different sets, and the two different sets contain vehicle tracking data under the same shot, removing the current match; and if there is single vehicle tracking data under the multiple shots that cannot be paired, generating a single cluster for the single vehicle tracking data.

[0015] In one embodiment, before merging the tracking data belonging to the same vehicle under the multiple shots based on the single-shot merged data corresponding to the multiple shots to generate unique tracking data of the same vehicle under the multiple shots, the method further includes: removing vehicle tracking data with a duration less than a preset duration from the single-shot merged data corresponding to the multiple shots; and / or removing vehicle tracking data with a vehicle size greater than a preset size from the single-shot merged data corresponding to the multiple shots.

[0016] In a second aspect, an embodiment of the present application provides a method for determining vehicle information, comprising: obtaining unique tracking data of a target vehicle in a target area under multiple lenses, wherein the unique tracking data is obtained using the method described in any of the above aspects; determining a user visit event for the target vehicle based on the unique tracking data; and determining the target vehicle's information of interest based on the user visit event.

[0017] In a third aspect, an embodiment of the present application provides a vehicle tracking data processing device, comprising:

[0018] An acquisition module is used to acquire the vehicle tracking data to be processed under multiple lenses in the target area;

[0019] A mapping module, configured to map the to-be-processed vehicle tracking data to a preset map space to obtain mapped vehicle tracking data;

[0020] a first merging module, configured to merge different tracking data belonging to the same vehicle under the same lens according to the mapped vehicle tracking data, to generate single-lens merged data under the same lens;

[0021] The second merging module is configured to merge the tracking data belonging to the same vehicle under the multiple shots according to the single-shot merged data corresponding to the multiple shots, and generate unique tracking data of the same vehicle under the multiple shots.

[0022] In a fourth aspect, an embodiment of the present application provides an electronic device, including:

[0023] at least one processor; and

[0024] a memory communicatively coupled to the at least one processor;

[0025] The memory stores instructions that can be executed by the at least one processor, and the instructions are executed by the at least one processor to enable the electronic device to execute the method described in any one of the above aspects.

[0026] In a fifth aspect, an embodiment of the present application provides a cloud device, including:

[0027] at least one processor; and

[0028] a memory communicatively coupled to the at least one processor;

[0029] The memory stores instructions that can be executed by the at least one processor, and the instructions are executed by the at least one processor to enable the cloud device to execute the method described in any one of the above aspects.

[0030] In a sixth aspect, an embodiment of the present application provides a computer-readable storage medium, wherein the computer-readable storage medium stores computer-executable instructions. When a processor executes the computer-executable instructions, the method described in any one of the above aspects is implemented.

[0031] In a seventh aspect, an embodiment of the present application provides a computer program product, including a computer program, which implements the method described in any of the above aspects when executed by a processor.

[0032] The vehicle tracking data processing method, device, storage medium, and program product provided in the embodiments of the present application solve the data inconsistency problem caused by perspective differences by mapping the vehicle tracking data from multiple lenses within the target area to a unified preset map space, thereby ensuring standardized data processing. Within a single lens, multiple tracking data of the same vehicle are merged to generate single-lens merged data, thereby reducing data redundancy. Then, by merging the vehicle tracking data across lenses, a unique global tracking data of the same vehicle under multiple lenses is generated. In this way, through a progressive merging mode from a single point to multiple points, the integration and optimization of multi-lens vehicle tracking data is achieved, which not only improves the continuity and accuracy of vehicle tracking in the multi-lens monitoring system, but also reduces the possibility of tracking interruption or misidentification caused by different shooting perspectives of multiple lenses, significantly improving the accuracy and efficiency of the monitoring system. BRIEF DESCRIPTION OF THE DRAWINGS

[0033] The accompanying drawings are incorporated into and constitute a part of the specification, illustrate embodiments consistent with the present application, and together with the specification, are used to explain the principles of the present application. Obviously, the drawings described below are some embodiments of the present invention, and it is clear that those skilled in the art can derive other drawings based on these drawings without inventive effort.

[0034] Figure 1 A schematic diagram of the structure of an electronic device provided in an embodiment of the present application;

[0035] Figure 2 A schematic diagram of an application scenario of a vehicle tracking data processing system provided in an embodiment of the present application;

[0036] Figure 3 A schematic diagram of the vehicle tracking data processing system architecture provided in an embodiment of the present application;

[0037] Figure 4 A flow chart of a vehicle tracking data processing method provided in an embodiment of the present application;

[0038] Figure 5 A coordinate mapping diagram provided in an embodiment of the present application;

[0039] Figure 6 A schematic diagram of bipartite graph assignment matching provided in an embodiment of the present application;

[0040] Figure 7 A schematic diagram of defining an overlap score between two vehicles provided in an embodiment of the present application;

[0041] Figure 8 A schematic diagram of a Euclidean distance fraction between two vehicles provided in an embodiment of the present application;

[0042] Figure 9 A schematic diagram of an aggregation conflict provided in an embodiment of the present application;

[0043] Figure 10 A flow chart of a vehicle tracking data processing method provided in an embodiment of the present application;

[0044] Figure 11 A flowchart of a method for determining vehicle information provided in an embodiment of the present application;

[0045] Figure 12 A schematic structural diagram of a vehicle tracking data processing device provided in an embodiment of the present application;

[0046] Figure 13 A schematic diagram of the structure of a cloud device provided in an embodiment of the present application.

[0047] The above drawings illustrate specific embodiments of the present application, which will be described in more detail below. These drawings and the textual description are not intended to limit the scope of the present application in any way, but rather to illustrate the concepts of the present application to those skilled in the art by reference to specific embodiments. DETAILED DESCRIPTION

[0048] Exemplary embodiments are described in detail herein, with examples illustrated in the accompanying drawings. When the following description refers to the drawings, identical numerals in different drawings represent identical or similar elements unless otherwise indicated. The embodiments described in the following exemplary embodiments are not intended to represent all embodiments consistent with this application.

[0049] The term "and / or" in this article is used to describe the association relationship of associated objects, specifically indicating that there can be three relationships. For example, A and / or B can mean: A exists alone, A and B exist at the same time, and B exists alone.

[0050] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, stored data, displayed data, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of relevant data must comply with the relevant laws, regulations and standards of relevant countries and regions, and provide corresponding operation entrances for users to choose to authorize or refuse.

[0051] In order to clearly describe the technical solutions of the embodiments of the present application, the following definitions are given first:

[0052] AI: Artificial Intelligence.

[0053] Map coordinates: refers to the plane coordinates that represent the location distribution of buildings in the actual scene.

[0054] ID: Identification, refers to identity recognition or identification.

[0055] Vehicle ID deduplication: The same vehicle photographed with different lenses is labeled with a unified ID to avoid the situation where one vehicle has multiple IDs.

[0056] Vehicle track: A vehicle tracking sequence corresponds to a target tracking sequence of a vehicle in a video, which is used to record information such as the vehicle's position, attributes, movement, and duration in the video.

[0057] Vehicle attributes: refers to the vehicle's brand, model, color and other attribute information.

[0058] Mutual exclusion check: Use objective laws to eliminate situations that cannot coexist or are obviously conflicting, so that the final result conforms to common sense.

[0059] ReID: Re-identification is a research topic in the field of computer vision, which aims to re-identify the same vehicle at different times and places by analyzing and matching the vehicle's visual features.

[0060] 3D: Three-Dimensional, three-dimensional.

[0061] LiDAR: Light Detection and Ranging, laser radar.

[0062] RGB-D camera: RGB+Depth camera, which combines RGB (red, green, and blue) color cameras with depth information acquisition technology. It can simultaneously capture the color information and distance information of an object, thereby achieving three-dimensional perception.

[0063] SLAM: Simultaneous Localization and Mapping, simultaneous positioning and map construction.

[0064] IOU: Intersection over Union, intersection over union.

[0065] The vehicle tracking data processing method of the embodiment of the present application can be applied to any field involving vehicle monitoring data.

[0066] As competition in the new energy vehicle market intensifies, accurately understanding customer demand for automotive products has become a growing concern for automakers. Deploying surveillance cameras in dealerships and leveraging machine vision-based artificial intelligence (AI) to analyze customer behavior during vehicle visits and test drives is one of the most effective ways to accurately gauge the popularity of new models.

[0067] In real-world scenarios, car dealerships are typically large, requiring a large number of cameras to achieve full coverage of the entire storefront. Some vehicles appear only in a few shots, while others are captured simultaneously by multiple cameras. To accurately capture the popularity of each model, it is necessary to accurately associate vehicles appearing in multiple shots and merge and de-duplicate vehicle IDs to avoid duplicate counting. This requires mapping the image space of each camera to a unified map space. Vehicles in the same location on this unified map space are considered the same vehicle. Therefore, spatial calibration of each camera is required to map the camera's imaging coordinates to map coordinates.

[0068] Map space calibration typically involves annotators searching for keypoints corresponding to the camera's view on a floor plan of the car dealership, thereby establishing a mapping relationship between map coordinates and the pixel coordinates of ground points captured by the camera. However, in practice, due to the limited number of available keypoints on the floor plan, reliable keypoints may not be visible under a specific lens. Annotators can only perform rough calibration, resulting in large errors. Furthermore, due to the large width of indoor surveillance cameras, the nonlinear spatial distortion of photographic imaging amplifies mapping errors at long distances and at image edges. This can lead to significant deviations in the position of the same vehicle on the floor plan when projected under different lenses. Simple vehicle ID merging based on spatial overlap often fails. For example, the same vehicle may have minimal overlap and cannot be merged, resulting in vehicle splitting. Different vehicles may also have some overlap, leading to incorrect merging.

[0069] Under such low annotation accuracy conditions, achieving highly accurate deduplication of cross-border vehicles is a practical challenge that must be overcome in real-world operations. Relying solely on spatial projection relationships cannot solve this problem. Other methods must be combined to compensate for the lack of spatial annotation accuracy and fully utilize other available information to reduce the possibility of incorrect merging.

[0070] Related technologies in the smart transportation sector also face the cross-border header merging problem (vehicle re-identification, or ReID). This primarily involves correlating the IDs of vehicles on the road from multiple traffic surveillance cameras. This approach primarily utilizes vehicle fingerprints (such as deep learning ReID features, attributes like model / color, etc.) or license plates for identification and association, eliminating the need for deduplication based on map coordinates. However, new cars in dealerships lack license plates, and license plates are difficult to capture from most angles. Therefore, this technology can only utilize fingerprint information based on the vehicle's appearance. Because indoor surveillance cameras often exhibit significant variations in vehicle posture, mutual occlusion, incomplete field of view, and poor differentiation between models of the same series, association based on vehicle fingerprints has a high error rate, making it difficult to meet the high-precision requirements of the business.

[0071] The key to indoor vehicle ID deduplication lies in the accuracy of the coordinate mapping between the camera imaging space and the map space where the building resides. Theoretically, if the coordinate projections between the two are very accurate, vehicle deduplication can be achieved based on the overlap of the projected positions. This involves camera 3D reconstruction technologies, including but not limited to manual modeling, LIDAR modeling, RGB-D depth camera modeling, and multi-view modeling. LIDAR modeling is suitable for sports cameras but is costly and unsuitable for surveillance scenarios. Car dealerships use standard surveillance cameras, making RGB-D depth camera modeling unsuitable. The only viable solutions are manual modeling and multi-view modeling. Manual modeling requires annotators to find corresponding matching points in each camera according to the building's floor plan, and then construct a mapping relationship between image coordinates and map coordinates. Because building floor plans consist of simple planar elements, it is often difficult to find a large number of accurate matching points in the surveillance view, resulting in large mapping errors. Multi-view modeling leverages the overlap between the fields of view of multiple surveillance cameras to reconstruct the 3D structure of the scene through camera parameter estimation and view matching. When cameras are densely distributed and the overlap between lenses is large, good reconstruction results can be achieved. However, in reality, some car dealerships have fewer surveillance cameras and minimal overlap between their lenses, resulting in large reconstruction errors or even failure. Therefore, both manual reconstruction and automatic reconstruction using multi-view photography carry the risk of large reconstruction errors. A simple approach based on spatial projection matching is unlikely to achieve highly accurate vehicle ID deduplication.

[0072] In summary, neither ReID based on vehicle attributes nor vehicle ID deduplication based on spatial mapping relationships can meet the high accuracy and versatility requirements under real-world conditions. Due to spatial layout and lens field of view limitations, vehicles in auto dealerships are scattered across multiple cameras, necessitating the association and deduplication of vehicles across these cameras under unified map coordinates. High-precision map calibration is costly and difficult, and calibration accuracy under real-world conditions is typically low. Accurately deduplicating vehicle IDs across multiple cameras with low calibration accuracy is a pressing technical challenge in this application.

[0073] In order to solve at least one of the above problems, the embodiment of the present application provides a vehicle tracking data processing solution. By mapping the vehicle tracking data from multiple lenses in the target area to a unified preset map space, the data inconsistency problem caused by the difference in viewing angles is solved, and the standardized processing of the data is ensured. In a single lens, multiple tracking data of the same vehicle are merged to generate single-lens merged data, thereby reducing data redundancy. Then, by merging the vehicle tracking data across lenses, a global unique tracking data of the same vehicle under multiple lenses is generated. In this way, through a progressive merging mode from a single point to multiple points, the integration and optimization of multi-lens vehicle tracking data is achieved, which not only improves the continuity and accuracy of vehicle tracking in the multi-lens monitoring system, but also reduces the possibility of tracking interruption or misidentification caused by different shooting angles of multiple lenses, significantly improving the accuracy and efficiency of the monitoring system.

[0074] In real-world scenarios, such as car dealerships, not only spatial location information but also vehicle temporal and attribute information can be utilized. For example, vehicles captured simultaneously in the same shot are definitely different vehicles, and vehicles of different models are definitely different vehicles. This multifaceted information can be combined and used to complement each other to meet business needs. Combining this information effectively reduces erroneous merging caused by projection bias and improves the accuracy of final vehicle deduplication.

[0075] The method proposed in the embodiment of the present application comprehensively utilizes the time information, spatial information and attribute information contained in the vehicle under the monitoring camera, realizes the full utilization of multi-source information through reasonable scheme design, and effectively realizes the high-accuracy merging and deduplication of vehicle IDs under low map positioning accuracy.

[0076] The following detailed description of some embodiments of the present application is provided in conjunction with the accompanying drawings. The following embodiments and features thereof may be combined with one another unless they conflict with each other. Furthermore, the sequence of steps in the following method embodiments is provided for illustrative purposes only and is not intended to be a strict limitation.

[0077] like Figure 1 As shown, this embodiment provides an electronic device 1, including: at least one processor 11 and a memory 12, Figure 1 A processor is used as an example. The processor 11 and the memory 12 are connected via a bus 10. The memory 12 stores instructions that can be executed by the processor 11. The instructions are executed by the processor 11 so that the electronic device 1 can execute all or part of the process of the method in the following embodiment, thereby not only improving the continuity and accuracy of vehicle tracking in a multi-camera monitoring system, but also reducing the possibility of tracking interruption or misidentification caused by different shooting angles of multiple cameras, significantly improving the accuracy and efficiency of the monitoring system.

[0078] In one embodiment, the electronic device 1 can be a mobile phone, tablet computer, laptop computer, desktop computer, or a large computing system composed of multiple computers. The electronic device 1 can also include an edge computing box or a computing server with a certain AI computing power.

[0079] Figure 2 Schematic diagram of a vehicle tracking data processing system application scenario 200 provided in an embodiment of the present application. Figure 2 As shown, the system includes: a server 210 and a terminal 220, wherein:

[0080] The server 210 may be a data platform that provides vehicle tracking data processing services, such as a video surveillance platform for a car store. In actual scenarios, a video surveillance platform may have multiple servers 210. Figure 2 Here, one server 210 is taken as an example.

[0081] The terminal 220 can be a computer, mobile phone, tablet or other device used by the user to log in to the video surveillance platform. There can also be multiple terminals 220. Figure 2 Two terminals 220 are used as an example for illustration.

[0082] The terminal 220 and the server 210 can transmit information via the Internet, so that the terminal 220 can access the data on the server 210. The terminal 220 and / or the server 210 can be implemented by the electronic device 1.

[0083] The vehicle tracking data processing solution of the embodiment of the present application can be deployed on the server 210, can also be deployed on the terminal 220, or can be deployed partially on the server 210 and partially on the terminal 220. In actual scenarios, the choice can be based on actual needs, and this embodiment does not limit it.

[0084] When the vehicle tracking data processing solution is fully or partially deployed on the server 210 , a calling interface may be opened to the terminal 220 to provide algorithm support to the terminal 220 .

[0085] The method provided in the embodiments of the present application can be implemented by executing corresponding software code on electronic device 1 and by interacting with a server. The electronic device 1 can be a local terminal device. When the method is run on a server, the method can be implemented and executed based on a cloud interaction system, which includes a server and a client device.

[0086] In a possible implementation, the method provided in the embodiment of the present application provides a graphical user interface through a terminal device, wherein the terminal device can be the local terminal device mentioned above, or it can be a client device in the cloud interaction system mentioned above.

[0087] like Figure 3 The figure shows a schematic diagram of the vehicle tracking data processing system architecture provided by an embodiment of the present application. Taking a surveillance camera as an image acquisition device as an example, the system mainly includes a surveillance camera, an edge computing device, a map survey and annotation platform, a central storage system and a central server, as well as an upper-layer business system. The embodiment of the present application is based on vehicle detection, tracking and identification at the edge, car store map construction and lens-to-map calibration, and uses this basic information to complete cross-lens vehicle ID deduplication and merging. Among them:

[0088] The surveillance camera is used to collect image information of the car store. There can be multiple surveillance cameras.

[0089] The edge computing devices are respectively connected to the corresponding surveillance cameras to complete vehicle detection, tracking and attribute recognition under a single lens. Optionally, the vehicle detection, tracking and attribute recognition methods under a single lens include but are not limited to various vehicle detection models, vehicle tracking methods, and vehicle attribute extraction, wherein the content of vehicle attribute extraction includes but is not limited to the vehicle's brand, color, license plate, ReID features, local identifiers, etc. Any technology related to vehicle detection, tracking, and attribute recognition and its improvements can be used as the underlying basis of the embodiments of the present application, and can be improved and replaced in a plug-in manner, and the embodiments of the present application are not limited to this.

[0090] The calibration platform is used to complete the survey of the car store site floor map and the calibration of the lens image space to the flat map space, and then store the mapping matrix in the storage system. Optionally, the map construction and lens calibration of the embodiment of the present application may include robot-based map survey, SLAM-based 3D field construction, multi-view scene modeling, etc. In principle, the more accurate the map construction and calibration, the higher the accuracy of the vehicle deduplication of the embodiment of the present application. Any effective map construction and calibration method can be applied to the embodiment of the present application.

[0091] The central server is used to receive the basic vehicle data reported by the edge computing devices corresponding to each camera (i.e., the vehicle tracking data to be processed, including but not limited to the results of vehicle detection, tracking, and attribute identification), read the mapping matrix from the storage system, and complete the deduplication and merging of vehicle IDs across cross-border headers, generate unique tracking data for the same vehicle under multiple cameras, and then store the merged results in the storage system.

[0092] The business system is used to read the merged results and extract the required business data from them, thus forming a complete business closed loop. For example, it can read the unique tracking data of the same vehicle under multiple cameras, determine the user visit event for the target vehicle based on the unique tracking data, and determine the attention information of the target vehicle based on the user visit event.

[0093] The solution of the embodiment of the present application is used to perform cross-lens merging of vehicle detection and tracking results reported by multiple lenses, and to achieve cross-lens vehicle ID deduplication. In terms of system composition, it includes two parts: vehicle detection and tracking on the edge side and cross-lens vehicle merging on the server side. On the edge side, first, based on the vehicle detection and tracking algorithm on the end side, vehicle detection and tracking are performed on each camera, and the vehicle track data stream is reported to the server. On the server side, the algorithm proposed in this application is used to perform cross-lens merging on the reported vehicle track data stream to obtain the vehicle result after ID deduplication, which serves other business applications (such as visiting customer volume detection, test drive customer volume detection, customer viewing time detection, etc.) data extraction scenarios.

[0094] Please see Figure 4 , which is a vehicle tracking data processing method according to an embodiment of the present application, the method can be Figure 1 The electronic device 1 shown is used to perform and can be applied to Figure 2-3 In the application scenario shown in , its implementation not only improves the continuity and accuracy of vehicle tracking in the multi-lens monitoring system, but also reduces the possibility of tracking interruption or misidentification caused by different shooting angles of multiple lenses, significantly improving the accuracy and efficiency of the monitoring system. Figure 3 For example, the central server shown in FIG is used as the execution end, and the method includes the following steps:

[0095] Step 401: Acquire vehicle tracking data to be processed under multiple lenses in a target area.

[0096] In this step, the target area can be a surveillance coverage area, such as the surveillance coverage area of ​​a car store. The vehicle tracking data to be processed can be the vehicle tracking data reported by various surveillance cameras, such as Figure 3 The vehicle tracking data reported to the server by the edge computing device includes but is not limited to the results of vehicle detection, tracking, and attribute recognition based on the surveillance video of the car store.

[0097] In one embodiment, step 401 may specifically include: receiving vehicle tracking data to be processed sent by an edge computing device corresponding to multiple lenses in a target area, where the vehicle tracking data to be processed is obtained by the edge computing device performing vehicle tracking detection based on monitoring data under the corresponding lenses.

[0098] In this step, the vehicle tracking data to be processed is received from the edge computing devices corresponding to multiple cameras in the target area. These data are generated by edge computing devices after analyzing the monitoring data under each camera and performing vehicle tracking detection, such as Figure 3 In the system shown, on the edge side, vehicle detection and tracking algorithms are implemented on each edge computing device. These algorithms detect and track vehicles on each surveillance camera within the dealership, and then transmit the vehicle tracking data stream to the central server. This allows the central server to obtain vehicle tracking data from different viewing angles and locations, improving the accuracy and reliability of vehicle tracking. Furthermore, using edge computing devices for preliminary data processing and analysis effectively reduces the computing burden on the central server, improving the overall system's response speed and processing efficiency. This distributed data processing architecture enhances the real-time and stability of the monitoring system.

[0099] Step 402: Map the vehicle tracking data to be processed to a preset map space to obtain mapped vehicle tracking data.

[0100] In this step, a preset map space is used to map the vehicle tracking data to be processed from different surveillance cameras into a unified map space. This ensures that the mapped vehicle tracking data has the same map space. This resolves data inconsistencies (e.g., spatial coordinates) caused by different viewpoints, ensures standardized data processing, and improves data computation efficiency. The preset map space can be the map space of the real-world scene of the target area, such as the real-world map space of the building housing the car dealership.

[0101] In one embodiment, the preset map space is the map space of the real-world scene covered by the target area. Step 402 may specifically include: determining the image coordinates of the vehicle in the surveillance image based on the vehicle tracking data to be processed. Determining a real-world map of the real-world scene covered by the target area. Mapping the image coordinates to the real-world map based on camera calibration data for multiple lenses to obtain the map coordinates of the vehicle on the real-world map. The mapped vehicle tracking data includes the map coordinates of the vehicle as viewed from multiple lenses.

[0102] In this embodiment, the preset map space corresponds to the map space of the real scene covered by the target area, such as the map space of the real scene covered by the building of a car dealership, which can provide a real geographic reference for vehicle tracking. By obtaining the vehicle tracking data to be processed, the system can determine the image coordinates of the vehicle in the monitoring image. Using the real space map of the target area and the camera calibration data of multiple lenses, these image coordinates are accurately mapped to the real space map, thereby obtaining the map coordinates of the vehicle on the real space map. The mapped vehicle tracking data contains the map coordinates of the vehicle under multiple lenses, enabling the system to continuously track and analyze the vehicle in a unified geographic space. This mapping technology improves the accuracy of vehicle location data and provides a reliable data foundation for subsequent different business applications.

[0103] like Figure 5 The figure below is a schematic diagram of coordinate mapping provided by an embodiment of the present application. Taking a surveillance image captured by a camera lens of a certain showroom in a car dealership as an example, coordinate mapping involves converting the image coordinates of each vehicle in the vehicle track captured by each camera lens into map coordinates of a unified display space map. Here, the vehicle image coordinates may refer to the image coordinates of the projection of the vehicle's chassis point on the ground (referred to as chassis point coordinates), and the mapped map coordinates refer to the corresponding position coordinates of each vehicle chassis point on the car dealership's floor map. The camera calibration data required for coordinate mapping can generally be obtained through camera calibration of the car dealership's surveillance cameras or multi-view reconstruction techniques. Taking manual camera calibration as an example, key points on the ground in the surveillance image and their corresponding points on the car dealership's floor map are selected as matching points. Based on three or more matching point pairs, a mapping matrix from image pixel coordinates to floor map coordinates is constructed. This mapping matrix can be used to map the vehicle's floor map from the vehicle's chassis point, thereby obtaining the physical location of the vehicle captured by the camera lens on the unified floor map.

[0104] Step 403: Based on the mapped vehicle tracking data, different tracking data belonging to the same vehicle under the same lens are merged to generate single-lens merged data under the same lens.

[0105] In this step, a single shot may generate multiple, discontinuous tracking data for the same vehicle due to factors such as occlusion, lighting changes, or vehicle movement. The mapped vehicle tracking data for each shot can be merged within a single shot to generate single-shot merged data. By merging these scattered tracking data, the system can generate continuous, complete vehicle tracking data. This not only reduces redundant information, but also lays the foundation for cross-shot data integration and global vehicle tracking.

[0106] In one embodiment, before step 403, the process further includes: performing data cleaning on the mapped vehicle tracking data to eliminate redundant data, where the redundant data includes one or more of the following data: vehicle tracking data for a specified time period, tracking data in which the number of vehicle key points is less than a preset number, and tracking data in which the area of ​​the vehicle in a preset map space is less than a preset area threshold.

[0107] In this embodiment, before single-shot merging, the mapped vehicle tracking data is first cleaned to remove redundant data. This redundant data includes, but is not limited to, vehicle tracking data from a specified time period, tracking data with fewer than a preset number of vehicle key points, and tracking data with a vehicle area within a preset map space less than a preset area threshold. Through this cleaning process, the system effectively removes unnecessary or low-quality data, thereby reducing data processing complexity and computational burden, ensuring that only high-quality and highly relevant data is used in the subsequent merging step, and improving the accuracy and reliability of the single-shot merged data.

[0108] Taking a car dealership monitoring scenario as an example, the vehicle tracking data for a specified time period could be data from the dealership's non-business hours. Before merging vehicle tracking data, vehicle tracks from non-business hours need to be removed. Non-business hours typically occur at night (for example, before 8:00 AM and after 9:00 PM). Due to poor lighting conditions, vehicle detection and attribute recognition results are unreliable. Furthermore, there are no customer visits during these hours. Therefore, vehicles from these hours can be removed beforehand to minimize the impact of low-quality reported data on vehicle track merging and reduce the subsequent data computation workload. Furthermore, vehicles that are distant, heavily obscured, or partially visible also need to be removed beforehand. These vehicles have a small or incomplete projection on the ground, making it difficult to effectively match their position with more complete vehicles captured by other cameras. Therefore, they also need to be filtered out before merging. The preset number and area threshold can be set based on actual needs. For example, the preset number could be 3, and the area threshold could be 2 square meters. Therefore, the filtering conditions could be as follows:

[0109] (a) Vehicle tracking data reporting time: time < 8:00 or time > 21:00.

[0110] (b) The number of keypoints of a vehicle is less than 3.

[0111] (c) The projected area of ​​the vehicle on the real-world map of the car dealership is less than 2 square meters.

[0112] If the vehicle tracking data meets any of the above conditions, the vehicle tracking data will be eliminated.

[0113] In one embodiment, step 403 may specifically include determining time information and location information corresponding to each vehicle tracking data item in the mapped vehicle tracking data. Under the same shot, candidate vehicle tracking data items with non-overlapping time periods are selected from the mapped vehicle tracking data items based on the time information. Based on the location information, different vehicle tracking data items with overlapping locations greater than a first threshold are merged within the candidate vehicle tracking data items under the same shot to generate single-shot merged data under the same shot.

[0114] In this embodiment, single-shot merging combines split tracks belonging to the same vehicle within a single shot, resulting in a single track for each vehicle, thus eliminating the problem of excessive number of vehicles due to track splitting. Taking the monitoring scenario of a car dealership as an example, the vehicles in a car dealership are generally stationary exhibit vehicles, so their positions are generally fixed. However, vehicles occasionally move from one exhibit to another, which can cause vehicle tracking to be interrupted, resulting in multiple split tracks. The gathering of people around a vehicle can also cause vehicle tracking to be interrupted, resulting in split tracks. Single-shot merging primarily utilizes the time period and map location of the tracks to achieve track merging.

[0115] First, the time and location information corresponding to each vehicle tracking data point is determined to facilitate accurate screening and merging. Within the same shot, the system uses the time information to select candidate vehicle tracking data points that do not overlap in time, ensuring temporal continuity during the merging process. Next, the system uses the location information to merge different vehicle tracking data points within these candidate data points where the overlap area exceeds a first threshold, generating single-shot merged data. This process effectively eliminates data redundancy and errors caused by time overlap or inconsistent locations, improving data accuracy and integrity.

[0116] The first threshold can be set according to actual needs, for example, the first threshold can be 40%. For any two vehicle tracking data vehicle track1 and vehicle track2 under a single shot, the single shot merging rules may include:

[0117] 1. If vehicle track1 and vehicle track2 overlap in time, they cannot be merged.

[0118] 2. If the overlap area of ​​vehicle track1 and vehicle track2 on the map exceeds 40%, the two are merged.

[0119] Rule 1 prevents misclassification of vehicles. Time overlap indicates the simultaneous presence of two vehicles, and therefore they must belong to different vehicles. Rule 2 addresses highly overlapping tracks, which are likely to belong to the same vehicle. Slight fluctuations in position may be due to human interference, leading to unstable detection results, or long-term occlusion, leading to splitting.

[0120] In one embodiment, step 403 specifically further includes: based on the position information, merging different vehicle tracking data in the candidate vehicle tracking data under the same shot, whose position overlap area is greater than the second threshold and less than the first threshold and whose vehicle attribute information is the same, to generate single-shot merged data under the same shot.

[0121] In this embodiment, within the same shot, not only time and location information are considered, but vehicle attribute information can also be incorporated as a basis for merging. First, within the same shot, the system filters out candidate vehicle tracking data with non-overlapping time based on time information. Based on location information, it filters out candidate vehicle tracking data with a location overlap area greater than a second threshold and less than a first threshold. Then, by comparing the vehicle attribute information within these filtered data, different vehicle tracking data with the same vehicle attribute information are identified. Vehicle attribute information can include discrete attribute information such as the vehicle's brand, sub-brand, and color, or continuous attribute information such as vehicle ReID features. By merging different vehicle tracking data with the same vehicle attribute information, single-shot merged data is generated. This ensures that even when location overlap is not significant, the system can accurately merge tracking data belonging to the same vehicle by matching vehicle attribute information. This effectively reduces the fragmentation of tracking data that can occur due to insignificant location overlap, improving data consistency and reliability.

[0122] The second threshold can be set according to actual needs and is smaller than the first threshold. For example, the second threshold can be 10%. Taking the aforementioned car dealership monitoring scenario as an example, for any two vehicle tracking data vehicle track1 and vehicle track2 under a single lens, the single lens merging rules can also include:

[0123] 3. If the overlapping area of ​​vehicle track 1 and vehicle track 2 on the real space map is less than 40% but more than 10%, and their attributes such as vehicle model and color are the same, then the two are merged.

[0124] Rule 3 addresses situations where vehicles experience some movement. In real-world scenarios, it's rare for vehicles within a dealership to move significant distances on the same day. If they're located close together and have identical attributes, it's likely just a partial movement within the dealership. This strategy effectively merges split tracks from the same shot, yielding a complete tracking sequence for the same vehicle in a single shot.

[0125] Step 404: Based on the single-shot merged data corresponding to the multiple shots, the tracking data belonging to the same vehicle under the multiple shots are merged to generate unique tracking data of the same vehicle under the multiple shots.

[0126] In this step, by merging vehicle tracking data across lenses, globally unique tracking data for the same vehicle under multiple lenses is generated, realizing the integration and optimization of multi-lens vehicle tracking data. This not only improves the continuity and accuracy of vehicle tracking in the multi-lens monitoring system, but also reduces the possibility of tracking interruption or misidentification caused by different shooting angles of multiple lenses, significantly improving the accuracy and efficiency of the monitoring system.

[0127] In one embodiment, before step 404, the process further includes: removing vehicle tracking data with a duration less than a preset duration from the single-shot merged data corresponding to the multiple shots, and / or removing vehicle tracking data with a vehicle size greater than a preset size from the single-shot merged data corresponding to the multiple shots.

[0128] In this embodiment, before merging the multi-shot tracking data, the single-shot merged data can be preprocessed to improve data quality and subsequent processing efficiency. Vehicle tracking data with a duration less than a preset duration can be removed from the single-shot merged data corresponding to the multiple shots, ensuring that only data with sufficient observation time is retained, thereby reducing erroneous data caused by brief appearances or misidentifications. Furthermore, vehicle tracking data with a size larger than a preset size can be removed to eliminate abnormal or inaccurate data. This data cleaning process effectively filters out low-quality data that could affect the accuracy of the merge, ensuring the accuracy and reliability of subsequent cross-shot data merging.

[0129] The duration refers to the total duration of the vehicle track obtained after stitching together the different track durations of the same vehicle within a single shot in step 403. The preset dimensions include, but are not limited to, the length, width, aspect ratio, or area of ​​the vehicle.

[0130] Taking the aforementioned monitoring scenario of a car dealership as an example, before merging cross-lens data, for the tracks merged by a single lens (i.e., single-lens merged data), obviously unreasonable tracks can be filtered out. Unreasonable situations include but are not limited to: 1. The life cycle of the vehicle track is too short. This is likely to be a misdetected vehicle or a short-term parked vehicle outdoors. 2. The size of the vehicle in the vehicle track is obviously unreasonable. This may be a vehicle that is only partially visible at the edge of the image, or it may be caused by reasons such as too large deviations in chassis point detection. It needs to be filtered out in time to avoid adverse effects on subsequent cross-border header merging. The preset duration and preset size can be set according to the actual scenario requirements. For example, the preset duration can be 2 hours, and the preset size can include a vehicle length threshold of 10 meters, a vehicle area threshold of 50 square meters, or a vehicle aspect ratio threshold of 10. At this time, the filtering strategy can be as follows:

[0131] 1. The life cycle of vehicle tracking is less than 2 hours.

[0132] 2. The length of the vehicle in the track exceeds 10 meters, the area exceeds 50 square meters, or the length-to-width ratio exceeds 10.

[0133] Vehicle tracks that meet any of the above rules in the single-shot merged data are eliminated.

[0134] In one embodiment, step 404 may specifically include determining the location information and / or vehicle attribute information corresponding to each vehicle tracking data item in the single-shot merged data. Based on the location information and / or vehicle attribute information, performing a one-to-one matching of the single-shot merged data items across multiple shots to obtain matching pairs of vehicle tracking data across different shots. The matching pairs across multiple shots are then aggregated to generate unique tracking data for the same vehicle across multiple shots.

[0135] In this embodiment, during the cross-shot data merging process, the location information and / or vehicle attribute information of the vehicle tracking data in each single-shot merged data is first determined to provide a basis for subsequent matching. The system then uses this information to perform one-to-one matching across multiple shots, forming matching pairs of vehicle tracking data from different shots. This matching process ensures accurate association of cross-shot data and identifies tracking data belonging to the same vehicle across different shots. These matching pairs are then aggregated to generate unique tracking data for the same vehicle across multiple shots. This effectively integrates tracking information from different perspectives and locations, providing complete and continuous vehicle tracking data.

[0136] Taking the aforementioned monitoring scene of the car dealership as an example, after the vehicle filtering and single-shot merging in the aforementioned embodiment, a "pure" vehicle track is obtained under each shot. "Pure" refers to the removal of noisy vehicles on the one hand. On the other hand, it means that the vehicle splitting problem has been solved. There is only one track for each vehicle, that is, each track corresponds to an independent vehicle. Cross-border head merging is to perform the merging of the same vehicle on this basis. Its core is to use the positional relationship of the vehicle on the unified real space map and the vehicle's own attributes to achieve accurate vehicle association. Cross-shot merging needs to meet the following principles:

[0137] 1. One-to-one matching: The vehicles in the two cameras to be matched are matched at most one-to-one, not many-to-one or one-to-many. That is, the vehicles in a single camera cannot be merged because the vehicles in a single camera have already been merged in the above step 403.

[0138] 2. Proximity similarity: Vehicles that can be merged must be located close to each other and have similar attributes.

[0139] One-to-one matching is achieved because vehicles in the same shot have been deduplicated during the single-shot merging phase. Each track retained corresponds to an independent vehicle, so these tracks cannot be merged. If there is an overlapping field of view between two shots, the vehicles under them can only be matched one-to-one to ensure that the vehicles in each shot are not merged. Proximity similarity means that only vehicles that are close enough in the real space map and have similar attributes can be merged. Optionally, if a vehicle in one shot has multiple similar vehicles in another shot, it should be matched to the one with the highest overall score.

[0140] In one embodiment, based on position information and / or vehicle attribute information, single-shot merged data from multiple shots are matched one-to-one to obtain matching pairs of vehicle tracking data from different shots. This includes calculating, for any two shots, similarity scores between the different vehicle tracking data from the two shots based on the position information and / or vehicle attribute information. Using the similarity scores as matching scores, bipartite graph assignment matching is performed on the different vehicle tracking data from the two shots with the goal of maximizing the matching score to obtain matching pairs corresponding to the different vehicle tracking data from the two shots.

[0141] In this embodiment, vehicle matching is performed for each of multiple shots. For any two shots, a similarity score is calculated between the different vehicle tracking data. This similarity score reflects the degree of similarity between the vehicles in different shots. This similarity score is used as a matching score. By maximizing the matching score, a bipartite graph assignment matching method is employed to determine the optimal matching pair of different vehicle tracking data from any two shots. This ensures that tracking data belonging to the same vehicle can be accurately identified and associated across different camera perspectives. By maximizing the matching score, mismatches can be effectively reduced, improving matching accuracy and reliability. This not only enhances the accuracy of cross-shot data integration but also provides a solid data foundation for subsequent vehicle tracking and analysis.

[0142] like Figure 6 As shown in FIG. 1 , a schematic diagram of bipartite graph assignment matching provided by an embodiment of the present application is provided. One-to-one matching is exactly a bipartite graph assignment problem. Assume that there are M cars under lens A (M is a positive integer, Figure 6 track11, track12, track13, track14 as an example) and N cars under lens B (N is a positive integer Figure 6 Tracks (using track21, track22, and track23 as an example) are assigned and matched based on their matching scores in a bipartite graph. Tracks from two cameras can only be matched one-to-one. The goal is to find the best matching method to maximize the overall matching score. This is a classic task assignment problem.

[0143] Optionally, embodiments of the present application can rapidly perform bipartite graph assignment matching using the "Hungarian algorithm." The similarity score between two vehicles in different shots is defined as the matching score between two vehicles in different shots. This can simultaneously satisfy both the one-to-one matching rule and the proximity similarity rule.

[0144] Optionally, the similarity scores between two vehicles under different lenses can be determined according to spatial distance and attribute similarity, including but not limited to the overlap, Euclidean distance and attribute similarity between two vehicles under different lenses. In actual scenarios, different similarity measurement methods can be selected according to actual needs, or multiple similarity measurement methods can be combined and then weighted summed to obtain the total similarity score.

[0145] Optionally, the overlap score is highly correlated with the position and orientation of the two vehicles. Only vehicles with similar positions and orientations can have a high overlap rate. The calculation method is as follows: Figure 7 As shown in Figure 2, for the quadrilaterals of two vehicles (vehicle 1 and vehicle 2) projected on the map plane, the overlap score is defined as the ratio of their intersection area to their joint area. Formula (1) is as follows:

[0146] (1)

[0147] Among them, quadrilateral 1 represents the position box of vehicle 1 in the real space map of the car store, quadrilateral 2 is the position box of vehicle 2 in the real space map of the car store, and IOU is the intersection-over-union ratio between quadrilateral 1 and quadrilateral 2. The intersection-over-union ratio can be defined as the overlap score between vehicle 1 and vehicle 2.

[0148] Alternatively, the distance score can also measure the distance between two vehicles. It is less powerful than the overlap score, but has a wider range of applications. The overlap score requires that there must be an intersection area between the two vehicles, otherwise the score is zero. Therefore, it cannot measure the relative distance between two vehicles without an intersection area. The Euclidean distance is a useful supplement to this. Figure 8 As shown in Figure 2, the Euclidean distance score Dist between vehicle 1 and vehicle 2 can be determined using the following formula (2):

[0149] (2)

[0150] Where center1 is the center point of the location box of vehicle 1 in the real-world map of the auto shop, center2 is the center point of the location box of vehicle 2 in the real-world map of the auto shop, w1 is the width of the circumscribed rectangle of vehicle 1's location box, and h1 is the length of the circumscribed rectangle of vehicle 1's location box. w2 is the width of the circumscribed rectangle of vehicle 2's location box, and h2 is the length of the circumscribed rectangle of vehicle 2's location box. Formula (2) uses the width and height of the circumscribed rectangle of the location box as normalization factors to eliminate differences in vehicle scale at different distances, resulting in a more robust measurement result.

[0151] Optionally, the attribute similarity between two vehicles can be determined by comparing the vehicle attribute information. Vehicle attributes can include one or more discrete attributes such as brand, sub-brand, and color, or continuous attributes such as vehicle ReID features. Here, sub-brand and color attributes are used as examples. Other attributes can be expanded accordingly based on their characteristics. For discrete attributes such as sub-brand and color, conditional scores can be configured according to the following formula (3) to numerically express discrete similarities:

[0152] (3)

[0153] Here, "Attr" represents the attribute similarity score between two vehicles. "Top n" is the statistical ranking of the multi-frame recognition results for a certain attribute (e.g., sub-brand) in the vehicle track sequence (n is a positive integer). "Top 1" represents the attribute recognition result with the largest percentage of the recognition results for that attribute in a single-shot vehicle track, and "Top 2" represents the second-largest percentage. For example, a vehicle track sequence contains 20 frames of attribute recognition results. Due to various interference factors, the attribute recognition results in each frame may not be completely consistent. Suppose that 12 frames have the attribute recognition result of "001" (sub-brand) of Brand A, 5 frames have the attribute recognition result of "002" (sub-brand) of Brand A, and 3 frames have the attribute recognition result of "003" (sub-brand) of Brand A. Then, the top 1 sub-brand for this vehicle track is "001", and the top 2 sub-brand is "002". Assuming the similarity score ranges from 0 to 1, if two vehicle tracks have the same top-1 sub-brand and color attributes (for example, both have the top-1 attribute 001 red), then their similarity score is assigned Attr = 1.0, representing maximum attribute similarity. This assignment is based on the importance and distinguishability of the attribute information. Generally, vehicle sub-brands are more distinguishing; vehicles with different sub-brands are definitely different vehicles. Because different vehicles can have the same color, and color is significantly affected by lighting, color is less distinguishable than brand. Therefore, vehicle sub-brands are given higher priority in scoring.

[0154] Alternatively, for continuous attributes (such as vehicle ReID features), the feature distance between two vehicles can be directly used to measure the similarity between the vehicles.

[0155] In actual scenarios, vehicle attribute information can be expanded as needed and is not limited to the sub-brands and colors listed above. The configuration method of attribute similarity can also be designed accordingly according to the characteristics of the attribute information, and is not limited to the conditional assignment method given in the above embodiment. For example, for large car dealerships, vehicles of multiple brands are exhibited at the same time, so the vehicle brand can be further subdivided into brands (such as car brand S, car brand P) and sub-brands (such as sub-brand S1 under car brand S, sub-brand P1 under car brand P). Some exhibition cars will also be affixed with model nameplates, and license plate detection and recognition can be further added to assist in the precise matching of vehicles. Vehicle ReID features include but are not limited to the vehicle's appearance features, local features, deep learning features, and spatiotemporal features. Therefore, vehicle ReID features are also valuable identification information, and the distance of ReID features can be used as a dimension to measure the similarity between two vehicles. The above-mentioned vehicle attribute information can be used as basic metadata in the embodiments of this application to improve the accuracy of vehicle matching.

[0156] Optionally, based on the three sub-scores of overlap, Euclidean distance, and attribute similarity, the comprehensive matching score between two vehicle tracks can be defined as:

[0157] (4)

[0158] Here, score represents the comprehensive matching score between two vehicle tracks. For any lens A and lens B, which contain m and n tracks respectively, the matching score (i, j) between the two lenses can be calculated using formula (4), and an mxn matching matrix M is obtained. Then, the Hungarian algorithm is used to solve the optimal assignment result, where m, n, and M are positive integers.

[0159] Optionally, threshold filtering can be performed to obtain the best dispatch result. Each matching pair has a corresponding matching score. Only two vehicle tracks with matching scores greater than a certain threshold truly belong to the same vehicle, ensuring the accuracy of the final result.

[0160] The above matching process takes the matching between two lenses as an example. For the multiple lenses contained in the car store, we can first perform pairwise matching, and then aggregate the results of the pairwise matching to obtain a comprehensive merge result, so as to completely merge the same car shot by multiple lenses with overlapping fields of view.

[0161] In actual scenarios, when aggregating multiple lenses, the "one-to-one matching" principle may be violated. The reasons can be found in Figure 9 As shown in the example, assume there are four cameras, A, B, C, and D, with two cars under camera A. Using pairwise merging (A<->B, B<->C, C<->D, and D<->A), if the two cars under camera A are directly aggregated, they will eventually be merged together. This violates the "one-to-one matching" principle and will result in incorrect aggregation. Therefore, this embodiment of the application provides a solution for conflict resolution during multi-channel aggregation, as follows:

[0162] In one embodiment, matching pairs under multiple lenses are aggregated in pairs to generate unique tracking data for the same vehicle under multiple lenses, including the following steps: traversing the matching pairs under multiple lenses, and if both vehicle tracking data in the current matching pair have not been aggregated, aggregating the two vehicle tracking data in the current matching pair into tracking data for the same vehicle. If the first vehicle tracking data in the current matching pair has been aggregated into a first set, the second vehicle tracking data in the current matching pair has not been aggregated, and the first set does not contain vehicle tracking data for the lens to which the second vehicle tracking data belongs, aggregating the second vehicle tracking data into the first set. If the current matching pair has been aggregated into two different sets, and vehicle tracking data for the same lens does not exist in the two different sets, merging the two different sets into a single set. If the current matching pair has been aggregated into two different sets, and vehicle tracking data for the same lens exists in the two different sets, discarding the current match. If single vehicle tracking data that cannot be paired exists under multiple lenses, generating a single cluster for the single vehicle tracking data.

[0163] In this embodiment, during the aggregation process, all matching pairs are traversed, and the two vehicle tracking data that have not yet been aggregated in the current matching pair are aggregated into tracking data of the same vehicle. In the case where the current matching pair has been aggregated, if the first vehicle tracking data in the current matching pair is already in the first set, and the second vehicle tracking data has not been aggregated and the vehicle tracking data under the lens to which the second vehicle tracking data belongs does not exist in the first set, then the second vehicle tracking data is added to the set. If the two vehicle tracking data of the current matching pair have been aggregated into two different sets respectively, and there is no vehicle tracking data under the same lens in these two sets, then the two sets are merged. Otherwise, if there is vehicle tracking data under the same lens in these two sets, in order to ensure that the vehicle tracking data under the same lens will not be aggregated, the current matching pair is eliminated. In this way, accurate aggregation of vehicle tracking data is ensured and duplication and incorrect matching are avoided.

[0164] In addition, for single vehicle tracking data that cannot be paired, a single cluster can be generated for it, ensuring accurate aggregation of vehicle tracking data.

[0165] Optionally, the polymerization process steps are exemplified as follows:

[0166] Input:

[0167] Matched pair sequence <(track1 i ,camera1 i ),(track2 i ,camera2 i ), score i >,i=1,2,…,N.

[0168] Initialization: Aggregate cluster collection = .

[0169] Among them, i represents the serial number of the matching pair, track1 i Indicates the vehicle track of the i-th matching pair under the lens camera1, track2 i Indicates the vehicle track score of the i-th matching pair under the lens camera2 i Indicates track1 i With track2 i Matching score between .

[0170] Step 1:

[0171] The input matching pair sequence is scored according to the matching score i Sort from largest to smallest.

[0172] Step 2:

[0173] Traverse the sorted matching pair sequence <(track1 i ,camera1 i ),(track2 i ,camera2 i ),score i >, find the set where track1_i is located in the current aggregation set and the set where track2_i is located :

[0174] a. If and , then add a new cluster to the aggregation set .

[0175] b. If and and camera1 is not included i The track under i ,camera1 i )join in At this time (track2 i ,camera2 i ) is the first vehicle tracking data that has been aggregated in the current matching pair, is the first set, and (track1i,camera1i) is the second vehicle tracking data.

[0176] c. If and and camera2 is not included i The track below, then (track2 i ,camera2 i )join in At this time, (track1i, camera1i) is the first vehicle tracking data that has been aggregated in the current matching pair. For the first set, (track2 i ,camera2 i ) is the second vehicle tracking data.

[0177] d. If and ,if and If there is no track with the same lens, and In contrast, if and If there are tracks under the same lens in the dataset, the introduction of this matching pair will cause a conflict and the matching pair needs to be removed.

[0178] Step 3:

[0179] For all (track1 i ,camera1 i ), if it is not in any of the sets in Step 2, then the track alone constitutes a cluster .

[0180] Output:

[0181] Aggregate Clusters = .

[0182] This method effectively avoids aggregation conflicts and ensures that the resulting aggregation results adhere to the "one-to-one matching" principle. In the aggregation results, tracks in each cluster belong to the same vehicle. For isolated tracks that are not in the clustered set in the single-shot merging results, meaning that the vehicle appears in only one shot, they constitute a separate vehicle.

[0183] In one embodiment, the focus of the car retail business is on in-store display vehicles. Car stores often use glass curtain wall designs, and vehicles parked outside the store will also be captured by the surveillance camera. To improve data accuracy, it is necessary to filter out vehicles parked outside the store. Filtering out vehicles outside the store can be achieved using the real-world space map of the car store. This embodiment provides the following filtering strategy:

[0184] (1) If the ratio of the intersection area between the projection of a vehicle’s track on the real-world map of the car store and the car store’s area to the vehicle’s projection area is less than a third threshold (which can be set according to actual conditions, such as 25%), the vehicle is considered to be an out-of-store vehicle.

[0185] (2) If the ratio of the intersection area between the projection of a vehicle’s track on the real-space map of the car store and the car store’s scope to the vehicle’s projection area is less than the fourth threshold (the fourth threshold can be set to be greater than the third threshold according to actual conditions, for example, the fourth threshold is 75%), and the vehicle is only captured by one camera, then the vehicle is an out-of-store vehicle.

[0186] The above filtering strategy can be used to filter the unique tracking data of the same vehicle under multiple lenses obtained in step 404, and the vehicle tracking data that meets any of the above filtering strategies can be eliminated from the unique tracking data to improve the accuracy of the final result.

[0187] like Figure 10 The figure shows a flow chart of a vehicle tracking data processing method provided by an embodiment of the present application. Taking the monitoring scenario of a car dealership as an example, the solution may include two parts: vehicle detection and tracking on the edge side and cross-lens vehicle merging on the service side. Assume that the car dealership includes multiple surveillance cameras, namely lens 1, lens 2...lens N (where N is a positive integer). Among them:

[0188] The edge side, based on the device-side vehicle detection and tracking algorithms, performs vehicle detection, attribute recognition, and tracking for each camera lens. The resulting track and accompanying metadata, such as vehicle attributes, are reported to the server. The server then performs coordinate mapping, detection box filtering, single-shot merging, track filtering, cross-border header merging, and filtering for vehicles outside of stores on the metadata reported by multiple cameras, achieving cross-camera vehicle merging and outputting the final merging results.

[0189] Through the six processing steps described above, this embodiment effectively eliminates interference factors such as false detection, split tracking, and vehicles parked outside the store, resulting in an accurate aggregation of vehicles within the store. Ultimately, each vehicle is assigned a globally unique ID, which incorporates detection and tracking results captured by multiple cameras from different perspectives. Each camera captures relevant information such as vehicle attributes and customer visit history. By aggregating these multiple monitoring results, comprehensive and automated vehicle-related data can be achieved. This allows for precise digital collection of a store's active customer base and the popularity of each model, providing timely and accurate vehicle market information.

[0190] It should be noted that the solution of this application is based on the underlying vehicle detection, tracking and map calibration. The above is only an example of the embodiment of this application. Any detection and tracking algorithm and map construction and calibration method in actual scenarios can serve as the underlying support for the embodiment of this application, and does not limit the scope of application of this application.

[0191] The vehicle tracking data processing method described above aims to solve the cross-border header merging problem for vehicles displayed in automotive stores. It specifically addresses the identification of stationary indoor vehicles across multiple cameras and determines their common location on a floor plan map. It proposes an effective solution that leverages multiple sources of information, including temporal, spatial, and attribute information. This method follows a hierarchical model, first performing single-point merging within a camera, then performing multi-point merging of cross-border headers. For cross-camera merging, it proposes one-to-one matching and nearest similarity merging principles. This method fully exploits the mutual exclusion check within a single piece of information and the complementary assistance between multiple pieces of information, overcoming potential merging errors caused by low calibration accuracy and achieving highly accurate cross-border header association, providing important support for the digitalization of automotive marketing.

[0192] First of all, for the special scenario of car dealerships, there is no need to emphasize the use of license plates or vehicle ReID features. On the one hand, these two features are not always applicable in car dealership scenarios. For example, some new cars do not have nameplates, and different lenses capture completely different perspectives of the vehicle (such as the front and back). The ReID features are very different, and vehicle merging based on license plates and ReID features will fail. Even the recognition of discrete attributes such as vehicle brand and color is not necessarily very accurate. Simply relying on vehicle attributes will inevitably lead to mis-aggregation and omission, and it is difficult to meet the accuracy requirements of business applications. The embodiment of the present application does not emphasize a specific vehicle attribute, but focuses on exploring the comprehensive application value of information in the three dimensions of time, space and attributes, and makes up for the lack of accuracy of single information through their mutual complementation. In response to the problem that technologies that have been effective in the field of smart transportation are no longer applicable in car dealership scenarios, the method proposed in the embodiment of the present application has overcome the shortcomings of the existing technology and has its own technical uniqueness.

[0193] Secondly, the solution provided by the embodiment of the present application solves the technical problem of vehicle merging under low annotation accuracy. If vehicle deduplication is achieved by relying on spatial matching based on high-precision calibration, the manual calibration method requires the calibrator to have very rich annotation experience, and multi-view shooting modeling requires the deployment of enough cameras with overlapping fields of view, which requires a lot of manpower and material resources, and brings great challenges to the implementation cost. Under the premise of low annotation accuracy and controllable implementation costs, how to achieve high-precision cross-lens vehicle deduplication is the technical challenge that the present invention aims to overcome. Regarding the technical difficulties of cross-border vehicle merging under conditions of low calibration accuracy:

[0194] (1) The embodiment of the present application proposes a progressive merging mode from single point to multi-point. First, low-quality data is filtered out by business hours and projected area, and then single-point merging is performed within a single shot by mutual exclusion in time and overlap in position. Single-point merging does not involve the problem of labeling accuracy, and both data quality and merging effect can be well guaranteed. Performing cross-border multi-point merging on the basis of obtaining high-quality single-point data can eliminate the interference of noise data to the greatest extent and reduce the difficulty of subsequent merging.

[0195] (2) In the cross-border header merging process involving calibration accuracy, based on single-point merging, the one-to-one matching and nearest similarity principles that need to be followed by multi-point matching are proposed, and the Hungarian matching algorithm is used to fully utilize these two principles. In view of the aggregation conflicts that may occur in multi-point cascade merging under low annotation accuracy, an effective conflict elimination algorithm is proposed, which effectively overcomes the merging errors that may be caused by insufficient annotation accuracy and greatly reduces the high requirements of annotation accuracy for high-accuracy merging. The merging effect required by the business can be achieved with lower manpower and cost investment, which has stronger practicality.

[0196] Through innovations in strategy design and underlying technologies, the embodiments of the present application achieve comprehensive utilization of multi-source and multi-dimensional information such as time information, spatial information, and attribute information, and ultimately achieve highly accurate ID deduplication of cross-border vehicles under low calibration accuracy conditions.

[0197] For details of each step of the above vehicle tracking data processing method, please refer to the relevant description of the above related embodiments, which will not be repeated here.

[0198] Please see Figure 11 , which is a flow chart of a method for determining vehicle information according to an embodiment of the present application, taking the determination of vehicle popularity in a monitoring scenario of a car dealership as an example, including:

[0199] Step 1101: Acquire unique tracking data of a target vehicle in a target area under multiple lenses. The unique tracking data is obtained by using the vehicle tracking data processing method of any of the aforementioned embodiments.

[0200] Step 1102: Determine a user visit event for the target vehicle based on the unique tracking data.

[0201] Step 1103: Determine the attention information of the target vehicle according to the user's visit event.

[0202] The solution of the embodiment of the present application, when it is necessary to perform information analysis on the target vehicle in the target area, first obtains the unique tracking data of the target vehicle in the target area under multiple lenses, and the unique tracking data is obtained by adopting the vehicle tracking data processing method of any of the aforementioned embodiments. The accuracy and consistency of the identification and tracking of the same target vehicle at different viewing angles and positions are ensured. The unique tracking data records event information related to the target vehicle. Based on the unique tracking data, user visit events for the target vehicle can be determined, and the user visit events reflect the user's actual attention behavior to the target vehicle. By analyzing the user visit events, the target vehicle's attention information can be further determined. The attention information may include indicators such as the user's interest level in the vehicle, attention duration, and interaction frequency, providing important data support for subsequent market analysis, user behavior research, and personalized recommendations.

[0203] For details of each step of the above-mentioned vehicle information determination method, please refer to the relevant description of the aforementioned related embodiments, which will not be repeated here.

[0204] Please see Figure 12 , which is a vehicle tracking data processing device 1200 according to an embodiment of the present application, which can be applied to Figure 1 The electronic device 1 shown can be applied to Figure 2-3 In the application scenario shown in , its implementation not only improves the continuity and accuracy of vehicle tracking in a multi-camera monitoring system, but also reduces the possibility of tracking interruptions or misidentification caused by different shooting angles of multiple cameras, significantly improving the accuracy and efficiency of the monitoring system. The device includes: an acquisition module 1201, a mapping module 1202, a first merging module 1203, and a second merging module 1204. The functional principles of each module are as follows:

[0205] The acquisition module 1201 is used to acquire the vehicle tracking data to be processed under multiple lenses in the target area.

[0206] The mapping module 1202 is used to map the vehicle tracking data to be processed to a preset map space to obtain mapped vehicle tracking data.

[0207] The first merging module 1203 is configured to merge different tracking data belonging to the same vehicle under the same lens according to the mapped vehicle tracking data, to generate single-lens merged data under the same lens.

[0208] The second merging module 1204 is configured to merge the tracking data belonging to the same vehicle under multiple shots according to the single shot merging data corresponding to the multiple shots, and generate unique tracking data of the same vehicle under multiple shots.

[0209] In one embodiment, the acquisition module 1201 is used to receive the vehicle tracking data to be processed sent by the edge computing device corresponding to multiple lenses in the target area. The vehicle tracking data to be processed is obtained by the edge computing device through vehicle tracking detection based on the monitoring data under the corresponding lens.

[0210] In one embodiment, the preset map space is the map space of the real-world scene covered by the target area. Mapping module 1202 is configured to determine the image coordinates of the vehicle in the surveillance image based on the vehicle tracking data to be processed. A real-world map of the real-world scene covered by the target area is determined. Based on camera calibration data for multiple lenses, the image coordinates are mapped to the real-world map to obtain the map coordinates of the vehicle on the real-world map. The mapped vehicle tracking data includes the map coordinates of the vehicle from the multiple lenses.

[0211] In one embodiment, the first merging module 1203 is configured to determine the time information and location information corresponding to each vehicle tracking data item in the mapped vehicle tracking data. Based on the time information, candidate vehicle tracking data items with non-overlapping time periods are selected from the mapped vehicle tracking data items for the same shot. Based on the location information, different vehicle tracking data items with overlapping locations greater than a first threshold are merged to generate single-shot merged data for the same shot.

[0212] In one embodiment, the first merging module 1203 is further configured to merge, based on the position information, different vehicle tracking data in the candidate vehicle tracking data under the same shot whose overlapping area is greater than the second threshold and less than the first threshold and whose vehicle attribute information is the same, to generate single-shot merged data under the same shot.

[0213] In one embodiment, the apparatus further includes: a first elimination module for performing data cleaning on the mapped vehicle tracking data to eliminate redundant data before merging different tracking data belonging to the same vehicle under the same lens to generate single-lens merged data under the same lens, wherein the redundant data includes one or more of the following data: vehicle tracking data within a specified time period, tracking data in which the number of vehicle key points is less than a preset number, and tracking data in which the area of ​​the vehicle in a preset map space is less than a preset area threshold.

[0214] In one embodiment, the second merging module 1204 is configured to determine the location information and / or vehicle attribute information corresponding to each vehicle tracking data item in the single-shot merged data. Based on the location information and / or vehicle attribute information, the single-shot merged data items across multiple shots are matched one-to-one to obtain matching pairs of vehicle tracking data across different shots. The matching pairs across multiple shots are then aggregated to generate unique tracking data for the same vehicle across multiple shots.

[0215] In one embodiment, the second merging module 1204 is specifically configured to calculate, for any two shots, a similarity score between different vehicle tracking data captured by the two shots based on position information and / or vehicle attribute information. The similarity score is used as a matching score, and bipartite graph assignment matching is performed on the different vehicle tracking data captured by the two shots to maximize the matching score, thereby obtaining matching pairs corresponding to the different vehicle tracking data captured by the two shots.

[0216] In one embodiment, the second merging module 1204 is specifically configured to traverse matching pairs under multiple lenses, and if both vehicle tracking data in the current matching pair have not been aggregated, aggregate the two vehicle tracking data in the current matching pair into tracking data of the same vehicle. If the first vehicle tracking data in the current matching pair has been aggregated into the first set, the second vehicle tracking data in the current matching pair has not been aggregated, and the first set does not contain vehicle tracking data under the lens to which the second vehicle tracking data belongs, aggregate the second vehicle tracking data into the first set. If the current matching pair has been aggregated into two different sets, and the two different sets do not contain vehicle tracking data under the same lens, merge the two different sets into the same set. If the current matching pair has been aggregated into two different sets, and the two different sets contain vehicle tracking data under the same lens, remove the current match. If there is single vehicle tracking data under multiple lenses that cannot be paired, generate a single cluster for the single vehicle tracking data.

[0217] In one embodiment, the apparatus further includes a second culling module configured to, before merging the tracking data belonging to the same vehicle under multiple shots based on the single-shot merging data corresponding to the multiple shots to generate unique tracking data for the same vehicle under the multiple shots, culling vehicle tracking data with a duration less than a preset duration from the single-shot merging data corresponding to the multiple shots, and / or culling vehicle tracking data with a size greater than a preset size from the single-shot merging data corresponding to the multiple shots.

[0218] For a detailed description of the vehicle tracking data processing device 1200 , please refer to the description of the relevant method steps in the above embodiment. The implementation principles and technical effects are similar and will not be repeated here in this embodiment.

[0219] Figure 13 This is a schematic diagram of the structure of a cloud device 130 provided in an exemplary embodiment of the present application. The cloud device 130 can be used to run the method provided in any of the above embodiments. Figure 13 As shown, the cloud device 130 may include: a memory 1304 and at least one processor 1305, Figure 13 A processor is used as an example.

[0220] The memory 1304 is used to store computer programs and can be configured to store various other data to support operations on the cloud device 130. The memory 1304 can be an object storage service (OSS).

[0221] The memory 1304 may be implemented by any type of volatile or non-volatile memory device or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic memory, flash memory, magnetic disk, or optical disk.

[0222] The processor 1305 is coupled to the memory 1304 and is used to execute the computer program in the memory 1304 to implement the solution provided by any of the above method embodiments. The specific functions and technical effects that can be achieved are not repeated here.

[0223] Furthermore, if Figure 13 The cloud device also includes: a firewall 1301, a load balancer 1302, a communication component 1306, a power supply component 1303 and other components. Figure 13 Only some components are shown schematically, which does not mean that the cloud device only includes Figure 13 Components shown.

[0224] In one embodiment, the above Figure 13 The communication component 1306 is configured to facilitate wired or wireless communication between the device in which the communication component 1306 resides and other devices. The device in which the communication component 1306 resides can access a wireless network based on a communication standard, such as WiFi, 2G, 3G, 4G, LTE (Long Term Evolution), 5G, or other mobile communication networks, or a combination thereof. In an exemplary embodiment, the communication component 1306 receives broadcast signals or broadcast-related information from an external broadcast management system via a broadcast channel. In an exemplary embodiment, the communication component 1306 also includes a near-field communication (NFC) module to facilitate short-range communication. For example, the NFC module can be implemented based on radio frequency identification (RFID) technology, infrared data association (IrDA) technology, ultra-wideband (UWB) technology, Bluetooth (BT) technology, or other technologies.

[0225] In one embodiment, the above Figure 13 The power supply component 1303 provides power to various components of the device where the power supply component 1303 is located. The power supply component 1303 may include a power management system, one or more power supplies, and other components associated with generating, managing, and distributing power to the device where the power supply component is located.

[0226] An embodiment of the present application further provides a computer-readable storage medium, in which computer-executable instructions are stored. When a processor executes the computer-executable instructions, the method of any of the aforementioned embodiments is implemented.

[0227] An embodiment of the present application also provides a computer program product, including a computer program, which implements the method of any of the aforementioned embodiments when executed by a processor.

[0228] In the several embodiments provided in this application, it should be understood that the disclosed devices and methods can be implemented in other ways. For example, the device embodiments described above are merely illustrative. For example, the module division is only a logical function division. In actual implementation, other division methods may be used. For example, multiple modules can be combined or integrated into another system, or some features can be ignored or not implemented.

[0229] The integrated modules implemented in the form of software function modules can be stored in a computer-readable storage medium. The software function modules stored in a storage medium include a number of instructions for causing a computer device (which can be a personal computer, server, or network device, etc.) or a processor to execute some of the steps of the methods of various embodiments of the present application.

[0230] It should be understood that the above-mentioned processor can be a central processing unit (CPU), or other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), etc. The general-purpose processor can be a microprocessor or any conventional processor. The steps of the method disclosed in the application can be directly implemented as being executed by a hardware processor, or can be executed by a combination of hardware and software modules in the processor. The memory may include high-speed RAM (Random Access Memory) memory, and may also include non-volatile storage NVM (NVM), such as at least one disk storage, and can also be a USB flash drive, a mobile hard disk, a read-only memory, a magnetic disk or an optical disk, etc.

[0231] The above-mentioned storage medium can be implemented by any type of volatile or non-volatile storage device, or a combination thereof, such as static random-access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic storage, flash memory, magnetic disk, or optical disk. The storage medium can be any available medium that can be accessed by a general-purpose or special-purpose computer.

[0232] An exemplary storage medium is coupled to a processor so that the processor can read information from the storage medium and write information to the storage medium. Of course, the storage medium can also be an integral part of the processor. The processor and the storage medium can be located in an application specific integrated circuit (ASIC). Of course, the processor and the storage medium can also exist as discrete components in an electronic device or a main control device.

[0233] It should be noted that, in this document, the terms "comprises," "includes," or any other variations thereof are intended to encompass non-exclusive inclusion, such that a process, method, apparel, or apparatus comprising a series of elements includes not only those elements but also other elements not explicitly listed, or elements inherent to such process, method, apparel, or apparatus. In the absence of further limitations, an element defined by the phrase "comprising a ..." does not exclude the presence of other identical elements in the process, method, apparel, or apparatus comprising the element.

[0234] The serial numbers of the above-mentioned embodiments of the present application are for description only and do not represent the advantages or disadvantages of the embodiments.

[0235] Through the description of the above embodiments, those skilled in the art can clearly understand that the above-mentioned embodiment methods can be implemented by means of software plus the necessary general hardware platform. Of course, they can also be implemented by hardware, but in many cases the former is a better embodiment. Based on this understanding, the technical solution of this application, or the part that contributes to the existing technology, can be embodied in the form of a software product. This computer software product is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk), and includes a number of instructions for enabling a terminal device (which can be a mobile phone, computer, server, air conditioner, or network device, etc.) to execute the methods of each embodiment of this application.

[0236] In the technical solution of this application, the collection, storage, use, processing, transmission, provision and disclosure of user data and other information involved comply with the provisions of relevant laws and regulations and do not violate public order and good morals.

[0237] The above are only preferred embodiments of the present application and do not limit the patent scope of the present application. Any equivalent structure or equivalent process transformation made using the contents of the present application specification and drawings, or directly or indirectly applied in other related technical fields, are also included in the patent protection scope of the present application.

Claims

1. A vehicle tracking data processing method, characterized in that: include: Obtain vehicle tracking data to be processed under multiple lenses in the target area; Mapping the to-be-processed vehicle tracking data to a preset map space to obtain mapped vehicle tracking data; Merging different tracking data belonging to the same vehicle under the same lens according to the mapped vehicle tracking data to generate single-lens merged data under the same lens; According to the single-shot merged data corresponding to the multiple shots, the tracking data belonging to the same vehicle under the multiple shots are merged to generate unique tracking data of the same vehicle under the multiple shots.

2. The method according to claim 1, characterized in that The method of obtaining the vehicle tracking data to be processed under multiple lenses in the target area includes: Receive the vehicle tracking data to be processed sent by the edge computing device corresponding to the multiple lenses in the target area, where the vehicle tracking data to be processed is obtained by the edge computing device through vehicle tracking detection based on the monitoring data under the corresponding lenses.

3. The method according to claim 1, characterized in that The preset map space is a map space of a real scene covered by the target area; Mapping the to-be-processed vehicle tracking data to a preset map space to obtain mapped vehicle tracking data includes: Determining the image coordinates of the vehicle in the monitoring image according to the vehicle tracking data to be processed; Determine a real space map of the real scene covered by the target area; According to the camera calibration data of the multiple lenses, the image coordinates are mapped to the real space map to obtain the map coordinates of the vehicle on the real space map, and the mapped vehicle tracking data includes the map coordinates of the vehicle under the multiple lenses.

4. The method according to claim 1, wherein The method of merging different tracking data belonging to the same vehicle under the same lens according to the mapped vehicle tracking data to generate single-lens merged data under the same lens includes: Determining time information and location information corresponding to each vehicle tracking data in the mapped vehicle tracking data; Under the same lens, selecting candidate vehicle tracking data with non-overlapping time from the mapped vehicle tracking data according to the time information; According to the position information, different vehicle tracking data with position overlap areas greater than a first threshold in the candidate vehicle tracking data under the same shot are merged to generate single-shot merged data under the same shot.

5. The method according to claim 4, characterized in that The step of merging different tracking data belonging to the same vehicle under the same lens according to the mapped vehicle tracking data to generate single-lens merged data under the same lens further includes: Based on the position information, different vehicle tracking data with the same vehicle attribute information and a position overlap area greater than a second threshold and less than the first threshold in the candidate vehicle tracking data under the same lens are merged to generate single-shot merged data under the same lens.

6. The method according to claim 1, wherein Before merging different tracking data belonging to the same vehicle under the same lens according to the mapped vehicle tracking data to generate single-lens merged data under the same lens, the method further includes: The mapped vehicle tracking data is cleaned to remove redundant data, where the redundant data includes one or more of the following: vehicle tracking data for a specified time period, tracking data in which the number of vehicle key points is less than a preset number, and tracking data in which the area of ​​the vehicle in the preset map space is less than a preset area threshold.

7. The method according to claim 1, characterized in that The step of merging the tracking data belonging to the same vehicle under the multiple lenses according to the single-lens merged data corresponding to the multiple lenses to generate unique tracking data of the same vehicle under the multiple lenses includes: Determining the position information and / or vehicle attribute information corresponding to each vehicle tracking data in the single-shot merged data; performing one-to-one matching on the single-shot merged data between the multiple shots according to the position information and / or the vehicle attribute information to obtain matching pairs of vehicle tracking data between different shots; The matching pairs under the multiple shots are aggregated in pairs to generate unique tracking data of the same vehicle under the multiple shots.

8. The method according to claim 7, characterized in that According to the position information and / or the vehicle attribute information, one-to-one matching is performed on the single-shot merged data between the multiple shots to obtain vehicle tracking data matching pairs between different shots, including: For any two shots, respectively calculating similarity scores between different vehicle tracking data under the any two shots based on the position information and / or the vehicle attribute information; The similarity score is used as a matching score, and bipartite graph assignment matching is performed on the different vehicle tracking data under the arbitrary two lenses for the purpose of maximizing the matching score to obtain the matching pairs corresponding to the different vehicle tracking data under the arbitrary two lenses.

9. The method according to claim 7, characterized in that Aggregating the matching pairs under the multiple shots in pairs to generate unique tracking data of the same vehicle under the multiple shots includes: Traversing the matching pairs under the multiple shots, if the two vehicle tracking data in the current matching pair are not aggregated, aggregating the two vehicle tracking data in the current matching pair into tracking data of the same vehicle; If the first vehicle tracking data in the current matching pair has been aggregated into the first set, and the second vehicle tracking data in the current matching pair has not been aggregated, and the vehicle tracking data under the lens to which the second vehicle tracking data belongs does not exist in the first set, then the second vehicle tracking data is aggregated into the first set; If the current matching pairs have been aggregated into two different sets, and there is no vehicle tracking data under the same lens in the two different sets, then the two different sets are merged into the same set; If the current matching pair has been aggregated into two different sets, and vehicle tracking data under the same lens exists in the two different sets, the current matching pair is discarded; If there is unpaired single vehicle tracking data under the multiple shots, a single cluster is generated for the single vehicle tracking data.

10. The method according to claim 1, characterized in that Before merging the tracking data belonging to the same vehicle under the multiple lenses according to the single-lens merging data corresponding to the multiple lenses to generate unique tracking data of the same vehicle under the multiple lenses, the method further includes: Eliminate vehicle tracking data with a duration shorter than a preset duration from the single-shot merged data corresponding to the multiple shots; and / or Vehicle tracking data with a vehicle size larger than a preset size is eliminated from the single-shot merged data corresponding to the multiple shots.

11. A method for determining vehicle information, characterized in that: include: Acquire unique tracking data of a target vehicle in a target area under multiple lenses, wherein the unique tracking data is obtained using the method according to any one of claims 1 to 10; determining a user visit event for the target vehicle according to the unique tracking data; The attention information of the target vehicle is determined according to the user visit event.

12. An electronic device, characterized in that: include: at least one processor; as well as a memory communicatively coupled to the at least one processor; The memory stores instructions that can be executed by the at least one processor, and the instructions are executed by the at least one processor to enable the electronic device to perform the method according to any one of claims 1 to 11.

13. A computer-readable storage medium, characterized in that The computer-readable storage medium stores computer-executable instructions, and when a processor executes the computer-executable instructions, the method according to any one of claims 1 to 11 is implemented.

14. A computer program product, characterized in that The invention comprises a computer program, which, when executed by a processor, implements the method according to any one of claims 1 to 11.

Images