Motion track redisk method and device based on video track and GPS fusion
By integrating and correcting video trajectories with GPS trajectories, the problems of insufficient trajectory review accuracy and limited application scenarios in the existing technology are solved, and more efficient and accurate trajectory review effects are achieved, supporting action analysis in complex scenarios.
Patent Information
- Application Number
- CN202510085290.2
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2025-01-20
- Publication Date
- 2025-05-16
AI Technical Summary
The existing technology has insufficient trajectory review accuracy and limited application scenarios in complex action scenarios, resulting in poor trajectory review efficiency.
The action trajectory review method based on the fusion of video trajectory and GPS is adopted. By obtaining GPS positioning information and video surveillance screen, the pedestrian feature recognition and field of view segmentation positioning algorithm are used to generate video trajectory, and the video trajectory and GPS trajectory are fused and corrected to obtain a more accurate action review trajectory.
It improves the accuracy and positioning efficiency of trajectory review, can achieve accurate trajectory recording and analysis in complex scenarios, and enhances the support capabilities of law enforcement and investigation.
Smart Images
Figure CN120014296A_ABST
Abstract
Description
Technical Field
[0001] The present application belongs to the field of trajectory review technology, and more specifically, relates to an action trajectory review method and device based on the fusion of video trajectory and GPS. Background Art
[0002] At present, the core of action trajectory review is to accurately record each space-time trajectory and locate it through GPS technology. In the process of reconnaissance or arrest, it is common to use GPS equipment for positioning and tracking. This positioning method can achieve accurate positioning in large-scale spaces of more than 10 meters in wide outdoor environments, but the positioning accuracy is insufficient in indoor scenes such as urban commercial buildings, street community campuses, etc. First, because the GPS signal is shielded by buildings, it is difficult to penetrate multiple layers of walls to reach indoors; second, the plane positioning accuracy is not high enough to locate the exact position in a small indoor place; third, the height deviation is large, the height error is 2-3 times the plane distance error, and the height value jumps greatly.
[0003] In order to make up for the insufficient accuracy of GPS indoor positioning, WiFi positioning and Bluetooth positioning technologies have also been applied. The application premise of such technologies is relatively harsh, requiring special base stations to be deployed indoors and the person being tracked to wear corresponding beacon positioning devices. Currently, they are only used in a few special places such as smart manufacturing factories and prisons, and are not universally applicable.
[0004] Therefore, how to solve the difficulty of trajectory review in complex action scenarios and improve the accuracy and positioning efficiency of trajectory review is a technical problem that needs to be solved urgently. Summary of the invention
[0005] In view of the defects of the prior art, the purpose of this application is to provide a method and device for action trajectory review based on the fusion of video trajectory and GPS, aiming to solve the problem of poor efficiency of trajectory review caused by insufficient trajectory review accuracy and limited application scenarios.
[0006] To achieve the above objectives, the present application provides an action trajectory review method based on the fusion of video trajectory and GPS, including:
[0007] Obtain GPS positioning information during the action and obtain GPS action trajectory;
[0008] Retrieving monitoring equipment operating within the target range of the GPS action trajectory, obtaining video monitoring images of the monitoring equipment during the action process, searching and identifying pedestrians in the video monitoring images according to pedestrian features, locating the positions of pedestrians in the video by using field of view segmentation, and connecting the positions in series to obtain the video trajectory; the pedestrian features include one or more of facial features, facial attribute features, and body features;
[0009] The video trajectory and the GPS action trajectory are subjected to trajectory fusion and trajectory correction to obtain an action review trajectory.
[0010] In some embodiments, searching and identifying pedestrians in the video surveillance image based on their features, locating the positions of pedestrians in the video using field of view segmentation, and concatenating the positions to obtain a video trajectory include:
[0011] Performing pedestrian detection on the video surveillance screen, and when a pedestrian is detected in the video surveillance screen, extracting facial features, facial attribute features, and body features of the pedestrian frame by frame to obtain a feature set of pedestrian features;
[0012] Perform feature comparison between the feature set and the established action library, and if any pedestrian feature is successfully matched, retain the video trajectory point corresponding to the current frame;
[0013] Obtain the longitude and latitude of the monitoring device, the camera installation height, the pitch angle of the center of the viewport, and the camera orientation angle, and calculate the longitude and latitude of the pedestrian's location through a field of view segmentation positioning algorithm using the longitude and latitude, the camera installation height, the pitch angle of the center of the viewport, and the camera orientation angle;
[0014] The above-mentioned feature comparison and field of view segmentation and positioning process is repeated to obtain all pedestrian position points, and all the pedestrian position points are connected in series in time sequence to obtain the video trajectory.
[0015] In some embodiments, the monitoring devices operating within the target range of the GPS motion trajectory are determined by:
[0016] Converting the GPS motion trajectory into a plane coordinate set; the plane coordinate set includes the first plane coordinates of each GPS positioning point;
[0017] Acquire the second plane coordinates of the monitoring device, and calculate the distance from the second plane coordinates to each first plane coordinate to determine the minimum distance;
[0018] When the minimum distance value is less than the target threshold, the monitoring device corresponding to the minimum distance value is used as the monitoring device operating within the target range of the GPS motion trajectory.
[0019] In some embodiments, the step of fusing and correcting the video trajectory and the GPS action trajectory to obtain an action review trajectory includes:
[0020] Fusing the video trajectory and the GPS action trajectory in chronological order to obtain a fused trajectory;
[0021] Traversing all the track points in the fused track, determining whether the track points have signal loss, position drift, and position conflict, and obtaining a determination result of the track points;
[0022] The trajectory is corrected based on the judgment result to obtain the action review trajectory.
[0023] In some embodiments, performing trajectory correction based on the judgment result includes:
[0024] When the time interval between two consecutive trajectory points is within a preset time range, it is determined that there is a short-term signal loss in the fused trajectory, and the trajectory is corrected by trajectory extrapolation;
[0025] When the distance between two consecutive trajectory points is greater than the preset distance and the time interval does not exceed the pre-examination time, the trajectory point is determined to be position drift, the discontinuity of the previous and next trajectory points is calculated, and the trajectory point with smaller discontinuity is retained for trajectory correction;
[0026] When the distance between the GPS motion track in the fused track and the video track points that appear alternately in the same time period is greater than the preset distance, it is determined that there is a position conflict in the fused track, and the GPS motion track is smoothly corrected based on the continuous track points of the video track.
[0027] In some embodiments, the trajectory correction by trajectory extrapolation is implemented using the following formula:
[0028]
[0029] Among them, p n is the trajectory correction point, p n-1 is the first track point before the track is lost, p n-2 is the second track point before the track is lost, t n-1 is the interval time between the first trajectory point and the second trajectory point, t n The price time of the trajectory correction point and the first trajectory point.
[0030] In some embodiments, the calculation of the discontinuity of the previous and next trajectory points and retaining the trajectory points with smaller discontinuity for trajectory correction is implemented by the following formula:
[0031]
[0032] Among them, p n-1 、p n 、p n+1 、p n+2 are four consecutive trajectory points, l n For p n Continuity of the front and back, ln+1 For p n+1 Continuity before and after.
[0033] In a second aspect, the present application also provides an action trajectory review device based on the fusion of video trajectory and GPS, comprising:
[0034] The GPS action trajectory acquisition module is used to obtain the GPS positioning information during the action process and obtain the GPS action trajectory;
[0035] A video trajectory acquisition module is used to retrieve monitoring equipment operating within the target range of the GPS action trajectory, obtain the video monitoring screen of the monitoring equipment during the action, search and identify the pedestrian in the video monitoring screen according to the pedestrian characteristics, and locate the position of the pedestrian in the video by using the field of view segmentation, and connect the positions in series to obtain the video trajectory; the pedestrian characteristics include one or more of facial features, facial attribute features and body features;
[0036] The trajectory fusion module is used to perform trajectory fusion and trajectory correction on the video trajectory and the GPS action trajectory to obtain an action review trajectory.
[0037] In a third aspect, the present application provides an electronic device comprising: at least one memory for storing programs; and at least one processor for executing the programs stored in the memory. When the program stored in the memory is executed, the processor is used to execute the method described in the first aspect or any possible implementation of the first aspect.
[0038] In a fourth aspect, the present application provides a computer-readable storage medium, which stores a computer program. When the computer program runs on a processor, the processor executes the method described in the first aspect or any possible implementation of the first aspect.
[0039] In a fifth aspect, the present application provides a computer program product. When the computer program product runs on a processor, the processor executes the method described in the first aspect or any possible implementation of the first aspect.
[0040] It can be understood that the beneficial effects of the second to fifth aspects mentioned above can be found in the relevant description of the first aspect mentioned above, and will not be repeated here.
[0041] In general, the above technical solutions conceived by this application have the following beneficial effects compared with the prior art:
[0042] (1) By integrating GPS action trajectories and video surveillance images, the system can more accurately track the target's location and activities during the action, thereby improving the accuracy and reliability of tracking. The combination of GPS action trajectories and video surveillance images can provide more detailed action information, thereby improving the accuracy and efficiency of trajectory review.
[0043] (2) This application uses video trajectories to correct GPS positioning and uses GPS positioning to fill in blind spots in video trajectories, thereby achieving accurate and coherent recording of indoor and outdoor actions and solving the problem of difficulty in trajectory review in complex action scenarios.
[0044] (3) This application can improve the reliability and integrity of the obtained action review trajectory by generating video trajectories and correcting GPS action trajectories. The generated action review trajectory can help law enforcement agencies conduct case investigation and analysis, restore and analyze the target's action process, and help discover clues, reconstruct case plots, and promote case detection. BRIEF DESCRIPTION OF THE DRAWINGS
[0045] Figure 1 It is a flowchart of an action trajectory review method based on video trajectory and GPS fusion provided in an embodiment of the present application;
[0046] Figure 2 It is a schematic diagram of the feature matching process provided in the embodiment of the present application;
[0047] Figure 3 It is one of the flow diagrams of the video segmentation and trajectory calculation diagram provided in the embodiment of the present application;
[0048] Figure 4 This is the second flow chart of the video segmentation and trajectory calculation schematic diagram provided in the embodiment of the present application;
[0049] Figure 5 It is a schematic diagram of the process of trajectory correction fusion provided in an embodiment of the present application;
[0050] Figure 6 It is a structural schematic diagram of an action trajectory replay device based on video trajectory and GPS fusion provided in an embodiment of the present application;
[0051] Figure 7 It is a schematic diagram of the structure of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION
[0052] In order to make the purpose, technical solution and advantages of the present application more clearly understood, the present application is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application.
[0053] The term "and / or" in this article is a description of the association relationship of associated objects, indicating that there can be three relationships. For example, A and / or B can represent: A exists alone, A and B exist at the same time, and B exists alone. The symbol " / " in this article indicates that the associated objects are in an or relationship, for example, A / B means A or B.
[0054] The terms "first" and "second" in the specification and claims herein are used to distinguish different objects rather than to describe a specific order of the objects. For example, a first response message and a second response message are used to distinguish different response messages rather than to describe a specific order of the response messages.
[0055] In the embodiments of the present application, words such as "exemplary" or "for example" are used to indicate examples, illustrations or descriptions. Any embodiment or design described as "exemplary" or "for example" in the embodiments of the present application should not be interpreted as being more preferred or more advantageous than other embodiments or designs. Specifically, the use of words such as "exemplary" or "for example" is intended to present related concepts in a specific way.
[0056] In the description of the embodiments of the present application, unless otherwise specified, "multiple" means two or more than two. For example, multiple processing units refer to two or more processing units, etc.; multiple elements refer to two or more elements, etc.
[0057] Next, the technical solutions provided in the embodiments of the present application are introduced.
[0058] Reference Figure 1 The present application provides an action trajectory review method based on the fusion of video trajectory and GPS, including:
[0059] S101. Obtain GPS positioning information during the action and obtain GPS action trajectory;
[0060] S102. Retrieve the monitoring equipment operating within the target range of the GPS action trajectory, obtain the video monitoring screen of the monitoring equipment during the action, search and identify the pedestrian in the video monitoring screen according to the pedestrian characteristics, and use the field of view segmentation to locate the position of the pedestrian in the video, and connect the positions in series to obtain the video trajectory; the pedestrian characteristics include one or more of facial features, facial attribute features and body features;
[0061] S103: Fusing and correcting the video trajectory and the GPS action trajectory to obtain an action replay trajectory.
[0062] First, the real-time location information of the target is obtained through a GPS device or a mobile device, and the location data is recorded to form a GPS action track. This track will show the target's movement path and location information during the action.
[0063] It should be noted that after obtaining the GPS positioning information, it is necessary to perform denoising preprocessing on the positioning information and use the preprocessed positioning information data to obtain the GPS movement trajectory.
[0064] Specifically, in this embodiment, GPS positioning information is collected through positioning devices such as smart phones, portable positioning devices, beacons, etc., and the GPS information is encapsulated into a custom format message of [id, collection time, longitude, latitude, altitude, speed, azimuth, plane accuracy, altitude accuracy, speed accuracy, azimuth accuracy] and returned through the message queue.
[0065] After receiving GPS information, the plane accuracy and altitude accuracy values of the GPS point are read. Points with plane, altitude and accuracy within 10 meters are marked as high-precision points, points with plane and altitude accuracy between 10-30 meters are recorded as low-precision points, and points with plane accuracy or altitude accuracy greater than 30 meters are discarded, thus forming a GPS movement trajectory.
[0066] Furthermore, the monitoring equipment running within the target range of the GPS action trajectory is retrieved, and the four static factors of the equipment's latitude and longitude, floor height, field of view, and time offset can be obtained. Then, the monitoring screen is obtained through the monitoring equipment, and the pedestrian characteristics are retrieved and identified to obtain the video trajectory. The monitoring equipment running within the target range will be retrieved, which may include fixed cameras, mobile cameras, or drones, etc., to obtain the video monitoring screens recorded by these devices during the action process.
[0067] Pedestrian features such as facial features, facial attribute features and body features are used for retrieval and recognition, so as to accurately identify the target person in the video surveillance screen and generate the target's movement trajectory in the video.
[0068] Finally, the video trajectory and GPS action trajectory are integrated and corrected. By comparing the data of the two, the system can correct the possible errors in the GPS action trajectory, and combine the video trajectory to supplement more accurate position and movement information, so as to obtain a more accurate action review trajectory. The review trajectory will comprehensively display the actual position and activity of the target during the action, providing detailed data support for subsequent analysis.
[0069] By integrating GPS positioning information and video surveillance images, and using pedestrian features for retrieval and identification, we can ultimately generate an accurate action review trajectory, comprehensively track and restore the target's action process, and provide a strong data foundation for subsequent analysis and decision-making.
[0070] By integrating GPS action trajectories and video surveillance images, the system can more accurately track the target's location and activities during the action, thereby improving the accuracy and credibility of tracking. The combination of GPS action trajectories and video surveillance images can provide more detailed action information, thereby improving the accuracy and efficiency of trajectory review.
[0071] Video trajectories are used to correct GPS positioning, and GPS positioning is used to fill in blind spots in video trajectories, achieving accurate and coherent recording of indoor and outdoor actions, and solving the problem of difficulty in trajectory review in complex action scenarios.
[0072] In some embodiments, searching and identifying pedestrians in the video surveillance image based on their features, locating the positions of pedestrians in the video using field of view segmentation, and concatenating the positions to obtain a video trajectory include:
[0073] Performing pedestrian detection on the video surveillance screen, and when a pedestrian is detected in the video surveillance screen, extracting facial features, facial attribute features, and body features of the pedestrian frame by frame to obtain a feature set of pedestrian features;
[0074] Perform feature comparison between the feature set and the established action library, and if any pedestrian feature is successfully matched, retain the video trajectory point corresponding to the current frame;
[0075] Obtain the longitude and latitude of the monitoring device, the camera installation height, the pitch angle of the center of the viewport, and the camera orientation angle, and calculate the longitude and latitude of the pedestrian's location through a field of view segmentation positioning algorithm using the longitude and latitude, the camera installation height, the pitch angle of the center of the viewport, and the camera orientation angle;
[0076] The above-mentioned feature comparison and field of view segmentation and positioning process is repeated to obtain all pedestrian position points, and all the pedestrian position points are connected in series in time sequence to obtain the video trajectory.
[0077] In this embodiment, the camera performs pedestrian detection, detects the presence of a person in the picture, extracts the facial features, facial attribute features, and body features of the pedestrian frame by frame, and compares the feature set of the pedestrian with the action library.
[0078] If none of the three features matches one, the pedestrian is judged not to be an action and can be discarded;
[0079] If one or more features match the same, the video track point is retained and recorded at the location;
[0080] If the matches of some features are different, the identification will be made based on the order of facial features, facial attribute features and body features.
[0081] This embodiment compares the extracted pedestrian feature set with the established action library to identify known pedestrian identities or behavioral features, thereby realizing rapid identification and tracking of target pedestrians. If the feature comparison is successful, the video trajectory points corresponding to the current frame are retained, and the feature comparison is repeated to gradually generate all the video trajectory points, which are connected in series to obtain a complete video trajectory. By combining video surveillance images and pedestrian feature extraction technology, the accuracy and reliability of target pedestrian identification and trajectory generation can be improved, thereby improving the overall efficiency of trajectory review.
[0082] In some embodiments, the monitoring devices operating within the target range of the GPS motion trajectory are determined by:
[0083] Converting the GPS motion trajectory into a plane coordinate set; the plane coordinate set includes the first plane coordinates of each GPS positioning point;
[0084] Acquire the second plane coordinates of the monitoring device, and calculate the distance from the second plane coordinates to each first plane coordinate to determine the minimum distance;
[0085] When the minimum distance value is less than the target threshold, the monitoring device corresponding to the minimum distance value is used as the monitoring device operating within the target range of the GPS motion trajectory.
[0086] In this embodiment, an example is taken to retrieve available monitoring equipment within a 50-meter range (the farthest identification distance of personnel features) of the GPS action trajectory, and the screen segmentation positioning algorithm is used to further locate the target position to form a video trajectory point set.
[0087] First, we traverse the cameras used in the action one by one and calculate the devices whose plane coordinates meet the condition of "50-meter range". Taking a certain camera as an example, we calculate the closest distance:
[0088] Assume that the plane coordinate point set of the GPS trajectory is P(x,y) = {p0,p1,...p n}, x is longitude, y is latitude, n is the number of line segments; the camera position is recorded as p c (x c ,y c ), calculate the distance between the camera and each segment on the trajectory, take the nth segment p of the trajectory n-1 (x n-1 ,y n-1 ),p n (x n ,y n )Two points:
[0089] p n-1 to p n for:
[0090] p n-1 to p c for:
[0091] Furthermore, it is determined whether the projection of the camera in the trajectory direction is on the trajectory segment, and the projection directionality criterion is calculated as:
[0092]
[0093] When 0≤f≤1, p c The projection of the point is within the line segment, and the minimum distance is
[0094] When 1 <f,p c The projection of the point is on the forward extension of line segment n, at a distance p n Click the closest.
[0095] When f<0, p c The projection of the point is on the backward extension of line segment n-1, the distance p n-1 Click the closest.
[0096] p c The distances between the two vectors are opposite, representing p c Distance from point p on segment n of the trajectory n-1 The closest distance is d n for:
[0097] Therefore, the shortest distance calculated according to different f values is:
[0098]
[0099] Traverse all trajectory segments and get the minimum distance as the minimum distance between the camera and the GPS action trajectory.
[0100] Furthermore, the process of face recognition and feature comparison is exemplified as follows:
[0101] (1) OpenCV is used to extract frames from the video, labelme is used to annotate the pedestrian dataset, and YOLOv3 is used to train and detect pedestrians in the video. Darknet-53 is selected as the backbone network to extract pedestrian features;
[0102] (2) The open source InsightFace project is used for face recognition. The dataset uses the Chinese Academy of Sciences Asian Face Dataset CAS-PEAL, and ResNet is used as the backbone network to extract facial features of pedestrians.
[0103] (3) FaceAttribute-FAN, an open source project of Tencent Youtu, is used for facial attribute recognition. The network structure uses Facial abstraction net, and the backbone network uses resnet. The focus is on identifying the identity of the person based on aspects such as whether the person has a beard, fatness or thinness, gender, whether the person has makeup, the size of the glasses, whether the person has hair, whether the person is wearing a hat, and whether the person is wearing glasses.
[0104] (4) The Fast-ReID project was used for pedestrian body feature recognition. The Market-1501 dataset of Tsinghua University was used for training and testing. Random patch preprocessing software was used for data augmentation. ResNet was selected as the backbone network. Max pooling was used for global body feature aggregation. The training strategy was set to Learningrate. Kuangshi Circle loss was used as the loss function for model training.
[0105] (5) Using one or more of facial features, facial attribute features, and body features, the surveillance images are gradually searched and identified. The specific process is as follows: Figure 2 shown.
[0106] (6) After locating the video track point, obtain the latitude and longitude of the device c (x c ,y c ), camera installation height H, viewport center pitch angle A (A < 90°), camera orientation angle B, and use the field of view segmentation positioning algorithm to calculate the latitude and longitude of the location. Figure 3 and Figure 4 As shown, position p p (x p ,y p ) The estimation method is as follows:
[0107] The horizontal distance d from the center of the camera image to the camera is:
[0108] d=H*tanA
[0109] Assuming the camera monitoring angle is 2C (C<45° and C≤A), the nearest shot d min , Maximum shooting distance d max for:
[0110] d min =H*tan(AC)
[0111]
[0112] Take a 1K standard definition camera (1920*1080 resolution), assuming that the nearest edge of the video frame is y min= 0, the farthest edge y of the video frame max =1080, the person’s position in the video frame is y1, then the distance d from the person to the camera is taken p for,
[0113]
[0114] For the deviation angle α relative to the camera's field of view p for:
[0115] α p =B+tan -1 (x1-960)
[0116] Therefore, the camera is taken as the center of the polar coordinates, the north direction is the positive direction of the polar coordinates, and the polar coordinates of the person relative to the camera can be expressed as (d p , α p ), assuming that the camera coordinates are (lon1, lat1) and the person's coordinates are (lon2, lat2), use the polar coordinate conversion rectangular coordinate formula (taking the average polar radius 6371.393*1000m):
[0117]
[0118] The longitude and latitude coordinates of the pedestrian are calculated, and the height is taken as the floor height where the camera is located.
[0119] In some embodiments, the step of fusing and correcting the video trajectory and the GPS action trajectory to obtain an action review trajectory includes:
[0120] Fusing the video trajectory and the GPS action trajectory in chronological order to obtain a fused trajectory;
[0121] Traversing all the track points in the fused track, determining whether the track points have signal loss, position drift, and position conflict, and obtaining a determination result of the track points;
[0122] The trajectory is corrected based on the judgment result to obtain the action review trajectory.
[0123] like Figure 5 As shown, the specific process of trajectory correction fusion in this application is as follows:
[0124] Get GPS tracks and video tracks in chronological order;
[0125] Read position point by point;
[0126] Determine whether the signal at this point is lost for a short time, if so, extrapolate the trajectory to estimate;
[0127] Determine whether the point has drifted, if so, remove the abnormal trajectory;
[0128] Determine whether the point has a position conflict, if so, correct the trajectory smoothly;
[0129] GPS track and video track fusion.
[0130] Further, performing trajectory correction based on the judgment result includes:
[0131] When the time interval between two consecutive trajectory points is within a preset time range, it is determined that there is a short-term signal loss in the fused trajectory, and the trajectory is corrected by trajectory extrapolation;
[0132] This is achieved using the following formula:
[0133]
[0134] Among them, p n is the trajectory correction point, p n-1 is the first track point before the track is lost, p n-2 is the second track point before the track is lost, t n-1 is the interval time between the first trajectory point and the second trajectory point, t n The price time of the trajectory correction point and the first trajectory point.
[0135] It should be noted that the preset time range is 1-15 minutes, that is, the time interval between two trajectory points is more than 1 minute but less than 15 minutes.
[0136] When the distance between two consecutive trajectory points is greater than the preset distance and the time interval does not exceed the pre-examination time, the trajectory point is determined to be position drift, the discontinuity of the previous and next trajectory points is calculated, and the trajectory point with smaller discontinuity is retained for trajectory correction;
[0137] This is achieved through the following formula:
[0138]
[0139] Among them, p n-1 、p n 、p n+1 、p n+2 are four consecutive trajectory points, l n For p n Continuity of the front and back, l n+1 For p n+1 Continuity before and after.
[0140] When the distance between the GPS motion track in the fused track and the video track points that appear alternately in the same time period is greater than the preset distance, it is determined that there is a position conflict in the fused track, and the GPS motion track is smoothly corrected based on the continuous track points of the video track.
[0141] Specifically, if the distance between the GPS track and the video track points that appear alternately in the same time period is greater than 1 km, it is defined as a position conflict. The GPS track is smoothed using the continuous video track points (the difference between the two video track points does not exceed 1 minute), and the GPS track points between the continuous video tracks are deleted. Assume that the starting point of the continuous video track is p n , the previous GPS track point p n-1 , then inserting a smooth trajectory point p between two points is:
[0142]
[0143] Reference Figure 6 The present application also provides an action trajectory review device based on the fusion of video trajectory and GPS, including:
[0144] The GPS action trajectory acquisition module 610 is used to acquire the GPS positioning information during the action process and obtain the GPS action trajectory;
[0145] The video track acquisition module 620 is used to retrieve the monitoring equipment running within the target range of the GPS action track, obtain the video monitoring screen of the monitoring equipment during the action, search and identify the pedestrian in the video monitoring screen according to the pedestrian characteristics, and locate the position of the pedestrian in the video by using the field of view segmentation, and connect the positions in series to obtain the video track; the pedestrian characteristics include one or more of facial features, facial attribute features and body features;
[0146] The trajectory fusion module 630 is used to perform trajectory fusion and trajectory correction on the video trajectory and the GPS action trajectory to obtain an action review trajectory.
[0147] In some embodiments, searching and identifying pedestrians in the video surveillance image based on their features, locating the positions of pedestrians in the video using field of view segmentation, and concatenating the positions to obtain a video trajectory include:
[0148] Performing pedestrian detection on the video surveillance screen, and when a pedestrian is detected in the video surveillance screen, extracting facial features, facial attribute features, and body features of the pedestrian frame by frame to obtain a feature set of pedestrian features;
[0149] Perform feature comparison between the feature set and the established action library, and if any pedestrian feature is successfully matched, retain the video trajectory point corresponding to the current frame;
[0150] Obtain the longitude and latitude of the monitoring device, the camera installation height, the pitch angle of the center of the viewport, and the camera orientation angle, and calculate the longitude and latitude of the pedestrian's location through a field of view segmentation positioning algorithm using the longitude and latitude, the camera installation height, the pitch angle of the center of the viewport, and the camera orientation angle;
[0151] The above-mentioned feature comparison and field of view segmentation and positioning process is repeated to obtain all pedestrian position points, and all the pedestrian position points are connected in series in time sequence to obtain the video trajectory.
[0152] In some embodiments, the monitoring devices operating within the target range of the GPS motion trajectory are determined by:
[0153] Converting the GPS motion trajectory into a plane coordinate set; the plane coordinate set includes the first plane coordinates of each GPS positioning point;
[0154] Acquire the second plane coordinates of the monitoring device, and calculate the distance from the second plane coordinates to each first plane coordinate to determine the minimum distance;
[0155] When the minimum distance value is less than the target threshold, the monitoring device corresponding to the minimum distance value is used as the monitoring device operating within the target range of the GPS motion trajectory.
[0156] In some embodiments, the step of fusing and correcting the video trajectory and the GPS action trajectory to obtain an action review trajectory includes:
[0157] Fusing the video trajectory and the GPS action trajectory in chronological order to obtain a fused trajectory;
[0158] Traversing all the track points in the fused track, determining whether the track points have signal loss, position drift, and position conflict, and obtaining a determination result of the track points;
[0159] The trajectory is corrected based on the judgment result to obtain the action review trajectory.
[0160] In some embodiments, performing trajectory correction based on the judgment result includes:
[0161] When the time interval between two consecutive trajectory points is within a preset time range, it is determined that there is a short-term signal loss in the fused trajectory, and the trajectory is corrected by trajectory extrapolation;
[0162] When the distance between two consecutive trajectory points is greater than the preset distance and the time interval does not exceed the pre-examination time, the trajectory point is determined to be position drift, the discontinuity of the previous and next trajectory points is calculated, and the trajectory point with smaller discontinuity is retained for trajectory correction;
[0163] When the distance between the GPS motion track in the fused track and the video track points that appear alternately in the same time period is greater than the preset distance, it is determined that there is a position conflict in the fused track, and the GPS motion track is smoothly corrected based on the continuous track points of the video track.
[0164] In some embodiments, the trajectory correction by trajectory extrapolation is implemented using the following formula:
[0165]
[0166] Among them, p n is the trajectory correction point, p n-1 is the first track point before the track is lost, p n-2 is the second track point before the track is lost, t n-1 is the interval time between the first trajectory point and the second trajectory point, t n The price time of the trajectory correction point and the first trajectory point.
[0167] In some embodiments, the calculation of the discontinuity of the previous and next trajectory points and retaining the trajectory points with smaller discontinuity for trajectory correction is implemented by the following formula:
[0168]
[0169] Among them, p n-1 、p n 、p n+1 、p n+2 are four consecutive trajectory points, l n For p n Continuity of the front and back, l n+1 For p n+1 Continuity before and after.
[0170] It can be understood that the detailed functional implementation of each of the above-mentioned units / modules can be found in the introduction of the aforementioned method embodiment, and will not be repeated here.
[0171] It should be understood that the above-mentioned device is used to execute the method in the above-mentioned embodiment. The implementation principle and technical effect of the corresponding program module in the device are similar to those described in the above-mentioned method. The working process of the device can refer to the corresponding process in the above-mentioned method, which will not be repeated here.
[0172] Reference Figure 7Based on the method in the above embodiment, the embodiment of the present application provides an electronic device, which may include: a processor (processor) 710, a communication interface (Communications Interface) 720, a memory (memory) 730 and a communication bus 740, wherein the processor 710, the communication interface 720, and the memory 730 communicate with each other through the communication bus 740. The processor 710 can call the logic instructions in the memory 730 to execute the method in the above embodiment.
[0173] In addition, the logic instructions in the above-mentioned memory 730 can be implemented in the form of software functional units and can be stored in a computer-readable storage medium when sold or used as an independent product. Based on this understanding, the technical solution of the present application, or the part that contributes to the prior art, or the part of the technical solution, can be embodied in the form of a software product, which is stored in a storage medium and includes a number of instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present application.
[0174] Based on the method in the above embodiment, an embodiment of the present application provides a computer-readable storage medium, which stores a computer program. When the computer program runs on a processor, the processor executes the method in the above embodiment.
[0175] Based on the method in the above embodiment, an embodiment of the present application provides a computer program product. When the computer program product runs on a processor, the processor executes the method in the above embodiment.
[0176] It is understandable that the processor in the embodiment of the present application can be a central processing unit (CPU), or other general-purpose processors, digital signal processors (DSP), application specific integrated circuits (ASIC), field programmable gate arrays (FPGA) or other programmable logic devices, transistor logic devices, hardware components or any combination thereof. The general-purpose processor can be a microprocessor or any conventional processor.
[0177] The method steps in the embodiments of the present application can be implemented by hardware or by a processor executing software instructions. The software instructions can be composed of corresponding software modules, and the software modules can be stored in random access memory (RAM), flash memory, read-only memory (ROM), programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), registers, hard disks, mobile hard disks, CD-ROMs, or any other form of storage medium known in the art. An exemplary storage medium is coupled to a processor so that the processor can read information from the storage medium and write information to the storage medium. Of course, the storage medium can also be a component of the processor. The processor and the storage medium can be located in an ASIC.
[0178] In the above embodiments, it can be implemented in whole or in part by software, hardware, firmware or any combination thereof. When implemented using software, it can be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, the process or function described in the embodiment of the present application is generated in whole or in part. The computer may be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions may be stored in a computer-readable storage medium or transmitted through the computer-readable storage medium. The computer instructions may be transmitted from a website site, a computer, a server or a data center to another website site, a computer, a server or a data center by wired (e.g., coaxial cable, optical fiber, digital subscriber line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) means. The computer-readable storage medium may be any available medium that a computer can access or a data storage device such as a server or a data center that includes one or more available media integrated. The available medium may be a magnetic medium (e.g., a floppy disk, a hard disk, a tape), an optical medium (e.g., a DVD), or a semiconductor medium (e.g., a solid state disk (SSD)), etc.
[0179] It should be understood that the various numerical numbers involved in the embodiments of the present application are only used for the convenience of description and are not used to limit the scope of the embodiments of the present application.
[0180] It will be easily understood by those skilled in the art that the above description is only a preferred embodiment of the present application and is not intended to limit the present application. Any modifications, equivalent substitutions and improvements made within the spirit and principles of the present application shall be included in the scope of protection of the present application.
Claims
1. A method for replaying an action trajectory based on the fusion of video trajectory and GPS, characterized in that: include: Obtain GPS positioning information during the action and obtain GPS action trajectory; Retrieving monitoring equipment operating within the target range of the GPS action trajectory, obtaining video monitoring images of the monitoring equipment during the action process, searching and identifying pedestrians in the video monitoring images according to pedestrian features, locating the positions of pedestrians in the video by using field of view segmentation, and connecting the positions in series to obtain the video trajectory; the pedestrian features include one or more of facial features, facial attribute features, and body features; The video trajectory and the GPS action trajectory are subjected to trajectory fusion and trajectory correction to obtain an action review trajectory.
2. The method for reconstructing the action trajectory based on the fusion of video trajectory and GPS according to claim 1 is characterized in that: The method of searching and identifying pedestrians in the video surveillance screen according to the characteristics of pedestrians, locating the positions of pedestrians in the video by using field of view segmentation, and connecting the positions in series to obtain the video trajectory includes: Performing pedestrian detection on the video surveillance screen, and when a pedestrian is detected in the video surveillance screen, extracting facial features, facial attribute features, and body features of the pedestrian frame by frame to obtain a feature set of pedestrian features; Perform feature comparison between the feature set and the established action library, and if any pedestrian feature is successfully matched, retain the video trajectory point corresponding to the current frame; Obtain the longitude and latitude of the monitoring device, the camera installation height, the pitch angle of the center of the viewport, and the camera orientation angle, and calculate the longitude and latitude of the pedestrian's location through a field of view segmentation positioning algorithm using the longitude and latitude, the camera installation height, the pitch angle of the center of the viewport, and the camera orientation angle; The above-mentioned feature comparison and field of view segmentation and positioning process is repeated to obtain all pedestrian position points, and all the pedestrian position points are connected in series in time sequence to obtain the video trajectory.
3. The method for reconstructing the action trajectory based on the fusion of video trajectory and GPS according to claim 1 is characterized in that: The monitoring devices operating within the target range of the GPS motion trajectory are determined in the following manner: Converting the GPS motion trajectory into a plane coordinate set; the plane coordinate set includes the first plane coordinates of each GPS positioning point; Acquire the second plane coordinates of the monitoring device, and calculate the distance from the second plane coordinates to each first plane coordinate to determine the minimum distance; When the minimum distance value is less than the target threshold, the monitoring device corresponding to the minimum distance value is used as the monitoring device operating within the target range of the GPS motion trajectory.
4. The method for reconstructing the action trajectory based on the fusion of video trajectory and GPS according to claim 1 is characterized in that: The step of fusing and correcting the video trajectory and the GPS action trajectory to obtain an action review trajectory includes: Fusing the video trajectory and the GPS action trajectory in chronological order to obtain a fused trajectory; Traversing all the track points in the fused track, determining whether the track points have signal loss, position drift, and position conflict, and obtaining a determination result of the track points; The trajectory is corrected based on the judgment result to obtain the action review trajectory.
5. The method for reconstructing the action trajectory based on the fusion of video trajectory and GPS according to claim 4 is characterized in that: The performing trajectory correction based on the judgment result includes: When the time interval between two consecutive trajectory points is within a preset time range, it is determined that there is a short-term signal loss in the fused trajectory, and the trajectory is corrected by trajectory extrapolation; When the distance between two consecutive trajectory points is greater than the preset distance and the time interval does not exceed the pre-examination time, the trajectory point is determined to be position drift, the discontinuity of the previous and next trajectory points is calculated, and the trajectory point with smaller discontinuity is retained for trajectory correction; When the distance between the GPS motion track in the fused track and the video track points that appear alternately in the same time period is greater than the preset distance, it is determined that there is a position conflict in the fused track, and the GPS motion track is smoothly corrected based on the continuous track points of the video track.
6. The method for reconstructing the action trajectory based on the fusion of video trajectory and GPS according to claim 5 is characterized in that: The trajectory correction by trajectory extrapolation is implemented by the following formula: Among them, p n is the trajectory correction point, p n-1 is the first track point before the track is lost, p n-2 is the second track point before the track is lost, t n-1 is the interval time between the first trajectory point and the second trajectory point, t n The price time of the trajectory correction point and the first trajectory point.
7. The method for reconstructing the action trajectory based on the fusion of video trajectory and GPS according to claim 5 is characterized in that: The calculation of the discontinuity of the trajectory points before and after, and retaining the trajectory points with smaller discontinuity for trajectory correction, is achieved by the following formula: Among them, p n-1 、p n 、p n+1 、p n+2 are four consecutive trajectory points, l n For p n Continuity of the front and back, l n+1 For p n+1 Continuity before and after.
8. An action trajectory review device based on the fusion of video trajectory and GPS, characterized in that: include: The GPS action trajectory acquisition module is used to obtain the GPS positioning information during the action process and obtain the GPS action trajectory; A video trajectory acquisition module is used to retrieve monitoring equipment operating within the target range of the GPS action trajectory, obtain the video monitoring screen of the monitoring equipment during the action, search and identify the pedestrian in the video monitoring screen according to the pedestrian characteristics, and locate the position of the pedestrian in the video by using the field of view segmentation, and connect the positions in series to obtain the video trajectory; the pedestrian characteristics include one or more of facial features, facial attribute features and body features; The trajectory fusion module is used to perform trajectory fusion and trajectory correction on the video trajectory and the GPS action trajectory to obtain an action review trajectory.
9. An electronic device, characterized in that: include: at least one memory for storing a computer program; At least one processor is used to execute the program stored in the memory. When the program stored in the memory is executed, the processor is used to execute the method according to any one of claims 1 to 7.
10. A computer-readable storage medium storing a computer program, characterized in that: When the computer program runs on a processor, the processor is caused to execute the method according to any one of claims 1 to 7.