An action trajectory review method based on the fusion of video trajectory and GPS positioning

Abstract

The present invention discloses a method for replaying an action trajectory by integrating video trajectory and GPS positioning, comprising: obtaining the GPS positioning information of the actor during the action process, and performing denoising preprocessing on the positioning information to form a GPS trajectory; retrieving the available monitoring equipment position points within a certain range in the GPS action trajectory, obtaining the static factors of the monitoring equipment, collecting the video monitoring screen of the action period, and retrieving and identifying the action in the video monitoring screen, while determining the latitude and longitude, altitude and time of the actor's location, and connecting them in time sequence to form a video trajectory; using the GPS trajectory as a low-credibility trajectory and the video trajectory as a high-credibility trajectory, performing trajectory correction fusion to form an action replay trajectory. The present invention utilizes video trajectory to correct GPS positioning and utilizes GPS positioning to fill in the blind spots of video trajectory, thereby achieving accurate and coherent recording of indoor and outdoor actions and solving the problem of difficult trajectory positioning in complex action scenarios.

Description

Technical Field

[0001] The present invention belongs to the technical field of action trajectory replay or backtracking, and in particular to an action trajectory replay method that integrates video trajectory and GPS positioning. Background Art

[0002] The core of action trajectory review is to accurately record each spatiotemporal trajectory. Traditional methods use a single GPS technology for positioning. During reconnaissance or arrest operations, the use of GPS devices for positioning and tracking has become relatively common. This positioning method can achieve accurate positioning in large-scale spaces of more than 10 meters in wide outdoor environments, but the positioning accuracy is insufficient in indoor scenarios such as urban commercial buildings, street community campuses, etc. First, because the GPS signal is shielded by buildings, it is difficult to penetrate multiple layers of walls to reach indoors; second, the plane positioning accuracy is not high enough to accurately locate the position in a small indoor space; third, the height deviation is large, the height error is 2-3 times the plane distance error, and the height value fluctuates greatly.

[0003] In order to make up for the insufficient accuracy of GPS indoor positioning, WiFi positioning and Bluetooth positioning technologies have also been applied. The application prerequisites of this type of technology are relatively harsh, requiring special base stations to be deployed indoors and the tracked person to wear corresponding beacon positioning equipment. Currently, only a few of them are installed in very few special places such as smart manufacturing factories and prisons, and they are not universally applicable.

[0004] With the widespread adoption of video surveillance systems, most indoor urban areas, such as commercial complexes, entertainment venues like bars and karaoke bars, smart campuses, safe communities, and hotels, are now equipped with video surveillance equipment. These devices record the time and location of individuals within the surveillance footage. Multiple cameras, combined with the order in which individuals appear, form a video trajectory. However, while video trajectories are highly accurate due to the fixed location of the surveillance equipment, they are often less continuous due to their limited field of view. Once an individual leaves the field of view, their movement trajectory becomes undetectable. Summary of the Invention

[0005] The purpose of the present invention is to address the problems existing in the above-mentioned prior art and provide a method for replaying an action trajectory by integrating video trajectory with GPS positioning.

[0006] The technical solution for achieving the purpose of the present invention is: a method for replaying an action trajectory by integrating video trajectory with GPS positioning, the method comprising:

[0007] Step 1: Obtain the GPS positioning information of the actor during the action, and perform denoising preprocessing on the positioning information to form a GPS trajectory;

[0008] Step 2: Retrieve available monitoring device locations within a certain range of the GPS action trajectory, obtain static factors of the monitoring device, collect video surveillance footage of the action period, search and identify the action in the video surveillance footage, and simultaneously determine the latitude and longitude, altitude, and time of the actor's location, and concatenate them into a video trajectory in chronological order; the static factors include: the latitude and longitude, altitude, field of view, and time offset of the monitoring device;

[0009] Step 3: Use the GPS trajectory as a low-confidence trajectory and the video trajectory as a high-confidence trajectory to perform trajectory correction and fusion to form an action review trajectory.

[0010] Furthermore, the video surveillance footage collected during the action period in step 2 specifically includes:

[0011] Simplify the GPS trajectory into a plane coordinate set P(x,y):

[0012] P(x,y) = {p0,p1,...,pm}

[0013] Where x is longitude, y is latitude, m is the number of segments, and pm is the mth segment trajectory;

[0014] For each monitoring device, calculate its distance to each trajectory and take the minimum distance. If the minimum distance is less than the preset distance, collect the video surveillance images of the monitoring device during the action period.

[0015] Furthermore, for each monitoring device, calculating the distance to each track segment specifically includes:

[0016] Install monitoring equipment The location is , Monitoring equipment longitude and latitude;

[0017] (1) Take two trajectory points on the mth trajectory , Track points The longitude and latitude of Track points longitude and latitude;

[0018] (2) Calculate trajectory vector , for:

[0019]

[0020]

[0021] (3) Determine whether the projection of the monitoring device in the trajectory direction is on the trajectory segment;

[0022] Calculation:

[0023]

[0024] If f < 0, return the monitoring device The actual distance to the m-th segment of the trajectory is:

[0025]

[0026] If f > 0, calculate the reference distance :

[0027]

[0028] If f > d, the actual distance is:

[0029]

[0030] If f < d, calculate the obtained projection point is:

[0031]

[0032] Then calculate the actual distance is:

[0033]

[0034] In the formula, are respectively the longitude and latitude of the projection point

[0035] Furthermore, the retrieval and recognition of actions in the video monitoring screen described in step 2 specifically include:

[0036] Perform pedestrian detection on the video monitoring screen. If a pedestrian is detected, extract the feature set of the pedestrian frame by frame, including several features;

[0037] Compare the feature set of the pedestrian with the action library. If none of the features in the feature set match a certain action in the action library, it is determined that the pedestrian has no action and is discarded; if one or several features match a certain action in the action library, retain the corresponding video trajectory points and record the positions corresponding to the video trajectory points; if several features match different actions, according to the preset feature priority order, use the matching result corresponding to the feature with the highest priority as the final recognition result.

[0038] Furthermore, the feature set includes one or more of face features, face attribute features, and body posture features.​

[0039] Furthermore, in step 2, the latitude and longitude, altitude, and time of the action location are determined by field of view segmentation and positioning. Specifically, the field of view of the monitoring device is segmented to determine the specific location of the foot in the image. Then, the position of the actor in the video is calculated based on the height, field of view angle, and GPS location of the monitoring device. Specifically, it includes:

[0040] Install monitoring equipment The location is , the height is H, the pitch angle of the viewport center is A, and the direction angle of the monitoring device is B, where Monitoring equipment latitude and longitude; monitoring equipment The pixels are h×w;

[0041] (1) Calculate the distance between the actor and the monitoring device distance for:

[0042]

[0043] Where, Indicates monitoring equipment Monitoring angle, is the vertical coordinate of the actor in the video frame;

[0044] (2) Calculating the actor's relative position to the monitoring device Deviation angle of the visual field midline for:

[0045]

[0046] (3) With the monitoring device as the polar coordinate center and the north direction as the positive direction of the polar coordinate, the polar coordinate of the actor relative to the monitoring device is expressed as ( , );

[0047] (4) Obtain the position of the actor in the video by converting the polar coordinates into rectangular coordinates .

[0048] Furthermore, step 3 specifically includes:

[0049] Combine GPS tracks and video tracks into the same track set in chronological order;

[0050] Traverse all points in the trajectory set to determine whether there is signal loss, position drift, or position conflict. If so, perform corresponding trajectory corrections to obtain the action replay trajectory.

[0051] Furthermore, in step 3, it is determined whether there is signal loss in the point trace and the trajectory is corrected, which specifically includes:

[0052] Determine whether the time interval between two consecutive high-precision trajectory points exceeds a first preset threshold and does not exceed a second preset threshold. If so, it indicates that there is a short-term signal loss; the high-precision trajectory point refers to a point where the plane accuracy and height accuracy values of the GPS point are within a preset range;

[0053] Estimate the path based on the direction and speed of the continuous high-precision trajectory points before the loss; specifically:

[0054] Take two consecutive high-precision trajectory points before the loss and , the interval between two points , estimate the nth trajectory point for:

[0055] .

[0056] Furthermore, in step 3, it is determined whether there is position drift in the point trace and the trajectory is corrected, which specifically includes:

[0057] Determine whether the distance between the two previous and next track points exceeds a third preset threshold, and whether the track time interval does not exceed a fourth preset threshold. If both are yes, it indicates that position drift exists;

[0058] Remove the points with poor continuity between the two trajectory points, specifically:

[0059] Assume that the two trajectory points are 、 ;

[0060] (1) Calculate the discontinuity of the two trajectory points:

[0061]

[0062] Where, for The discontinuity value corresponding to the point, for The adjacent point before the point;

[0063] (2) For the two preceding and following trajectory points, the trajectory point with a relatively large discontinuity value is removed.

[0064] Furthermore, in step 3, it is determined whether there is a position conflict in the point trace and the trajectory is corrected, which specifically includes:

[0065] Determine whether the distance between the GPS track and the video location appearing at the same time is greater than a fifth preset threshold, if so, indicating that there is a location conflict;

[0066] The GPS track is smoothed using continuous video track points, and the GPS track points between continuous video tracks are deleted, where the time difference between the two video track points does not exceed the sixth threshold. Specifically:

[0067] Assume the starting point of the continuous video trajectory is , the previous GPS track point is , then insert a smooth trajectory point between the two points for:

[0068] .

[0069] Compared with the prior art, the present invention has the following significant advantages:

[0070] (1) The present invention uses video trajectory to correct GPS positioning, uses GPS positioning to fill in the blind spots of video trajectory, and uses low-precision and high-precision trajectory points to achieve information complementarity, thereby achieving accurate and coherent recording of indoor and outdoor actions, and solving the problem of difficult trajectory positioning in complex indoor and outdoor action scenes.

[0071] (2) In response to abnormal situations such as position conflicts, trajectory drift, and short-term signal loss that occur during use, innovative trajectory correction methods such as a smoothing correction algorithm for video trajectory points to GPS trajectories and a drift point trajectory extrapolation estimation algorithm are proposed. This achieves the goal of accurately locating the trajectory of the action personnel, with the trajectory accuracy controlled within 50m. It can also improve the trajectory density and form a stable and continuous trajectory, providing strong support for action review.

[0072] (3) The solution achieves low-cost, high-precision and high-density mobile positioning by combining two positioning methods. The method used can be implemented using a camera and a GPS chip, without the need for special equipment or special positioning devices, and has low dependence on equipment. The method has a wide range of applications and strong universality.

[0073] The present invention is further described in detail below with reference to the accompanying drawings. BRIEF DESCRIPTION OF THE DRAWINGS

[0074] Figure 1 The figure is a flow chart of a trajectory review method in one embodiment.

[0075] Figure 2 FIG. 4 is a flow chart of feature matching in one embodiment.

[0076] Figure 3 Schematic diagram of video segmentation and trajectory calculation in one embodiment.

[0077] Figure 4 This is a flow chart of trajectory correction fusion in one embodiment. DETAILED DESCRIPTION

[0078] In order to make the purpose, technical solutions and advantages of this application more clear, the following further describes this application in detail with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain this application and are not intended to limit this application.

[0079] It should be noted that if the embodiments of the present invention involve directional indications (such as up, down, left, right, front, back, etc.), such directional indications are only used to explain the relative position relationship, movement status, etc. between the various components under a certain specific posture (as shown in the accompanying drawings). If the specific posture changes, the directional indication will also change accordingly.

[0080] In addition, if there are descriptions involving "first", "second", etc. in the embodiments of the present invention, the descriptions of "first", "second", etc. are only for descriptive purposes and cannot be understood as indicating or implying their relative importance or implicitly indicating the number of the indicated technical features. Therefore, the features specified as "first" and "second" may explicitly or implicitly include at least one of such features. In addition, the technical solutions between the various embodiments can be combined with each other, but this must be based on the fact that ordinary technicians in this field can implement them. When the combination of technical solutions is contradictory or cannot be implemented, it should be deemed that such a combination of technical solutions does not exist and is not within the scope of protection required by the present invention.

[0081] In one embodiment, combined Figure 1 , provides an action trajectory review method for integrating video trajectory and GPS positioning application, the method comprising:

[0082] Step 1: Obtain the GPS positioning information of the actor during the action, and perform denoising preprocessing on the positioning information to form a GPS trajectory;

[0083] Step 2: Retrieve available monitoring device locations within a certain range of the GPS action trajectory, obtain static factors of the monitoring device, collect video surveillance footage of the action period, search and identify the action in the video surveillance footage, and simultaneously determine the latitude and longitude, altitude, and time of the actor's location, and concatenate them into a video trajectory in chronological order; the static factors include: the latitude and longitude, altitude, field of view, and time offset of the monitoring device;

[0084] Step 3: Use the GPS trajectory as a low-confidence trajectory and the video trajectory as a high-confidence trajectory to perform trajectory correction fusion to obtain a continuous and accurate action review trajectory.

[0085] Furthermore, in one embodiment, the step 2 of collecting video surveillance images during the action period specifically includes:

[0086] Simplify the GPS trajectory into a plane coordinate set P(x,y):

[0087] P(x,y) = {p0,p1,...,pm}

[0088] Where x is longitude, y is latitude, m is the number of segments, and pm is the mth segment trajectory;

[0089] For each monitoring device, calculate its distance to each trajectory and take the minimum distance. If the minimum distance is less than the preset distance, collect the video surveillance images of the monitoring device during the action period.

[0090] Furthermore, in one embodiment, calculating the distance from each monitoring device to each trajectory segment specifically includes:

[0091] Install monitoring equipment The location is , Monitoring equipment longitude and latitude;

[0092] (1) Take two trajectory points on the mth trajectory , Track points The longitude and latitude of Track points longitude and latitude;

[0093] (2) Calculate trajectory vector , for:

[0094]

[0095]

[0096] (3) Determine whether the projection of the monitoring device in the trajectory direction is on the trajectory segment;

[0097] calculate:

[0098]

[0099] If f<0, return to the monitoring device The actual distance to the mth track for:

[0100]

[0101] If f > 0, calculate the reference distance :

[0102]

[0103] If f > d, the actual distance is:

[0104]

[0105] If f < d, calculate the projected point is:

[0106]

[0107] Then calculate the actual distance is:

[0108]

[0109] In the formula, are respectively the longitude and latitude of the projected point .

[0110] Furthermore, in one of the embodiments, in combination with Figure 2 , the retrieval and recognition of actions in the video surveillance screen in step 2 specifically includes:

[0111] Perform pedestrian detection on the video surveillance screen. If a pedestrian is detected, extract the feature set of the pedestrian frame by frame, including several features;

[0112] Compare the feature set of the pedestrian with the action library. If all features in the feature set do not match a certain action in the action library, it is determined that the pedestrian has no action and is discarded; if one or several features match a certain action in the action library, retain the corresponding video trajectory points and record the positions corresponding to the video trajectory points; if several features match different actions, according to the preset feature priority order, take the matching result corresponding to the feature with the highest priority as the final recognition result.

[0113] Preferably here, the feature set includes one or more of face features, face attribute features, and body posture features.

[0114] Preferably here, the priority order is: face features, face attribute features, body posture features.

[0115] Furthermore, in one embodiment, in step 2, the latitude and longitude, altitude, and time of the action location are determined by means of field of view segmentation and positioning, specifically: the field of view of the monitoring device is segmented to determine the specific location of the foot in the image, and then the location of the actor in the video is calculated based on the height, field of view angle, and GPS location of the monitoring device; combined with Figure 3 , specifically including:

[0116] Install monitoring equipment The location is , the height is H, the pitch angle of the viewport center is A, and the direction angle of the monitoring device is B, where Monitoring equipment latitude and longitude; monitoring equipment The pixels are h×w;

[0117] (1) Calculate the distance between the actor and the monitoring device distance for:

[0118]

[0119] Where, Indicates monitoring equipment Monitoring angle, is the vertical coordinate of the actor in the video frame;

[0120] Here, if If the value of is greater than 50, The value is 50.

[0121] (2) Calculating the actor's relative position to the monitoring device Deviation angle of the visual field midline for:

[0122]

[0123] (3) With the monitoring device as the polar coordinate center and the north direction as the positive direction of the polar coordinate, the polar coordinate of the actor relative to the monitoring device is expressed as ( , );

[0124] (4) Obtain the position of the actor in the video by converting the polar coordinates into rectangular coordinates .

[0125] Furthermore, in one embodiment, in combination Figure 4 , step 3 specifically includes:

[0126] Combine GPS tracks and video tracks into the same track set in chronological order;

[0127] Traverse all points in the trajectory set to determine whether there is signal loss, position drift, or position conflict. If so, perform corresponding trajectory corrections to obtain the action replay trajectory.

[0128] Here, signal loss, position drift, and position conflict are judged in sequence.

[0129] Furthermore, in one embodiment, in step 3, determining whether there is signal loss in the point trace and performing trajectory correction specifically includes:

[0130] Determine whether the time interval between two consecutive high-precision trajectory points exceeds a first preset threshold and does not exceed a second preset threshold. If so, it indicates that there is a short-term signal loss; the high-precision trajectory point refers to a point where the plane accuracy and height accuracy values of the GPS point are both within a preset range; preferably, the preset range is 30 meters;

[0131] Estimate the path based on the direction and speed of the continuous high-precision trajectory points before the loss; specifically:

[0132] Take two consecutive high-precision trajectory points before the loss and , the interval between two points , estimate the nth trajectory point for:

[0133] .

[0134] Here preferably, the first preset threshold is 1 minute and the second preset threshold is 15 minutes.

[0135] Furthermore, in one embodiment, in step 3, determining whether the point trace has position drift and performing trajectory correction specifically includes:

[0136] Determine whether the distance between the two previous and next track points exceeds a third preset threshold, and whether the track time interval does not exceed a fourth preset threshold. If both are yes, it indicates that position drift exists;

[0137] Remove the points with poor continuity between the two trajectory points, specifically:

[0138] Assume that the two trajectory points are 、 ;

[0139] (1) Calculate the discontinuity of the two trajectory points:

[0140]

[0141] Where, for The discontinuity value corresponding to the point, for The adjacent point before the point;

[0142] (2) For the two preceding and following trajectory points, the trajectory point with a relatively large discontinuity value is removed.

[0143] Here preferably, the third preset threshold is 1 kilometer and the second preset threshold is 1 minute.

[0144] Furthermore, in one embodiment, in step 3, determining whether there is a position conflict in the point trace and performing trajectory correction specifically includes:

[0145] Determine whether the distance between the GPS track and the video location appearing at the same time is greater than a fifth preset threshold, if so, indicating that there is a location conflict;

[0146] The GPS track is smoothed using continuous video track points, and the GPS track points between continuous video tracks are deleted, where the time difference between the two video track points does not exceed the sixth threshold. Specifically:

[0147] Assume the starting point of the continuous video trajectory is , the previous GPS track point is , then insert a smooth trajectory point between the two points for:

[0148] .

[0149] Here, preferably, the fifth preset threshold is 1 kilometer.

[0150] In one embodiment, a motion trajectory review system integrating video trajectory and GPS positioning is provided, the system comprising:

[0151] The first module is used to obtain the GPS positioning information of the actor during the action, and perform denoising preprocessing on the positioning information to form a GPS trajectory;

[0152] The second module is used to retrieve the available monitoring device locations within a certain range of the GPS action trajectory, obtain the static factors of the monitoring device, collect video surveillance footage of the action period, search and identify the action in the video surveillance footage, and simultaneously determine the latitude and longitude, altitude and time of the actor's location, and concatenate them into a video trajectory in time sequence. The static factors include: the latitude and longitude, altitude, field of view of the monitoring device, and time offset;

[0153] The third module is used to use the GPS trajectory as a low-credibility trajectory and the video trajectory as a high-credibility trajectory to perform trajectory correction and fusion to form an action review trajectory.

[0154] Regarding the specific limitations of the motion trajectory replay system for the fusion application of video trajectory and GPS positioning, please refer to the limitations of the motion trajectory replay method for the fusion application of video trajectory and GPS positioning above, which will not be repeated here. Each module in the above-mentioned motion trajectory replay system for the fusion application of video trajectory and GPS positioning can be implemented in whole or in part by software, hardware and a combination thereof. The above-mentioned modules can be embedded in or independent of the processor in the computer device in the form of hardware, or can be stored in the memory of the computer device in the form of software, so that the processor can call and execute the operations corresponding to the above modules.

[0155] In one embodiment, a computer device is provided, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the computer program, the following is achieved:

[0156] Step 1: Obtain the GPS positioning information of the actor during the action, and perform denoising preprocessing on the positioning information to form a GPS trajectory;

[0157] Step 2: Retrieve available monitoring device locations within a certain range of the GPS action trajectory, obtain static factors of the monitoring device, collect video surveillance footage of the action period, search and identify the action in the video surveillance footage, and simultaneously determine the latitude and longitude, altitude, and time of the actor's location, and concatenate them into a video trajectory in chronological order; the static factors include: the latitude and longitude, altitude, field of view, and time offset of the monitoring device;

[0158] Step 3: Use the GPS trajectory as a low-confidence trajectory and the video trajectory as a high-confidence trajectory to perform trajectory correction and fusion to form an action review trajectory.

[0159] For the specific limitations of each step, please refer to the limitations of the action trajectory review method for the fusion application of video trajectory and GPS positioning above, which will not be repeated here.

[0160] In one embodiment, a computer-readable storage medium is provided, on which a computer program is stored, and when the computer program is executed by a processor, the computer program implements:

[0161] Step 1: Obtain the GPS positioning information of the actor during the action, and perform denoising preprocessing on the positioning information to form a GPS trajectory;

[0162] Step 2: Retrieve available monitoring device locations within a certain range of the GPS action trajectory, obtain static factors of the monitoring device, collect video surveillance footage of the action period, search and identify the action in the video surveillance footage, and simultaneously determine the latitude and longitude, altitude, and time of the actor's location, and concatenate them into a video trajectory in chronological order; the static factors include: the latitude and longitude, altitude, field of view, and time offset of the monitoring device;

[0163] Step 3: Use the GPS trajectory as a low-confidence trajectory and the video trajectory as a high-confidence trajectory to perform trajectory correction and fusion to form an action review trajectory.

[0164] For the specific limitations of each step, please refer to the limitations of the action trajectory review method for the fusion application of video trajectory and GPS positioning above, which will not be repeated here.

[0165] As a specific example, the present invention is described in detail in one of the embodiments.

[0166] 1. Use portable positioning equipment to collect GPS positioning information during the action, perform denoising preprocessing on the positioning information, and form a GPS trajectory;

[0167] 1) Use smart phones, portable positioning devices, beacons and other positioning devices to collect GPS positioning information, encapsulate the GPS information into a custom format message of [id, collection time, longitude, latitude, altitude, speed, azimuth, plane accuracy, altitude accuracy, speed accuracy, azimuth accuracy], and send it back through the message queue RabbitMQ;

[0168] 2) After receiving GPS information, the system reads the horizontal accuracy and altitude accuracy values of the GPS points. Points with horizontal and altitude accuracy within 30 meters are marked as high-precision points. Points with horizontal and altitude accuracy between 30-100 meters are recorded as low-precision points. Points with horizontal accuracy or altitude accuracy greater than 100 meters are discarded, thus forming the original GPS track (hereinafter referred to as the GPS track).

[0169] 2. Retrieve the available monitoring devices within 500 meters of the GPS movement trajectory, locate them from the device screen, and use the screen segmentation positioning algorithm to generate the video trajectory;

[0170] 1) Simplify the GPS trajectory into a set of trajectory plane coordinate points , the camera position is recorded as , is the longitude, is the latitude, is the number of line segments, and the number of cameras within 500 meters of the GPS track is calculated. Assume that the camera position is , take the trajectory part Two points;

[0171] Calculate the trajectory vector , which is

[0172]

[0173]

[0174] Judge whether the projection of the camera in the trajectory direction is on the trajectory segment:

[0175]

[0176] When f < 0, return the distance which is:

[0177]

[0178] When f > 0, calculate the reference distance :

[0179]

[0180] If f > d, the distance is:

[0181]

[0182] If f < d, the projection point can be obtained which is:

[0183]

[0184] Then calculate the actual distance which is:

[0185]

[0186] Traverse all trajectory segments according to the above method, and obtain the minimum distance as the minimum distance between the camera and the trajectory.

[0187] 2) Use openCV to extract frames from the video, use labelme to label the pedestrian dataset, and use YOLOv3 to train and detect pedestrians in the video. Among them, Darknet-53 is selected as the backbone network to extract pedestrian features.

[0188] 3) Use the open-source InsightFace project for face recognition. The dataset uses the Chinese Academy of Sciences Asian face dataset CAS-PEAL, and ResNet is used as the backbone network to extract the face features of pedestrians.

[0189] 4) FaceAttribute-FAN, an open-source project from Tencent Youtu, is used for facial attribute recognition. The network structure uses Facial Abstraction Net, and the backbone network uses ResNet. The focus is on identifying facial features such as beard, weight, gender, makeup, glasses size, hair, hat, and glasses.

[0190] 5) We used the Fast-ReID project for pedestrian body feature recognition, using the Market-1501 dataset from Tsinghua University for training and testing. We used Random Patch preprocessing software for data augmentation, selected ResNet as the backbone network, used max pooling for global body feature aggregation, set the training strategy to LearningRate, and used Kuangshi Circle Loss as the loss function for model training.

[0191] 6) Using one or more of facial features, facial attribute features, and body features, gradually search and identify in the monitoring screen. The specific process is as follows: Figure 2 shown.

[0192] 7) After positioning, obtain the latitude and longitude of the camera device , camera height H, viewport center pitch angle A (A < 90°), camera direction angle B four parameters, the field of view segmentation positioning algorithm is used to calculate the latitude and longitude of the location as follows Figure 2 As shown, the actor's position in the video The estimation method is as follows:

[0193] The horizontal distance from the center of the camera image to the camera for:

[0194]

[0195] Assuming the camera monitoring angle is 2C (C < 45° and C < A), the nearest shot , Maximum shooting distance for:

[0196]

[0197]

[0198] like >50, Take 50.

[0199] Take a 1K standard definition camera (1920*1080 resolution), assuming the nearest edge of the video frame is =0, the farthest edge of the video frame =1080, the position of the person in the video frame is , then take the distance between the actor and the camera for,

[0200]

[0201] The deviation angle of the actor relative to the center line of the camera's field of view for:

[0202]

[0203] Therefore, the camera is taken as the polar coordinate center, the north direction is the positive direction of the polar coordinate, and the polar coordinate relative to the camera can be expressed as ( , ), the position of the actor in the video can be obtained by converting the polar coordinates into rectangular coordinates.

[0204] 3. Combine GPS tracks and video tracks into the same track set in chronological order , traverse all trajectory points at once, determine whether there are signal loss, position drift, or position conflict problems, and use corresponding processing methods to correct the trajectory to obtain a continuous and accurate action review trajectory .

[0205] 1) Short-term signal loss correction

[0206] If the time interval between two consecutive trajectory points is more than 1 minute but less than 15 minutes, it is defined as short-term signal loss and trajectory extrapolation is used for correction. points There is a track loss, and the interval time with the previous point , take the trajectory point before loss and , the interval between two points , calculate the nth trajectory point for:

[0207]

[0208] 2) Position drift correction

[0209] If the distance between the two trajectories exceeds 1 km and the time interval between the trajectories does not exceed 30 seconds, it is defined as position drift and the points with poor continuity between the two points need to be removed for correction.

[0210] Assume that three consecutive points are 、 、 ,point Discontinuity for:

[0211]

[0212] Calculate the discontinuity between two points before and after 、 , retaining good continuity ( small) points.

[0213] 3) Position conflict correction

[0214] If the distance between the GPS track and the video track points that appear alternately in the same time period is greater than 1 km, it is defined as a position conflict. The GPS track is smoothed using the continuous video track points (the time difference between the two video track points does not exceed 1 minute), and the GPS track points between the continuous video tracks are deleted. Assume that the starting point of the continuous video track is , the previous GPS track point , then insert a smooth trajectory point between the two points for:

[0215] .

[0216] The above shows and describes the basic principles, main features and advantages of the present invention. Those skilled in the art should understand that the present invention is not limited to the above embodiments. The above embodiments and descriptions are only illustrative of the principles of the present invention. Without departing from the spirit and scope of the present invention, any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present invention shall be included in the scope of protection of the present invention.

Claims

1. A method for replaying an action trajectory by integrating video trajectory and GPS positioning, characterized in that: The method comprises: Step 1: Obtain the GPS positioning information of the actor during the action, and perform denoising preprocessing on the positioning information to form a GPS trajectory; Step 2: Retrieve available monitoring device locations within a certain range of the GPS action trajectory, obtain static factors of the monitoring device, collect video surveillance footage of the action period, search and identify the action in the video surveillance footage, and simultaneously determine the latitude and longitude, altitude, and time of the actor's location, and concatenate them into a video trajectory in chronological order; the static factors include: the latitude and longitude, altitude, field of view, and time offset of the monitoring device; Step 3: Use the GPS trajectory as a low-confidence trajectory and the video trajectory as a high-confidence trajectory to perform trajectory correction and fusion to form an action review trajectory; The video surveillance footage collected during the action period in step 2 specifically includes: Simplify the GPS trajectory into a plane coordinate set P(x,y): P(x,y) = {p0,p1,...,pm}; Where x is longitude, y is latitude, m is the number of segments, and pm is the mth segment trajectory; For each monitoring device, calculate its distance to each trajectory and take the minimum distance. If the minimum distance is less than the preset distance, collect the video surveillance footage of the monitoring device during the action period; In step 2, the latitude and longitude, altitude, and time of the action are determined through field of view segmentation and positioning. Specifically, the field of view of the monitoring device is segmented to determine the specific location of the foot in the image. Then, the position of the actor in the video is calculated based on the height, field of view angle, and GPS location of the monitoring device.

2. The method for replaying an action trajectory by integrating video trajectory and GPS positioning according to claim 1 is characterized in that: The calculation of the distance from each monitoring device to each track segment specifically includes: Install monitoring equipment The location is , Monitoring equipment longitude and latitude; (1) Take two trajectory points on the mth trajectory , Track points The longitude and latitude of Track points longitude and latitude; (2) Calculate trajectory vector , for: ； ； (3) Determine whether the projection of the monitoring device in the trajectory direction is on the trajectory segment; calculate: ； If f<0, return to the monitoring device The actual distance to the mth track for: ； If f>0, calculate the reference distance : ； If f>d, the actual distance for: ； If f < d, the calculated projection point is as follows: ； Then calculate the actual distance for: ； Where, The projection points The longitude and latitude of .

3. The method for replaying an action trajectory by integrating video trajectory and GPS positioning according to claim 1 is characterized in that: Step 2 of searching and identifying actions in the video surveillance footage specifically includes: Perform pedestrian detection on the video surveillance screen. If a pedestrian is detected, extract the pedestrian's feature set frame by frame, including several features; The pedestrian's feature set is compared with the action library. If all features in the feature set do not match an action in the action library, it is determined that the pedestrian has no action and is discarded; if one or several features match an action in the action library, the corresponding video trajectory point is retained and the corresponding position of the video trajectory point is recorded; if the actions matched by certain features are different, the matching result corresponding to the feature with the highest priority is used as the final recognition result according to the preset feature priority order.

4. The method for replaying an action trajectory by integrating video trajectory and GPS positioning according to claim 3 is characterized in that: The feature set includes one or more of facial features, facial attribute features, and body features.

5. The method for replaying an action trajectory by integrating video trajectory and GPS positioning according to claim 1 is characterized in that: Step 3 specifically includes: Combine GPS tracks and video tracks into the same track set in chronological order; Traverse all points in the trajectory set to determine whether there is signal loss, position drift, or position conflict. If so, perform corresponding trajectory corrections to obtain the action replay trajectory.

6. The method for replaying an action trajectory by integrating video trajectory and GPS positioning according to claim 5 is characterized in that: In step 3, determine whether there is signal loss in the trace and perform trajectory correction, which includes: Determine whether the time interval between two consecutive high-precision trajectory points exceeds a first preset threshold and does not exceed a second preset threshold. If so, it indicates that there is a short-term signal loss; the high-precision trajectory point refers to a point where the plane accuracy and height accuracy values of the GPS point are within a preset range; Estimate the path based on the direction and speed of the continuous high-precision trajectory points before the loss; specifically: Take two consecutive high-precision trajectory points before the loss and , the interval between two points , estimate the nth trajectory point for: 。 7. The method for replaying an action trajectory by integrating video trajectory and GPS positioning according to claim 5 is characterized in that: In step 3, determine whether the point trace has position drift and perform trajectory correction, which specifically includes: Determine whether the distance between the two previous and next track points exceeds a third preset threshold, and whether the track time interval does not exceed a fourth preset threshold. If both are yes, it indicates that position drift exists; Remove the points with poor continuity between the two trajectory points, specifically: Assume that the two trajectory points are 、 ; (1) Calculate the discontinuity of the two trajectory points: ； Where, for The discontinuity value corresponding to the point, for The adjacent point before the point; (2) For the two preceding and following trajectory points, the trajectory point with a relatively large discontinuity value is removed.

8. The method for replaying an action trajectory by integrating video trajectory and GPS positioning according to claim 5 is characterized in that: In step 3, determine whether there is a position conflict in the point trace and perform trajectory correction, which specifically includes: Determine whether the distance between the GPS track and the video location appearing at the same time is greater than a fifth preset threshold, if so, indicating that there is a location conflict; The GPS track is smoothed using continuous video track points, and the GPS track points between continuous video tracks are deleted, where the time difference between the two video track points does not exceed the sixth threshold. Specifically: Assume the starting point of the continuous video trajectory is , the previous GPS track point is , then insert a smooth trajectory point between the two points for: 。 9. An action trajectory review system based on the video trajectory and GPS positioning fusion application of the method according to any one of claims 1 to 8, characterized in that: The system comprises: The first module is used to obtain the GPS positioning information of the actor during the action, and perform denoising preprocessing on the positioning information to form a GPS trajectory; The second module is used to retrieve the available monitoring device locations within a certain range of the GPS action trajectory, obtain the static factors of the monitoring device, collect video surveillance footage of the action period, search and identify the action in the video surveillance footage, and simultaneously determine the latitude and longitude, altitude and time of the actor's location, and concatenate them into a video trajectory in time sequence. The static factors include: the latitude and longitude, altitude, field of view of the monitoring device, and time offset; The third module is used to use the GPS trajectory as a low-credibility trajectory and the video trajectory as a high-credibility trajectory to perform trajectory correction and fusion to form an action review trajectory.

10. A computer device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein: When the processor executes the computer program, the method according to any one of claims 1 to 8 is implemented.

Images