A motion trajectory comparison method, system and terminal device
By receiving test tasks, acquiring and fusing video images and labeling images of multiple monitoring devices, generating and comparing motion trajectory routes, the problems of incomplete fusion of motion trajectory and difficulty in automatic comparison in the prior art are solved, and the trajectory routes are realized Improved accuracy verification and comparison efficiency.
Patent Information
- Application Number
- CN202411169959.8
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2024-08-23
- Publication Date
- 2024-11-29
- Estimated Expiration
- 2044-08-23
AI Technical Summary
When the existing technology applies deep learning algorithms in industrial products, there is manual dependence on motion trajectory fusion and lack of system management in the testing process, so it is impossible to effectively fuse multi-channel video image data, resulting in incomplete trajectory routes of the motion target and the inability to automatically compare depth. Learning algorithms and labeling image data affects the accuracy and comparison efficiency of trajectory fusion.
By receiving test tasks, we obtain video images and labeled images collected by multiple monitoring devices, perform image stitching and fusion, label time stamps and target features, detect target features, generate trajectory routes, and perform trajectory comparisons, and output comparison results Verify the accuracy of trajectory fusion and improve the comparison efficiency.
Complete acquisition and automatic comparison of the trajectory route of the moving target are achieved, the accuracy of trajectory fusion is verified, the comparison efficiency is improved, manual participation is reduced, and the applicability is wider.
Smart Images

Figure CN118692013B_ABST
Abstract
Description
Technical Field
[0001] The present application relates to the field of deep learning technology, and in particular to a motion trajectory comparison method and system, and a terminal device. Background Art
[0002] At present, in the process of applying deep learning algorithms to industrial products, there are problems such as a certain degree of manual dependence and lack of system management in the testing process. Especially in the aspect of motion trajectory fusion, the multi-channel video image data is not fused, and a relatively complete trajectory route of the moving target cannot be obtained. The trajectory route of the moving target obtained by the deep learning algorithm cannot be automatically compared with the corresponding trajectory route obtained by the annotated image data, so the accuracy of motion trajectory fusion cannot be verified. At the same time, motion trajectory comparison requires more manual participation, which affects the comparison efficiency. Summary of the invention
[0003] The embodiments of the present application provide a motion trajectory comparison method and system, and a terminal device, which are used to fuse multiple video image data to obtain a complete trajectory route of a moving target, and automatically compare it with the marked trajectory route, thereby verifying the accuracy of trajectory fusion and improving comparison efficiency.
[0004] The present application provides a motion trajectory comparison method, including:
[0005] When a test task is received, video images and annotated images collected by multiple monitoring devices in a specified monitoring area are obtained based on the test task; the annotated images are obtained by splicing image frames of the video images at the same time, marking the timestamp of the corresponding time on each spliced image, and marking the target features of each moving target including at least target ID, target type, coordinate frame, target coordinates, and moving speed;
[0006] According to a set period, image frames at the same time are obtained from each of the video images for image fusion, and timestamps of corresponding times are marked on each fused image; the annotated images with the same timestamp as the fused images are determined to form an annotated image set; target detection is performed on each of the fused images to obtain target features of multiple detected targets; the target IDs of the same detected target in different fused images are the same;
[0007] For each of the detected targets and each of the annotated targets included in the annotated image set, executing: determining the trajectory points of the target in the monitoring area according to the target coordinates of the target, marking a time tag on each of the trajectory points, and generating a trajectory route of the target in the monitoring area according to each of the trajectory points and the time tag corresponding to each of the trajectory points;
[0008] The trajectory of each detected target is compared with the trajectory of each marked target, and the comparison result is output.
[0009] By this method, when a test task is received, video images and annotated images collected by multiple monitoring devices in a specified monitoring area are obtained based on the test task; the annotated images are obtained by splicing image frames at the same time of each of the video images, marking the timestamp of the corresponding time on each spliced image, and marking the target features of each moving target including at least the target ID, target type, coordinate frame, target coordinates, and movement speed; according to a set period, image frames at the same time are obtained from each of the video images for image fusion, and the timestamp of the corresponding time is marked on each fused image; the annotated images with the same timestamp as the fused images are determined to form an annotated image set; and the fused images are fused together. The target features of multiple detected targets are obtained by combining the images; the target ID of the same detected target in different fused images is the same; for each detected target and each annotated target included in the annotated image set, the following are performed: the trajectory point of the target in the monitoring area is determined according to the target coordinates of the target, and a time tag is marked on each trajectory point, and the trajectory route of the target in the monitoring area is generated according to each trajectory point and the time tag corresponding to each trajectory point; the trajectory route of each detected target is compared with the trajectory route of each annotated target, and the comparison result is output, so as to verify the accuracy of trajectory fusion, improve the comparison efficiency, and have wider applicability.
[0010] In some embodiments, the comparing the trajectory of each detected target with the trajectory of each marked target and outputting the comparison result includes:
[0011] For each of the detection targets, based on the target type and coordinate frame of the detection target, when the target type of any of the labeled targets is different from the target type of the detection target, and / or the intersection and union ratio of the coordinate frame and the coordinate frame of the detection target is less than a first preset value, it is determined that there is no labeled target that matches the detection target, and a comparison result that the detection target is a virtual captured target is output.
[0012] Through this method, it is possible to determine the labeled target that matches the detected target, so that the accuracy of target detection and labeled image data can be determined based on the target matching result.
[0013] In some embodiments, when it is determined that there is a marked target that matches the detected target, a first corresponding time after the matched marked target moves a first set distance from the first trajectory point is determined according to the movement speed of the matched marked target and the time tag of the first trajectory point of the trajectory route;
[0014] Respectively determining the trajectory points that the detection target and the matched labeled target pass through when moving from the first trajectory point to the first corresponding moment;
[0015] Based on the trajectory points passed by the detection target and the trajectory points passed by the matched marked target, the distance between each pair of trajectory points with the same time label is calculated. When the number of trajectory point pairs whose distance is less than the second preset value is less than the third preset value, it is determined that the trajectory route of the detection target does not match the trajectory route of the matched marked target, and the comparison result that the trajectory route of the detection target is an unmatched trajectory route is output.
[0016] Through this method, it is possible to determine whether the trajectory route of each detected target is a matching trajectory route or a non-matching trajectory route, thereby verifying the accuracy of trajectory fusion.
[0017] In some embodiments, when it is determined that the trajectory route of the detected target matches the trajectory route of the matched marked target, a second corresponding moment when the matched marked target moves to a second set distance from the last trajectory point is determined according to the movement speed of the matched marked target;
[0018] The second corresponding moment minus the preset time is used as the start time, and the second corresponding moment plus the preset time is used as the end time, and the trajectory points of the detected target and the matched marked target moving from the start time to the end time are determined respectively;
[0019] Determine the distance between each pair of two trajectory points with the same time tag according to the trajectory points passed by the detection target and the trajectory points passed by the matching labeled target;
[0020] When the minimum value of the distance is not less than a fourth preset value, a comparison result indicating that the trajectory route of the detection target is an incomplete trajectory route is output.
[0021] Through this method, the integrity of the trajectory route can be further determined on the basis of detecting that the trajectory route of the target is a matching trajectory route, thereby verifying the accuracy of trajectory fusion.
[0022] In some embodiments, when the minimum value of the distance is less than a fourth preset value, the distance between each pair of two trajectory points with the same time tags is determined according to the trajectory route of the detected target and the trajectory route of the matched labeled target;
[0023] For each pair of two trajectory points with the same time tag, if the distance is less than the fifth preset value, the comparison result of the corresponding trajectory point of the detection target is output as the correct trajectory point; otherwise, the comparison result of the corresponding trajectory point of the detection target is output as the wrong trajectory point.
[0024] Through this method, when the trajectory route of the detected target is a matching trajectory route and is complete, it is possible to further determine whether each trajectory point of the trajectory route is correct, thereby verifying the accuracy of trajectory fusion.
[0025] In some embodiments, after comparing the trajectory of each detected target with the trajectory of each marked target and outputting the comparison result, the method further includes:
[0026] Determine the number of detection trajectory routes according to the target ID of each detection target; determine the number of annotated trajectory routes according to the target ID of each annotated target; and obtain a capture rate by dividing the number of annotated trajectory routes by the number of detection trajectory routes; and
[0027] The number of detected trajectory routes minus the number of incomplete trajectory routes is used to obtain the number of complete trajectory routes; the number of detected trajectory routes minus the number of unmatched trajectory routes is used to obtain the number of matched trajectory routes; the number of complete trajectory routes is divided by the number of matched trajectory routes to obtain the completeness rate; and
[0028] Counting the total number of the correct trajectory points and dividing the result by the total number of trajectory points of each detection target to obtain the accuracy rate;
[0029] Generate a test report including at least the capture rate, completeness rate, and accuracy rate.
[0030] Through this method, after completing the comparison between the trajectory routes of each detected target and the trajectory routes of each labeled target, the capture rate, completeness rate and accuracy rate can be determined using the comparison results. This facilitates the subsequent analysis of the accuracy of trajectory fusion and labeled image data based on these three indicators, and quickly locates and analyzes problems and optimizes the corresponding parameters.
[0031] In some embodiments, the method further comprises:
[0032] Generate a fused video according to each of the fused images and a timestamp corresponding to each of the fused images;
[0033] According to the detection target selected by the user, when playing the fused video, the trajectory route of the detection target and the trajectory route of the matching marked target with the same time label as the corresponding timestamp are determined respectively according to the timestamp corresponding to the currently played fused image, and the trajectory points are highlighted, so that the detection target and the matching marked target move to the corresponding trajectory points within the monitoring area as the fused video is played.
[0034] Through this method, the detected target and the marked target can move in real time in the monitoring area as the fused video is played, which can more intuitively judge the correctness of the matching between the detected trajectory route and the marked trajectory route.
[0035] The present application provides a motion trajectory comparison system, including:
[0036] An acquisition module is used for acquiring video images and annotated images collected by multiple monitoring devices in a specified monitoring area based on the test task when a test task is received; the annotated images are obtained by splicing image frames of the video images at the same time, marking the timestamp of the corresponding time on each spliced image, and marking target features of each moving target including at least target ID, target type, coordinate frame, target coordinates, and moving speed;
[0037] The detection module is used to obtain image frames at the same time from each of the video images according to a set period, perform image fusion, and mark the timestamp of the corresponding time on each fused image; determine the annotated images with the same timestamp as the fused images to form an annotated image set; perform target detection on each of the fused images to obtain target features of multiple detected targets; the target ID of the same detected target in different fused images is the same;
[0038] A generating module, for each of the detected targets and each of the annotated targets included in the annotated image set, executing: determining the trajectory points of the target in the monitoring area according to the target coordinates of the target, marking a time tag on each of the trajectory points, and generating a trajectory route of the target in the monitoring area according to each of the trajectory points and the time tag corresponding to each of the trajectory points;
[0039] The comparison module is used to compare the trajectory of each detected target with the trajectory of each marked target, and output the comparison result.
[0040] Through this system, multi-channel video image data can be fused to obtain the complete trajectory of the moving target, and automatically compared with the marked trajectory, so as to verify the accuracy of trajectory fusion and improve comparison efficiency.
[0041] Another embodiment of the present application provides a terminal device, which includes a memory and a processor, wherein the memory is used to store program instructions, and the processor is used to call the program instructions stored in the memory, and execute any of the above methods according to the obtained program.
[0042] Furthermore, according to an embodiment, for example, a computer program product for a computer is provided, which includes software code portions, and when the product is run on the computer, these software code portions are used to perform the steps of the method defined above. The computer program product may include a computer-readable medium having the software code portions stored thereon. Furthermore, the computer program product may be directly loaded into the internal memory of the computer and / or sent via a network through at least one of an upload process, a download process, and a push process.
[0043] Another embodiment of the present application provides a computer-readable storage medium, wherein the computer-readable storage medium stores computer-executable instructions, and the computer-executable instructions are used to enable the computer to execute any of the above methods. BRIEF DESCRIPTION OF THE DRAWINGS
[0044] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings required for use in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.
[0045] Figure 1 A schematic diagram of the overall process of a motion trajectory comparison method provided in an embodiment of the present application;
[0046] Figure 2 A schematic diagram of a motion trajectory route provided in an embodiment of the present application;
[0047] Figure 3 A schematic diagram of a motion trajectory comparison result provided in an embodiment of the present application;
[0048] Figure 4 A detailed schematic diagram of a motion trajectory comparison result provided in an embodiment of the present application;
[0049] Figure 5 A schematic diagram of the structure of a motion trajectory comparison system provided in an embodiment of the present application;
[0050] Figure 6 A schematic diagram of a specific flow chart of a motion trajectory comparison method provided in an embodiment of the present application;
[0051] Figure 7 A schematic diagram of a specific process for determining a labeled target that matches a detection target provided in an embodiment of the present application;
[0052] Figure 8 A schematic diagram of a specific process for determining whether the trajectory route of a detection target matches provided in an embodiment of the present application;
[0053] Fig. 9 A schematic diagram of a specific process for determining whether the trajectory route of a detection target is complete provided in an embodiment of the present application;
[0054] Fig.10 A schematic diagram of a process for generating a test report provided in an embodiment of the present application;
[0055] Fig.11 A schematic diagram of the structure of a terminal device provided in an embodiment of the present application. DETAILED DESCRIPTION
[0056] The following is a clear and complete description of the technical solutions in the embodiments of the present application in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.
[0057] The embodiments of the present application provide a motion trajectory comparison method and system, and a terminal device, which are used to fuse multiple video image data to obtain a complete trajectory route of a moving target, and automatically compare it with the marked trajectory route, thereby verifying the accuracy of trajectory fusion and improving comparison efficiency.
[0058] Among them, the method and the device are based on the same application concept. Since the method and the device solve the problem in a similar principle, the implementation of the device and the method can refer to each other, and the repeated parts will not be repeated.
[0059] The terms "first", "second", etc. (if any) in the specification and claims of the embodiments of the present application and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchangeable where appropriate, so that the embodiments described herein can be implemented in a sequence other than the content illustrated or described herein. In addition, the terms "including" and "having" and any of their variations are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device comprising a series of steps or units is not necessarily limited to those steps or units clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.
[0060] The following examples and embodiments are to be understood as illustrative examples only. Although this specification may refer to "one", "an" or "some" examples or embodiments in several places, this does not mean that each such reference relates to the same example or embodiment, nor does it mean that the feature applies only to a single example or embodiment. Individual features of different embodiments may also be combined to provide other embodiments. In addition, terms such as "including" and "comprising" should be understood as not limiting the described embodiments to consist only of those features that have been mentioned; such examples and embodiments may also include features, structures, units, modules, etc. that are not specifically mentioned.
[0061] The following is a detailed description of each embodiment of the present application in conjunction with the accompanying drawings. It should be noted that the display order of the embodiments of the present application only represents the order of the embodiments, and does not represent the advantages and disadvantages of the technical solutions provided by the embodiments.
[0062] It should be noted that the technical solution provided in the embodiment of the present application is explained by taking the video images and annotated images collected by multiple monitoring devices in a monitoring area as input data, fusing each video image to obtain the trajectory route of the detected target, and comparing it with the trajectory route of the annotated target included in the annotated image as an example, but is not limited to this.
[0063] See also Figure 1 The present application provides a motion trajectory comparison method, for example, by using a monitoring device to perform the method, on which a demo program encapsulating a trajectory fusion algorithm is running, and the demo program can perform image fusion, target detection, and motion trajectory comparison, including:
[0064] Step S101, when a test task is received, video images and annotated images collected by multiple monitoring devices in a specified monitoring area are obtained based on the test task; the annotated images are obtained by splicing image frames of the video images at the same time, marking the timestamp of the corresponding time on each spliced image, and marking the target features of each moving target including at least target ID, target type, coordinate frame, target coordinates, and moving speed;
[0065] The monitoring area, such as an intersection or a crossroad, has multiple monitoring devices installed at different locations in the monitoring area.
[0066] Labeling images means manually scaling the image frames at the same time according to a certain ratio, stitching the video images together, and marking the target attribute information such as the target ID, target type, coordinate frame, target coordinates, motion speed, target color, etc. of each moving target on the stitched image, so as to verify the trajectory route of the detected target output by the trajectory fusion algorithm.
[0067] Step S102: according to a set period, image frames at the same time are obtained from each of the video images for image fusion, and the timestamp of the corresponding time is marked on each fused image; the annotated images with the same timestamp as the fused images are determined to form an annotated image set; target detection is performed on each of the fused images to obtain target features of multiple detected targets; the target ID of the same detected target in different fused images is the same;
[0068] The set period, for example 40 to 80 seconds, can be set according to actual needs.
[0069] The image fusion and target detection can be implemented using existing technologies and will not be described in detail here.
[0070] Step S103, for each of the detected targets and each of the annotated targets included in the annotated image set, executing: determining the trajectory points of the target in the monitoring area according to the target coordinates of the target, marking a time tag on each of the trajectory points, and generating a trajectory route of the target in the monitoring area according to each of the trajectory points and the time tag corresponding to each of the trajectory points;
[0071] For example, according to the 3D coordinates of the target, the trajectory points of the target are marked on the bird's-eye view drawn according to the high-precision map of the monitoring area, and the trajectory route of the target on the bird's-eye view is generated according to each trajectory point and the corresponding time tag, for example Figure 2 As shown in the figure, id91 represents the detected target, id98 represents the labeled target, the darker points represent the trajectory points of the detected target, and the lighter points represent the trajectory points of the labeled target; the straight line represents the lane line, and the right angle represents the movement direction of the target.
[0072] Step S104: performing a motion trajectory comparison between the trajectory route of each detected target and the trajectory route of each marked target, and outputting a comparison result.
[0073] For each detected target’s trajectory, the detected target’s trajectory is compared with the trajectory of each labeled target, and the comparison result is output, for example: Figure 3 As shown, the target source correctly indicates that the trajectory route of the detected target matches the trajectory route of the matching annotated target. The comparison results can be displayed through the user interface. Users can search the comparison results by conditions, sort the comparison results, and view the details of the comparison results, such as Figure 4 As shown in the figure, h, l and w represent the detected target or the marked target trajectory point in the bird's eye view Figure 3The displacement or position change in a dimension; 01-04 represents 4 channels, i.e. 4 monitoring devices; the target angle refers to the direction of the target, which is obtained by converting the target in the calibrated coordinate system based on the calibrated reference direction, where the reference direction in the calibrated coordinate system is the true north direction. In the details interface of the comparison results, users can view the fused video generated during the comparison process, the trajectory route on the bird's-eye view, the trajectory points of the detection target currently viewed by the user, and the trajectory points of the matching annotated targets. They can also view the movement of the trajectory points when playing the fused video. Therefore, users can verify and check the results based on the comparison results, quickly determine whether the trajectory route of the detection target is matched, complete, and correct, and quickly determine the reasons for mismatch, incompleteness, and incorrectness, such as: the target features of the detection target obtained are inaccurate, the annotated image is wrong, and there are many interference sources in the annotated image.
[0074] In some embodiments, the comparing the trajectory of each detected target with the trajectory of each marked target and outputting the comparison result includes:
[0075] For each of the detection targets, based on the target type and coordinate frame of the detection target, when the target type of any of the labeled targets is different from the target type of the detection target, and / or the intersection and union ratio of the coordinate frame and the coordinate frame of the detection target is less than a first preset value, it is determined that there is no labeled target that matches the detection target, and a comparison result that the detection target is a virtual captured target is output.
[0076] The first preset value, such as 0.9, can be set according to actual needs.
[0077] If the target type of the detected target is the same as the target type of the marked target, and the intersection and union ratio of the coordinate frame of the detected target and the coordinate frame of the marked target is greater than or equal to a first preset value, then the detected target and the marked target are the same moving target, otherwise, they are not the same moving target.
[0078] The detected target is matched with the labeled target, and three matching results can be obtained:
[0079] (1) If there is a moving target in the detection target but not in the annotation target, then the moving target in the detection target is a virtual target. Therefore, the original video image can be checked to determine whether the moving target exists. If it exists, it means that the annotation image data is wrong and needs to be corrected. If it does not exist, there is a deviation in the image fusion and target detection process and the algorithm needs to be corrected.
[0080] (2) If a moving target is not included in the detection target but is included in the annotated target, then the moving target in the annotated target is a missed target. Therefore, the video image can be checked to determine whether the moving target exists. If it exists, it means that there is a deviation in the image fusion and target detection process, and the algorithm needs to be corrected. If it does not exist, it means that there is an error in the annotated image data, and the annotated image data needs to be corrected.
[0081] (3) If both the detected target and the annotated target have a moving target, the detection result is consistent with the annotated data, indicating that there is no deviation in the image fusion and target detection process.
[0082] In some embodiments, when it is determined that there is a marked target that matches the detected target, a first corresponding moment after the matched marked target moves a first set distance (e.g., 5 meters, which can be set according to actual needs) from the first track point is determined according to the movement speed of the matched marked target and the time tag of the first track point of the track route;
[0083] Respectively determining the trajectory points that the detection target and the matched labeled target pass through when moving from the first trajectory point to the first corresponding moment;
[0084] Based on the trajectory points passed by the detection target and the trajectory points passed by the matched marked target, the distance between each pair of trajectory points with the same time label is calculated. When the number of trajectory point pairs whose distance is less than a second preset value (for example, 1 meter) is less than a third preset value (for example, 2), it is determined that the trajectory route of the detection target does not match the trajectory route of the matched marked target, and the comparison result that the trajectory route of the detection target is an unmatched trajectory route is output.
[0085] By comparing the first few track points of the two track routes, we can determine whether the two track routes match. For example, the marked target starts from the first track point and moves for 5 meters, passing through a total of 5 track points, track points 2-6. During this period of time, the detected target also passes through 5 track points, track points 02-06, among which track points 2 and track points 02, track points 3 and track points 03, etc., track points 6 and track points 06 are track point pairs with the same time tags. The distance between these 5 pairs of track points is calculated. If the distance between at least 2 pairs of track points is less than 1 meter, the track route of the detected target matches the track route of the marked target; on the contrary, if the distance between 1 or 0 pairs of track points is less than 1 meter, the track route of the detected target does not match the track route of the marked target, indicating that there is a large deviation in the track fusion process, and the track fusion algorithm needs to be corrected.
[0086] It should be noted that the matched marked target may also move from the first trajectory point for a set time period, and the matching pairs of trajectory points passed by the detection target and the matched marked target within the set time period are determined. If the matching pairs are greater than a third preset value, the trajectory route of the detection target matches the trajectory route of the matched marked target.
[0087] In some embodiments, when it is determined that the trajectory route of the detected target matches the trajectory route of the matched marked target, a second corresponding moment when the matched marked target moves to a second set distance (e.g., 1 meter, which can be set as needed) from the last trajectory point is determined according to the movement speed of the matched marked target;
[0088] The second corresponding moment minus a preset time (for example, 3 seconds, which can be set according to actual needs) is used as the start time, and the second corresponding moment plus the preset time is used as the end time, and the trajectory points of the detected target and the matched marked target from the start time to the end time are determined respectively;
[0089] Determine the distance between each pair of two trajectory points with the same time tag according to the trajectory points passed by the detection target and the trajectory points passed by the matching labeled target;
[0090] When the minimum value of the distance is not less than a fourth preset value (eg, 5 meters, which can be set according to actual needs), a comparison result indicating that the trajectory route of the detected target is an incomplete trajectory route is output.
[0091] By comparing several track points close to the last track point of the two track routes, it is determined whether the track route of the detection target is a complete track route. For example, the marked target passes through 3 track points during this period of time, namely track points 7-9, and the detection target also passes through 3 track points during this period of time, namely 07-09. If track points 7 and 07, track points 8 and 08, and track points 9 and 09 are track point pairs with the same time tags, and the distances between these three pairs of track points are 6 meters, 7 meters, and 8 meters, then the minimum distance is 6 meters. If it is greater than 5 meters, the track route of the detection target is an incomplete track route, indicating that the later motion trajectory has deviations during the track fusion process, and the positioning problem can be analyzed based on this.
[0092] In some embodiments, when the minimum value of the distance is less than a fourth preset value, the distance between each pair of two trajectory points with the same time tags is determined according to the trajectory route of the detected target and the trajectory route of the matched labeled target;
[0093] For each pair of two trajectory points with the same time label, if the distance is less than the fifth preset value (for example, 1 meter, which can be set according to actual needs), the corresponding trajectory point of the detection target is output as the comparison result of the correct trajectory point; otherwise, the corresponding trajectory point of the detection target is output as the comparison result of the incorrect trajectory point.
[0094] When the trajectory route of the detected target matches the trajectory route of the matched marked target and is a complete trajectory route, it is determined whether each trajectory point on the trajectory route of the detected target is a correct trajectory point, that is, whether the distance with the corresponding trajectory point of the matched marked target is less than a preset value.
[0095] In some embodiments, after comparing the trajectory of each detected target with the trajectory of each marked target and outputting the comparison result, the method further includes:
[0096] Determine the number of detection trajectory routes according to the target ID of each detection target; determine the number of annotated trajectory routes according to the target ID of each annotated target; and obtain a capture rate by dividing the number of annotated trajectory routes by the number of detection trajectory routes; and
[0097] The number of detected trajectory routes minus the number of incomplete trajectory routes is used to obtain the number of complete trajectory routes; the number of detected trajectory routes minus the number of unmatched trajectory routes is used to obtain the number of matched trajectory routes; the number of complete trajectory routes is divided by the number of matched trajectory routes to obtain the completeness rate; and
[0098] Counting the total number of the correct trajectory points and dividing the result by the total number of trajectory points of each detection target to obtain the accuracy rate;
[0099] Generate a test report including at least the capture rate, completeness rate, and accuracy rate.
[0100] In order to intuitively judge the correctness of the matching between the detected trajectory route and the marked trajectory route, in some embodiments, the method further includes:
[0101] Generate a fused video according to each of the fused images and a timestamp corresponding to each of the fused images;
[0102] According to the detection target selected by the user, when playing the fused video, the trajectory route of the detection target and the trajectory route of the matching marked target with the same time label as the corresponding timestamp are determined respectively according to the timestamp corresponding to the currently played fused image, and the trajectory points are highlighted, so that the detection target and the matching marked target move to the corresponding trajectory points within the monitoring area as the fused video is played (that is, the trajectories of the detection target and the marked target move in real time within the monitoring area as the video is played, and the trajectory movement and route of the monitored target can be seen).
[0103] See also Figure 5 , a motion trajectory comparison system provided in an embodiment of the present application includes:
[0104] The acquisition module 100 is used for acquiring video images and annotated images collected by multiple monitoring devices in a specified monitoring area based on the test task when a test task is received; the annotated images are obtained by splicing image frames of the video images at the same time, marking the timestamp of the corresponding time on each spliced image, and marking the target features of each moving target including at least target ID, target type, coordinate frame, target coordinates, and moving speed;
[0105] The detection module 200 is used to obtain image frames at the same time from each of the video images according to a set period, perform image fusion, and mark the timestamp of the corresponding time on each fused image; determine the annotated images with the same timestamp as the fused images to form an annotated image set; perform target detection on each of the fused images to obtain target features of multiple detected targets; the target ID of the same detected target in different fused images is the same;
[0106] The generating module 300 is used to execute, for each of the detected targets and each of the annotated targets included in the annotated image set: determining the trajectory points of the target in the monitoring area according to the target coordinates of the target, marking a time tag on each of the trajectory points, and generating a trajectory route of the target in the monitoring area according to each of the trajectory points and the time tag corresponding to each of the trajectory points;
[0107] The comparison module 400 is used to compare the trajectory of each detected target with the trajectory of each marked target, and output the comparison result.
[0108] Several specific method flow examples are given below. Embodiment 1:
[0109] See also Figure 6The present application provides a motion trajectory comparison method, for example, by encapsulating a demo program with a motion trajectory fusion algorithm to execute the method, the demo program can perform image fusion, target detection, trajectory route generation and motion trajectory comparison, and the specific steps include:
[0110] Step S601: when a test task is received, video images and annotated images collected by multiple monitoring devices in a specified monitoring area are obtained according to the test task;
[0111] Step S602: according to a set period, image frames at the same time are obtained from each video image for image fusion, and a timestamp of the corresponding time is marked on each fused image;
[0112] In this step, at a certain time interval, image frames at the same moment are extracted from each video image for image fusion to obtain a fused image, and the timestamp of the image frame captured by the monitoring device is marked on the fused image.
[0113] Step S603: for each fused image, determine from the annotated images the annotated images with the same timestamp as the fused image to form an annotated image set;
[0114] Ensure that the fused image and the annotated image are images at the same time to ensure that the moving target obtained using the fused image later is consistent with the moving target in the annotated image, so that the trajectory of the annotated target in the annotated image can be used to verify the accuracy of the trajectory of the detected target, or whether there is any annotation error in the annotated image.
[0115] Step S604: Detect targets on each fused image to obtain target IDs, target types, target coordinates, coordinate frames, and motion speeds of multiple detected targets;
[0116] Step S605: for each target, according to the target coordinates of the target, mark the trajectory point of the target at the corresponding position of the bird's-eye view, mark each trajectory point with a time tag, and generate the trajectory route of the target in the bird's-eye view according to each trajectory point and the time tag corresponding to each trajectory point;
[0117] It should be noted that the target refers to each detected target and each annotated target included in the annotated image set; the bird's-eye view is a panoramic top-down stereoscopic view of the monitoring area drawn using a high viewpoint perspective method.
[0118] Step S606: for each detected target, determine whether there is a marked target matching the detected target in the marked targets, if so, proceed to step S607; if not, output a comparison result that the detected target is a virtual captured target;
[0119] Step S607, determining whether the trajectory route of the detected target matches the trajectory route of the matched marked target, if so, proceeding to step S608; if not, outputting a comparison result that the trajectory route of the detected target is a non-matching trajectory route;
[0120] Step S608: judging whether the trajectory of the detected target is complete according to the trajectory of the detected target and the trajectory of the matched marked target, and if so, proceeding to step S609; if not, outputting a comparison result that the trajectory of the detected target is an incomplete trajectory;
[0121] Step S609: determining the distance between each pair of two trajectory points with the same time tags according to the trajectory route of the detected target and the trajectory route of the matched labeled target;
[0122] Step S610: for each pair of two trajectory points with the same time tag, if the distance is less than a preset value, output the comparison result that the corresponding trajectory point of the detection target is a correct trajectory point; otherwise, output the comparison result that the corresponding trajectory point of the detection target is an incorrect trajectory point;
[0123] Step S611, determine whether each detection target has completed target matching, if not, proceed to step S606; if yes, proceed to step S612;
[0124] Step S612, determining whether there is an unmatched annotated target in the annotated image set, if so, outputting a comparison result that the annotated target is a missed target; if not, proceeding to step S613;
[0125] Step S613: Calculate the capture rate, completeness rate, and accuracy rate, and generate a test report that includes at least these three indicators.
[0126] For example, in the test report shown in Table 1, the number of annotated trajectories refers to the number of trajectory routes of the annotated target (the number of trajectory routes corresponds to the number of different annotated targets included in the annotated image set); the number of detected trajectories refers to the number of trajectory routes of the detected target (the number of trajectory routes corresponds to the number of different detected targets); the total number of trajectory points refers to the sum of the trajectory points of each detected target; capture rate = number of annotated trajectories / number of detected trajectories; completeness rate = number of complete trajectory routes / number of matched trajectory routes; accuracy rate = number of correct trajectory points / total trajectory points.
[0127] Table 1 Test report
[0128] Embodiment 2:
[0129] See also Figure 7The above step S606 determines whether there is a marked target that matches the detection target, and the specific steps include:
[0130] Step S701, respectively calculating the intersection and union ratios of the coordinate frame of the detected target and the coordinate frames of each labeled target;
[0131] Step S702, determining whether there is at least one intersection-to-union ratio greater than or equal to a first preset value, if so, proceeding to step S703; if not, outputting a comparison result that the detected target is a virtual grasp target;
[0132] Step S703: for the marked target whose intersection-over-union ratio is greater than the first preset value, determine whether its target type is the same as the target type of the detected target. If so, the detected target matches the marked target. If not, output a comparison result that the detected target is a virtual captured target.
[0133] It should be noted that it is also possible to first determine whether there is a labeled target of the same target type as the detection target. If so, calculate the intersection and union ratio of the coordinate frame of the detection target and the coordinate frame of the labeled target of the same target type to determine whether there is a labeled target that matches the detection target. Embodiment three:
[0134] See also Figure 8 The above step S607 determines whether the trajectory of the detected target matches the trajectory of the matched marked target, and the specific steps include:
[0135] Step S801, determining a first corresponding time after the matched marked object moves a first set distance from the first track point according to the movement speed of the matched marked object and the time tag of the first track point of the track route;
[0136] Step S802, respectively determining the trajectory points that the detected target and the matched labeled target pass through when moving from the first trajectory point to the first corresponding moment;
[0137] Step S803, calculating the distance between each pair of two trajectory points with the same time tag according to the trajectory points passed by the detected target and the trajectory points passed by the matched labeled target;
[0138] Step S804, determine whether the number of trajectory point pairs whose distance is less than the second preset value is greater than or equal to the third preset value. If so, the trajectory route of the detection target is a matching trajectory route; if not, output a comparison result that the trajectory route of the detection target is a non-matching trajectory route. Embodiment 4:
[0139] See also Fig. 9 The above step S608 determines whether the trajectory route of the detection target is complete, and the specific steps include:
[0140] Step S901: determining, according to the movement speed of the matched marked object, a second corresponding moment when the matched marked object moves to a second set distance from the last track point;
[0141] Step S902: Taking the second corresponding moment minus the preset time as the start time and the second corresponding moment plus the preset time as the end time, respectively determine the trajectory points of the detected target and the matching annotated target from the start time to the end time;
[0142] Step S903: determining the distance between each pair of two trajectory points with the same time tags according to the trajectory points passed by the detected target and the trajectory points passed by the matched labeled target;
[0143] Step S904: Determine the minimum value of the distance according to the distances calculated in step S903;
[0144] Step S905, determine whether the minimum value of the distance is greater than or equal to the fourth preset value. If so, output the comparison result that the trajectory route of the detection target is an incomplete trajectory route; if not, the trajectory route of the detection target is a complete trajectory route. Embodiment five:
[0145] See also Fig.10 The specific steps of generating the test report in step S613 include:
[0146] Step S1001: Determine the number of detection trajectory routes according to the target ID of each detection target; and determine the number of annotated trajectory routes according to the target ID of each annotated target;
[0147] Step S1002: Divide the number of marked trajectory routes by the number of detected trajectory routes to obtain a capture rate;
[0148] Step S1003: subtract the number of incomplete trajectory routes from the number of detected trajectory routes to obtain the number of complete trajectory routes; subtract the number of mismatched trajectory routes from the number of detected trajectory routes to obtain the number of matched trajectory routes;
[0149] Step S1004: Divide the number of complete trajectory routes by the number of matching trajectory routes to obtain a completeness rate;
[0150] Step S1005, count the total number of correct trajectory points, divide it by the total number of trajectory points of each detection target, and obtain the accuracy rate;
[0151] Step S1006: Generate a test report including at least capture rate, completeness rate, and accuracy rate.
[0152] It should be noted that the calculation of the capture rate, completeness rate, and accuracy rate can be performed sequentially or simultaneously.
[0153] The following is an introduction to the equipment or device provided in the embodiments of the present application, in which the explanations or examples of technical features that are the same or corresponding to those described in the above method will not be repeated later.
[0154] See also Fig.11 , a terminal device provided in an embodiment of the present application includes:
[0155] The processor 110 is used to read the program in the memory 130 and execute the following process:
[0156] When a test task is received, video images and annotated images collected by multiple monitoring devices in a specified monitoring area are obtained based on the test task; the annotated images are obtained by splicing image frames of the video images at the same time, marking the timestamp of the corresponding time on each spliced image, and marking the target features of each moving target including at least target ID, target type, coordinate frame, target coordinates, and moving speed;
[0157] According to a set period, image frames at the same time are obtained from each of the video images for image fusion, and timestamps of corresponding times are marked on each fused image; the annotated images with the same timestamp as the fused images are determined to form an annotated image set; target detection is performed on each of the fused images to obtain target features of multiple detected targets; the target IDs of the same detected target in different fused images are the same;
[0158] For each of the detected targets and each of the annotated targets included in the annotated image set, executing: determining the trajectory points of the target in the monitoring area according to the target coordinates of the target, marking a time tag on each of the trajectory points, and generating a trajectory route of the target in the monitoring area according to each of the trajectory points and the time tag corresponding to each of the trajectory points;
[0159] The trajectory of each detected target is compared with the trajectory of each marked target, and the comparison result is output.
[0160] In some embodiments, the comparing the trajectory of each detected target with the trajectory of each marked target and outputting the comparison result includes:
[0161] For each of the detection targets, based on the target type and coordinate frame of the detection target, when the target type of any of the labeled targets is different from the target type of the detection target, and / or the intersection and union ratio of the coordinate frame and the coordinate frame of the detection target is less than a first preset value, it is determined that there is no labeled target that matches the detection target, and a comparison result that the detection target is a virtual captured target is output.
[0162] In some embodiments, the processor 110 is further configured to execute:
[0163] When it is determined that there is a marked target that matches the detected target, determining a first corresponding time after the matched marked target moves a first set distance from the first trajectory point according to the movement speed of the matched marked target and the time tag of the first trajectory point of the trajectory route;
[0164] Respectively determining the trajectory points that the detection target and the matched labeled target pass through when moving from the first trajectory point to the first corresponding moment;
[0165] Based on the trajectory points passed by the detection target and the trajectory points passed by the matched marked target, the distance between each pair of trajectory points with the same time label is calculated. When the number of trajectory point pairs whose distance is less than the second preset value is less than the third preset value, it is determined that the trajectory route of the detection target does not match the trajectory route of the matched marked target, and the comparison result that the trajectory route of the detection target is an unmatched trajectory route is output.
[0166] In some embodiments, the processor 110 is further configured to execute:
[0167] When it is determined that the trajectory route of the detected target matches the trajectory route of the matched marked target, determining a second corresponding moment when the matched marked target moves to a second set distance from the last trajectory point according to the movement speed of the matched marked target;
[0168] The second corresponding moment minus the preset time is used as the start time, and the second corresponding moment plus the preset time is used as the end time, and the trajectory points of the detected target and the matched marked target moving from the start time to the end time are determined respectively;
[0169] Determine the distance between each pair of two trajectory points with the same time tag according to the trajectory points passed by the detection target and the trajectory points passed by the matching labeled target;
[0170] When the minimum value of the distance is not less than a fourth preset value, a comparison result indicating that the trajectory route of the detection target is an incomplete trajectory route is output.
[0171] In some embodiments, the processor 110 is further configured to execute:
[0172] When the minimum value of the distance is less than a fourth preset value, determining the distance between each pair of two trajectory points with the same time tags according to the trajectory route of the detected target and the trajectory route of the matched labeled target;
[0173] For each pair of two trajectory points with the same time tag, if the distance is less than the fifth preset value, the comparison result of the corresponding trajectory point of the detection target is output as the correct trajectory point; otherwise, the comparison result of the corresponding trajectory point of the detection target is output as the wrong trajectory point.
[0174] In some embodiments, after comparing the trajectory of each detected target with the trajectory of each marked target and outputting the comparison result, the method further includes:
[0175] Determine the number of detection trajectory routes according to the target ID of each detection target; determine the number of annotated trajectory routes according to the target ID of each annotated target; and obtain a capture rate by dividing the number of annotated trajectory routes by the number of detection trajectory routes; and
[0176] The number of detected trajectory routes minus the number of incomplete trajectory routes is used to obtain the number of complete trajectory routes; the number of detected trajectory routes minus the number of unmatched trajectory routes is used to obtain the number of matched trajectory routes; the number of complete trajectory routes is divided by the number of matched trajectory routes to obtain the completeness rate; and
[0177] Counting the total number of the correct trajectory points and dividing the result by the total number of trajectory points of each detection target to obtain the accuracy rate;
[0178] Generate a test report including at least the capture rate, completeness rate, and accuracy rate.
[0179] In some embodiments, the processor 110 is further configured to execute:
[0180] Generate a fused video according to each of the fused images and a timestamp corresponding to each of the fused images;
[0181] According to the detection target selected by the user, when playing the fused video, the trajectory route of the detection target and the trajectory route of the matching marked target with the same time label as the corresponding timestamp are determined respectively according to the timestamp corresponding to the currently played fused image, and the trajectory points are highlighted, so that the detection target and the matching marked target move to the corresponding trajectory points within the monitoring area as the fused video is played.
[0182] In some embodiments, the terminal device provided in the embodiments of the present application also includes a transceiver 120 for receiving and sending data to a monitoring device under the control of the processor 110 .
[0183] Among them, Fig.11 In the embodiment of the present invention, the bus architecture may include any number of interconnected buses and bridges, specifically linking various circuits of one or more processors represented by processor 110 and memory represented by memory 130. The bus architecture may also link various other circuits such as peripherals, voltage regulators, and power management circuits, which are well known in the art and are not further described herein. The bus interface provides an interface. The transceiver 120 may be a plurality of components, namely, a transmitter and a receiver, providing a unit for communicating with various other devices over a transmission medium.
[0184] The processor 110 is responsible for managing the bus architecture and general processing, and the memory 130 can store data used by the processor 110 when performing operations.
[0185] In some embodiments, the processor 110 may be a CPU (central processing unit), an ASIC (Application Specific Integrated Circuit), an FPGA (Field-Programmable Gate Array), or a CPLD (Complex Programmable Logic Device).
[0186] The embodiment of the present application provides a computing device, which may be a desktop computer, a portable computer, a smart phone, a tablet computer, a personal digital assistant (PDA), etc. The computing device may include a central processing unit (CPU), a memory, an input / output device, etc. The input device may include a keyboard, a mouse, a touch screen, etc. The output device may include a display device, such as a liquid crystal display (LCD), a cathode ray tube (CRT), etc.
[0187] The memory may include a read-only memory (ROM) and a random access memory (RAM), and provides the processor with program instructions and data stored in the memory. In the embodiment of the present application, the memory may be used to store the program of any of the methods provided in the embodiment of the present application.
[0188] The processor calls the program instructions stored in the memory, and the processor is used to execute any of the methods provided in the embodiments of the present application according to the obtained program instructions.
[0189] The present application embodiment also provides a computer program product or computer program, which includes a computer instruction, which is stored in a computer-readable storage medium. The processor of the computer device reads the computer instruction from the computer-readable storage medium, and the processor executes the computer instruction so that the computer device performs any of the methods described in the above embodiments. The program product can use any combination of one or more readable media. The readable medium can be a readable signal medium or a readable storage medium. The readable storage medium can be, for example, - but not limited to - an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or device, or any combination of the above. More specific examples of readable storage media (non-exhaustive list) include: an electrical connection with one or more wires, a portable disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above.
[0190] The embodiment of the present application provides a computer-readable storage medium for storing computer program instructions used by the apparatus provided in the embodiment of the present application, which includes a program for executing any of the methods provided in the embodiment of the present application. The computer-readable storage medium may be a non-transitory computer-readable medium.
[0191] The computer-readable storage medium can be any available medium or data storage device that can be accessed by a computer, including but not limited to magnetic storage (such as floppy disks, hard disks, magnetic tapes, magneto-optical disks (MO), etc.), optical storage (such as CD, DVD, BD, HVD, etc.), and semiconductor storage (such as ROM, EPROM, EEPROM, non-volatile memory (NANDFLASH), solid-state drive (SSD)), etc.
[0192] Those skilled in the art will appreciate that the embodiments of the present application may be provided as methods, systems, or computer program products. Therefore, the present application may adopt the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware. Moreover, the present application may adopt the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program codes.
[0193] The present application is described with reference to the flowcharts and / or block diagrams of the methods, devices (systems), and computer program products according to the present application. It should be understood that each process and / or block in the flowchart and / or block diagram, as well as the combination of processes and / or blocks in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.
[0194] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.
[0195] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.
[0196] Obviously, those skilled in the art can make various changes and modifications to the present application without departing from the spirit and scope of the present application. Thus, if these modifications and variations of the present application fall within the scope of the claims of the present application and their equivalents, the present application is also intended to include these modifications and variations.
Claims
1. A motion trajectory comparison method, characterized in that: include: When a test task is received, video images and annotated images collected by multiple monitoring devices in a specified monitoring area are obtained based on the test task; The annotated image is obtained by splicing image frames of the video images at the same time, marking the timestamp of the corresponding time on each spliced image, and marking the target features of each moving target including at least target ID, target type, coordinate frame, target coordinates, and moving speed; According to a set period, image frames at the same time are obtained from each of the video images for image fusion, and a timestamp of the corresponding time is marked on each fused image; Determine that the annotated images having the same timestamp as the fused images constitute an annotated image set; Performing target detection on each of the fused images to obtain target features of multiple detected targets; The target ID of the same detection target in different fused images is the same; For each of the detected targets and each of the annotated targets included in the annotated image set, executing: determining the trajectory points of the target in the monitoring area according to the target coordinates of the target, marking a time tag on each of the trajectory points, and generating a trajectory route of the target in the monitoring area according to each of the trajectory points and the time tag corresponding to each of the trajectory points; Comparing the trajectory of each detected target with the trajectory of each marked target, and outputting the comparison result; The step of comparing the trajectory of each detected target with the trajectory of each marked target and outputting the comparison result includes: For each of the detection targets: Determine a marked target that matches the detected target according to the target type and the coordinate frame of the detected target; the target type of the matched marked target is the same as the target type of the detected target, and the intersection value of the coordinate frame and the coordinate frame of the detected target is greater than or equal to a first preset value; According to the trajectory route of the detected target and the trajectory route of the matched marked target, when it is determined that the two trajectory routes match, according to the movement speed of the matched marked target, determine the second corresponding moment when the matched marked target moves to a second set distance from the last trajectory point; The second corresponding moment minus the preset time is used as the start time, and the second corresponding moment plus the preset time is used as the end time, and the trajectory points of the detected target and the matched marked target moving from the start time to the end time are determined respectively; Determine the distance between each pair of two trajectory points with the same time tag according to the trajectory points passed by the detection target and the trajectory points passed by the matching labeled target; When the minimum value of the distance is not less than a fourth preset value, a comparison result indicating that the trajectory route of the detection target is an incomplete trajectory route is output.
2. The method according to claim 1, characterized in that The step of comparing the trajectory of each detected target with the trajectory of each marked target and outputting the comparison result further includes: For each of the detection targets, based on the target type and coordinate frame of the detection target, when the target type of any of the labeled targets is different from the target type of the detection target, and / or the intersection and union ratio of the coordinate frame and the coordinate frame of the detection target is less than a first preset value, it is determined that there is no labeled target that matches the detection target, and a comparison result that the detection target is a virtual captured target is output.
3. The method according to claim 1, characterized in that According to the trajectory route of the detection target and the trajectory route of the matching marked target, when it is determined that the two trajectory routes do not match, outputting a comparison result that the trajectory route of the detection target is an unmatched trajectory route; Wherein, determining that the two trajectory routes do not match, according to the trajectory route of the detected target and the trajectory route of the matching marked target, includes: Determine, according to the movement speed of the matched marked object and the time tag of the first track point of the track route, a first corresponding time after the matched marked object moves a first set distance from the first track point; Respectively determining the trajectory points that the detection target and the matched labeled target pass through when moving from the first trajectory point to the first corresponding moment; Based on the trajectory points passed by the detection target and the trajectory points passed by the matched marked target, the distance between each pair of trajectory points with the same time label is calculated. When the number of trajectory point pairs whose distance is less than the second preset value is less than the third preset value, it is determined that the trajectory route of the detection target does not match the trajectory route of the matched marked target.
4. The method according to claim 1, characterized in that: When the minimum value of the distance is less than a fourth preset value, determining the distance between each pair of two trajectory points with the same time tags according to the trajectory route of the detected target and the trajectory route of the matched labeled target; For each pair of two trajectory points with the same time tag, if the distance is less than the fifth preset value, the comparison result of the corresponding trajectory point of the detection target is output as the correct trajectory point; otherwise, the comparison result of the corresponding trajectory point of the detection target is output as the wrong trajectory point.
5. The method according to claim 4, characterized in that After comparing the trajectory of each detected target with the trajectory of each marked target and outputting the comparison result, the method further includes: Determine the number of detection trajectory routes according to the target ID of each detection target; determine the number of annotated trajectory routes according to the target ID of each annotated target; and obtain a capture rate by dividing the number of annotated trajectory routes by the number of detection trajectory routes; and The number of detected trajectory routes minus the number of incomplete trajectory routes is used to obtain the number of complete trajectory routes; the number of matching trajectory routes is determined according to the target ID of the detection target determined to be matched by the trajectory route; the number of complete trajectory routes is divided by the number of matching trajectory routes to obtain the completeness rate; and Counting the total number of the correct trajectory points and dividing the result by the total number of trajectory points of each detection target to obtain the accuracy rate; Generate a test report including at least the capture rate, completeness rate, and accuracy rate.
6. The method according to claim 1, characterized in that The method further comprises: Generate a fused video according to each of the fused images and a timestamp corresponding to each of the fused images; According to the detection target selected by the user, when playing the fused video, the trajectory route of the detection target and the trajectory route of the matching marked target with the same time label as the corresponding timestamp are determined respectively according to the timestamp corresponding to the currently played fused image, and the trajectory points are highlighted, so that the detection target and the matching marked target move to the corresponding trajectory points within the monitoring area as the fused video is played.
7. A motion trajectory comparison system, characterized in that: include: An acquisition module, configured to, when receiving a test task, acquire video images and annotated images collected by a plurality of monitoring devices in a specified monitoring area based on the test task; The annotated image is obtained by splicing image frames of the video images at the same time, marking the timestamp of the corresponding time on each spliced image, and marking the target features of each moving target including at least target ID, target type, coordinate frame, target coordinates, and moving speed; A detection module, used to obtain image frames at the same time from each of the video images according to a set period, perform image fusion, and mark a timestamp of the corresponding time on each fused image; Determine that the annotated images having the same timestamp as the fused images constitute an annotated image set; Performing target detection on each of the fused images to obtain target features of multiple detected targets; The target ID of the same detection target in different fused images is the same; A generating module, for each of the detected targets and each of the annotated targets included in the annotated image set, executing: determining the trajectory points of the target in the monitoring area according to the target coordinates of the target, marking a time tag on each of the trajectory points, and generating a trajectory route of the target in the monitoring area according to each of the trajectory points and the time tag corresponding to each of the trajectory points; A comparison module, used to compare the trajectory of each detected target with the trajectory of each marked target, and output the comparison result; The step of comparing the trajectory of each detected target with the trajectory of each marked target and outputting the comparison result includes: For each of the detection targets: Determine a marked target that matches the detected target according to the target type and the coordinate frame of the detected target; the target type of the matched marked target is the same as the target type of the detected target, and the intersection value of the coordinate frame and the coordinate frame of the detected target is greater than or equal to a first preset value; According to the trajectory route of the detected target and the trajectory route of the matched marked target, when it is determined that the two trajectory routes match, according to the movement speed of the matched marked target, determine the second corresponding moment when the matched marked target moves to a second set distance from the last trajectory point; The second corresponding moment minus the preset time is used as the start time, and the second corresponding moment plus the preset time is used as the end time, and the trajectory points of the detected target and the matched marked target moving from the start time to the end time are determined respectively; Determine the distance between each pair of two trajectory points with the same time tag according to the trajectory points passed by the detection target and the trajectory points passed by the matching labeled target; When the minimum value of the distance is not less than a fourth preset value, a comparison result indicating that the trajectory route of the detection target is an incomplete trajectory route is output.
8. A terminal device, characterized in that: include: A memory for storing program instructions; A processor, configured to call the program instructions stored in the memory, and execute the method according to any one of claims 1 to 6 according to the obtained program.
9. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores computer-executable instructions, and the computer-executable instructions are used to enable the computer to execute the method according to any one of claims 1 to 6.
Citation Information
Patent Citations
Motor vehicle track matching method, device and equipment
CN118365911A
