Video speed measurement method and device, electronic equipment and storage medium

Abstract

The invention discloses a video speed measurement method and device, electronic equipment and a storage medium, and the method comprises the steps: obtaining a video data set, the video data set comprises a plurality of videos, the data content of each of the plurality of videos comprises a plurality of vehicles, and the video data set is subjected to video spatialization to obtain a video data set; the video spatialization is used for establishing a mapping model between pixel coordinates and geographic space coordinates; performing target detection on the video data set to obtain a detection result taking the vehicle as a detection target; performing target tracking on the detection result to obtain a tracking identifier of the detection result; and measuring the speed of the plurality of vehicles based on the detection result and the tracking identifier by using the mapping model to obtain a speed measurement result. Therefore, according to the scheme, special hardware equipment does not need to be additionally deployed, and the vehicle can be detected and tracked by utilizing the existing video resources and the target detection and target tracking algorithm capability, so that the deployment cost and the maintenance difficulty can be greatly reduced. Moreover, on the basis of target detection and tracking, precise measurement and calculation of the vehicle speed can be further realized in combination with spatialized data attributes.

Description

Technical Field

[0001] This application relates to the field of computer technology, and in particular to a video speed measurement method, apparatus, electronic device, and storage medium. Background Technology

[0002] In the current process of intelligentizing transportation and urban governance, video surveillance systems have become one of the important means of capturing the operational status of road traffic, widely serving scenarios such as traffic flow monitoring, road safety control, and comprehensive urban management. However, most traditional video surveillance systems can only complete the real-time presentation and basic storage functions of video images, making it difficult to meet the needs of in-depth intelligent analysis, especially in the key application of vehicle speed detection, where the shortcomings are particularly prominent.

[0003] Therefore, current mainstream vehicle speed measurement solutions often rely on dedicated hardware equipment such as radar or laser ranging. Although these solutions can achieve the speed measurement goal to a certain extent, they have problems such as high deployment costs and high maintenance difficulty. Summary of the Invention

[0004] This application provides a video speed measurement method, device, electronic device, and storage medium that can achieve vehicle speed measurement without dedicated hardware, thereby reducing deployment costs and maintenance difficulty.

[0005] In a first aspect, embodiments of this application provide a video speed measurement method, including:

[0006] A video dataset is obtained, which includes multiple videos. The data content of each video includes multiple vehicles. The video dataset is spatialized and used to establish a mapping model between pixel coordinates and geospatial coordinates.

[0007] Target detection is performed on the video dataset to obtain detection results with vehicles as the detection targets;

[0008] The detection results are used for target tracking to obtain the tracking identifier of the detection results;

[0009] Using the mapping model, the speed of the multiple vehicles is measured based on the detection results and the tracking identifier to obtain the speed measurement results.

[0010] Optionally, the step of performing object detection on the video dataset to obtain detection results with vehicles as the detection target includes:

[0011] The video dataset is extracted frame by frame to obtain the image to be processed;

[0012] The target detection model is used to perform target detection on the image to be processed to obtain initial detection results. The initial detection results include the type of each of the multiple detected targets, the detection box, and the confidence level of the detection box.

[0013] Based on the types of the multiple detection targets, the detection result is determined from the initial detection result, and the type of the detection target corresponding to the detection result is a vehicle.

[0014] Optionally, the step of performing target detection on the image to be processed based on the target detection model to obtain initial detection results includes:

[0015] Based on the target detection model, target detection is performed on the image to be processed to obtain the type of the same target, multiple initial detection boxes, and the confidence of the multiple initial detection boxes;

[0016] The multiple initial detection boxes are deduplicated to obtain the detection boxes for the same detection target and the confidence scores of the detection boxes for the same detection target.

[0017] Optionally, the detection result includes the confidence level of the detection frame for each of the plurality of vehicles;

[0018] For the first vehicle among the plurality of vehicles, the step of performing target tracking on the detection result to obtain a tracking identifier for the detection result includes:

[0019] Based on the confidence level of the detection box for each vehicle, a plurality of first detection boxes and a plurality of second detection boxes are determined, wherein the confidence level of the plurality of first detection boxes is higher than a confidence threshold and the confidence level of the plurality of second detection boxes is lower than a confidence threshold.

[0020] Based on the trajectory of the first vehicle in the previous frames, the trajectory of the first vehicle in the current frame is predicted to obtain the predicted bounding box of the first vehicle.

[0021] Determine the first intersection-union ratio (IUU) between the predicted bounding box of the first vehicle and the plurality of first detection boxes;

[0022] Using the Hungarian algorithm, based on the first intersection-union ratio, the predicted bounding box of the first vehicle is matched with the plurality of first detection boxes to obtain a first matching result;

[0023] If the first matching result indicates that the predicted bounding box of the first vehicle successfully matches the first target detection box in the plurality of first detection boxes, the target trajectory box of the first vehicle is determined based on the predicted bounding box of the first vehicle and the first target detection box, and the tracking identifier of the first vehicle is generated based on the target trajectory box of the first vehicle.

[0024] Optionally, after generating the tracking identifier of the first vehicle based on the target trajectory bounding box of the first vehicle, the method further includes:

[0025] Determine the second intersection-union ratio between the predicted bounding box of the second vehicle (excluding the first vehicle) and the plurality of second detection boxes;

[0026] Using the Hungarian algorithm, based on the second intersection-union ratio, the predicted bounding box of the second vehicle is matched with the plurality of second detection boxes to obtain a second matching result;

[0027] If the second matching result indicates that the predicted bounding box of the second vehicle successfully matches the second target detection box in the plurality of second detection boxes, the target trajectory box of the second vehicle is determined based on the predicted bounding box of the second vehicle and the second target detection box, and the tracking identifier of the second vehicle is generated based on the target trajectory box of the second vehicle.

[0028] Optionally, after generating the tracking identifier of the second vehicle based on the target trajectory bounding box of the second vehicle, the method further includes:

[0029] Using the Hungarian algorithm, based on the first intersection-union ratio, the predicted bounding box of the third vehicle is matched with the other first detection boxes in the plurality of first detection boxes except the first target detection box, to obtain a third matching result. The third vehicle is the vehicle among the plurality of vehicles other than the first vehicle and the second vehicle.

[0030] In the case where the third matching result indicates that the predicted bounding box of the third vehicle is successfully matched with the third target detection box in the other first detection boxes, the target trajectory box of the third vehicle is determined based on the predicted bounding box of the third vehicle and the third target detection box, and the tracking identifier of the third vehicle is generated based on the target trajectory box of the third vehicle.

[0031] If the third matching result indicates that the predicted bounding box of the third vehicle fails to match any of the other first detection boxes, the predicted bounding box of the third vehicle is deleted.

[0032] If the third matching result indicates that all other first detection boxes fail to match the third vehicle, and the difference between the confidence level of the other first detection boxes and the confidence level threshold is less than the difference threshold, then the predicted box of the third vehicle is deleted.

[0033] Optionally, for a target vehicle among the plurality of vehicles, the step of using the mapping model to measure the speed of the plurality of vehicles based on the detection results and the tracking identifier to obtain speed measurement results includes:

[0034] Based on the tracking identifier of the target vehicle, the detection box corresponding to the tracking identifier of the target vehicle is bound to the target vehicle;

[0035] Based on the mapping model, the pixel coordinates of all detection boxes bound to the target vehicle are converted into geospatial coordinates;

[0036] Anomaly analysis is performed on all the detection boxes to identify and remove abnormal detection boxes, thus obtaining the target detection boxes;

[0037] If the distance between the center point of the first frame detection box and the center point of the current frame detection box is greater than the center point distance threshold, it is determined that the target vehicle is in a moving state.

[0038] The speed measurement result of the target vehicle is determined based on the geospatial coordinates of the first frame detection box, the geospatial coordinates of the last frame detection box in the target detection box, and the frame time interval corresponding to the first frame detection box and the last frame detection box.

[0039] Secondly, embodiments of this application provide a video speed measurement device, comprising:

[0040] The data acquisition module is used to acquire a video dataset, which includes multiple videos. The data content of each video includes multiple vehicles. The video dataset is spatialized, and the video spatialization is used to establish a mapping model between pixel coordinates and geospatial coordinates.

[0041] The target detection module is used to perform target detection on the video dataset to obtain detection results with vehicles as the detection targets;

[0042] The target tracking module is used to track the target based on the detection result and obtain the tracking identifier of the detection result;

[0043] The speed measurement module is used to measure the speed of the multiple vehicles based on the detection results and the tracking identifier using the mapping model, and to obtain the speed measurement results.

[0044] Thirdly, embodiments of this application provide an electronic device, the device including: a processor, a memory, and a system bus;

[0045] The processor and the memory are connected via the system bus;

[0046] The memory is used to store a program, the program including instructions that, when executed by the processor, cause the processor to perform any of the implementation steps of the video speed measurement method described above.

[0047] Fourthly, embodiments of this application provide a computer-readable storage medium storing a computer program, which, when executed by a processor, implements any of the implementation steps of the video speed measurement method described above.

[0048] As can be seen from the above technical solutions, the embodiments of this application have the following advantages:

[0049] In this embodiment, firstly, a video dataset is acquired, comprising multiple videos, each containing data on multiple vehicles. The video dataset undergoes video spatialization, which establishes a mapping model between pixel coordinates and geographic spatial coordinates. Next, target detection is performed on the video dataset to obtain detection results with vehicles as the target. Target tracking is then performed on the detection results to obtain tracking identifiers. Subsequently, the mapping model is used to measure the speed of multiple vehicles based on the detection results and tracking identifiers, yielding the speed measurement results. As can be seen, this solution eliminates the need for dedicated hardware deployment. By utilizing existing video resources and target detection and tracking algorithms, vehicle detection and tracking can be achieved, significantly reducing deployment costs and maintenance complexity. Furthermore, since the video dataset undergoes video spatialization, the video data is associated with geographic spatial information. Therefore, based on target detection and tracking, and combined with spatialized data attributes, the vehicle's trajectory can be transformed into positional changes at the geographic coordinate level, thereby enabling accurate calculation of vehicle speed. Attached Figure Description

[0050] Figure 1 A flowchart of a video speed measurement method provided in an embodiment of this application;

[0051] Figure 2 This is a schematic diagram illustrating the effect of a video speed measurement method provided in an embodiment of this application;

[0052] Figure 3 This is a schematic diagram of the structure of a video speed measuring device provided in an embodiment of this application. Detailed Implementation

[0053] To facilitate understanding, the relevant terms used in this plan will be explained below.

[0054] Video spatialization refers to the process of establishing a mapping model between video pixel coordinates and geospatial coordinates when the video itself does not possess geospatial information. This enables the mutual conversion between video pixel coordinates and geospatial coordinates, supporting the fusion display of video data and geospatial data.

[0055] Object detection is one of the core tasks in computer vision. It uses algorithms to identify specific objects in images or videos and determine their location and category. Object detection is widely used in fields such as autonomous driving, security monitoring, medical image analysis, and smart retail.

[0056] Target tracking is a computer vision technique designed to identify and track the position and trajectory of a specific target within a video sequence. It is commonly used in fields such as surveillance, autonomous driving, and robot navigation, enabling real-time analysis of target states and serving as a key technology for achieving advanced visual understanding.

[0057] As mentioned earlier, current mainstream vehicle speed measurement solutions often rely on dedicated hardware such as radar or laser ranging. While these solutions can achieve the speed measurement goal to a certain extent, they suffer from high deployment costs and high maintenance difficulties.

[0058] Furthermore, with the rapid development of artificial intelligence technology, target detection and tracking algorithms based on video data are becoming increasingly mature. However, existing algorithms mostly stop at the target detection and classification stage, making it difficult to achieve spatial representation and computation. Therefore, they often remain at the image-level recognition level and cannot support accurate vehicle speed calculation. In other words, current algorithms are difficult to transform into spatial data and decision support that can directly serve traffic monitoring, smart city management, and the development of the low-altitude economy. This results in weak application scenario expansion capabilities and difficulty in adapting to the development needs of smart transportation and new urban applications.

[0059] To address the aforementioned issues, this application provides a video speed measurement method, comprising: first, acquiring a video dataset including multiple videos, each containing data on multiple vehicles; and second, spatializing the video dataset to establish a mapping model between pixel coordinates and geographic spatial coordinates; third, performing target detection on the video dataset to obtain detection results with vehicles as the detection targets, and then tracking the detection results to obtain tracking identifiers; finally, using the mapping model, measuring the speed of multiple vehicles based on the detection results and tracking identifiers to obtain the speed measurement result. As can be seen, this solution eliminates the need for dedicated hardware deployment, utilizing existing video resources and target detection and tracking algorithms to replace dedicated hardware functions, thereby significantly reducing deployment costs and maintenance complexity.

[0060] Furthermore, because the video dataset has undergone spatialization, this means the video data has been linked to geospatial information. Therefore, based on target detection and tracking, and combined with spatialized data attributes, the vehicle's trajectory can be transformed into positional changes at the geographic coordinate level, thereby enabling more accurate calculation of vehicle speed. Moreover, the speed measurement results can be directly integrated into traffic monitoring platforms, smart city management systems, and the low-altitude economic information transmission field, adapting to the data application needs of different scenarios.

[0061] It should be noted that the implementation subject of the video speed measurement method in this application is not limited. For example, the video speed measurement method in this application can be applied to data processing devices such as servers or terminal devices. The server can be a standalone server, a cluster server, or a cloud server. The terminal device can be an electronic device such as a smartphone, computer, personal digital assistant (PDA), or tablet computer.

[0062] To make the objectives, technical solutions, and advantages of the embodiments of this application clearer, the technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those of ordinary skill in the art without creative effort are within the scope of protection of this application.

[0063] Figure 1 This is a flowchart illustrating a video speed measurement method provided in an embodiment of this application. (In conjunction with...) Figure 1 As shown, the video speed measurement method provided in this application embodiment may include the following steps S101-S104.

[0064] S101: Obtain the video dataset, which includes multiple videos. The data content of each video includes multiple vehicles. The video dataset is spatialized, which is used to establish a mapping model between pixel coordinates and geospatial coordinates.

[0065] In this embodiment, multiple videos can be pre-added using SuperMap iDesktopX (a cross-platform, full-featured desktop GIS software), for example, by adding local video file paths and / or online video links, thus obtaining a video dataset. Next, the software spatializes each video in the video dataset. In this way, since the video dataset has already been spatialized, the pixel coordinates of the vehicle's location obtained through object detection and tracking can be converted into geospatial coordinates with spatial information. After obtaining the real geospatial coordinates and combining them with time, the vehicle speed can be calculated.

[0066] S102: Perform object detection on the video dataset to obtain detection results with vehicles as the detection target.

[0067] In the object detection process, the first step is to extract the video dataset frame by frame to obtain the image to be processed.

[0068] Next, target detection is performed on the image to be processed based on the target detection model to obtain the initial detection results, which include the type of each of the multiple detected targets, the detection box, and the confidence of the detection box.

[0069] Then, based on the types of multiple detection targets, the detection result can be determined from the initial detection results, and the type of the detection target corresponding to the detection result is a vehicle.

[0070] In this way, based on the type of the target being detected, the detection results for vehicles can be quickly and accurately filtered out, thus achieving vehicle target detection in the video dataset.

[0071] In practical applications, for any target in the image to be processed, there may be multiple initial detection boxes. Therefore, during target detection, the optimal result can be determined from multiple initial detection boxes as the detection box for that target. To facilitate understanding, an example is provided below.

[0072] As an example, the image to be processed is first analyzed using an object detection model to obtain the type of the same object, multiple initial detection boxes, and the confidence scores of these initial detection boxes. Then, the multiple initial detection boxes are deduplicated to obtain the detection boxes for the same object and their confidence scores.

[0073] Furthermore, in practical applications, since object detection models generally have requirements for the color mode and dimensions of the input image, the image to be processed can be preprocessed before object detection to adapt to the input requirements of the object detection model.

[0074] As an example, the object detection model takes the YOLO (You Only Look Once) model. Since the YOLO model generally requires the width and height of the input image to be a multiple of 32, such as 640 (width) * 640 (height) pixels, and also requires the color mode to be in the order of Blue Green Red (BGR) channels, the image to be processed obtained frame by frame can be converted to the aspect ratio and corresponding color mode required by the object detection model, and then the object detection model can be used to perform object detection on the processed image.

[0075] S103: Perform target tracking on the detection results to obtain the tracking identifier of the detection results.

[0076] As mentioned earlier, the detection results can include the confidence level of the detection boxes for each of the multiple vehicles. Therefore, for ease of understanding, the ByteTrack algorithm will be used as an example to explain the target tracking process in detail below.

[0077] As one possible implementation method, for ease of understanding, the matching process at this stage will be described below using the first vehicle among multiple vehicles as an example.

[0078] First, based on the confidence level of each vehicle's detection box, multiple first detection boxes and multiple second detection boxes can be determined. The confidence levels of the first detection boxes are higher than a confidence threshold, while the confidence levels of the second detection boxes are lower than the confidence threshold. Traditional tracking algorithms typically only use detection boxes with high confidence levels for target tracking, which can easily lead to the loss of some targets. The ByteTrack algorithm, however, can further utilize detection boxes with lower confidence levels to continue the trajectory and improve continuity.

[0079] Next, based on the trajectory of the first vehicle in the previous frames, the trajectory of the first vehicle in the current frame is predicted to obtain the predicted bounding box of the first vehicle. Specifically, a tracking trajectory can be created for the first vehicle, and then the motion features of the first vehicle (such as speed, direction, and position change trends in pixels) can be extracted. Then, a motion estimation algorithm (such as Kalman Filter) is used to predict the possible position range of the first vehicle in the current frame based on the trajectory of the first vehicle in the previous frames, that is, the predicted bounding box of the first vehicle.

[0080] Next, the first intersection-over-union ratio (IOU) between the predicted bounding box of the first vehicle and the multiple first detection boxes can be determined. Here, the intersection-over-union ratio (IOU) is an important indicator in computer vision used to measure the degree of overlap between two detection boxes. In this application embodiment, the determination method of the first intersection-over-union ratio is not limited, and it can be obtained by any existing or future formula for calculating the intersection-over-union ratio.

[0081] Then, using the Hungarian algorithm, based on the first intersection-union ratio, the predicted bounding box of the first vehicle is matched with multiple first detection boxes to obtain the first matching result.

[0082] In this way, if the first matching result indicates that the predicted bounding box of the first vehicle successfully matches the first target detection box in multiple first detection boxes, the target trajectory box of the first vehicle can be determined based on the predicted bounding box and the first target detection box, and a tracking identifier for the first vehicle can be generated based on the target trajectory box. The target trajectory box of the first vehicle can be calculated using a Kalman filter algorithm from the predicted bounding box and the first target detection box, and the box corresponding to the tracking trajectory created for the first vehicle mentioned above can be updated with this target trajectory box.

[0083] If the first matching result indicates that the predicted bounding box of the first vehicle fails to match with multiple first detection boxes, the predicted bounding box of the first vehicle can be temporarily stored and the next stage of the matching process can begin.

[0084] To facilitate understanding, the following example uses the second vehicle (excluding the first vehicle) among multiple vehicles to illustrate the matching process in this stage. This second vehicle can be a vehicle that failed to match in the previous stage.

[0085] Accordingly, firstly, the second intersection-union ratio (IUU) between the predicted bounding box of the second vehicle and multiple second detection boxes can be determined. That is, at this point, the detection boxes with lower confidence can be matched again using the algorithm function mentioned above.

[0086] Next, using the Hungarian algorithm, based on the second intersection-union ratio, the predicted bounding box of the second vehicle is matched with multiple second detection boxes to obtain the second matching result.

[0087] If the second matching result indicates that the predicted bounding box of the second vehicle successfully matches the second target detection box in multiple second detection boxes, the target trajectory box of the second vehicle is determined based on the predicted bounding box of the second vehicle and the second target detection box, and a tracking identifier for the second vehicle is generated based on the target trajectory box of the second vehicle. The method for determining the target trajectory box of the second vehicle can be referred to the method for determining the target trajectory box of the first vehicle described above, and will not be repeated here.

[0088] If the second matching result indicates that the predicted bounding box of the second vehicle fails to match with multiple second detection boxes, the predicted bounding box of the second vehicle can be temporarily stored and the process can continue to the next stage of the matching process.

[0089] To facilitate understanding, the following example uses a third vehicle (other than the first and second vehicles) to illustrate the matching process in this stage. This third vehicle can be a vehicle that failed to be matched in the first two stages.

[0090] Accordingly, firstly, the Hungarian algorithm can be used to match the predicted bounding box of the third vehicle with other first detection boxes (excluding the first target detection box) in multiple first detection boxes based on the first intersection-union ratio, to obtain the third matching result.

[0091] Here, the third matching result can be any of the following possibilities: the predicted bounding box of the third vehicle successfully matches the third target detection box in other first detection boxes; the predicted bounding box of the vehicle successfully matches the third target detection box in other first detection boxes; or all other first detection boxes fail to match the third vehicle. For ease of understanding, the implementation process corresponding to these three cases is described below.

[0092] As an example, if the third matching result indicates that the predicted bounding box of the third vehicle successfully matches the third target detection box in other first detection boxes, the target trajectory box of the third vehicle is determined based on the predicted bounding box of the third vehicle and the third target detection box, and a tracking identifier for the third vehicle is generated based on the target trajectory box of the third vehicle. The method for determining the target trajectory box of the third vehicle can be found in the section on determining the target trajectory box of the third vehicle described above, and will not be repeated here.

[0093] As another example, if the third matching result indicates that the predicted bounding box of the third vehicle fails to match any of the other first detection boxes, the predicted bounding box of the third vehicle is deleted.

[0094] As another example, if the third matching result indicates that all other first detection boxes fail to match the third vehicle, and the difference between the confidence score of the other first detection boxes and the confidence score threshold is less than the difference threshold, then the predicted box of the third vehicle is deleted. Here, the difference threshold can be 0.1, meaning that when the confidence score of the other first detection boxes is greater than the confidence score threshold + 0.1, a new tracking trajectory can be created for the third vehicle, while when the confidence score of the other first detection boxes is less than the confidence score threshold + 0.1, the predicted box of the third vehicle is deleted.

[0095] Based on the above steps, a unique tracking identifier corresponding to the detection result can be obtained quickly and accurately. Subsequently, the detection result and the tracking identifier can be used to measure the speed of the detection target, thereby reducing deployment costs and maintenance difficulty.

[0096] S104: Using a mapping model, speeds of multiple vehicles are measured based on detection results and tracking markers to obtain speed measurement results.

[0097] In the specific implementation of speed measurement, taking a target vehicle among multiple vehicles as an example, firstly, based on the target vehicle's tracking identifier, the detection frame corresponding to the target vehicle's tracking identifier can be bound to the target vehicle. It should be noted that the target vehicle can be any vehicle among multiple vehicles that has completed the above binding steps.

[0098] Next, as mentioned earlier, the mapping model is a mapping model between pixel coordinates and geospatial coordinates. Based on this, the pixel coordinates of all detection boxes bound to the target vehicle can be converted into geospatial coordinates using this mapping model. That is, the pixel coordinates corresponding to the four vertices of each detection box are converted into geospatial coordinates.

[0099] Then, outlier analysis can be performed on all detection boxes to identify and remove abnormal detection boxes, thus obtaining the target detection boxes. In practical applications, slight shaking of the detection boxes of the camera or vehicle may cause a shift in geospatial coordinates, which may be mistakenly interpreted as vehicle movement, and the coordinate error can be significant, leading to additional computational resources being consumed to calculate its speed. Therefore, to reduce the waste of computational resources, in this embodiment, after obtaining the geospatial coordinates of all detection boxes of the target vehicle, outlier analysis can be performed on each detection box first. After removing abnormal detection boxes, the remaining target detection boxes are used to determine whether the target vehicle is in a moving state, and then its speed is measured.

[0100] In practice, firstly, a coordinate system is established with the top-left corner of the image to be processed as the origin (0,0), and the positive X-axis pointing to the right and the positive Y-axis pointing downwards. In this coordinate system, the X-value increases with horizontal rightward movement, and the Y-value increases with vertical downward movement. After video spatialization, the larger the Y-value of a pixel in the image to be processed, and the closer its X-value is to the center point of the X-axis, the higher the accuracy of converting video pixel coordinates to geospatial coordinates. Therefore, the straight-line distance between the pixel coordinates corresponding to the four vertices of each detection box and the pixel coordinates of the bottom point in the image can be calculated to obtain the target pixel closest to the bottom point. The bottom point in the image refers to a preset reference point slightly below the vertical and horizontal midline of the image. Next, all detection boxes of the target vehicle are traversed to convert the coordinates of the target pixels corresponding to all detection boxes into geospatial coordinates. Then, cluster analysis can be performed on the geospatial coordinates of all target pixels. Coordinates that deviate significantly from the mainstream clusters can be removed as outliers, and the corresponding detection boxes can be removed.

[0101] After obtaining the target detection box, the distance between the center point of the first frame detection box and the center point of the current frame detection box can be calculated. If this distance is greater than a center point distance threshold, it can be determined that the target vehicle is moving. If the distance is less than the center point distance threshold, it can be determined that the target vehicle is not moving. In this way, this solution can determine whether the target vehicle is moving, reducing the waste of computing resources, and can also support the identification of abnormal parking phenomena, such as abnormal parking on highways, thereby expanding practical application scenarios.

[0102] Furthermore, in practice, the tracking identifier for the same vehicle may change. For example, the tracking identifier for the same vehicle might be 2, but in some frames it might become 4 or another number. Therefore, when implementing speed measurement, taking the target vehicle as an example, we can first filter based on the number of tracking identifiers for the target vehicle in all frames. When the number is less than the threshold, the target vehicle can be ignored, and its speed will not be measured.

[0103] Finally, the speed measurement result of the target vehicle can be determined based on the geospatial coordinates of the first frame detection box, the geospatial coordinates of the last frame detection box within the target detection box, and the frame time interval between the first and last detection boxes. In other words, after obtaining the geospatial coordinates of the first and last detection boxes, the straight-line distance can be calculated using these two coordinates, and the quotient of this straight-line distance and the frame time interval can be used as the speed of the target vehicle, i.e., the speed measurement result.

[0104] It should be noted that in practical applications, there may be situations where a vehicle circles around and returns to its starting point. In this case, the speed calculated using the first and last detection frames will be 0. Therefore, to improve the effectiveness of the speed measurement results, taking the target vehicle as an example, when the number of detection frames corresponding to the target vehicle exceeds a certain amount or the total frame time interval exceeds a certain duration, some older detection frames can be removed, and the latest detection frames can be retained, thus ensuring that the corresponding detection frames are always up-to-date.

[0105] Combination Figure 2 As shown, the video speed measurement scheme provided in this application embodiment can not only measure the speed of a moving vehicle, but also identify abnormal parking behavior on the road.

[0106] Based on the above steps S101-S104, in this embodiment, firstly, a video dataset is acquired, comprising multiple videos, each containing data on multiple vehicles. The video dataset undergoes video spatialization, which is used to establish a mapping model between pixel coordinates and geographic spatial coordinates. Next, target detection is performed on the video dataset to obtain detection results with vehicles as the detection targets. Target tracking is then performed on the detection results to obtain tracking identifiers. Subsequently, the mapping model is used to measure the speed of multiple vehicles based on the detection results and tracking identifiers, yielding the speed measurement results. It is evident that this solution eliminates the need for additional dedicated hardware. By utilizing existing video resources and target detection and tracking algorithms, vehicle detection and tracking can be achieved, significantly reducing deployment costs and maintenance complexity. Furthermore, since the video dataset undergoes video spatialization, the video data is associated with geographic spatial information. Therefore, based on target detection and tracking, and combined with spatialized data attributes, the vehicle's trajectory can be transformed into positional changes at the geographic coordinate level, thereby enabling accurate calculation of vehicle speed.

[0107] Based on the video speed measurement method provided in the above embodiments, this application can also provide a video speed measurement device. The video speed measurement device will now be described in conjunction with the embodiments and accompanying drawings.

[0108] Figure 3 This is a schematic diagram of a video speed measurement device provided in an embodiment of this application. (Combined with...) Figure 3 As shown, the video speed measuring device 300 provided in this embodiment includes:

[0109] The data acquisition module 301 is used to acquire a video dataset, which includes multiple videos. The data content of each video includes multiple vehicles. The video dataset is spatialized, and the video spatialization is used to establish a mapping model between pixel coordinates and geographic spatial coordinates.

[0110] The target detection module 302 is used to perform target detection on the video dataset to obtain detection results with vehicles as the detection targets;

[0111] The target tracking module 303 is used to track the target based on the detection result and obtain the tracking identifier of the detection result;

[0112] The speed measurement module 304 is used to measure the speed of the multiple vehicles based on the detection results and the tracking identifier using the mapping model, and to obtain the speed measurement result.

[0113] Optionally, the target detection module 302 includes:

[0114] The video processing module is used to extract the video dataset frame by frame to obtain the image to be processed;

[0115] The target detection submodule is used to perform target detection on the image to be processed based on the target detection model to obtain initial detection results. The initial detection results include the type, detection box, and confidence score of each of the multiple detected targets.

[0116] The detection result determination module is used to determine the detection result from the initial detection result based on the type of each of the plurality of detection targets, wherein the type of the detection target corresponding to the detection result is a vehicle.

[0117] Optionally, the target detection submodule is specifically used for:

[0118] Based on the target detection model, target detection is performed on the image to be processed to obtain the type of the same target, multiple initial detection boxes, and the confidence of the multiple initial detection boxes;

[0119] The multiple initial detection boxes are deduplicated to obtain the detection boxes for the same detection target and the confidence scores of the detection boxes for the same detection target.

[0120] Optionally, the detection result includes the confidence level of the detection frame for each of the plurality of vehicles;

[0121] For the first vehicle among the plurality of vehicles, the target tracking module 303 is specifically used for:

[0122] Based on the confidence level of the detection box for each vehicle, a plurality of first detection boxes and a plurality of second detection boxes are determined, wherein the confidence level of the plurality of first detection boxes is higher than a confidence threshold and the confidence level of the plurality of second detection boxes is lower than a confidence threshold.

[0123] Based on the trajectory of the first vehicle in the previous frames, the trajectory of the first vehicle in the current frame is predicted to obtain the predicted bounding box of the first vehicle.

[0124] Determine the first intersection-union ratio (IUU) between the predicted bounding box of the first vehicle and the plurality of first detection boxes;

[0125] Using the Hungarian algorithm, based on the first intersection-union ratio, the predicted bounding box of the first vehicle is matched with the plurality of first detection boxes to obtain a first matching result;

[0126] If the first matching result indicates that the predicted bounding box of the first vehicle successfully matches the first target detection box in the plurality of first detection boxes, the target trajectory box of the first vehicle is determined based on the predicted bounding box of the first vehicle and the first target detection box, and the tracking identifier of the first vehicle is generated based on the target trajectory box of the first vehicle.

[0127] Optionally, the target tracking module 303 is further configured to:

[0128] Determine the second intersection-union ratio between the predicted bounding box of the second vehicle (excluding the first vehicle) and the plurality of second detection boxes;

[0129] Using the Hungarian algorithm, based on the second intersection-union ratio, the predicted bounding box of the second vehicle is matched with the plurality of second detection boxes to obtain a second matching result;

[0130] If the second matching result indicates that the predicted bounding box of the second vehicle successfully matches the second target detection box in the plurality of second detection boxes, the target trajectory box of the second vehicle is determined based on the predicted bounding box of the second vehicle and the second target detection box, and the tracking identifier of the second vehicle is generated based on the target trajectory box of the second vehicle.

[0131] Optionally, the target tracking module 303 is further configured to:

[0132] Using the Hungarian algorithm, based on the first intersection-union ratio, the predicted bounding box of the third vehicle is matched with the other first detection boxes in the plurality of first detection boxes except the first target detection box, to obtain a third matching result. The third vehicle is the vehicle among the plurality of vehicles other than the first vehicle and the second vehicle.

[0133] In the case where the third matching result indicates that the predicted bounding box of the third vehicle is successfully matched with the third target detection box in the other first detection boxes, the target trajectory box of the third vehicle is determined based on the predicted bounding box of the third vehicle and the third target detection box, and the tracking identifier of the third vehicle is generated based on the target trajectory box of the third vehicle.

[0134] If the third matching result indicates that the predicted bounding box of the third vehicle fails to match any of the other first detection boxes, the predicted bounding box of the third vehicle is deleted.

[0135] If the third matching result indicates that all other first detection boxes fail to match the third vehicle, and the difference between the confidence level of the other first detection boxes and the confidence level threshold is less than the difference threshold, then the predicted box of the third vehicle is deleted.

[0136] Optionally, for a target vehicle among the plurality of vehicles, the speed measuring module 304 is specifically used for:

[0137] Based on the tracking identifier of the target vehicle, the detection box corresponding to the tracking identifier of the target vehicle is bound to the target vehicle;

[0138] Based on the mapping model, the pixel coordinates of all detection boxes bound to the target vehicle are converted into geospatial coordinates;

[0139] Anomaly analysis is performed on all the detection boxes to identify and remove abnormal detection boxes, thus obtaining the target detection boxes;

[0140] If the distance between the center point of the first frame detection box and the center point of the current frame detection box is greater than the center point distance threshold, it is determined that the target vehicle is in a moving state.

[0141] The speed measurement result of the target vehicle is determined based on the geospatial coordinates of the first frame detection box, the geospatial coordinates of the last frame detection box in the target detection box, and the frame time interval corresponding to the first frame detection box and the last frame detection box.

[0142] Furthermore, embodiments of this application also provide an electronic device, including: a processor, a memory, and a system bus;

[0143] The processor and the memory are connected via the system bus;

[0144] The memory is used to store one or more programs, the one or more programs including instructions that, when executed by the processor, cause the processor to perform any of the implementation steps of the video speed measurement method described above.

[0145] Furthermore, embodiments of this application also provide a computer-readable storage medium storing instructions that, when executed on an electronic device, enable any of the above-described video speed measurement method steps to be implemented.

[0146] As can be seen from the above description of the embodiments, those skilled in the art can clearly understand that all or part of the steps in the methods of the above embodiments can be implemented by means of software plus necessary general-purpose hardware platforms. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product can be stored in a storage medium, such as ROM / RAM, magnetic disk, optical disk, etc., and includes several instructions to cause a computer device (which may be a personal computer, a server, or a network communication device such as a media gateway, etc.) to execute the methods described in various embodiments or some parts of the embodiments of this application. It should be noted that the various embodiments in this specification are described in a progressive manner, and each embodiment focuses on describing the differences from other embodiments. The same or similar parts between the various embodiments can be referred to mutually.

[0147] The system disclosed in the embodiments is described in a relatively simple manner because it corresponds to the method disclosed in the embodiments. For relevant details, please refer to the method section.

[0148] It should also be noted that, in this document, relational terms such as "first" and "second" are used only to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.

[0149] The above description of the disclosed embodiments enables those skilled in the art to make or use this application. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of this application. Therefore, this application is not to be limited to the embodiments shown herein, but is to be accorded the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. A video speed measurement method, characterized in that, include: A video dataset is obtained, which includes multiple videos. The data content of each video includes multiple vehicles. The video dataset is spatialized and used to establish a mapping model between pixel coordinates and geospatial coordinates. Target detection is performed on the video dataset to obtain detection results with vehicles as the detection targets; The detection results are used for target tracking to obtain the tracking identifier of the detection results; Using the mapping model, the speed of the multiple vehicles is measured based on the detection results and the tracking identifier to obtain the speed measurement results.

2. The video speed measurement method according to claim 1, characterized in that, The step of performing object detection on the video dataset to obtain detection results with vehicles as the detection targets includes: The video dataset is extracted frame by frame to obtain the image to be processed; The target detection model is used to perform target detection on the image to be processed to obtain initial detection results. The initial detection results include the type of each of the multiple detected targets, the detection box, and the confidence level of the detection box. Based on the types of the multiple detection targets, the detection result is determined from the initial detection result, and the type of the detection target corresponding to the detection result is a vehicle.

3. The video speed measurement method according to claim 2, characterized in that, The step of performing target detection on the image to be processed based on the target detection model to obtain initial detection results includes: Based on the target detection model, target detection is performed on the image to be processed to obtain the type of the same target, multiple initial detection boxes, and the confidence of the multiple initial detection boxes; The multiple initial detection boxes are deduplicated to obtain the detection boxes for the same detection target and the confidence scores of the detection boxes for the same detection target.

4. The video speed measurement method according to claim 1, characterized in that, The detection results include the confidence level of the detection frame for each of the plurality of vehicles; For the first vehicle among the plurality of vehicles, the step of performing target tracking on the detection result to obtain a tracking identifier for the detection result includes: Based on the confidence level of the detection box for each vehicle, a plurality of first detection boxes and a plurality of second detection boxes are determined, wherein the confidence level of the plurality of first detection boxes is higher than a confidence threshold and the confidence level of the plurality of second detection boxes is lower than a confidence threshold. Based on the trajectory of the first vehicle in the previous frames, the trajectory of the first vehicle in the current frame is predicted to obtain the predicted bounding box of the first vehicle. Determine the first intersection-union ratio (IUU) between the predicted bounding box of the first vehicle and the plurality of first detection boxes; Using the Hungarian algorithm, based on the first intersection-union ratio, the predicted bounding box of the first vehicle is matched with the plurality of first detection boxes to obtain a first matching result; If the first matching result indicates that the predicted bounding box of the first vehicle successfully matches the first target detection box in the plurality of first detection boxes, the target trajectory box of the first vehicle is determined based on the predicted bounding box of the first vehicle and the first target detection box, and the tracking identifier of the first vehicle is generated based on the target trajectory box of the first vehicle.

5. The video speed measurement method according to claim 4, characterized in that, After generating the tracking identifier of the first vehicle based on the target trajectory bounding box of the first vehicle, the method further includes: Determine the second intersection-union ratio between the predicted bounding box of the second vehicle (excluding the first vehicle) and the plurality of second detection boxes; Using the Hungarian algorithm, based on the second intersection-union ratio, the predicted bounding box of the second vehicle is matched with the plurality of second detection boxes to obtain a second matching result; If the second matching result indicates that the predicted bounding box of the second vehicle successfully matches the second target detection box in the plurality of second detection boxes, the target trajectory box of the second vehicle is determined based on the predicted bounding box of the second vehicle and the second target detection box, and the tracking identifier of the second vehicle is generated based on the target trajectory box of the second vehicle.

6. The video speed measurement method according to claim 5, characterized in that, After generating the tracking identifier of the second vehicle based on the target trajectory bounding box of the second vehicle, the method further includes: Using the Hungarian algorithm, based on the first intersection-union ratio, the predicted bounding box of the third vehicle is matched with the other first detection boxes in the plurality of first detection boxes except the first target detection box, to obtain a third matching result. The third vehicle is the vehicle among the plurality of vehicles other than the first vehicle and the second vehicle. In the case where the third matching result indicates that the predicted bounding box of the third vehicle is successfully matched with the third target detection box in the other first detection boxes, the target trajectory box of the third vehicle is determined based on the predicted bounding box of the third vehicle and the third target detection box, and the tracking identifier of the third vehicle is generated based on the target trajectory box of the third vehicle. If the third matching result indicates that the predicted bounding box of the third vehicle fails to match any of the other first detection boxes, the predicted bounding box of the third vehicle is deleted. If the third matching result indicates that all other first detection boxes fail to match the third vehicle, and the difference between the confidence level of the other first detection boxes and the confidence level threshold is less than the difference threshold, then the predicted box of the third vehicle is deleted.

7. The video speed measurement method according to any one of claims 1 to 6, characterized in that, For the target vehicle among the plurality of vehicles, the speed of the plurality of vehicles is measured using the mapping model based on the detection results and the tracking identifier to obtain the speed measurement results, including: Based on the tracking identifier of the target vehicle, the detection box corresponding to the tracking identifier of the target vehicle is bound to the target vehicle; Based on the mapping model, the pixel coordinates of all detection boxes bound to the target vehicle are converted into geospatial coordinates; Anomaly analysis is performed on all the detection boxes to identify and remove abnormal detection boxes, thus obtaining the target detection boxes; If the distance between the center point of the first frame detection box and the center point of the current frame detection box is greater than the center point distance threshold, it is determined that the target vehicle is in a moving state. The speed measurement result of the target vehicle is determined based on the geospatial coordinates of the first frame detection box, the geospatial coordinates of the last frame detection box in the target detection box, and the frame time interval corresponding to the first frame detection box and the last frame detection box.

8. A video speed measurement device, characterized in that, include: The data acquisition module is used to acquire a video dataset, which includes multiple videos. The data content of each video includes multiple vehicles. The video dataset is spatialized, and the video spatialization is used to establish a mapping model between pixel coordinates and geospatial coordinates. The target detection module is used to perform target detection on the video dataset to obtain detection results with vehicles as the detection targets; The target tracking module is used to track the target based on the detection result and obtain the tracking identifier of the detection result; The speed measurement module is used to measure the speed of the multiple vehicles based on the detection results and the tracking identifier using the mapping model, and to obtain the speed measurement results.

9. An electronic device, characterized in that, The device includes: a processor, a memory, and a system bus; The processor and the memory are connected via the system bus; The memory is used to store a program, the program including instructions that, when executed by the processor, cause the processor to perform the steps of the video speed measurement method according to any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program that, when executed by a processor, implements the steps of the video speed measurement method as described in any one of claims 1 to 7.

Images