Roadside perception camera offline calibration method, device and electronic equipment

Abstract

The application relates to the technical field of perception camera calibration, and provides a roadside perception camera offline calibration method, device and electronic equipment. The method comprises the following steps: performing quality detection on video data collected by a roadside perception camera, determining target video data for roadside perception camera calibration, and distributing the target video data to at least one multi-access edge computing (MEC), wherein the at least one MEC is used for performing calibration of the roadside perception camera, and the at least one MEC is selected from a plurality of candidate MECs according to an algorithm power index and a load balancing strategy of the plurality of candidate MECs.

Description

Technical Field

[0001] This application relates to the field of sensing camera calibration technology, and in particular to an offline calibration method, device and electronic equipment for roadside sensing cameras. Background Technology

[0002] In the calibration of roadside perception cameras, related technologies concentrate all video data on a few multi-access edge computing (MEC) devices for processing. This results in excessive computational pressure on these MEC devices, while other idle MEC devices cannot be effectively utilized, thus reducing the overall efficiency of roadside perception camera calibration.

[0003] Application content This application provides a method, apparatus, and electronic device for offline calibration of roadside sensing cameras, in order to solve the problem of how to improve the efficiency of offline calibration of roadside sensing cameras.

[0004] Firstly, this application provides an offline calibration method for roadside sensing cameras, including: The video data collected by the roadside sensing camera is subjected to quality inspection to determine the target video data for the calibration of the roadside sensing camera. The target video data is allocated to at least one multi-access edge computing (MEC), and at least one MEC is used to perform the calibration of roadside perception cameras; At least one MEC is selected from multiple candidate MECs based on their computing power metrics and load balancing strategies.

[0005] In some embodiments, the load balancing strategy includes: prioritizing the MEC with higher idle time among the candidate MECs, and prioritizing the MEC closer to the roadside sensing camera when the idle time is the same.

[0006] In some embodiments, computing power metrics include at least one of graphics processing unit (GPU) utilization, video memory usage, and task queue length.

[0007] In some embodiments, quality inspection is performed on video data acquired by roadside sensing cameras, including: The quality of video data collected by roadside perception cameras is tested according to the detection indicators, which include at least one of the following: video frame rate, frame interval, time synchronization, resolution, and network fluctuation.

[0008] In some embodiments, the calibration process for roadside sensing cameras includes: The features of each vehicle in the target video data are matched with the features of true vehicles in the true vehicle feature benchmark library to determine the target vehicles in the target video data that match the true vehicle features. The pixel coordinates corresponding to the target vehicle are correlated across frames to form the pixel trajectory of the target vehicle; The pixel coordinates corresponding to each trajectory point on the pixel trajectory of the target vehicle are matched with the world coordinates corresponding to each trajectory point on the pixel trajectory of the target vehicle to obtain the matrix parameters for the calibration of the roadside perception camera.

[0009] In some embodiments, the features of each vehicle in the target video data are matched with the features of ground truth vehicles in the ground truth vehicle feature benchmark library to determine the target vehicles in the target video data that match the features of ground truth vehicles, including: Based on the features of ground truth vehicle images from multiple angles, a benchmark library of ground truth vehicle features is constructed. The features of each vehicle in the target video data are detected to obtain the feature vector of each vehicle. Calculate the similarity between the feature vector of each vehicle and the features of true vehicles in the true vehicle feature benchmark library; Vehicles that meet the similarity criteria among all vehicles are identified as target vehicles.

[0010] In some embodiments, at least one MEC is also used to upload target video data and / or matrix parameters calibrated by roadside perception cameras to the platform storage space.

[0011] In some embodiments, the method further includes: Send the matrix parameters calibrated by the roadside perception camera to the MEC corresponding to the roadside perception camera.

[0012] Secondly, this application also provides an offline calibration device for a roadside sensing camera, comprising: The detection unit is used to perform quality inspection on the video data collected by the roadside perception camera and determine the target video data for the calibration of the roadside perception camera. The allocation unit is used to allocate target video data to at least one MEC, and the at least one MEC is used to perform calibration of the roadside perception camera.

[0013] Thirdly, this application also provides an electronic device, including a processor and a memory storing a computer program, wherein the processor executes the program to implement the offline calibration method for roadside sensing cameras described in the first aspect.

[0014] Fourthly, this application also provides a processor-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the offline calibration method for roadside sensing cameras described in the first aspect.

[0015] Fifthly, this application also provides a computer program product, including a computer program that, when executed by a processor, implements the offline calibration method for roadside sensing cameras described in the first aspect.

[0016] The roadside sensing camera offline calibration method, apparatus, and electronic equipment provided in this application select candidate MECs from multiple candidate MECs to perform roadside sensing camera calibration through computing power indicators and load balancing strategies. This distributes the roadside sensing camera calibration process, which was originally concentrated on a few MECs, to multiple idle MECs for parallel processing. Without adding any new hardware, the calibration time of roadside sensing cameras is shortened, and the efficiency of offline roadside sensing camera calibration is improved. Attached Figure Description

[0017] To more clearly illustrate the technical solutions in this application or related technologies, the accompanying drawings used in the description of the embodiments or related technologies will be briefly introduced below. Obviously, the accompanying drawings described below are some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0018] Figure 1 This is a flowchart illustrating the offline calibration method for roadside sensing cameras provided in an embodiment of this application.

[0019] Figure 2 This is a flowchart illustrating the offline calibration process of a roadside sensing camera provided in an embodiment of this application.

[0020] Figure 3 This is a schematic diagram of the structure of the offline calibration device for roadside sensing cameras provided in an embodiment of this application.

[0021] Figure 4 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application. Detailed Implementation

[0022] In the embodiments of this application, the term "and / or" describes the relationship between associated objects, indicating that three relationships can exist. For example, A and / or B can represent three cases: A alone, A and B simultaneously, and B alone. The character " / " generally indicates that the preceding and following associated objects have an "or" relationship.

[0023] In the embodiments of this application, the term "multiple" refers to two or more, and other quantifiers are similar.

[0024] In the embodiments of this application, the terms "first," "second," etc., are used to distinguish similar objects, and not to describe a specific order or sequence. It should be understood that such terms can be used interchangeably where appropriate so that embodiments of this application can be implemented in orders other than those illustrated or described herein, and the objects distinguished by "first" and "second" are generally of the same class, and the number of objects is not limited; for example, the first object can be one or more.

[0025] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of this application, and not all of the embodiments. Based on the embodiments of this application, all other embodiments obtained by those of ordinary skill in the art without creative effort are within the scope of protection of this application.

[0026] To facilitate a clearer understanding of the technical solutions of the various embodiments of this application, some technical content related to the various embodiments of this application will be introduced first.

[0027] As a core component of intelligent road construction, roadside perception, through the deployment of diverse sensors such as LiDAR, cameras, and millimeter-wave radar, enables real-time and accurate collection of information on the road environment, traffic participants, and road conditions. It forms the foundation for building a vehicle-road cooperative perception system and supporting the efficient operation of the vehicle-road-cloud integrated system. However, in actual project delivery, the calibration process for roadside perception cameras is cumbersome. On one hand, the calibration process involves multiple steps, including video acquisition, model calculation, and error verification, relying on manual operation, making it complex and time-consuming. On the other hand, the calibration data processing lacks unified management; files are stored haphazardly, version tracking is difficult, and this easily leads to problems such as parameter mismatch and data loss. These issues not only extend the project delivery cycle but also result in high rework rates. Furthermore, calibration errors or file anomalies can cause algorithmic errors and distorted perception data, becoming a significant technical challenge that urgently needs to be addressed for the large-scale implementation of intelligent road construction.

[0028] Existing roadside perception calibration solutions from roadside vendors typically employ a method of directly acquiring camera video streams and centrally transmitting them to at least a few Multi-Access Edge Computing (MEC) devices to run calibration AI detection. Because all video data is processed on a small number of MEC devices, the computational load on these devices becomes excessive, while other idle MEC devices within the project are not effectively utilized. This not only reduces the overall efficiency of calibration but may also lead to performance bottlenecks under high load, affecting the real-time performance and accuracy of calibration, making it difficult to support the parallel calibration requirements of large-scale locations.

[0029] In existing AI-based vehicle detection calibration processes, the identification (ID) of ground truth vehicles typically requires manual review of videos, manual matching of ground truth vehicle features, and selection of the corresponding ID. This process is not only time-consuming and labor-intensive but also highly dependent on human experience. For example, relying on experience to judge when a ground truth vehicle will appear at an intersection increases workload and is prone to errors due to human factors. Furthermore, the lack of standardization and automation makes it difficult to guarantee the consistency and accuracy of each calibration, directly affecting the reliability of the calibration results. In the video acquisition stage, there is a lack of pre-processing checks on key indicators such as frame rate, frame interval, timing, resolution, and network fluctuations. Whether the acquired video data meets the input requirements of the AI ​​detection algorithm is often only revealed during the detection phase. If it does not meet the standards, re-acquisition is required, resulting in significant rework.

[0030] Therefore, existing roadside sensing camera calibration schemes have significant shortcomings in calibration efficiency, making it difficult to meet the requirements for high-efficiency calibration. Based on this, this application provides an offline calibration method, apparatus, and electronic device for roadside sensing cameras. By using computing power metrics and a load balancing strategy, candidate MECs are selected from multiple candidate MECs to perform roadside sensing camera calibration. This distributes the calibration process, which was originally concentrated on a few MECs, across multiple idle MECs for parallel processing. Without adding any new hardware, the calibration time for roadside sensing cameras is shortened, and the efficiency of offline roadside sensing camera calibration is improved.

[0031] Figure 1 This is a flowchart illustrating the offline calibration method for roadside sensing cameras provided in an embodiment of this application, as shown below. Figure 1 As shown, the method includes the following steps 101 and 102.

[0032] Step 101: Perform quality inspection on the video data collected by the roadside sensing camera to determine the target video data for roadside sensing camera calibration.

[0033] Step 102: Assign the target video data to at least one multi-access edge computing (MEC), where at least one MEC is used to perform calibration of the roadside perception camera.

[0034] At least one MEC is selected from multiple candidate MECs based on their computing power metrics and load balancing strategies.

[0035] Specifically, the method provided in this application embodiment can be executed by a centralized platform or cloud control system, which can manage multiple roadside units (RSUs) or MEC nodes deployed at different intersections or road sections.

[0036] Roadside sensing cameras can be camera devices deployed on both sides of a road to sense road traffic participants, such as bullet cameras, PTZ cameras, fisheye cameras, etc., without specific limitations. Before being put into use or after the location changes during use, roadside sensing cameras usually need to be calibrated to establish the transformation relationship between the image coordinate system and the world coordinate system.

[0037] The target video data refers to video data that meets quality inspection standards and can be used in the roadside sensing camera calibration process. During the video acquisition process of roadside sensing cameras, the data quality of the acquired video can be checked through inspection indicators to ensure that the acquired video data meets the data quality requirements of the roadside sensing camera calibration process. Different processing measures are taken for video data that meets or does not meet the quality standards.

[0038] The platform's acquisition tools pre-check the link and device status before acquiring video data. During video acquisition, relevant quality inspection indicators are statistically analyzed frame by frame using a sliding window, for example, a fixed sliding window of 1 second or 2 frames. For video data that fails the quality inspection, an alarm is triggered in real time, and the data segment is marked as invalid, prompting a re-acquisition. For video segments that all meet the quality inspection standards, they are identified as target video data and proceed to subsequent storage and uploading stages. By performing data quality checks before and during acquisition, the number of rework attempts and error rates during the calibration process can be effectively reduced, improving overall work efficiency and accuracy.

[0039] After determining the target video data, it can be allocated to at least one multi-access edge computing (MEC) for calibrating roadside sensing cameras. The MEC can be edge computing devices deployed at intersections, road sections, etc., and possesses a certain level of AI inference computing power. The platform's intelligent scheduling function can distribute the collected video data to MECs at various locations, fully utilizing idle MEC resources and leveraging their distributed computing capabilities to achieve parallel processing of the roadside sensing camera calibration process. Before allocation, the target video data can be segmented. For example, the target video data for roadside sensing camera calibration can be divided into multiple independent calibration subtasks based on dimensions such as location, camera, and time period. Each calibration subtask can be executed independently by a specific MEC, and these subtasks can be processed in parallel. The rules for subtask segmentation can be determined based on parameters such as video duration, number of MECs, and number of cameras, and are not specifically limited here.

[0040] The platform can perform computing power probing on multiple candidate MECs, collect computing power metrics of each candidate MEC in real time, and maintain a global computing power resource table accordingly. Then, based on this global computing power resource table, a load balancing strategy is used to distribute the calibrated subtasks obtained from the partitioning to the selected MECs. The computing power metrics can be indicators that reflect the current computing power and resource utilization of the candidate MECs, and the load balancing strategy can be a strategy that can reasonably distribute the task load among the candidate MECs.

[0041] The offline calibration method for roadside sensing cameras provided in this application selects candidate MECs from multiple candidate MECs for roadside sensing camera calibration through computing power indicators and load balancing strategies. This distributes the roadside sensing camera calibration process, which was originally concentrated on a few MECs, to multiple idle MECs for parallel processing. Without adding any hardware, this shortens the calibration time for roadside sensing cameras and improves the efficiency of offline roadside sensing camera calibration.

[0042] In some embodiments, the load balancing strategy includes: prioritizing the MEC with higher idle time among the candidate MECs, and prioritizing the MEC closer to the roadside sensing camera when the idle time is the same.

[0043] Specifically, idleness can be a parameter used to quantify the currently available computing resources of candidate MECs. Idleness can be calculated based on the computing power indicators of candidate MECs. For example, the resource usage can be normalized, weighted, and summed, with the complement value taken as the idleness. Alternatively, a hierarchical approach can be used, such as dividing candidate MECs into several levels like high idle, medium idle, and low idle. The specific calculation method for idleness is not specifically limited here. If a candidate MEC currently has no executing calibration subtasks and low resource usage, it indicates a high idleness. If a candidate MEC is executing real-time sensing tasks or processing many calibration subtasks, it indicates a low idleness. MECs with higher idleness are prioritized. When screening candidate MECs, idleness is used as the first priority selection criterion. Candidate MECs with high idleness are included in the selection scope first, thereby allocating calibration subtasks to idle MEC resources as much as possible, avoiding situations where some MECs are overloaded while others are idle.

[0044] When multiple candidate MECs have similar or identical idle levels—for example, if the difference in idle levels among multiple candidate MECs is less than a preset difference, or if multiple candidate MECs are at the same idle level—the MEC closer to the roadside sensing camera can be prioritized. Distance refers to network distance, such as network hop count, transmission path length, network transmission latency, available bandwidth, or other indicators reflecting network proximity. Since the target video data is typically large, assigning the calibration subtask to the nearest MEC can shorten the time required for video data to be transmitted from the storage location to the MEC performing the calibration, reducing the risk of packet loss and congestion during transmission.

[0045] The platform can traverse multiple candidate MECs based on a global computing resource table maintained by computing power metrics. First, it sorts them by idle time from highest to lowest. If idle times are the same, it sorts them by distance from the roadside perception camera from closest to farthest. Then, according to the sorting result, the calibration subtasks are sequentially distributed to the GPUs of the top-ranked candidate MECs for execution. When the number of calibration subtasks received by a candidate MEC reaches its capacity limit, it can continue to allocate them to the next-ranked candidate MECs until all calibration subtasks have been allocated.

[0046] In some embodiments, the platform can also reschedule failed calibration subtasks. For example, when a calibration subtask times out or fails to execute on its assigned MEC, it can be reassigned to another idle MEC with the highest current idle level and closest proximity, following the same load balancing strategy, thereby ensuring the overall completion rate of calibration tasks. Furthermore, the execution of the load balancing strategy can be periodic or event-triggered, such as triggering an allocation decision whenever a new calibration subtask is generated or the global computing power resource table is updated.

[0047] The offline calibration method for roadside sensing cameras provided in this application uses idle time as the first priority and distance as the second priority for load balancing. This method can utilize the computing resources of idle MECs and avoid a single MEC handling too many calibration tasks. By allocating data based on network proximity, the transmission overhead and latency of video data are reduced, the completion time of calibration tasks is shortened, and the calibration efficiency of roadside sensing cameras is improved.

[0048] In some embodiments, computing power metrics include at least one of graphics processing unit (GPU) utilization, video memory usage, and task queue length.

[0049] Specifically, GPU utilization can be defined as the percentage of time that the GPU computing units in a candidate MEC are in an active state during the statistical period, reflecting the GPU's workload. GPU utilization can be collected through driver interfaces, monitoring tools, or management agents provided by the candidate MEC's ​​operating system, or through other common methods of obtaining GPU utilization, which are not specifically limited here. For example, if the GPU utilization of a candidate MEC is 20%, it means that the MEC's ​​GPU has sufficient computing power to perform the roadside camera calibration process, and new calibration subtasks can be assigned to this MEC. If the GPU utilization of a candidate MEC is 95%, it means that the MEC's ​​GPU is close to full load, and it is not suitable to assign new calibration subtasks to this MEC.

[0050] The video memory usage status can refer to the GPU memory usage status of candidate MECs, which can include one or more of the following: occupied video memory capacity, remaining available video memory capacity, and video memory utilization rate. Since roadside cameras need to load detection models and video data into video memory during calibration, if the remaining video memory of a candidate MEC is insufficient, it cannot accommodate the model and data required for the calibration subtask. In this case, even if its GPU utilization is low, the calibration subtask may not execute correctly. Therefore, when selecting an MEC, the platform can compare the remaining available video memory with the estimated video memory requirements of the calibration subtask and select the candidate MEC whose remaining video memory meets the requirements.

[0051] The task queue length represents the number of tasks currently queued for execution on a candidate MEC. This can be the number of calibrated subtasks assigned to the MEC but not yet executed, or the number of other computational tasks being performed by the MEC. The task queue length reflects the load on the candidate MEC over a future period. Candidate MECs with longer task queues will experience heavier loads later on, even if current GPU utilization is low, thus their selection priority can be lowered.

[0052] The platform can obtain the aforementioned computing power metrics through computing power probing. For example, the platform can collect real-time computing power metrics such as GPU utilization, memory usage, and current task queue length for each MEC location, and maintain a global computing power resource table accordingly. The computing power metrics can be obtained by agents deployed on each MEC periodically reporting to the platform, by the platform actively polling each MEC, or by a combination of both; no specific limitation is made here. The global computing power resource table can record information such as the identifier of each candidate MEC, its location, the maximum and minimum values ​​of various computing power metrics, and the update time, as well as other information related to the computing power metrics; no specific limitation is made here. The computing power metrics and the maintained global computing power resource table can provide real-time and accurate decision-making basis for load balancing strategies.

[0053] The offline calibration method for roadside sensing cameras provided in this application selects MECs based on GPU utilization, memory usage, and task queue length. This method can accurately determine the real-time computing power of MECs, avoiding calibration failures or timeouts caused by MECs with insufficient resources being assigned to them. This improves the success rate and efficiency of calibration subtasks.

[0054] In some embodiments, quality inspection is performed on video data acquired by roadside sensing cameras, including: The quality of video data collected by roadside perception cameras is tested according to the detection indicators, which include at least one of the following: video frame rate, frame interval, time synchronization, resolution, and network fluctuation.

[0055] Specifically, video frame rate represents the effective frame rate per unit of time. A frame rate that is too low can cause fast-moving targets to be missed. When calculating the true smooth frame rate of a video, static or frozen frames can be removed first. Static frames can be identified by comparing the pixel differences between consecutive frames; for example, if the pixel difference between two adjacent frames is less than a preset difference threshold, the subsequent frame is considered a static frame. The smooth frame rate of the video data can be set to at least 25 frames per second, the proportion of static frames to be less than 5%, and the frame interval fluctuation to be no more than ±20% of one frame period. Only under these conditions can the video data be retained.

[0056] Frame interval can represent the stability of the time interval between adjacent frames. If the fluctuation is too large, it indicates that there is a frame drop phenomenon. For example, if the frame interval fluctuation is greater than 20% or less than 20% of one frame period, it can be said that the stability of the time interval between adjacent frames of the video data is poor.

[0057] Time synchronization can represent the consistency between the video timestamp and the platform or Real Time Kinematic (RTK) timestamp. It can be represented by the Network Time Protocol (NTP), and the time synchronization deviation between the video timestamp and the RTK timestamp can be set to no more than 10 milliseconds.

[0058] Resolution indicates the level of detail in an image. A resolution of at least 1920×1080 can be set, meaning the image resolution must be at least 1080P. Simultaneously, image sharpness can be calculated using Laplacian variance; a higher Laplacian variance value indicates a sharper image, eliminating blurry or out-of-focus scenes. Alternatively, while meeting the resolution requirement, sharpness can be set to be at least 60% to 70% of the average of acceptable samples. The specific percentage can be adjusted based on actual conditions and is not specifically limited here.

[0059] Network fluctuations can be analyzed by calculating the packet loss rate using the expected number of packets received and the actual number of packets received, and then smoothing out the fluctuations in packet arrival times using a standard recursive algorithm. A packet loss rate of no more than 0.5% and network fluctuations of no more than 30 milliseconds can be considered acceptable for transmission quality.

[0060] Before data acquisition, the platform's data collection tool performs a pre-check of the link and device status. During acquisition, it statistically analyzes the aforementioned indicators frame by frame using a sliding window. If any hard criterion fails to meet the standard or the overall quality score falls below the threshold, an alarm is triggered in real time, and the data segment is marked as invalid, prompting a re-acquisition. Only video segments that meet all the standards can be used as target video data for subsequent storage and uploading. The overall quality score can be a weighted fusion of the scores of various indicators; the weights and thresholds are not specifically limited here.

[0061] The offline calibration method for roadside sensing cameras provided in this application performs quality checks on video data based on multiple detection indicators, including frame rate, frame interval, timing, resolution, and network fluctuation. This method can detect and remove unqualified data during the acquisition stage, ensuring that the target video data meets the quality requirements for roadside sensing camera calibration and effectively reducing the error rate during the roadside sensing camera calibration process.

[0062] In some embodiments, the calibration process for roadside sensing cameras includes: The features of each vehicle in the target video data are matched with the features of true vehicles in the true vehicle feature benchmark library to determine the target vehicles in the target video data that match the true vehicle features. The pixel coordinates corresponding to the target vehicle are correlated across frames to form the pixel trajectory of the target vehicle; The pixel coordinates corresponding to each trajectory point on the pixel trajectory of the target vehicle are matched with the world coordinates corresponding to each trajectory point on the pixel trajectory of the target vehicle to obtain the matrix parameters for the calibration of the roadside perception camera.

[0063] Specifically, the ground truth vehicle can be a vehicle equipped with high-precision positioning equipment that can provide ground truth world coordinates, such as a vehicle equipped with RTK equipment. The ground truth vehicle feature benchmark library can be a pre-built ground truth vehicle appearance feature library.

[0064] Kalman filtering can be used to predict the target vehicle's position in the next frame, and inter-frame matching can be performed using an association cost matrix. The cost can combine both position and appearance factors; for example, the cost can be expressed as a weighted sum of position and appearance costs. It can be represented as: in, The i-th row and j-th column of the matrix represents the cost between the i-th pixel trajectory and the j-th detection box in the current frame. The larger the value, the more mismatched the two are. Used to balance the relative importance of position and appearance items. The larger the value, the more important the location information. The smaller the size, the more important the appearance. The predicted bounding box is obtained by predicting the trajectory of the i-th pixel using Kalman filtering. This refers to the j-th detection box detected in the current image frame, i.e., the ground truth detection box. This represents the degree of overlap between the predicted bounding box and the detected bounding box. This represents the appearance feature vector of the i-th pixel trajectory. This represents the appearance feature vector corresponding to the j-th detection box. express and The feature similarity function. This indicates that they look the same. This indicates a difference in appearance.

[0065] The cost can be minimized using the Hungarian algorithm. By establishing a one-to-one association, the complete and continuous pixel trajectory of the ground truth vehicle under the field of view of the roadside camera can be obtained. h represents the pixel column of the pixel trajectory, and g represents the pixel row of the pixel trajectory. This represents the pixel coordinates of the vehicle in the image at time t.

[0066] World coordinates can be obtained using the World Geodetic System 1984 (WGS84) coordinate system. For example, the pixel coordinates of the ground truth vehicle in the image can be aligned with the world coordinates of its simultaneous RTK ground truth value by timestamp, establishing a set of corresponding point pairs of pixel coordinates and world coordinates. This is due to the timestamps of the pixel trajectory points. Timestamps of RTK truth values Since they belong to different clocks, they can first be aligned according to the time synchronization, and then the RTK sequence can be linearly interpolated to the image time to obtain corresponding point pairs at the same time. It can be represented as: in, , This represents the actual sampling times of two adjacent RTK positioning systems. express The vehicle position output by RTK at any given time. express The vehicle position output by RTK at any given time.

[0067] Interpolation can be performed at adjacent RTK sampling times. and Performed between, satisfying Latitude and longitude can be projected onto a local Cartesian coordinate system, such as the Gauss-Kruger projection or the Universal Transverse Mercator (UTM) projection, to obtain planar pixel coordinates. The projection method is not specifically limited here.

[0068] Since the ground can be approximated as a plane, planar pixel coordinates with world plane coordinates Satisfying homography, i.e. in, It is a 3×3 homography matrix, which can be used to describe the projection transformation between two planes. s is the scale factor.

[0069] The problem can be solved using Direct Linear Transformation (DLT). H is expanded row-wise into a 9-dimensional vector, and N sets of corresponding points are stacked to form a 2N×9 matrix A, which constitutes the equation. To improve numerical stability, pixel coordinates and world coordinates can be normalized first, for example, by translating to the centroid or scaling to change the average distance. Then, singular value decomposition (SVD) is performed on A, followed by inverse normalization to obtain H.

[0070] Calculate the reprojection error for each point by mapping the pixel to world coordinates via H and comparing the result with the RTK ground truth. It can be represented as: in, This is the homogeneous normalized projection function. It can be used for... Outlier pairs exceeding the threshold are removed, and H is recalculated. This process is repeated until the root-mean-square error (RMSE) of the difference converges to meet the calibration accuracy requirements. The resulting matrix parameter H represents the extrinsic parameter calibration result for this camera.

[0071] The offline calibration method for roadside sensing cameras provided in this application ensures the consistency between the timestamps of pixel trajectory points and the timestamps of RTK ground truth values ​​through timestamp alignment during the matching process. Normalized DLT solution and iterative outlier removal improve the accuracy of homography matrix determination, thereby enhancing the automation and accuracy of roadside sensing camera calibration.

[0072] In some embodiments, the features of each vehicle in the target video data are matched with the features of ground truth vehicles in the ground truth vehicle feature benchmark library to determine the target vehicles in the target video data that match the features of ground truth vehicles, including: Based on the features of ground truth vehicle images from multiple angles, a benchmark library of ground truth vehicle features is constructed. The features of each vehicle in the target video data are detected to obtain the feature vector of each vehicle. Calculate the similarity between the feature vector of each vehicle and the features of true vehicles in the true vehicle feature benchmark library; Vehicles that meet the similarity criteria among all vehicles are identified as target vehicles.

[0073] Specifically, images of the benchmark vehicle from multiple angles, including the front, rear, sides, and top, can be pre-collected. These images are then uploaded to the platform, and their appearance features are extracted to construct a benchmark feature library for the benchmark vehicle. Appearance features can include one or more of the following: vehicle color, model, license plate, texture, and depth feature vectors. As the benchmark vehicle moves within the camera's field of view, its orientation constantly changes. A single-angle benchmark image is insufficient for reliable matching under various orientations; multi-angle benchmark images can accommodate different vehicle orientations. Features can then be extracted from the benchmark vehicle to obtain its feature vector q.

[0074] AI object detection algorithms can be used to detect vehicle targets in target video data. The detection algorithm outputs candidate bounding boxes and class confidence scores for each frame. Non-maximum suppression (NMS) can be used to remove duplicates based on the intersection-over-union (IoU) ratio. IoU is the ratio of the intersection area to the union area of ​​two candidate boxes. When the IoU is greater than the suppression threshold... By suppressing candidate boxes with low confidence, vehicle targets with confidence levels not lower than the lower confidence threshold can be retained for subsequent matching. The lower confidence threshold can be adjusted according to the actual application scenario. Features can be extracted for each retained candidate vehicle to obtain the feature vector p of each candidate vehicle.

[0075] The cosine similarity between the feature vector p of each vehicle and the feature vector q of the ground truth vehicle in the benchmark library can be calculated, and the maximum response can be taken for multi-angle benchmarks. The maximum similarity among the features of a candidate vehicle and the features of the benchmarks at the front, rear, side, and top angles is taken as the matching score of the candidate target. Similarity can also be calculated using other methods such as the negative correlation transformation of Euclidean distance, which are not specifically limited here. A similarity threshold can be set, and a candidate match is considered when the matching score is not lower than the similarity threshold. To suppress false matches, a candidate vehicle can be considered a vehicle that meets the similarity condition only if it stably matches the same benchmark vehicle within K consecutive frames. For example, the percentage of frames in K consecutive frames where the candidate target successfully matches the same ground truth vehicle can be defined as the hit rate within the window. When the hit rate within the window is not lower than the hit rate threshold, the candidate vehicle is determined to be a ground truth vehicle. The value of the hit rate threshold can be adjusted according to the actual scenario, which is not specifically limited here.

[0076] The offline calibration method for roadside sensing cameras provided in this application uses multi-angle reference images for automatic matching, eliminating the need for manual judgment of the time and characteristics of the true vehicle based on experience. Through confidence filtering and continuous multi-frame stability determination, it effectively suppresses false matching, improves the accuracy of roadside sensing camera calibration, and provides reliable target vehicle input for subsequent sensing camera calibration processes.

[0077] In some embodiments, at least one MEC is also used to upload target video data and / or matrix parameters calibrated by roadside perception cameras to the platform storage space.

[0078] Specifically, target video data that passes quality verification can be stored in the MEC (Multi-access Edge Computing) at the current location. The platform then uploads the video data from the MEC to the cloud platform's storage space. Videos captured by roadside sensing cameras are initially stored on nearby MECs to avoid data loss due to network instability during acquisition. After quality detection confirms the video segment as the target video data, the platform centrally schedules its upload from the MEC to the cloud, creating data that can be stored long-term. The upload timing can be immediate after acquisition, in batches during network downtime, or on demand when the platform issues calibration tasks; no specific limitations are specified here.

[0079] After each MEC (Multi-access Edge Computing) obtains the homography matrix H through the roadside camera calibration process, it can transmit the matrix parameters, along with intermediate detection results such as ground truth vehicle trajectories and pixel coordinate pairs, to the platform for storage. The platform can save the matrix parameters to the platform using storage indices corresponding to the points and cameras, ensuring that each set of matrix parameters corresponds to its respective roadside sensing camera. The platform can also uniformly store and manage the versions of the generated matrix parameter files, such as recording the time of each calibration, the batch of target video data used, and calibration accuracy indicators. The specific content of the matrix parameter files is not specifically limited here.

[0080] The offline calibration method for roadside sensing cameras provided in this application uploads target video data and / or matrix parameters for roadside sensing camera calibration to the platform storage space, realizing centralized storage of roadside sensing camera calibration-related data. This provides a reliable data foundation for the verification of calibration results and the subsequent transmission of matrix parameters, and improves the maintainability of the roadside sensing camera calibration process.

[0081] In some embodiments, the method further includes: Send the matrix parameters calibrated by the roadside perception camera to the MEC corresponding to the roadside perception camera.

[0082] Specifically, the MEC corresponding to the roadside perception camera is the MEC that needs to run the perception algorithm using the camera's calibration parameters. The MEC corresponding to the roadside perception camera and the aforementioned MEC performing calibration calculations can be the same MEC or different MECs; no specific limitation is made here. Since the calibration calculations are distributed to idle MECs according to a load balancing strategy, the MEC performing a camera calibration subtask may be located in other locations, and the matrix parameters after calibration ultimately need to be returned to the MEC where the camera is located for use by the perception algorithm.

[0083] After the perception algorithm is deployed, the platform can automatically send the matching matrix parameter file to the corresponding MEC based on the location and camera index, thus automatically configuring the external parameter calibration parameters. The trigger condition for sending the file can be the completion event of the perception algorithm deployment, such as the MEC reporting a ready status to the platform after installation or upgrade, which triggers the sending. The trigger condition can also be other forms, such as confirmation by platform administrators; no specific limitation is made here.

[0084] The platform can locate the network address of the MEC corresponding to the point in the matrix parameter file to be distributed, based on the point and camera indexes. The platform then transmits the matrix parameter file to the MEC via the communication link between the platform and the MEC. Upon receiving the file, the MEC can perform integrity verification, such as checksums and comparisons. If the verification passes, the MEC loads the matrix parameters into the perception algorithm's configuration directory or configuration interface, enabling the perception algorithm to convert the pixel coordinates of detected targets in the image into world coordinates. The MEC can also send a confirmation message back to the platform after the configuration takes effect. The platform then updates the distribution status of the camera calibration parameters accordingly. If distribution fails or verification fails, the platform can automatically retry or issue an alarm.

[0085] The offline calibration method for roadside sensing cameras provided in this application automatically sends matrix parameters to the corresponding MEC after the sensing algorithm is deployed, realizing automatic configuration of external parameter calibration parameters, avoiding parameter mismatch and overlay loss caused by manual copying, and improving the reliability of large-scale roadside sensing system deployment.

[0086] Figure 2 The flowchart for offline calibration of roadside sensing cameras provided in the embodiments of this application is as follows: Figure 2 As shown, the ground truth vehicle first traverses the sensing points to prepare for data collection, including video acquisition. The acquired video data is not used directly but must first undergo quality verification. This verification is based on indicators such as video frame rate, frame interval, timing, resolution, and network fluctuations. Only video data that meets the quality standards can proceed to the next stage; otherwise, it needs to be re-acquired. Quality-compliant video data is stored and uploaded. Simultaneously, the platform collects and manages multi-angle benchmark images from the ground truth vehicle in parallel to build a feature library. Next, the platform schedules computing power based on computing power indicators and load balancing strategies, allocating calibration tasks to the most suitable idle MEC nodes. On the selected MEC, the calibration process for the roadside sensing cameras is executed.

[0087] The calibration process for roadside perception cameras includes: First, using a feature benchmark library, automatic detection and matching of ground truth vehicles are performed to accurately identify target vehicles from the video stream. Then, the pixel trajectory of the vehicle is matched with the world coordinate trajectory obtained through devices such as RTK. Finally, the camera's matrix parameters are calculated based on the matched point pairs. The generated parameters undergo projection verification to ensure their accuracy. Only parameters that pass the verification are saved to the platform and uniformly distributed to the corresponding business MEC to take effect.

[0088] The offline calibration device for roadside sensing cameras provided in the embodiments of this application is described below. The offline calibration device for roadside sensing cameras described below and the offline calibration method for roadside sensing cameras described above can be referred to in correspondence with each other.

[0089] Figure 3 This is a schematic diagram of the structure of the offline calibration device for roadside sensing cameras provided in the embodiments of this application, as shown below. Figure 3 As shown, the device includes: The detection unit 310 is used to perform quality detection on the video data collected by the roadside perception camera and determine the target video data for the calibration of the roadside perception camera. The allocation unit 320 is used to allocate target video data to at least one MEC, and the at least one MEC is used to perform calibration of the roadside perception camera.

[0090] In some embodiments, the load balancing strategy includes: prioritizing the MEC with higher idle time among the candidate MECs, and prioritizing the MEC closer to the roadside sensing camera when the idle time is the same.

[0091] In some embodiments, computing power metrics include at least one of graphics processing unit (GPU) utilization, video memory usage, and task queue length.

[0092] In some embodiments, quality inspection is performed on video data acquired by roadside sensing cameras, including: The quality of video data collected by roadside perception cameras is tested according to the detection indicators, which include at least one of the following: video frame rate, frame interval, time synchronization, resolution, and network fluctuation.

[0093] In some embodiments, the calibration process for roadside sensing cameras includes: The features of each vehicle in the target video data are matched with the features of true vehicles in the true vehicle feature benchmark library to determine the target vehicles in the target video data that match the true vehicle features. The pixel coordinates corresponding to the target vehicle are correlated across frames to form the pixel trajectory of the target vehicle; The pixel coordinates corresponding to each trajectory point on the pixel trajectory of the target vehicle are matched with the world coordinates corresponding to each trajectory point on the pixel trajectory of the target vehicle to obtain the matrix parameters for the calibration of the roadside perception camera.

[0094] In some embodiments, the features of each vehicle in the target video data are matched with the features of ground truth vehicles in the ground truth vehicle feature benchmark library to determine the target vehicles in the target video data that match the features of ground truth vehicles, including: Based on the features of ground truth vehicle images from multiple angles, a benchmark library of ground truth vehicle features is constructed. The features of each vehicle in the target video data are detected to obtain the feature vector of each vehicle. Calculate the similarity between the feature vector of each vehicle and the features of true vehicles in the true vehicle feature benchmark library; Vehicles that meet the similarity criteria among all vehicles are identified as target vehicles.

[0095] In some embodiments, at least one MEC is also used to upload target video data and / or matrix parameters calibrated by roadside perception cameras to the platform storage space.

[0096] In some embodiments, the device further includes: The transmitting unit is used to send the matrix parameters calibrated by the roadside perception camera to the MEC corresponding to the roadside perception camera.

[0097] It should be noted that the roadside sensing camera offline calibration device provided in this application embodiment can implement all the method steps implemented in the above method embodiment and can achieve the same technical effect. Here, the parts that are the same as those in the method embodiment and the beneficial effects will not be described in detail.

[0098] Figure 4 is a schematic diagram of the structure of the electronic device provided in an embodiment of this application, as shown below. Figure 4 As shown, the electronic device may include: a processor 410, a communication interface 420, a memory 430, and a communication bus 440, wherein the processor 410, the communication interface 420, and the memory 430 communicate with each other via the communication bus 440. The processor 410 can call the computer program in the memory 430 to execute the steps of the roadside perception camera offline calibration method provided in the above embodiments, such as including: The video data collected by the roadside sensing camera is subjected to quality inspection to determine the target video data for the calibration of the roadside sensing camera. The target video data is allocated to at least one multi-access edge computing (MEC), and at least one MEC is used to perform the calibration of roadside perception cameras; At least one MEC is selected from multiple candidate MECs based on their computing power metrics and load balancing strategies.

[0099] Furthermore, the logical instructions in the aforementioned memory 430 can be implemented as software functional units and, when sold or used as independent products, can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or a part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0100] On the other hand, embodiments of this application also provide a computer program product, which includes a computer program that can be stored on a non-transitory computer-readable storage medium. When the computer program is executed by a processor, the computer can perform the steps of the roadside perception camera offline method provided in the above embodiments, such as including: The video data collected by the roadside sensing camera is subjected to quality inspection to determine the target video data for the calibration of the roadside sensing camera. The target video data is allocated to at least one multi-access edge computing (MEC), and at least one MEC is used to perform the calibration of roadside perception cameras; At least one MEC is selected from multiple candidate MECs based on their computing power metrics and load balancing strategies.

[0101] It should be noted that the computer program product provided in this application embodiment can implement all the method steps implemented in the above method embodiment and can achieve the same technical effect. Here, the parts that are the same as those in the method embodiment and the beneficial effects will not be described in detail.

[0102] On the other hand, embodiments of this application also provide a processor-readable storage medium storing a computer program for causing a processor to execute the steps of the roadside perception camera offline calibration method provided in the above embodiments, including, for example: The video data collected by the roadside sensing camera is subjected to quality inspection to determine the target video data for the calibration of the roadside sensing camera. The target video data is allocated to at least one multi-access edge computing (MEC), and at least one MEC is used to perform the calibration of roadside perception cameras; At least one MEC is selected from multiple candidate MECs based on their computing power metrics and load balancing strategies.

[0103] It should be noted that the processor-readable storage medium provided in this application embodiment can implement all the method steps implemented in the above method embodiment and can achieve the same technical effect. Here, the parts that are the same as those in the method embodiment and the beneficial effects will not be described in detail.

[0104] The processor-readable storage medium can be any available medium or data storage device that the processor can access, including but not limited to magnetic memory (e.g., floppy disk, hard disk, magnetic tape, magneto-optical disk (MO)), optical memory (e.g., CD, DVD, BD, HVD), and semiconductor memory (e.g., ROM, EPROM, EEPROM, non-volatile memory (NAND FLASH), solid-state drive (SSD)).

[0105] The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs. Those skilled in the art can understand and implement this without any creative effort.

[0106] Through the above description of the embodiments, those skilled in the art can clearly understand that each embodiment can be implemented by means of software plus necessary general-purpose hardware platforms, and of course, it can also be implemented by hardware. Based on this understanding, the above technical solutions, in essence or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, magnetic disk, optical disk, etc., and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute the methods described in the various embodiments or some parts of the embodiments.

[0107] Those skilled in the art will understand that embodiments of this application can be provided as methods, systems, or computer program products. Therefore, this application can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, this application can take the form of a computer program product implemented on one or more computer-usable storage media (including, but not limited to, disk storage and optical storage) containing computer-usable program code.

[0108] This application is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of this application. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer-executable instructions. These computer-executable instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart... Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0109] These processor-executable instructions may also be stored in a processor-readable memory that can direct a computer or other programmable data processing device to operate in a particular manner, such that the instructions stored in the processor-readable memory produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0110] These processors can execute instructions that can also be loaded onto a computer or other programmable data processing device, causing a series of operational steps to be performed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable device for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0111] Obviously, those skilled in the art can make various modifications and variations to this application without departing from the spirit and scope of this application. Therefore, if such modifications and variations fall within the scope of the claims of this application and their equivalents, this application also intends to include such modifications and variations.

Claims

1. An offline calibration method for roadside sensing cameras, characterized in that, The method includes: The video data collected by the roadside sensing camera is subjected to quality inspection to determine the target video data used for the calibration of the roadside sensing camera. The target video data is allocated to at least one multi-access edge computing (MEC), and the at least one MEC is used to perform the calibration of the roadside perception camera; The at least one MEC is selected from the multiple candidate MECs based on their computing power metrics and load balancing strategies.

2. The offline calibration method for roadside sensing cameras according to claim 1, characterized in that, The load balancing strategy includes: prioritizing the MEC with higher idle rate among the candidate MECs, and prioritizing the MEC closer to the roadside sensing camera when the idle rate is the same.

3. The offline calibration method for roadside sensing cameras according to claim 1, characterized in that, The computing power metrics include at least one of the following: GPU utilization, video memory usage, and task queue length.

4. The offline calibration method for roadside sensing cameras according to any one of claims 1 to 3, characterized in that, The quality inspection of video data collected by roadside sensing cameras includes: The video data collected by the roadside perception camera is subjected to quality testing based on the detection indicators, which include at least one of the following: video frame rate, frame interval, time synchronization, resolution, and network fluctuation.

5. The offline calibration method for roadside sensing cameras according to any one of claims 1 to 3, characterized in that, The calibration process for the roadside sensing camera includes: The features of each vehicle in the target video data are matched with the features of true vehicles in the true vehicle feature benchmark library to determine the target vehicles in the target video data that match the features of true vehicles. The pixel coordinates corresponding to the target vehicle are correlated across frames to form the pixel trajectory of the target vehicle; The pixel coordinates corresponding to each trajectory point on the pixel trajectory of the target vehicle are matched with the world coordinates corresponding to each trajectory point on the pixel trajectory of the target vehicle to obtain the matrix parameters for the calibration of the roadside perception camera.

6. The offline calibration method for roadside sensing cameras according to claim 5, characterized in that, The step of matching the features of each vehicle in the target video data with the features of ground truth vehicles in the ground truth vehicle feature benchmark library to determine the target vehicles in the target video data that match the features of ground truth vehicles includes: Based on the features of ground truth vehicle images from multiple angles, a benchmark library of ground truth vehicle features is constructed. The features of each vehicle in the target video data are detected to obtain the feature vector of each vehicle. Calculate the similarity between the feature vectors of each vehicle and the features of true vehicles in the true vehicle feature benchmark library; The vehicles that meet the similarity criteria among the vehicles are identified as the target vehicles.

7. The offline calibration method for roadside sensing cameras according to claim 5, characterized in that, The at least one MEC is also used to upload the target video data and / or the matrix parameters calibrated by the roadside perception camera to the platform storage space.

8. The offline calibration method for roadside sensing cameras according to claim 5, characterized in that, The method further includes: The matrix parameters calibrated by the roadside sensing camera are sent to the MEC corresponding to the roadside sensing camera.

9. An offline calibration device for a roadside sensing camera, characterized in that, The device includes: The detection unit is used to perform quality detection on the video data collected by the roadside perception camera and determine the target video data used for the calibration of the roadside perception camera. An allocation unit is used to allocate the target video data to at least one MEC, the at least one MEC being used to perform calibration of the roadside perception camera.

10. An electronic device comprising a processor and a memory storing a computer program, characterized in that, When the processor executes the computer program, it implements the offline calibration method for roadside sensing cameras as described in any one of claims 1 to 8.

11. A processor-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by the processor, it implements the offline calibration method for roadside sensing cameras as described in any one of claims 1 to 8.

12. A computer program product, comprising a computer program, characterized in that, When the computer program is executed by the processor, it implements the offline calibration method for roadside sensing cameras as described in any one of claims 1 to 8.

Images