Method, device and electronic equipment for verifying results of roadside camera calibration

Abstract

The application discloses a roadside camera calibration result verification method and device and electronic equipment. The method comprises the following steps: acquiring a roadside camera calibration result and a road image collected by a roadside camera; detecting a vehicle target in the road image by using a preset target detection model to obtain a vehicle target detection result, wherein the vehicle target detection result comprises a vehicle detection frame and a vehicle type; determining the size of the vehicle target according to the vehicle detection frame and the roadside camera calibration result; and verifying the roadside camera calibration result according to the size of the vehicle target and the vehicle type to obtain a verification result of the roadside camera calibration result. The application detects the vehicle target in the image, determines the vehicle body size in combination with the previously calibrated roadside camera calibration result, verifies whether the previously calibrated roadside camera calibration result is available at present based on the fixed characteristics of the vehicle body size, improves the verification efficiency of the calibration result, and re-calibrates the calibration result in a timely manner when the calibration result is unavailable.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of roadside equipment, and in particular to a roadside camera calibration result verification method and device and electronic equipment. BACKGROUND

[0002] The perception of a roadside camera usually needs to use the transformation relationship between the image coordinate system and the real world coordinate system, so that the pixel position of the target detected in the image can be transformed into the absolute position in the real world, thereby using the absolute position for trajectory prediction, providing decision control to the vehicle end, or synchronously displaying on the digital twin to achieve the purpose of intelligent monitoring of urban roads.

[0003] The transformation relationship from the image coordinate system to the world coordinate system can be obtained through prior calibration. For example, the calibration method can use a checkerboard to calibrate the camera intrinsic parameters, calibrate the transformation relationship from the image to the real ground, and combine the world longitude and latitude information provided by the RTK (Real-time kinematic, real-time dynamic difference) positioning device to achieve the purpose of calibration.

[0004] However, the usually calibrated transformation relationship will be greatly deviated after the camera angle is adjusted, or the calibration files of each camera are confused, or the calibration is wrong, etc., so the accuracy of the calibration result needs to be verified. However, since there are many roadside cameras, and the deployment process may be wrong in the calibration link, manual checking one by one is too tedious and inefficient, and errors in the calibration result cannot be found in time. SUMMARY

[0005] The embodiments of the present application provide a roadside camera calibration result verification method and device and electronic equipment to improve the verification efficiency of the roadside camera calibration result.

[0006] The embodiments of the present application adopt the following technical solutions:

[0007] In a first aspect, the embodiments of the present application provide a roadside camera calibration result verification method, and the method comprises:

[0008] Obtaining a roadside camera calibration result and a road image collected by a roadside camera;

[0009] Detecting a vehicle target in the road image by using a preset target detection model to obtain a vehicle target detection result, the vehicle target detection result comprising a vehicle detection frame and a vehicle type;

[0010] Determining the size of the vehicle target according to the vehicle detection frame and the roadside camera calibration result;

[0011] verify the roadside camera calibration result according to the size of the vehicle target and the vehicle type, to obtain a verification result of the roadside camera calibration result.

[0012] Optionally, the vehicle bounding boxes include a plurality of, and the determining the size of the vehicle target according to the vehicle bounding boxes and the roadside camera calibration result includes:

[0013] determining a target vehicle bounding box from the plurality of vehicle bounding boxes by using a preset bounding box screening strategy;

[0014] determining the size of the vehicle target according to the target vehicle bounding box and the roadside camera calibration result.

[0015] Optionally, the roadside camera is a close-range camera or a long-range camera, and the determining the target vehicle bounding box from the plurality of vehicle bounding boxes by using the preset bounding box screening strategy includes:

[0016] determining a longitudinal relative distance of a bottom edge of each vehicle bounding box to a bottom edge of the road image and a transverse relative distance of a center of each vehicle bounding box to a center of the road image;

[0017] determining whether there is a vehicle bounding box in each vehicle bounding box, whose longitudinal relative distance of the bottom edge to the bottom edge of the road image is less than a first preset longitudinal distance threshold, and whose transverse relative distance of the center to the center of the road image is less than a first preset transverse distance threshold;

[0018] if there is, taking the vehicle bounding box with the smallest longitudinal relative distance to the bottom edge of the road image and the smallest transverse relative distance to the center of the road image as the target vehicle bounding box.

[0019] Optionally, the roadside camera is a fisheye camera, and the determining the target vehicle bounding box from the plurality of vehicle bounding boxes by using the preset bounding box screening strategy includes:

[0020] determining a longitudinal relative distance and a transverse relative distance of a center of each vehicle bounding box to a center of the road image;

[0021] determining whether there is a vehicle bounding box in each vehicle bounding box, whose longitudinal relative distance of the center to the center of the road image is less than a second preset longitudinal distance threshold, and whose transverse relative distance of the center to the center of the road image is less than a second preset transverse distance threshold;

[0022] if there is, taking the vehicle bounding box with the smallest longitudinal relative distance and the smallest transverse relative distance to the center of the road image as the target vehicle bounding box.

[0023] Optionally, the vehicle bounding box comprises image position coordinates of a bottom-left corner point of the vehicle bounding box and image position coordinates of a bottom-right corner point of the vehicle bounding box, the roadside camera calibration result comprises a transformation relationship between an image coordinate system and a world coordinate system, and the determining of the size of the vehicle target according to the vehicle bounding box and the roadside camera calibration result comprises:

[0024] converting the image position coordinates of the bottom-left corner point and the image position coordinates of the bottom-right corner point into the world coordinate system according to the transformation relationship between the image coordinate system and the world coordinate system to obtain world position coordinates of the bottom-left corner point and world position coordinates of the bottom-right corner point;

[0025] determining the size of the vehicle target according to the world position coordinates of the bottom-left corner point and the world position coordinates of the bottom-right corner point.

[0026] Optionally, the roadside camera is a camera of a non-intersection road section, and the verifying of the roadside camera calibration result according to the size of the vehicle target and the vehicle type comprises:

[0027] determining a first preset size threshold corresponding to the vehicle target according to the vehicle type;

[0028] comparing the size of the vehicle target with the first preset size threshold corresponding to the vehicle target to obtain a first comparison result;

[0029] determining the verification result of the roadside camera calibration result according to the first comparison result.

[0030] Optionally, the roadside camera is a camera of an intersection road section, and the verifying of the roadside camera calibration result according to the size of the vehicle target and the vehicle type comprises:

[0031] determining a vehicle driving direction corresponding to the vehicle bounding box;

[0032] determining a second preset size threshold corresponding to the vehicle target according to the vehicle type and the vehicle driving direction corresponding to the vehicle bounding box;

[0033] comparing the size of the vehicle target with the second preset size threshold corresponding to the vehicle target to obtain a second comparison result;

[0034] determining the verification result of the roadside camera calibration result according to the second comparison result.

[0035] Optionally, the determining of the second preset size threshold corresponding to the vehicle target according to the vehicle type and the vehicle driving direction corresponding to the vehicle bounding box comprises:

[0036] In a case where the vehicle driving direction corresponding to the vehicle detection frame is a direction parallel to the orientation of the roadside camera, a preset vehicle body width threshold corresponding to the vehicle target is determined according to the vehicle type;

[0037] In a case where the vehicle driving direction corresponding to the vehicle detection frame is a direction perpendicular to the orientation of the roadside camera, a preset vehicle body length threshold corresponding to the vehicle target is determined according to the vehicle type.

[0038] In a second aspect, an embodiment of the present application further provides a device for verifying a calibration result of a roadside camera, and the device comprises:

[0039] an acquisition unit configured to acquire the calibration result of the roadside camera and a road image collected by the roadside camera;

[0040] a detection unit configured to detect a vehicle target in the road image by using a preset target detection model to obtain a vehicle target detection result, the vehicle target detection result comprising a vehicle detection frame and a vehicle type;

[0041] a determination unit configured to determine a size of the vehicle target according to the vehicle detection frame and the calibration result of the roadside camera;

[0042] a verification unit configured to verify the calibration result of the roadside camera according to the size of the vehicle target and the vehicle type to obtain a verification result of the calibration result of the roadside camera.

[0043] In a third aspect, an embodiment of the present application further provides an electronic device, comprising:

[0044] a processor; and

[0045] a memory arranged to store computer executable instructions that, when executed, cause the processor to perform any of the aforementioned methods.

[0046] In a fourth aspect, an embodiment of the present application further provides a computer readable storage medium storing one or more programs, which, when executed by an electronic device comprising a plurality of applications, cause the electronic device to perform any of the aforementioned methods.

[0047] The at least one technical solution adopted in this application embodiment can achieve the following beneficial effects: The verification method for roadside camera calibration results in this application embodiment first acquires the roadside camera calibration results and the road image captured by the roadside camera; then, it uses a preset target detection model to detect vehicle targets in the road image to obtain vehicle target detection results, which include vehicle detection boxes and vehicle types; then, it determines the size of the vehicle target based on the vehicle detection boxes and the roadside camera calibration results; finally, it verifies the roadside camera calibration results based on the size of the vehicle target and the vehicle type to obtain the verification result of the roadside camera calibration results. The verification method for roadside camera calibration results in this application embodiment detects vehicle targets in the image and determines the vehicle body size by combining the pre-calibrated roadside camera calibration results. Based on the fixed vehicle body size, it verifies whether the pre-calibrated roadside camera calibration results are accurate and usable at present, which improves the verification efficiency of the calibration results so as to promptly recalibrate unusable calibration results. Attached Figure Description

[0048] The accompanying drawings, which are included to provide a further understanding of this application and form part of this application, illustrate exemplary embodiments and are used to explain this application, but do not constitute an undue limitation of this application. In the drawings:

[0049] Figure 1 This is a flowchart illustrating a method for verifying the calibration results of a roadside camera according to an embodiment of this application.

[0050] Figure 2 This is a schematic diagram of the screening results of a vehicle detection frame in an embodiment of this application;

[0051] Figure 3 This is a schematic diagram of the screening results of another vehicle detection frame in an embodiment of this application;

[0052] Figure 4 This is a schematic diagram of the structure of a verification device for the calibration results of a roadside camera in an embodiment of this application;

[0053] Figure 5 This is a schematic diagram of the structure of an electronic device according to an embodiment of this application. Detailed Implementation

[0054] To make the objectives, technical solutions, and advantages of this application clearer, the technical solutions of this application will be clearly and completely described below in conjunction with specific embodiments and corresponding drawings. Obviously, the described embodiments are only a part of the embodiments of this application, and not all of them. Based on the embodiments in this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0055] The technical solutions provided by the embodiments of the present application are described in detail below with reference to the drawings.

[0056] The present application provides a kind of verification method of roadside camera calibration result, as shown in Figure 1 The present application provides a kind of verification method of roadside camera calibration result, as shown in

[0057] Step S110, the roadside camera calibration result and the road image collected by roadside camera are acquired.

[0058] Since the verification method of roadside camera calibration result of the present application is mainly applied to the verification link after the calibration of roadside camera, the roadside camera calibration result that has been calibrated in advance needs to be acquired first, which can include the transformation relationship between image coordinate system and world coordinate system, based on which any point in the image collected by roadside camera can be converted into real world, which can provide more abundant perception information for automatic driving vehicles in actual road scene.

[0059] In addition, the present application also needs to acquire the road image currently collected by roadside camera first as the basic data for subsequent verification whether the calibration result is available.

[0060] Step S120, the vehicle target in the road image is detected using a preset target detection model, to obtain vehicle target detection result, and the vehicle target detection result includes vehicle detection frame and vehicle type.

[0061] After obtaining the road image currently collected by roadside camera, existing target detection model such as target detection model based on YOLO etc. Convolutional neural network training can be used to detect vehicle target in road image, so as to obtain vehicle target detection result, which can specifically include all vehicle detection frames and corresponding vehicle types detected in road image, and vehicle type can be car, bus or truck type, etc.

[0062] Step S130, the size of vehicle target is determined according to the vehicle detection frame and the roadside camera calibration result.

[0063] The vehicle detection frame output by the preset target detection model specifically contains the position information of the vehicle detection frame in the image coordinate system, and the calibration result of roadside camera contains the transformation relationship between image coordinate system and world coordinate system, so based on the transformation relationship, the size of the vehicle corresponding to the vehicle detection frame in real world can be determined.

[0064] Step S140, verifying the roadside camera calibration result according to the size of the vehicle target and the vehicle type, to obtain a verification result of the roadside camera calibration result.

[0065] Since the size of the vehicle target is determined based on the roadside camera calibration result, if the roadside camera calibration result is accurate enough, the size of the vehicle target obtained therefrom is also close to the real size of the vehicle.

[0066] If the roadside camera calibration result has a large error or is no longer applicable to the current camera angle, the size of the vehicle target obtained therefrom will deviate greatly from the real size of the vehicle.

[0067] In addition, the real size of different types of vehicles also has obvious differences, so the embodiment of the present application can verify the roadside camera calibration result by combining the size of the vehicle target and the vehicle type, so as to obtain a verification result of the roadside camera calibration result, that is, whether the previously calibrated roadside camera calibration result is currently available.

[0068] In order to ensure the verification efficiency of the calibration result, the above-mentioned verification method of the roadside camera calibration result of the embodiment of the present application can be executed by the roadside, of course, in actual application, it can also be executed by the cloud, and the person skilled in the art can flexibly set the execution subject according to the actual demand, which is not limited here.

[0069] The verification method of the roadside camera calibration result of the embodiment of the present application verifies whether the previously calibrated roadside camera calibration result is accurate and available at present based on the fixed characteristics of the vehicle body size by detecting the vehicle target in the image and determining the size of the vehicle target combined with the previously calibrated roadside camera calibration result, which improves the verification efficiency of the calibration result, so as to timely recalibrate the unusable calibration result.

[0070] In some embodiments of the present application, the vehicle detection frame includes a plurality of vehicle detection frames, and the size of the vehicle target is determined according to the vehicle detection frame and the roadside camera calibration result, which includes: determining a target vehicle detection frame in the plurality of vehicle detection frames by using a preset detection frame screening strategy; and determining the size of the vehicle target according to the target vehicle detection frame and the roadside camera calibration result.

[0071] When a road image is detected by using a preset target detection model, multiple vehicle detection frames may be detected, but not all vehicle detection frames are necessarily suitable as the basis for verifying the roadside camera calibration result, which may be affected by the type of roadside camera.

[0072] 5Based on this, the embodiments of the present application can use the pre-defined detection box screening strategy to screen a plurality of vehicle detection boxes, determine the target vehicle detection box therefrom, and then determine the size of the vehicle target corresponding to the target vehicle detection box in combination with the road side camera calibration result.

[0073] Of course, it should be noted that the target detection box meeting the above requirements can also be multiple, and the vehicle size corresponding to each target detection box can be determined, and then the road side camera calibration result can be verified in combination with the vehicle type corresponding to each target detection box, for example, if the vehicle target size corresponding to one target detection box deviates greatly from the true size, it is considered that the road side camera calibration result is unusable.

[0074] In some embodiments of the present application, the road side camera is a close-up camera or a long-range camera, and the determination of the target vehicle detection box from the plurality of vehicle detection boxes using the preset detection box screening strategy includes: determining the longitudinal relative distance of the bottom edge of each vehicle detection box to the bottom edge of the road image and the horizontal relative distance of the center of each vehicle detection box to the center of the road image; determining whether there is a vehicle detection box in each vehicle detection box whose longitudinal relative distance of the bottom edge to the bottom edge of the road image is less than a first preset longitudinal distance threshold, and whose horizontal relative distance of the center to the center of the road image is less than a first preset horizontal distance threshold; if so, the vehicle detection box with the smallest longitudinal relative distance to the bottom edge of the road image and the smallest horizontal relative distance to the center of the road image is taken as the target vehicle detection box.

[0075] The road side cameras currently deployed in the road generally include close-up cameras, long-range cameras and fisheye cameras. The close-up cameras are used to shoot the road conditions in the near distance, the long-range cameras are used to shoot the road conditions in the far distance, and the fisheye cameras are mainly used to compensate for the blind area of the close-up cameras and the long-range cameras, and are used to shoot the road conditions at the downward viewing angle of the camera.

[0076] Different types of road side cameras will have different degrees of deviation compared to the real world. The different degrees of deviation here mainly refer to the differences between the close-up cameras, the long-range cameras and the fisheye cameras. For the close-up cameras and the long-range cameras, the main problem is that the image pixels of the far distance deviate greatly, while for the fisheye cameras, the main problem is image distortion, and the farther away from the center of the image, the greater the degree of pixel distortion.

[0077] Based on this, different preset detection box screening strategies can be adopted for different types of roadside cameras to screen vehicle detection boxes. The preset detection box screening strategy designed for the close-up camera or the long-range camera in the embodiments of the present application. For the close-up camera or the long-range camera, considering the problem of "near large and far small", when screening the vehicle detection box, the longitudinal relative distance between the bottom edge of each vehicle detection box and the bottom edge of the road image and the horizontal relative distance between the center of each vehicle detection box and the center of the road image can be determined first. The smaller the longitudinal relative distance between the bottom edge of the vehicle detection box and the bottom edge of the image, and the smaller the horizontal relative distance between the center of the vehicle detection box and the center of the road image, the smaller the pixel offset error and the distortion degree corresponding to the vehicle detection box. Therefore, the error of the vehicle size calculated based on the vehicle detection box is less affected by the image itself.

[0078] Of course, in order to ensure the quality of the target vehicle detection box, a first preset longitudinal distance threshold and a first preset horizontal distance threshold can also be set to constrain the longitudinal relative distance and the horizontal relative distance. If the two threshold requirements are not met at the same time, the detection result of the current frame image can be discarded, and the next frame road image is acquired for detection, until the vehicle detection box detected in the image meets the above two threshold requirements at the same time.

[0079] On this basis, the vehicle detection box with the smallest longitudinal relative distance between the bottom edge of the vehicle detection box and the bottom edge of the image and the smallest horizontal relative distance between the center of the vehicle detection box and the center of the road image can be taken as the final target detection box. Since the target photographed by the roadside camera at this position is basically parallel to the driving direction of the road, the target vehicle detection box is usually the detection box with the smallest pixel offset, the smallest distortion degree, and the two sides of the detected rectangular box closest to the left and right sides of the vehicle body. As shown in Figure 2 The vehicle detection box pointed by the arrow in the screening result diagram of the vehicle detection box in the embodiments of the present application is the final target detection box.

[0080] In some embodiments of the present application, the roadside camera is a fisheye camera, and the target vehicle detection box is determined from the multiple vehicle detection boxes by using the preset detection box screening strategy, including: determining the longitudinal relative distance and the horizontal relative distance between the center of each vehicle detection box and the center of the road image; determining whether there is a vehicle detection box whose longitudinal relative distance and horizontal relative distance between the center of the vehicle detection box and the center of the road image are less than the second preset longitudinal distance threshold and the second preset horizontal distance threshold; if there is, the vehicle detection box with the smallest longitudinal relative distance and horizontal relative distance to the center of the road image is taken as the target vehicle detection box.

[0081] The preset detection frame screening strategy designed for the fisheye camera mainly aims to solve the image distortion problem of the fisheye camera. The farther away from the center of the image, the greater the degree of pixel distortion, and vice versa, the closer to the center of the image, the smaller the degree of pixel distortion. Therefore, the longitudinal relative distance and the transverse relative distance between the center of each vehicle detection frame and the center of the road image can be determined first. The smaller the longitudinal relative distance and the transverse relative distance between the center of the vehicle detection frame and the center of the road image, the smaller the degree of image distortion, and the smaller the error of the vehicle size calculated based on the vehicle detection frame.

[0082] In order to ensure the quality of the target vehicle detection frame, the second preset longitudinal distance threshold and the second preset transverse distance threshold can be further set to constrain the longitudinal relative distance and the transverse relative distance respectively. If the two threshold requirements are not met at the same time, the detection result of the current frame image can be discarded, and the next frame road image is acquired for detection until the vehicle detection frame in the image meets the two threshold requirements at the same time.

[0083] On this basis, the vehicle detection frame with the smallest longitudinal relative distance and transverse relative distance between the center of the vehicle detection frame and the center of the image can be taken as the final target detection frame. Since the target detection frame is closest to the center of the image, the target vehicle detection frame has the smallest distortion degree, and the two sides of the detected rectangular frame are closest to the left and right sides of the vehicle body. As shown in Figure 3 The arrow points to the final target detection frame.

[0084] Of course, for the image distortion problem of the fisheye camera, the existing image distortion correction algorithm can also be used for distortion correction processing to obtain a distortion-corrected image for target detection. Since the fisheye camera is downward, the deviation of the multiple vehicle detection frames detected in the distortion-corrected image is small, so the actual situation can be flexibly selected.

[0085] It should be noted that the purpose of the above embodiment of screening the vehicle detection frame is to minimize the influence of the image itself on the vehicle size calculation, thereby improving the accuracy of the influence of the road side camera calibration result on the vehicle size calculation.

[0086] In some embodiments of the present application, the vehicle bounding box includes image position coordinates of a bottom-left corner point and a bottom-right corner point of the vehicle bounding box, the roadside camera calibration result includes a transformation relationship between an image coordinate system and a world coordinate system, and determining the size of the vehicle target according to the vehicle bounding box and the roadside camera calibration result includes: converting the image position coordinates of the bottom-left corner point and the bottom-right corner point to the world coordinate system respectively according to the transformation relationship between the image coordinate system and the world coordinate system to obtain world position coordinates of the bottom-left corner point and the bottom-right corner point; and determining the size of the vehicle target according to the world position coordinates of the bottom-left corner point and the bottom-right corner point.

[0087] Since the transformation relationship between the image coordinate system and the world coordinate system included in the roadside camera calibration result specifically refers to the position transformation of the roadside camera to the ground, and the bottom-left corner point and the bottom-right corner point of the four corner points of the vehicle bounding box correspond to ground points, the image position coordinates of the bottom-left corner point and the bottom-right corner point of the vehicle bounding box can be converted to the world coordinate system based on the previously calibrated transformation relationship between the image coordinate system and the world coordinate system, so as to obtain the absolute positions of the two corner points in the real world.

[0088] Since the left and right sides of the vehicle bounding box selected in the foregoing embodiments are close to the left and right sides of the vehicle body, the size information of the vehicle target can be calculated according to the distance between the absolute positions of the two corner points in the real world, which may be the width of the vehicle body or the length of the vehicle body, mainly depending on the driving direction of the vehicle target relative to the roadside camera.

[0089] In some embodiments of the present application, the roadside camera is a camera for a non-intersection section, and verifying the roadside camera calibration result according to the size of the vehicle target and the vehicle type to obtain a verification result of the roadside camera calibration result includes: determining a first preset size threshold corresponding to the vehicle target according to the vehicle type; comparing the size of the vehicle target with the first preset size threshold corresponding to the vehicle target to obtain a first comparison result; and determining the verification result of the roadside camera calibration result according to the first comparison result.

[0090] After the size of the vehicle target is obtained based on the roadside camera calibration result in the foregoing embodiments, the size of the vehicle target can be compared with a real size threshold corresponding to the vehicle target, and then it is determined whether the size of the vehicle target obtained based on the roadside camera calibration result is accurate.

[0091] Since some of the roadside cameras currently deployed on roads are deployed on non-intersection sections, i.e. normal driving sections, the cameras are parallel to the driving direction of the road, and some of the roadside cameras are deployed on intersection sections, such as crossroads, where multiple driving directions of the road exist, so the cameras may be parallel or perpendicular to the driving direction of the road. The orientation of the camera relative to the driving direction of the road also affects the type of the size of the vehicle target calculated, for example, if the orientation of the camera is parallel to the driving direction of the road, the size of the vehicle target calculated is the width of the vehicle body, and if the orientation of the camera is perpendicular to the driving direction of the road, the size of the vehicle target calculated is the length of the vehicle body.

[0092] Based on this, for the roadside cameras deployed on non-intersection sections, since the driving direction of the road is relatively single, the driving direction of all the vehicle targets detected from the road image collected by the roadside camera is basically parallel to the orientation of the roadside camera, so it is not necessary to judge the driving direction of the vehicle target to directly determine that the size of the vehicle calculated based on the two corner points of the bottom side of the vehicle detection box is the width of the vehicle body, and then the threshold value corresponding to the width of the vehicle body set in advance can be used to measure the accuracy of the calculation of the width of the vehicle body, and then to verify whether the calibration result of the roadside camera is available.

[0093] In addition, different types of vehicles correspond to different widths of the vehicle body, for example, the width of the vehicle body of a small car is obviously smaller than the width of the vehicle body of a truck, so when determining the threshold value corresponding to the width of the vehicle body, the type of the vehicle detected before can also be combined to determine the width of the vehicle body threshold value currently used for judgment, i.e. the first preset size threshold value, if the currently calculated width of the vehicle body is obviously greater or smaller than the first preset size threshold value, it indicates that the width of the vehicle body is not accurate, and the corresponding calibration result of the roadside camera is not available, and a reminder can be initiated to complete the recalibration of the roadside camera as soon as possible.

[0094] In some embodiments of the present application, the roadside camera is a camera for an intersection section, and the verification of the calibration result of the roadside camera according to the size of the vehicle target and the type of the vehicle includes: determining the driving direction of the vehicle corresponding to the vehicle detection box; determining a second preset size threshold value corresponding to the vehicle target according to the type of the vehicle and the driving direction of the vehicle corresponding to the vehicle detection box; comparing the size of the vehicle target with the second preset size threshold value corresponding to the vehicle target to obtain a second comparison result; and determining the verification result of the calibration result of the roadside camera according to the second comparison result.

[0095] For the roadside camera deployed at the intersection section, due to the relatively complex driving direction of the road, the camera orientation may be parallel or perpendicular to the driving direction of the road. Therefore, the driving direction of the vehicle corresponding to the current vehicle detection frame can be determined first. For example, the driving direction of the vehicle can be determined according to the traffic light information corresponding to the vehicle, or the moving direction of the vehicle target in the image can be determined by target tracking in adjacent frames. How to determine the driving direction of the vehicle can be selected flexibly by those skilled in the art in combination with the prior art.

[0096] After determining the driving direction of the vehicle corresponding to the vehicle detection frame, it can be determined whether the vehicle size calculated based on the two corner points of the bottom edge of the vehicle detection frame is the vehicle body width or the vehicle body length, and then different vehicle body size thresholds are determined and compared.

[0097] In some embodiments of the present application, the second preset size threshold corresponding to the vehicle target is determined according to the vehicle type and the driving direction of the vehicle corresponding to the vehicle detection frame. If the driving direction of the vehicle corresponding to the vehicle detection frame is parallel to the orientation of the roadside camera, a preset vehicle body width threshold corresponding to the vehicle target is determined according to the vehicle type. If the driving direction of the vehicle corresponding to the vehicle detection frame is perpendicular to the orientation of the roadside camera, a preset vehicle body length threshold corresponding to the vehicle target is determined according to the vehicle type.

[0098] If the driving direction of the vehicle is parallel to the orientation of the roadside camera, it means that the two corner points of the bottom edge of the vehicle detection frame detected by the roadside camera in the visual angle correspond to the vehicle body width, and then the corresponding vehicle body width threshold can be determined in combination with the vehicle type corresponding to the vehicle detection frame. If the driving direction of the vehicle is perpendicular to the orientation of the roadside camera, it means that the two corner points of the bottom edge of the vehicle detection frame detected by the roadside camera in the visual angle correspond to the vehicle body length, and then the corresponding vehicle body length threshold can be determined in combination with the vehicle type corresponding to the vehicle detection frame.

[0099] The present application also provides a verification device 400 for the calibration result of the roadside camera, as shown in Figure 4 The structure diagram of the verification device for the calibration result of the roadside camera in the present application is provided, and the device 400 comprises an acquisition unit 410, a detection unit 420, a determination unit 430 and a verification unit 440, wherein:

[0100] The acquisition unit 410 is configured to acquire the calibration result of the roadside camera and the road image collected by the roadside camera.

[0101] The detection unit 420 is configured to detect the vehicle target in the road image by using a preset target detection model to obtain a vehicle target detection result, wherein the vehicle target detection result comprises a vehicle detection frame and a vehicle type.

[0102] determining, by a determining unit 430, a size of a vehicle target according to the vehicle bounding box and the roadside camera calibration result;

[0103] verifying, by a verifying unit 440, the roadside camera calibration result according to the size of the vehicle target and the vehicle type, to obtain a verification result of the roadside camera calibration result.

[0104] In some embodiments of the present application, the vehicle bounding boxes include a plurality of vehicle bounding boxes, and the determining unit 430 is specifically configured to: determine a target vehicle bounding box from the plurality of vehicle bounding boxes by using a preset bounding box screening strategy; and determine the size of the vehicle target according to the target vehicle bounding box and the roadside camera calibration result.

[0105] In some embodiments of the present application, the roadside camera is a close-range camera or a long-range camera, and the determining unit 430 is specifically configured to: determine a longitudinal relative distance between a bottom edge of each vehicle bounding box and a bottom edge of the road image and a transverse relative distance between a center of each vehicle bounding box and a center of the road image; determine whether there is a vehicle bounding box in each vehicle bounding box, in which a longitudinal relative distance between a bottom edge of the vehicle bounding box and a bottom edge of the road image is less than a first preset longitudinal distance threshold, and a transverse relative distance between a center of the vehicle bounding box and a center of the road image is less than a first preset transverse distance threshold; and if there is, determine a vehicle bounding box with the smallest longitudinal relative distance to the bottom edge of the road image and the smallest transverse relative distance to the center of the road image as the target vehicle bounding box.

[0106] In some embodiments of the present application, the roadside camera is a fisheye camera, and the determining unit 430 is specifically configured to: determine a longitudinal relative distance and a transverse relative distance between a center of each vehicle bounding box and a center of the road image; determine whether there is a vehicle bounding box in each vehicle bounding box, in which a longitudinal relative distance between a center of the vehicle bounding box and a center of the road image is less than a second preset longitudinal distance threshold, and a transverse relative distance between a center of the vehicle bounding box and a center of the road image is less than a second preset transverse distance threshold; and if there is, determine a vehicle bounding box with the smallest longitudinal relative distance and the smallest transverse relative distance to the center of the road image as the target vehicle bounding box.

[0107] In some embodiments of the present application, the vehicle bounding box includes image position coordinates of a bottom-left corner point and a bottom-right corner point of the vehicle bounding box, and the roadside camera calibration result includes a transformation relationship between an image coordinate system and a world coordinate system. The determination unit 430 is specifically configured to: convert the image position coordinates of the bottom-left corner point and the bottom-right corner point to the world coordinate system respectively according to the transformation relationship between the image coordinate system and the world coordinate system, to obtain world position coordinates of the bottom-left corner point and world position coordinates of the bottom-right corner point; and determine the size of the vehicle target according to the world position coordinates of the bottom-left corner point and the world position coordinates of the bottom-right corner point.

[0108] In some embodiments of the present application, the roadside camera is a camera of a non-intersection road section, and the verification unit 440 is specifically configured to: determine a first preset size threshold corresponding to the vehicle target according to the vehicle type; compare the size of the vehicle target with the first preset size threshold corresponding to the vehicle target to obtain a first comparison result; and determine a verification result of the roadside camera calibration result according to the first comparison result.

[0109] In some embodiments of the present application, the roadside camera is a camera of an intersection road section, and the verification unit 440 is specifically configured to: determine a vehicle driving direction corresponding to the vehicle bounding box; determine a second preset size threshold corresponding to the vehicle target according to the vehicle type and the vehicle driving direction corresponding to the vehicle bounding box; compare the size of the vehicle target with the second preset size threshold corresponding to the vehicle target to obtain a second comparison result; and determine a verification result of the roadside camera calibration result according to the second comparison result.

[0110] In some embodiments of the present application, the verification unit 440 is specifically configured to: in a case where the vehicle driving direction corresponding to the vehicle bounding box is a direction parallel to the orientation of the roadside camera, determine a preset vehicle body width threshold corresponding to the vehicle target according to the vehicle type; and in a case where the vehicle driving direction corresponding to the vehicle bounding box is a direction perpendicular to the orientation of the roadside camera, determine a preset vehicle body length threshold corresponding to the vehicle target according to the vehicle type.

[0111] It can be understood that the roadside camera calibration result verification device described above can realize each step of the roadside camera calibration result verification method provided in the foregoing embodiments, and the related explanations about the roadside camera calibration result verification method are all applicable to the roadside camera calibration result verification device, which will not be described here again.

[0112] Figure 5 is a structural schematic diagram of an electronic device of an embodiment of the present application. Please refer to Figure 5At the hardware level, the electronic device includes a processor, and optionally further includes an internal bus, a network interface, and a memory. The memory can include a memory such as a random-access memory (RAM), and can also include a non-volatile memory such as a read-only memory (ROM).

[0113] The electronic device can also include other hardware required by the business, such as at least one magnetic disk memory.

[0114] The processor, the network interface, and the memory can be connected to each other through the internal bus, which can be an industry standard architecture (ISA) bus, a peripheral component interconnect (PCI) bus, or an extended industry standard architecture (EISA) bus, etc. The bus can be divided into an address bus, a data bus, and a control bus. For ease of representation, Figure 5 Only one bidirectional arrow is used in the figure, but

[0115] It does not mean that there is only one bus or one type of bus.

[0116] The memory is used to store programs. Specifically, the program can include program code including computer operation instructions. The memory can include a memory and a non-volatile memory, and provide instructions and data to the processor.

[0117] The processor reads the corresponding computer program from the non-volatile memory into the memory and then runs, and forms a verification device for the roadside camera calibration result at the logical level. The processor executes the program stored in the memory, and is specifically used for the following operations:

[0118] Obtain a roadside camera calibration result and a road image collected by a roadside camera;

[0119] Detect a vehicle target in the road image using a preset target detection model to obtain a vehicle target detection result, the vehicle target detection result including a vehicle detection frame and a vehicle type;

[0120] Determine the size of the vehicle target according to the vehicle detection frame and the roadside camera calibration result;

[0121] Verify the roadside camera calibration result according to the size of the vehicle target and the vehicle type to obtain a verification result of the roadside camera calibration result.

[0122] The above is as stated in this application. Figure 1 The five methods executed by the verification device for roadside camera calibration results disclosed in the illustrated embodiment can be applied to a processor, or implemented by a processor. The processor may be an integrated circuit chip.

[0123] The chip has signal processing capabilities. In implementation, each step of the above method can be completed by integrated logic circuits in the hardware of the processor or by instructions in software form. The processor can be a general-purpose processor, including a central processing unit (CPU), a network processor (NP), etc.; it can also be a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components. It can implement or execute the methods, steps, and logic block diagrams disclosed in the embodiments of this application. The general-purpose processor can be a microprocessor or any conventional processor. The steps of the methods disclosed in the embodiments of this application can be directly embodied in the execution of a hardware decoding processor, or executed by a combination of hardware and software modules in the decoding processor. The software modules can be located in random access memory, flash memory, read-only memory, programmable read-only memory, electrically erasable programmable memory, registers, or other mature storage media in the art. The storage medium is located in the memory, and the processor reads the information in the memory and, in conjunction with its hardware, completes the steps of the above method.

[0124] The electronic device can also perform Figure 1 A method for verifying the calibration results of roadside cameras, and the implementation of the verification device for the calibration results of roadside cameras in... Figure 1 The functions of the embodiments shown are not described in detail here.

[0125] This application also proposes a computer-readable storage medium that stores one or more programs, the programs including instructions that, when executed by an electronic device including multiple applications, enable the electronic device to perform... Figure 1 The method executed by the verification device for the calibration results of the roadside camera in the illustrated embodiment is specifically used to perform:

[0126] Acquire the calibration results of the roadside cameras and the road images captured by the roadside cameras;

[0127] The vehicle target in the road image is detected by using a preset target detection model, to obtain a vehicle target detection result, which includes a vehicle detection frame and a vehicle type;

[0128] The size of the vehicle target is determined according to the vehicle detection frame and the road-side camera calibration result;

[0129] The road-side camera calibration result is verified according to the size of the vehicle target and the vehicle type, to obtain a verification result of the road-side camera calibration result.

[0130] Those skilled in the art will understand that embodiments of the present application can be provided as methods, systems, or computer program products. Therefore, the present application can take the form of an entirely hardware embodiment, an entirely software embodiment, or an embodiment combining software and hardware aspects. Moreover, the present application can take the form of a computer program product implemented on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROMs, optical storage, etc.) containing computer-usable program code.

[0131] The present application is described with reference to flowcharts and / or block diagrams of methods, apparatus (systems) and computer program products according to embodiments of the present application. It should be understood that each flow and / or block in the flowcharts and / or block diagrams, as well as combinations of flows and / or blocks in the flowcharts and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing apparatus to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing apparatus produce an apparatus that implements the functions specified in the flowcharts and / or block diagrams. Figure 1 The functions specified in one or more flows and / or blocks Figure 1 The functions specified in one or more flows and / or blocks

[0132] These computer program instructions can also be stored in a computer-readable memory that can direct the computer or other programmable data processing apparatus to work in a specific manner, so that the instructions stored in the computer-readable memory produce a manufactured product including instruction apparatus, which implements the functions specified in the flowcharts and / or block diagrams. Figure 1 The functions specified in one or more flows and / or blocks Figure 1 The functions specified in one or more flows and / or blocks

[0133] These computer program instructions can also be loaded onto a computer or other programmable data processing apparatus, so that a series of operation steps are performed on the computer or other programmable data processing apparatus to produce a computer-implemented process, so that the instructions executed on the computer or other programmable data processing apparatus provide a process for implementing the functions specified in the flowcharts and / or block diagrams. Figure 1 The functions specified in one or more flows and / or blocks Figure 1steps of a function specified in one or more blocks.

[0134] In one typical arrangement, the computing device includes one or more processors (CPUs), input / output interfaces, network interfaces, and memory.

[0135] The memory can include non-persistent memory and / or volatile memory, such as random access memory (RAM) and / or cache memory, non-volatile memory, such as read-only memory (ROM), EPROM, and / or flash memory, etc. The memory is an example of computer readable media.

[0136] Computer readable media includes permanent and non-permanent, removable and non-removable media implemented in any method or technology for storage of information such as computer readable instructions, data structures, program modules or other data. Examples of computer storage media include, but are not limited to, phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technology, compact disc read-only memory (CD-ROM), digital versatile discs (DVDs) or other optical storage, magnetic cassettes, magnetic tapes, magnetic disk storage or other magnetic storage devices, or any other non-transmission medium that can be used to store information accessible to a computing device. According to the definition herein, computer readable media does not include transitory media, such as modulated data signals and carrier waves.

[0137] It should also be noted that the terms "comprising," "including," and any other variation thereof are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements does not include only those elements but can also include other elements not expressly listed or inherent to such process, method, article, or apparatus. Without further limitation, an element preceded by "comprises a" does not, without more constraints, foreclose the existence of additional identical elements in the process, method, article, or apparatus that includes the recited element.

[0138] Those skilled in the art will appreciate that embodiments of the present application can be devised for a method, a system, or a computer program product. Accordingly, the present application can take the form of an entirely hardware embodiment, an entirely software embodiment or an embodiment combining software and hardware aspects. Furthermore, the present application can take the form of a computer program product on one or more computer readable storage media (including, but not limited to, disk memory, CD-ROMs, optical storage devices, etc.) embodying computer readable program code.

[0139] The above merely provides an example of the present application, and is not intended to limit the present application. Any modification, equivalent replacement, improvement, etc. within the spirit and principle of the present application should be included in the scope of claims of the present application.

Claims

1. A method for verifying the calibration results of a roadside camera, wherein, The method includes: Acquire the calibration results of the roadside cameras and the road images captured by the roadside cameras; The vehicle targets in the road image are detected using a preset target detection model to obtain vehicle target detection results, which include vehicle detection boxes and vehicle types. The size of the vehicle target is determined based on the vehicle detection frame and the calibration results of the roadside camera. The calibration results of the roadside camera are verified based on the size of the vehicle target and the vehicle type to obtain the verification results of the roadside camera calibration results; The vehicle detection frame includes multiple frames, and determining the size of the vehicle target based on the vehicle detection frame and the roadside camera calibration result includes: The target vehicle detection frame is determined from multiple vehicle detection frames by using preset detection frame filtering strategies corresponding to different types of roadside cameras; The size of the vehicle target is determined based on the target vehicle detection frame and the roadside camera calibration results.

2. The method as described in claim 1, wherein, The roadside camera can be a close-up or a long-range camera. A preset detection box filtering strategy is used to determine the target vehicle detection box from multiple vehicle detection boxes, including: Determine the vertical relative distance between the bottom edge of each vehicle detection frame and the bottom edge of the road image, and the horizontal relative distance between the center of each vehicle detection frame and the center of the road image; Determine whether there are any vehicle detection frames in each vehicle detection frame whose vertical relative distance between the bottom edge of the vehicle detection frame and the bottom edge of the road image is less than a first preset vertical distance threshold, and whose horizontal relative distance between the center of the vehicle detection frame and the center of the road image is less than a first preset horizontal distance threshold. If it exists, the vehicle detection box with the smallest vertical relative distance to the bottom edge of the road image and the smallest horizontal relative distance to the center of the road image is taken as the target vehicle detection box.

3. The method as described in claim 1, wherein, The roadside camera is a fisheye camera. A preset detection box filtering strategy is used to determine the target vehicle detection box from multiple vehicle detection boxes, including: Determine the longitudinal and lateral relative distances between the center of each vehicle detection box and the center of the road image; Determine whether there are any vehicle detection frames in each vehicle detection frame whose vertical relative distance between the center of the vehicle detection frame and the center of the road image is less than a second preset vertical distance threshold, and whose horizontal relative distance between the center of the vehicle detection frame and the center of the road image is less than a second preset horizontal distance threshold; If it exists, the vehicle detection box with the smallest longitudinal and lateral relative distances to the center of the road image will be used as the target vehicle detection box.

4. The method as described in claim 1, wherein, The vehicle detection bounding box includes the image position coordinates of the left corner point and the right corner point of the bottom edge of the vehicle detection bounding box. The roadside camera calibration result includes the transformation relationship between the image coordinate system and the world coordinate system. Determining the size of the vehicle target based on the vehicle detection bounding box and the roadside camera calibration result includes: Based on the transformation relationship between the image coordinate system and the world coordinate system, the image position coordinates of the bottom left corner point and the bottom right corner point are transformed to the world coordinate system respectively to obtain the world position coordinates of the bottom left corner point and the world position coordinates of the bottom right corner point. The size of the vehicle target is determined based on the world position coordinates of the left corner point of the bottom edge and the world position coordinates of the right corner point of the bottom edge.

5. The method as described in claim 1, wherein, The roadside camera is a camera located on a non-intersection road section. The verification of the roadside camera calibration results based on the size of the vehicle target and the vehicle type yields the following verification results: Determine the first preset size threshold corresponding to the vehicle target based on the vehicle type; The size of the vehicle target is compared with a first preset size threshold corresponding to the vehicle target to obtain a first comparison result; The verification result of the roadside camera calibration result is determined based on the first comparison result.

6. The method of claim 1, wherein, The roadside camera is a camera located at an intersection. The verification of the roadside camera calibration results based on the size and type of the vehicle target, yielding verification results including: Determine the vehicle's driving direction corresponding to the vehicle detection frame; The second preset size threshold corresponding to the vehicle target is determined based on the vehicle type and the vehicle driving direction corresponding to the vehicle detection frame; The size of the vehicle target is compared with a second preset size threshold corresponding to the vehicle target to obtain a second comparison result; The verification result of the roadside camera calibration result is determined based on the second comparison result.

7. The method of claim 6, wherein, The step of determining the second preset size threshold corresponding to the vehicle target based on the vehicle type and the vehicle driving direction corresponding to the vehicle detection frame includes: If the vehicle driving direction corresponding to the vehicle detection frame is parallel to the direction of the roadside camera, then the preset vehicle width threshold corresponding to the vehicle target is determined according to the vehicle type. If the vehicle's travel direction corresponding to the vehicle detection frame is perpendicular to the direction of the roadside camera, then a preset vehicle length threshold corresponding to the vehicle target is determined according to the vehicle type.

8. A verification device for roadside camera calibration results, wherein, The device includes: The acquisition unit is used to acquire the calibration results of the roadside camera and the road images captured by the roadside camera. The detection unit is used to detect vehicle targets in the road image using a preset target detection model, and obtain vehicle target detection results, which include vehicle detection boxes and vehicle types. A determining unit is used to determine the size of the vehicle target based on the vehicle detection frame and the roadside camera calibration result; The verification unit is used to verify the calibration result of the roadside camera based on the size of the vehicle target and the vehicle type, and to obtain the verification result of the roadside camera calibration result; The vehicle detection frame includes multiple frames, and the determining unit is specifically used for: The target vehicle detection frame is determined from multiple vehicle detection frames by using preset detection frame filtering strategies corresponding to different types of roadside cameras; The size of the vehicle target is determined based on the target vehicle detection frame and the roadside camera calibration results.

9. An electronic device, comprising: processor; as well as A memory configured to store computer-executable instructions, which, when executed, cause the processor to perform the method of any one of claims 1 to 7.

10. A computer-readable storage medium storing one or more programs, which, when executed by an electronic device including a plurality of applications, cause the electronic device to perform the method of any one of claims 1 to 7.

Images