Calibration method, device and equipment of radar and video acquisition device and storage medium

Abstract

The application provides a radar and video acquisition device calibration method and device, equipment and a storage medium, comprising: from the data collected by the radar and video acquisition device for the same scene, intercepting radar images and corresponding video images and displaying them side by side, obtaining the radar target and the corresponding image target selected by the user as a matching target pair, calculating the calibration matrix based on this, obtaining the radar target position mark and displaying it in the video image as the calibration result, if the calibration result does not meet the requirements, obtaining the newly added matching target pair selected by the user and recalculating the calibration matrix until the calibration result meets the requirements. In the embodiment of the application, the user can determine the calibration point and the corresponding image target from the radar data and the video image for the same scene at the same time displayed side by side, without the user measuring the accurate radar coordinates and image coordinates of the calibration point, so that the calibration can be based on the actually collected data, improving the accuracy and operability of the calibration.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of device calibration, in particular to a radar and video acquisition device calibration method, device, apparatus and storage medium. BACKGROUND

[0002] Currently, in many fields, in order to accurately detect target objects, point cloud data collected by radar and images collected by video acquisition devices are used for positioning. For example, in the fields of traffic, security, and automobiles, radar and video images are used for target object detection.

[0003] In order to ensure that the radar data of the target object and the video image correspond to each other, the radar and video acquisition device need to be calibrated. Calibration is to determine the conversion relationship between two coordinate systems, and in this application, it refers to the conversion between the radar coordinate system and the image coordinate system.

[0004] In related technologies, the calibration method of the radar and video acquisition device is usually as follows: first, in the case that there is no target object to be detected in the shooting scene, video images of the scene environment are collected. Then, based on the cognition of the scene environment and the video images of the scene environment, the calibration personnel manually selects a quadrilateral on the radar display screen of the scene environment, sets four vertices of the quadrilateral as target points participating in calibration, and estimates the coordinate values of the target points in the radar coordinate system. For example, based on the four points of the pedestrian crossing in the video image, a quadrilateral is selected on the radar display screen corresponding to the four points of the pedestrian crossing, and the coordinate values of the four points are estimated. Then, based on the coordinates of the target points in the radar coordinate system and the coordinates in the video image, a calibration matrix is obtained. Finally, the calibration matrix is used to perform coordinate conversion on other points in the radar data, and based on the coordinate conversion result, the calibration effect is obtained, and in the case that the calibration effect meets the requirements, the calibration is completed.

[0005] In this process, the calibration personnel needs to continuously adjust the coordinate values of the target points in the radar coordinate system according to the calibration effect to realize the calibration.

[0006] Or the coordinates of the four calibration points in the radar coordinate system are measured manually, which usually consumes a lot of human resources, especially in the case of multiple devices.

[0007] It can be seen that in the related technology, the coordinate values of the target points in the radar coordinate system are manually estimated by the calibration personnel, and the calibration based on the manually estimated coordinate values is not accurate enough. SUMMARY

[0008] The purpose of the embodiments of the present application is to provide a radar and video acquisition device calibration method, device, apparatus and storage medium to improve the accuracy of calibration. The specific technical solutions are as follows:

[0009] In one aspect of the present application, a calibration method of radar and video acquisition equipment is provided, comprising:

[0010] acquiring radar data and video data collected by the radar and video acquisition equipment for the same scene;

[0011] based on the radar data and video data, intercepting a radar image containing multiple radar targets and a video image at the same time containing multiple image targets corresponding to the radar image;

[0012] displaying the video image and the radar image side by side on a display;

[0013] obtaining at least four non-collinear radar targets selected by a user based on the current display content as calibration points, and image targets corresponding to the calibration points, and taking the calibration points and the corresponding image targets as matching target pairs;

[0014] calculating a current calibration matrix based on image coordinate information and radar coordinate information of the matching target pairs;

[0015] based on the current calibration matrix, performing coordinate conversion on each radar target in the radar image to obtain converted position information of each radar target in the image coordinate system;

[0016] based on the converted position information of each radar target, generating corresponding radar target position markers, and displaying each radar target position marker in the video image as a calibration result;

[0017] if the user determines that the calibration result does not meet the requirements, obtaining at least one non-calibration point radar target selected by the user based on the current display content as a new calibration point, and new image targets corresponding to each new calibration point, and taking the new calibration point and the corresponding new image target as a new matching target pair; returning to the step of calculating the current calibration matrix based on the image coordinate information and the radar coordinate information of the matching target pairs until the user determines that the calibration result meets the requirements.

[0018] In one embodiment of the present application, the step of obtaining at least four non-collinear radar targets selected by a user based on the current display content as calibration points, and image targets corresponding to each calibration point, comprises:

[0019] based on the user's operation of dragging at least four non-collinear radar targets in the radar image to the video image, taking the at least four radar targets as calibration points, determining the current positions of the images of the calibration points dragged into the video image, and obtaining image targets corresponding to the current positions as image targets corresponding to the calibration points;

[0020] or,

[0021] Based on the clicking operation of the user on the selected image target of each calibration point, the selected image target is determined as the image target corresponding to each calibration point.

[0022] In an embodiment of the present application, the step of obtaining at least one non-calibration point radar target selected by the user based on the current display content as a new calibration point and a new image target corresponding to each new calibration point when the user determines that the calibration result does not meet the requirements comprises:

[0023] Based on the operation of the user dragging the non-calibration point radar target selected in the radar image to the video image, the non-calibration point radar target is determined as a new calibration point, the current position of each new calibration point dragged to the video image is obtained, and the image target corresponding to the current position is obtained as a new image target corresponding to the new calibration point.

[0024] Or,

[0025] Based on the clicking operation of the user on the selected image target of each new calibration point in the video image, the selected image target is determined as a new image target corresponding to each new calibration point.

[0026] In an embodiment of the present application, the image target comprises a first image target detected by an image detection algorithm or a second image target not detected by an image detection algorithm.

[0027] The step of obtaining at least four non-collinear radar targets selected by the user based on the current display content as calibration points and image targets corresponding to each calibration point, and taking the calibration points and corresponding image targets as matching target pairs comprises:

[0028] At least four non-collinear radar targets selected by the user based on the current display content are obtained as calibration points.

[0029] Based on the clicking operation of the user on the selected image target of each calibration point in the video image, the selected first image target and / or second image target is determined as the image target corresponding to each calibration point.

[0030] The step of obtaining at least one non-calibration point radar target selected by the user based on the current display content as a new calibration point and a new image target corresponding to each new calibration point comprises:

[0031] At least one non-calibration point radar target selected by the user based on the current display content is obtained as a new calibration point.

[0032] The first image target and / or the second image target clicked by the user in the video image based on the click operation of the selected each newly added calibration point is determined as the newly added image target corresponding to the newly added calibration point.

[0033] In an embodiment of the present application, in the step of obtaining at least four non-collinear radar targets selected by the user based on the current display content as calibration points and the image targets corresponding to each calibration point, the step further comprises: performing zooming operation on the image locally based on the image local zooming instruction input by the user for the part of the image that is not clear; and / or, in the step of obtaining at least one non-calibration point radar target selected by the user based on the current display content as a newly added calibration point and the newly added image target corresponding to each newly added calibration point, the step further comprises: performing zooming operation on the image locally based on the image local zooming instruction input by the user for the part of the image that is not clear.

[0034] In a second aspect of the present application, a calibration device for radar and video acquisition equipment is provided, comprising:

[0035] A data acquisition module is configured to acquire radar data and video data collected by the radar and video acquisition equipment for the same scene;

[0036] A cutting module is configured to cut radar images containing a plurality of radar targets and video images at the same time containing a plurality of image targets based on the radar data and video data;

[0037] A display module is configured to display the video images and radar images side by side on a display;

[0038] A matching target pair determination module is configured to obtain at least four non-collinear radar targets selected by the user based on the current display content as calibration points and the image targets corresponding to each calibration point, and to determine the calibration points and the corresponding image targets as matching target pairs;

[0039] A calibration matrix calculation module is configured to calculate a current calibration matrix based on the image coordinate information and radar coordinate information of each matching target pair;

[0040] A coordinate conversion module is configured to perform coordinate conversion on each radar target in the radar images based on the current calibration matrix to obtain converted position information of each radar target in the image coordinate system;

[0041] A calibration result acquisition module is configured to generate corresponding radar target position markers based on the converted position information of each radar target, and to display each radar target position marker in the video images as a calibration result;

[0042] The matching target pair determination module is configured to determine, based on the operation of the user dragging at least four non-collinear radar targets in the radar image to the video image, the at least four radar targets as calibration points, determine current positions of images of the calibration points in the video image, and obtain image targets corresponding to the current positions as image targets corresponding to the calibration points.

[0043] In an embodiment of the present application, the matching target pair determination module is configured to determine, based on the operation of the user dragging at least four non-collinear radar targets in the radar image to the video image, the at least four radar targets as calibration points, determine current positions of images of the calibration points in the video image, and obtain image targets corresponding to the current positions as image targets corresponding to the calibration points.

[0044] Or,

[0045] The matching target pair determination module is configured to determine, based on the operation of the user dragging at least four non-collinear radar targets in the radar image to the video image, the at least four radar targets as calibration points, determine current positions of images of the calibration points in the video image, and obtain image targets corresponding to the current positions as image targets corresponding to the calibration points.

[0046] In an embodiment of the present application, the matching target pair determination module is configured to determine, based on the operation of the user dragging at least four non-collinear radar targets in the radar image to the video image, the at least four radar targets as calibration points, determine current positions of images of the calibration points in the video image, and obtain image targets corresponding to the current positions as image targets corresponding to the calibration points.

[0047] Or,

[0048] The matching target pair determination module is configured to determine, based on the operation of the user dragging at least four non-collinear radar targets in the radar image to the video image, the at least four radar targets as calibration points, determine current positions of images of the calibration points in the video image, and obtain image targets corresponding to the current positions as image targets corresponding to the calibration points.

[0049] In an embodiment of the present application, the image target includes a first image target detected by an image detection algorithm or a second image target not detected by the image detection algorithm.

[0050] The matching target pair determination module is configured to determine, based on the operation of the user dragging at least four non-collinear radar targets in the radar image to the video image, the at least four radar targets as calibration points.

[0051] The matching target pair determination module is configured to determine, based on the operation of the user dragging at least four non-collinear radar targets in the radar image to the video image, the at least four radar targets as calibration points, determine current positions of images of the calibration points in the video image, and obtain image targets corresponding to the current positions as image targets corresponding to the calibration points.

[0052] The newly added matching target pair acquisition module is configured to obtain at least one non-designated point radar target selected by the user based on the current display content as a newly added designated point;

[0053] Based on the click operation of the user on each newly added designated point in the video image, the first image target and / or the second image target that is clicked and selected is determined as a newly added image target corresponding to the newly added designated point pair.

[0054] In an embodiment of the present application, the matching target pair determination module is further configured to perform an image local magnification operation on the image based on an image local magnification instruction input by the user for the local part of the image that is not clear.

[0055] The newly added matching target pair acquisition module is further configured to perform an image local magnification operation on the image based on an image local magnification instruction input by the user for the local part of the image that is not clear.

[0056] Another aspect of the embodiment of the present application also provides an electronic device, characterized by comprising a processor, a communication interface, a memory and a communication bus, wherein the processor, the communication interface and the memory complete communication with each other through the communication bus.

[0057] The memory is configured to store a computer program.

[0058] The processor is configured to execute the program stored on the memory, so as to realize the calibration method steps of the radar and video acquisition device according to any one of the above embodiments.

[0059] The embodiment of the present application also provides a computer readable storage medium, characterized by storing a computer program in the computer readable storage medium, and the computer program is executed by the processor to realize the calibration method steps of the radar and video acquisition device according to any one of the above embodiments.

[0060] The embodiment of the present application also provides a computer program product containing instructions, which, when running on a computer, causes the computer to execute the calibration method of the radar and video acquisition device according to any one of the above embodiments.

[0061] The embodiment of the present application has the following beneficial effects:

[0062] The radar and video acquisition device calibration method provided in the embodiment of the application obtains radar data and video data collected by a radar and a video acquisition device for the same scene, intercepts a radar image containing a plurality of radar targets and a video image of the same time containing a plurality of image targets, displays the video image and the radar image side by side, obtains at least four non-collinear radar targets selected by a user as calibration points, determines image targets corresponding to the calibration points, takes the calibration points and the corresponding image targets as matching target pairs, calculates a current calibration matrix based on image coordinate information and radar coordinate information of the matching target pairs, performs coordinate conversion on each radar target in the radar image based on the current calibration matrix, obtains position information of each radar target in an image coordinate system after conversion, generates corresponding radar target position marks based on the position information, and displays each radar target position mark in the video image as a calibration result. If the calibration result does not meet the requirement, at least one non-calibration point radar target selected by the user based on the current display content is obtained as a new calibration point, and a new image target corresponding to each new calibration point is obtained, the new calibration point and the corresponding new image target are taken as a new matching target pair, and a new calibration matrix is calculated based on the matching target pairs to obtain a calibration result until the user determines that the calibration result meets the requirement. In the embodiment of the application, the user can determine the calibration points and the corresponding image targets from the radar data and the video image of the same time for the same scene displayed side by side, without the user giving accurate radar coordinates and image coordinates of the calibration points, so that the calibration can be performed based on the actually collected data, thereby improving the accuracy and operability of the calibration of the radar and the video acquisition device.

[0063] Of course, implementing any product or method of the application does not necessarily require achieving all the advantages described above at the same time. BRIEF DESCRIPTION OF DRAWINGS

[0064] In order to more clearly illustrate the technical solutions in the embodiments of the application or the prior art, the following will briefly introduce the drawings needed in the embodiments or prior art description. Obviously, the drawings in the following description only constitute some embodiments of the application, and other embodiments can also be obtained according to these drawings for those skilled in the art.

[0065] Figure 1 A flowchart of the radar and video acquisition device calibration method provided in the embodiment of the application;

[0066] Figure 2 An actual display situation diagram of the video image and the radar image in the embodiment of the application;

[0067] Figure 3 A display diagram of the video image and the radar image in the embodiment of the application;

[0068] Figure 4 An effect diagram after one calibration in an embodiment of the present application;

[0069] Figure 5 An example diagram of a deviation between a preset calibration position and a coordinate conversion position in an embodiment of the present application;

[0070] Figure 6 A diagram of obtaining an image target corresponding to a new calibration point in an embodiment of the present application;

[0071] Figure 7 A diagram of a final calibration effect in an embodiment of the present application;

[0072] Figure 8 A diagram of an actual final calibration effect in an embodiment of the present application;

[0073] Figure 9 A structural diagram of a calibration device of a radar and video acquisition device provided by an embodiment of the present application;

[0074] Figure 10 A structural diagram of an electronic device provided by an embodiment of the present application. DETAILED DESCRIPTION

[0075] The technical solutions in the embodiments of the present application will be described clearly and completely below with reference to the drawings in the embodiments of the present application. Obviously, the described embodiments are only some of the embodiments of the present application, but not all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art based on the present application belong to the scope of protection of the present application.

[0076] In order to improve the accuracy of calibration of a radar and video acquisition device, an embodiment of the present application provides a calibration method, device, electronic device and storage medium of a radar and video acquisition device.

[0077] Reference is made to Figure 1 , Figure 1 A flow diagram of a calibration method of a radar and video acquisition device provided by an embodiment of the present application, which can specifically include the following steps:

[0078] Step S110, radar data and video data collected by a radar and video acquisition device for a same scene are obtained;

[0079] Step S120, based on the radar data and video data, a radar image containing multiple radar targets and a video image at the same time containing multiple image targets are intercepted;

[0080] Step S130, the video image and the radar image are displayed side by side on a display.

[0081] In step S140, at least four non-collinear radar targets selected by the user based on the current display content are obtained as calibration points, and image targets corresponding to the calibration points are obtained as matching target pairs.

[0082] In step S150, a current calibration matrix is calculated based on the image coordinate information and the radar coordinate information of the matching target pairs.

[0083] In step S160, coordinate conversion is performed on each radar target in the radar image based on the current calibration matrix, and converted position information of each radar target in the image coordinate system is obtained.

[0084] In step S170, corresponding radar target position markers are generated based on the converted position information of each radar target, and each radar target position marker is displayed in the video image as a calibration result.

[0085] In step S180, at least one non-calibration point radar target selected by the user based on the current display content is obtained as a new calibration point when the user determines that the calibration result does not meet the requirements, and a new image target corresponding to each new calibration point is obtained as a new matching target pair. The step of calculating the current calibration matrix based on the image coordinate information and the radar coordinate information of the matching target pairs is returned to step S150 until the user determines that the calibration result meets the requirements.

[0086] The radar and video acquisition device calibration method provided in the embodiments of the present application intercepts a radar image containing a plurality of radar targets and a video image of the same time containing a plurality of image targets from radar data and video data collected by a radar and video acquisition device for the same scene, and displays the video image and the radar image side by side. Then, at least four non-collinear radar targets selected by a user are obtained as calibration points, and image targets corresponding to the calibration points are determined. The calibration points and the corresponding image targets are taken as matching target pairs. Based on image coordinate information and radar coordinate information of each matching target pair, a current calibration matrix is calculated. Based on the current calibration matrix, coordinate conversion is performed on each radar target in the radar image to obtain position information of each radar target in the image coordinate system after conversion. Based on the position information, corresponding radar target position markers are generated and displayed in the video image as a calibration result. If the calibration result does not meet the requirements, at least one non-calibration point radar target selected by the user based on the current display content is obtained as a new calibration point, and a new image target corresponding to each new calibration point is obtained. The new calibration point and the corresponding new image target are taken as a new matching target pair. Based on each matching target pair, the calibration matrix is recalculated to obtain the calibration result until the user determines that the calibration result meets the requirements. In the embodiments of the present application, the user can determine the calibration points and the corresponding image targets from the radar data and the video image of the same time for the same scene displayed side by side. The user does not need to provide accurate radar coordinates and image coordinates of the calibration points. The calibration can be performed based on the actual collected data, thereby improving the accuracy and operability of the calibration of the radar and video acquisition device.

[0087] In the embodiments of the present application, the radar and video acquisition device described above can be installed for the same scene. The installation positions of the two devices can have an angle difference, that is, the two devices can not be installed at the same position, but the installation positions of the two devices are preferably kept at the same height, so that the radar data and the video data collected by the two devices are more consistent. For example, for intersection vehicle monitoring, the radar device and the video acquisition device can be uniformly installed on the left side of the road, or the radar device and the video acquisition device can be installed on the left and right sides of the road, respectively, and can be installed at a position 5-6 m away from the ground.

[0088] In the embodiment of the present application, the video acquisition device can be a camera erected at an intersection, and the radar can be a millimeter wave radar. The millimeter wave radar, i.e., a radar working in a millimeter wave band, has a strong ability to penetrate fog, smoke and dust, and has the characteristics of all-weather (except heavy rain) and all-day. Meanwhile, the millimeter wave radar can identify multiple targets at the same time, and can distinguish very small targets. In addition, the millimeter wave radar also has good anti-interference performance. Therefore, in the embodiment of the present application, the use of the millimeter wave radar can make the radar data obtained more accurate, and further improve the accuracy of the calibration of the radar and the video acquisition device.

[0089] In the embodiment of the present application, when the radar image and the video image at the same time are intercepted, the radar targets in the radar image or the image targets in the video image are distributed as evenly as possible, so as to further improve the accuracy of the calibration.

[0090] In the embodiment of the present application, after the radar image and the corresponding video image at the same time are intercepted, the two can be displayed side by side on the display, and the user can clearly and intuitively determine the image target corresponding to each radar target based on the radar targets in the radar image and the image targets in the video image.

[0091] As shown in Figure 2 , Figure 2 is a schematic diagram of actual display of the radar image and the video image at the same time in the embodiment of the present application, Figure 2 the radar image and the corresponding video image at the same time intercepted from the radar data and the video data collected for the same scene are displayed side by side. Figure 2 The left image is a video image intercepted from the video data collected by the video acquisition device. In addition to the vehicle target, the image can also display lane number, vehicle number, vehicle type, vehicle speed and other information. Figure 2 The right image is the corresponding radar image. In addition to the radar target corresponding to the vehicle, the radar image can also display the vehicle number and the driving track information.

[0092] Figure 3 is a schematic diagram of display of the radar target and the video image obtained by simplifying the actual display diagram in the embodiment of the present application. The video image Figure 3 on the left) contains 8 image targets (small cars), and the labels are 1, 2, 3, 4, 5, 6, 7 and 8. The radar image Figure 3 on the right) at the same time also contains 8 radar targets indicated by dots, and the labels are also 1, 2, 3, 4, 5, 6, 7 and 8.

[0093] Figure 2 , Figure 3The targets in the radar image and the video image are uniformly distributed, and by displaying the two images side by side, the user can intuitively see the correspondence between the radar targets and the image targets from the image.

[0094] As a specific implementation of the embodiment of the present application, the step of obtaining, in step S140, at least four non-collinear radar targets selected by the user based on the current display content as calibration points and the image targets corresponding to the calibration points can specifically include:

[0095] Based on the operation of the user dragging at least four non-collinear radar targets in the radar image to the video image, the at least four radar targets are taken as calibration points, the image of each calibration point is determined to be dragged to the current position in the video image, and the image target corresponding to the current position is obtained as the image target corresponding to the calibration point.

[0096] When the user selects the calibration points, the user can make the distribution of the calibration points as uniform as possible to avoid the selected calibration points being too dispersed or too concentrated, which leads to a large calculation error of the calibration matrix. The number of the calibration points selected by the user is preferably not less than four, and to enable the calibration matrix to be normally calculated, the four calibration points are preferably not on the same straight line.

[0097] After the user selects the calibration points, the user can drag the calibration points to the corresponding positions in the video image. Specifically, the user can drag the above calibration points to the lower edge center positions of the corresponding image targets by a mouse or a touch screen, and the system can determine that the position to which the calibration point is dragged is the image target corresponding to the calibration point.

[0098] In other embodiments, the image target corresponding to the calibration point can also be determined by the user clicking. Specifically, the image target clicked by the user can be determined as the image target corresponding to the calibration point based on the clicking operation of the user on the image target in the video image.

[0099] In the embodiment of the present disclosure, for one calibration point, the calibration point and the image target corresponding to the calibration point can be taken as a matching target pair. In the embodiment of the present disclosure, there can be multiple matching target pairs.

[0100] After the above matching target pairs are determined, the current calibration matrix can be calculated based on the image coordinate information of the image target in each matching target pair and the radar coordinate information of the calibration point. The radar coordinate is the coordinate of the calibration point in the radar data (in the radar coordinate system), and the image coordinate is the coordinate of the image target corresponding to the calibration point in the video data (in the image coordinate system).

[0101] In the embodiment of the present application, the current calibration matrix can be calculated by the following method:

[0102] In homogeneous coordinates, assume that a point p(x i ,y i ,1) is transformed by the H matrix to p'(x' i ,y' i ,1), that is, p' = H * p, H is the current calibration matrix to be solved, x i ,y i are the radar coordinates of the calibration points, i is the label of the calibration point, and x' i ,y' i are the image coordinates of the image target corresponding to the calibration point.

[0103]

[0104] Generally, for perspective transformation, let h 33 = 1, at this time, the H matrix has 8 degrees of freedom, so at least 4 pairs of feature points are needed to solve. For the case of n > 4 pairs of feature points (overdetermined equation), solving the p' = H * p equation group can be converted to solving the homogeneous equation group A * h = b.

[0105] From formula (1), we have:

[0106]

[0107] When the number of calibration points is 4, write it in matrix form:

[0108]

[0109] For the case of n > 4 pairs of feature points (overdetermined equation), solving the p' = H * p equation group can be converted to solving the homogeneous equation group A * h = b, which can be solved by the least square method. The characteristic value and characteristic vector of the coefficient matrix A are obtained. The H matrix has been solved, and subsequent selection can be performed by random sampling consensus (RANSC).

[0110] After the current calibration matrix is calculated based on the above calibration points and corresponding image targets, the radar coordinates of each radar target in the intercepted radar image can be converted based on the current calibration matrix and the radar coordinates of each radar target, to obtain the converted position information of each radar coordinate in the image coordinate system.

[0111] After the above radar targets are converted, each radar target can be displayed at the corresponding position in the video image according to the converted position information, and the similarity of the corresponding position of each radar target in the video image and the position of the image target corresponding to each radar target is observed to determine whether the calibration effect meets the requirements. If it meets the requirements, the calibration is completed. Whether the calibration effect meets the requirements can be determined by the user based on actual experiments and use requirements.

[0112] As a specific embodiment of the present application, in order to make the calibration effect more intuitive, the calibration points and non-calibration points can be distinguished by different colors.

[0113] Based on Figure 3 As shown in the example, Figure 4 As shown, Figure 4 is a schematic diagram of the effect after one calibration in an embodiment of the present application:

[0114] Figure 4 In the example, the user selects radar targets 1, 2, 3, and 5 as calibration points, and drags the above four calibration points to the center of the lower edge of the image targets 1, 2, 3, and 5 in the video image. The coordinates of the position can be used as the coordinates of the calibration points in the image coordinate system of the video image. Then, the calibration matrix is calculated based on the radar coordinates of the above calibration points and the corresponding image coordinates, and all radar targets in the radar image are projected into the video image based on the calibration matrix, as shown in the left image in Figure 4 In the example, the projection positions of the calibration points are completely consistent with the positions of the corresponding image targets, and in the non-calibration points, the projection position of the radar target 4 has a small error with the position of the corresponding image target, but the projection points of the radar targets 6, 7, and 8, which are far away from the calibration points 1, 2, 3, and 5, still have a large gap with the corresponding image targets, that is, Figure 4 The calibration effect shown in the example does not meet the requirements.

[0115] In the case where the calibration result does not meet the requirements, the user can select other non-calibration point radar targets as new calibration points based on the current display content, and select new image targets corresponding to the new calibration points. As a specific embodiment of the present application, the user can select at least one radar target with an error greater than a preset threshold from the other radar targets of the non-calibration points in the radar data as a new calibration point based on the above coordinate conversion result.

[0116] Through the above step S170, the image targets and the projection points of the corresponding radar calibration points can be displayed on the same image, and the user can intuitively see the projection points of the radar targets with large errors from the image, so that the radar targets corresponding to the projection points with large errors can be selected as new calibration points.

[0117] The above error generally refers to the deviation of the preset calibration position and the coordinate conversion position. For example, in a traffic scene, the radar detection point is used as the calibration point, and the midpoint of the lower edge of the image detection frame is used as the preset calibration position. In theory, the coordinate conversion position of any target after radar coordinate conversion should be at the midpoint of the lower edge of the image frame. However, due to various reasons, there is a certain deviation between the calibration point and the preset calibration position.

[0118] Referring to Figure 5 wherein the dark point P'(x',y') of the vehicle image is the coordinate position of the radar target projected onto the video image according to the calibration matrix, and the light point P(x,y) adjacent to the vehicle image is the preset calibration position (i.e. the accurate position of the radar target in the video image). In the absence of errors, the dark point P'(x',y') should coincide with the light point P(x,y). At this time, the error is defined as:

[0119] x-direction error: Δx = abs(x-x')

[0120] y-direction error: Δy = abs(y-y')

[0121] The threshold of the error is generally given according to experience, and generally, the error threshold of Δx and Δy can be defined as not greater than half of the width and height of the target image detection frame.

[0122] In the embodiment of the present application, an x-direction threshold and a y-direction threshold can be preset, and if any of the above direction errors is greater than the corresponding threshold, the radar target of the corresponding non-calibration point can be selected as a newly added calibration point.

[0123] In the embodiment of the present application, when the user selects the corresponding newly added image target based on the selected newly added calibration point, the user can also use the dragging or clicking method to obtain the newly added image target. Specifically, based on the operation of the user dragging the non-calibration point radar target selected in the radar image to the video image, the non-calibration point radar target is taken as a newly added calibration point, the current position of each newly added calibration point dragged to the video image is determined, and the image target corresponding to the current position is obtained as the newly added image target corresponding to the newly added calibration point. Alternatively, based on the clicking operation of the user on the image target in the video image for each newly added calibration point selected, the image target clicked and selected is determined as the newly added image target corresponding to each newly added calibration point.

[0124] As a specific implementation, after the user determines the newly added calibration point, the user can move the newly added calibration point to the corresponding image target by the mouse or touch screen dragging method. The system can determine the position of the calibration point moved to as the image target corresponding to the calibration point. Of course, in the embodiment of the present application, the user can also select the newly added calibration point and the image target corresponding to the newly added calibration point by the clicking method. Specifically, the user can select a newly added calibration point by clicking the mouse or screen, and click the image target corresponding to the newly added calibration point in the image target image by clicking the mouse or screen. The system can determine the image target clicked by the user after selecting the newly added calibration point as the image target corresponding to the calibration point, and the user can click the image target corresponding to each newly added calibration point in the image target image after selecting each newly added calibration point.

[0125] In an embodiment of the present application, the image target can include a first image target with a target frame detected by an image detection algorithm or a second image target not detected by the image detection algorithm.

[0126] In this way, after the user selects the calibration point or adds the calibration point from the radar target data, the first image target and / or the second image target clicked by the user in the video image can be determined as the image target corresponding to each calibration point based on the click operation of the user on the selected calibration point. Alternatively, the first image target and / or the second image target clicked by the user in the video image can be determined as the newly added image target corresponding to the newly added calibration point based on the click operation of the user on the selected newly added calibration point.

[0127] For example, if the corresponding image target is determined by the mouse or touch screen dragging method, if the image target is the first target image of the target frame detected by the image detection algorithm, the user can drag the corresponding radar target to the lower edge center position of the image target frame; if the image target is the second target image of the target frame not detected by the image detection algorithm, the user can directly drag the corresponding radar target to the center position below the second target image.

[0128] If the corresponding image target is determined by the click method, the user can directly click the target frame of the first target image or the second target image, so as to determine the target in the target frame or the second target image corresponding to the target as the image target.

[0129] In addition, in the embodiment, for the target far away in the video image, the size of the target on the image is small, and the user cannot see the target clearly. In the process of determining the image target or the newly added image target, the user can input the image local magnification instruction for the local image, and the image local magnification instruction is used to magnify the local image, so that the user can determine the image target corresponding to the calibration point or the newly added calibration point accurately.

[0130] In the embodiment of the present application, after the newly added calibration point and the corresponding newly added image target are selected, the newly added calibration point and the corresponding newly added image target can be taken as a newly added matching target pair, and a new calibration matrix can be calculated based on the image coordinate information and the radar coordinate information of the newly added matching target pair and the original matching target pair, and the projection of each radar target in the radar image is performed again to determine whether the calibration effect meets the requirements.

[0131] Based on the above Figure 4 For example, as Figure 6As shown, since the projection positions of the non-calibration points 6, 7, and 8 are greatly different from the positions of the corresponding image targets, the three non-calibration points 6, 7, and 8 with large deviations can be dragged to the central positions of the lower edges of the corresponding image targets, and the radar targets 6, 7, and 8 are added to the calibration points. The calibration matrix is recalculated based on the calibration points 1, 2, 3, 5, 6, 7, and 8 to obtain a new current calibration matrix.

[0132] As shown in FIG. 6, Figure 7 As shown, based on the new calibration matrix, each radar target is projected into the video image. The projection positions of the calibration points 1, 2, 3, 5, 6, 7, and 8, and the non-calibration point 4 are all relatively small compared with the positions of the corresponding image targets, and thus the calibration meets the requirements and the calibration is completed.

[0133] As shown in FIG. 6, Figure 8 Figure 8 FIG. 7 is a schematic diagram of an actual calibration result in an embodiment of the present application. In the figure, the radar target frame (the smaller frame on the vehicle) and the image target frame (the larger frame on the vehicle) of each vehicle are coincident, that is, the calibration result meets the requirements and achieves a good calibration effect.

[0134] In the embodiment of the present application, the above-mentioned new calibration points can be manually selected by a user or realized through automatic target matching.

[0135] It can be seen that, in the traditional calibration method of the radar and the video acquisition device, the calibration personnel need to have certain professional knowledge to select a suitable quadrilateral and estimate the coordinates of each target point in the radar coordinate system. Moreover, the calibration personnel need to constantly adjust the coordinate values of the target points according to the calibration effect, and thus it is difficult to give a calibration result in a short time. In comparison, the calibration method of the radar and the video acquisition device provided in the embodiment of the present application calibrates based on the actually collected data, and does not require the calibration personnel to estimate the coordinate values of the radar targets, so that the calibration result is more accurate. Therefore, the user can quickly and conveniently calibrate the video acquisition device and the radar without certain professional knowledge, and the calibration method can be used in various scenes.

[0136] Based on the same inventive concept as the above-mentioned calibration method of the radar and the video acquisition device, an embodiment of the present application further provides a calibration device of a radar and a video acquisition device, as shown in FIG. 8, which can include: Figure 9

[0137] The data acquisition module 910 can be configured to acquire radar data and video data collected by the radar and the video acquisition device for the same scene.

[0138] The intercepting module 920 can be configured to intercept, based on the radar data and the video data, a radar image containing multiple radar targets and a video image at the same time containing multiple image targets.​​

[0139] The display module 930 can be configured to display the video image and the radar image side by side on a display.

[0140] The matching target pair determination module 940 can be configured to obtain at least four non-collinear radar targets selected by a user as calibration points based on current display content, and image targets corresponding to the calibration points, and take the calibration points and the corresponding image targets as matching target pairs.

[0141] The calibration matrix calculation module 950 can be configured to calculate a current calibration matrix based on image coordinate information and radar coordinate information of the matching target pairs.

[0142] The coordinate conversion module 960 can be configured to perform coordinate conversion on each radar target in the radar image based on the current calibration matrix to obtain converted position information of each radar target in an image coordinate system.

[0143] The calibration result acquisition module 970 can be configured to generate corresponding radar target position markers based on the converted position information of each radar target, display each radar target position marker in the video image as a calibration result.

[0144] The new matching target pair acquisition module 980 can be configured to obtain at least one non-calibration point radar target selected by a user as a new calibration point based on current display content in a case where the user determines that the calibration result does not meet requirements, and new image targets corresponding to the new calibration points, and take the new calibration points and the corresponding new image targets as new matching target pairs; return to the step of calculating the current calibration matrix based on the image coordinate information and the radar coordinate information of the matching target pairs until the user determines that the calibration result meets requirements.

[0145] The calibration device of the radar and video acquisition equipment provided in the embodiment of the present application intercepts a radar image containing a plurality of radar targets and a video image at the same time containing a plurality of image targets corresponding to the radar image from radar data and video data collected by the radar and video acquisition equipment for the same scene, and displays the video image and the radar image side by side. Then, at least four non-collinear radar targets selected by a user are obtained as calibration points, and image targets corresponding to the calibration points are determined. The calibration points and the corresponding image targets are taken as matching target pairs. Based on image coordinate information and radar coordinate information of each matching target pair, a current calibration matrix is calculated. Based on the current calibration matrix, coordinate conversion is performed on each radar target in the radar image to obtain position information of each radar target in the image coordinate system after conversion. Based on the position information, corresponding radar target position markers are generated and displayed in the video image as a calibration result. If the calibration result does not meet the requirements, at least one non-calibration point radar target selected by the user based on the current display content is obtained as a new calibration point, and a new image target corresponding to each new calibration point is obtained. The new calibration point and the corresponding new image target are taken as a new matching target pair. Based on each matching target pair, a calibration matrix is recalculated to obtain a calibration result until the user determines that the calibration result meets the requirements. In the embodiment of the present application, the user can determine the calibration points and the corresponding image targets from the radar data and the video image at the same time for the same scene displayed side by side, without the user giving accurate radar coordinates and image coordinates of the calibration points. This makes it possible to calibrate based on the actual collected data during calibration, thereby improving the accuracy and operability of the calibration of the radar and video acquisition equipment.

[0146] As a specific implementation of the embodiment of the present application, the matching target pair determination module 940 can be used to determine the image targets corresponding to the calibration points based on the operation of the user dragging at least four non-collinear radar targets in the radar image to the video image, taking the at least four radar targets as the calibration points, and obtaining the image targets at the current positions of the image targets dragged to the video image as the image targets corresponding to the calibration points.

[0147] Or,

[0148] Based on the clicking operation of the user on the image targets in the video image for each selected calibration point, the image target clicked is determined as the image target corresponding to each calibration point.

[0149] As a specific implementation of the embodiment of the present application, the newly added matching target pair acquisition module 980 can be used to determine, as a newly added image target corresponding to each newly added calibration point, an image target corresponding to a current position to which each newly added non-calibration point radar target selected by the user is dragged in the video image.

[0150] Or,

[0151] Based on the clicking operation of the user on the image target in the video image for each selected newly added calibration point, the image target clicked and selected is determined as the newly added image target corresponding to each newly added calibration point.

[0152] As a specific implementation of the embodiment of the present application, the image target includes a first image target detected by an image detection algorithm or a second image target not detected by the image detection algorithm.

[0153] The matching target pair determination module 940 can be used to obtain at least four non-collinear radar targets selected by the user as calibration points based on the current display content.

[0154] Based on the clicking operation of the user on the first image target and / or the second image target in the video image for each selected calibration point, the first image target and / or the second image target clicked and selected is determined as the image target corresponding to each calibration point.

[0155] The newly added matching target pair acquisition module 980 can be used to obtain at least one non-calibration point radar target selected by the user as a newly added calibration point based on the current display content.

[0156] Based on the clicking operation of the user on the first image target and / or the second image target in the video image for each selected newly added calibration point, the first image target and / or the second image target clicked and selected is determined as the newly added image target corresponding to the newly added calibration point.

[0157] As a specific implementation of the embodiment of the present application, the matching target pair determination module 940 can also be used to perform zoom-in operation on the image part based on the image part zoom-in instruction input by the user for the part of the image that is not clear; and / or,

[0158] The newly added matching target pair acquisition module can also be used to perform zoom-in operation on the image part based on the image part zoom-in instruction input by the user for the part of the image that is not clear.

[0159] The embodiment of the present application also provides an electronic device, such as Figure 10As shown, the device includes a processor 1001, a communication interface 1002, a memory 1003 and a communication bus 1004, wherein the processor 1001, the communication interface 1002 and the memory 1003 communicate with each other through the communication bus 1004,

[0160] The memory 1003 is used to store computer programs.

[0161] The processor 1001 is used to execute the programs stored in the memory 1003 to realize the following steps:

[0162] Obtain radar data and video data collected by a radar and a video collection device for the same scene;

[0163] Based on the radar data and the video data, intercept a radar image containing multiple radar targets and a video image at the same time containing multiple image targets corresponding to the radar image;

[0164] Display the video image and the radar image side by side on a display;

[0165] Obtain at least four non-collinear radar targets selected by a user based on the current display content as calibration points, and image targets corresponding to the calibration points, and take the calibration points and the corresponding image targets as matching target pairs;

[0166] Based on the image coordinate information and the radar coordinate information of the matching target pairs, calculate a current calibration matrix;

[0167] Based on the current calibration matrix, perform coordinate conversion on each radar target in the radar image to obtain converted position information of each radar target in an image coordinate system;

[0168] Based on the converted position information of each radar target, generate corresponding radar target position markers, and display each radar target position marker in the video image as a calibration result;

[0169] When the user determines that the calibration result does not meet the requirements, obtain at least one non-calibration point radar target selected by the user based on the current display content as a new calibration point, and new image targets corresponding to each new calibration point, and take the new calibration point and the corresponding new image target as a new matching target pair; return to the step of calculating the current calibration matrix based on the image coordinate information and the radar coordinate information of the matching target pairs until the user determines that the calibration result meets the requirements.

[0170] The radar and video acquisition device calibration method provided in the embodiment of the present application intercepts a radar image containing a plurality of radar targets and a video image of the same time containing a plurality of image targets from radar data and video data collected by the radar and video acquisition device for the same scene, and displays the video image and the radar image side by side, then acquires at least four non-collinear radar targets selected by a user as calibration points, determines the image targets corresponding to the calibration points, takes the calibration points and the corresponding image targets as matching target pairs, calculates a current calibration matrix based on the image coordinate information and the radar coordinate information of each matching target pair, performs coordinate conversion on each radar target in the radar image based on the current calibration matrix to obtain the converted position information of each radar target in the image coordinate system, generates corresponding radar target position markers based on the position information, and displays each radar target position marker in the video image as a calibration result, if the calibration result does not meet the requirements, acquires at least one non-calibration point radar target selected by the user based on the current display content as a new calibration point, and each new image target corresponding to each new calibration point, takes the new calibration point and the corresponding new image target as a new matching target pair, and recalculates the calibration matrix based on each matching target pair to obtain the calibration result until the user determines that the calibration result meets the requirements. In the embodiment of the present application, the user can determine the calibration points and the corresponding image targets from the radar data and the video image of the same time for the same scene displayed side by side, without the user giving the accurate radar coordinates and image coordinates of the calibration points, so that the calibration can be based on the actual collected data, thereby improving the accuracy and operability of the calibration of the radar and video acquisition device.

[0171] The communication bus mentioned above can be a Peripheral Component Interconnect (PCI) bus or an Extended Industry Standard Architecture (EISA) bus, etc. The communication bus can be divided into an address bus, a data bus, a control bus, etc. For convenience of representation, only one thick line is used in the figure, but it does not mean that there is only one bus or one type of bus.

[0172] The communication interface is used for communication between the above-mentioned electronic device and other devices.

[0173] The memory can include a Random Access Memory (RAM) and can also include a Non-Volatile Memory (NVM), such as at least one disk memory. Optionally, the memory can also be at least one storage device located away from the aforementioned processor.

[0174] The processor described above can be a general processor, including a central processing unit (CPU), a network processor (NP), etc.; can also be a digital signal processor (DSP), an application specific integrated circuit (ASIC), a field-programmable gate array (FPGA) or other programmable logic device, a discrete gate or transistor logic device, a discrete hardware component.

[0175] In another embodiment provided by the present application, a computer readable storage medium is provided, and the computer readable storage medium stores a computer program. The computer program is executed by a processor to implement the steps of the calibration method of the radar and video acquisition device in any of the above embodiments.

[0176] In another embodiment provided by the present application, a computer program product is provided, and the computer program product includes instructions. When the computer program product is executed on a computer, the computer is caused to execute the calibration method of the radar and video acquisition device in any of the above embodiments.

[0177] In the above embodiments, all or part of the embodiments can be implemented by software, hardware, firmware or any combination thereof. When implemented by software, all or part of the embodiments can be implemented in the form of a computer program product. The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, all or part of the processes or functions described in the embodiments of the present application are generated. The computer can be a general-purpose computer, a special-purpose computer, a computer network or other programmable device. The computer instructions can be stored in a computer readable storage medium or transmitted from one computer readable storage medium to another computer readable storage medium, for example, the computer instructions can be transmitted from one website, computer, server or data center to another website, computer, server or data center through wired (such as coaxial cable, optical fiber, digital subscriber line (DSL)) or wireless (such as infrared, wireless, microwave, etc.) mode. The computer readable storage medium can be any available medium that can be accessed by a computer or a data storage device such as a server, data center, etc. integrated with one or more available media. The available medium can be a magnetic medium (for example, floppy disk, hard disk, magnetic tape), an optical medium (for example, DVD), or a semiconductor medium (for example, solid state disk (SSD)) and the like.

[0178] It is to be noted that, in the present text, the relative terms such as first and second and the like are used merely to differentiate one entity or operation from another entity or operation, without necessarily requiring or implying any such actual relationship or order between such entities or operations. Moreover, the terms "comprising", "including", or any other variant thereof are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements does not include only those elements but can also include other elements not expressly listed or inherent to such process, method, article, or apparatus. Without further limitation, an element preceded by "comprises a" does not exclude the existence of additional identical elements in the process, method, article, or apparatus that comprises the recited element.

[0179] Each of the embodiments in the present specification is described in a related manner, and the same or similar parts between the embodiments can be referred to each other. Each of the embodiments focuses on the difference from other embodiments. In particular, for the device, electronic device, storage medium, and program product embodiments, since they are basically similar to the method embodiments, the description is relatively simple, and the relevant parts can be referred to the part of the method embodiment.

[0180] The above only describes the preferred embodiments of the present application and is not intended to limit the protection scope of the present application. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present application shall be included in the protection scope of the present application.

Claims

1. A calibration method for radar and video acquisition equipment, characterized in that, include: Acquire radar data and video data collected by radar and video acquisition devices for the same scene; Based on the radar data and video data, capture radar images containing multiple radar targets and corresponding video images containing multiple image targets at the same moment. The video images and radar images are displayed side by side on the monitor; Obtain at least four non-collinear radar targets selected by the user based on the currently displayed content as calibration points, and the image targets corresponding to each calibration point, and use the calibration points and the corresponding image targets as matching target pairs; Based on the image coordinate information and radar coordinate information of each matched target pair, calculate the current calibration matrix; Based on the current calibration matrix, coordinate transformation is performed on each radar target in the radar image to obtain the transformed position information of each radar target in the image coordinate system; Based on the converted position information of each radar target, a corresponding radar target position marker is generated, and the radar target position markers are displayed in the video image as calibration results. If the user determines that the calibration result does not meet the requirements, at least one non-calibration point radar target selected based on the current display content is used as a new calibration point, and the new image targets corresponding to each new calibration point are used as new matching target pairs. The process of calculating the current calibration matrix based on the image coordinate information and radar coordinate information of each matched target pair continues until the user confirms that the calibration result meets the requirements.

2. The method according to claim 1, characterized in that, The step of obtaining at least four non-collinear radar targets selected by the user based on the currently displayed content as calibration points, and the image targets corresponding to each calibration point, includes: Based on the user's operation of dragging at least four non-collinear radar targets in the radar image to the video image, the at least four radar targets are used as calibration points. The current position of each calibration point in the video image is determined, and the image target corresponding to the current position is obtained as the image target corresponding to the calibration point. or, Based on the user's click operation on the image target in the video image at each selected calibration point, the clicked image target is identified as the image target corresponding to each calibration point.

3. The method according to claim 1, characterized in that, The step of obtaining at least one non-calibration point radar target selected by the user as a new calibration point based on the currently displayed content when the calibration result does not meet the requirements, and the new image targets corresponding to each new calibration point, includes: Based on the user's operation of dragging a non-calibrated radar target selected in the radar image into the video image, the non-calibrated radar target is taken as a new calibration point. The position of each new calibration point is determined to be dragged to the current position in the video image, and the image target corresponding to the current position is obtained as the new image target corresponding to the new calibration point. or, Based on the user's click operation on the image target in the video image for each newly added calibration point, the clicked image target is identified as the newly added image target corresponding to each newly added calibration point.

4. The method according to claim 1, characterized in that, The image target includes: a first image target whose bounding box is detected by an image detection algorithm or a second image target that is not detected by an image detection algorithm; The step of obtaining at least four non-collinear radar targets selected by the user based on the currently displayed content as calibration points, and the image targets corresponding to each calibration point, and using the calibration points and the corresponding image targets as matching target pairs, includes: Obtain at least four non-collinear radar targets selected by the user based on the currently displayed content as calibration points; Based on the user's click operation on the first image target and / or the second image target in the video image for each selected calibration point, the first image target and / or the second image target that is clicked and selected is determined as the image target corresponding to each calibration point; The steps of selecting at least one non-calibration point radar target based on the current display content as a new calibration point, and the new image targets corresponding to each new calibration point, include: The system obtains at least one non-calibration radar target selected by the user based on the currently displayed content as a new calibration point. Based on the user's click operation on the first image target and / or the second image target in the video image for each selected newly added calibration point, the clicked first image target and / or the second image target are determined as the newly added image target corresponding to the newly added calibration point.

5. The method according to claim 1, characterized in that, The step of obtaining at least four non-collinear radar targets selected by the user based on the current display content as calibration points, and the image targets corresponding to each calibration point, further includes: Based on the user's input of a local image magnification command for a blurry area of ​​the image, perform a magnification operation on that area; and / or, The steps of selecting at least one non-calibration point radar target as a new calibration point based on the current display content, and the new image targets corresponding to each new calibration point, further include: Based on the user's input of a local image magnification command for a blurry part of the image, the magnification operation is performed on that part of the image.

6. A calibration device for radar and video acquisition equipment, characterized in that, include: The data acquisition module is used to acquire radar data and video data collected by radar and video acquisition devices for the same scene; The interception module is used to intercept radar images containing multiple radar targets and corresponding video images containing multiple image targets at the same moment, based on the radar data and video data. A display module is used to display the video images and radar images side by side on a display screen; The target pair determination module is used to obtain at least four non-collinear radar targets selected by the user based on the current display content as calibration points, and the image targets corresponding to each calibration point, and to use the calibration points and the corresponding image targets as a target pair; The calibration matrix calculation module is used to calculate the current calibration matrix based on the image coordinate information and radar coordinate information of each matched target pair; The coordinate transformation module is used to perform coordinate transformation on each radar target in the radar image based on the current calibration matrix, so as to obtain the transformed position information of each radar target in the image coordinate system; The calibration result acquisition module is used to generate corresponding radar target position markers based on the converted position information of each radar target, and display each radar target position marker in the video image as the calibration result; A new matching target pair acquisition module is added to obtain at least one non-calibration point radar target selected by the user based on the currently displayed content as a new calibration point when the calibration result does not meet the requirements, as well as the new image targets corresponding to each new calibration point, and the new calibration point and the corresponding new image targets are used as new matching target pairs. The process of calculating the current calibration matrix based on the image coordinate information and radar coordinate information of each matched target pair continues until the user confirms that the calibration result meets the requirements.

7. The apparatus according to claim 6, characterized in that, The matching target pair determination module is used to determine the current position of the image of each calibration point in the video image based on the user's operation of dragging at least four non-collinear radar targets in the radar image to the video image, and obtain the image target corresponding to the current position as the image target corresponding to the calibration point. or, Based on the user's click operation on the image target in the video image at each selected calibration point, the clicked image target is identified as the image target corresponding to each calibration point.

8. The apparatus according to claim 6, characterized in that, The newly added matching target pair acquisition module is used to, based on the user's operation of dragging a non-calibrated radar target selected in the radar image into the video image, take the non-calibrated radar target as a new calibration point, determine the current position of each new calibration point dragged to the video image, and obtain the image target corresponding to the current position as the new image target corresponding to the new calibration point. or, Based on the user's click operation on the image target in the video image for each newly added calibration point, the clicked image target is identified as the newly added image target corresponding to each newly added calibration point.

9. The apparatus according to claim 6, characterized in that, The image target includes: a first image target whose bounding box is detected by an image detection algorithm or a second image target that is not detected by an image detection algorithm; The target pair determination module is used to obtain at least four non-collinear radar targets selected by the user based on the currently displayed content as calibration points; Based on the user's click operation on the first image target and / or the second image target in the video image for each selected calibration point, the first image target and / or the second image target that is clicked and selected is determined as the image target corresponding to each calibration point; The newly added target pair acquisition module is used to obtain at least one non-calibration point radar target selected by the user based on the currently displayed content as a new calibration point; Based on the user's click operation on the first image target and / or the second image target in the video image for each selected newly added calibration point, the clicked first image target and / or the second image target are determined as the newly added image target corresponding to the newly added calibration point.

10. The apparatus according to claim 6, characterized in that, The matching target pair determination module is further configured to perform a magnification operation on a local area of ​​the image based on a user's input of a local magnification command for a blurry part of the image; and / or, The newly added matching target pair acquisition module is also used to perform a magnification operation on a local part of the image based on the user's input of an image magnification command for a local part of the image that is not clear.

11. An electronic device, characterized in that, It includes a processor, a communication interface, a memory, and a communication bus, wherein the processor, the communication interface, and the memory communicate with each other through the communication bus; Memory, used to store computer programs; A processor, when executing a program stored in memory, implements the steps of the method described in any one of claims 1-5.

12. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program that, when executed by a processor, implements the steps of the method described in any one of claims 1-5.

Images