Automatic calibration method, device and equipment for radar and medium

Abstract

The application discloses a radar and video automatic calibration method, device, equipment and medium. The method comprises the following steps: determining a coordinate conversion model based on a perspective transformation matrix, wherein the coordinate conversion model comprises the relationship among a plurality of coordinate conversion parameters, to-be-converted coordinates and converted coordinates; determining a candidate coordinate conversion parameter combination according to the initial range and step length of the plurality of coordinate conversion parameters, and determining the coordinate matching parameter corresponding to the candidate coordinate conversion parameter combination according to the to-be-measured radar coordinates and to-be-measured video coordinates based on the coordinate conversion model; and determining a target coordinate conversion parameter combination from the candidate coordinate conversion parameter combination according to the coordinate matching parameter. The application determines the coordinate conversion parameter by determining the covering relationship between the video coordinates and the radar coordinates through the coordinate conversion model, solves the problem of difficulty in determining the corresponding relationship between the video coordinates and the radar coordinates, and improves the efficiency and accuracy of radar and video calibration in a complex scene.

Description

Technical Field

[0001] This invention relates to the field of radar signal technology, and in particular to an automatic radar calibration method, apparatus, equipment, and medium. Background Technology

[0002] With the development of intelligent transportation technology, the detection effect of a single sensor can no longer meet the increasingly higher demands. At present, the fusion of multiple sensors can improve the accuracy of roadside perception, provide richer perception data, thereby accelerating the implementation of vehicle-road cooperation and improving traffic safety. For example, radar-visual integrated machines are gradually becoming the mainstream traffic sensors.

[0003] Since radar sensors and video sensors acquire data relatively independently, their acquisition coordinate systems are inconsistent. In order to describe the acquired information of both sensors as the same object, it is necessary to obtain the correspondence between their coordinate system transformations, that is, to perform joint calibration between different sensors.

[0004] Existing calibration methods are mainly divided into manual calibration and automatic calibration. Manual calibration requires experienced operators, and each device needs to be manually calibrated, which is labor-intensive and time-consuming. Most current automatic calibration methods rely on finding multiple radar coordinates and video coordinates with corresponding relationships, and determining the calibration result based on multiple sets of correspondences. However, due to the complexity of the detection scene, it is difficult to determine the radar coordinates and video coordinates with corresponding relationships. If the correspondence is determined incorrectly, it will cause the calibration result to be incorrect, which in turn will affect the fusion result between sensors. Summary of the Invention

[0005] This invention provides an automatic calibration method, apparatus, device, and medium for radar and image acquisition equipment to solve the problems of difficult and inaccurate calibration between radar equipment and image acquisition equipment.

[0006] According to one aspect of the present invention, an automatic radar calibration method is provided, applicable to radar equipment and image acquisition equipment with overlapping detection areas, comprising:

[0007] The coordinate transformation model is determined based on the perspective transformation matrix, wherein the coordinate transformation model includes multiple coordinate transformation parameters, coordinates to be transformed, and the relationship between the transformed coordinates;

[0008] Candidate coordinate transformation parameter combinations are determined based on the initial range and step size of the plurality of coordinate transformation parameters, and coordinate matching parameters corresponding to the candidate coordinate transformation parameter combinations are determined based on the coordinate transformation model and the radar coordinates and video coordinates to be tested; wherein, the radar coordinates and video coordinates to be tested are acquired by the radar device and the image acquisition device at the same time from the detection targets in the overlapping detection area;

[0009] The target coordinate transformation parameter combination is determined from the candidate coordinate transformation parameter combinations based on the coordinate matching parameters.

[0010] According to another aspect of the present invention, an automatic radar calibration device is provided, applicable to radar equipment and image acquisition equipment having overlapping detection areas, comprising:

[0011] The model determination module is used to determine the coordinate transformation model based on the perspective transformation matrix, wherein the coordinate transformation model includes multiple coordinate transformation parameters, coordinates to be transformed, and the relationship between the transformed coordinates;

[0012] The coordinate matching parameter determination module is used to determine candidate coordinate transformation parameter combinations based on the initial range and step size of the plurality of coordinate transformation parameters, and to determine the coordinate matching parameters corresponding to the candidate coordinate transformation parameter combinations based on the coordinate transformation model and the radar coordinates and video coordinates to be tested; wherein, the radar coordinates and video coordinates to be tested are acquired by the radar device and the image acquisition device at the same time from the detection targets in the overlapping detection area;

[0013] The coordinate transformation parameter determination module is used to determine the target coordinate transformation parameter combination from the candidate coordinate transformation parameter combinations based on the coordinate matching parameters.

[0014] According to another aspect of the present invention, an electronic device is provided, the electronic device comprising:

[0015] At least one processor; and

[0016] A memory communicatively connected to the at least one processor; wherein,

[0017] The memory stores a computer program that can be executed by the at least one processor, which enables the at least one processor to perform the automatic calibration method for radar vision according to any embodiment of the present invention.

[0018] According to another aspect of the present invention, a computer-readable storage medium is provided, the computer-readable storage medium storing computer instructions for causing a processor to execute and implement the automatic calibration method for radar vision according to any embodiment of the present invention.

[0019] The technical solution of this invention determines the coordinate transformation parameters by determining the coverage relationship between video coordinates and radar coordinates through a coordinate transformation model, which solves the problem of difficulty in determining the correspondence between video coordinates and radar coordinates and improves the efficiency and accuracy of radar calibration in complex scenarios.

[0020] It should be understood that the description in this section is not intended to identify key or essential features of the embodiments of the present invention, nor is it intended to limit the scope of the invention. Other features of the invention will become readily apparent from the following description. Attached Figure Description

[0021] To more clearly illustrate the technical solutions in the embodiments of the present invention, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0022] Figure 1 This is a flowchart of an automatic radar calibration method provided in Embodiment 1 of the present invention;

[0023] Figure 2 This is a flowchart of another automatic radar calibration method provided in Embodiment 2 of the present invention;

[0024] Figure 3 This is a flowchart of another automatic radar calibration method provided by an embodiment of the present invention;

[0025] Figure 4 This is a schematic diagram of the structure of an automatic radar calibration device according to Embodiment 3 of the present invention;

[0026] Figure 5 This is a schematic diagram of the structure of an electronic device that implements the automatic calibration method for radar vision according to an embodiment of the present invention. Detailed Implementation

[0027] To enable those skilled in the art to better understand the present invention, the technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings of the embodiments of the present invention. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort should fall within the scope of protection of the present invention.

[0028] It should be noted that the terms "candidate," "target," etc., used in the specification, claims, and accompanying drawings of this invention are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that embodiments of the invention described herein can be implemented in orders other than those illustrated or described herein. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover a non-exclusive inclusion; for example, a process, method, system, product, or apparatus that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus.

[0029] Example 1

[0030] Figure 1 This is a flowchart of an automatic radar-visual calibration method provided in Embodiment 1 of the present invention. This embodiment is applicable to the joint calibration of radar equipment and image acquisition equipment, and is applied to radar equipment and image acquisition equipment with overlapping detection areas. The method can be executed by an automatic radar-visual calibration device, which can be implemented in hardware and / or software. This automatic radar-visual calibration device can be configured in a radar-visual integrated machine or server, or other devices with computing capabilities. Figure 1 As shown, the method includes:

[0031] S110. Determine the coordinate transformation model based on the perspective transformation matrix, wherein the coordinate transformation model includes multiple coordinate transformation parameters, the coordinates to be transformed, and the relationship between the transformed coordinates.

[0032] The perspective transformation matrix transforms coordinates from one coordinate system to another. The purpose of joint calibration of radar and video sensors is to unify the information collected by the two sensors in the same coordinate system. Therefore, it is necessary to obtain the correspondence between the two coordinate system transformations.

[0033] The perspective transformation matrix can be expressed as the following formula:

[0034]

[0035] Where X, Y, and Z represent the three coordinates after the transmission transformation, i.e., the transformed coordinates, and x and y represent the two-dimensional coordinates before the transmission transformation, i.e., the coordinates to be transformed. Since the video coordinate system has strong visibility, in this embodiment of the invention, the coordinates to be transformed are radar coordinates and the transformed coordinates are video coordinates, so as to unify the radar coordinates under the video coordinate system.

[0036] Since the video coordinate system is a two-dimensional coordinate system, by calculating the above perspective transformation matrix and using the Z coordinate to transform the transformed coordinates to a two-dimensional coordinate system, we obtain the following expression:

[0037]

[0038] Where x′ and y′ are the transformed coordinates after transmission transformation (i.e., video coordinates).

[0039] Since this embodiment uses a perspective transformation matrix for transformation in a two-dimensional coordinate system, for ease of calculation, c3 = 1, thus obtaining the coordinate transformation model. The expression for the coordinate transformation model is as follows:

[0040]

[0041] Where (x,y) are the coordinates to be transformed, (x′,y′) are the transformed coordinates, and the coordinate transformation parameter is any parameter in the set {a1,a2,a3,b1,b2,b3,c1,c2}.

[0042] Specifically, the process of calibrating video coordinates and radar coordinates is the process of determining all coordinate transformation parameters in the coordinate transformation model.

[0043] S120. Determine the candidate coordinate transformation parameter combination based on the initial range and step size of multiple coordinate transformation parameters, and determine the coordinate matching parameters corresponding to the candidate coordinate transformation parameter combination based on the coordinate transformation model and the radar coordinates and video coordinates to be measured.

[0044] The radar coordinates and video coordinates to be measured are obtained by the radar equipment and the image acquisition equipment simultaneously from the detected target in the overlapping detection area. Since the radar coordinates and video coordinates are acquired in their respective coordinate systems, to establish the correspondence between them, the target object corresponding to the coordinates needs to be located in the overlapping detection area of ​​the two devices. Furthermore, to ensure consistency in positional correspondence, the corresponding target objects need to be from the same moment in time. For example, for ease of calculation, the radar coordinates and video coordinates to be measured are the centroid coordinates of the target vehicle in the overlapping detection area. That is, the vehicle centroid in the overlapping detection area is determined by the radar detection results as the radar coordinates to be measured, and the vehicle centroid in the overlapping detection area is determined by the image acquisition results as the video coordinates to be measured.

[0045] In the coordinate transformation model, each coordinate transformation parameter has its own initial range and step size. The initial range can be determined based on the theoretical parameters of the radar and image acquisition equipment. That is, the theoretical coordinate transformation parameters are determined during the manufacturing process of the radar and image acquisition equipment. However, due to errors in actual manufacturing and application, deviations in the theoretical parameters may occur. Therefore, different devices need to be individually calibrated during actual use to ensure the accuracy of the calibration results. Since the deviations will not deviate too much from the theoretical parameters, to improve calibration efficiency, an initial range for the coordinate transformation parameters is determined based on the theoretical parameters, and the final accurate coordinate transformation parameters are determined within this initial range. The step size can be determined based on the actual calibration accuracy requirements and the length of the initial range, and is not limited here. For example, the step size for all coordinate transformation parameters can be the same or different.

[0046] Specifically, the value of each coordinate transformation parameter is determined according to its initial range and step size. These values ​​are then combined to obtain candidate coordinate transformation parameter combinations. For example, if there are 8 coordinate transformation parameters, each with an initial range of [0,5] and a step size of 1, then each parameter has 6 possible values, resulting in 6 candidate coordinate transformation parameter combinations. 8 There are 6 results. Substituting different combinations of candidate coordinate transformation parameters into the coordinate transformation model yields 6. 8 Several different candidate coordinate transformation models.

[0047] If the combination of candidate coordinate transformation parameters in the candidate coordinate transformation model is correct, then there will be a one-to-one correspondence between the radar coordinates to be measured and the video coordinates to be measured. That is, the closer the candidate coordinate transformation model is to the final result, the higher the proportion of successful matching between the radar coordinates to be measured and the video coordinates to be measured after model transformation and matching. Therefore, the correctness of the candidate coordinate transformation parameters can be determined based on the coordinate matching parameters between the radar coordinates to be measured and the video coordinates to be measured. The coordinate matching parameters characterize the matching ratio between the radar coordinates to be measured and the video coordinates to be measured.

[0048] In a feasible embodiment, based on a coordinate transformation model, coordinate matching parameters corresponding to candidate coordinate transformation parameter combinations are determined according to the radar coordinates to be measured and the video coordinates to be measured, including:

[0049] The coordinates of the radar to be tested are determined as the coordinates to be transformed, and the coordinates of the video to be tested are determined as the transformed coordinates; or, the coordinates of the video to be tested are determined as the coordinates to be transformed, and the coordinates of the radar to be tested are determined as the transformed coordinates.

[0050] The set of regions to be covered is determined based on the transformed coordinates; wherein, the set of regions to be covered includes multiple regions to be covered, and each region to be covered includes at least one transformed coordinate;

[0051] The matching coordinates corresponding to the coordinates to be transformed are determined based on the candidate coordinate transformation model corresponding to the candidate coordinate transformation parameter combination.

[0052] The coordinate matching parameters corresponding to the candidate coordinate transformation parameter combinations are determined based on the number of matching regions in the target area to be covered set; wherein, the target area to be covered is the area to be covered that contains at least one coordinate to be matched.

[0053] According to the coordinate transformation model, the radar coordinates to be measured can be either the coordinates to be transformed or the transformed coordinates. Correspondingly, the video coordinates to be measured are either the transformed coordinates or the coordinates to be transformed. For example, based on the above example, this embodiment uses the radar coordinates to be transformed as the coordinates to be measured and the video coordinates to be measured as the transformed coordinates for illustration.

[0054] The area to be covered is the region obtained by expanding the matching error range based on the transformed coordinates. Since the radar coordinates and video coordinates to be measured are obtained based on different sensor principles and different sensor measurement accuracies, even if they correspond to the same target, there may be a certain error range. Therefore, an error range is set according to the device's recognition accuracy. That is, the area within the error range expanded outward from the transformed coordinates belongs to the area to be covered corresponding to those transformed coordinates. For example, if the coordinates to be transformed are radar coordinates and the transformed coordinates are video coordinates, then the video coordinate system is divided into m*n regions based on the video coordinates. If a region contains more than or equal to one video coordinate to be measured, then that region is the area to be covered, and the set of these areas to be covered is called the area to be covered set. The coordinates to be matched are the coordinate information corresponding to the coordinates to be transformed after being transformed by the candidate coordinate transformation model. That is, if the coordinates to be matched after being transformed by the candidate coordinate transformation model are located in any area to be covered, then it is determined that the coordinates to be transformed match the transformed coordinates corresponding to that area to be covered, i.e., it is determined that the radar coordinates to be measured match the video coordinates to be measured corresponding to that area to be covered.

[0055] For example, a candidate coordinate transformation model is determined based on the combination of candidate coordinate transformation parameters. The coordinates of the radar to be measured are substituted into the right side of the equation of the candidate coordinate transformation model to obtain the coordinates to be matched. If the candidate coordinate transformation model is the final model, then the coordinates to be matched should match the coordinates of the video to be measured corresponding to the coordinates of the radar to be measured. Therefore, the area to be covered that includes at least one coordinate to be matched is determined as the target area to be covered, and the number of target areas to be covered in the set of areas to be covered is determined. The coordinate matching parameters are determined based on this number, and the candidate coordinate transformation models are traversed to determine the corresponding coordinate matching parameters for each area.

[0056] S130. Determine the target coordinate transformation parameter combination from the candidate coordinate transformation parameter combinations based on the coordinate matching parameters.

[0057] Since the coordinate matching parameter characterizes the matching degree between the coordinates of the video to be tested and the coordinates of the radar to be tested, the candidate coordinate transformation parameter combination corresponding to the largest coordinate matching parameter is determined as the target coordinate transformation parameter combination, and the target coordinate transformation model determined based on this target coordinate transformation parameter combination is the radar-visual calibration result.

[0058] In one feasible embodiment, S130 includes:

[0059] The candidate coordinate transformation parameter combination with the largest corresponding coordinate matching parameter among the candidate coordinate transformation parameter combinations is determined as the candidate coordinate transformation parameter combination corresponding to the radar coordinates to be tested and the video coordinates to be tested.

[0060] If there are a preset number of sets of radar coordinates and video coordinates to be measured that have the same combination of candidate coordinate transformation parameters, then that candidate coordinate transformation parameter combination is determined to be the target coordinate transformation parameter combination.

[0061] Input the coordinate set data corresponding to the radar coordinates and video coordinates of a frame to be tested into the candidate coordinate transformation model corresponding to the candidate coordinate transformation parameter combination. The candidate coordinate transformation parameter combination with the largest coordinate matching parameter is the candidate coordinate transformation parameter combination corresponding to the coordinate set data of that frame.

[0062] After determining the candidate coordinate transformation parameter combinations corresponding to a frame coordinate group, the candidate coordinate transformation parameter combinations are stored in the parameter table. It is then determined whether there is any candidate coordinate transformation parameter combination in the parameter table whose quantity is greater than the preset quantity. If so, it means that the candidate coordinate transformation parameter combination meets the requirements and is the target coordinate transformation parameter combination.

[0063] For example, the candidate coordinate transformation parameter combination corresponding to the radar coordinates and video coordinates to be tested in the first frame is called combination 1, and the candidate coordinate transformation parameter combination corresponding to the radar coordinates and video coordinates to be tested in the second frame is called combination 2. Combination 1, combination 2 and the candidate coordinate transformation parameter combination of subsequent frames are stored in a parameter table. It is determined whether the number of occurrences of combination 1 and combination 2 in the parameter table is greater than a preset number threshold. If it is greater, combination 1 or combination 2 is determined to be the target coordinate transformation parameter combination.

[0064] Since a candidate coordinate transformation parameter combination is the one that achieves the highest matching degree between the radar coordinates and the video coordinates in a single frame, but due to the existence of randomness, if a certain candidate coordinate transformation parameter combination achieves the highest matching degree among multiple frames of radar coordinates and video coordinates, the randomness can be eliminated, and it can be determined as the target coordinate transformation parameter combination, thus improving the accuracy of the calibration results.

[0065] In a feasible embodiment, after determining the candidate coordinate transformation parameter combination with the largest corresponding coordinate matching parameter among the candidate coordinate transformation parameter combinations as the candidate coordinate transformation parameter combination corresponding to the radar coordinates to be measured and the video coordinates to be measured, the method further includes:

[0066] If the number of possible coordinate transformation parameter combinations reaches the convergence count of any coordinate transformation parameter, then the current value range of the coordinate transformation parameter is converged based on the matching frequency of each value of the coordinate transformation parameter within the possible coordinate transformation parameter combinations, to obtain the convergence range of the coordinate transformation parameter; wherein, the convergence count is determined based on the number of values ​​in the initial range of the coordinate transformation parameter and the preset confidence level;

[0067] The candidate coordinate transformation parameter combinations are updated based on the convergence range of the coordinate transformation parameters;

[0068] Based on the updated candidate coordinate transformation parameter combinations, determine the next set of candidate coordinate transformation parameter combinations corresponding to the radar coordinates and video coordinates to be tested.

[0069] To improve the efficiency of traversing candidate coordinate transformation parameters, the range of candidate coordinate transformation parameter combinations is narrowed during the traversal process based on the aforementioned matching results, that is, the matching range of coordinate transformation parameters is converged to reduce computation time.

[0070] Specifically, the number of candidate coordinate transformation parameter combinations represents the number of calculations required for the radar coordinates and video coordinates under test. That is, once the number of candidate coordinate transformation parameter combinations reaches the convergence count of any one of the coordinate transformation parameters, the range corresponding to that coordinate transformation parameter is converged. This converged range is smaller than the initial range, and the ratio of the sum of the occurrences of each value within the converged range in the candidate coordinate transformation parameter combinations to the sum of the occurrences of each value within the unconverged range is greater than a preset confidence level. Candidate coordinate transformation parameter combinations outside the convergence range are eliminated, reducing the number of candidate coordinate transformation parameter combinations that need to be calculated subsequently and improving the calculation efficiency of coordinate matching parameters.

[0071] For example, after determining the candidate coordinate transformation parameter combinations corresponding to the radar coordinates to be measured in a radar image frame and the video coordinates to be measured in the corresponding video image frame, it is determined whether the current number of calculation frames has reached the convergence count for any coordinate transformation parameter. For example, if the convergence count for the first coordinate transformation parameter is 5 times and the convergence count for the second coordinate transformation parameter is 8 times, then after calculating 5 radar video image sets, the range of the first coordinate transformation parameter is converged once; after calculating 8 radar video image sets, the range of the second coordinate transformation parameter is converged once; and subsequently, after calculating 10 radar video image sets, the range of the first coordinate transformation parameter is converged again, and so on. Optionally, since the convergence operation also consumes time, and the range of values ​​for the coordinate transformation parameters will decrease after the convergence operation, thereby reducing the computational time of the solution, in order to balance these two time consumptions, the coordinate transformation parameter is converged every... A convergence operation is performed after each solution operation. That is, the convergence number of the coordinate transformation parameters is... Where α is the preset confidence level, which can be determined according to the calibration accuracy and is not limited here; a is the initial number of values ​​for the coordinate transformation parameter, which is determined according to the initial range and step size.

[0072] In a feasible embodiment, the convergence range of the coordinate transformation parameter is obtained by converging the current value range of the coordinate transformation parameter based on the matching frequency of each value of the coordinate transformation parameter within the selected coordinate transformation parameter combination, including:

[0073] The convergence range is determined based on the following conditions:

[0074] and

[0075] Among them, [p min1 ,p max1 [p] represents the current value range. min2 ,p max2 [ ] represents the convergence range, f i For [p] min2 ,p max2 The matching frequency of the i-th value in the matrix, f j For [p] min1 ,p max1 The matching frequency of the j-th value in the equation, where α is the pre-set confidence level.

[0076] Specifically, the frequency of each value in the candidate coordinate transformation parameter combination is determined as the matching frequency. Since the candidate coordinate transformation parameter combination is the one with the highest coordinate matching parameter in a frame of radar coordinates and video coordinates, that is, the parameter combination that is closest to the final result, the more frequently any value of any coordinate transformation parameter appears in the candidate coordinate transformation parameter combination, the closer it is to the final result.

[0077] Specifically, according to and A defined convergence range ensures that the values ​​within this range cover most of the hit values, guaranteeing both a narrower range after convergence, reducing computational load, and that excluded values ​​do not contribute to the final result. The range of coordinate transformation parameters in the subsequent solution process converges to the interval [p]. min2 ,p max2 The step size remains unchanged.

[0078] The technical solution of this invention determines the coordinate transformation parameters by determining the coverage relationship between video coordinates and radar coordinates through a coordinate transformation model, which solves the problem of difficulty in determining the correspondence between video coordinates and radar coordinates and improves the efficiency and accuracy of radar calibration in complex scenarios.

[0079] Example 2

[0080] Figure 2 This is a flowchart of an automatic radar calibration method provided in Embodiment 2 of the present invention. In this embodiment, the coordinate transformation parameters in the above embodiments are grouped. The coordinate transformation parameters include a first coordinate transformation parameter and a second coordinate transformation parameter. The first coordinate transformation parameter is any parameter in the set {a1,a2,b1,b2,c1,c2}, and the second coordinate transformation parameter is any parameter in the set {a3,b3}. Figure 2 As shown, the method includes:

[0081] S210. Determine the coordinate transformation model based on the perspective transformation matrix.

[0082] S220. Based on the element information in the coordinate transformation model, deform the coordinate transformation model to obtain the deformed model of the coordinate transformation model.

[0083] The deformation model includes a first coordinate transformation parameter, and the coordinate transformation model includes a first coordinate transformation parameter and a second coordinate transformation parameter.

[0084] As shown in the coordinate transformation model above, the denominators of the two equations have the same information. Therefore, the coordinate transformation model can be transformed to obtain a deformed model. Specifically, the expression of the deformed model is:

[0085]

[0086] From this deformation model, we can see that it includes only 6 coordinate transformation parameters: {a1,a2,b1,b2,c1,c2}. Therefore, based on the deformation model, the coordinate transformation parameters are divided into the first coordinate transformation parameter and the second coordinate transformation parameter.

[0087] Specifically, as shown in the above embodiments, if there are a total of 8 coordinate transformation parameters, each with an initial range of [0,5] and a step size of 1, then each coordinate transformation parameter has 6 possible values, resulting in 6 candidate coordinate transformation parameter combinations. 8 The result is that for each frame of the radar coordinates and video coordinates to be measured, 6... 8 There are several coordinate matching parameters. However, if the coordinate transformation parameters are divided, and the first coordinate transformation parameter is determined based on the deformation model, then only two unknown coordinate transformation parameters exist in the coordinate transformation model. Then, the second coordinate transformation parameter is determined, reducing the number of calculations to 6. 6 +6 2 The number of calculations was drastically reduced, thus improving computational efficiency.

[0088] S230. Determine the candidate coordinate transformation parameter combination based on the initial range and step size of multiple coordinate transformation parameters.

[0089] S240. Based on the coordinate transformation model, the deformed model determines the first coordinate matching parameter corresponding to the candidate first coordinate transformation parameter combination in the candidate coordinate transformation parameter combination according to the radar coordinates to be measured and the video coordinates to be measured.

[0090] Based on the candidate first coordinate transformation parameter combination of the first coordinate transformation parameter, the corresponding candidate deformation model is determined. The radar coordinates to be tested and the video coordinates to be tested are substituted into the candidate deformation model, and the first coordinate matching parameter of each candidate deformation model is determined respectively.

[0091] For example, the set R = {(x1,y1),(x2,y2)...(x...} n ,y n Let V = {(x′1,y′1),(x′2,y′2)…(x′1,y′1),} represent the radar target to be measured, that is, the set of coordinates of all targets captured in a single radar frame; set V = {(x′1,y′1),(x′2,y′2)…(x′1,y′2)} m ,y′ m Let V represent the coordinates of the video to be tested, that is, the set of coordinates of all targets captured in a single frame of the video. The coordinates in V are calculated using the left-hand side of the deformation model to obtain a set of points. By setting the parameter ε according to the device's recognition accuracy, the set of areas to be covered can be obtained. Substituting the coordinates in R into the right-hand side of the deformation model, we can obtain a set of points A′ containing the first coordinate transformation parameter. By iterating through A′ according to the candidate first coordinate transformation parameter combinations, we can determine the number of regions in the region set P to be covered corresponding to each candidate first coordinate transformation parameter combination that contain more than or equal to 1 points in A′. This number is then determined as the first coordinate matching parameter corresponding to that candidate first coordinate transformation parameter combination.

[0092] S250. Determine the target first coordinate transformation parameter combination from the candidate first coordinate transformation parameter combinations based on the first coordinate matching parameters.

[0093] The candidate first coordinate transformation parameter combination with the largest corresponding coordinate matching parameter among the candidate first coordinate transformation parameter combinations is determined as the target first coordinate transformation parameter combination corresponding to the radar coordinates to be measured and the video coordinates to be measured.

[0094] Optionally, the target first coordinate transformation parameter combination is determined from the candidate first coordinate transformation parameter combinations based on the first coordinate matching parameters, including:

[0095] The candidate first coordinate transformation parameter combination with the largest first coordinate matching parameter among the candidate first coordinate transformation parameter combinations is determined as the first candidate coordinate transformation parameter combination corresponding to the radar coordinates to be tested and the video coordinates to be tested.

[0096] If there are a preset number of sets of radar coordinates and video coordinates to be measured that have the same first candidate coordinate transformation parameter combination, then the first candidate coordinate transformation parameter combination is determined as the target first coordinate transformation parameter combination.

[0097] Specifically, after traversing all candidate first coordinate transformation parameter combinations, the candidate first coordinate transformation parameter combination with the largest first coordinate matching parameter is recorded in the first parameter table as the first candidate coordinate transformation parameter combination. Optionally, if multiple candidate first coordinate transformation parameter combinations have the same first coordinate matching parameter and are the maximum value, all of these candidate first coordinate transformation parameter combinations are recorded in the first parameter table. Since the first candidate coordinate transformation parameter combination should theoretically equal the number of coordinates in the video to be tested when it is the final result, but considering that there is a certain error in the device's coordinate recognition, it can be considered that the first parameter table records several sets of approximate values ​​close to the true value. Therefore, after determining the first candidate coordinate transformation parameter combination corresponding to a frame of coordinates, the first candidate coordinate transformation parameter combination is stored in the first parameter table, and it is determined whether there is any first candidate coordinate transformation parameter combination in the first parameter table whose number is greater than the preset number. If so, it means that the first candidate coordinate transformation parameter combination meets the requirements and is the target first coordinate transformation parameter combination. If not, the calculation of the first coordinate matching parameters of the next frame of video coordinates and radar coordinates to be tested continues.

[0098] Optionally, after determining the candidate first coordinate transformation parameter combination with the largest first coordinate matching parameter among the candidate first coordinate transformation parameter combinations as the first candidate coordinate transformation parameter combination corresponding to the radar coordinates to be measured and the video coordinates to be measured, the method further includes:

[0099] If the number of combinations of first candidate coordinate transformation parameters reaches the convergence count of any first coordinate transformation parameter, then the current value range of the first coordinate transformation parameter is converged according to the matching frequency of each value of the first coordinate transformation parameter within the first candidate coordinate transformation parameter combination, to obtain the convergence range of the first coordinate transformation parameter; wherein, the convergence count is determined based on the number of values ​​in the initial range of the first coordinate transformation parameter and the preset confidence level;

[0100] The candidate combinations of first coordinate transformation parameters are updated based on the convergence range of the first coordinate transformation parameters;

[0101] Based on the updated candidate first coordinate transformation parameter combination, determine the first candidate coordinate transformation parameter combination corresponding to the next set of radar coordinates and video coordinates to be tested.

[0102] S260. Based on the coordinate transformation model that determines the first coordinate transformation parameter combination of the target, determine the second coordinate matching parameter corresponding to the candidate second coordinate transformation parameter combination in the candidate coordinate transformation parameter combination according to the radar coordinates to be measured and the video coordinates to be measured.

[0103] After determining the first coordinate transformation parameter, there are two unknown coordinate transformation parameters in the coordinate transformation model, which are the second coordinate transformation parameters. Based on the candidate combinations of the second coordinate transformation parameters, the corresponding second candidate coordinate transformation model is determined. The radar coordinates and video coordinates to be measured are substituted into the second candidate coordinate transformation model, and the second coordinate matching parameter for each second candidate coordinate transformation model is determined.

[0104] For example, based on the above example, according to the range and threshold of (x′, y′) in the video coordinate set V to be tested, the video coordinate system is divided into m*n regions. If any region contains more than or equal to 1 coordinate from V, the set of these regions is called the region set Q to be covered. Substituting each coordinate in R into the right-hand side of the second candidate coordinate transformation model, a set of points V′ containing the second coordinate transformation parameters can be obtained. By iterating through V′ according to the candidate second coordinate transformation parameter combinations, the number of regions in the region set Q to be covered corresponding to each candidate second coordinate transformation parameter combination that contain more than or equal to 1 points from V′ is determined, and this number is determined as the second coordinate matching parameter corresponding to that candidate second coordinate transformation parameter combination.

[0105] S270. Determine the target second coordinate transformation parameter combination from the candidate second coordinate transformation parameter combinations based on the second coordinate matching parameters.

[0106] The candidate second coordinate transformation parameter combination with the largest corresponding coordinate matching parameter among the candidate second coordinate transformation parameter combinations is determined as the target second coordinate transformation parameter combination corresponding to the radar coordinates to be measured and the video coordinates to be measured.

[0107] Optionally, the target second coordinate transformation parameter combination is determined from the candidate second coordinate transformation parameter combinations based on the second coordinate matching parameters, including:

[0108] The candidate second coordinate transformation parameter combination with the largest corresponding second coordinate matching parameter among the candidate second coordinate transformation parameter combinations is determined as the second candidate coordinate transformation parameter combination corresponding to the radar coordinates to be tested and the video coordinates to be tested.

[0109] If there are preset groups of radar coordinates and video coordinates to be measured that have the same second candidate coordinate transformation parameter combination, then the second candidate coordinate transformation parameter combination is determined as the target second coordinate transformation parameter combination.

[0110] Specifically, after traversing all candidate second coordinate transformation parameter combinations, the candidate second coordinate transformation parameter combination with the largest second coordinate matching parameter is recorded in the second parameter table as the second candidate coordinate transformation parameter combination. Optionally, if multiple sets of candidate second coordinate transformation parameter combinations have the same second coordinate matching parameter and are the maximum value, then all these multiple sets of candidate second coordinate transformation parameter combinations are recorded in the second parameter table. Since the second candidate coordinate transformation parameter combination should theoretically equal the number of coordinates in the video to be tested when it is the final result, but considering that there is a certain error in the device's coordinate recognition, it can be considered that the second parameter table records several sets of approximate values ​​close to the true value. Therefore, after determining the second candidate coordinate transformation parameter combination corresponding to a frame of coordinates, the second candidate coordinate transformation parameter combination is stored in the second parameter table, and it is determined whether there is any second candidate coordinate transformation parameter combination in the second parameter table whose number is greater than the preset number. If so, it means that the second candidate coordinate transformation parameter combination meets the requirements and is the target second coordinate transformation parameter combination. If not, the calculation of the second coordinate matching parameters of the next frame of video coordinates and radar coordinates to be tested continues.

[0111] Optionally, after determining the candidate second coordinate transformation parameter combination with the largest second coordinate matching parameter among the candidate second coordinate transformation parameter combinations as the second candidate coordinate transformation parameter combination corresponding to the radar coordinates to be measured and the video coordinates to be measured, the method further includes:

[0112] If the number of combinations of second candidate coordinate transformation parameters reaches the convergence count of any second coordinate transformation parameter, then the current value range of the second coordinate transformation parameter is converged according to the matching frequency of each value of the second coordinate transformation parameter within the combination of second candidate coordinate transformation parameters, to obtain the convergence range of the second coordinate transformation parameter; wherein, the convergence count is determined based on the number of values ​​in the initial range of the second coordinate transformation parameter and the preset confidence level;

[0113] The candidate combinations of second coordinate transformation parameters are updated based on the convergence range of the second coordinate transformation parameters;

[0114] Based on the updated candidate second coordinate transformation parameter combinations, determine the next set of second candidate coordinate transformation parameter combinations corresponding to the radar coordinates and video coordinates to be measured.

[0115] The technical solution of this invention reduces calibration time, improves calibration speed, and enhances compatibility in complex scenarios by determining coordinate transformation parameters in groups.

[0116] Figure 3 A flowchart of a preferred embodiment of the automatic calibration method for radar vision provided by the present invention is shown below. Figure 3 As shown, among them<R,V> The set of target coordinates in the synchronized radar frame and video frame is defined by the Num operation, which is the first coordinate transformation parameter operation, table T1 is the first parameter table, the Convergence operation is the parameter convergence operation, the Den operation is the second coordinate transformation parameter operation, and table T2 is the second parameter table.

[0117] Specifically, the radar coordinates and video coordinates to be measured are obtained from a synchronized radar frame and video frame.<R,V> The first coordinate transformation parameter is calculated based on a deformable model of the coordinate transformation model. Specifically, the first coordinate matching parameter is determined based on the radar coordinates and video coordinates to be measured, identifying the first coordinate matching parameter corresponding to the candidate first coordinate transformation parameter combination. After iterating through all candidate first coordinate transformation parameter combinations, the candidate first coordinate transformation parameter combination with the largest first coordinate matching parameter is recorded in the first parameter table as the first candidate coordinate transformation parameter combination. (Single frame)<R,V> After the corresponding Num operation is completed, it is determined whether there exists a set of candidate coordinate transformation parameter combinations in the first parameter table whose record count is greater than or equal to a preset threshold. If so, this set of candidate coordinate transformation parameter combinations is identified as the target value of the first coordinate transformation parameter; otherwise, the next frame is obtained.<R,V> Perform Num operations.

[0118] Proceeding to the next frame<R,V> Before the Num operation, a parameter convergence operation is performed on the first coordinate transformation parameter. Optionally, if the number of first candidate coordinate transformation parameter combinations in the first parameter table reaches the convergence count of any first coordinate transformation parameter, then the current value range of the first coordinate transformation parameter is converged according to the matching frequency of each value of the first coordinate transformation parameter within the first candidate coordinate transformation parameter combination to obtain the convergence range of the first coordinate transformation parameter; wherein, the convergence count is determined based on the number of values ​​in the initial range of the first coordinate transformation parameter and a preset confidence level; the candidate first coordinate transformation parameter combinations are updated based on the convergence range of the first coordinate transformation parameter; based on the updated candidate first coordinate transformation parameter combinations, the next frame is determined.<R,V> Num operations.

[0119] After Num calculation, a set of target first coordinate transformation parameters that meet the expectations are determined. These parameters are then substituted into the coordinate transformation model to obtain a model containing only the unknown second coordinate transformation parameters. Den calculation is performed based on this model, determining the second coordinate matching parameter corresponding to the candidate second coordinate transformation parameter combination from among the candidate coordinate transformation parameter combinations based on the radar coordinates and video coordinates to be measured. After traversing all candidate second coordinate transformation parameter combinations, the candidate second coordinate transformation parameter combination with the largest second coordinate matching parameter is recorded in the second parameter table as the second candidate coordinate transformation parameter combination. (Single frame)<R,V> After the corresponding Den operation is completed, it is determined whether there exists a set of second candidate coordinate transformation parameter combinations in the second parameter table whose record count is greater than or equal to a preset threshold. If so, this set of second candidate coordinate transformation parameter combinations is identified as the target value of the second coordinate transformation parameter; otherwise, the next frame is obtained.<R,V> Perform the Den operation.

[0120] Proceeding to the next frame<R,V> Before the Den operation, a parameter convergence operation is performed on the second coordinate transformation parameters. Optionally, if the number of second candidate coordinate transformation parameter combinations in the second parameter table reaches the convergence count of any second coordinate transformation parameter, then the current value range of the second coordinate transformation parameter is converged based on the matching frequency of each value of the second coordinate transformation parameter within the second candidate coordinate transformation parameter combination to obtain the convergence range of the second coordinate transformation parameter; wherein, the convergence count is determined based on the number of values ​​in the initial range of the second coordinate transformation parameter and a preset confidence level; the candidate second coordinate transformation parameter combinations are updated based on the convergence range of the second coordinate transformation parameter; based on the updated candidate second coordinate transformation parameter combinations, the next frame is determined.<R,V> Den operation.

[0121] For example, due to different frames<R,V> The Num and Den operations are incoherent, therefore they can be processed simultaneously across multiple frames using multiple threads.<R,V> Perform Num and Den operations to improve calibration efficiency.

[0122] Example 3

[0123] Figure 4 This is a schematic diagram of the structure of an automatic radar calibration device provided in Embodiment 3 of the present invention. Figure 4 As shown, this device is applied to radar equipment and image acquisition equipment with overlapping detection areas, and includes:

[0124] The model determination module 410 is used to determine the coordinate transformation model based on the perspective transformation matrix. The coordinate transformation model includes multiple coordinate transformation parameters, coordinates to be transformed, and the relationship between the transformed coordinates.

[0125] The coordinate matching parameter determination module 420 is used to determine candidate coordinate transformation parameter combinations based on the initial range and step size of the plurality of coordinate transformation parameters, and to determine the coordinate matching parameters corresponding to the candidate coordinate transformation parameter combinations based on the coordinate transformation model and the radar coordinates and video coordinates to be tested; wherein, the radar coordinates and video coordinates to be tested are acquired by the radar device and the image acquisition device at the same time from the detection targets in the overlapping detection area;

[0126] The coordinate transformation parameter determination module 430 is used to determine the target coordinate transformation parameter combination from the candidate coordinate transformation parameter combinations based on the coordinate matching parameters.

[0127] Optional, the coordinate matching parameter determination module is specifically used for:

[0128] The radar coordinates to be tested are determined as the coordinates to be converted, and the video coordinates to be tested are determined as the converted coordinates; or, the video coordinates to be tested are determined as the coordinates to be converted, and the radar coordinates to be tested are determined as the converted coordinates.

[0129] The set of regions to be covered is determined based on the transformed coordinates; wherein, the set of regions to be covered includes multiple regions to be covered, and each region to be covered includes at least one transformed coordinate;

[0130] The candidate coordinate transformation model corresponding to the candidate coordinate transformation parameter combination is used to determine the coordinate to be transformed and the coordinate to be matched. The coordinate matching parameter corresponding to the candidate coordinate transformation parameter combination is determined according to the number of matching areas of the target area to be covered in the target area set. The target area to be covered is an area to be covered that contains at least one coordinate to be matched.

[0131] Optionally, after determining the coordinate transformation model based on the perspective transformation matrix, the method further includes:

[0132] The coordinate transformation model is deformed based on the element information in the coordinate transformation model to obtain the deformed model of the coordinate transformation model;

[0133] The deformation model includes a first coordinate transformation parameter, and the coordinate transformation model includes a first coordinate transformation parameter and a second coordinate transformation parameter.

[0134] Optional, the coordinate matching parameter determination module is specifically used for:

[0135] Based on the deformed model of the coordinate transformation model, the first coordinate matching parameter corresponding to the candidate first coordinate transformation parameter combination in the candidate coordinate transformation parameter combination is determined according to the radar coordinates to be measured and the video coordinates to be measured.

[0136] The target first coordinate transformation parameter is determined from the candidate first coordinate transformation parameter combinations based on the first coordinate matching parameter;

[0137] Based on the coordinate transformation model that determines the first coordinate transformation parameters of the target, the second coordinate matching parameter is determined according to the radar coordinates to be measured and the video coordinates to be measured.

[0138] Optional, a coordinate transformation parameter determination module, specifically used for:

[0139] The candidate coordinate transformation parameter combination with the largest corresponding coordinate matching parameter among the candidate coordinate transformation parameter combinations is determined as the candidate coordinate transformation parameter combination corresponding to the radar coordinates to be tested and the video coordinates to be tested.

[0140] If there are a preset number of sets of radar coordinates and video coordinates to be measured that have the same combination of candidate coordinate transformation parameters, then the candidate coordinate transformation parameter combination is determined to be the target coordinate transformation parameter combination.

[0141] Optionally, the device further includes a parameter convergence module, comprising:

[0142] The parameter range convergence unit is used to converge the current value range of the coordinate transformation parameter based on the matching frequency of each value of the coordinate transformation parameter within the selected coordinate transformation parameter combination if the number of combinations of candidate coordinate transformation parameters reaches the convergence count of any coordinate transformation parameter, thereby obtaining the convergence range of the coordinate transformation parameter; wherein, the convergence count is determined based on the number of values ​​of the coordinate transformation parameter within the initial range and a preset confidence level;

[0143] The parameter combination update unit is used to update the candidate coordinate transformation parameter combinations with undetermined coordinate matching parameters based on the convergence range of the coordinate transformation parameter.

[0144] The coordinate matching parameter determination unit is used to determine the next set of candidate coordinate transformation parameter combinations corresponding to the radar coordinates and video coordinates to be measured, based on the updated candidate coordinate transformation parameter combinations.

[0145] Optional, parameter range convergence unit, specifically used for:

[0146] The convergence range is determined based on the following conditions:

[0147] and

[0148] Among them, [p min1 ,p max1 [p] represents the current value range. min2 ,p max2 [ ] represents the convergence range, f i For [p] min2 ,p max2 The matching frequency of the i-th value in the matrix, f j For [p] min1 ,p max1 The matching frequency of the j-th value in the equation, where α is the pre-set confidence level.

[0149] The automatic radar calibration device provided in this embodiment of the invention can execute the automatic radar calibration method provided in any embodiment of the invention, and has the corresponding functional modules and beneficial effects of the method.

[0150] The acquisition, storage, use, and processing of data in this application comply with relevant national laws and regulations and do not violate public order and good morals.

[0151] Example 4

[0152] According to embodiments of this disclosure, this disclosure also provides an electronic device, a readable storage medium, and a computer program product.

[0153] Figure 5A schematic diagram of an electronic device 10 that can be used to implement embodiments of the present invention is shown. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device can also represent various forms of mobile devices, such as personal digital processors, cellular phones, smartphones, wearable devices (e.g., helmets, glasses, watches, etc.), and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely illustrative and are not intended to limit the implementation of the invention described and / or claimed herein.

[0154] like Figure 5 As shown, the electronic device 10 includes at least one processor 11 and a memory, such as a read-only memory (ROM) 12 or a random access memory (RAM) 13, communicatively connected to the at least one processor 11. The memory stores computer programs executable by the at least one processor. The processor 11 can perform various appropriate actions and processes based on the computer program stored in the ROM 12 or loaded from storage unit 18 into the RAM 13. The RAM 13 may also store various programs and data required for the operation of the electronic device 10. The processor 11, ROM 12, and RAM 13 are interconnected via a bus 14. An input / output (I / O) interface 15 is also connected to the bus 14.

[0155] Multiple components in electronic device 10 are connected to I / O interface 15, including: input unit 16, such as keyboard, mouse, etc.; output unit 17, such as various types of displays, speakers, etc.; storage unit 18, such as disk, optical disk, etc.; and communication unit 19, such as network card, modem, wireless transceiver, etc. Communication unit 19 allows electronic device 10 to exchange information / data with other devices through computer networks such as the Internet and / or various telecommunications networks.

[0156] Processor 11 can be a variety of general-purpose and / or special-purpose processing components with processing and computing capabilities. Some examples of processor 11 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various special-purpose artificial intelligence (AI) computing chips, various processors running machine learning model algorithms, a digital signal processor (DSP), and any suitable processor, controller, microcontroller, etc. Processor 11 performs the various methods and processes described above, such as the method of automatic calibration.

[0157] In some embodiments, the automatic calibration method for radar vision can be implemented as a computer program tangibly contained in a computer-readable storage medium, such as storage unit 18. In some embodiments, part or all of the computer program can be loaded and / or mounted on electronic device 10 via ROM 12 and / or communication unit 19. When the computer program is loaded into RAM 13 and executed by processor 11, one or more steps of the automatic calibration method for radar vision described above can be performed. Alternatively, in other embodiments, processor 11 can be configured to perform automatic calibration method for radar vision by any other suitable means (e.g., by means of firmware).

[0158] Various embodiments of the systems and techniques described above herein can be implemented in digital electronic circuit systems, integrated circuit systems, field-programmable gate arrays (FPGAs), application-specific integrated circuits (ASICs), application-specific reference products (ASSPs), systems-on-a-chip (SoCs), complex programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various embodiments may include implementations in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which may be a dedicated or general-purpose programmable processor, capable of receiving data and instructions from a storage system, at least one input device, and at least one output device, and transferring data and instructions to the storage system, the at least one input device, and the at least one output device.

[0159] Computer programs used to implement the methods of the present invention may be written in any combination of one or more programming languages. These computer programs may be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing device, such that when executed by the processor, the computer programs cause the functions / operations specified in the flowcharts and / or block diagrams to be performed. The computer programs may be executed entirely on a machine, partially on a machine, or as a standalone software package, partially on a machine and partially on a remote machine, or entirely on a remote machine or server.

[0160] In the context of this invention, a computer-readable storage medium can be a tangible medium that may contain or store a computer program for use by or in conjunction with an instruction execution system, apparatus, or device. A computer-readable storage medium may include, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatus, or devices, or any suitable combination thereof. Alternatively, a computer-readable storage medium may be a machine-readable signal medium. More specific examples of machine-readable storage media include electrical connections based on one or more wires, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fibers, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination thereof.

[0161] To provide interaction with a user, the systems and techniques described herein can be implemented on an electronic device having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user; and a keyboard and pointing device (e.g., a mouse or trackball) through which the user provides input to the electronic device. Other types of devices can also be used to provide interaction with the user; for example, feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including sound input, voice input, or tactile input).

[0162] The systems and technologies described herein can be implemented in computing systems that include backend components (e.g., as data servers), or computing systems that include middleware components (e.g., application servers), or computing systems that include frontend components (e.g., user computers with graphical user interfaces or web browsers through which users can interact with implementations of the systems and technologies described herein), or any combination of such backend, middleware, or frontend components. The components of the system can be interconnected via digital data communication of any form or medium (e.g., communication networks). Examples of communication networks include local area networks (LANs), wide area networks (WANs), blockchain networks, and the Internet.

[0163] A computing system can include clients and servers. Clients and servers are generally located far apart and typically interact through communication networks. The client-server relationship is created by computer programs running on the respective computers and having a client-server relationship with each other. The server can be a cloud server, also known as a cloud computing server or cloud host, which is a hosting product within the cloud computing service system to address the shortcomings of traditional physical hosts and VPS services, such as high management difficulty and weak business scalability.

[0164] It should be understood that the various forms of processes shown above can be used, with steps reordered, added, or deleted. For example, the steps described in this invention can be executed in parallel, sequentially, or in different orders, as long as the desired result of the technical solution of this invention can be achieved, and no limitation is imposed herein.

[0165] The specific embodiments described above do not constitute a limitation on the scope of protection of this invention. Those skilled in the art should understand that various modifications, combinations, sub-combinations, and substitutions can be made according to design requirements and other factors. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of this invention should be included within the scope of protection of this invention.

Claims

1. An automatic calibration method for radar equipment, applied to radar devices and image acquisition devices with overlapping detection areas, characterized in that, include: The coordinate transformation model is determined based on the perspective transformation matrix, wherein the coordinate transformation model includes multiple coordinate transformation parameters, coordinates to be transformed, and the relationship between the transformed coordinates; Candidate coordinate transformation parameter combinations are determined based on the initial range and step size of the plurality of coordinate transformation parameters, and coordinate matching parameters corresponding to the candidate coordinate transformation parameter combinations are determined based on the coordinate transformation model and the radar coordinates and video coordinates to be tested; wherein, the radar coordinates and video coordinates to be tested are acquired by the radar device and the image acquisition device at the same time from the detection targets in the overlapping detection area; The target coordinate transformation parameter combination is determined from the candidate coordinate transformation parameter combinations based on the coordinate matching parameters; Specifically, based on the coordinate transformation model, the coordinate matching parameters corresponding to the candidate coordinate transformation parameter combinations are determined according to the radar coordinates to be measured and the video coordinates to be measured, including: The radar coordinates to be tested are determined as the coordinates to be converted, and the video coordinates to be tested are determined as the converted coordinates; or, the video coordinates to be tested are determined as the coordinates to be converted, and the radar coordinates to be tested are determined as the converted coordinates. The set of regions to be covered is determined based on the transformed coordinates; wherein, the set of regions to be covered includes multiple regions to be covered, and each region to be covered includes at least one transformed coordinate; Based on the candidate coordinate transformation model corresponding to the candidate coordinate transformation parameter combination, determine the matching coordinates corresponding to the coordinates to be transformed; The coordinate matching parameters corresponding to the candidate coordinate transformation parameter combination are determined based on the number of matching regions of the target area to be covered in the target area set; wherein, the target area to be covered is an area to be covered that contains at least one coordinate to be matched.

2. The method according to claim 1, characterized in that, After determining the coordinate transformation model based on the perspective transformation matrix, the method further includes: The coordinate transformation model is deformed based on the element information in the coordinate transformation model to obtain the deformed model of the coordinate transformation model; The deformation model includes a first coordinate transformation parameter, and the coordinate transformation model includes a first coordinate transformation parameter and a second coordinate transformation parameter.

3. The method according to claim 2, characterized in that, Based on the coordinate transformation model, coordinate matching parameters corresponding to the candidate coordinate transformation parameter combinations are determined according to the radar coordinates to be measured and the video coordinates to be measured, including: Based on the deformed model of the coordinate transformation model, the first coordinate matching parameter corresponding to the candidate first coordinate transformation parameter combination in the candidate coordinate transformation parameter combination is determined according to the radar coordinates to be measured and the video coordinates to be measured. The target first coordinate transformation parameter combination is determined from the candidate first coordinate transformation parameter combinations based on the first coordinate matching parameter; Based on the coordinate transformation model that determines the first coordinate transformation parameter combination of the target, the second coordinate matching parameter is determined according to the radar coordinates to be measured and the video coordinates to be measured.

4. The method according to any one of claims 1 to 3, characterized in that, Determining the target coordinate transformation parameter combination from the candidate coordinate transformation parameter combinations based on the coordinate matching parameters includes: The candidate coordinate transformation parameter combination with the largest corresponding coordinate matching parameter among the candidate coordinate transformation parameter combinations is determined as the candidate coordinate transformation parameter combination corresponding to the radar coordinates to be tested and the video coordinates to be tested. If there are a preset number of sets of radar coordinates and video coordinates to be measured that have the same combination of candidate coordinate transformation parameters, then the candidate coordinate transformation parameter combination is determined to be the target coordinate transformation parameter combination.

5. The method according to claim 4, characterized in that, After determining the candidate coordinate transformation parameter combination with the largest corresponding coordinate matching parameter among the candidate coordinate transformation parameter combinations as the candidate coordinate transformation parameter combination corresponding to the radar coordinates to be measured and the video coordinates to be measured, the method further includes: If the number of candidate coordinate transformation parameter combinations reaches the convergence count of any coordinate transformation parameter, then the current value range of the coordinate transformation parameter is converged based on the matching frequency of each value of the coordinate transformation parameter within the candidate coordinate transformation parameter combination to obtain the convergence range of the coordinate transformation parameter; wherein, the convergence count is determined based on the number of values ​​within the initial range of the coordinate transformation parameter and a preset confidence level; The candidate coordinate transformation parameter combination is updated based on the convergence range of the coordinate transformation parameter; Based on the updated candidate coordinate transformation parameter combinations, determine the next set of candidate coordinate transformation parameter combinations corresponding to the radar coordinates and video coordinates to be tested.

6. The method according to claim 5, characterized in that, The convergence range of the coordinate transformation parameter is obtained by converging the current value range of the coordinate transformation parameter based on the matching frequency of each value of the coordinate transformation parameter within the selected coordinate transformation parameter combination, including: The convergence range is determined based on the following conditions: and ; in, This represents the current value range. For the convergence range, for The matching frequency of the i-th value. for The matching frequency of the j-th value in the set. For pre-set reliability.

7. An automatic calibration device for radar, applied to radar equipment and image acquisition equipment with overlapping detection areas, characterized in that, include: The model determination module is used to determine the coordinate transformation model based on the perspective transformation matrix, wherein the coordinate transformation model includes multiple coordinate transformation parameters, coordinates to be transformed, and the relationship between the transformed coordinates; The coordinate matching parameter determination module is used to determine candidate coordinate transformation parameter combinations based on the initial range and step size of the plurality of coordinate transformation parameters, and to determine the coordinate matching parameters corresponding to the candidate coordinate transformation parameter combinations based on the coordinate transformation model and the radar coordinates and video coordinates to be tested; wherein, the radar coordinates and video coordinates to be tested are acquired by the radar device and the image acquisition device at the same time from the detection targets in the overlapping detection area; A coordinate transformation parameter determination module is used to determine a target coordinate transformation parameter combination from the candidate coordinate transformation parameter combinations based on the coordinate matching parameters; The coordinate matching parameter determination module is specifically used for: The radar coordinates to be tested are determined as the coordinates to be converted, and the video coordinates to be tested are determined as the converted coordinates; or, the video coordinates to be tested are determined as the coordinates to be converted, and the radar coordinates to be tested are determined as the converted coordinates. The set of regions to be covered is determined based on the transformed coordinates; wherein, the set of regions to be covered includes multiple regions to be covered, and each region to be covered includes at least one transformed coordinate; The candidate coordinate transformation model corresponding to the candidate coordinate transformation parameter combination is used to determine the coordinate to be transformed and the coordinate to be matched. The coordinate matching parameter corresponding to the candidate coordinate transformation parameter combination is determined according to the number of matching areas of the target area to be covered in the target area set. The target area to be covered is an area to be covered that contains at least one coordinate to be matched.

8. An electronic device, characterized in that, The electronic device includes: At least one processor; and A memory communicatively connected to the at least one processor; wherein, The memory stores a computer program that can be executed by the at least one processor, the computer program being executed by the at least one processor to enable the at least one processor to perform the automatic calibration method for radar vision as described in any one of claims 1-6.

9. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores computer instructions that cause a processor to execute the automatic calibration method for radar vision as described in any one of claims 1-6.

Images