Calibration Parameter Determination Method, Device, and Computer-Readable Storage Medium

Abstract

The present application discloses a method, device, and computer-readable storage medium for determining calibration parameters. The method includes: obtaining a frame matching pair including a target, where the frame matching pair includes a radar frame and an image frame respectively collected by a radar and a camera at the same moment; respectively performing target extraction on the radar frame and the image frame in the frame matching pair; performing target matching with the overlapping field of view region of the radar and the camera as a matching constraint condition to obtain a target matching pair; adding the target matching pair to a target matching pair set; and obtaining a position transformation matrix between the radar and the camera based on the target matching pair set. Through the above manner, the accuracy of the position transformation matrix can be improved, and the calibration parameter determination method provided by the present application is easy to implement and has low complexity.

Description

Technical Field

[0001] This application relates to the field of computer technology, and particularly to a method, device, and computer-readable storage medium for determining calibration parameters. Background Art

[0002] In the field of monitoring, the research direction is gradually shifting from single-dimensional perception (such as cameras or radars) to multi-dimensional perception (such as cameras and radars). The image frames collected by cameras have rich external features such as texture, color, and shape, and can perform tasks such as classification, recognition, and semantic segmentation. The radar frames collected by radars can achieve target detection and positioning tasks. Integrating radars and cameras can play a complementary role, obtain richer target attributes, and achieve richer tasks.

[0003] Before integrating radars and cameras to obtain the attributes of targets, parameter calibration is required. However, current methods for determining calibration parameters either have low accuracy, are difficult to implement, or have high complexity. Summary of the Invention

[0004] This application provides a method, device, and computer-readable storage medium for determining calibration parameters, which can solve the problems that existing methods for determining calibration parameters either have low accuracy, are difficult to implement, or have high complexity.

[0005] To solve the above technical problems, one technical solution adopted by this application is: to provide a method for determining calibration parameters. The method includes: obtaining a frame matching pair containing a target, where the frame matching pair includes a radar frame and an image frame respectively collected by a radar and a camera at the same moment; respectively extracting targets from the radar frame and the image frame in the frame matching pair; performing target matching with the overlapping field of view area of the radar and the camera as a matching constraint condition to obtain a target matching pair; adding the target matching pair to a target matching pair set; and obtaining a position transformation matrix between the radar and the camera based on the target matching pair set.

[0006] To solve the above technical problems, another technical solution adopted by this application is: to provide a calibration parameter device, which includes a processor and a memory connected to the processor. The memory stores program instructions; the processor is configured to execute the program instructions stored in the memory to implement the above method.

[0007] To solve the above technical problems, yet another technical solution adopted by this application is: to provide a computer-readable storage medium storing program instructions, which can implement the above method when the program instructions are executed.

[0008] In the above manner, the present application performs target extraction and target matching on the radar frame and the image frame in the frame matching pair to obtain the target matching pair, adds the target matching pair to the target matching pair set, and obtains the position transformation matrix between the radar and the camera based on the target matching pair set. Since the target matching is performed with the overlapping field of view area between the radar and the camera as the matching constraint condition, the obtained target matching pair is in the overlapping field of view area between the radar and the camera, so the accuracy of the target matching pair can be improved. Using the target matching pair for the calculation of the position transformation matrix can improve the accuracy of the position transformation matrix. Moreover, the calibration parameter determination method provided by the present application has a wide application range and does not require a specific calibration object, so it is easier to implement. In addition, the calibration parameter determination method provided by the present application has a simple process flow, reducing the complexity. BRIEF DESCRIPTION OF THE DRAWINGS

[0009] Figure 1 is a schematic diagram of the application scenario for calibration basis;

[0010] Figure 2 is a schematic flowchart of an embodiment of the calibration parameter determination method of the present application;

[0011] Figure 3 is Figure 2 the specific flowchart of S15 in

[0012] Figure 4 is a schematic diagram of the overlapping field of view area in the image frame;

[0013] Figure 5 is a schematic diagram of a preset geometric distribution;

[0014] Figure 6 is a schematic flowchart of another embodiment of the calibration parameter determination method of the present application;

[0015] Figure 7 is Figure 6 the specific flowchart of S25 in

[0016] Figure 8 is a schematic flowchart of yet another embodiment of the calibration parameter determination method of the present application;

[0017] Figure 9 is a schematic diagram of the overlapping field of view area in the radar frame and the image frame;

[0018] Figure 10 is a schematic flowchart of yet another embodiment of the calibration parameter determination method of the present application;

[0019] Figure 11 is a schematic flowchart of yet another embodiment of the calibration parameter determination method of the present application;

[0020] Figure 12It is a schematic diagram of the relationship of the overlapping area of the actual scene, the radar, and the camera's field of view;

[0021] Figure 13 It is a schematic flowchart of another embodiment of the calibration parameter determination method of the present application;

[0022] Figure 14 is Figure 13 The specific flowchart of S61 in

[0023] Figure 15 It is a schematic flowchart of a specific example of the calibration parameter determination method of the present application;

[0024] Figure 16 It is a schematic diagram of the target detection and fusion system;

[0025] Figure 17 It is a schematic structural diagram of an embodiment of the calibration parameter determination device of the present application;

[0026] Figure 18 It is a schematic structural diagram of an embodiment of the computer-readable storage medium of the present application. Detailed implementation manners

[0027] Next, the technical solutions in the embodiments of the present application will be clearly and completely described in conjunction with the accompanying drawings in the embodiments of the present application. Obviously, the described embodiments are only a part of the embodiments of the present application, rather than all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative efforts fall within the protection scope of the present application.

[0028] The terms "first", "second", and "third" in the present application are only for descriptive purposes, and cannot be understood as indicating or implying relative importance or implicitly specifying the quantity of the indicated technical features. Thus, the features defined with "first", "second", and "third" may explicitly or implicitly include at least one of such features. In the description of the present application, the meaning of "a plurality" is at least two, such as two, three, etc., unless otherwise specifically and clearly defined.

[0029] Referring to "embodiment" herein means that a specific feature, structure, or characteristic described in connection with the embodiment may be included in at least one embodiment of the present application. The phrase appears in various places in the specification and does not necessarily refer to the same embodiment, nor is it an independent or alternative embodiment mutually exclusive with other embodiments. Those skilled in the art will explicitly and implicitly understand that, without conflict, the embodiments described herein may be combined with other embodiments.

[0030] Before introducing the calibration parameter determination method provided by the present application, the relevant definitions of the actual scene based on which the calibration is performed are first explained:

[0031] The actual scenarios for calibration can be roads (lanes, sidewalks), conveyor belts, etc. The actual scenarios include a number of movement channels extending in a first direction and arranged side by side in a second direction, where the first direction is perpendicular to the second direction. Ideally, a target in the actual scenario moves in a movement channel in the first direction. The target can be a vehicle, a person, an item carried on a conveyor belt, etc.

[0032] Combined with Figure 1 Illustrate the actual scenario with an example, such as Figure 1 As shown, the actual scenario is a lane, which includes four lanes M1, M2, M3, and M4 extending in the first direction and arranged side by side in the second direction. The first direction is the extension direction of M1 / M2 / M3 / M4, and the second direction is perpendicular to the extension direction of M1 / M2 / M3 / M4.

[0033] The following introduces the calibration parameter determination method provided by this application:

[0034] Figure 2 It is a schematic flowchart of an embodiment of the calibration parameter determination method of this application. It should be noted that if there are substantially the same results, this embodiment is not limited to Figure 2 the process sequence shown. As Figure 2 shown, this embodiment may include:

[0035] S11: Obtain frame matching pairs containing the target.

[0036] Among them, the frame matching pairs include a radar frame and an image frame respectively collected by the radar and the camera at the same moment.

[0037] The execution subject of the method embodiment of this application is a calibration parameter determination device. The calibration parameter determination device can be a radar, a camera, a radar-vision device, an electronic device (such as a computer, a mobile phone) that establishes a communication connection with the radar or the camera, or a server, etc.

[0038] The radar can be a millimeter-wave radar, an ultrasonic radar, a lidar, etc. The fields of view of the radar and the camera at least partially overlap. The radar and the camera can be different modules in a radar-vision device, or can be separate independent devices. Both the radar frame and the image frame contain the target.

[0039] S12: Extract the target from the radar frame and the image frame in the frame matching pairs respectively.

[0040] Target extraction is performed on the radar frame, and the targets in the radar frame can be obtained. The targets in the radar frame can include the IDs of the targets in the radar frame and the positions of the targets in the radar frame (i.e., the position of the target in the three-dimensional coordinate system OX0Y0Z0 of the radar frame O(Ox, Oy, Oz)). Based on the IDs of the targets and the positions of the targets in a number of already acquired radar frames, the trajectories of the targets in the radar frame can be obtained.

[0041] Target extraction is performed on the image frame, and the targets in the image frame can be obtained. The targets in the image frame can include the IDs of the targets in the image frame and the positions of the targets in the image frame (i.e., the position of the target in the coordinate system of the image frame I(Ix, Iy)). Based on the IDs of the targets and the positions of the targets in a number of already acquired image frames, the trajectories of the targets in the image frame can be obtained.

[0042] S13: Target matching is performed with the overlapping area of the fields of view of the radar and the camera as the matching constraint condition to obtain target matching pairs.

[0043] Target matching is based on the positions of the targets in the radar frame and the image frame. Before applying the positions of the targets in S12 to target matching, a reference plane can be fitted based on the positions of the targets in the radar frame; the positions of the targets in the radar frame are converted from the three-dimensional coordinate system of the radar to the two-dimensional coordinate system of the reference plane (also known as the two-dimensional coordinate system of the radar). The reference plane is the plane (ground plane) on which the targets in the actual scene travel. For example, according to O i (Ox i , Oy i , Oz i ), a reference plane π can be fitted and the normal vector n of π can be obtained. A three-dimensional rotation and translation transformation is performed according to n to convert the positions of the targets in the radar frame from the three-dimensional coordinate system OX0Y0Z0 of the radar frame to the two-dimensional coordinate system X R Y R Z R of the radar, and R i (Rx i , Ry i ) is obtained:

[0044] R i (Rx i , Ry i , 0) = [R|t] * O i (Ox i , Oy i , Oz i ).

[0045] For the schematic illustration of converting the positions of the targets in the radar frame from the three-dimensional coordinate system of the radar to the two-dimensional coordinate system of the reference plane, reference can be made to Figure 12 mentioned later.

[0046] Target matching with the overlapping field of view region of the radar and the camera as the matching constraint condition means matching the targets in the overlapping field of view region in the radar frame and the image frame. A target matching pair consists of a pair of targets in the radar frame and the image frame that are in the overlapping field of view region and are matched.

[0047] The overlapping field of view region of the radar and the camera can be a priori overlapping field of view region or a posteriori overlapping field of view region. The a priori overlapping field of view region is obtained by pre - setting. The a posteriori overlapping field of view region is calculated based on the set of target matching pairs, and the number of target matching pairs included in the set of target matching pairs is greater than or equal to the first quantity threshold. That is, when the number of target matching pairs accumulated in the set of target matching pairs is greater than or equal to the first quantity threshold, the a posteriori overlapping field of view region is calculated based on the set of target matching pairs. For a detailed description of the set of target matching pairs, please refer to the description of the following steps.

[0048] S14: Add the target matching pair to the set of target matching pairs.

[0049] The target matching pairs in the set of target matching pairs are accumulated frame - by - frame. That is to say, for each acquired frame of matching pairs, after obtaining the target matching pairs in this frame of matching pairs, the target matching pairs are added to the set of target matching pairs. The set of target matching pairs consists of the target matching pairs in several acquired frames of matching pairs.

[0050] S15: Based on the set of target matching pairs, obtain the position transformation matrix between the radar and the camera.

[0051] The position transformation matrix is a homography matrix. For example, it is the homography matrix H, and the calculation formula can be as follows:

[0052] In some embodiments, the position transformation matrix can be obtained by performing multiple iterations of selecting the minimum model through the RANSAC algorithm.

[0053] In some embodiments, with reference to Figure 3 , S15 may include the following sub - steps:

[0054] S151: Select a second preset number of target matching pairs from the set of target matching pairs.

[0055] The target matching pairs can be selected arbitrarily, or the target matching pairs closest to the four edges of the overlapping field of view region (the lower edge and the upper edge along the first direction, and the lower edge and the upper edge along the second direction) can be selected. The second preset number can be 4, 5, etc.

[0056] In some embodiments, the second preset quantity is 4, and the second preset quantity of target matching pairs respectively have maximum or minimum values along the first direction or the second direction in the image frame, that is, the positions of the targets in the image frame included in the second preset quantity of target matching pairs respectively have maximum or minimum values along the first direction or the second direction. Alternatively, the second preset quantity of target matching pairs respectively have maximum or minimum values along the first direction or the second direction in the radar frame, that is, the positions of the targets in the radar frame included in the second preset quantity of target matching pairs respectively have maximum or minimum values along the first direction or the second direction. At this time, the selected target matching pairs can be called extreme value target matching pairs.

[0057] Combined with Figure 4 Illustrate by way of example. Figure 4 is a schematic diagram of the field of view coincidence area in the image frame, as Figure 4 shown, the field of view coincidence area includes 9 target matching pairs 0 to 8. Select the target matching pair 8 that has a minimum value along the first direction and a minimum value along the second direction in the image frame, the target matching pair 7 that has a minimum value along the first direction and a maximum value along the second direction in the image frame, the target matching pair 0 that has a maximum value along the first direction and a minimum value along the second direction in the image frame, and the target matching pair 2 that has a maximum value along the first direction and a maximum value along the second direction in the image frame.

[0058] S152: Determine whether the position distributions of the selected second preset quantity of target matching pairs in the image frame and in the radar frame both conform to the preset position distribution.

[0059] For example, referring to the targets in the radar frame as radar targets and the targets in the image frame as camera targets, the selected 4 target matching pairs are respectively target matching pair 1 (camera target 1, radar target 1), target matching pair 2 (camera target 2, radar target 2), target matching pair 3 (camera target 3, radar target 3), and target matching pair 4 (camera target 4, radar target 4), then determine whether the position distributions of camera targets 1 to 4 and the position distributions of radar targets 1 to 4 both conform to the preset position distribution.

[0060] It can be understood that the preset position distribution can be embodied as a preset shape, such as a quadrilateral or a pentagon, or can be embodied as a preset direction, such as a clockwise direction or a counterclockwise direction. The position distribution conforming to the preset position distribution, that is, the similarity between the position distribution and the preset combined distribution is greater than the similarity threshold, means that the selected second preset quantity of target matching pairs are sufficiently dispersed, which is used to obtain the position transformation matrix and can improve the robustness of the position transformation matrix. And, compared with the method of obtaining the position transformation matrix by RANSAC, it can reduce the time consumption.

[0061] If the preset position distribution is met, execute S153; if the preset position distribution is not met, execute S154.

[0062] S153: Obtain a position transformation matrix based on the selected second preset number of target matching pairs.

[0063] S154: Filter the selected second preset number of target matching pairs from the set of target matching pairs, and return to the step of obtaining frame matching pairs.

[0064] That is, in the case of not meeting the preset geometric distribution, filter the selected second preset number of target matching pairs, and obtain new frame matching pairs to repeat the foregoing steps until a position transformation matrix is obtained.

[0065] Through the implementation of this embodiment, the present application performs target extraction and target matching on the radar frame and the image frame in the frame matching pair to obtain target matching pairs, adds the target matching pairs to the set of target matching pairs, and obtains the position transformation matrix between the radar and the camera based on the set of target matching pairs. Since the target matching is performed with the overlapping field of view area of the radar and the camera as the matching constraint condition, the obtained target matching pairs are in the overlapping field of view area of the radar and the camera, so the accuracy of the target matching pairs can be improved. Using the target matching pairs for the calculation of the position transformation matrix can improve the accuracy of the position transformation matrix.

[0066] In addition, the calibration parameter determination method in the related art needs to be implemented in a laboratory environment (including specific calibration objects, such as checkerboards) before leaving the factory, and is only applicable to the case where the camera and the radar are different modules of a radar-vision device. The calibration parameter determination method provided by the present application is not only applicable to the case where the camera and the radar are different modules of a radar-vision device, but also applicable to the case where the camera and the radar are independently set (for example, a camera has been set on the road, and a radar needs to be set on the road. For cost considerations, instead of directly replacing the camera with a radar-vision device, a radar is added to the road). Therefore, the applicable range is wide; and it can be implemented without relying on specific calibration objects, and is more tolerant of the calibration scene. Therefore, the calibration parameter determination method provided by the present application is easier to implement.

[0067] In addition, the calibration parameter determination method in the related art needs to perform calibration based on the attitude data of the radar, the attitude data of the camera, the internal parameters of the camera, and the installation height of the radar-vision device, and the implementation is highly complex. The calibration parameter determination method provided by the present application can achieve calibration through the set of target matching pairs in the overlapping field of view area of the radar frame and the image frame collected by the radar and the camera. Therefore, the process is simple and the complexity is reduced.

[0068] Figure 6It is a schematic flowchart of another embodiment of the method for determining calibration parameters of the present application. It should be noted that if there are substantially the same results, this embodiment is not limited to Figure 6 the shown process sequence. This embodiment is a further extension of S13. As Figure 6 shown, this embodiment may include:

[0069] S21: Determine whether there is a posterior field-of-view coincidence region.

[0070] Determining whether there is a posterior field-of-view coincidence region means determining whether the prior field-of-view coincidence region has been corrected based on the target matching points in the prior field-of-view coincidence region to obtain the posterior field-of-view coincidence region.

[0071] If there is a posterior field-of-view coincidence region, then execute S22. If there is no posterior field-of-view coincidence region, then execute S23 - S25.

[0072] S22: Perform target matching based on the posterior field-of-view coincidence region.

[0073] Performing target matching based on the posterior field-of-view coincidence region means matching the targets in the image frame and the radar frame that are in the posterior field-of-view coincidence region.

[0074] S23: Determine whether the number of target matching pairs in the target matching pair set is greater than or equal to the first quantity threshold.

[0075] The number of target matching pairs in the target matching pair set refers to the number of target matching pairs that are accumulated and retained in the target matching pair set based on a number of acquired matching frame pairs. If the number of target matching pairs is less than the first quantity threshold, it means that the representativeness of the target matching pair set is insufficient and cannot be used to correct the prior field-of-view coincidence region to obtain the posterior field-of-view coincidence region. If the number of target matching pairs is greater than or equal to the first quantity threshold, it means that the representativeness of the target matching pair set is sufficient and can be used to correct the prior field-of-view coincidence region to obtain the posterior field-of-view coincidence region.

[0076] If it is less than the first quantity threshold, then execute S24; if it is greater than or equal to the first quantity threshold, then execute S25.

[0077] S24: Perform target matching based on the preset prior field-of-view coincidence region.

[0078] Performing target matching based on the prior field-of-view coincidence region means matching the targets in the image frame and the radar frame that are in the prior field-of-view coincidence region.

[0079] S25: Correct the prior field-of-view coincidence region based on the target matching pair set to obtain the posterior field-of-view coincidence region.

[0080] After executing S25, the step of obtaining frame matching pairs can be returned.

[0081] Alternatively, after executing S25, the currently accumulated frame matching pairs can be filtered for noise and then used to obtain a position transformation matrix. For example, if the current frame matching pair is frame matching pair 5, a set of target matching pairs is accumulated based on frame matching pairs 1-5. Based on the set of target matching pairs, the prior field-of-view coincidence region is corrected to obtain a posterior field-of-view coincidence region. Based on the posterior field-of-view coincidence region, the target matching pairs in the set of target matching pairs that are outside the posterior field-of-view coincidence region are filtered out. Based on the filtered set of target matching pairs, a position transformation matrix is obtained.

[0082] In some embodiments, referring to Figure 7 , the correction of the prior field-of-view coincidence region in S25 may include:

[0083] S251: Select a first preset number of target matching pairs from the set of target matching pairs.

[0084] The selected target matching pairs are on the same side edge closest to the prior field-of-view coincidence region in the radar frame along a first direction.

[0085] The target matching pairs being on the same side edge closest to the prior field-of-view coincidence region in the radar frame along a first direction means that the targets in the radar frame included are at the minimum distance from the same side edge of the prior field-of-view coincidence region in the radar field of view along the first direction.

[0086] The prior field-of-view coincidence region has a lower side edge and an upper side edge along the first direction. Correspondingly, a first preset number of target matching pairs that are closest to the lower side edge of the prior field-of-view coincidence region in the radar frame along the first direction can be selected for subsequent steps to determine the position of the lower side edge of the posterior field-of-view coincidence region in the radar field of view and the position of the upper side edge of the posterior field-of-view coincidence region in the camera field of view; a first preset number of target matching pairs that are closest to the upper side edge of the prior field-of-view coincidence region in the radar frame along the first direction can be selected for subsequent steps to determine the position of the upper side edge of the posterior field-of-view coincidence region in the radar field of view and the position of the lower side edge of the posterior field-of-view coincidence region in the camera field of view.

[0087] S252: Calculate the position difference between each selected target matching pair and other selected target matching pairs in the image frame along the first direction.

[0088] The position difference between different target matching pairs in the image frame along the first direction refers to the difference in the positions of the targets in the image frames included in different target matching pairs along the first direction.

[0089] S253: Count the number of times the position difference of each selected target matching pair falls within a preset first difference range.

[0090] S254: Determine the edge position of the posterior field-of-view coincidence region along the first direction based on the target matching pair with the largest number of occurrences.

[0091] The position of the target in the image frame included in the target matching pair with the largest number of occurrences along the first direction can be used as the edge position of the posterior field-of-view coincidence region along the first direction under the camera field of view; the position of the target in the radar frame included in the target matching pair with the largest number of occurrences along the first direction can be used as the edge position of the posterior field-of-view coincidence region along the first direction under the radar field of view.

[0092] Among them, when the first preset number of target matching pairs are the ones closest to the lower edge of the prior field-of-view coincidence region along the first direction in the radar frame, the position of the target in the radar frame included in the target matching pair with the largest number of occurrences along the first direction can be used as the lower edge position of the posterior field-of-view coincidence region along the first direction under the radar field of view, and the position of the target in the image frame included in the target matching pair with the largest number of occurrences along the first direction can be used as the upper edge position of the posterior field-of-view coincidence region along the first direction under the camera field of view. When the first preset number of target matching pairs are the ones closest to the upper edge of the prior field-of-view coincidence region along the first direction in the radar frame, the position of the target in the radar frame included in the target matching pair with the largest number of occurrences along the first direction can be used as the upper edge position of the posterior field-of-view coincidence region along the first direction under the radar field of view, and the position of the target in the image frame included in the target matching pair with the largest number of occurrences along the first direction can be used as the lower edge position of the posterior field-of-view coincidence region along the first direction under the camera field of view.

[0093] The following takes an example to illustrate S251 - S254:

[0094] The set of target matching pairs includes target matching pair i = {R i (R i x, R i y), I i (I i x, I i y)}, i = 1 - 10, Y represents the first direction, X represents the second direction, R i represents the target in the radar frame included in target matching pair i, I i represents the target in the image frame included in target matching pair i, R i (R i x, R i y) represents the position of the target in the radar frame included in target matching pair i, R i x represents the position along the second direction, R i y represents the position along the first direction, I i (I i x, Ii y) represents the position of the target in the image frame included in the target matching pair i, I i x represents the position along the second direction, I i y represents the position along the first direction.

[0095] The lower edge position and the upper edge position of the prior field of view coincidence region along the first direction are y L and y H . Among them, the lower edge position of the prior field of view coincidence region along the first direction under the camera field of view is y IL , and the lower edge position of the prior field of view coincidence region along the first direction under the radar field of view is y RL ; the upper edge position of the prior field of view coincidence region along the first direction under the camera field of view is y IH , and the upper edge position of the prior field of view coincidence region along the first direction under the radar field of view is y RH .

[0096] 1) Select R i The distance from y to y RL The first preset number (5) of target matching pairs 1-5 with the smallest distance.

[0097] 2) Calculate I j The position difference between y and I m y, j = 1-5, m = 1-5 and m ≠ j, and count the number of times the position difference falls within the first difference range to obtain the target matching pair 1 with the largest number of times.

[0098] 3) Determine the R included in the target matching pair 1 j y as the lower edge position of the posterior field of view coincidence region along the first direction under the radar field of view; determine the I included in the target matching pair 1 j y as the upper edge position of the posterior field of view coincidence region along the first direction under the camera field of view.

[0099] The determination of the upper edge position of the posterior field of view coincidence region along the first direction under the radar field of view and the lower edge position of the posterior field of view coincidence region along the first direction under the camera field of view is the same as that in 1)-3) of this example and will not be elaborated here.

[0100] In other embodiments, different from S251-S254, it may also be based on the image frame to select the first preset number of target matching pairs, and then on the basis of the first preset number of target matching pairs, determine the target matching pair with the largest number of times based on the radar frame, and then determine the edge position of the posterior field of view coincidence region along the first direction.

[0101] Figure 8 It is a schematic flowchart of another embodiment of the calibration parameter determination method of the present application. It should be noted that if there are substantially the same results, this embodiment does notFigure 8 It is limited to the shown process sequence. This embodiment is a further expansion of S13. As Figure 8 shown, this embodiment may include:

[0102] S31: Divide the field-of-view overlapping region in the radar frame into a first sub-region and a second sub-region.

[0103] The first sub-region and the second sub-region are respectively adjacent to and spaced from the two side edges of the field-of-view overlapping region along a first direction, and the target moves from the first sub-region to the second sub-region.

[0104] That is, the first sub-region is adjacent to the lower side edge of the field-of-view overlapping region along the first direction, the second sub-region is adjacent to the upper side edge of the field-of-view overlapping region along the first direction, and the first sub-region and the second sub-region are spaced apart.

[0105] S32: Divide the field-of-view overlapping region in the image frame into a third sub-region and a fourth sub-region.

[0106] The third sub-region and the fourth sub-region are respectively adjacent to and spaced from the two side edges of the field-of-view overlapping region along the first direction, and the target moves from the third sub-region to the fourth sub-region.

[0107] That is, the third sub-region is adjacent to the lower side edge of the field-of-view overlapping region along the first direction, the second sub-region is adjacent to the upper side edge of the field-of-view overlapping region along the first direction, and the first sub-region and the second sub-region are spaced apart.

[0108] S33: Determine whether the target pairs to be matched extracted from the radar frame and the image frame are respectively located in the first sub-region and the fourth sub-region, or respectively located in the second sub-region and the third sub-region.

[0109] A target pair to be matched is composed of a target to be matched in the radar frame and a target to be matched in the image frame.

[0110] The target pairs to be matched are respectively located in the first sub-region and the fourth sub-region, which means that the target in the radar frame included in the target pair to be matched is located in the first sub-region, and the target in the image frame is located in the fourth sub-region. The same applies to the target pairs to be matched respectively located in the second sub-region and the third sub-region.

[0111] If so, execute S34; otherwise, execute S35.

[0112] S34: The target pair to be matched is a non-target matching pair.

[0113] S35: The target pair to be matched is a target matching pair.

[0114] It can be understood that the closer the target is to the radar, the smaller the position of the target along the first direction relative to the radar (near radar target), and vice versa (far radar target). The closer the target is to the camera, the larger the position of the target along the first direction relative to the camera (near camera target), and vice versa (far camera target). Therefore, the lower and upper edges of the radar's field of view along the first direction are opposite to the lower and upper edges of the camera's field of view along the first direction. The lower edge of the field of view coincidence area in the radar frame along the first direction corresponds to the upper edge of the field of view coincidence area in the image frame along the first direction, and the upper edge of the field of view coincidence area in the radar frame along the first direction corresponds to the lower edge of the field of view coincidence area in the image frame along the first direction. The first sub-region corresponds to the fourth sub-region, and the second sub-region corresponds to the third sub-region. The field of view of the camera and the field of view of the radar may not completely coincide, and correspondingly, the field of view coincidence areas in the image frame and the radar frame may not completely coincide. Therefore, the target located in the second sub-region (far radar target) is regarded as the target that is in the field of view coincidence area in the radar frame but not in the field of view coincidence area in the image frame, that is, the target that is visible to the radar but not to the camera. The target located in the third sub-region (near camera target) is regarded as the target that is in the field of view coincidence area in the image frame but not in the field of view coincidence area in the radar frame, that is, the target that is visible to the camera but not to the radar.

[0115] Therefore, regarding the target pairs to be matched located in the second sub-region and the third sub-region respectively, and the target pairs to be matched located in the first sub-region and the fourth sub-region respectively as non-target matching pairs can avoid the problem of false matching caused by the inconsistency of the field of view coincidence areas in the image frame and the radar frame.

[0116] Combined with Figure 9 For example, Figure 9 is a schematic diagram of the field of view coincidence area in the radar frame and the image frame, where A represents the field of view coincidence area in the radar frame (the lower edge Y RL and the upper edge Y RH ), A1 represents the first sub-region A2 represents the second sub-region B represents the field of view coincidence area in the image frame (the lower edge Y IL and the upper edge Y IH ), B3 represents the third sub-region B4 represents the fourth sub-region The target motion direction is the first direction. The motion direction of the target in the image frame is from A1 to A2, and the motion direction of the target in the radar frame is from B3 to B4. If the target pairs to be matched in A and B appear in A1 and B4 respectively, or appear in A2 and B3 respectively, then the target pair to be matched is a non-matching target pair, otherwise it is a matching target pair.

[0117] Figure 10 It is a schematic flowchart of another embodiment of the method for determining calibration parameters of the present application. It should be noted that if there are substantially the same results, this embodiment is not limited to Figure 10 the flowchart sequence shown. This embodiment is a further extension of the above embodiment, and S41 to S45 are steps that can be included before S13. As Figure 10 shown, this embodiment may include:

[0118] S41: Determine whether the number of targets in the radar frame and the image frame is 1 respectively.

[0119] If the number of targets in both the radar frame and the image frame is 1, then execute S42; if one of the number of targets in the radar frame and the image frame is greater than 1 and the other is greater than or equal to 1, then execute S43; otherwise execute S45.

[0120] S42: Determine the targets in the radar frame and the image frame as the target pairs to be matched.

[0121] S43: Determine whether the number of targets in the motion channels on the same side edge closest to the field of view coincidence region along the second direction in the radar frame and the image frame is 1 respectively.

[0122] That is, determine whether the number of targets in the motion channels on the lower side edge closest to the field of view coincidence region along the second direction in the radar frame and the image frame is 1 respectively, and whether the number of targets in the motion channels on the upper side edge closest to the field of view coincidence region along the second direction in the radar frame and the image frame is 1 respectively.

[0123] The motion channels where each target in the radar frame is located can be determined, so as to determine the motion channels on the lower side edge and the upper side edge closest to the field of view coincidence region along the second direction in the radar frame, and determine the number of included targets. Among them, the motion channels where each target in the radar frame is located can be determined by the position of the target on the lower side edge closest to the field of view coincidence region, the position on the upper side edge closest to the field of view coincidence region, and the width of the motion channel in the radar frame.

[0124] The motion channels where each target in the image frame is located can be determined, so as to determine the motion channels on the lower side edge and the upper side edge closest to the field of view coincidence region along the second direction in the image frame, and determine the number of included targets. Among them, the motion channels where each target in the image frame is located can be determined by the pre - defined motion channel area, and the pre - defined motion channel area can be embodied as the linear equations of each motion channel.

[0125] If so, then execute S44; if not, then execute S45.

[0126] S44: Determine the targets in the motion channels on the same side edge closest to the field of view coincidence region along the second direction as the target pairs to be matched.

[0127] That is, determine that in the radar frame and the image frame, the two targets in the motion channels on the lower side edge closest to the field of view coincidence region along the second direction are the target pairs to be matched, and the two targets in the motion channels on the upper side edge closest to the field of view coincidence region along the second direction are the target pairs to be matched.

[0128] S45: Determine that there are no target matching pairs in the radar frame and the image frame.

[0129] Figure 11 It is a schematic flowchart of another embodiment of the calibration parameter determination method of the present application. It should be noted that if there are substantially the same results, this embodiment is not limited to Figure 11 the shown process sequence. This embodiment is a further expansion of the above embodiment, and S51 to S57 are steps that can be included after S15. As Figure 11 shown, this embodiment may include:

[0130] S51: Use the currently calculated position transformation matrix to reproject the positions of the targets in the target matching pair set in the radar frame to the image frame to obtain the verification positions.

[0131] S52: Calculate the reprojection error based on the verification positions and the positions of the targets in the image frame.

[0132] S53: Determine whether the reprojection error of the currently calculated position transformation matrix is less than the reprojection error of the previously retained position transformation matrix.

[0133] If it is less, execute S54; otherwise, execute S55.

[0134] S54: Update the previously retained position transformation matrix with the currently calculated position transformation matrix.

[0135] After executing S54, execute S55.

[0136] S55: Determine whether the calibration end condition is satisfied.

[0137] The calibration end condition may include that the number of target matching pairs included in the target matching pair set is greater than the target matching pair number threshold, the number of iterations (i.e., the number of times of calculating the position transformation matrix) is greater than the number threshold, the reprojection error of the currently calculated position transformation matrix is less than the error threshold, and so on.

[0138] If it is satisfied, execute S56; otherwise, execute S57.

[0139] S56: End the calibration and use the last retained position transformation matrix as the calibration result of the position transformation matrix.

[0140] S57: Return to the step of obtaining frame matching pairs.

[0141] That is, a new calculation of the position transformation matrix is performed using the new frame matching pairs until the calibration end condition is met.

[0142] The following uses an example to illustrate S51 - S57:

[0143] 1) Use the currently calculated position transformation matrix H K , and re - project the position R i (R i x, R i y) of the targets in the target matching pair set M to the image frame to obtain the verification position I i ’(I i x’, I i y’). Calculate the Euclidean distance between the position I i (I i x, I i y) of the targets in the image frame and I i ’(I i x’, I i y’). Sum and average all the Euclidean distances to obtain the reprojection error of H K as E k . Similarly, calculate the reprojection error E K-1 of the previously reserved position transformation matrix H k-1 .

[0144] 2) Determine whether E k is less than E k-1 ; if it is less, then update H k with H k-1 , and proceed to 3); otherwise, do not update H k with H k-1 , and directly proceed to 3).

[0145] 3) Determine whether the number of iterations is greater than the iteration threshold; if it is greater, end the calibration and use the last reserved position transformation matrix as the calibration result of the position transformation matrix; if it is not greater, return to the step of obtaining the frame matching degree.

[0146] Through the implementation of this embodiment, the present application adopts the idea of iterative optimization. During the process of accumulating target matching pairs frame by frame, the position transformation matrix with the smallest reprojection error is retained until the calibration end condition is met, and the last retained position transformation matrix is used as the calibration result. Thus, compared with the method of obtaining the position transformation matrix through RANSAC, it can reduce the calculation time while ensuring the accuracy of the calibration result of the position transformation matrix. Further, there may be a rotation between the field of view of the radar and the field of view of the camera, that is, there is a deflection angle. And target matching has rotational invariance. Therefore, if the targets extracted from the radar frame are directly applied to target matching, it may lead to very poor matching results. Combined with Figure 12 for illustration, Figure 12 is a schematic diagram of the relationship of the overlapping area of the actual scene, the radar, and the camera's field of view. As Figure 12 shown, the actual scene is a lane including four moving lanes, and two adjacent moving lanes are separated by lane lines. P is the schematic diagram of the camera's field of view, and Q is the schematic diagram of the radar's field of view. θ represents the deflection angle between the radar's and the camera's fields of view.

[0147] Therefore, after target extraction and before target matching, the error caused by the rotation between the field of view of the radar and the field of view of the camera can be reduced or even eliminated. Specifically, it can be as follows:

[0148] Figure 13 is a schematic flowchart of another embodiment of the method for determining the calibration parameters of the present application. It should be noted that if there are substantially the same results, this embodiment is not limited to Figure 13 the shown process sequence. This embodiment is a further expansion of the above embodiment, and S61 - S62 are steps that can be included after S12 and before S13. As Figure 13 shown, this embodiment may include:

[0149] S61: Obtain the deflection angle of the radar's field of view relative to the camera's field of view.

[0150] Combined with reference to Figure 14 , S61 may include the following sub - steps:

[0151] S611: Obtain the first slope of the movement direction of the target in the radar frame relative to the first direction in the radar frame, and the second slope of the movement direction of the target in the image frame relative to the first direction in the image frame.

[0152] For the first slope: The first slope can be obtained based on the trajectories of at least one target in the radar frame. Alternatively, when the number of targets in the radar frame is greater than a second quantity threshold, calculate the first slope difference between the first slope of each target in the radar frame and the first slopes of other targets in the radar frame; count the number of times the first slope difference of each target in the radar frame falls within a second difference range; determine the first slope of the target in the radar frame with the largest number of times as the first slope of the movement direction of the target in the radar frame relative to the first direction in the radar frame. The first direction in the radar frame is the first direction coordinate axis in the coordinate system of the radar frame.

[0153] Illustrate the latter case by way of example:

[0154] The slopes of each target in the radar frame form a set C = {k i}, where i = 1 to 5, calculate the first slope difference between k i and k j , where j = 1 to 5 and i ≠ j; count the number of times n i of the first slope difference of k i falls within the second difference range; determine the first slope of k i when n i is the largest as the first slope of the movement direction of the target in the radar frame relative to the first direction in the radar frame.

[0155] It can be understood that the first slope of the target in the radar frame reflects the movement direction of the target in the radar frame. The larger the corresponding number of times, the closer it means to the correct movement direction of the target in the actual scenario. Therefore, determining the first slope of the target in the radar frame with the largest number of times as the first slope of the movement direction of the target in the radar frame relative to the first direction in the radar frame can better reflect the correct movement direction of the target in the actual scenario.

[0156] For the second slope: It can be obtained by using the dividing line (such as a lane line) of different movement channels and the camera installation height information. If the field of view of the camera is facing the actual scenario, that is, the projection vector of the optical axis of the camera on the plane of the actual scenario is approximately parallel to the extension direction of the travel channel in the actual scenario, the second slope can be directly set to 0.

[0157] S612: Determine the deflection angle based on the first slope and the second slope.

[0158] The difference between the first slope and the second slope can be calculated, and the deflection angle can be obtained by converting the result of the difference calculation.

[0159] S62: Correct the position of the target in the radar frame based on the deflection angle.

[0160] The formula for correction can be as follows:

[0161] R i(x j , y j ) = [cos(θ), sin(θ); -sin(θ), cos(θ)] * R’ i (x’ j , y’ j );

[0162] Where θ represents the deflection angle, and R’ i (x’ j , y’ j ) represents the position of the target in the radar frame, and R i (x j , y j ) represents the calibration result.

[0163] Through the calibration parameter determination method of the present application, the determined calibration parameters may include a position transformation matrix, or include a position transformation matrix and a deflection angle.

[0164] The following combines Figure 15 Taking an example, the calibration parameter determination method provided by the present application is described in detail:

[0165] The camera is facing the actual scene, that is, the projection vector of the optical axis of the camera on the plane of the actual scene is approximately parallel to the extension direction of the travel channel in the actual scene.

[0166] 1) Obtain frame matching pairs (radar frame and image frame).

[0167] 2) Extract targets from the radar frame and the image frame respectively to obtain the targets in the radar frame and the targets in the image frame.

[0168] 3) Determine whether there is a deflection angle between the fields of view of the radar and the camera; if so, go to 7); otherwise, go to 4).

[0169] 4) Determine whether the number of targets in the radar frame that has been obtained is greater than the second quantity threshold; if it is greater than the second quantity threshold, go to 5) - 6); otherwise, return to 1).

[0170] 5) Obtain the deflection angle based on the targets in the radar frame. For the specific implementation process, refer to the description of the previous embodiment and will not be elaborated here.

[0171] 6) Correct the positions of the targets in the radar frame based on the deflection angle.

[0172] 7) Determine whether there is a posterior field of view coincidence area; if so, go to 11); otherwise, go to 8) - 10).

[0173] 8) Match the targets in the prior field of view coincidence area in the radar frame and the image frame to obtain target matching pairs; and add the target matching pairs to the target matching pair set.

[0174] 9) Determine whether the number of target matching pairs in the target matching pair set is greater than the first quantity threshold; if it is greater, proceed to 10); otherwise, return to 1).

[0175] 10) Based on the target matching pair set, correct the prior field of view coincidence region to obtain the posterior field of view coincidence region. After executing 10), return to 1).

[0176] 11) Match the targets in the radar frame and the image frame that are in the posterior field of view coincidence region to obtain target matching pairs; and add the target matching pairs to the target matching pair set.

[0177] 12) Select 4 extreme value target matching pairs from the target matching pair set.

[0178] 13) Determine whether the 4 extreme value target matching pairs conform to the preset geometric distribution; if they conform, proceed to 15); otherwise, proceed to 14).

[0179] 14) Filter the 4 extreme value target matching pairs from the target matching pair set and return to 1).

[0180] 15) Based on the 4 extreme value target matching pairs, obtain the currently calculated position transformation matrix.

[0181] 16) Obtain the currently calculated position transformation matrix H K of the reprojection error E k and determine whether E k is less than the reprojection error E K-1 of the previously retained position transformation matrix H k-1 ; if it is less, proceed to 17); otherwise, proceed to 18).

[0182] 17) Update H K with H K-1 .

[0183] 18) Determine whether the calibration end condition is satisfied. If it is satisfied, end the calibration; otherwise, return to 1).

[0184] The following combines Figure 16 to illustrate the application scenario of target detection and fusion of the radar and the camera after calibration in the form of an example:

[0185] Figure 16 is a schematic diagram of the target detection and fusion system, as Figure 16As shown in the figure, the target detection and fusion device includes a camera 11, a radar 12, and a target detection and fusion device 13. The camera 11 is used to collect image frames, the radar 12 is used to collect radar frames, and the target detection and fusion device 13 performs target detection based on the image frames and radar frames to obtain the image attributes and radar attributes of the targets respectively, and then fuses the two attributes based on the calibration parameters.

[0186] Figure 17 It is a schematic structural diagram of an embodiment of the calibration parameter determination device of the present application. As Figure 17 shown in the figure, the calibration parameter determination device includes a processor 21 and a memory 22 coupled to the processor 21.

[0187] Among them, the memory 22 stores program instructions for implementing the method of any of the above embodiments; the processor 21 is used to execute the program instructions stored in the memory 22 to implement the steps of the above method embodiments. Among them, the processor 21 can also be called a CPU (Central Processing Unit, central processing unit). The processor 21 may be an integrated circuit chip with signal processing capabilities. The processor 21 can also be a general-purpose processor, a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components. The general-purpose processor can be a microprocessor or the processor can also be any conventional processor, etc.

[0188] The calibration parameter determination device can be a radar, a camera, a radar-vision device, an electronic device (such as a computer, a mobile phone) that establishes a communication connection with the radar or the camera, or a server, etc.

[0189] Figure 18 It is a schematic structural diagram of an embodiment of the computer-readable storage medium of the present application. As Figure 18 shown in the figure, the computer-readable storage medium 30 of the embodiment of the present application stores program instructions 31, and when the program instructions 31 are executed, the method provided in the above embodiments of the present application is implemented. Among them, the program instructions 31 can form a program file and be stored in the above computer-readable storage medium 30 in the form of a software product, so that a computer device (which can be a personal computer, a server, or a network device, etc.) or a processor can execute all or part of the steps of the methods of various embodiments of the present application. And the aforementioned computer-readable storage medium 30 includes: various media that can store program codes such as a USB flash drive, a mobile hard disk, a read-only memory (ROM, Read-Only Memory), a random access memory (RAM, Random Access Memory), a magnetic disk, or an optical disc, or a terminal device such as a computer, a server, a mobile phone, or a tablet.

[0190] In several embodiments provided by the present application, it should be understood that the disclosed systems, devices, and methods can be implemented in other ways. For example, the device embodiments described above are merely illustrative. For example, the division of units is only a logical function division. In actual implementation, there may be other division methods. For example, multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the displayed or discussed coupling or direct coupling or communication connection between each other can be an indirect coupling or communication connection through some interfaces, devices, or units, and can be in electrical, mechanical, or other forms.

[0191] In addition, each functional unit in various embodiments of the present application can be integrated in a processing unit, or each unit can exist physically alone, or two or more units can be integrated in one unit. The above-mentioned integrated unit can be implemented in the form of hardware or in the form of a software functional unit. The above is only the implementation manner of the present application, and does not limit the patent scope of the present application. Any equivalent structure or equivalent process transformation made by using the content of the specification and drawings of the present application, or directly or indirectly applied in other related technical fields, is equally included in the patent protection scope of the present application.

Claims

1. A method for determining calibration parameters, characterized in that, Including: Obtaining frame matching pairs including a target, where the frame matching pairs include a radar frame and an image frame respectively collected by a radar and a camera at the same moment; Respectively performing target extraction on the radar frame and the image frame in the frame matching pairs; Performing target matching with the overlapping region of the fields of view of the radar and the camera as a matching constraint condition to obtain target matching pairs; Adding the target matching pairs to a target matching pair set; Based on the target matching pair set, obtaining a position transformation matrix between the radar and the camera; Wherein, the performing target matching with the overlapping region of the fields of view of the radar and the camera as a matching constraint condition includes: judging whether there is a posterior overlapping region of the fields of view; if there is the posterior overlapping region of the fields of view, performing target matching based on the posterior overlapping region of the fields of view; if there is no the posterior overlapping region of the fields of view, judging whether the number of target matching pairs in the target matching pair set is less than a first quantity threshold; if it is less than the first quantity threshold, performing target matching based on a preset prior overlapping region of the fields of view; if it is greater than or equal to the first quantity threshold, correcting the prior overlapping region of the fields of view based on the target matching pair set to obtain the posterior overlapping region of the fields of view; Wherein, the actual scene for calibration includes a plurality of motion channels extending along a first direction and arranged side by side along a second direction, the first direction is perpendicular to the second direction, and a target moves along the first direction in one of the motion channels; the correcting the prior overlapping region of the fields of view based on the target matching pair set to obtain the posterior overlapping region of the fields of view includes: selecting a first preset number of target matching pairs from the target matching pair set, and the selected target matching pairs are on the same side edge closest to the prior overlapping region of the fields of view along the first direction in the radar frame; calculating the position differences between each selected target matching pair and other selected target matching pairs along the first direction in the image frame; counting the number of times that the position differences of each selected target matching pair fall within a preset first difference range; determining the edge position of the posterior overlapping region of the fields of view along the first direction based on the target matching pair with the largest number of times; 2. The method according to claim 1, wherein The actual scene includes a plurality of motion channels extending along a first direction and arranged side by side along a second direction, the first direction is perpendicular to the second direction, and a target moves along the first direction in one of the motion channels; the performing target matching with the overlapping region of the fields of view of the radar and the camera as a matching constraint condition includes: Dividing the overlapping region of the fields of view in the radar frame into a first sub-region and a second sub-region, the first sub-region and the second sub-region are respectively adjacent to and spaced from the two side edges of the overlapping region of the fields of view along the first direction, and the target moves from the first sub-region to the second sub-region; Divide the field-of-view coincidence region in the image frame into a third sub-region and a fourth sub-region. The third sub-region and the fourth sub-region are respectively adjacent to and spaced from the two side edges of the field-of-view coincidence region along the first direction, and the target moves from the third sub-region to the fourth sub-region. If the target pairs to be matched extracted from the radar frame and the image frame are respectively located in the first sub-region and the fourth sub-region, or respectively located in the second sub-region and the third sub-region, then determine that the target pairs to be matched are non-target matching pairs.

3. The method according to claim 1, wherein The actual scene includes a plurality of motion channels extending along a first direction and arranged side by side along a second direction. The first direction is perpendicular to the second direction, and a target moves along the first direction in one of the motion channels. Before performing target matching with the field-of-view coincidence region of the radar and the camera as the matching constraint condition, the method further includes: If the number of targets in both the radar frame and the image frame is 1, then determine the targets in the radar frame and the image frame as the target pairs to be matched. If one of the number of targets in the radar frame and the image frame is greater than 1, the other is greater than or equal to 1, and the number of targets in the motion channels closest to the same side edge of the field-of-view coincidence region along the second direction is 1, then determine the targets in the motion channels closest to the same side edge of the field-of-view coincidence region along the second direction as the target pairs to be matched.

4. The method according to claim 1, characterized in that The obtaining the position transformation matrix between the radar and the camera based on the set of target matching pairs includes: Select a second preset number of target matching pairs from the set of target matching pairs. Judge whether the position distributions of the selected second preset number of target matching pairs in the image frame and in the radar frame both conform to a preset position distribution. If both conform to the preset position distribution, then obtain the position transformation matrix based on the selected second preset number of target matching pairs. If they do not both conform to the preset position distribution, then filter the selected second preset number of target matching pairs from the set of target matching pairs and return to the step of obtaining frame matching pairs.

5. The method according to claim 4, characterized in that, The actual scene includes a plurality of motion channels extending along a first direction and arranged side by side along a second direction. The first direction is perpendicular to the second direction, and a target moves along the first direction in one of the motion channels. The second preset number is 4, and the second preset number of target matching pairs respectively have maximum or minimum values along the first direction or the second direction in the image frame.

6. The method according to claim 1, wherein The method further includes: Using the currently calculated position transformation matrix, re-project the positions of the targets in the radar frame in the set of target matching pairs to the image frame to obtain verification positions. Calculate the reprojection error based on the verification positions and the positions of the targets in the image frame. If the reprojection error of the currently calculated position transformation matrix is less than the reprojection error of the previously retained position transformation matrix, then update the previously retained position transformation matrix with the currently calculated position transformation matrix.

7. The method according to claim 6, wherein The method further includes: If the reprojection error of the currently calculated position transformation matrix is not less than the reprojection error of the previously retained position transformation matrix, then determine whether the calibration end condition is satisfied; If the calibration end condition is not satisfied, then return to the step of obtaining frame matching pairs.

8. The method according to claim 1, wherein After respectively performing target extraction on the radar frame and the image frame in the frame matching pair, the method further includes: Obtain the deflection angle of the field of view of the radar relative to the field of view of the camera; Correct the position of the target in the radar frame based on the deflection angle.

9. The method according to claim 8, wherein The actual scene includes a plurality of motion channels extending along a first direction and arranged side by side along a second direction, the first direction being perpendicular to the second direction, and a target moving along the first direction in one of the motion channels; The obtaining the deflection angle of the field of view of the radar relative to the field of view of the camera includes: Obtain a first slope of the moving direction of the target in the radar frame relative to the first direction in the radar frame, and a second slope of the moving direction of the target in the image frame relative to the first direction in the image frame; Determine the deflection angle based on the first slope and the second slope.

10. The method according to claim 9, characterized in that, The step of obtaining the first slope of the moving direction of the target in the radar frame relative to the first direction in the radar frame includes: Calculate a first slope difference between the first slope of each target in the radar frame and the first slopes of other targets in the radar frame, the number of targets in the radar frame being greater than a second quantity threshold; Count the number of times the first slope difference of each target in the radar frame falls within a second difference range; Determine the first slope of the target in the radar frame with the largest number of times as the first slope of the moving direction of the target in the radar frame relative to the first direction in the radar frame.

11. The method according to claim 1, wherein After respectively performing target extraction on the radar frame and the image frame in the frame matching pair, the method further includes: Fit a reference plane based on the positions of the targets in the radar frame; Convert the positions of the targets in the radar frame from the three-dimensional coordinate system of the radar to the two-dimensional coordinate system of the reference plane.

12. A calibration parameter determination device, characterized in that, Includes a processor and a memory connected to the processor, wherein, The memory stores program instructions; The processor is configured to execute the program instructions stored in the memory to implement the method according to any one of claims 1-11.

13. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores program instructions that can be executed by a processor, and when executed, implements the method according to any one of claims 1-11.

Images