Leighting joint calibration method and device

Abstract

The invention relates to the technical field of multi-sensor joint calibration, and discloses a thunder-vision joint calibration method and device, and the method comprises the steps: determining first visual track information based on a visual sensor, and determining first radar track information based on a radar sensor; based on the target object, executing corresponding object association operation on the visual sensor and the radar sensor to obtain a calibration object association result; according to the calibration object association result, the first visual trajectory information and the first radar trajectory information, determining a trajectory relationship result corresponding to the target object based on a visual level and a radar level; and according to a trajectory relationship result, determining second visual trajectory information and second radar trajectory information of the target object based on the target driving area. Therefore, the accuracy and reliability of the calculated radar track information and visual track information can be improved, the accuracy, reliability, efficiency and convenience of thunder vision calibration are improved, and the stability and reliability of thunder vision application are improved.

Description

Technical Field

[0001] The present invention relates to the technical field of multi-sensor joint calibration, and in particular to a radar-vision joint calibration method and device. Background Art

[0002] With the construction and optimization of intelligent transportation systems, radar sensors and visual sensors play an important role. They can be applied to multiple demand areas such as traffic flow monitoring, vehicle identification and tracking, and traffic accident handling. Therefore, joint calibration of radar and visual sensors can effectively promote the development of related technologies.

[0003] At present, the main method for joint calibration of radar and vision is that the staff conducts joint calibration through subjective consciousness. However, joint calibration of radar and vision based on subjective consciousness will be affected by many factors, such as the poor mental state of the staff during joint calibration, and different staff members have different considerations and focuses on analyzing the same radar and vision situation. To a certain extent, even under the same conditions, the joint calibration results for the same radar and vision situation will be biased, making the accuracy and efficiency of joint calibration of radar and vision low. It can be seen that it is particularly important to provide a new joint calibration method of radar and vision to improve the accuracy and efficiency of joint calibration of radar and vision. Summary of the invention

[0004] The present invention provides a radar-vision joint calibration method and device, which can improve the accuracy and efficiency of the radar-vision joint calibration.

[0005] In order to solve the above technical problems, the first aspect of the present invention discloses a radar-visual joint calibration method, the method comprising:

[0006] Determine first visual track information of the target object based on the visual sensor, and determine first radar track information of the target object based on the radar sensor;

[0007] Based on the target object, performing corresponding object association operations on the visual sensor and the radar sensor to obtain a calibration object association result;

[0008] Determine, according to the calibration object association result, the first visual track information and the first radar track information, a track relationship result corresponding to the target object based on a visual level and a radar level;

[0009] According to the trajectory relationship result, second visual trajectory information and second radar trajectory information of the target object based on the target driving area are determined.

[0010] As an optional implementation, in the first aspect of the present invention, the first visual track information includes one or more first visual track coordinate points of the target object; the first radar track information includes one or more first radar track coordinate points of the target object;

[0011] And, determining the trajectory relationship result corresponding to the target object based on the visual level and the radar level according to the calibration object association result, the first visual trajectory information and the first radar trajectory information, comprises:

[0012] According to the calibration object association result and the preset association coordinate determination method, performing corresponding coordinate point matching operations on all the first visual track coordinate points and all the first radar coordinate points to obtain one or more coordinate point matching sets;

[0013] For each of the coordinate point matching sets, determining the basic coordinate relationship corresponding to the first visual track coordinate point and the first radar track coordinate point in the coordinate point matching set according to a preset coordinate relationship calculation method;

[0014] According to all the coordinate point matching sets and their corresponding basic coordinate relationships, the trajectory relationship results corresponding to the target object based on the visual level and the radar level are determined.

[0015] As an optional implementation, in the first aspect of the present invention, after determining the second visual track information and the second radar track information of the target object based on the target driving area according to the track relationship result, the method further includes:

[0016] Determining a trajectory evaluation reference content that can be used to verify and optimize the second visual trajectory information and the second radar trajectory information, the trajectory evaluation reference content including standardized trajectory information and / or standardized trajectory relationship results;

[0017] According to the trajectory evaluation reference content, determining whether the second visual trajectory information and the second radar trajectory information meet a preset update optimization requirement condition;

[0018] When the judgment result is yes, performing corresponding trajectory information update optimization operations on the second visual trajectory information and the second radar trajectory information according to the trajectory evaluation reference content;

[0019] The standardized track information, the second visual track information and the second radar track information all include corresponding track coordinate points.

[0020] As an optional implementation manner, in the first aspect of the present invention, after performing corresponding object association operations on the visual sensor and the radar sensor based on the target object to obtain a calibration object association result, the method further includes:

[0021] Determining whether the target object meets a preset trajectory rationalization condition according to the first visual trajectory information and the first radar trajectory information;

[0022] When it is determined that the target object meets the trajectory rationalization condition, the step of determining the trajectory relationship result corresponding to the target object based on the visual level and the radar level according to the calibration object association result, the first visual trajectory information and the first radar trajectory information is executed.

[0023] As an optional embodiment, in the first aspect of the present invention, the method further comprises:

[0024] When it is determined that the target object does not satisfy the trajectory rationalization condition, analyzing a first impact on the data collection capability of the radar sensor according to the determined first scene information corresponding to the radar sensor, and analyzing a second impact on the data collection capability of the visual sensor according to the determined second scene information corresponding to the visual sensor;

[0025] According to the first affected condition and the second affected condition, performing corresponding trajectory information adjustment operations on the first radar trajectory information and the first visual trajectory information to obtain a trajectory information adjustment result;

[0026] Determine a trajectory relationship result according to the trajectory information adjustment result, and perform the step of determining the second visual trajectory information and the second radar trajectory information of the target object based on the target driving area according to the trajectory relationship result;

[0027] Among them, the first scene information includes one or more of atmospheric wind information, earth gravity information, temperature and humidity information and other noise factor parameter information; the second scene information includes one or more of lighting condition information, natural climate information and other scene recognition object information.

[0028] As an optional implementation manner, in the first aspect of the present invention, judging whether the target object meets a preset trajectory rationalization condition according to the first visual trajectory information and the first radar trajectory information includes:

[0029] Determine first track time information corresponding to the first visual track information, and determine second track time information corresponding to the first radar track information;

[0030] According to the first trajectory time information and the second trajectory time information, all the first visual trajectory coordinate points and all the first radar trajectory coordinate points are divided into a first trajectory coordinate set that meets a preset time coincidence condition and / or a second trajectory coordinate set that does not meet a preset time coincidence condition;

[0031] Determining a first trajectory coincidence degree for an actual trajectory route according to the first trajectory coordinate set, and / or determining a second trajectory coincidence degree for a predicted trajectory route according to the second trajectory coordinate set;

[0032] According to the first trajectory overlap degree and / or the second trajectory overlap degree, determining whether the target object meets a preset trajectory overlap condition;

[0033] When it is determined that the target object satisfies the trajectory coincidence condition, determining that the target object satisfies a preset trajectory rationalization condition;

[0034] When it is determined that the target object does not satisfy the trajectory coincidence condition, it is determined that the target object does not satisfy a preset trajectory rationalization condition.

[0035] As an optional implementation manner, in the first aspect of the present invention, performing corresponding trajectory information adjustment operations on the first radar trajectory information and the first visual trajectory information according to the first affected situation and the second affected situation to obtain a trajectory information adjustment result includes:

[0036] According to the first affected situation, determining the target radar track coordinate point that needs to be adjusted and optimized from all the first radar track coordinate points, and / or, according to the second affected situation, determining the target visual track coordinate point that needs to be adjusted and optimized from all the first visual track coordinate points;

[0037] Determine a trajectory information optimization scheme corresponding to the target radar trajectory coordinate point and / or the target visual trajectory coordinate point according to the first affected situation, the second affected situation, the first radar trajectory information, and the second radar trajectory information;

[0038] A trajectory information adjustment result is determined according to the trajectory information optimization scheme, the first radar trajectory information, and the second radar trajectory information.

[0039] The second aspect of the present invention discloses a radar-visual joint calibration device, the device comprising:

[0040] A track information acquisition module, used to determine first visual track information of the target object based on the visual sensor, and to determine first radar track information of the target object based on the radar sensor;

[0041] An object association module, configured to perform corresponding object association operations on the visual sensor and the radar sensor based on the target object to obtain a calibration object association result;

[0042] A trajectory relationship determination module, used to determine the trajectory relationship result corresponding to the target object based on the visual level and the radar level according to the calibration object association result, the first visual trajectory information and the first radar trajectory information;

[0043] The trajectory information calculation module is used to determine the second visual trajectory information and the second radar trajectory information of the target object based on the target driving area according to the trajectory relationship result.

[0044] As an optional implementation, in the second aspect of the present invention, the first visual track information includes one or more first visual track coordinate points of the target object; the first radar track information includes one or more first radar track coordinate points of the target object;

[0045] And, the trajectory relationship determination module determines the trajectory relationship result corresponding to the target object based on the visual level and the radar level according to the calibration object association result, the first visual trajectory information and the first radar trajectory information, specifically including:

[0046] According to the calibration object association result and the preset association coordinate determination method, performing corresponding coordinate point matching operations on all the first visual track coordinate points and all the first radar coordinate points to obtain one or more coordinate point matching sets;

[0047] For each of the coordinate point matching sets, determining the basic coordinate relationship corresponding to the first visual track coordinate point and the first radar track coordinate point in the coordinate point matching set according to a preset coordinate relationship calculation method;

[0048] According to all the coordinate point matching sets and their corresponding basic coordinate relationships, the trajectory relationship results corresponding to the target object based on the visual level and the radar level are determined.

[0049] As an optional implementation, in the second aspect of the present invention, the device further includes:

[0050] an optimization information determination module, configured to determine, after the trajectory information calculation module determines the second visual trajectory information and the second radar trajectory information of the target object based on the target driving area according to the trajectory relationship result, a trajectory evaluation reference content that can be used to verify and optimize the second visual trajectory information and the second radar trajectory information, wherein the trajectory evaluation reference content includes standardized trajectory information and / or standardized trajectory relationship result;

[0051] A first judgment module, used to judge whether the second visual trajectory information and the second radar trajectory information meet a preset update optimization requirement condition according to the trajectory evaluation reference content;

[0052] a trajectory information optimization module, configured to perform corresponding trajectory information update optimization operations on the second visual trajectory information and the second radar trajectory information according to the trajectory evaluation reference content when the first judgment module determines that the second visual trajectory information and the second radar trajectory information meet the update optimization requirement condition;

[0053] The standardized track information, the second visual track information and the second radar track information all include corresponding track coordinate points.

[0054] As an optional implementation, in the second aspect of the present invention, the device further includes:

[0055] The second judgment module is used for, after the object association module performs corresponding object association operations on the visual sensor and the radar sensor based on the target object, and obtains the calibration object association result, judging whether the target object meets the preset trajectory rationalization condition according to the first visual trajectory information and the first radar trajectory information; when the judgment result is yes, the trajectory relationship determination module executes the step of determining the trajectory relationship result corresponding to the target object based on the visual level and the radar level according to the calibration object association result, the first visual trajectory information and the first radar trajectory information.

[0056] As an optional implementation, in the second aspect of the present invention, the device further includes:

[0057] a trajectory information adjustment module, configured to, when the second judgment module determines that the target object does not meet the trajectory rationalization condition, analyze a first affected condition of the radar sensor on the data collection capability according to the determined first scene information corresponding to the radar sensor, and analyze a second affected condition of the visual sensor on the data collection capability according to the determined second scene information corresponding to the visual sensor; and perform corresponding trajectory information adjustment operations on the first radar trajectory information and the first visual trajectory information according to the first affected condition and the second affected condition to obtain a trajectory information adjustment result;

[0058] The trajectory relationship determination module is further used to determine a trajectory relationship result according to the trajectory information adjustment result, and trigger the trajectory information calculation module to execute the step of determining the second visual trajectory information and the second radar trajectory information of the target object based on the target driving area according to the trajectory relationship result;

[0059] Among them, the first scene information includes one or more of atmospheric wind information, earth gravity information, temperature and humidity information and other noise factor parameter information; the second scene information includes one or more of lighting condition information, natural climate information and other scene recognition object information.

[0060] As an optional implementation, in the second aspect of the present invention, the second judgment module determines whether the target object meets the preset trajectory rationalization condition according to the first visual trajectory information and the first radar trajectory information, specifically including:

[0061] Determine first track time information corresponding to the first visual track information, and determine second track time information corresponding to the first radar track information;

[0062] According to the first trajectory time information and the second trajectory time information, all the first visual trajectory coordinate points and all the first radar trajectory coordinate points are divided into a first trajectory coordinate set that meets a preset time coincidence condition and / or a second trajectory coordinate set that does not meet a preset time coincidence condition;

[0063] Determining a first trajectory coincidence degree for an actual trajectory route according to the first trajectory coordinate set, and / or determining a second trajectory coincidence degree for a predicted trajectory route according to the second trajectory coordinate set;

[0064] According to the first trajectory overlap degree and / or the second trajectory overlap degree, determining whether the target object meets a preset trajectory overlap condition;

[0065] When it is determined that the target object satisfies the trajectory coincidence condition, determining that the target object satisfies a preset trajectory rationalization condition;

[0066] When it is determined that the target object does not satisfy the trajectory coincidence condition, it is determined that the target object does not satisfy a preset trajectory rationalization condition.

[0067] As an optional implementation, in the second aspect of the present invention, the trajectory information adjustment module performs corresponding trajectory information adjustment operations on the first radar trajectory information and the first visual trajectory information according to the first affected situation and the second affected situation, and the manner of obtaining the trajectory information adjustment result specifically includes:

[0068] According to the first affected situation, determining the target radar track coordinate point that needs to be adjusted and optimized from all the first radar track coordinate points, and / or, according to the second affected situation, determining the target visual track coordinate point that needs to be adjusted and optimized from all the first visual track coordinate points;

[0069] Determine a trajectory information optimization scheme corresponding to the target radar trajectory coordinate point and / or the target visual trajectory coordinate point according to the first affected situation, the second affected situation, the first radar trajectory information, and the second radar trajectory information;

[0070] A trajectory information adjustment result is determined according to the trajectory information optimization scheme, the first radar trajectory information, and the second radar trajectory information.

[0071] The third aspect of the present invention discloses another radar-visual joint calibration device, the device comprising:

[0072] A memory storing executable program code;

[0073] a processor coupled to the memory;

[0074] The processor calls the executable program code stored in the memory to execute a radar-visual joint calibration method disclosed in the first aspect of the present invention.

[0075] A fourth aspect of the present invention discloses a computer storage medium, wherein the computer storage medium stores computer instructions, and when the computer instructions are called, they are used to execute a radar-visual joint calibration method disclosed in the first aspect of the present invention.

[0076] Compared with the prior art, the embodiments of the present invention have the following beneficial effects:

[0077] In an embodiment of the present invention, the first visual track information of the target object is determined based on the visual sensor, and the first radar track information of the target object is determined based on the radar sensor; based on the target object, the corresponding object association operation is performed on the visual sensor and the radar sensor to obtain a calibration object association result; according to the calibration object association result, the first visual track information and the first radar track information, the track relationship result corresponding to the target object based on the visual level and the radar level is determined; according to the track relationship result, the second visual track information and the second radar track information of the target object based on the target driving area are determined. It can be seen that the present invention can determine the track relationship result according to the determined first radar track information, the first visual track information and the object association result, and then determine the second radar track information and the second visual track information according to the track relationship result, which is conducive to improving the comprehensiveness and rationality of a radar-visual joint calibration method, and then is conducive to improving the accuracy and reliability of the calculated second radar track information and the second visual track information, so as to improve the accuracy and reliability of the radar-visual calibration, and is conducive to improving the efficiency and convenience of the radar-visual calibration, and further is conducive to improving the stability and reliability of the radar-visual application. BRIEF DESCRIPTION OF THE DRAWINGS

[0078] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the following briefly introduces the drawings required for use in the description of the embodiments. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0079] Figure 1 It is a flow chart of a radar-visual joint calibration method disclosed in an embodiment of the present invention;

[0080] Figure 2 It is a flow chart of another radar-visual joint calibration method disclosed in an embodiment of the present invention;

[0081] Figure 3 It is a structural schematic diagram of a radar-visual joint calibration device disclosed in an embodiment of the present invention;

[0082] Figure 4 It is a structural schematic diagram of another radar-visual joint calibration device disclosed in an embodiment of the present invention;

[0083] Figure 5 It is a structural schematic diagram of another radar-visual joint calibration device disclosed in an embodiment of the present invention. DETAILED DESCRIPTION

[0084] In order to enable those skilled in the art to better understand the scheme of the present invention, the technical scheme in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0085] The terms "first", "second", etc. in the specification and claims of the present invention and the above-mentioned drawings are used to distinguish different objects rather than to describe a specific order. In addition, the terms "including" and "having" and any variations thereof are intended to cover non-exclusive inclusions. For example, a process, method, device, product or end including a series of steps or units is not limited to the listed steps or units, but may optionally include steps or units that are not listed, or may optionally include other steps or units inherent to these processes, methods, products or ends.

[0086] Reference to "embodiments" herein means that a particular feature, structure, or characteristic described in conjunction with the embodiments may be included in at least one embodiment of the present invention. The appearance of the phrase in various places in the specification does not necessarily refer to the same embodiment, nor is it an independent or alternative embodiment that is mutually exclusive with other embodiments. It is explicitly and implicitly understood by those skilled in the art that the embodiments described herein may be combined with other embodiments.

[0087] The present invention discloses a radar-visual joint calibration method and device, which can determine the track relationship result according to the determined first radar track information, the first visual track information and the object association result, and then determine the second radar track information and the second visual track information according to the track relationship result, which is conducive to improving the comprehensiveness and rationality of a radar-visual joint calibration method, and then is conducive to improving the accuracy and reliability of the calculated second radar track information and the second visual track information, thereby helping to improve the accuracy and reliability of radar-visual calibration, and is conducive to improving the efficiency and convenience of radar-visual calibration, and further is conducive to improving the stability and reliability of radar-visual application. The following are detailed descriptions.

[0088] Embodiment 1

[0089] See also Figure 1 , Figure 1 : is a flow chart of a radar-visual joint calibration method disclosed in an embodiment of the present invention. Figure 1 The described method may be applied to a radar-visual joint calibration device, wherein the device may include a server, wherein the server includes a local server or a cloud server, which is not limited in the embodiment of the present invention. Figure 1 As shown, the radar-visual joint calibration method includes the following operations:

[0090] 101. Determine first visual track information of a target object based on a visual sensor, and determine first radar track information of the target object based on a radar sensor.

[0091] Optionally, the first visual track information may be determined by, but is not limited to, image information such as color and shape of the target object, which is not limited in this embodiment of the present invention.

[0092] Optionally, the first radar trajectory information may be determined by, but is not limited to, radar information such as a positioning point, speed, and position of the target object, and is not limited in this embodiment of the present invention.

[0093] Optionally, the first visual trajectory information may include but is not limited to visual trajectory coordinate points and / or other information that can reflect the visual trajectory, etc., which is not limited in the embodiment of the present invention.

[0094] Optionally, the first radar track information may include but is not limited to radar track coordinate points and / or other information that can reflect the radar track, etc., which is not limited in this embodiment of the present invention.

[0095] Optionally, the object type of the target object may be a vehicle, a person, or other subject objects, which is not limited in the embodiment of the present invention.

[0096] 102. Based on the target object, perform corresponding object association operations on the visual sensor and the radar sensor to obtain a calibration object association result.

[0097] Further optionally, the above-mentioned performing corresponding object association operations on the visual sensor and the radar sensor based on the target object to obtain the calibration object association result may include:

[0098] Associating and binding the target object related content collected by the visual sensor with the target object related content collected by the radar sensor to obtain a calibration object association result; or

[0099] The operations performed by the user in response to the object association requirements of the visual sensor and the radar sensor (ie, manual association operations) are detected to obtain calibration object association results.

[0100] Further optionally, the above-mentioned operation of associating and binding the target object related content collected by the visual sensor with the target object related content collected by the radar sensor may be associating and binding the target object collected by the visual sensor with the target object collected by the radar sensor, or may be associating and binding the target object collected by the visual sensor and its related information (such as visual trajectory information, etc.) with the target object collected by the radar sensor and its related information (such as radar trajectory information, etc.), which is not limited to the embodiments of the present invention.

[0101] 103. Determine the trajectory relationship results corresponding to the target object based on the visual level and the radar level according to the calibration object association result, the first visual trajectory information and the first radar trajectory information.

[0102] Optionally, the trajectory relationship result corresponding to the target object based on the visual level and the radar level can be understood as the coordinate relationship between the visual level trajectory coordinate point and the radar level trajectory coordinate point of the target object, which is not limited in the embodiment of the present invention.

[0103] 104. Determine, according to the trajectory relationship result, second visual trajectory information and second radar trajectory information of the target object based on the target driving area.

[0104] Optionally, the second visual track information may include, but is not limited to, one or more second visual track coordinate points of the target object, which is not limited in this embodiment of the present invention.

[0105] Optionally, the second radar track information may include, but is not limited to, one or more second radar track coordinate points of the target object, which is not limited in this embodiment of the present invention.

[0106] Further optionally, the method may also include the following operations:

[0107] When it is necessary to determine the required trajectory information of the target object, the target trajectory information is determined from the second visual trajectory information and the second radar trajectory information according to the determined trajectory information requirement content of the target object as the required trajectory information of the target object.

[0108] It can be seen that the radar and visual joint calibration method described in the embodiment of the present invention can determine the trajectory relationship result based on the determined first radar trajectory information, the first visual trajectory information and the object association result, and then determine the second radar trajectory information and the second visual trajectory information based on the trajectory relationship result, which is conducive to improving the comprehensiveness and rationality of a radar and visual joint calibration method, and further conducive to improving the accuracy and reliability of the calculated second radar trajectory information and the second visual trajectory information, thereby facilitating improving the accuracy and reliability of radar and visual calibration, as well as improving the efficiency and convenience of radar and visual calibration, and further conducive to improving the stability and reliability of radar and visual applications.

[0109] In an optional embodiment, the first visual track information includes one or more first visual track coordinate points of the target object; the first radar track information includes one or more first radar track coordinate points of the target object;

[0110] Further optionally, determining the trajectory relationship result corresponding to the target object based on the visual level and the radar level according to the calibration object association result, the first visual track information and the first radar track information may include:

[0111] According to the calibration object association result and the preset association coordinate determination method, corresponding coordinate point matching operations are performed on all first visual track coordinate points and all first radar coordinate points to obtain one or more coordinate point matching sets;

[0112] For each coordinate point matching set, determining the basic coordinate relationship corresponding to the first visual track coordinate point and the first radar track coordinate point in the coordinate point matching set according to a preset coordinate relationship calculation method;

[0113] According to the matching set of all coordinate points and their corresponding basic coordinate relationships, the trajectory relationship results corresponding to the target object based on the visual level and radar level are determined.

[0114] Optionally, each coordinate point matching set may include a first visual track coordinate point and a first radar track coordinate point matching the first visual track coordinate point, which is not limited in the embodiment of the present invention.

[0115] Optionally, the method for determining associated coordinates may include, but is not limited to, one or more of a method for determining associated coordinates for the same time node, a method for determining associated coordinates for the same environmental scene features, a method for determining associated coordinates for the same character features, and other methods for determining associated coordinates based on related parameters, and the embodiments of the present invention are not limited thereto.

[0116] Further optionally, the above-mentioned determination of the trajectory relationship result corresponding to the target object based on the visual level and the radar level according to the matching set of all coordinate points and their corresponding basic coordinate relationship may include:

[0117] Directly integrate all coordinate point matching sets and their corresponding basic coordinate relationships to determine the trajectory relationship results corresponding to the target object based on the visual level and the radar level; or,

[0118] According to the preset coordinate relationship synthesis method, the corresponding coordinate relationship synthesis operation is performed on all coordinate point matching sets and their corresponding basic coordinate relationships to obtain the trajectory relationship results corresponding to the target object based on the visual level and radar level.

[0119] It can be seen that this optional embodiment can determine one or more coordinate point matching sets and their corresponding basic coordinate relationships, and then determine the trajectory relationship results of the target object, which is conducive to improving the comprehensiveness and rationality of the trajectory relationship result determination method, and then is conducive to improving the accuracy and reliability of the determined trajectory relationship results, thereby facilitating improving the accuracy and reliability of the radar vision calibration results determined based on the trajectory relationship results and the predicted new radar trajectory information and new visual trajectory information.

[0120] In another optional embodiment, after determining the second visual track information and the second radar track information of the target object based on the target driving area according to the track relationship result, the method may further include the following operations:

[0121] Determine a trajectory evaluation reference content that can be used to verify and optimize the second visual trajectory information and the second radar trajectory information, the trajectory evaluation reference content including standardized trajectory information and / or standardized trajectory relationship results;

[0122] According to the trajectory evaluation reference content, judging whether the second visual trajectory information and the second radar trajectory information meet the preset update optimization requirement conditions;

[0123] When the judgment result is yes, performing corresponding trajectory information update optimization operations on the second visual trajectory information and the second radar trajectory information according to the trajectory evaluation reference content;

[0124] The standardized track information, the second visual track information and the second radar track information all include corresponding track coordinate points.

[0125] Optionally, standardized trajectory information can be understood as accurate, newly recorded radar trajectory information and visual trajectory information for the target object; further optionally, the standardized trajectory information may include but is not limited to at least one standardized radar trajectory coordinate point and / or at least one standardized visual trajectory coordinate point, which is not limited in the embodiments of the present invention.

[0126] Optionally, the standardized trajectory relationship result may be understood as an accurate, newly obtained trajectory relationship result for the target object, which is not limited in the embodiment of the present invention.

[0127] Further optionally, judging whether the second visual trajectory information and the second radar trajectory information meet a preset update optimization requirement condition according to the trajectory evaluation reference content may include:

[0128] Determine expected trajectory information corresponding to the second visual trajectory information and the second radar trajectory information according to the trajectory evaluation reference content;

[0129] Determining whether the second visual trajectory information and the second radar trajectory information match the expected trajectory information;

[0130] When the judgment result is yes, determining that the second visual track information and the second radar track information do not meet the preset update optimization requirement condition;

[0131] When the judgment result is no, it is determined that the second visual track information and the second radar track information meet the preset update optimization requirement condition.

[0132] Further optionally, the performing corresponding trajectory information updating and optimizing operations on the second visual trajectory information and the second radar trajectory information according to the trajectory evaluation reference content may include:

[0133] According to the trajectory evaluation reference content, determining the abnormal trajectory information of the target from the second visual trajectory information and the second radar trajectory information;

[0134] According to the trajectory evaluation reference content, corresponding update and adjustment operations are performed on the target abnormal trajectory information.

[0135] It can be seen that this optional embodiment can perform corresponding trajectory information update optimization operations on the second visual trajectory information and the second radar trajectory information according to the determined trajectory evaluation reference content, which is beneficial to improving the comprehensiveness and integrity of a radar-visual joint calibration method, and further beneficial to improving the accuracy, reliability and timeliness of the update optimization of the visual trajectory information and the radar trajectory information, thereby helping to improve the accuracy and reliability of the visual trajectory information and the radar trajectory information, and further beneficial to improving the accuracy and reliability of the radar-visual joint calibration.

[0136] Embodiment 2

[0137] See also Figure 2 , Figure 2 FIG. 1 is a flow chart of another radar-visual joint calibration method disclosed in an embodiment of the present invention. Figure 2 The described method may be applied to a radar-visual joint calibration device, wherein the device may include a server, wherein the server includes a local server or a cloud server, which is not limited in the embodiment of the present invention. Figure 2 As shown, the radar-visual joint calibration method includes the following operations:

[0138] 201. Determine first visual track information of a target object based on a visual sensor, and determine first radar track information of the target object based on a radar sensor.

[0139] 202. Based on the target object, perform corresponding object association operations on the visual sensor and the radar sensor to obtain a calibration object association result.

[0140] 203. Determine whether the target object meets a preset trajectory rationalization condition according to the first visual trajectory information and the first radar trajectory information.

[0141] Optionally, the above-mentioned judgment of whether the target object satisfies the preset trajectory rationalization condition can be understood as: judging whether the first visual trajectory information matches the first radar trajectory information, if they match, the trajectory rationalization condition is met; if they do not match, the trajectory rationalization condition is not met, and further, judging which information in the first visual trajectory information and the first radar trajectory information is abnormal, and adjusting the abnormal information, which is not limited in the embodiment of the present invention.

[0142] 204. When it is determined that the target object meets the trajectory rationalization condition, the trajectory relationship result corresponding to the target object based on the visual level and the radar level is determined according to the calibration object association result, the first visual trajectory information and the first radar trajectory information.

[0143] 205. Determine, according to the trajectory relationship result, second visual trajectory information and second radar trajectory information of the target object based on the target driving area.

[0144] In the embodiment of the present invention, for other descriptions of step 201 to step 205, please refer to other detailed descriptions of step 101 to step 104 in embodiment 1, and the embodiment of the present invention will not be repeated here.

[0145] It can be seen that the embodiment of the present invention can determine the trajectory relationship result according to the determined first radar trajectory information, the first visual trajectory information and the object association result, and then determine the second radar trajectory information and the second visual trajectory information according to the trajectory relationship result, which is conducive to improving the comprehensiveness and rationality of a radar-visual joint calibration method, and then is conducive to improving the accuracy and reliability of the calculated second radar trajectory information and the second visual trajectory information, thereby helping to improve the accuracy and reliability of radar-visual calibration, and is conducive to improving the efficiency and convenience of radar-visual calibration, and further is conducive to improving the stability and reliability of radar-visual application; and, it is also possible to determine whether the target object meets the trajectory rationalization condition according to the first visual trajectory information and the first radar trajectory information, and if so, perform the subsequent trajectory relationship result determination operation, which is conducive to improving the comprehensiveness and integrity of a radar-visual joint calibration method, and then is conducive to improving the accuracy and reliability of the subsequently determined trajectory relationship result, the second visual trajectory information and the second radar trajectory information, and effectively deal with the situation where the subsequently determined trajectory relationship result, the second visual trajectory information and the second radar trajectory information are inaccurate due to the first visual trajectory information and the first radar trajectory information.

[0146] In an optional embodiment, the method may further include the following operations:

[0147] When it is determined that the target object does not meet the trajectory rationalization condition, analyzing a first impact on the data collection capability of the radar sensor according to the determined first scene information corresponding to the radar sensor, and analyzing a second impact on the data collection capability of the visual sensor according to the determined second scene information corresponding to the visual sensor;

[0148] According to the first affected situation and the second affected situation, performing corresponding track information adjustment operations on the first radar track information and the first visual track information to obtain a track information adjustment result;

[0149] According to the trajectory information adjustment result, a trajectory relationship result is determined, and the step of determining the second visual trajectory information and the second radar trajectory information of the target object based on the target driving area is performed according to the trajectory relationship result;

[0150] Among them, the first scene information includes one or more of atmospheric wind information, earth gravity information, temperature and humidity information and other noise factor parameter information; the second scene information includes one or more of lighting condition information, natural climate information and other scene recognition object information.

[0151] Optionally, the first affected situation of the data collection capability of the radar sensor can be used to determine one or more of the affected specific trajectory coordinate points, the affected specific trajectory, the specific affected degree, the specific affected condition, the specific affected phenomenon, and other information that can reflect the specific affected situation, which is not limited in the embodiments of the present invention.

[0152] Optionally, the second affected situation of the data collection capability of the visual sensor can be used to determine one or more of the affected specific trajectory coordinate points, the affected specific trajectory, the specific affected degree, the specific affected condition, the specific affected phenomenon, and other information that can reflect the specific affected situation, which is not limited in the embodiments of the present invention.

[0153] Optionally, the trajectory information adjustment result may include, but is not limited to, trajectory information in the first radar trajectory information and the first visual trajectory information that does not require information adjustment operation, and trajectory information that requires information adjustment operation and has completed information adjustment, which is not limited in the embodiment of the present invention.

[0154] Further optionally, the specific implementation method of determining the trajectory relationship result based on the trajectory information adjustment result can refer to but is not limited to the above-mentioned specific implementation method of determining the trajectory relationship result corresponding to the target object based on the visual level and the radar level according to the calibration object association result, the first visual trajectory information and the first radar trajectory information, and the embodiment of the present invention is not limited thereto.

[0155] It can be seen that this optional embodiment can provide corresponding response operations for the situation where the target object does not meet the trajectory rationalization conditions, and provide a trajectory information adjustment method for the first radar trajectory information and the first visual trajectory information, which is conducive to improving the comprehensiveness and rationality of the trajectory information adjustment method, and further helps to improve the accuracy and reliability of the subsequently determined trajectory relationship results, the second visual trajectory information and the second radar trajectory information.

[0156] In another optional embodiment, the first visual trajectory information and the first radar trajectory information are used to determine whether the target object meets a preset trajectory rationalization condition, which may include:

[0157] Determine first track time information corresponding to the first visual track information, and determine second track time information corresponding to the first radar track information;

[0158] According to the first trajectory time information and the second trajectory time information, all the first visual trajectory coordinate points and all the first radar trajectory coordinate points are divided into a first trajectory coordinate set that meets a preset time coincidence condition and / or a second trajectory coordinate set that does not meet the preset time coincidence condition;

[0159] Determining a first trajectory coincidence degree for the actual trajectory route according to the first trajectory coordinate set, and / or determining a second trajectory coincidence degree for the predicted trajectory route according to the second trajectory coordinate set;

[0160] Determining whether the target object meets a preset trajectory coincidence condition according to the first trajectory coincidence degree and / or the second trajectory coincidence degree;

[0161] When it is determined that the target object satisfies the trajectory coincidence condition, determining that the target object satisfies a preset trajectory rationalization condition;

[0162] When it is determined that the target object does not satisfy the trajectory coincidence condition, it is determined that the target object does not satisfy a preset trajectory rationalization condition.

[0163] Optionally, the first trajectory time information may include, but is not limited to, a collection time and / or occurrence time corresponding to each first visual trajectory coordinate point included in the first visual trajectory information, which is not limited in the embodiment of the present invention.

[0164] Optionally, the second trajectory time information may include, but is not limited to, the collection time and / or occurrence time corresponding to each first radar trajectory coordinate point included in the second radar trajectory information, which is not limited in this embodiment of the present invention.

[0165] Further optionally, the above-mentioned dividing all the first visual trajectory coordinate points and all the first radar trajectory coordinate points into a first trajectory coordinate set that meets a preset time coincidence condition and / or a second trajectory coordinate set that does not meet a preset time coincidence condition according to the first trajectory time information and the second trajectory time information may include:

[0166] According to the first trajectory time information and the second trajectory time information, determining a first trajectory coordinate set that meets a preset time coincidence condition from all first visual trajectory coordinate points and all first radar trajectory coordinate points;

[0167] The track coordinate points other than the first track coordinate set are determined from all the first visual track coordinate points and all the first radar track coordinate points as a second track coordinate set that does not meet a preset time coincidence condition.

[0168] Optionally, for satisfying the preset time coincidence condition and not satisfying the preset time coincidence condition, an example is given: for example, there is a first visual trajectory coordinate point and a first radar trajectory coordinate point whose occurrence time is both the first ten seconds, and / or there is a first radar trajectory coordinate point whose occurrence time is the last ten seconds and a first visual trajectory coordinate point whose occurrence time is both the last twenty seconds, then they are determined to be the first trajectory coordinate set that satisfies the preset time coincidence condition, and the first radar trajectory coordinate point whose occurrence time is the last ten seconds and the first visual trajectory coordinate point whose occurrence time is both the last twenty seconds are determined to be the second trajectory coordinate set that does not satisfy the preset time coincidence condition; for another example, when there is a first visual trajectory coordinate point whose occurrence time is the first ten seconds and a first radar trajectory coordinate point whose occurrence time is the last ten seconds, and there is no first radar trajectory coordinate point whose occurrence time is the first ten seconds, then the first visual trajectory coordinate point whose occurrence time is the first ten seconds and the first radar trajectory coordinate point whose occurrence time is the last ten seconds are determined to be the second trajectory coordinate set that does not satisfy the preset time coincidence condition, and other situations are similarly obtainable, and the embodiments of the present invention are not limited thereto.

[0169] Optionally, the first trajectory coincidence of the actual trajectory route can be understood as directly performing a trajectory coincidence analysis on the existing radar trajectory coordinate points and visual trajectory coordinate points with the same occurrence time, which is not limited in the embodiment of the present invention.

[0170] Optionally, for the second trajectory coincidence of the predicted trajectory route, it can be understood that for the existing radar trajectory coordinate points and visual trajectory coordinate points at different occurrence times, the predicted radar trajectory coordinate points and predicted visual trajectory coordinate points of the target object at the same occurrence time are first predicted and determined, and then the trajectory coincidence analysis is performed, which is not limited in the embodiments of the present invention.

[0171] Further optionally, determining the first trajectory coincidence degree for the actual trajectory route according to the first trajectory coordinate set may include:

[0172] A first actual trajectory route is determined according to each first visual trajectory coordinate point in the first trajectory coordinate set, and a second actual trajectory route is determined according to each first radar trajectory coordinate point in the first trajectory coordinate set; and a first trajectory overlap degree is determined according to the first actual trajectory route and the second actual trajectory route.

[0173] Further optionally, determining the second trajectory coincidence degree for the predicted trajectory route according to the second trajectory coordinate set may include:

[0174] The first predicted trajectory route is determined according to each first visual trajectory coordinate point in the second trajectory coordinate set and the determined predicted trajectory time period, and the second predicted trajectory route is determined according to each first radar trajectory coordinate point in the second trajectory coordinate set and the determined predicted trajectory time period; the second trajectory overlap is determined according to the first predicted trajectory route and the second predicted trajectory route.

[0175] Further optionally, judging whether the target object satisfies a preset trajectory coincidence condition according to the first trajectory coincidence degree and / or the second trajectory coincidence degree may include:

[0176] Determining a comprehensive trajectory overlap degree according to the first trajectory overlap degree and / or the second trajectory overlap degree;

[0177] Determine whether the comprehensive trajectory overlap is greater than or equal to a preset trajectory overlap threshold;

[0178] When the result of the judgment is yes, it is determined that the target object meets the preset trajectory coincidence condition;

[0179] When the judgment result is no, it is determined that the target object does not meet the preset trajectory coincidence condition.

[0180] Further optionally, determining the comprehensive trajectory overlap according to the first trajectory overlap and / or the second trajectory overlap may include:

[0181] When the first trajectory coincidence degree or the second trajectory coincidence degree is included, the first trajectory coincidence degree or the second trajectory coincidence degree is used to determine the comprehensive trajectory coincidence degree;

[0182] When the first trajectory overlap and the second trajectory overlap are included, the comprehensive trajectory overlap is determined according to the first trajectory overlap, the second trajectory overlap and a preset weighting scheme.

[0183] It can be seen that this optional embodiment can determine the first trajectory overlap for the actual trajectory route and / or the second trajectory overlap for the predicted trajectory route and then determine the trajectory overlap condition satisfaction result of the target object, and determine the trajectory rationalization condition satisfaction result based on the trajectory overlap condition satisfaction result, which is conducive to improving the comprehensiveness and rationality of the method for determining the trajectory overlap condition satisfaction result, and then is conducive to the comprehensiveness and rationality of the trajectory rationalization condition satisfaction result, and is conducive to improving the accuracy and reliability of the determined trajectory overlap condition satisfaction result, thereby being conducive to improving the accuracy and reliability of the determined trajectory rationalization condition satisfaction result.

[0184] In yet another optional embodiment, performing corresponding trajectory information adjustment operations on the first radar trajectory information and the first visual trajectory information according to the first affected situation and the second affected situation to obtain a trajectory information adjustment result may include:

[0185] According to the first affected situation, determining the target radar track coordinate point that needs to be adjusted and optimized from all the first radar track coordinate points, and / or, according to the second affected situation, determining the target visual track coordinate point that needs to be adjusted and optimized from all the first visual track coordinate points;

[0186] Determine a trajectory information optimization scheme corresponding to a target radar trajectory coordinate point and / or a target visual trajectory coordinate point according to the first affected situation, the second affected situation, the first radar trajectory information, and the second radar trajectory information;

[0187] A trajectory information adjustment result is determined according to the trajectory information optimization scheme, the first radar trajectory information, and the second radar trajectory information.

[0188] It can be seen that this optional embodiment can determine the target radar trajectory coordinate points and target visual trajectory coordinate points that need to be adjusted and optimized, and determine the trajectory information optimization plan to obtain the trajectory information adjustment result, which is beneficial to improving the comprehensiveness and rationality of the trajectory information adjustment method of the first radar trajectory information and the first visual trajectory information, and thus is beneficial to improving the adjustment accuracy and reliability of the first radar trajectory information and the first visual trajectory information, thereby helping to improve the accuracy and reliability of the determined trajectory information adjustment result.

[0189] Embodiment 3

[0190] See also Figure 3 , Figure 3 Schematic diagram of the structure of a radar-visual joint calibration device disclosed in an embodiment of the present invention. Figure 3 The described device may include a server, wherein the server includes a local server or a cloud server, which is not limited in the embodiment of the present invention. Figure 3 As shown, the radar-visual joint calibration device may include:

[0191] The track information acquisition module 301 is used to determine first visual track information of the target object based on the visual sensor, and to determine first radar track information of the target object based on the radar sensor.

[0192] The object association module 302 is used to perform corresponding object association operations on the visual sensor and the radar sensor based on the target object to obtain a calibration object association result.

[0193] The track relationship determination module 303 is used to determine the track relationship results corresponding to the target object based on the visual level and the radar level according to the calibration object association result, the first visual track information and the first radar track information.

[0194] The track information calculation module 304 is used to determine the second visual track information and the second radar track information of the target object based on the target driving area according to the track relationship result.

[0195] It can be seen that implementation Figure 3 The described radar and visual joint calibration device can determine the trajectory relationship result based on the determined first radar trajectory information, the first visual trajectory information and the object association result, and then determine the second radar trajectory information and the second visual trajectory information based on the trajectory relationship result, which is beneficial to improving the comprehensiveness and rationality of a radar and visual joint calibration method, and further beneficial to improving the accuracy and reliability of the calculated second radar trajectory information and the second visual trajectory information, thereby helping to improve the accuracy and reliability of radar and visual calibration, as well as helping to improve the efficiency and convenience of radar and visual calibration, and further helping to improve the stability and reliability of radar and visual applications.

[0196] In an optional embodiment, the first visual track information includes one or more first visual track coordinate points of the target object; and the first radar track information includes one or more first radar track coordinate points of the target object.

[0197] Further optionally, the trajectory relationship determination module 303 determines the trajectory relationship result corresponding to the target object based on the visual level and the radar level according to the calibration object association result, the first visual trajectory information and the first radar trajectory information, specifically including:

[0198] According to the calibration object association result and the preset association coordinate determination method, corresponding coordinate point matching operations are performed on all first visual track coordinate points and all first radar coordinate points to obtain one or more coordinate point matching sets;

[0199] For each coordinate point matching set, determining the basic coordinate relationship corresponding to the first visual track coordinate point and the first radar track coordinate point in the coordinate point matching set according to a preset coordinate relationship calculation method;

[0200] According to the matching set of all coordinate points and their corresponding basic coordinate relationships, the trajectory relationship results corresponding to the target object based on the visual level and radar level are determined.

[0201] It can be seen that implementation Figure 4 The described device can determine one or more coordinate point matching sets and their corresponding basic coordinate relationships, and then determine the trajectory relationship results of the target object, which is conducive to improving the comprehensiveness and rationality of the trajectory relationship result determination method, and then is conducive to improving the accuracy and reliability of the determined trajectory relationship results, thereby facilitating improving the accuracy and reliability of the radar vision calibration results determined based on the trajectory relationship results and the predicted new radar trajectory information and new visual trajectory information.

[0202] In another optional embodiment, Figure 4 As shown, the device may also include:

[0203] The optimization information determination module 305 is used to determine the trajectory evaluation reference content that can be used to verify and optimize the second visual trajectory information and the second radar trajectory information after the trajectory information calculation module 304 determines the second visual trajectory information and the second radar trajectory information of the target object based on the target driving area according to the trajectory relationship result, and the trajectory evaluation reference content includes standardized trajectory information and / or standardized trajectory relationship results.

[0204] The first judgment module 306 is used to judge whether the second visual trajectory information and the second radar trajectory information meet the preset update optimization requirement conditions according to the trajectory evaluation reference content.

[0205] The trajectory information optimization module 307 is used to perform corresponding trajectory information update optimization operations on the second visual trajectory information and the second radar trajectory information according to the trajectory evaluation reference content when the first judgment module 306 judges that the second visual trajectory information and the second radar trajectory information meet the update optimization requirement conditions;

[0206] The standardized track information, the second visual track information and the second radar track information all include corresponding track coordinate points.

[0207] It can be seen that implementation Figure 4 The described device can also perform corresponding trajectory information update optimization operations on the second visual trajectory information and the second radar trajectory information according to the determined trajectory evaluation reference content, which is beneficial to improving the comprehensiveness and integrity of a radar-visual joint calibration method, and further beneficial to improving the accuracy, reliability and timeliness of the update optimization of the visual trajectory information and the radar trajectory information, thereby helping to improve the accuracy and reliability of the visual trajectory information and the radar trajectory information, and further beneficial to improving the accuracy and reliability of the radar-visual joint calibration.

[0208] In yet another optional embodiment, Figure 4 As shown, the device may also include:

[0209] The second judgment module 308 is used to perform corresponding object association operations on the visual sensor and the radar sensor based on the target object in the object association module 302, and after obtaining the calibration object association result, judge whether the target object meets the preset trajectory rationalization condition according to the first visual trajectory information and the first radar trajectory information; when the judgment result is yes, the trajectory relationship determination module 303 executes the step of determining the trajectory relationship result corresponding to the target object based on the visual level and the radar level according to the calibration object association result, the first visual trajectory information and the first radar trajectory information.

[0210] It can be seen that implementation Figure 4 The described device can also determine whether the target object meets the trajectory rationalization conditions based on the first visual trajectory information and the first radar trajectory information. If so, it can perform subsequent trajectory relationship result determination operations, which is beneficial to improving the comprehensiveness and integrity of a radar-visual joint calibration method, and thus is beneficial to improving the accuracy and reliability of the subsequently determined trajectory relationship results, the second visual trajectory information and the second radar trajectory information, and effectively deal with the situation where the subsequently determined trajectory relationship results, the second visual trajectory information and the second radar trajectory information are inaccurate due to the first visual trajectory information and the first radar trajectory information.

[0211] In yet another optional embodiment, Figure 4 As shown, the device may also include:

[0212] The trajectory information adjustment module 309 is used to analyze a first affected situation of the data acquisition capability of the radar sensor according to the determined first scene information corresponding to the radar sensor, and to analyze a second affected situation of the data acquisition capability of the visual sensor according to the determined second scene information corresponding to the visual sensor when the second judgment module 308 determines that the target object does not meet the trajectory rationalization condition; and to perform corresponding trajectory information adjustment operations on the first radar trajectory information and the first visual trajectory information according to the first affected situation and the second affected situation to obtain a trajectory information adjustment result.

[0213] The trajectory relationship determination module 303 is further used to determine the trajectory relationship result according to the trajectory information adjustment result, and trigger the trajectory information calculation module 304 to perform a step of determining the second visual trajectory information and the second radar trajectory information of the target object based on the target driving area according to the trajectory relationship result;

[0214] Among them, the first scene information includes one or more of atmospheric wind information, earth gravity information, temperature and humidity information and other noise factor parameter information; the second scene information includes one or more of lighting condition information, natural climate information and other scene recognition object information.

[0215] It can be seen that implementation Figure 4 The described device can also provide corresponding response operations for situations where the target object does not meet the trajectory rationalization conditions, and provide trajectory information adjustment methods for the first radar trajectory information and the first visual trajectory information, which is conducive to improving the comprehensiveness and rationality of the trajectory information adjustment method, and thus is conducive to improving the accuracy and reliability of the subsequently determined trajectory relationship results, the second visual trajectory information and the second radar trajectory information.

[0216] In another optional embodiment, the second judgment module 308 judges whether the target object meets the preset trajectory rationalization condition according to the first visual trajectory information and the first radar trajectory information in a manner specifically including:

[0217] Determine first track time information corresponding to the first visual track information, and determine second track time information corresponding to the first radar track information;

[0218] According to the first trajectory time information and the second trajectory time information, all the first visual trajectory coordinate points and all the first radar trajectory coordinate points are divided into a first trajectory coordinate set that meets a preset time coincidence condition and / or a second trajectory coordinate set that does not meet the preset time coincidence condition;

[0219] Determining a first trajectory coincidence degree for the actual trajectory route according to the first trajectory coordinate set, and / or determining a second trajectory coincidence degree for the predicted trajectory route according to the second trajectory coordinate set;

[0220] Determining whether the target object meets a preset trajectory coincidence condition according to the first trajectory coincidence degree and / or the second trajectory coincidence degree;

[0221] When it is determined that the target object satisfies the trajectory coincidence condition, determining that the target object satisfies a preset trajectory rationalization condition;

[0222] When it is determined that the target object does not satisfy the trajectory coincidence condition, it is determined that the target object does not satisfy a preset trajectory rationalization condition.

[0223] It can be seen that implementation Figure 4 The described device can also determine a first trajectory overlap degree for an actual trajectory route and / or a second trajectory overlap degree for a predicted trajectory route, and thereby determine a trajectory overlap condition satisfaction result for the target object, and determine a trajectory rationalization condition satisfaction result based on the trajectory overlap condition satisfaction result, which is beneficial to improving the comprehensiveness and rationality of the method for determining the trajectory overlap condition satisfaction result, and thereby is beneficial to improving the comprehensiveness and rationality of the trajectory rationalization condition satisfaction result, and is beneficial to improving the accuracy and reliability of the determined trajectory overlap condition satisfaction result, and thereby is beneficial to improving the accuracy and reliability of the determined trajectory rationalization condition satisfaction result.

[0224] In another optional embodiment, the trajectory information adjustment module 309 performs corresponding trajectory information adjustment operations on the first radar trajectory information and the first visual trajectory information according to the first affected situation and the second affected situation, and the manner of obtaining the trajectory information adjustment result specifically includes:

[0225] According to the first affected situation, determining the target radar track coordinate point that needs to be adjusted and optimized from all the first radar track coordinate points, and / or, according to the second affected situation, determining the target visual track coordinate point that needs to be adjusted and optimized from all the first visual track coordinate points;

[0226] Determine a trajectory information optimization scheme corresponding to a target radar trajectory coordinate point and / or a target visual trajectory coordinate point according to the first affected situation, the second affected situation, the first radar trajectory information, and the second radar trajectory information;

[0227] A trajectory information adjustment result is determined according to the trajectory information optimization scheme, the first radar trajectory information, and the second radar trajectory information.

[0228] It can be seen that implementation Figure 4 The described device can also determine the target radar trajectory coordinate points and target visual trajectory coordinate points that need to be adjusted and optimized, and determine the trajectory information optimization plan to obtain the trajectory information adjustment result, which is conducive to improving the comprehensiveness and rationality of the trajectory information adjustment method of the first radar trajectory information and the first visual trajectory information, and thus is conducive to improving the adjustment accuracy and reliability of the first radar trajectory information and the first visual trajectory information, thereby helping to improve the accuracy and reliability of the determined trajectory information adjustment result.

[0229] Embodiment 4

[0230] See also Figure 5 , Figure 5 is a structural diagram of another radar-visual joint calibration device disclosed in an embodiment of the present invention. Figure 5 The described device may include a server, wherein the server includes a local server or a cloud server, which is not limited in the embodiment of the present invention. Figure 5 As shown, the device may include:

[0231] A memory 401 storing executable program codes;

[0232] a processor 402 coupled to the memory 401;

[0233] Furthermore, it may also include an input interface 403 and an output interface 404 coupled to the processor 402;

[0234] The processor 402 calls the executable program code stored in the memory 401 to execute the steps in the radar-visual joint calibration method described in the first or second embodiment.

[0235] Embodiment 5

[0236] An embodiment of the present invention discloses a computer storage medium storing a computer program for electronic data exchange, wherein the computer program enables a computer to execute the steps of a radar-visual joint calibration method described in Embodiment 1 or Embodiment 2.

[0237] Embodiment 6

[0238] An embodiment of the present invention discloses a computer program product, which includes a non-transitory computer-readable storage medium storing a computer program, and the computer program is operable to enable a computer to execute the steps in a radar-visual joint calibration method described in Embodiment 1 or Embodiment 2.

[0239] The device embodiments described above are only illustrative, wherein the modules described as separate components may or may not be physically separated, and the components displayed as modules may or may not be physical modules, i.e., they may be located in one place, or they may be distributed on multiple network modules. Some or all of the modules may be selected according to actual needs to achieve the purpose of the scheme of this embodiment. Those of ordinary skill in the art may understand and implement it without creative work.

[0240] Through the specific description of the above embodiments, those skilled in the art can clearly understand that each implementation method can be implemented by means of software plus a necessary general hardware platform, and of course, it can also be implemented by hardware. Based on such an understanding, the above technical solution can be essentially or partly contributed to the prior art in the form of a software product, and the computer software product can be stored in a computer-readable storage medium, and the storage medium includes a read-only memory (ROM), a random access memory (RAM), a programmable read-only memory (PROM), an erasable programmable read-only memory (EPROM), a one-time programmable read-only memory (OTPROM), an electronically erasable rewritable read-only memory (EEPROM), a compact disc (CD-ROM) or other optical disc storage, a magnetic disk storage, a magnetic tape storage, or any other computer-readable medium that can be used to carry or store data.

[0241] Finally, it should be noted that the method and device for combined radar and vision calibration disclosed in the embodiments of the present invention only disclose the preferred embodiments of the present invention, which are only used to illustrate the technical solution of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that the technical solutions described in the aforementioned embodiments may still be modified, or some of the technical features thereof may be replaced by equivalents. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A radar-visual joint calibration method, characterized in that: The method comprises: Determine first visual track information of the target object based on the visual sensor, and determine first radar track information of the target object based on the radar sensor; Based on the target object, performing corresponding object association operations on the visual sensor and the radar sensor to obtain a calibration object association result; Determine, according to the calibration object association result, the first visual track information and the first radar track information, a track relationship result corresponding to the target object based on a visual level and a radar level; According to the trajectory relationship result, second visual trajectory information and second radar trajectory information of the target object based on the target driving area are determined.

2. The method for combined radar and visual calibration according to claim 1, characterized in that: The first visual track information includes one or more first visual track coordinate points of the target object; the first radar track information includes one or more first radar track coordinate points of the target object; And, determining the trajectory relationship result corresponding to the target object based on the visual level and the radar level according to the calibration object association result, the first visual trajectory information and the first radar trajectory information, comprises: According to the calibration object association result and the preset association coordinate determination method, performing corresponding coordinate point matching operations on all the first visual track coordinate points and all the first radar coordinate points to obtain one or more coordinate point matching sets; For each of the coordinate point matching sets, determining the basic coordinate relationship corresponding to the first visual track coordinate point and the first radar track coordinate point in the coordinate point matching set according to a preset coordinate relationship calculation method; According to all the coordinate point matching sets and their corresponding basic coordinate relationships, the trajectory relationship results corresponding to the target object based on the visual level and the radar level are determined.

3. The radar-visual joint calibration method according to claim 1, characterized in that: After determining the second visual track information and the second radar track information of the target object based on the target driving area according to the track relationship result, the method further includes: Determining a trajectory evaluation reference content that can be used to verify and optimize the second visual trajectory information and the second radar trajectory information, the trajectory evaluation reference content including standardized trajectory information and / or standardized trajectory relationship results; According to the trajectory evaluation reference content, determining whether the second visual trajectory information and the second radar trajectory information meet a preset update optimization requirement condition; When the judgment result is yes, performing corresponding trajectory information update optimization operations on the second visual trajectory information and the second radar trajectory information according to the trajectory evaluation reference content; The standardized track information, the second visual track information and the second radar track information all include corresponding track coordinate points.

4. A radar-visual joint calibration method according to any one of claims 1 to 3, characterized in that: After performing corresponding object association operations on the visual sensor and the radar sensor based on the target object to obtain a calibration object association result, the method further includes: Determining whether the target object meets a preset trajectory rationalization condition according to the first visual trajectory information and the first radar trajectory information; When it is determined that the target object meets the trajectory rationalization condition, the step of determining the trajectory relationship result corresponding to the target object based on the visual level and the radar level according to the calibration object association result, the first visual trajectory information and the first radar trajectory information is executed.

5. The method for combined radar and visual calibration according to claim 4, characterized in that: The method further comprises: When it is determined that the target object does not satisfy the trajectory rationalization condition, analyzing a first impact on the data collection capability of the radar sensor according to the determined first scene information corresponding to the radar sensor, and analyzing a second impact on the data collection capability of the visual sensor according to the determined second scene information corresponding to the visual sensor; According to the first affected condition and the second affected condition, performing corresponding trajectory information adjustment operations on the first radar trajectory information and the first visual trajectory information to obtain a trajectory information adjustment result; Determine a trajectory relationship result according to the trajectory information adjustment result, and perform the step of determining the second visual trajectory information and the second radar trajectory information of the target object based on the target driving area according to the trajectory relationship result; Among them, the first scene information includes one or more of atmospheric wind information, earth gravity information, temperature and humidity information and other noise factor parameter information; the second scene information includes one or more of lighting condition information, natural climate information and other scene recognition object information.

6. The method for combined radar and visual calibration according to claim 4, characterized in that: The determining, based on the first visual trajectory information and the first radar trajectory information, whether the target object satisfies a preset trajectory rationalization condition includes: Determine first track time information corresponding to the first visual track information, and determine second track time information corresponding to the first radar track information; According to the first trajectory time information and the second trajectory time information, all the first visual trajectory coordinate points and all the first radar trajectory coordinate points are divided into a first trajectory coordinate set that meets a preset time coincidence condition and / or a second trajectory coordinate set that does not meet a preset time coincidence condition; Determining a first trajectory coincidence degree for an actual trajectory route according to the first trajectory coordinate set, and / or determining a second trajectory coincidence degree for a predicted trajectory route according to the second trajectory coordinate set; According to the first trajectory overlap degree and / or the second trajectory overlap degree, determining whether the target object meets a preset trajectory overlap condition; When it is determined that the target object satisfies the trajectory coincidence condition, determining that the target object satisfies a preset trajectory rationalization condition; When it is determined that the target object does not satisfy the trajectory coincidence condition, it is determined that the target object does not satisfy a preset trajectory rationalization condition.

7. The method for combined radar and visual calibration according to claim 5, characterized in that: The performing corresponding trajectory information adjustment operations on the first radar trajectory information and the first visual trajectory information according to the first affected situation and the second affected situation to obtain a trajectory information adjustment result includes: According to the first affected situation, determining the target radar track coordinate point that needs to be adjusted and optimized from all the first radar track coordinate points, and / or, according to the second affected situation, determining the target visual track coordinate point that needs to be adjusted and optimized from all the first visual track coordinate points; Determine a trajectory information optimization scheme corresponding to the target radar trajectory coordinate point and / or the target visual trajectory coordinate point according to the first affected situation, the second affected situation, the first radar trajectory information, and the second radar trajectory information; A trajectory information adjustment result is determined according to the trajectory information optimization scheme, the first radar trajectory information, and the second radar trajectory information.

8. A combined radar and visual calibration device, characterized in that: The device comprises: A track information acquisition module, used to determine first visual track information of the target object based on the visual sensor, and to determine first radar track information of the target object based on the radar sensor; An object association module, configured to perform corresponding object association operations on the visual sensor and the radar sensor based on the target object to obtain a calibration object association result; A trajectory relationship determination module, used to determine the trajectory relationship result corresponding to the target object based on the visual level and the radar level according to the calibration object association result, the first visual trajectory information and the first radar trajectory information; The trajectory information calculation module is used to determine the second visual trajectory information and the second radar trajectory information of the target object based on the target driving area according to the trajectory relationship result.

9. A combined radar and visual calibration device, characterized in that: The device comprises: A memory storing executable program code; a processor coupled to the memory; The processor calls the executable program code stored in the memory to execute a radar-visual joint calibration method as described in any one of claims 1-7.

10. A computer storage medium, characterized in that: The computer storage medium stores computer instructions, and when the computer instructions are called, they are used to execute a radar-visual joint calibration method as described in any one of claims 1-7.

Images