Calibration method, device, electronic device and storage medium

Abstract

The present disclosure proposes a calibration method, device, electronic device and storage medium, which relate to the field of artificial intelligence technology, specifically to the field of virtual reality, augmented reality and autonomous driving technology. The solution is: based on the virtual calibration environment corresponding to the target vehicle and the on-board data information output by the on-board sensor model, the reference pose information of the on-board sensor and the virtual calibration scene corresponding to the target vehicle under the reference pose information are determined; the on-board data information output by the on-board sensor model under the reference pose information and the virtual calibration scene are fused to obtain a target fused image; based on the target fused image, the reference pose information is calibrated to determine the target pose information of the on-board sensor. In this way, various calibration scenes can be built quickly and at low cost, and the pose information of the on-board sensor can be calibrated in the virtual calibration environment, thereby improving the calibration efficiency and accuracy of the pose information of the on-board sensor.

Description

Technical Field

[0001] The present disclosure relates to the field of artificial intelligence technology, specifically to the field of virtual reality, augmented reality and autonomous driving technology, and in particular to calibration methods, devices, electronic devices and storage media. Background Art

[0002] Currently, when a vehicle is in motion (e.g., autonomous driving), multiple sensors must work together to sense and locate the vehicle. This collaborative work requires that the position and posture information of the onboard sensors (lidar and cameras) be known and pre-calibrated. Therefore, calibrating the position and posture information of onboard sensors is crucial. Summary of the Invention

[0003] The present disclosure provides a calibration method, device, electronic device and storage medium.

[0004] According to one aspect of the present disclosure, a calibration method is provided, comprising: simulating a calibration environment in which a target vehicle is located and an on-board sensor of the target vehicle respectively to obtain a virtual calibration environment and an on-board sensor model corresponding to the target vehicle; determining reference pose information of the on-board sensor and a virtual calibration scene corresponding to the target vehicle under the reference pose information based on the virtual calibration environment and the on-board data information output by the on-board sensor model; fusing the on-board data information output by the on-board sensor model under the reference pose information and the virtual calibration scene to obtain a target fused image; and calibrating the reference pose information based on the target fused image to determine the target pose information of the on-board sensor.

[0005] According to another aspect of the present disclosure, a calibration device is provided, including: a simulation module for simulating a calibration environment in which a target vehicle is located and the on-board sensors of the target vehicle, respectively, to obtain a virtual calibration environment and an on-board sensor model corresponding to the target vehicle; a determination module for determining, based on the virtual calibration environment and the on-board data information output by the on-board sensor model, reference pose information of the on-board sensor and a virtual calibration scene corresponding to the target vehicle under the reference pose information; a fusion module for fusing the on-board data information output by the on-board sensor model under the reference pose information and the virtual calibration scene to obtain a target fused image; and a calibration module for calibrating the reference pose information based on the target fused image to determine the target pose information of the on-board sensor.

[0006] According to another aspect of the present disclosure, an electronic device is provided, comprising: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to execute the method described in the embodiment of the first aspect of the present disclosure.

[0007] According to another aspect of the present disclosure, a non-transitory computer-readable storage medium storing computer instructions is provided, wherein the computer instructions are used to enable the computer to execute the method described in the embodiment of the first aspect of the present disclosure.

[0008] According to another aspect of the present disclosure, a computer program product is provided, including a computer program, wherein when the computer program is executed by a processor, the computer program implements the method described in the embodiment of the first aspect of the present disclosure.

[0009] It should be understood that the contents described in this section are not intended to identify the key or important features of the embodiments of the present disclosure, nor are they intended to limit the scope of the present disclosure. Other features of the present disclosure will become readily understood through the following description. BRIEF DESCRIPTION OF THE DRAWINGS

[0010] The accompanying drawings are provided to facilitate a better understanding of the present invention and do not constitute a limitation of the present disclosure.

[0011] Figure 1 is a schematic diagram according to a first embodiment of the present disclosure;

[0012] Figure 2 is a schematic diagram of a virtual calibration environment in an embodiment of the present application;

[0013] Figure 3 is a schematic diagram according to a second embodiment of the present disclosure;

[0014] Figure 4 is a schematic diagram according to a third embodiment of the present disclosure;

[0015] Figure 5 is a schematic diagram according to a fourth embodiment of the present disclosure;

[0016] Figure 6 is a schematic diagram according to a fifth embodiment of the present disclosure;

[0017] Figure 7 is a schematic diagram according to a sixth embodiment of the present disclosure;

[0018] Figure 8 A flow chart of a calibration method provided in an embodiment of the present disclosure;

[0019] Figure 9 is a schematic diagram according to a seventh embodiment of the present disclosure;

[0020] Figure 10 A schematic block diagram of an example electronic device that can be used to implement embodiments of the present disclosure is shown. DETAILED DESCRIPTION

[0021] The following description of exemplary embodiments of the present disclosure is made in conjunction with the accompanying drawings, including various details of the embodiments of the present disclosure to facilitate understanding. These details should be considered as merely exemplary. Therefore, those skilled in the art will recognize that various changes and modifications may be made to the embodiments described herein without departing from the scope and spirit of the present disclosure. Similarly, for the sake of clarity and conciseness, descriptions of well-known functions and structures are omitted in the following description.

[0022] In related technologies, the position and posture information of vehicle sensors is calibrated by manually holding a calibration device (such as a calibration plate) or fixing the calibration device with a bracket after placing the vehicle at a real calibration site. If the design needs to be modified, the calibration device needs to be remade and the position and posture information of the vehicle sensors needs to be confirmed again. However, due to the limited real calibration site, some calibration requirements are difficult to achieve. In addition, the calibration equipment has a long production cycle, is inconvenient to replace, consumes a lot of manpower, is expensive, is greatly affected by surrounding environmental factors, and has low calibration efficiency.

[0023] Therefore, in order to solve the above problems, the present disclosure proposes a calibration method, device, electronic device and storage medium.

[0024] The following describes the calibration method, device, electronic device, and storage medium of the embodiments of the present disclosure with reference to the accompanying drawings.

[0025] Figure 1 It should be noted that the calibration method implemented in the present disclosure is configured in a calibration device for illustration, and the calibration device can be applied to any electronic device to enable the electronic device to perform the calibration function.

[0026] Among them, the electronic device can be any device with computing capabilities, such as a personal computer (PC), a mobile terminal, etc. The mobile terminal can be, for example, a mobile phone, a tablet computer, a personal digital assistant, a wearable device, and other hardware devices with various operating systems, touch screens and / or display screens.

[0027] like Figure 1 As shown, the calibration method may include the following steps:

[0028] Step 101 : Simulating the calibration environment of the target vehicle and the onboard sensors of the target vehicle respectively to obtain a virtual calibration environment and an onboard sensor model corresponding to the target vehicle.

[0029] In order to reduce the construction cost of the calibration environment and improve the calibration efficiency of the vehicle's on-board sensors, in the embodiment of the present disclosure, the calibration environment of the target vehicle can be simulated to construct a virtual calibration environment corresponding to the target vehicle. At the same time, the on-board sensors of the target vehicle can also be simulated to obtain an on-board sensor model.

[0030] Step 102 : Determine reference pose information of the vehicle-mounted sensor and a virtual calibration scene corresponding to the target vehicle under the reference pose information based on the virtual calibration environment and the vehicle-mounted data information output by the vehicle-mounted sensor model.

[0031] Furthermore, if Figure 2 As shown, the onboard sensor model (lidar model and camera model) is loaded into the designated location of the target vehicle model, and the onboard data information output by the onboard sensor model is obtained. The onboard data information can be used to indicate the position information of the target vehicle and the image information of the calibration environment in which the target vehicle is located. Furthermore, based on the onboard data information, the relative position information between the target vehicle and the corresponding calibration device can be determined. Further, it can be determined whether the relative position information matches the set position information. If the relative position information does not match the set position information, the position information of the onboard sensor can be adjusted to obtain the reference position information of the onboard sensor.

[0032] Furthermore, based on the reference posture information of the on-board sensor, the posture information of the on-board sensor model in the virtual calibration environment can be adjusted to the reference posture information, and then the on-board data information output by the on-board sensor model under the reference posture information can be obtained. According to the target vehicle model, the calibration device model and the on-board data information corresponding to the reference posture information, a virtual calibration scene corresponding to the target vehicle can be generated, wherein the virtual calibration scene may include the target vehicle model, the calibration device model and the posture information between the target vehicle model and the calibration device model corresponding to the reference posture information.

[0033] Step 103 , fusing the vehicle data information output by the vehicle sensor model under the reference pose information and the virtual calibration scene to obtain a target fused image.

[0034] Furthermore, the on-board data information output by the on-board sensor model under the reference posture information is obtained. Based on the on-board data information, the image information of the calibration environment in which the target vehicle is located can be determined. Then, the image information of the calibration environment in which the target vehicle is located is fused with the virtual calibration scene to obtain a target fused image.

[0035] Step 104 : calibrate the reference pose information based on the target fusion image to determine the target pose information of the vehicle-mounted sensor.

[0036] Furthermore, based on the target fusion image, the position information of the target object in the target fusion image is determined. When the position information of the target object in the target fusion image meets expectations, the reference pose information is used as the target pose information of the on-board sensor. When the position information of the target object in the target fusion image does not meet expectations, the reference pose information of the on-board sensor is adjusted (for example, the shooting angle and shooting direction of the camera), and the target fusion image is re-determined based on the adjusted reference pose information until the position information of the reference device in the target fusion image meets expectations, and the pose information of the on-board sensor at this time is used as the target pose information.

[0037] In summary, the calibration environment of the target vehicle and the on-board sensors of the target vehicle are simulated respectively to obtain the virtual calibration environment and on-board sensor model corresponding to the target vehicle; according to the virtual calibration environment and the on-board data information output by the on-board sensor model, the reference posture information of the on-board sensor and the virtual calibration scene corresponding to the target vehicle under the reference posture information are determined; the on-board data information output by the on-board sensor model under the reference posture information and the virtual calibration scene are fused to obtain the target fused image; according to the target fused image, the reference posture information is calibrated to determine the target posture information of the on-board sensor, thereby, through By simulating the calibration environment in which the target vehicle is located and the on-board sensors of the target vehicle respectively, various calibration scenarios can be built quickly and at low cost. Then, the reference pose information of the on-board sensor is determined based on the virtual calibration environment and the on-board data information output by the on-board sensor model, and the pose information of the on-board sensor can be calibrated efficiently in the virtual calibration environment. In addition, the on-board data information output by the on-board sensor model under the reference pose information and the virtual calibration scene are fused to obtain a target fused image. Based on the target fused image, the reference pose information is calibrated, which further improves the accuracy of the pose information of the on-board sensor.

[0038] In order to clearly illustrate how the above embodiment determines the reference pose information of the on-board sensor and the virtual calibration scene corresponding to the target vehicle under the reference pose information based on the virtual calibration environment and the on-board data information output by the on-board sensor model, the present disclosure proposes another calibration method.

[0039] Figure 3 is a schematic diagram according to the second embodiment of the present disclosure.

[0040] like Figure 3 As shown, the calibration method may include the following steps:

[0041] Step 301 : Simulate the calibration environment of the target vehicle and the onboard sensors of the target vehicle respectively to obtain a virtual calibration environment and onboard sensor model corresponding to the target vehicle.

[0042] Step 302 : Calibrate the pose information of the vehicle-mounted sensor according to the virtual calibration environment and the vehicle-mounted data information output by the vehicle-mounted sensor model to obtain reference pose information.

[0043] As a possible implementation method of an embodiment of the present disclosure, the relative posture information between the target vehicle and the calibration device indicated by the on-board data information can be compared with the set posture information between the target vehicle and the calibration device to determine whether the relative posture information matches the set posture information. When the relative posture information does not match the set posture information, the posture information of the on-board sensor is adjusted so that the relative posture information matches the set posture information. The posture information of the on-board sensor corresponding to the relative posture information matches the set posture information is used as the reference posture information of the on-board sensor.

[0044] Step 303: Determine a virtual calibration scene corresponding to the target vehicle based on the reference posture information.

[0045] Furthermore, based on the reference pose information of the on-board sensor, the pose information of the on-board sensor model in the virtual calibration environment can be adjusted, and the pose information between the target vehicle model output by the on-board sensor model after the pose information adjustment and the calibration device model can be obtained. Furthermore, based on the target vehicle model, the calibration device model, and the pose information between the target vehicle model and the calibration device model, the virtual calibration scene corresponding to the target vehicle can be obtained.

[0046] In step 304 , the vehicle data information output by the vehicle sensor model under the reference posture information and the virtual calibration scene are fused to obtain a target fused image.

[0047] Step 305 : Calibrate the reference pose information based on the target fusion image to determine the target pose information of the vehicle-mounted sensor.

[0048] It should be noted that the execution process of step 301 and steps 304 to 305 can be implemented in any way in the embodiments of the present disclosure, and the embodiments of the present disclosure do not limit this and will not be described in detail.

[0049] In summary, the pose information of the on-board sensor is calibrated according to the virtual calibration environment and the on-board data information output by the on-board sensor model to obtain reference pose information; according to the reference pose information, the virtual calibration scene corresponding to the target vehicle is determined, thereby realizing the calibration of the pose information of the on-board sensor in the virtual calibration environment, improving the calibration efficiency of the pose information of the on-board sensor, and at the same time, according to the reference pose information obtained by calibration, the virtual calibration scene corresponding to the target vehicle can be effectively determined, thereby fusing the on-board data information and the virtual calibration scene to obtain the target fusion image; according to the target fusion image, the reference pose information is calibrated to further improve the accuracy of the pose information of the on-board sensor.

[0050] In order to more clearly illustrate how the above-mentioned embodiment determines the reference posture information of the on-board sensor and the virtual calibration scene corresponding to the target vehicle under the reference posture information based on the virtual calibration environment and the on-board data information output by the on-board sensor model, the present disclosure proposes another calibration method.

[0051] Figure 4 is a schematic diagram according to a third embodiment of the present disclosure.

[0052] like Figure 4 As shown, the calibration method may include the following steps:

[0053] Step 401 : Simulate the calibration environment of the target vehicle and the onboard sensors of the target vehicle respectively to obtain a virtual calibration environment and onboard sensor model corresponding to the target vehicle.

[0054] Step 402 : determining initial pose information of the onboard sensor based on the pose information of the onboard sensor model on the target vehicle model in the virtual calibration environment.

[0055] In an embodiment of the present disclosure, the on-board sensor model can be set at a specified position of the target vehicle model, so that the posture information of the on-board sensor model on the target vehicle model can be obtained, and the posture information of the on-board sensor model on the target vehicle model can be used as the initial posture information of the on-board sensor.

[0056] Step 403: Adjust the initial position and posture information of the vehicle-mounted sensor according to the vehicle-mounted data information to obtain reference position and posture information of the vehicle-mounted sensor.

[0057] In order to accurately determine the reference pose information of the on-board sensor, as an example, the relative pose information between the target vehicle and the corresponding calibration device is determined based on the on-board data information; based on the set pose information between the target vehicle and the corresponding calibration device and the difference between the relative pose information, the initial pose information of the on-board sensor is adjusted to obtain the reference pose information of the on-board sensor.

[0058] In the disclosed embodiment, the on-board data information can be used to indicate the position information of the target vehicle and the position information of the calibration device. Then, based on the position information of the target vehicle and the position information of the calibration device, the relative position information between the target vehicle and the calibration device can be determined. Then, based on the difference between the relative position information and the corresponding set position information, the initial position information of the on-board sensor is adjusted to obtain the reference position information of the on-board sensor, so as to minimize the difference between the relative position information and the corresponding set position information. Thus, according to the on-board data information, the initial position information of the on-board sensor is adjusted to accurately determine the reference position information of the on-board sensor.

[0059] As another example, a virtual calibration environment is rendered in three dimensions to display relative posture information between a target vehicle in simulated data information and a corresponding calibration device; in response to a first user operation, reference posture information of the on-board sensor is determined; wherein the reference posture information is generated based on the difference between the set posture information and the relative posture information between the target vehicle and the corresponding calibration device; and the initial posture information of the on-board sensor is adjusted based on the reference posture information.

[0060] That is to say, in order to make the relative posture information between the target vehicle and the corresponding calibration device more intuitively displayed, so that the posture information of the on-board sensor can be calibrated more accurately, in the embodiment of the present disclosure, the relevant code can be executed to perform three-dimensional rendering of the virtual calibration environment, such as rendering to a virtual reality (VR) head-mounted display device. The user can determine the set posture information between the target vehicle and the corresponding calibration device and the difference between the relative posture information based on the relative posture information between the target vehicle and the calibration device. Therefore, the user can determine the reference posture information to which the on-board sensor is to be adjusted based on the difference. The user can input the reference posture information to which the on-board sensor is to be adjusted into the calibration device. Therefore, the calibration device can adjust the initial posture information of the on-board sensor based on the reference posture information input by the user.

[0061] Step 404: Determine a virtual calibration scene corresponding to the target vehicle based on the reference posture information.

[0062] Step 405 : Fusing the vehicle data information output by the vehicle sensor model under the reference pose information and the virtual calibration scene to obtain a target fused image.

[0063] Step 406 : Calibrate the reference pose information based on the target fusion image to determine the target pose information of the vehicle-mounted sensor.

[0064] It should be noted that the execution process of steps 401 to 402 and steps 404 to 406 can be implemented in any way in the embodiments of the present disclosure, and the embodiments of the present disclosure do not limit this and will not be described in detail.

[0065] In summary, the initial pose information of the on-board sensor is determined based on the pose information of the on-board sensor model on the target vehicle model in the virtual calibration environment; the initial pose information of the on-board sensor is adjusted according to the on-board data information to obtain the reference pose information of the on-board sensor. Thus, the reference pose information of the on-board sensor is determined based on the virtual calibration environment and the on-board data information output by the on-board sensor model, and the pose information of the on-board sensor can be effectively calibrated in the virtual calibration environment.

[0066] In order to clearly illustrate how the initial pose information of the on-board sensor is determined based on the pose information of the on-board sensor model on the target vehicle model in the virtual calibration environment in the above embodiment, the present disclosure proposes another calibration method.

[0067] Figure 5 is a schematic diagram according to a fourth embodiment of the present disclosure.

[0068] like Figure 5 As shown, the calibration method may include the following steps:

[0069] Step 501 : Simulate the calibration environment of the target vehicle and the onboard sensors of the target vehicle respectively to obtain a virtual calibration environment and onboard sensor model corresponding to the target vehicle.

[0070] Step 502: Load the vehicle-mounted sensor model onto the target vehicle model in the virtual calibration environment, and obtain the position and posture information of the vehicle-mounted sensor model.

[0071] In an embodiment of the present disclosure, relevant code can be executed to load the on-board sensor model onto the target vehicle model in the virtual calibration environment, and then, the position information of the on-board sensor model can be determined based on the on-board data information output by the on-board sensor model.

[0072] Step 503: Using the pose information of the vehicle-mounted sensor model as the initial pose information of the vehicle-mounted sensor.

[0073] Furthermore, the pose information of the vehicle-mounted sensor model can be used as the initial pose information of the vehicle-mounted sensor.

[0074] Step 504 : adjusting the initial position and posture information of the vehicle-mounted sensor according to the vehicle-mounted data information to obtain reference position and posture information of the vehicle-mounted sensor.

[0075] Step 505: Determine the virtual calibration scene corresponding to the target vehicle based on the reference posture information.

[0076] Step 506 : Fusing the vehicle data information output by the vehicle sensor model under the reference pose information and the virtual calibration scene to obtain a target fused image.

[0077] Step 507 : Calibrate the reference pose information based on the target fusion image to determine the target pose information of the vehicle-mounted sensor.

[0078] It should be noted that the execution process of step 501 and steps 504 to 507 can be implemented in any way in the embodiments of the present disclosure, and the embodiments of the present disclosure do not limit this and will not be described in detail.

[0079] In summary, the on-board sensor model is loaded onto the target vehicle model in the virtual calibration environment to obtain the pose information of the on-board sensor model; the pose information of the on-board sensor model on the target vehicle model is used as the initial pose information of the on-board sensor. Thus, the initial pose information of the on-board sensor can be effectively determined based on the pose information of the on-board sensor model.

[0080] In order to clearly illustrate how the above embodiment simulates the calibration environment of the target vehicle and the on-board sensors of the target vehicle respectively to obtain the virtual calibration environment and on-board sensor model corresponding to the target vehicle, the present disclosure proposes another calibration method.

[0081] Figure 6 is a schematic diagram according to a fifth embodiment of the present disclosure.

[0082] like Figure 6 As shown, the calibration method may include the following steps:

[0083] Step 601 : Simulate the target vehicle, the calibration equipment corresponding to the target vehicle, and the calibration site where the target vehicle is located, respectively, to obtain a target vehicle model, a calibration equipment model, and a calibration environment model corresponding to the target vehicle.

[0084] In order to quickly and cost-effectively build various calibration scenarios, in the disclosed embodiments, any type of target vehicle can be simulated separately to obtain a target vehicle model corresponding to the target vehicle. At the same time, calibration equipment of any size can also be simulated to obtain a calibration equipment model. In addition, the calibration site where the target vehicle is located can also be simulated to obtain a calibration environment model.

[0085] Step 602 : Load the target vehicle model, calibration equipment model, and calibration site model to generate a virtual calibration environment corresponding to the target vehicle.

[0086] Furthermore, by executing relevant codes, the target vehicle model, the calibration equipment model and the calibration site model can be loaded, thereby generating a virtual calibration environment corresponding to the target vehicle.

[0087] Step 603: Simulate the onboard sensor of the target vehicle to obtain an onboard sensor model.

[0088] In addition, in order to realize the calibration of the posture information of the on-board sensors in the virtual calibration environment, the on-board sensors of the target vehicle can also be simulated to obtain the on-board sensor model. Thus, the on-board sensor model can be set at the specified position of the target vehicle model to obtain the on-board data information output by the on-board sensor model.

[0089] Step 604 : Determine reference pose information of the vehicle-mounted sensor and a virtual calibration scene corresponding to the target vehicle under the reference pose information based on the virtual calibration environment and the vehicle-mounted data information output by the vehicle-mounted sensor model.

[0090] Step 605 , fusing the vehicle data information output by the vehicle sensor model under the reference pose information and the virtual calibration scene to obtain a target fused image.

[0091] Step 606 , calibrating the reference pose information based on the target fusion image to determine the target pose information of the vehicle-mounted sensor.

[0092] It should be noted that the execution process of steps 604 to 606 can be implemented in any of the embodiments of the present disclosure, and the embodiments of the present disclosure do not limit this and will not be described in detail.

[0093] In summary, by simulating the target vehicle, the calibration equipment corresponding to the target vehicle, and the calibration site where the target vehicle is located, the target vehicle model, calibration equipment model, and calibration environment model corresponding to the target vehicle are obtained; the target vehicle model, calibration equipment model, and calibration site model are loaded to generate a virtual calibration environment corresponding to the target vehicle; the on-board sensors of the target vehicle are simulated to obtain the on-board sensor model. Therefore, by simulating the calibration environment where the target vehicle is located and the on-board sensors of the target vehicle, various calibration scenarios can be built quickly and at low cost.

[0094] In order to clearly illustrate how the above embodiment calibrates the reference pose information of the vehicle-mounted sensor based on the target fusion image to determine the target pose information of the vehicle-mounted sensor, the present disclosure proposes another calibration method.

[0095] Figure 7 is a schematic diagram according to a sixth embodiment of the present disclosure.

[0096] like Figure 7 As shown, the calibration method may include the following steps:

[0097] Step 701 : Simulate the calibration environment of the target vehicle and the onboard sensors of the target vehicle respectively to obtain a virtual calibration environment and onboard sensor model corresponding to the target vehicle.

[0098] Step 702 : Determine reference pose information of the vehicle-mounted sensor and a virtual calibration scene corresponding to the target vehicle under the reference pose information based on the virtual calibration environment and the vehicle-mounted data information output by the vehicle-mounted sensor model.

[0099] Step 703 , fusing the vehicle-mounted data information corresponding to the target vehicle under the reference posture information and the virtual calibration scene to obtain a target fused image.

[0100] Step 704 : Detect the target object in the target fused image to obtain position information of the target object.

[0101] In the disclosed embodiments, a target detection algorithm or target detection model may be used to detect the target object in the target fusion image to obtain the target object's location information. It should be noted that the target detection model may be a trained neural network model that has learned the correspondence between the target fusion image and the target object's location information.

[0102] Step 705 : adjusting the reference pose information of the vehicle-mounted sensor according to the difference between the position information of the target object and the set position information of the target object to obtain the target pose information of the vehicle-mounted sensor.

[0103] Furthermore, when there is a difference between the position information of the target object and the set position information of the target object, the reference posture information of the vehicle-mounted sensor can be adjusted, and the target fusion image can be re-determined based on the adjusted reference posture information until the difference between the position information of the target object in the target fusion image and the set position information of the target object is minimized, and the posture information of the vehicle-mounted sensor at this time is used as the target posture information.

[0104] In addition, in order to facilitate analysis by relevant personnel, the target posture information of the vehicle-mounted sensor can be obtained and displayed, and in response to the second user operation, the target posture information of the vehicle-mounted sensor can be stored.

[0105] It should be noted that the execution process of steps 701 to 703 can be implemented in any of the embodiments of the present disclosure, and the embodiments of the present disclosure do not limit this and will not be described in detail.

[0106] In summary, the target object in the target fusion image is detected to obtain the position information of the target object; according to the difference between the position information of the target object and the set position information of the target object, the reference pose information of the on-board sensor is adjusted to obtain the target pose information of the on-board sensor. Thus, the reference pose information is calibrated according to the target fusion image, which can further improve the accuracy of the pose information of the on-board sensor.

[0107] In order to clearly illustrate the above embodiment, an example is now given for illustration.

[0108] For example, if Figure 8 As shown, Figure 8 A flowchart of a calibration method provided in an embodiment of the present disclosure.

[0109] exist Figure 8 In the calibration method, the following steps may be included:

[0110] 1. Scene model loading: Load the target vehicle model, calibration site model and calibration environment model to generate a virtual calibration environment corresponding to the target vehicle;

[0111] 2. Vehicle sensor simulation: simulate the target vehicle's onboard sensors to obtain a vehicle sensor model, load the onboard sensor model into the specified location of the target vehicle model, and obtain the onboard data information output by the onboard sensor model;

[0112] 3. Scene rendering: Rendering the virtual calibration environment into the VR headset;

[0113] 4. VR control: The operator can actually see the posture information between the target vehicle and the calibration device. The operator can compare the posture information between the target vehicle and the calibration device with the set posture information between the target vehicle and the calibration device to adjust the posture information of the on-board sensor and obtain the reference posture information of the on-board sensor to minimize the difference between the posture information between the target vehicle and the calibration device and the corresponding set posture information. Then, the on-board data information output by the on-board sensor model under the reference posture information is fused with the corresponding virtual calibration scene to obtain a target fused image. According to the position information of the target object in the target fused image, the reference posture information is further calibrated to obtain the target posture information of the on-board sensor.

[0114] 5. Save the target posture information of the vehicle-mounted sensor.

[0115] The calibration method of the embodiment of the present disclosure simulates the calibration environment of the target vehicle and the on-board sensor of the target vehicle respectively to obtain a virtual calibration environment and an on-board sensor model corresponding to the target vehicle; determines the reference posture information of the on-board sensor and the virtual calibration scene corresponding to the target vehicle under the reference posture information according to the virtual calibration environment and the on-board data information output by the on-board sensor model; fuses the on-board data information output by the on-board sensor model under the reference posture information and the virtual calibration scene to obtain a target fused image; calibrates the reference posture information according to the target fused image to determine the on-board sensor The target pose information of the vehicle is obtained by simulating the calibration environment of the target vehicle and the on-board sensors of the target vehicle respectively, so that various calibration scenarios can be built quickly and at low cost. Then, the reference pose information of the on-board sensor is determined according to the virtual calibration environment and the on-board data information output by the on-board sensor model, and the pose information of the on-board sensor can be calibrated efficiently in the virtual calibration environment. In addition, the on-board data information and the virtual calibration scene are fused to obtain a target fused image. According to the target fused image, the reference pose information is calibrated, which further improves the accuracy of the pose information of the on-board sensor.

[0116] In order to implement the above embodiments, the present disclosure proposes a calibration device.

[0117] Figure 9 is a schematic diagram according to a seventh embodiment of the present disclosure.

[0118] like Figure 9 As shown, the calibration device 900 includes: a simulation module 910 , a determination module 920 , a fusion module 930 and a calibration module 940 .

[0119] Among them, the simulation module 910 is used to simulate the calibration environment in which the target vehicle is located and the on-board sensors of the target vehicle respectively to obtain the virtual calibration environment and on-board sensor model corresponding to the target vehicle; the determination module 920 is used to determine the reference pose information of the on-board sensor and the virtual calibration scene corresponding to the target vehicle under the reference pose information based on the virtual calibration environment and the on-board data information output by the on-board sensor model; the fusion module 930 is used to fuse the on-board data information output by the on-board sensor model under the reference pose information and the virtual calibration scene to obtain a target fused image; the calibration module 940 is used to calibrate the reference pose information according to the target fused image to determine the target pose information of the on-board sensor.

[0120] As a possible implementation method of the embodiment of the present disclosure, the determination module 920 is used to: calibrate the posture information of the on-board sensor according to the virtual calibration environment and the on-board data information output by the on-board sensor model to obtain reference posture information; and determine the virtual calibration scene corresponding to the target vehicle according to the reference posture information.

[0121] As a possible implementation method of an embodiment of the present disclosure, the determination module 920 is also used to: determine the initial pose information of the vehicle-mounted sensor based on the pose information of the vehicle-mounted sensor model on the target vehicle model in the virtual calibration environment; adjust the initial pose information of the vehicle-mounted sensor based on the vehicle-mounted data information to obtain the reference pose information of the vehicle-mounted sensor.

[0122] As a possible implementation method of the embodiment of the present disclosure, the determination module 920 is also used to: determine the relative posture information between the target vehicle and the corresponding calibration device based on the on-board data information; adjust the initial posture information of the on-board sensor based on the set posture information between the target vehicle and the corresponding calibration device and the difference between the relative posture information to obtain the reference posture information of the on-board sensor.

[0123] As a possible implementation of the embodiment of the present disclosure, the calibration device 900 further includes: a rendering module and an adjustment module.

[0124] Among them, the rendering module is used to perform three-dimensional rendering of the virtual calibration environment to display the relative posture information between the target vehicle and the corresponding calibration device in the simulation data information; the determination module 920 is also used to determine the reference posture information of the on-board sensor in response to the first user operation; wherein, the reference posture information is generated based on the set posture information between the target vehicle and the corresponding calibration device and the difference between the relative posture information; the adjustment module is used to adjust the initial posture information of the on-board sensor according to the reference posture information.

[0125] As a possible implementation of the embodiment of the present disclosure, the determination module 920 is further configured to: load the vehicle sensor model onto the target vehicle model in the virtual calibration environment;

[0126] And obtain the pose information of the on-board sensor model on the target vehicle model; use the pose information of the on-board sensor model on the target vehicle model as the initial pose information of the on-board sensor.

[0127] As a possible implementation method of the embodiment of the present disclosure, the simulation module 910 is used to: simulate the target vehicle, the calibration equipment corresponding to the target vehicle, and the calibration site where the target vehicle is located, respectively, to obtain the target vehicle model, calibration equipment model, and calibration environment model corresponding to the target vehicle; load the target vehicle model, calibration equipment model, and calibration site model where the target vehicle is located to generate a virtual calibration environment corresponding to the target vehicle; simulate the on-board sensors of the target vehicle to obtain the on-board sensor model.

[0128] As a possible implementation method of an embodiment of the present disclosure, the calibration module 940 is used to: detect the target object in the target fusion image to obtain the position information of the target object; adjust the reference pose information of the on-board sensor according to the difference between the position information of the target object and the set position information of the target object to obtain the target pose information of the on-board sensor.

[0129] As a possible implementation of the embodiment of the present disclosure, the calibration device 900 further includes: a processing module and a storage module.

[0130] Among them, the processing module is used to obtain and display the target posture information of the vehicle-mounted sensor; the storage module is used to store the target posture information of the vehicle-mounted sensor in response to the second user operation.

[0131] The calibration device of the embodiment of the present disclosure simulates the calibration environment of the target vehicle and the on-board sensor of the target vehicle respectively to obtain a virtual calibration environment and an on-board sensor model corresponding to the target vehicle; determines the reference posture information of the on-board sensor and the virtual calibration scene corresponding to the target vehicle under the reference posture information according to the virtual calibration environment and the on-board data information output by the on-board sensor model; fuses the on-board data information output by the on-board sensor model under the reference posture information and the virtual calibration scene to obtain a target fused image; calibrates the reference posture information according to the target fused image to determine the on-board sensor The target pose information of the vehicle is obtained by simulating the calibration environment of the target vehicle and the on-board sensors of the target vehicle respectively, so that various calibration scenarios can be built quickly and at low cost. Then, the reference pose information of the on-board sensor is determined according to the virtual calibration environment and the on-board data information output by the on-board sensor model, and the pose information of the on-board sensor can be calibrated efficiently in the virtual calibration environment. In addition, the on-board data information and the virtual calibration scene are fused to obtain a target fused image. According to the target fused image, the reference pose information is calibrated, which further improves the accuracy of the pose information of the on-board sensor.

[0132] It should be noted that in the technical solutions disclosed herein, the collection, storage, use, processing, transmission, provision and disclosure of user personal information are all carried out with the user's consent, and are in compliance with relevant laws and regulations and do not violate public order and good morals.

[0133] In order to implement the above embodiments, the present disclosure also proposes an electronic device, comprising: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions that can be executed by the at least one processor, and the instructions are executed by the at least one processor so that the at least one processor can execute the calibration method of the above embodiments.

[0134] In order to implement the above embodiments, the present disclosure further proposes a non-transitory computer-readable storage medium storing computer instructions, wherein the computer instructions are used to enable a computer to execute the calibration method described in the above embodiments.

[0135] In order to implement the above embodiments, the present disclosure further proposes a computer program product, including a computer program, which implements the calibration method described in the above embodiments when executed by a processor.

[0136] According to an embodiment of the present application, the present application also provides an electronic device, a readable storage medium and a computer program product.

[0137] Figure 10 A schematic block diagram of an example electronic device 1000 that can be used to implement embodiments of the present disclosure is shown. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device can also represent various forms of mobile devices, such as personal digital assistants, cellular phones, smartphones, wearable devices, and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely examples and are not intended to limit the implementation of the present disclosure described and / or claimed herein.

[0138] like Figure 10 As shown, the device 1000 includes a computing unit 1001, which can perform various appropriate actions and processes according to a computer program stored in a read-only memory (ROM) 1002 or a computer program loaded from a storage unit 1008 into a random access memory (RAM) 1003. Various programs and data required for the operation of the device 1000 can also be stored in the RAM 1003. The computing unit 1001, the ROM 1002, and the RAM 1003 are connected to each other via a bus 1004. An input / output (I / O) interface 1005 is also connected to the bus 1004.

[0139] Various components in device 1000 are connected to I / O interface 1005, including an input unit 1006, such as a keyboard, mouse, etc.; an output unit 1007, such as various types of displays, speakers, etc.; a storage unit 1008, such as a magnetic disk, optical disk, etc.; and a communication unit 1009, such as a network card, modem, wireless communication transceiver, etc. The communication unit 1009 allows device 1000 to exchange information / data with other devices via a computer network such as the Internet and / or various telecommunication networks.

[0140] The computing unit 1001 can be a variety of general and / or special processing components with processing and computing capabilities. Some examples of the computing unit 1001 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various dedicated artificial intelligence (AI) computing chips, various computing units that run machine learning model algorithms, digital signal processors (DSPs), and any appropriate processors, controllers, microcontrollers, etc. The computing unit 1001 performs the various methods and processes described above, such as the calibration method. For example, in some embodiments, the calibration method can be implemented as a computer software program that is tangibly contained in a machine-readable medium, such as the storage unit 1008. In some embodiments, part or all of the computer program can be loaded and / or installed on the device 1000 via the ROM 1002 and / or the communication unit 1009. When the computer program is loaded into the RAM 1003 and executed by the computing unit 1001, one or more steps of the calibration method described above can be performed. Alternatively, in other embodiments, the computing unit 1001 can be configured to perform the calibration method by any other appropriate means (e.g., by means of firmware).

[0141] Various embodiments of the systems and techniques described herein can be implemented in digital electronic circuit systems, integrated circuit systems, field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), system-on-chip systems (SOCs), programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various embodiments can include being implemented in one or more computer programs that are executable and / or interpreted on a programmable system that includes at least one programmable processor, which can be a special purpose or general purpose programmable processor that can receive data and instructions from a storage system, at least one input device, and at least one output device, and transmit data and instructions to the storage system, the at least one input device, and the at least one output device.

[0142] The program code for implementing the method of the present disclosure can be written in any combination of one or more programming languages. These program codes can be provided to a processor or controller of a general-purpose computer, a special-purpose computer, or other programmable data processing device so that when the program code is executed by the processor or controller, the functions / operations specified in the flow chart and / or block diagram are implemented. The program code can be executed entirely on the machine, partially on the machine, as a stand-alone software package, partially on the machine and partially on a remote machine, or entirely on a remote machine or server.

[0143] In the context of the present disclosure, a machine-readable medium can be a tangible medium that can contain or store a program for use by or in conjunction with an instruction execution system, device or equipment. A machine-readable medium can be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium can include, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or equipment, or any suitable combination of the foregoing. A more specific example of a machine-readable storage medium can include an electrical connection based on one or more lines, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

[0144] To provide interaction with a user, the systems and techniques described herein can be implemented on a computer having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user; and a keyboard and pointing device (e.g., a mouse or trackball) through which the user can provide input to the computer. Other types of devices can also be used to provide interaction with the user; for example, the feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including acoustic input, voice input, or tactile input).

[0145] The systems and techniques described herein can be implemented in a computing system that includes back-end components (e.g., as a data server), or a computing system that includes middleware components (e.g., an application server), or a computing system that includes front-end components (e.g., a user computer with a graphical user interface or a web browser through which a user can interact with implementations of the systems and techniques described herein), or a computing system that includes any combination of such back-end components, middleware components, or front-end components. The components of the system can be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include: a local area network (LAN), a wide area network (WAN), the Internet, and a blockchain network.

[0146] A computer system may include a client and a server. The client and server are generally remote from each other and typically interact through a communication network. The client-server relationship arises through computer programs running on the respective computers and having a client-server relationship with each other. The server may also be a server in a distributed system or a server integrated with a blockchain.

[0147] It's important to note that artificial intelligence (AI) is the study of how computers can simulate certain human thought processes and intelligent behaviors (such as learning, reasoning, thinking, and planning). This encompasses both hardware and software technologies. AI hardware technologies generally include sensors, specialized AI chips, cloud computing, distributed storage, and big data processing. AI software technologies primarily encompass computer vision, speech recognition, natural language processing, machine learning / deep learning, big data processing, and knowledge graphs.

[0148] It should be understood that the various forms of the processes shown above can be used to reorder, add, or delete steps. For example, the steps described in this disclosure can be performed in parallel, sequentially, or in a different order, as long as the desired results of the technical solutions disclosed in this disclosure can be achieved. This is not a limitation herein.

[0149] The above specific embodiments do not constitute a limitation on the scope of protection of this disclosure. Those skilled in the art will appreciate that various modifications, combinations, sub-combinations, and substitutions may be made based on design requirements and other factors. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of this disclosure shall be included within the scope of protection of this disclosure.

Claims

1. A calibration method, comprising: Simulating the calibration environment of the target vehicle and the on-board sensors of the target vehicle respectively to obtain a virtual calibration environment and on-board sensor model corresponding to the target vehicle; Determining reference pose information of the on-board sensor and a virtual calibration scene corresponding to the target vehicle under the reference pose information based on the virtual calibration environment and the on-board data information output by the on-board sensor model; Fusing the vehicle data information output by the vehicle sensor model under the reference pose information and the virtual calibration scene to obtain a target fused image; The reference pose information is calibrated according to the target fusion image to determine the target pose information of the vehicle-mounted sensor.

2. The method according to claim 1, wherein The determining, based on the virtual calibration environment and the vehicle data information output by the vehicle sensor model, reference pose information of the vehicle sensor and a virtual calibration scene corresponding to the target vehicle under the reference pose information, includes: Calibrate the pose information of the on-board sensor according to the virtual calibration environment and the on-board data information output by the on-board sensor model to obtain reference pose information; Determine a virtual calibration scene corresponding to the target vehicle based on the reference pose information.

3. The method according to claim 2, wherein: The step of calibrating the pose information of the vehicle-mounted sensor according to the virtual calibration environment and the vehicle-mounted data information output by the vehicle-mounted sensor model to obtain reference pose information includes: Determining initial pose information of the on-board sensor according to pose information of the on-board sensor model on the target vehicle model in the virtual calibration environment; The initial position information of the vehicle-mounted sensor is adjusted according to the vehicle-mounted data information to obtain reference position information of the vehicle-mounted sensor.

4. The method according to claim 3, wherein: The adjusting the initial pose information of the vehicle-mounted sensor according to the vehicle-mounted data information to obtain reference pose information of the vehicle-mounted sensor includes: Determining relative position information between the target vehicle and the corresponding calibration device based on the vehicle-borne data information; According to the set posture information between the target vehicle and the corresponding calibration device and the difference between the relative posture information, the initial posture information of the on-board sensor is adjusted to obtain the reference posture information of the on-board sensor.

5. The method according to claim 4, wherein The method further comprises: Performing three-dimensional rendering on the virtual calibration environment to display relative pose information between the target vehicle and the corresponding calibration device in the simulation data information; In response to a first user operation, determining reference pose information of the vehicle-mounted sensor; wherein the reference pose information is generated based on the difference between the set pose information between the target vehicle and the corresponding calibration device and the relative pose information; The initial posture information of the vehicle-mounted sensor is adjusted according to the reference posture information.

6. The method according to claim 3, wherein: The determining, based on the pose information of the on-board sensor model on the target vehicle model in the virtual calibration environment, the initial pose information of the on-board sensor includes: Loading the vehicle sensor model on the target vehicle model in the virtual calibration environment, and obtaining the position and posture information of the vehicle-mounted sensor model; The position and posture information of the vehicle-mounted sensor model is used as the initial position and posture information of the vehicle-mounted sensor.

7. The method according to claim 1, wherein The calibration environment of the target vehicle and the on-board sensor of the target vehicle are simulated respectively to obtain a virtual calibration environment and an on-board sensor model corresponding to the target vehicle, including: Simulating the target vehicle, the calibration equipment corresponding to the target vehicle, and the calibration site where the target vehicle is located, respectively, to obtain a target vehicle model, a calibration equipment model, and a calibration environment model corresponding to the target vehicle; Loading the target vehicle model, the calibration equipment model, and the calibration site model where the target vehicle is located to generate a virtual calibration environment corresponding to the target vehicle; The onboard sensor of the target vehicle is simulated to obtain the onboard sensor model.

8. The method according to claim 1, wherein The step of calibrating the reference pose information according to the target fusion image to determine the target pose information of the vehicle-mounted sensor includes: Detecting a target object in the target fused image to obtain position information of the target object; The reference position information is adjusted according to a difference between the position information of the target object and the set position information of the target object to obtain the target position information of the vehicle-mounted sensor.

9. The method according to any one of claims 1 to 8, wherein The method further comprises: Acquiring and displaying target pose information of the vehicle-mounted sensor; In response to a second user operation, the target posture information of the vehicle-mounted sensor is stored.

10. A calibration device comprising: A simulation module is used to simulate the calibration environment of the target vehicle and the on-board sensors of the target vehicle respectively to obtain a virtual calibration environment and on-board sensor model corresponding to the target vehicle; A determination module is used to determine the reference pose information of the on-board sensor and the virtual calibration scene corresponding to the target vehicle under the reference pose information based on the virtual calibration environment and the on-board data information output by the on-board sensor model; a fusion module, configured to fuse the vehicle data information output by the vehicle sensor model under the reference pose information and the virtual calibration scene to obtain a target fused image; A calibration module is used to calibrate the reference pose information according to the target fusion image to determine the target pose information of the vehicle-mounted sensor.

11. The device according to claim 10, wherein The determining module is configured to: Calibrate the pose information of the on-board sensor according to the virtual calibration environment and the on-board data information output by the on-board sensor model to obtain reference pose information; Determine a virtual calibration scene corresponding to the target vehicle based on the reference pose information.

12. The device according to claim 11, wherein The determining module is further configured to: Determining initial pose information of the on-board sensor according to pose information of the on-board sensor model on the target vehicle model in the virtual calibration environment; The initial position information of the vehicle-mounted sensor is adjusted according to the vehicle-mounted data information to obtain reference position information of the vehicle-mounted sensor.

13. The device according to claim 12, wherein The determining module is further configured to: Determining relative position information between the target vehicle and the corresponding calibration device based on the vehicle-borne data information; According to the set posture information between the target vehicle and the corresponding calibration device and the difference between the relative posture information, the initial posture information of the on-board sensor is adjusted to obtain the reference posture information of the on-board sensor.

14. The device according to claim 13, wherein The device further comprises: a rendering module, configured to perform three-dimensional rendering of the virtual calibration environment to display relative pose information between the target vehicle and the corresponding calibration device in the simulation data information; The determination module is further configured to determine reference pose information of the vehicle-mounted sensor in response to a first user operation; wherein the reference pose information is generated based on the difference between the set pose information between the target vehicle and the corresponding calibration device and the relative pose information; An adjustment module is used to adjust the initial posture information of the vehicle-mounted sensor according to the reference posture information.

15. The device according to claim 12, wherein The determining module is further configured to: Loading the vehicle sensor model on the target vehicle model in the virtual calibration environment, and obtaining the position and posture information of the vehicle-mounted sensor model; The position and posture information of the on-board sensor model on the target vehicle model is used as the initial position and posture information of the on-board sensor.

16. The device according to claim 10, wherein The simulation module is used to: Simulating the target vehicle, the calibration equipment corresponding to the target vehicle, and the calibration site where the target vehicle is located, respectively, to obtain a target vehicle model, a calibration equipment model, and a calibration environment model corresponding to the target vehicle; Loading the target vehicle model, the calibration equipment model, and the calibration site model where the target vehicle is located to generate a virtual calibration environment corresponding to the target vehicle; The onboard sensor of the target vehicle is simulated to obtain the onboard sensor model.

17. The device according to claim 10, wherein The calibration module is used to: Detecting a target object in the target fused image to obtain position information of the target object; According to the difference between the position information of the target object and the set position information of the target object, the reference position information of the vehicle-mounted sensor is adjusted to obtain the target position information of the vehicle-mounted sensor.

18. The device according to any one of claims 10 to 17, wherein: The device further comprises: A processing module, configured to obtain and display target pose information of the vehicle-mounted sensor; The storage module is used to store the target posture information of the vehicle-mounted sensor in response to a second user operation.

19. An electronic device comprising: at least one processor; as well as a memory communicatively connected to the at least one processor; wherein, The memory stores instructions that can be executed by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to perform the method according to any one of claims 1 to 9.

20. A non-transitory computer-readable storage medium storing computer instructions, wherein: The computer instructions are used to cause the computer to execute the method according to any one of claims 1 to 9.

21. A computer program product comprising a computer program, which, when executed by a processor, implements the method according to any one of claims 1 to 9.

Images