Calibration method and device for joint application sensor parameters

Abstract

The present application discloses a calibration method and device for joint application sensor parameters, wherein the method includes: in the process of joint application sensor movement, determining the static environment object included in the natural environment as the target environment object; measuring the target environment object by the first sensor in multiple positions and postures, and determining the first position information of the target environment object; determining the target optimization parameter and each group of values ​​of the target optimization parameter, converting the relative position information of the first position information, and obtaining the second position information corresponding to each group of target optimization parameter values ​​in a unified coordinate system; based on the position data of the target environment object in each position in each second position information, determining the target optimization parameter optimization value, and determining the optimization value as the calibration parameter of the first sensor and the second sensor. The method can perform online and real-time parameter calibration for the joint application sensor based on the static objects in the environment.

Description

Technical Field

[0001] The present application relates to the field of sensor application technology, and in particular to a method and device for calibrating parameters of a combined application sensor. Background Art

[0002] The research and application of intelligent automation systems are gradually emerging. In some fields, many achievements that can be put into application have emerged, such as self-driving cars, robots and other technologies. The movement process of these automation systems often needs to be based on effective environmental perception. For example, self-driving cars need to effectively identify objects in the environment, such as roads, traffic signs and signals, and other traffic participants during driving, and accurately determine the relative relationship between their own state and the environment to ensure the effective implementation of autonomous driving and driving safety. The environmental data obtained by the sensors and other equipment components carried by the system is the basis of environmental perception. By processing the obtained sensor data, various objects in the environment can be detected and identified in real time, and the relative relationship between environmental objects and themselves can be determined, thereby making correct decisions.

[0003] Independently applied sensors and other components are easily affected and restricted by application conditions, thus affecting the reliability of the application process. For example, the camera may be blinded in strong light and unable to obtain effective image data, the lidar may fail in foggy or dusty weather, and the GPS component may be inaccurate or unable to locate in special terrains, such as tunnels and under bridges. When sensors and other components encounter poor application conditions or fail themselves, the system will not be able to obtain the corresponding type of data and cause the system to fail. The integration of different types of sensors has become one of the effective means to improve the reliability and robustness of the system.

[0004] In multi-sensor fusion applications, an important step includes calibrating the external parameters of each jointly applied sensor to determine the coordinate position relationship between each jointly applied sensor component, so that each sensor can work effectively together, especially for sensors related to object position and related applications of object measurement, accurate parameter calibration between sensors is the basis for environmental object perception. In the prior art, the parameter calibration of jointly applied sensors usually needs to be carried out by professional equipment or venues, and the flexibility of the calibration method is poor, which is not conducive to timely proof of calibration parameter errors. Therefore, the technical problem that needs to be solved by those skilled in the art is to provide a more flexible and effective calibration method for fusion sensor parameters. Summary of the invention

[0005] The embodiments of the present application provide a method and device for calibrating parameters of a combined application sensor, which can realize online, real-time parameter calibration of the combined application sensor based on static markers in the natural environment.

[0006] This application provides the following solutions:

[0007] A method for calibrating joint application sensor parameters, comprising:

[0008] In the process of moving the combined application sensor, a static environment object included in the natural environment is determined as a target environment object; the combined application sensor includes a first sensor and a second sensor;

[0009] Measuring the target environment object by using the first sensor in a plurality of postures to determine first position information of the target environment object relative to the first sensor;

[0010] Determine the target optimization parameters and each group of values ​​of the target optimization parameters, convert the relative position information of the first position information, and obtain the second position information corresponding to each group of target optimization parameter values ​​in a unified coordinate system;

[0011] Based on the position data of the target environment object at each posture in each of the second position information, an optimization value of the target optimization parameter is determined, and the optimization value is determined as the calibration parameter of the first sensor and the second sensor.

[0012] A calibration method for jointly applying sensor parameters, comprising:

[0013] Receiving an instruction from an external device to calibrate parameters of a sensor used in a carrier;

[0014] Determine target optimization parameters and each group of values ​​of the target optimization parameters;

[0015] Determine a target environmental object, and during the movement of the carrier, measure the target environmental object by using the combined application sensor based on each set of values ​​of the target optimization parameter to obtain measurement data;

[0016] According to the measured data, an optimized value of a target optimized parameter is determined, and the optimized value is determined as a calibration parameter of the combined application sensor.

[0017] A method for verifying calibration parameters of a combined application sensor, comprising:

[0018] The external device sends a command to the carrier system to calibrate the parameters of the combined application sensor in the carrier;

[0019] After receiving the instruction, the carrier system determines the target optimization parameters and each group of values ​​of the target optimization parameters; determines the target environmental object, and during the movement of the carrier, based on the each group of values ​​of the target optimization parameters, measures the target environmental object through the combined application sensor to obtain measurement data; determines the optimized value of the target optimization parameter based on the measurement data, and uses the optimized value to calibrate the parameters of the combined application sensor.

[0020] A calibration device for jointly applying sensor calibration parameters, comprising:

[0021] A static object determination unit, used to determine static environmental objects included in the natural environment as target environmental objects during the movement of the combined application sensor; the combined application sensor includes a first sensor and a second sensor;

[0022] A position measuring unit, configured to measure the target environment object by using the first sensor in a plurality of positions and postures, and determine first position information of the target environment object relative to the first sensor;

[0023] A position conversion unit, used to determine the target optimization parameters and each group of target optimization parameter values, convert the relative position information of the first position information, and obtain the second position information corresponding to each group of target optimization parameter values ​​in a unified coordinate system;

[0024] A parameter determination unit is used to determine the optimization value of the target optimization parameter based on the position data of the target environment object in each posture in each second position information, and determine the optimization value as the calibration parameter of the first sensor and the second sensor.

[0025] A calibration device for jointly applying sensor calibration parameters, comprising:

[0026] A calibration instruction receiving unit, used to receive an instruction from an external device to calibrate the parameters of the combined application sensor in the carrier;

[0027] A parameter value determination unit, used to determine target optimization parameters and each group of values ​​of the target optimization parameters;

[0028] A measurement data determination unit, used for determining a target environment object, and obtaining measurement data by measuring the target environment object through the combined application sensor based on each set of values ​​of the target optimization parameter during the movement of the carrier;

[0029] The optimization parameter determination unit is used to determine the optimization value of the target optimization parameter according to the measurement data, and determine the optimization value as the calibration parameter of the joint application sensor.

[0030] A calibration device for jointly applying sensor calibration parameters, comprising:

[0031] A calibration instruction issuing unit, used for the external device to issue an instruction to the carrier system to calibrate the parameters of the combined application sensor in the carrier;

[0032] The calibration processing unit is used to determine the target optimization parameters and each group of values ​​of the target optimization parameters after the carrier system receives the instruction; determine the target environment object, and during the movement of the carrier, based on the each group of values ​​of the target optimization parameters, measure the target environment object through the combined application sensor to obtain measurement data; determine the optimized value of the target optimization parameter based on the measurement data, and use the optimized value to calibrate the parameters of the combined application sensor.

[0033] An electronic device, comprising:

[0034] one or more processors; and

[0035] A memory associated with the one or more processors, the memory being used to store program instructions, wherein when the program instructions are read and executed by the one or more processors, the following operations are performed:

[0036] In the process of moving the combined application sensor, a static environment object included in the natural environment is determined as a target environment object; the combined application sensor includes a first sensor and a second sensor;

[0037] Measuring the target environment object by using the first sensor in a plurality of postures to determine first position information of the target environment object relative to the first sensor;

[0038] Determine the target optimization parameters and each group of values ​​of the target optimization parameters, convert the relative position information of the first position information, and obtain the second position information corresponding to each group of target optimization parameter values ​​in a unified coordinate system;

[0039] Based on the position data of the target environment object at each posture in each of the second position information, an optimization value of the target optimization parameter is determined, and the optimization value is determined as the calibration parameter of the first sensor and the second sensor.

[0040] An electronic device, comprising:

[0041] one or more processors; and

[0042] A memory associated with the one or more processors, the memory being used to store program instructions, wherein when the program instructions are read and executed by the one or more processors, the following operations are performed:

[0043] Receiving an instruction from an external device to calibrate parameters of a sensor used in a carrier;

[0044] Determine target optimization parameters and each group of values ​​of the target optimization parameters;

[0045] Determine a target environmental object, and during the movement of the carrier, measure the target environmental object by using the combined application sensor based on each set of values ​​of the target optimization parameter to obtain measurement data;

[0046] According to the measured data, an optimized value of a target optimized parameter is determined, and the optimized value is determined as a calibration parameter of the combined application sensor.

[0047] An electronic device, comprising:

[0048] one or more processors; and

[0049] A memory associated with the one or more processors, the memory being used to store program instructions, wherein when the program instructions are read and executed by the one or more processors, the following operations are performed:

[0050] The external device sends a command to the carrier system to calibrate the parameters of the combined application sensor in the carrier;

[0051] After receiving the instruction, the carrier system determines the target optimization parameters and each group of values ​​of the target optimization parameters; determines the target environmental object, and during the movement of the carrier, based on the each group of values ​​of the target optimization parameters, measures the target environmental object through the combined application sensor to obtain measurement data; determines the optimized value of the target optimization parameter based on the measurement data, and uses the optimized value to calibrate the parameters of the combined application sensor.

[0052] According to the specific embodiments provided in this application, this application discloses the following technical effects:

[0053] Through this application, during the movement of the combined application sensor, the static environment objects included in the natural environment can be determined as the target environment objects; the combined application sensor includes a first sensor and a second sensor; the target environment object is measured by the first sensor in multiple positions and postures, and the first position information of the target environment object relative to the first sensor is determined; the target optimization parameters and each group of values ​​of the target optimization parameters are determined, and the relative position information of the first position information is converted to obtain the second position information corresponding to each group of target optimization parameter values ​​in a unified coordinate system; based on the position data of the target environment object in each position in each second position information, the optimized value of the target optimization parameter is determined, and the optimized value is determined as the calibration parameter of the first sensor and the second sensor. Through this method, online and real-time parameter calibration of the combined application sensor can be realized based on static markers in the natural environment. When the system is in operation, real-time error correction can be performed on the time and space parameter-related working conditions of the jointly applied sensors without requiring offline maintenance of the equipment, achieving high efficiency and greatly reducing costs compared to existing offline methods. Time delay parameters that cannot be accurately calibrated in existing technologies can be calibrated, effectively suppressing the actual impact of delays between sensors on sensing results, and improving the safety of the operation of the automation system.

[0054] Of course, any product implementing the present application does not necessarily need to achieve all of the advantages described above at the same time. BRIEF DESCRIPTION OF THE DRAWINGS

[0055] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings required for use in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0056] Figure 1 is a schematic diagram of a multi-sensor fusion application provided in an embodiment of the present application;

[0057] Figure 2 is a flow chart of the first method provided in an embodiment of the present application;

[0058] Figure 3 is a flow chart of the second method provided in an embodiment of the present application;

[0059] Figure 4 is a flowchart of the third method provided in an embodiment of the present application;

[0060] Figure 5 is a schematic diagram of a first device provided in an embodiment of the present application;

[0061] Figure 6is a schematic diagram of a second device provided in an embodiment of the present application;

[0062] Figure 7 is a schematic diagram of a third device provided in an embodiment of the present application;

[0063] Figure 8 It is a schematic diagram of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0064] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. All other embodiments obtained by ordinary technicians in this field based on the embodiments in the present application belong to the scope of protection of this application.

[0065] In intelligent systems such as self-driving cars, many different types of sensors are usually used to obtain different types of environmental data and complete different environmental perception tasks based on the obtained data, providing diverse options for the organization of the system. For example, image acquisition devices can obtain images of the surrounding environment, and based on the analysis of image data, environmental objects and object types can be determined to achieve functions such as traffic signal recognition and lane keeping; radar equipment can emit sound waves or electromagnetic waves to illuminate the target object and receive its echo to obtain rich information about the target object, such as the distance to the target object, the rate of change of distance, size, shape and other information.

[0066] In systems that use different types of sensors, the reliability of sensors and their data is one of the key factors to consider. Independently applied sensor components are easily affected and restricted by application conditions, which affects the reliability of the application process. When an autonomous vehicle suddenly loses certain sensor data, the backup plan is often insufficient or the implementation cost is very high. Compared with independently applied sensors, sensor fusion applications can make up for their many shortcomings. The fusion application of different types of sensors at different application levels can bring different technical effects, achieve better fault tolerance, complementarity, real-time performance, economy, etc., and produce rich application effects in different application directions. For example, some sensor combinations can achieve a similar function during the period when one of the sensors temporarily fails, thereby ensuring that the system can continue to operate during this period; another effect is that when a certain sensor fails, a temporary alternative can be provided to provide buffer time, providing more sufficient buffer time for the system to handle abnormal situations. For example, in an autonomous driving system, image sensors and IMU (Inertial Measurement Unit) sensors can be fused and applied. When the image sensor fails due to strong light or other conditions, the lane keeping function can still be implemented for a certain period of time based on the last image data and the relative motion information provided by the IMU. This time period can be used to prompt personnel to intervene in the operation. Compared with the sudden failure of the image sensor when it is used independently, the fusion application of the two provides a buffer time to improve safety. In addition, some sensor combinations can provide functional reliability that was previously only provided by high-cost components in a very economical way, etc.

[0067] In multi-sensor fusion applications, each sensor can be the same or different types of sensors. The sensors used in combination usually need to be calibrated, including the calibration of the external parameters of each sensor to determine the coordinate position relationship between the sensor components. Calibration parameters are the basis for the normal operation of each sensor. Better calibration parameters can effectively improve the accuracy of environmental perception and achieve better system effectiveness. Figure 1As shown in FIG. 1 , it is a schematic diagram of a multi-sensor fusion application, wherein the sensor assembly for joint application includes sensor 110 and sensor 120, which are two different types of sensors for joint application. The connection shown in the figure reflects the positional relationship between the sensors. In practical applications, the sensors for joint application may not be directly connected through components, but may be installed at different parts of the carrier, such as the center of the chassis and the roof of the autonomous driving vehicle. As shown in the figure, the two sensors use the corresponding coordinate system 111 and coordinate system 121 respectively, which usually do not overlap, that is, the coordinate systems of the two sensors are not unified. Due to reasons such as manufacturing and installation, it is a common situation in practical applications that the sensor coordinates are not unified and therefore parameter calibration is required. The determination of the relative coordinate position relationship of the joint application sensors can be carried out through calibration operations. After parameter calibration, the determined coordinate position relationship of the sensor is usually reflected as a set of parameters for coordinate conversion, such as parameters such as relative rotation angle and relative translation. When performing data fusion processing of the two sensors, the environmental data can be converted into a coordinate system based on the calibration parameters. It can be seen that accurate sensor calibration parameters are the basis for the joint application sensor to provide effective data.

[0068] In the prior art, the calibration of the parameters of the combined application sensors usually needs to be carried out by professional equipment and venues. The implementation basis includes providing professional media or tools, such as calibration workshops containing laser radars, high-precision maps, etc., and checking the sensor parameters or recalibrating based on calibration objects or places containing standardized coordinate information. The high degree of specialization of the detection and calibration sites / tools, etc., has resulted in the inability of the existing methods to be converted into terminal applications, and the failure detection and correction of the calibration parameters need to be carried out offline. This means that when the system is in operation, such as during the driving of an autonomous vehicle, it is impossible to perform real-time online detection and error correction on the parameter conditions of the combined application sensors. The failure of the calibration parameters may cause sensor data errors and abnormal environmental perception functions, thereby reducing the automation level of the system and even bringing safety hazards. In particular, for the delay in data processing between different sensors, the synchronization methods used in the prior art cannot be accurately described by external parameter calibration, and cannot effectively suppress the actual impact of the delay between sensors on the sensing results.

[0069] The embodiment of the present application provides a calibration method for sensor parameters based on natural scene objects in a moving state. The sensors used in the method may include a first sensor and a second sensor. The first sensor is used to detect the target environment object. Specifically, the first sensor can be used to generate sensing data about the target environment object, and the speed of the target environment object, the position description information, etc. can be determined according to the sensing data of the first sensor. The typical first sensor is a millimeter wave radar; the second sensor is mainly used to provide a coordinate system conversion reference value. Specifically, when it is necessary to unify the data obtained by the same sensor in different states or the data obtained by different sensors into a consistent coordinate system, the relative reference value of the coordinate system can be provided to facilitate the fusion of data. Under the existing technical conditions, most carrier systems are active on the surface, and the frequently changing postures of the carrier system during the movement can be unified into the earth coordinate system for description, so the choice of the unified coordinate system is mainly the earth coordinate system, or the navigation coordinate system. The second sensor can also provide information such as its own speed (in most cases, the speed is equivalent to the carrier system movement speed), and the typical second sensor is an IMU inertial measurement unit. Specifically, in the scenario of the millimeter-wave radar-IMU joint application sensor, the millimeter-wave radar can be used to detect the position of objects in the environment, etc., and obtain the position information of the objects in the environment relative to the millimeter-wave radar coordinate system; afterwards, the relative position obtained by the millimeter-wave radar can be converted into a position in the earth coordinate system through calculations using the positioning information provided by the IMU and the calibration parameters between the millimeter-wave radar and the IMU. And calibrating the parameters of the joint application sensors may include calibrating the translation and rotation parameters between the sensors, so as to process the data obtained by the joint application sensors based on the correct calibration parameters, thereby obtaining more accurate environmental perception results. In addition to calibrating spatial parameters such as translation and rotation, the method provided in the embodiment of the present application can also calibrate the time delay parameters between sensors to suppress the actual impact of the delay between sensors on the sensing results.

[0070] The embodiment of the present application describes a method for calibrating the parameters of a sensor (group) in motion when the first sensor and the second sensor are used in combination. The principle of the method is that for an object in a natural environment, if it is stationary, then no matter what posture the first sensor uses to detect it, the detected result should be consistent, that is, the position points detected in different postures should be overlapped or substantially overlapped. Since the final detection result needs to be converted using the reference information provided by the second sensor and the calibration parameters between the first sensor and the second sensor, the factor for detecting whether the calibration parameters of the sensor are accurate is whether the calibration parameters between the first sensor and the second sensor are applicable to the current physical state of the two. In other words, the position detection can be performed on a stationary object in different postures by the first sensor, and the coordinate system conversion reference value provided by the second sensor and the calibration parameters between the two are converted, and the position point information converted to a unified coordinate system is used as sample data, and then when optimizing the target parameters, the optimized values ​​of each target parameter when the position points overlap or the overlap degree is higher than a certain threshold are searched, so it can be explained that the optimized values ​​of the target parameters obtained as calibration parameters are applicable to the current physical state of the first sensor and the second sensor. Therefore, the embodiment of the present application can calibrate the parameters between the first sensor and the second sensor, that is, the combined application sensor, through the above principle.

[0071] Specifically, the embodiment of the present application mainly detects static environmental objects in the natural environment as target environmental objects, measures the target environmental objects at different positions of the first sensor to obtain their relative position information relative to the first sensor, and then converts the relative position information of the first position information according to the coordinate system conversion reference value and each group of target optimization parameter values, and obtains the second position information corresponding to each group of target optimization parameter values ​​in the unified coordinate system; the position data of the target environmental object at each position in each second position information is used as a sample, and the target parameter values ​​when the position points overlap or the overlap degree is higher than a certain threshold are searched to determine the calibration parameters of the first sensor and the second sensor. This method can perform online sensor parameter calibration during the operation of the carrier system, such as online calibration during the driving of an autonomous driving car, and does not rely on the intervention of external professional equipment, does not require the use of a specific site, does not require offline maintenance of the equipment, and achieves high efficiency and greatly reduces the cost compared to the existing offline method, and can be performed in real time when the device is online, which can effectively improve the safety of the intelligent system during operation. The following is a detailed introduction to the sensor parameter calibration method in conjunction with a specific embodiment.

[0072] Embodiment 1

[0073] See also Figure 2, is a flow chart of a method for calibrating sensor parameters for a combined application provided in Embodiment 1 of the present application. The method may include the following steps:

[0074] S210: during the movement of the combined application sensor, determining a static environment object included in the natural environment as a target environment object; the combined application sensor includes a first sensor and a second sensor;

[0075] Different from the offline calibration method using professional calibration equipment and venues in the prior art, the method provided in the first embodiment of the present application can realize the online calibration of the parameters of the joint application sensor during the movement process, and can use a large number of static environment objects in the natural environment as reference objects, such as buildings, billboards, signboards, etc. First, in the process of the movement of the joint application sensor, the static environment objects included in the natural environment can be determined as target environment objects. Usually, while there are a certain number of static environment objects around the carrier system, there will also be certain moving objects. The movement or static mentioned here can be a motion state relative to a unified coordinate system, such as relative to the earth coordinate system. In the process of determining the static environment objects in the natural environment as the target, it is necessary to distinguish between the static environment objects and the moving objects. When making a specific distinction, it can be judged according to the speed of the measured natural environment object. If the speed of a natural environment object is zero or less than a threshold value (close to zero), the object is judged to be a static environment object. The static environment object can be determined as a target environment object as a marker, or a static environment object that meets certain conditions can be determined as a target environment object, for example, a static environment object with a reflection area greater than a certain area threshold is determined as a target environment object. A plurality of different static environment objects may be determined as target environment objects to form a target environment object group.

[0076] The speed of the object in the natural environment can be detected by the first sensor, or by other sensors in the system. The combined application sensor may include a first sensor and a second sensor. The first sensor may be used to measure the environmental object information, for example, data for measuring the speed of the target object may be obtained. For example, the first sensor may be a millimeter wave radar, which can measure the relative speed of the target object by emitting radio waves and measuring echoes, that is, the first speed information of the target object in the natural environment relative to the first sensor may be obtained by measuring the first sensor. When the speed of the target object is measured by other sensors in the system, the other sensors can move synchronously or approximately synchronously with the first sensor because they are installed on the same carrier as the first sensor; the other sensors may be sensors that do not involve parameter calibration, but when the object in the natural environment is determined to be the target environment object, it is necessary to further determine the identification information of the object in order to identify the object, and in the next step, the position data of the same static object is measured by the first sensor. That is, the identification of the static object can be carried out by the first sensor, or by other sensors in the system, but the position data of the static object is used as the basis for calibrating the sensor parameters, and the determination of the position of the target environment object in the subsequent steps needs to be carried out by the sensor (first sensor) involved in parameter calibration.

[0077] The relative speed of the target object measured by the first sensor or other sensors. In addition, the speed of the first sensor itself can also be determined by the information provided by the sensor components in the carrier, for example, the speed of the first sensor itself can be determined by the information provided by the sensor components such as the GNSS (Global Navigation Satellite System) module, the IMU inertial measurement unit, or the IMU inertial measurement unit with RTK as an auxiliary unit, as the second speed information of the first sensor, and the second speed information is the speed of the first sensor in the earth coordinate system. If the first speed information and the second speed information meet certain conditions, such as being equal or very close in size and opposite in direction, it can be considered that the target object is a static object that can be used as a target environment object in the unified coordinate system (earth coordinate system). The sensor that provides information to determine the speed of the first sensor itself can be a second sensor. For example, when the second sensor is an IMU inertial measurement unit, the carrier speed can be obtained by solving the linear acceleration and rotation angular rate output by the IMU, which can be used as the approximate speed of the first sensor itself; or an IMU with RTK as an auxiliary unit can determine the first sensor's own speed as the second speed information based on the positioning information provided by the RTK auxiliary unit, that is, the first sensor's own speed can be obtained through the positioning information provided by the second sensor. For example, if the second sensor is an IMU with RTK as an auxiliary unit, the second speed information of the first sensor can be determined through the positioning information provided by the second sensor. Alternatively, the second speed information of the first sensor can also be determined by other sensor components in the carrier system that have a certain positional relationship with the first sensor.

[0078] The jointly applied sensor includes a first sensor and a second sensor. The first sensor is used to measure the information of the environmental object. For example, the first sensor can be a millimeter wave radar, which can measure the position, speed and other information of the environmental object by emitting electromagnetic waves and obtaining the point cloud formed by the echo. The second sensor can be used to provide a coordinate system conversion reference value. For example, the second sensor can include an IMU inertial measurement unit, which has a built-in acceleration sensor and a gyroscope, which can measure the linear acceleration and rotation angular rate from three directions. The attitude, speed and displacement of the carrier can be obtained by solving the calculation. The actual speed information of the target object in the natural environment is determined according to the first speed and the second speed, and whether the object in the natural environment is a static environmental object is determined according to the actual speed information. For example, when the jointly applied sensors are millimeter wave radar and IMU, the above-mentioned first speed information and second speed information can be obtained respectively by these two jointly applied sensors, and the speed of the target object in the earth coordinate system can be obtained by the following calculation:

[0079]

[0080] Among them, v obj is the instantaneous relative velocity of the target measured by the millimeter-wave radar, l is the translation parameter of the combined sensor, Indicates the rotation parameters of the combined application sensor. The translation parameters and rotation parameters can be pre-calibrated parameters before this calibration. imu represents the speed output by the IMU unit, and ω represents the angular velocity output by the IMU unit. When the speed v of the target object in the unified coordinate system is determined obj When it is zero or a negligible value close to zero, it can be determined that the target object is a static object and can be used as a target environment object or as a candidate for a target environment object.

[0081] For environmental objects that have been determined as static objects, they can be further screened in a certain way to determine objects that are more suitable as target environmental objects, improve the effectiveness of the selected target objects, and improve the accuracy of the calibration results. For example, a static object with a reflection area greater than a specific area value can be determined as a target environmental object. When multiple static objects are used as reference objects for calibration, if static environmental objects exceeding the required number are detected in the environment, the static environmental object with better signal feedback can be selected as the target environmental object. In addition, the calibration task of the combined application of sensor calibration parameters can be triggered under specific conditions, for example, when a specific signal sending device can be set in the road, when the autonomous driving vehicle receives a specific signal from the signal sending device in the road during the journey, the corresponding sensor calibration task can be started. In addition to objects such as buildings and signboards, static objects can also include objects specially designed for sensor calibration parameter purposes. For example, it is not easy to find static objects suitable as target environmental objects on sections of the road. Special objects for sensor calibration parameter purposes can be set. Such dedicated static objects can be used in conjunction with the aforementioned signal sending device to reduce the search frequency for static targets in sensor calibration tasks. This combination can also be applied to smart roads.

[0082] The carrier system can receive instructions for calibrating the parameters of the combined application sensors. The instructions can come from outside the carrier system, such as the example of the smart road mentioned above, or from inside the carrier system. After determining the instructions for calibration, the calibration task of the combined application sensor parameters can be started. The instructions for calibrating the parameters of the combined application sensors can be formed in a variety of ways. For example, the instructions for calibrating the parameters of the combined application sensors can be determined according to a preset time. In specific implementation, a timer or other application can be set in the carrier system, and the instructions for calibrating the parameters of the combined application sensors can be determined according to the preset time to start the verification task; or, the calibration task can be started according to the mileage status of the carrier system. For example, in an autonomous driving vehicle system, the calibration task is performed according to the mileage status of the autonomous driving vehicle. When the carrier mileage count reaches a specific value or value range, the instruction for calibrating the parameters of the combined application sensors is determined.

[0083] During the movement of the carrier, the calibration task can also be started in real time when a more serious bump occurs. First, the bump degree of the carrier system can be determined, for example, the bump degree of the system can be determined based on the change in the acceleration obtained by the IMU unit in the carrier. For example, when the acceleration vector exceeds a specific threshold, it is determined that the system has a more obvious bump. In this case, the calibration of the joint application sensor parameters can be started, that is, the instruction for calibrating the joint application sensor parameters can be determined based on the bump degree information during the movement of the carrier, and the generated instruction is received and executed. In addition, a human-computer interaction interface can also be provided, and the instruction for calibrating the joint application sensor parameters can be determined based on the user's operation on the human-computer interaction interface. In actual applications, the joint application sensor parameters can also be verified first, and then the instruction for calibrating the joint application sensor parameters can be determined based on the verification results. During the calibration process, the following online calibration method based on the measured data of the target environment object can be applied.

[0084] S220: measuring the target environment object by using the first sensor in a plurality of postures to determine first position information of the target environment object relative to the first sensor;

[0085] The target environment object is determined, and then the target environment object can be measured by the first sensor. The measurement can include multiple measurements of the target environment object at different positions. The content of the measurement mainly includes determining the first position information of the target environment object in the first sensor coordinate system. The number of target environment objects can include one or more. Compared with a single target environment object, the position information related to multiple environment objects is collected as sample data at the same time, and the sensor calibration parameters are optimized based on the sample data related to multiple environment objects. The optimized value of the calibration parameters is determined, which can effectively improve the accuracy of the calibration parameters and can be used as a preferred method. Of course, selecting a single target environment object is also feasible. The implementation subject of the measurement is the first sensor. The first sensor can have the function of measuring the information of the environment object, for example, it can have the function of detecting the relative speed, relative position and other information of the environment object. For example, the first sensor can be a millimeter wave radar, a laser radar, etc. Taking the millimeter wave radar as an example, it can emit electromagnetic waves and receive echoes to measure the position, speed and other information of the target environment object. The first position information of multiple target environment objects obtained at different positions is shown in Table 1:

[0086] Table 1

[0087]

[0088] The first position information shown in Table 1 includes the position information of m target environmental objects in n positions. For example, the first sensor measures environmental object 3 in position 1 and obtains the position of the environmental object in the position p. 13 . The posture information can be provided by the second sensor. For example, when the second sensor is an IMU inertial measurement unit, real-time posture information can be obtained through the sensing information of the IMU. Usually, the posture information is also used as the posture information of the carrier, and the posture information usually changes in real time with the movement of the carrier. The "relative position" in the first position information can be the relative position of each environmental object relative to the first sensor in the corresponding posture. It can be imagined that when the posture changes, the carrier system is usually in the process of moving. In different postures of the carrier system, the relative position of the same target environmental object relative to the first sensor will be different.

[0089] In order to obtain the first position information of the target environment object in the first sensor coordinate system in a specific implementation, the target environment object can be measured by the first sensor in multiple positions to obtain point cloud data in each position, and then the first position information of each target environment object in each position can be determined based on the point cloud data in different positions. In a specific implementation, the point cloud data of each target environment object in each position can be clustered, and the first position information can be determined based on the clustering results. For example, the first position information of the target environment object in multiple positions can be obtained by clustering the point cloud. The first position information, for example, can be the center point of the target environment object obtained by clustering the point cloud data. Determining the center point of a target environment object based on the point cloud data can be achieved using the above-mentioned clustering method, such as the K-mean method or other point trace condensation methods.

[0090] Since when the millimeter wave radar or the like measures static markers, the obtained point cloud data may contain noise that does not belong to the static environmental object markers, the points that do not belong to the static environmental objects may be filtered before using the point cloud data to determine the first position information. In specific implementation, for the point cloud data of a target environmental object in a certain posture, feature matching may be performed based on relevant parameter information generated by some radars. For example, before clustering, feature matching may be performed on the point cloud data of the target environmental object based on parameter information related to the point cloud data obtained by the first sensor, and points that do not belong to the target environmental object may be filtered out to eliminate interference data as much as possible and obtain more accurate first position information. Specifically, when filtering points that do not belong to the target environmental object, it may be based on parameter information related to the point cloud data, such as posture information, first sensor reflection cross-section information, or Doppler state information. Of course, filtering may also be performed using two or more of these types of information in combination to obtain better filtering effects.

[0091] S230: determining a target optimization parameter and each group of values ​​of the target optimization parameter, converting the relative position information of the first position information, and obtaining second position information corresponding to each group of target optimization parameter values ​​in a unified coordinate system;

[0092] Determine the target optimization parameters that need to be optimized. The target optimization parameters may include the spatial parameters between the first sensor and the second sensor, such as the rotation parameters and translation parameters between the first sensor and the second sensor. In addition, the time delay parameters between the first sensor and the second sensor that are difficult to calibrate by the existing method may also be included. The second sensor may include an inertial measurement unit. The process of parameter optimization can be regarded as a process of finding the optimization value in the parameter space. Usually, this process uses multiple sets of parameter values, and searches to make the objective function (cost function) value meet the optimization goal. The optimization goal may be to search for the minimum value of the cost function, or to make the cost function value meet specific requirements. In the process of parameter optimization, multiple combinations of target optimization parameters may be determined. The values ​​of each set of target optimization parameters used may be determined in a variety of ways, such as the method of exhaustively enumerating candidate parameter combinations on the parameter grid, the gradient descent method, the Newton method, and the like. According to the coordinate system conversion reference value and each set of values ​​of the target optimization parameters, the relative position information of the first position information is converted to obtain the second position information corresponding to each set of target optimization parameter values ​​in the unified coordinate system. It can be seen that for each set of optimization parameter values, there will be a corresponding second position information. For example, as shown in Table 2:

[0093] Table 2

[0094]

[0095] Where R k represents a set of optimized parameter values, Represents the corresponding second position information obtained by transformation based on the values ​​of the group of optimization parameters and the coordinate system transformation reference value. Usually represents a sequence of positions.

[0096] The first position information is the relative position of each target environmental object relative to the first sensor in each posture. The relative position of the same target environmental object relative to the first sensor in different postures will be different, but the target environmental object is a static landmark in the natural world. In a unified coordinate system, such as the earth coordinate system, the same target environmental object in different postures has coordinate consistency. In this way, although the relative positions in different postures are different, if the coordinate system of each relative position information is converted according to the coordinate system conversion reference value provided by the second sensor and the values ​​of the same set of target optimization parameters, and converted to a unified earth coordinate system, under the condition that the values ​​of the target optimization parameters are accurate, the unified coordinate system coordinates obtained when measuring the same static marker in different postures should be consistent. This consistency is reflected in the overlap or deviation of the position data being less than a specific threshold.

[0097] According to the coordinate system conversion reference value provided by the second sensor and each set of values ​​of the target optimization parameter, the relative position information in the first position information in the coordinate system of the first sensor can be converted to obtain the second position information in the unified coordinate system. The second sensor can provide a coordinate system conversion reference value including the carrier posture and can also include position information. For example, the position information can be determined by an IMU with RTK as an auxiliary unit, and the position information can also be measured by other positioning sensor devices.

[0098] In specific implementation, the influence of the time delay parameters between the jointly applied sensors can also be considered, and the target optimization parameters include the time delay parameters. First, the position compensation amount of each relative position information under each posture in the first position information can be obtained according to the time delay parameter value in each set of values ​​of the target optimization parameters, the angular rate information output by the inertial measurement unit, and the speed and angle information output by the first sensor. Taking the joint application of the millimeter wave radar and the IMU unit as an example, the millimeter wave radar is the first sensor and the IMU unit is the second sensor. The position compensation amount can be calculated in the following way:

[0099]

[0100] Among them, τ 0 is the time delay parameter value between the millimeter-wave radar and the IMU unit, ω is the angular rate output by the IMU unit, v and α are the speed and angle information output by the millimeter-wave radar, respectively. Then, the second position information corresponding to the values ​​of each group of target optimization parameters in the unified coordinate system can be determined according to the corresponding attitude matrix of the second sensor, the translation and rotation parameter values ​​of each group of values ​​of the target optimization parameters, the position information of the second sensor in the unified coordinate system, and the position compensation amount determined above. The second position information may include the position information of each relative position in the first position information in the unified coordinate system (usually the earth coordinate system). When making a specific determination, it can be carried out according to the following method:

[0101]

[0102] in, is the attitude matrix output by the IMU unit, l and are the translation and rotation parameter values ​​for each set of target optimization parameters, is the relative position in the first position information, for example, it may be the relative position information relative to the target environment object of the first sensor, represents the position information of the second sensor in the unified coordinate system. When multiple target environment objects are used for calibration, the obtained X i The data in Table 2 may include:

[0103] Table 3

[0104]

[0105] The data in Table 2 can be expressed as the position sequence X of each environmental object in the unified coordinate system under different postures i :

[0106] {X 1 ,X 2 ,…,X n};

[0107] X i represents the second position information under the i-th pose. When multiple target environment objects are used, X i The position information of a set of target environment objects, namely, the position sequence X i It can be expressed as:

[0108] {(x 11 ,x 12 ,…,x 1m ),(x 21 ,x 22 ,…,x 2m ),…,(x n1 ,x n2 ,……,x nm )};

[0109] Where m is the number of target environment objects. When there are k sets of target optimization parameter values, k similar position sequences can be obtained.

[0110] S240: Determine an optimization value of a target optimization parameter based on the position data of the target environment object at each posture in each of the second position information, and determine the optimization value as a calibration parameter of the first sensor and the second sensor.

[0111] During the movement of the carrier, the relative position of the same target environment object relative to the first sensor in different postures will be different, but when converted to the position in the unified coordinate system, it should be the same or have an acceptable deviation error when the external parameters of the sensor are normal. On the contrary, the value of the target optimization parameter can be measured according to the size of this deviation error to measure whether it is an optimized value. For each set of target optimization parameters, the pros and cons of the values ​​of the target optimization parameters can be determined according to the overlap between the second position information of the target environment object obtained by conversion under multiple postures. The pros and cons of the values ​​of each set of target optimization parameters can be measured with the corresponding second position data. It can be done in different ways, for example, a specific method can be used to calculate the deviation of the target environment object position measurement data under each posture, and determine the difference between the deviation and the preset value, and determine the suitability of a set of target optimization parameters according to the size of the difference. In one implementation, if the deviation is less than the preset value, it can be considered that the search condition is met.

[0112] In another implementation, for the position sequence of the second position information obtained by converting the values ​​of a set of target optimization parameters, the evaluation value of the calibration parameter can be determined based on the position measurement data of the target environment object in each posture, and the values ​​of each set of target optimization parameters can correspond to their respective evaluation values, and the value of the group target optimization parameter corresponding to the minimum value of each evaluation value is determined as the optimization value. The evaluation value can be determined by a cost function, and the cost function can be used to find sensor calibration parameters and can also be used to evaluate the quality of calibration parameters.

[0113] For example, when calibrating the jointly applied sensor, the following cost function may be used for the position sequence of the second position information obtained by converting the values ​​of a set of target optimization parameters:

[0114]

[0115] Where n is the number of poses, X i It is the position sequence of each environmental object in the unified coordinate system under different postures, as mentioned above.

[0116] For each set of target optimization parameters, k , there is a position sequence Correspondingly, the corresponding evaluation value J can be obtained by using the cost function calculation. The minimum value can be searched among all the evaluation values ​​J, and the corresponding set of target optimization parameter values ​​can be used as the optimization value, that is, each second position information is input into the preset cost function respectively, and the function value corresponding to the value of each set of target optimization parameters is determined; wherein each second position information includes the position data of each target environment object in each posture in a unified coordinate system; the minimum value among each function value is determined, and the value of the target optimization parameter corresponding to the minimum value is determined as the optimization value, and the obtained optimization value is determined as the calibration parameter of the first sensor and the second sensor, and the obtained calibration parameter is used to calibrate the parameters of the joint application sensor.

[0117] The above introduces the calibration method of the joint application sensor parameters provided in the first embodiment of the present application. The method can determine the static environmental objects included in the natural environment as the target environmental objects during the movement of the joint application sensor; the joint application sensor includes a first sensor and a second sensor, the first sensor is used to measure the environmental object information, and the second sensor is used to provide a coordinate system conversion reference value; the target environmental object is measured by the first sensor in a variety of postures to determine the first position information of the target environmental object relative to the first sensor; the target optimization parameters are determined, and the relative position information of the first position information is converted according to the coordinate system conversion reference value and each group of values ​​of the target optimization parameters, to obtain the second position information corresponding to each group of target optimization parameter values ​​in a unified coordinate system; based on the position data of the target environmental object in each posture in each second position information, the optimization value of the target optimization parameter is determined, and the optimization value is determined as the calibration parameter of the first sensor and the second sensor. Through this method, online and real-time parameter calibration of the jointly applied sensors can be achieved during the operation of the system. The parameter calibration does not rely on the intervention of external professional equipment, nor does it rely on specific sites. It can be carried out during the operation of the carrier system, such as during the driving of an autonomous vehicle, based on sample data of objects in a natural state measured by online sensors. When the system is in operation, real-time error correction can be performed on the time and space parameter-related working conditions of the jointly applied sensors. Offline maintenance of the equipment is not required, and high efficiency is achieved with greatly reduced costs compared to existing offline methods. Time delay parameters that cannot be accurately calibrated in existing technologies can be calibrated, effectively suppressing the actual impact of delays between sensors on sensing results, and providing effective protection for the safe operation of the automation system.

[0118] Embodiment 2

[0119] The second embodiment of the present application provides a method for calibrating the parameters of the sensor used in a joint application. The calibration of the parameters of the sensor used in a joint application in a carrier can be initiated by receiving a command from an external device. The method can be applied to scenarios such as carrier permission control. For example, when the carrier is an autonomous driving vehicle, the sensor parameter calibration can be performed on the autonomous driving vehicle system. Entry is allowed only after the sensors related to the autonomous driving vehicle are correctly calibrated, thereby improving the safety of autonomous driving vehicles, smart roads and other applications. By automatically calibrating the data measured based on natural markers, the efficiency of sensor parameter calibration is improved and the cost of parameter calibration is reduced. Figure 3 As shown, the method may include the following steps:

[0120] S310: receiving an instruction from an external device to calibrate parameters of a joint application sensor in a carrier;

[0121] The carrier system can receive instructions from the external device end to calibrate the parameters of the combined application sensors in the carrier. For example, when the carrier is an autonomous driving vehicle, it can receive instructions from the roadside equipment during movement. After receiving the instructions, it starts to calibrate the parameters of the combined application sensors in the carrier system. The external device end can communicate with the carrier system and send parameter calibration instructions under specific applications. For example, when the robot enters the working state, the autonomous driving vehicle enters a specific road, or the carrier system passes through a specific calibration task area set for it, etc. Under specific application conditions, the carrier movement speed will affect the calibration accuracy. When performing parameter calibration, the movement speed of the carrier device can be limited to better ensure the accuracy of the calibration. When the carrier enters the calibration task area, it receives instructions from the external device end to calibrate the parameters of the combined application sensors in the carrier. After receiving the instructions, the carrier device can adjust the current speed of the carrier to the preset speed limit value. The speed limit value can be set by the carrier system according to the preset value or can also be specified by the external device end, for example, in the form of an instruction parameter and sent to the carrier system together with the instruction.

[0122] S320: Determine target optimization parameters and values ​​of each group of target optimization parameters;

[0123] After receiving the instruction from the external device, the carrier system can determine the target optimization parameters that need to be optimized. The target optimization parameters can include the spatial parameters between the joint application sensors, such as the rotation parameters and translation parameters between the joint application sensors. In addition, it can also include the time delay parameters between the joint application sensors that are difficult to calibrate with existing methods. Usually, the calibration process uses multiple groups of parameter values, which can determine the target optimization parameters and the values ​​of each group of target optimization parameters.

[0124] S330: determining a target environment object, and in the process of the carrier moving, measuring the target environment object by using the combined application sensor based on each group of values ​​of the target optimization parameter to obtain measurement data;

[0125] After determining the target optimization parameters and each group of values ​​of the target optimization parameters, the target environment object can be determined. In the process of moving the carrier, based on each group of values ​​of the target optimization parameters, the target environment object is measured by the combined application sensor to obtain measurement data. The target environment object can be a static environment object that exists in large quantities in the natural environment. Of course, it can also be an artificial marker, such as a simple plane sign, etc., that is, the method does not require expensive professional tools or equipment, low cost and easy to implement. In specific implementation, the method of measuring the target environment object by the combined application sensor in Example 1 can be adopted. The combined application sensor includes a first sensor and a second sensor. In the process of the combined application sensor moving with the carrier, the target environment object can be measured by the first sensor in a variety of postures to determine the first position information of the target environment object relative to the first sensor; based on each group of values ​​of the target optimization parameters, the relative position information of the first position information is converted to obtain the second position information corresponding to each group of values ​​of the target optimization parameters in the unified coordinate system. Usually, each group of target optimization parameter values ​​will correspond to a measurement data group, and each group of measurement data constitutes a set, and the obtained set is used as the measurement data.

[0126] S340: Determine an optimized value of a target optimized parameter based on the measured data, and determine the optimized value as a calibration parameter of the combined application sensor.

[0127] According to the measured data, the optimized value of the target optimization parameter is determined. The process can be a process of searching for the optimized value in the parameter space. When implementing it specifically, you can refer to the specific implementation in Example 1. For example, it can be similar to the process of step 230 and step 240 in Example 1. For a set of values ​​of the target optimization parameter, determine the overlap of the second position information in each posture, and determine the set of values ​​with the highest overlap of each set of values ​​as the optimized value. The specific process will not be repeated here. After determining the optimized value of the target optimization parameter, the determined optimized value can be determined as the calibration parameter of the joint application sensor to perform parameter calibration on the joint application sensor.

[0128] The determined calibration parameters are used to calibrate the parameters of the joint application sensor, and the calibration completion information can also be sent to the external device end after the calibration is completed. The external device end can perform corresponding measures according to the calibration completion information to determine the next action of the carrier system, and the carrier system can receive the action instructions sent by the external device end determined according to the calibration completion information. Taking the carrier system as an autonomous driving vehicle system as an example, receiving the action instructions sent by the external device end determined according to the calibration completion information can be receiving the pass permission action instructions sent by the external device end determined according to the calibration completion information. The pass permission action instructions can include permission instructions to pass through a preset detection area, or permission instructions to enter a preset road, etc. For autonomous driving application scenarios, the carrier may include an autonomous driving vehicle, and the external device includes roadside equipment in a smart road.

[0129] The calibration method of the joint application sensor parameters provided in the second embodiment of the present application can determine the target optimization parameters and each group of values ​​of the target optimization parameters after receiving the instruction issued by the external device end to calibrate the parameters of the joint application sensor in the carrier; determine the target environment object, and in the process of the movement of the carrier, based on each group of values ​​of the target optimization parameters, measure the target environment object through the joint application sensor to obtain measurement data; according to the obtained measurement data, determine the optimization value of the target optimization parameter, and determine the optimization value as the calibration parameter of the joint application sensor to calibrate the parameters of the joint application sensor. Through this method, the online and real-time parameter calibration of the joint application sensor can be realized during the operation of the system. The calibration of the parameters does not rely on the intervention of external professional equipment, nor does it rely on a specific site, etc. During the operation of the carrier system, the temporal and spatial parameter-related working conditions of the joint application sensor can be corrected in real time, and the equipment does not require offline maintenance, so as to achieve high efficiency and greatly reduce the cost compared with the existing offline method. After the calibration is completed, the calibration completion information can also be sent to the external device end. The external device end can execute corresponding measures according to the calibration completion information to determine the next action of the carrier system, such as limiting the behavior of the carrier system. The time delay parameters that cannot be accurately calibrated in the existing technology can be calibrated, effectively suppressing the actual impact of the delay between sensors on the sensing results and ensuring the safe operation of the automation system.

[0130] Embodiment 3

[0131] Embodiment 3 of the present application provides a method for verifying the calibration parameters of a combined application sensor, which is characterized in that it is a method corresponding to the method of embodiment 2 and described from the external device side. This method can be applied to external devices other than the carrier, such as when the carrier is an autonomous driving vehicle, and the external device can be a roadside device. Figure 4 As shown, the method may include:

[0132] S410: The external device sends an instruction to the carrier system to calibrate the parameters of the joint application sensor in the carrier;

[0133] The external device side automatically issues an instruction to the carrier system to calibrate the parameters of the joint application sensors. The carrier system can receive instructions for calibrating the parameters of the joint application sensors issued by the external device side. For example, when the carrier is an autonomous driving vehicle, it can receive instructions from the roadside equipment during movement. After receiving the instructions, it starts to calibrate the parameters of the joint application sensors in the carrier system. In the application of setting the detection area, the external device side can detect the carrier entering the preset calibration area. When a carrier is detected entering the preset calibration area, a calibration instruction is issued to the corresponding carrier system that has entered.

[0134] S420: After receiving the instruction, the carrier system determines the target optimization parameters and each group of values ​​of the target optimization parameters; determines the target environmental object, and during the movement of the carrier, based on the each group of values ​​of the target optimization parameters, measures the target environmental object through the combined application sensor to obtain measurement data; determines the optimized values ​​of the target optimization parameters based on the measurement data, and uses the optimized values ​​to calibrate the parameters of the combined application sensor.

[0135] After the carrier system receives the instruction, the target optimization parameters may include the spatial parameters between the joint application sensors, such as the rotation parameters and translation parameters between the joint application sensors. In addition, it may also include the time delay parameters between the joint application sensors that are difficult to calibrate with existing methods. Usually, the calibration process uses multiple sets of parameter values, which can determine the target optimization parameters and each set of values ​​of the target optimization parameters. After the target optimization parameters and each set of values ​​of the target optimization parameters are determined, the target environment object can be determined. During the movement of the carrier, based on the each set of values ​​of the target optimization parameters, the target environment object is measured by the joint application sensor to obtain measurement data. The target environment object can be a static environment object that exists in large numbers in the natural environment. Of course, it can also be an artificial marker, such as a simple plane sign, etc., that is, this method does not require expensive professional tools or equipment, is low-cost and easy to implement. In specific implementation, the method of measuring the target environment object by the combined application sensor in Example 1 can be adopted. The combined application sensor includes a first sensor and a second sensor. In the process of the combined application sensor moving with the carrier, the target environment object can be measured by the first sensor in a variety of postures to determine the first position information of the target environment object relative to the first sensor; the relative position information of the first position information is converted to obtain the second position information corresponding to each group of target optimization parameter values ​​in a unified coordinate system. According to the measured data, the optimized value of the target optimization parameter is determined, and the process can be a process of finding the optimized value in the parameter space. In specific implementation, the specific implementation in Example 1 can be referred to, such as the process similar to step 230 and step 240 in Example 1. For a group of values ​​of the target optimization parameter, the overlap of the second position information in each posture is determined, and the group of values ​​with the highest overlap of each group of values ​​is determined as the optimized value. The specific process will not be repeated here. After determining the optimized value of the target optimization parameter, the determined optimized value can be determined as the calibration parameter of the combined application sensor to calibrate the parameters of the combined application sensor.

[0136] Embodiment 3 of the present application provides a method for calibrating parameters of a joint application sensor, in which an instruction for calibrating the parameters of the joint application sensor in the carrier can be sent from an external device end to a carrier system; after receiving the instruction, the carrier system determines the target optimization parameters and each group of values ​​of the target optimization parameters; determines the target environment object, and in the process of the carrier moving, based on each group of values ​​of the target optimization parameters, measures the target environment object through the joint application sensor to obtain measurement data; according to the measurement data, determines the optimization value of the target optimization parameter, and uses the optimization value to calibrate the parameters of the joint application sensor. Through this method, online and real-time parameter calibration of the joint application sensor can be achieved during the operation of the system. The calibration of the parameters does not rely on the intervention of external professional equipment, nor does it rely on a specific site, etc. During the operation of the carrier system, the temporal and spatial parameter-related working conditions of the joint application sensor can be corrected in real time, and the equipment does not require offline maintenance, so as to achieve high efficiency and greatly reduce the cost compared with the existing offline method. After the calibration is completed, the calibration completion information can also be sent to the external device end. The external device can execute corresponding measures based on the calibration completion information to determine the next action of the carrier system, such as limiting the behavior of the carrier system, thereby improving the safety of the operation of the automation system.

[0137] Corresponding to the calibration method for the combined application sensor parameters provided in the first embodiment of the present application, a calibration device for the combined application sensor parameters is also provided, such as Figure 5 As shown, the device may include:

[0138] The static object determination unit 510 is used to determine the static environmental objects included in the natural environment as the target environmental objects during the movement of the combined application sensor; the combined application sensor includes a first sensor and a second sensor, the first sensor is used to measure the environmental object information, and the second sensor is used to provide a coordinate system conversion reference value;

[0139] A position measuring unit 520, configured to measure the target environment object by using the first sensor in a plurality of positions and postures, and determine first position information of the target environment object relative to the first sensor;

[0140] The position conversion unit 530 is used to determine the target optimization parameter, and convert the relative position information of the first position information according to the coordinate system conversion reference value and each group of target optimization parameter values, to obtain the second position information corresponding to each group of target optimization parameter values ​​in the unified coordinate system;

[0141] The parameter determination unit 540 is used to determine the optimization value of the target optimization parameter based on the position data of the target environment object at each posture in each second position information, and determine the optimization value as the calibration parameter of the first sensor and the second sensor.

[0142] In one implementation, the static object determination unit 510 may be configured to:

[0143] Obtaining first speed information of an object in a natural environment relative to the first sensor by measuring with the first sensor;

[0144] determining a movement speed of the first sensor as second speed information;

[0145] The actual speed information of the object in the natural environment is determined according to the first speed and the second speed, and whether the object in the natural environment is a static environment object is determined according to the actual speed information.

[0146] When determining the movement speed of the first sensor as the second speed information, the movement speed of the first sensor may be determined as the second speed information through positioning information provided by the second sensor.

[0147] In another implementation, the location determination unit 520 may include:

[0148] A clustering subunit, used to determine point cloud data obtained by measuring the target environment object at each posture by the first sensor; and

[0149] The point cloud data obtained at each pose are clustered respectively, and the first position information of the target environment object at each pose is determined according to the clustering results.

[0150] In this embodiment, the device may further include:

[0151] The data filtering unit is used to perform feature matching on the point cloud data of the target environment object according to parameter information related to the point cloud data obtained by the first sensor before clustering processing, and filter out points in the point cloud data that do not belong to the target environment object.

[0152] The parameter information related to the point cloud data may include posture information, first sensor reflection cross-section information and / or Doppler state information.

[0153] The parameter determination unit 540 includes:

[0154] The first parameter determination subunit is used to determine the spatial parameters between the first sensor and the second sensor, and / or the time delay parameters as target optimization parameters; wherein the spatial parameters include the rotation parameters and the translation parameters between the first sensor and the second sensor.

[0155] The second sensor may include an inertial measurement unit, and the position conversion unit 530 may include:

[0156] A compensation amount acquisition subunit is used to obtain a position compensation amount of each relative position information at each position in the first position information according to the time delay parameter value of each group of target optimization parameter values, the angular rate information output by the inertial measurement unit, and the speed and angle information output by the first sensor;

[0157] The position conversion subunit is used to determine each second position information in the unified coordinate system according to the corresponding posture matrix output by the second sensor in each posture, the translation and rotation parameter values ​​between the first sensor and the second sensor, the position information of the second sensor in the unified coordinate system, and the position compensation amount.

[0158] Each set of values ​​of the target optimization parameter may correspond to a second position information, and the parameter determination unit 540 may include:

[0159] The second parameter determination subunit is used to input each second position information into a preset cost function to determine the function value corresponding to the value of each group of target optimization parameters; wherein each second position information includes the position data of each target environment object in a unified coordinate system at each posture;

[0160] The optimization value determination subunit is used to determine the minimum value among the function values, and determine the value of the target optimization parameter corresponding to the minimum value as the optimization value.

[0161] In one implementation, two or more static environmental objects included in the natural environment may be determined as target environmental objects. The first sensor may include a millimeter wave radar, and the second sensor may include an inertial measurement unit.

[0162] In another implementation, the joint application sensor parameter calibration device may further include:

[0163] a calibration instruction determination unit, used to determine an instruction for calibrating parameters of the combined application sensor;

[0164] The calibration task starting unit is used to start the calibration task of the joint application sensor parameters after receiving the calibration instruction.

[0165] The calibration instruction determination unit may be specifically used for:

[0166] An instruction for calibrating the parameters of the combined application sensor is determined according to a preset time.

[0167] The calibration instruction determination unit may be further used for:

[0168] When the vehicle mileage count reaches a specific value or value range, an instruction to calibrate the combined application sensor parameters is determined.

[0169] The calibration instruction determination unit may be further used for:

[0170] Instructions for calibrating the parameters of the combined application sensor are determined based on information about the degree of bumpiness during the movement of the carrier.

[0171] The calibration instruction determination unit may be further used for:

[0172] A human-computer interaction interface is provided, and instructions for calibrating the parameters of the combined application sensor are determined according to the user's operation on the human-computer interaction interface.

[0173] Alternatively, the calibration instruction determination unit may be specifically used to:

[0174] The parameters of the combined application sensor are verified, and instructions for calibrating the parameters of the combined application sensor are determined according to the verification results.

[0175] The above introduces the calibration device for the parameters of the combined application sensor provided in the embodiment of the present application. The device can determine the static environmental objects included in the natural environment as the target environmental objects during the movement of the combined application sensor; the combined application sensor includes a first sensor and a second sensor, the first sensor is used to measure the environmental object information, and the second sensor is used to provide a coordinate system conversion reference value; the target environmental object is measured by the first sensor in a variety of postures to determine the first position information of the target environmental object relative to the first sensor; the target optimization parameters are determined, and the relative position information of the first position information is converted according to the coordinate system conversion reference value and each group of values ​​of the target optimization parameters, respectively, to obtain the second position information corresponding to each group of target optimization parameter values ​​in a unified coordinate system; based on the position data of the target environmental object in each posture in each second position information, the optimization value of the target optimization parameter is determined, and the optimization value is determined as the calibration parameter of the first sensor and the second sensor. Through this device, online and real-time parameter calibration of the jointly applied sensors can be realized during the operation of the system. The parameter calibration does not rely on the intervention of external professional equipment, nor does it rely on specific sites. It can be carried out during the operation of the carrier system, such as during the driving of an autonomous vehicle, based on sample data of objects in a natural state measured by online sensors. When the system is in operation, real-time error correction can be performed on the time and space parameter-related working conditions of the jointly applied sensors. Offline maintenance of the equipment is not required, and high efficiency is achieved with greatly reduced costs compared to existing offline methods. Time delay parameters that cannot be accurately calibrated in the existing technology can be calibrated, effectively suppressing the actual impact of the delay between sensors on the sensing results, and improving the safety of the operation of the automation system.

[0176] Corresponding to the calibration method for the combined application of sensor calibration parameters provided in the second embodiment of the present application, another calibration device for the combined application of sensor calibration parameters is also provided, such as Figure 6 As shown, the device may include:

[0177] The calibration instruction receiving unit 610 is used to receive an instruction issued by an external device to calibrate the parameters of the joint application sensor in the carrier;

[0178] A parameter value determination unit 620, used to determine target optimization parameters and values ​​of each group of target optimization parameters;

[0179] The measurement data determination unit 630 is used to determine the target environment object, and in the process of the carrier moving, based on each group of values ​​of the target optimization parameter, measure the target environment object through the combined application sensor to obtain measurement data;

[0180] The optimization parameter determination unit 640 is used to determine the optimization value of the target optimization parameter according to the measurement data, and determine the optimization value as the calibration parameter of the joint application sensor.

[0181] Wherein, the combined application sensor may include a first sensor and a second sensor;

[0182] The measurement data determination unit 630 may be used to:

[0183] In the process of the combined application sensor moving with the carrier, the target environment object is measured by the first sensor in a variety of positions to determine the first position information of the target environment object relative to the first sensor;

[0184] The relative position information of the first position information is converted to obtain second position information corresponding to each group of target optimization parameter values ​​in a unified coordinate system.

[0185] In this implementation, the optimization parameter determination unit 640 may be used to:

[0186] For a set of values ​​of the target optimization parameter, the overlap of the second position information in each posture is determined, and a set of values ​​with the highest overlap among the sets of values ​​is determined as the optimized value.

[0187] In another embodiment, the calibration device for the combined application sensor calibration parameter may further include:

[0188] The completion information sending unit is used to perform parameter calibration on the combined application sensor using the determined calibration parameters, and send calibration completion information to the external device end after the calibration is completed.

[0189] In this embodiment, the calibration device for the combined application sensor calibration parameter may further include:

[0190] The advanced instruction receiving unit is used to receive an action instruction sent by an external device and determined according to the calibration completion information.

[0191] The carrier can be an autonomous driving vehicle, an advanced command receiving unit, which can be used for:

[0192] Receive the permission action instruction sent by the external device based on the calibration completion information.

[0193] The passing permission action instruction may include a permission instruction for allowing passing through a preset detection area, or a permission instruction for allowing entering a preset road.

[0194] The calibration device for the combined application sensor calibration parameters may also include:

[0195] The carrier speed adjustment unit is used to adjust the current speed of the carrier to a preset speed limit value.

[0196] Among them, the carrier may include an autonomous driving vehicle, and the corresponding external equipment may include roadside equipment in a smart road.

[0197] Corresponding to the calibration method for the combined application of sensor calibration parameters provided in the third embodiment of the present application, another calibration device for the combined application of sensor calibration parameters is provided, such as Figure 7 As shown, the device may include:

[0198] The calibration instruction issuing unit 710 is used for the external device to issue an instruction to the carrier system to calibrate the parameters of the joint application sensor in the carrier;

[0199] The calibration processing unit 720 is used to determine the target optimization parameters and each group of values ​​of the target optimization parameters after the carrier system receives the instruction; determine the target environment object, and during the movement of the carrier, based on the each group of values ​​of the target optimization parameters, measure the target environment object through the combined application sensor to obtain measurement data; determine the optimized value of the target optimization parameter based on the measurement data, and use the optimized value to calibrate the parameters of the combined application sensor.

[0200] In one embodiment, the calibration device for the combined application sensor calibration parameter may further include:

[0201] The entry detection unit is used for detecting a carrier entering a preset calibration area from an external device end. When it is detected that a carrier enters the preset calibration area, the calibration instruction is sent to the corresponding carrier system.

[0202] The present application also provides an electronic device, which may include:

[0203] one or more processors; and

[0204] A memory associated with the one or more processors, the memory being used to store program instructions, wherein when the program instructions are read and executed by the one or more processors, the following operations are performed:

[0205] In the process of moving the combined application sensor, a static environment object included in the natural environment is determined as a target environment object; the combined application sensor includes a first sensor and a second sensor;

[0206] Measuring the target environment object by using the first sensor in a plurality of postures to determine first position information of the target environment object relative to the first sensor;

[0207] Determine the target optimization parameters and each group of values ​​of the target optimization parameters, convert the relative position information of the first position information, and obtain the second position information corresponding to each group of target optimization parameter values ​​in a unified coordinate system;

[0208] Based on the position data of the target environment object at each posture in each of the second position information, an optimization value of the target optimization parameter is determined, and the optimization value is determined as the calibration parameter of the first sensor and the second sensor.

[0209] in, Figure 8 The architecture of an electronic device is exemplarily shown. For example, device 800 may be a mobile phone, a computer, a digital broadcast terminal, a messaging device, a game console, a tablet device, a medical device, a fitness device, a personal digital assistant, an aircraft, etc.

[0210] Reference Figure 8 , device 800 may include one or more of the following components: a processing component 802 , a memory 804 , a power component 806 , a multimedia component 808 , an audio component 810 , an input / output (I / O) interface 812 , a sensor component 814 , and a communication component 816 .

[0211] The processing component 802 generally controls the overall operation of the device 800, such as operations associated with display, phone calls, data communications, camera operations, and recording operations. The processing component 802 may include one or more processors 820 to execute instructions to complete the video playback method provided by the technical solution of the present disclosure, when the preset conditions are met, a traffic compression request is generated and sent to the server, wherein the traffic compression request records information for triggering the server to obtain the target focus area, and the traffic compression request is used to request the server to prioritize the bit rate of the video content in the target focus area; according to the bit stream file returned by the server, the video content corresponding to the bit stream file is played, wherein the bit stream file is the whole or part of the steps of the video file obtained by the server performing bit rate compression processing on the video content outside the target focus area according to the traffic compression request. In addition, the processing component 802 may include one or more modules to facilitate the interaction between the processing component 802 and other components. For example, the processing component 802 may include a multimedia module to facilitate the interaction between the multimedia component 808 and the processing component 802.

[0212] The memory 804 is configured to store various types of data to support operations on the device 800. Examples of such data include instructions for any application or method operating on the device 800, contact data, phone book data, messages, pictures, videos, etc. The memory 804 can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic memory, flash memory, magnetic disk or optical disk.

[0213] The power supply component 806 provides power to the various components of the device 800. The power supply component 806 can include a power management system, one or more power supplies, and other components associated with generating, managing, and distributing power for the device 800.

[0214] The multimedia component 808 includes a screen that provides an output interface between the device 800 and the user. In some embodiments, the screen may include a liquid crystal display (LCD) and a touch panel (TP). If the screen includes a touch panel, the screen may be implemented as a touch screen to receive input signals from the user. The touch panel includes one or more touch sensors to sense touch, slide, and gestures on the touch panel. The touch sensor may not only sense the boundaries of the touch or slide action, but also detect the duration and pressure associated with the touch or slide operation. In some embodiments, the multimedia component 808 includes a front camera and / or a rear camera. When the device 800 is in an operating mode, such as a shooting mode or a video mode, the front camera and / or the rear camera may receive external multimedia data. Each front camera and rear camera may be a fixed optical lens system or have a focal length and optical zoom capability.

[0215] The audio component 810 is configured to output and / or input audio signals. For example, the audio component 810 includes a microphone (MIC), and when the device 800 is in an operating mode, such as a call mode, a recording mode, and a speech recognition mode, the microphone is configured to receive an external audio signal. The received audio signal can be further stored in the memory 804 or sent via the communication component 816. In some embodiments, the audio component 810 also includes a speaker for outputting audio signals.

[0216] I / O interface 812 provides an interface between processing component 802 and peripheral interface modules, such as keyboards, click wheels, buttons, etc. These buttons may include but are not limited to: home button, volume button, start button, and lock button.

[0217] The sensor assembly 814 includes one or more sensors for providing various aspects of status assessment for the device 800. For example, the sensor assembly 814 can detect the open / closed state of the device 800, the relative positioning of components, such as the display and keypad of the device 800, and the sensor assembly 814 can also detect the position change of the device 800 or a component of the device 800, the presence or absence of user contact with the device 800, the orientation or acceleration / deceleration of the device 800, and the temperature change of the device 800. The sensor assembly 814 may include a proximity sensor configured to detect the presence of nearby objects without any physical contact. The sensor assembly 814 may also include a light sensor, such as a CMOS or CCD image sensor, for use in imaging applications. In some embodiments, the sensor assembly 814 may also include an accelerometer, a gyroscope sensor, a magnetic sensor, a pressure sensor, or a temperature sensor.

[0218] The communication component 816 is configured to facilitate wired or wireless communication between the device 800 and other devices. The device 800 can access a wireless network based on a communication standard, such as WiFi, 2G or 3G, or a combination thereof. In an exemplary embodiment, the communication component 816 receives a broadcast signal or broadcast-related information from an external broadcast management system via a broadcast channel. In an exemplary embodiment, the communication component 816 also includes a near field communication (NFC) module to facilitate short-range communication. For example, the NFC module can be implemented based on radio frequency identification (RFID) technology, infrared data association (IrDA) technology, ultra-wideband (UWB) technology, Bluetooth (BT) technology and other technologies.

[0219] In an exemplary embodiment, the device 800 may be implemented by one or more application-specific integrated circuits (ASICs), digital signal processors (DSPs), digital signal processing devices (DSPDs), programmable logic devices (PLDs), field programmable gate arrays (FPGAs), controllers, microcontrollers, microprocessors, or other electronic components to perform the above methods.

[0220] In an exemplary embodiment, a non-temporary computer-readable storage medium including instructions is also provided, such as a memory 804 including instructions, and the above instructions can be executed by the processor 820 of the device 800 to complete the video playback method provided by the technical solution of the present disclosure, when the preset conditions are met, a traffic compression request is generated and sent to the server, wherein the traffic compression request records information for triggering the server to obtain the target focus area, and the traffic compression request is used to request the server to give priority to guaranteeing the bit rate of the video content in the target focus area; according to the bit stream file returned by the server, the video content corresponding to the bit stream file is played, wherein the bit stream file is a video file obtained by the server performing bit rate compression processing on the video content outside the target focus area according to the traffic compression request. For example, the non-temporary computer-readable storage medium can be a ROM, a random access memory (RAM), a CD-ROM, a tape, a floppy disk, an optical data storage device, etc.

[0221] The present application also provides another electronic device, which may include:

[0222] one or more processors; and

[0223] A memory associated with the one or more processors, the memory being used to store program instructions, wherein when the program instructions are read and executed by the one or more processors, the following operations are performed:

[0224] Receiving an instruction from an external device to calibrate parameters of a sensor used in a carrier;

[0225] Determine target optimization parameters and each group of values ​​of the target optimization parameters;

[0226] Determine a target environmental object, and during the movement of the carrier, measure the target environmental object by using the combined application sensor based on each set of values ​​of the target optimization parameter to obtain measurement data;

[0227] According to the measured data, an optimized value of a target optimized parameter is determined, and the optimized value is determined as a calibration parameter of the combined application sensor.

[0228] An embodiment of the present application provides another electronic device, which may include:

[0229] one or more processors; and

[0230] A memory associated with the one or more processors, the memory being used to store program instructions, wherein when the program instructions are read and executed by the one or more processors, the following operations are performed:

[0231] The external device sends a command to the carrier system to calibrate the parameters of the combined application sensor in the carrier;

[0232] After receiving the instruction, the carrier system determines the target optimization parameters and each group of values ​​of the target optimization parameters; determines the target environmental object, and during the movement of the carrier, based on the each group of values ​​of the target optimization parameters, measures the target environmental object through the combined application sensor to obtain measurement data; determines the optimized value of the target optimization parameter based on the measurement data, and uses the optimized value to calibrate the parameters of the combined application sensor.

[0233] It can be known from the description of the above implementation methods that those skilled in the art can clearly understand that the present application can be implemented by means of software plus a necessary general hardware platform. Based on such an understanding, the technical solution of the present application can be essentially or partly contributed to the prior art in the form of a software product, which can be stored in a storage medium such as ROM / RAM, a magnetic disk, an optical disk, etc., and includes several instructions for enabling a computer device (which can be a personal computer, a server, or a network device, etc.) to execute the methods described in the various embodiments of the present application or certain parts of the embodiments.

[0234] Each embodiment in this specification is described in a progressive manner, and the same or similar parts between the embodiments can refer to each other, and each embodiment focuses on the differences from other embodiments. In particular, for the system or system embodiment, since it is basically similar to the method embodiment, the description is relatively simple, and the relevant parts can refer to the partial description of the method embodiment. The system and system embodiments described above are merely schematic, wherein the units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, they may be located in one place, or they may be distributed on multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the scheme of this embodiment. Ordinary technicians in this field can understand and implement it without creative work.

[0235] The above is a detailed introduction to the calibration method and device for the joint application sensor parameters provided by the present application. This article uses specific examples to illustrate the principles and implementation methods of the present application. The description of the above embodiments is only used to help understand the method and core ideas of the present application. At the same time, for those skilled in the art, according to the ideas of the present application, there will be changes in the specific implementation methods and application scope. In summary, the content of this specification should not be understood as limiting the present application.

Claims

1. A method for calibrating sensor parameters by joint application, characterized in that: include: After an autonomous driving vehicle carrying a combined application sensor starts a sensor calibration task while traveling on a road, a static environmental object included in the natural environment is determined as a target environmental object; the combined application sensor includes a first sensor for measuring environmental object information and a second sensor for providing a coordinate system conversion reference value; the static environmental object includes a building or signboard-type object existing in the road environment, or an object specially provided for sensor calibration parameter purposes on a road section where static objects suitable as target environmental objects are not easily found; when determining a static marker, first speed information of the detected object relative to the first sensor is obtained by measuring with the first sensor, second speed information of the first sensor is obtained by the positioning information provided by the second sensor, actual speed information of the detected object is determined based on the first speed and the second speed, and a static marker is determined based on the actual speed information; Measuring the target environment object by using the first sensor in a plurality of postures to determine first position information of the target environment object relative to the first sensor; Determine the target optimization parameters and each group of target optimization parameter values, and use the coordinate system conversion reference value provided by the second sensor to convert the relative position information of the first position information to obtain the second position information corresponding to each group of target optimization parameter values ​​in the unified coordinate system; Taking the position data of the target environment object in each posture in each second position information as a sample, search for the optimized value of each target optimization parameter when the position points overlap or the overlap degree is higher than a certain threshold, and determine the optimized value as the calibration parameter between the first sensor and the second sensor.

2. The method according to claim 1, characterized in that The second speed information includes: The movement speed of the first sensor is determined as second speed information according to the positioning information provided by the second sensor.

3. The method according to claim 1, characterized in that The determining of the target optimization parameters and the values ​​of each group of the target optimization parameters, and converting the relative position information of the first position information, includes: Determine the target optimization parameters and each set of values ​​of the target optimization parameters, and transform the relative position information of the first position information according to the coordinate system conversion reference value and each set of values ​​of the target optimization parameters.

4. The method according to claim 1, characterized in that: The determining first position information of the target environment object relative to the first sensor includes: Determine point cloud data obtained by measuring the target environment object at each posture by the first sensor; The point cloud data obtained at each posture are clustered respectively, and the first position information of the target environment object at each posture is determined according to the clustering results.

5. The method according to claim 4, characterized in that Also includes: Before the clustering process, feature matching is performed on the point cloud data of the target environment object according to parameter information related to the point cloud data obtained by the first sensor, and points in the point cloud data that do not belong to the target environment object are filtered out.

6. The method according to claim 4, characterized in that The parameter information related to the point cloud data includes posture information, first sensor reflection cross-section information and / or Doppler state information.

7. The method according to claim 1, characterized in that Determining the target optimization parameters includes: Determine a spatial parameter between the first sensor and the second sensor as a target optimization parameter; wherein the spatial parameter includes a rotation parameter and a translation parameter between the first sensor and the second sensor.

8. The method according to claim 1, characterized in that Determining the target optimization parameters includes: A time delay parameter between the first sensor and the second sensor is determined as a target optimization parameter.

9. The method according to claim 8, characterized in that The second sensor includes an inertial measurement unit, and the relative position information of the first position information is converted to obtain the second position information corresponding to each group of target optimization parameter values ​​in a unified coordinate system, including: Obtaining position compensation amounts of relative position information at each position in the first position information according to the time delay parameter value of each set of target optimization parameter values, the angular rate information output by the inertial measurement unit, and the speed and angle information output by the first sensor; The second position information in the unified coordinate system is determined according to the corresponding posture matrix output by the second sensor in each posture, the translation and rotation parameter values ​​between the first sensor and the second sensor, the position information of the second sensor in the unified coordinate system, and the position compensation amount.

10. The method according to any one of claims 1 to 9, characterized in that: Each set of values ​​of the target optimization parameters corresponds to a second position information, and the position data of the target environment object in each position in each of the second position information is used as a sample to search for the optimized values ​​of each target optimization parameter when the position points coincide or the coincidence degree is higher than a certain threshold, including: Input each of the second position information into a preset cost function respectively to determine the function value corresponding to the value of each group of target optimization parameters; wherein each of the second position information includes the position data of each target environment object in a unified coordinate system at each posture; The minimum value among the function values ​​is determined, and the value of the target optimization parameter corresponding to the minimum value is determined as the optimized value.

11. The method according to any one of claims 1 to 9, characterized in that: The step of determining a static environment object included in the natural environment as a target environment object includes: Two or more static environmental objects included in the natural environment are determined as target environmental objects.

12. The method according to any one of claims 1 to 9, characterized in that: The first sensor includes a millimeter wave radar, and the second sensor includes an inertial measurement unit.

13. The method according to any one of claims 1 to 9, characterized in that: Also includes: Determine instructions for calibrating parameters of the combined application sensor; After receiving the instruction to calibrate, the calibration task of the combined application sensor parameters is started.

14. The method according to claim 13, characterized in that The instruction for determining to calibrate the parameters of the combined application sensor includes: An instruction for calibrating the parameters of the combined application sensor is determined according to a preset time.

15. The method according to claim 13, characterized in that The instruction for determining to calibrate the parameters of the combined application sensor includes: When the vehicle mileage count reaches a specific value or value range, an instruction to calibrate the combined application sensor parameters is determined.

16. The method according to claim 13, characterized in that The instruction for determining to calibrate the parameters of the combined application sensor includes: Instructions for calibrating the parameters of the combined application sensor are determined based on information about the degree of bumpiness during the movement of the carrier.

17. The method according to claim 13, characterized in that The instruction for determining to calibrate the parameters of the combined application sensor includes: A human-computer interaction interface is provided, and instructions for calibrating the parameters of the combined application sensor are determined according to the user's operation on the human-computer interaction interface.

18. The method according to claim 13, characterized in that The instruction for determining to calibrate the parameters of the combined application sensor includes: The parameters of the combined application sensor are verified, and instructions for calibrating the parameters of the combined application sensor are determined according to the verification results.

19. A method for calibrating joint application sensor parameters, characterized in that: include When an autonomous driving vehicle equipped with a combined application sensor is traveling on a road, an instruction for calibrating parameters of the combined application sensor in the autonomous driving vehicle is received from an external device; the combined application sensor includes a first sensor for measuring environmental object information and a second sensor for providing a coordinate system conversion reference value; Determine target optimization parameters and each group of values ​​of the target optimization parameters; Determine a target environment object. During the traveling of the autonomous driving vehicle on the road, based on each group of values ​​of the target optimization parameter, the target environment object is measured by the combined application sensor to obtain measurement data. The target environment object includes a static environment object included in the natural environment. The static environment object includes a building or signboard-type object existing in the road environment, or an object specially provided for sensor calibration parameter purposes on a road section where static objects suitable as target environment objects are not easily found. When determining a static marker, first speed information of the detected object relative to the first sensor is obtained by measuring with the first sensor, second speed information of the first sensor is obtained by positioning information provided by the second sensor, actual speed information of the detected object is determined based on the first speed and the second speed, and a static marker is determined based on the actual speed information. According to the measured data, an optimized value of a target optimized parameter is determined, and the optimized value is determined as a calibration parameter of the combined application sensor.

20. The method according to claim 19, characterized in that Also includes: The determined calibration parameters are used to calibrate the parameters of the combined application sensor, and after the calibration is completed, calibration completion information is sent to the external device end.

21. The method according to claim 20, characterized in that Also includes: Receive an action instruction sent by the external device and determined according to the calibration completion information.

22. The method according to claim 21, characterized in that The receiving the action instruction sent by the external device and determined according to the calibration completion information includes: Receive the permission action instruction sent by the external device end according to the calibration completion information.

23. The method according to claim 22, characterized in that The passing permission action instruction includes: a permission instruction for allowing passing through a preset detection area, or a permission instruction for allowing entering a preset road.

24. The method according to claim 19, characterized in that Also includes: Adjust the current speed of the carrier to the preset speed limit value.

25. According to the method according to any one of claims 19-24, the external device includes roadside equipment in a smart road.

26. A method for verifying the calibration parameters of a combined sensor application, characterized in that: include: When an autonomous driving vehicle equipped with a combined application sensor is traveling on a road, an external device sends an instruction to the autonomous driving vehicle system to calibrate parameters of the combined application sensor in the autonomous driving vehicle; the combined application sensor includes a first sensor for measuring environmental object information and a second sensor for providing a coordinate system conversion reference value; After receiving the instruction, the autonomous driving vehicle system determines the target optimization parameters and each set of values ​​of the target optimization parameters; Determine a target environment object, and during the movement of the autonomous driving vehicle, measure the target environment object through the combined application sensor based on each group of values ​​of the target optimization parameter to obtain measurement data; determine the optimized value of the target optimization parameter according to the measurement data, and use the optimized value to calibrate the parameters of the combined application sensor; The target environment objects include static environment objects included in the natural environment; the static environment objects include buildings or signboard objects existing in the road environment, or objects specially set up for sensor calibration parameter purposes on road sections where static objects suitable as target environment objects are not easily found; when determining static markers, first speed information of the detection object relative to the first sensor is obtained by measuring with the first sensor, second speed information of the first sensor is obtained by positioning information provided by the second sensor, actual speed information of the detection object is determined based on the first speed and the second speed, and static markers are determined based on the actual speed information.

27. The method according to claim 26, characterized in that Also includes: The external device detects a carrier entering a preset calibration area, and when it is detected that a carrier enters the preset calibration area, the calibration instruction is sent to the corresponding carrier system.

28. A calibration device for joint application of sensor calibration parameters, characterized in that: include: A static object determination unit is used to determine static environmental objects included in the natural environment as target environmental objects after the autonomous driving vehicle carrying a combined application sensor starts a sensor calibration task while traveling on the road; the combined application sensor includes a first sensor for measuring environmental object information and a second sensor for providing a coordinate system conversion reference value; the static environmental objects include buildings or signboard-type objects existing in the road environment, or objects specially provided for sensor calibration parameter purposes on sections of the road where static objects suitable as target environmental objects are not easily found; when determining static markers, first speed information of the detected object relative to the first sensor is obtained by measuring with the first sensor, second speed information of the first sensor is obtained by positioning information provided by the second sensor, actual speed information of the detected object is determined based on the first speed and the second speed, and static markers are determined based on the actual speed information; A position measuring unit, configured to measure the target environment object by using the first sensor in a plurality of positions and postures, and determine first position information of the target environment object relative to the first sensor; a position conversion unit, used to determine the target optimization parameters and each group of values ​​of the target optimization parameters, and to convert the relative position information of the first position information using the coordinate system conversion reference value provided by the second sensor to obtain second position information corresponding to each group of target optimization parameter values ​​in a unified coordinate system; A parameter determination unit is used to use the position data of the target environment object in each posture in each second position information as a sample, search for the optimized value of each target optimization parameter when the position points overlap or the overlap degree is higher than a certain threshold, and determine the optimized value as the calibration parameter between the first sensor and the second sensor.

29. A calibration device for joint application of sensor calibration parameters, characterized in that: include: A calibration instruction receiving unit, used for receiving an instruction issued by an external device to calibrate parameters of a combined application sensor in an autonomous driving vehicle while the autonomous driving vehicle carrying the combined application sensor is traveling on a road; the combined application sensor includes a first sensor for measuring environmental object information and a second sensor for providing a coordinate system conversion reference value; A parameter value determination unit, used to determine target optimization parameters and each group of values ​​of the target optimization parameters; a measurement data determination unit, for determining a target environment object, wherein during the traveling of the autonomous driving vehicle on the road, the target environment object is measured by the combined application sensor based on each group of values ​​of the target optimization parameter to obtain measurement data; the target environment object includes a static environment object included in the natural environment; the static environment object includes a building or signboard-type object existing in the road environment, or an object specially provided for sensor calibration parameter purposes on a road section where static objects suitable as target environment objects are not easily found; when determining a static marker, first speed information of the detected object relative to the first sensor is obtained by measuring with the first sensor, second speed information of the first sensor is obtained by positioning information provided by the second sensor, actual speed information of the detected object is determined based on the first speed and the second speed, and a static marker is determined based on the actual speed information; The optimization parameter determination unit is used to determine the optimization value of the target optimization parameter according to the measurement data, and determine the optimization value as the calibration parameter of the joint application sensor.

30. A calibration device for joint application of sensor calibration parameters, characterized in that: include: a calibration instruction issuing unit, configured to issue an instruction to calibrate the parameters of the combined application sensor in the autonomous driving vehicle from an external device end to the autonomous driving vehicle system when the autonomous driving vehicle carrying the combined application sensor is traveling on a road; the combined application sensor includes a first sensor and a second sensor; A calibration processing unit, configured to determine target optimization parameters and sets of values ​​of the target optimization parameters after the autonomous driving vehicle system receives the instruction; Determine a target environment object, and during the movement of the autonomous driving vehicle, measure the target environment object through the combined application sensor based on each group of values ​​of the target optimization parameter to obtain measurement data; determine the optimized value of the target optimization parameter according to the measurement data, and use the optimized value to calibrate the parameters of the combined application sensor; The target environment objects include static environment objects included in the natural environment; the static environment objects include buildings or signboard objects existing in the road environment, or objects specially set up for sensor calibration parameter purposes on road sections where static objects suitable as target environment objects are not easily found; when determining static markers, first speed information of the detection object relative to the first sensor is obtained by measuring with the first sensor, second speed information of the first sensor is obtained by positioning information provided by the second sensor, actual speed information of the detection object is determined based on the first speed and the second speed, and static markers are determined based on the actual speed information.

31. An electronic device, characterized in that: include: one or more processors; as well as A memory associated with the one or more processors, the memory being used to store program instructions, wherein the program instructions, when read and executed by the one or more processors, execute the steps of the method as claimed in any one of claims 1 to 27.

Images