Multi-sensor joint calibration method, device and equipment and storage medium

Abstract

The invention provides a multi-sensor joint calibration method, device and equipment and a storage medium, and belongs to the technical field of vehicle environment perception detection, and the method comprises the steps: calibrating a first sensor and a first target point on a target plane, and obtaining a calibration parameter of the first sensor; a second sensor and a second target point on the target plane are calibrated, and calibration parameters of the second sensor are obtained; and correcting the calibration parameter of the second sensor according to the calibration parameter of the first sensor, the first target point and the second target point. According to the technical scheme provided by the embodiment of the invention, the accurate alignment of the data among the plurality of sensors in the unified coordinate system can be ensured, so that the overall calibration precision is improved, and meanwhile, the improvement of the calibration precision is also beneficial to reducing system faults and false alarm conditions caused by calibration errors.

Description

Technical Field

[0001] The present invention relates to the field of vehicle environment perception and detection technology, and in particular to a multi-sensor joint calibration method, device, equipment and storage medium. Background Art

[0002] Multi-sensor calibration refers to determining the spatial transformation relationship (rotation and translation) between different sensor coordinate systems in order to achieve data fusion, collaborative perception and system modeling. It is widely used in scenarios such as autonomous driving, robots, and drones. Currently, related intelligent driving functions, especially those mainly in high-level core functions, involve more sensors. Multi-sensor calibration is crucial for the operation of autonomous driving systems. It not only lays the foundation for subsequent mapping, positioning, perception and control, but also affects the driving status of the vehicle.

[0003] Sensor calibration is an essential part of the automotive manufacturing process. Time-consuming calibration directly extends production cycles, impacting vehicle manufacturing cycle times and reducing production efficiency. Calibration requires significant testing and human resources, and prolonged calibration can exacerbate resource constraints, especially during peak production periods.

[0004] Therefore, it is urgent to design a new calibration method that can improve the calibration efficiency while ensuring the accuracy of sensor calibration. Summary of the Invention

[0005] The present invention aims to solve at least one of the technical problems existing in the prior art and proposes a multi-sensor joint calibration method, device, equipment and storage medium.

[0006] In a first aspect, an embodiment of the present invention provides a multi-sensor joint calibration method, comprising:

[0007] Calibrate the first sensor with a first target point on the target plane to obtain calibration parameters of the first sensor;

[0008] Calibrate the second sensor with a second target point on the target plane to obtain calibration parameters of the second sensor;

[0009] The calibration parameters of the second sensor are calibrated according to the calibration parameters of the first sensor, the first target point, and the second target point.

[0010] In some embodiments, calibrating the first sensor with a first target point on a target plane to obtain calibration parameters of the first sensor includes:

[0011] Select three non-collinear random points to solve the planar function of the target plane;

[0012] Determine the normal vector based on the solved plane function and select the first target point;

[0013] Based on the normal vector and the first target point, solve the rotation matrix and translation matrix.

[0014] In some embodiments, the first target point satisfies coordinate parameters on the target plane and has a non-oblique positional relationship with the first sensor.

[0015] In some embodiments, the process of solving the plane function of the target plane includes:

[0016] Initialize the plane function of the target plane with respect to the Z plane;

[0017] Acquire three non-collinear random points on the target plane by a first sensor;

[0018] According to the random points obtained, the parameter coefficients of the plane function are solved.

[0019] In some embodiments, after solving the plane function of the target plane, the method further includes:

[0020] Selecting a plurality of random calibration points, and determining whether the plurality of random calibration points are on the plane function;

[0021] The random calibration points that are not on the plane function are deleted, and the random calibration points on the plane function are used as the first target point and the second target point.

[0022] In some embodiments, the process of determining whether a plurality of random calibration points are on the plane function includes:

[0023] Determine whether the sum of squares of the residuals between the random calibration points and the plane function is less than a first preset threshold;

[0024] Determine whether the derivative of the residual sum of squares with respect to the parameter coefficient of the plane function is less than a second preset threshold;

[0025] If the residual sum of squares is less than a preset threshold and the derivative approaches zero, the multiple random calibration points are determined to be on the plane function; otherwise, the multiple random calibration points are determined not to be on the plane function, and multiple random calibration points are reselected.

[0026] In some embodiments, after solving the plane function of the target plane, the method further includes: determining four corner points of the target plane.

[0027] In a second aspect, an embodiment of the present invention provides a multi-sensor joint calibration device, including:

[0028] a first acquisition module, configured to calibrate the first sensor with a first target point on a target plane, and acquire calibration parameters of the first sensor;

[0029] a second acquisition module, configured to calibrate the second sensor with a second target point on the target plane, and obtain calibration parameters of the second sensor;

[0030] A parameter correction module is used to correct the calibration parameters of the second sensor according to the calibration parameters of the first sensor, the first target point and the second target point.

[0031] In a third aspect, an embodiment of the present invention provides an electronic device, the electronic device including:

[0032] at least one processor; and a memory communicatively connected to the at least one processor; wherein,

[0033] The memory stores a computer program that can be executed by at least one processor. The computer program is executed by the at least one processor to enable the at least one processor to perform the steps of the method of any embodiment of the present invention.

[0034] In a fourth aspect, an embodiment of the present invention provides a computer-readable storage medium, wherein the computer-readable storage medium stores computer instructions, and the computer instructions are used to enable a processor to implement the steps of the method of any embodiment of the present invention when executed.

[0035] Compared with the prior art, the present invention has the following beneficial effects:

[0036] The multi-sensor joint calibration method provided by the present invention first calibrates a first sensor with a first target point on a target plane to obtain the calibration parameters of the first sensor. Then, a second sensor is calibrated with the first target point on the target plane to obtain the calibration parameters of the first sensor. Finally, the calibration parameters of the second sensor are corrected based on the calibration parameters of the first sensor, the first target point, and the second target point. The technical solution provided by the present invention enables calibration of all sensors in the same coordinate system, improving the overall calibration accuracy of the sensors. BRIEF DESCRIPTION OF THE DRAWINGS

[0037] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the following briefly introduces the drawings required for use in the description of the embodiments. Obviously, the drawings described below are only preferred embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0038] Figure 1 A schematic diagram of a multi-sensor joint calibration method provided in an embodiment of the present invention;

[0039] Figure 2 A front view of a target calibration plane provided by an embodiment of the present invention;

[0040] Figure 3 A side view of a target calibration plane provided by an embodiment of the present invention;

[0041] Figure 4 A structural block diagram of a multi-sensor joint calibration device provided by an embodiment of the present invention;

[0042] Figure 5 This is a structural block diagram of an electronic device provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0043] The following will clearly and completely describe the technical solutions in the embodiments of the present invention in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of the present invention.

[0044] To enable those skilled in the art to better understand the technical solutions of the present invention, exemplary embodiments of the present invention are described below in conjunction with the accompanying drawings, including various details of the embodiments of the present invention to facilitate understanding. These details should be considered merely exemplary. Therefore, those skilled in the art should recognize that various changes and modifications may be made to the embodiments described herein without departing from the scope and spirit of the present invention. Similarly, for the sake of clarity and conciseness, descriptions of well-known functions and structures are omitted in the following description.

[0045] In the absence of conflict, the various embodiments of the present invention and the various features therein may be combined with each other.

[0046] As used herein, the term "and / or" includes any and all combinations of one or more of the associated listed items.

[0047] The terms used herein are only used to describe specific embodiments and are not intended to limit the present invention. As used herein, the singular forms "a" and "the" are also intended to include the plural forms, unless the context clearly indicates otherwise. It will also be understood that when the terms "comprising" and / or "made of..." are used in this specification, the presence of the features, wholes, steps, operations, elements and / or components is specified, but the presence or addition of one or more other features, wholes, steps, operations, elements, components and / or groups thereof is not excluded. Similar words such as "connected" or "connected" are not limited to physical or mechanical connections, but can include electrical connections, whether direct or indirect.

[0048] Unless otherwise defined, all terms (including technical and scientific terms) used herein have the same meaning as commonly understood by one of ordinary skill in the art. It will also be understood that terms such as those defined in commonly used dictionaries should be interpreted as having a meaning consistent with their meaning in the context of the relevant art and the present invention, and will not be interpreted as having an idealized or overly formal meaning unless expressly defined as such herein.

[0049] The collection, storage, use, processing, transmission, provision, and disclosure of user personal information in the technical solution of this invention complies with relevant laws and regulations and does not violate public order and good morals. The use of user data in this technical solution complies with relevant national laws and regulations (e.g., the "Information Security Technology Personal Information Security Specification"). For example, appropriate measures are implemented to control access to personal information; the display of personal information is subject to prescribed restrictions; the purpose of using personal information does not exceed the scope of direct or reasonable relevance; and when using personal information, explicit identification is eliminated to avoid pinpointing specific individuals.

[0050] In related technologies, common multi-sensor calibration methods are mainly divided into two categories:

[0051] 1) Targetless calibration methods: This involves calibrating the spatiotemporal parameters of multiple integrated sensors without the use of artificial calibration targets. These methods are based on scene features, motion information, and other methods, enabling the fusion of multi-sensor data within a unified spatiotemporal framework. Some scene feature-based methods utilize natural features in the environment for calibration, eliminating the need for additional targets. Motion-based methods estimate calibration parameters using sensor motion data. However, different types of targetless calibration methods vary in accuracy, robustness, and computational complexity.

[0052] 2) Calibration with a target: This involves using a calibration plate, a flat plate with a fixed-pitch pattern array and a reflectivity that meets LiDAR requirements. This plate can be photographed by a camera or used by a radar for data collection. Common calibration methods include: Calibration of individual sensors. Calibration of automotive sensors is a complex and time-consuming process that requires specialized facilities, equipment, and personnel, and consumes significant time and labor costs. Calibration of multiple sensors against a large target, however, cannot cover all sensors, and when the calibration areas selected by multiple sensors do not overlap, calibration anomalies can result in irreversible errors.

[0053] In order to solve at least one of the technical problems existing in the above-mentioned related technologies, the present invention provides a multi-sensor joint calibration method. Figure 1A flowchart of a multi-sensor joint calibration method provided in an embodiment of the present invention is provided. The method can be performed by a multi-sensor joint calibration device, which can be implemented in software and / or hardware and can be configured in an electronic device.

[0054] like Figure 1 As shown, the method specifically includes:

[0055] S1, calibrating a first sensor with a first target point on a target plane to obtain calibration parameters of the first sensor.

[0056] It should be noted that common sensor calibration in automotive systems includes camera calibration and radar calibration. The first sensor is any one of multiple sensors in the automotive system. The first target points are multiple points on the target plane that assist in calibration of the first sensor, and the locations of these points do not cause sensor distortion.

[0057] The first sensor is calibrated with the first target point on the target plane, and the second sensor is calibrated with the second target point on the target plane. Then, the calibration coordinate system of the first sensor and the calibration coordinate system of the second sensor are unified through the conversion relationship between the calibration parameters, so that other sensors can be calibrated with points on the target plane in the same coordinate system.

[0058] Before calibrating the first sensor, you can first set the theoretical center point coordinates of the target plane. By identifying the mark on the target plane, determine the center point coordinate information. Set the center point coordinates to .

[0059] Figure 2 The front view of the target plane provided by the embodiment of the present invention is as follows: Figure 2 As shown, in the front view of the plane, there are 4 corner points, namely , , and .

[0060] Figure 3 A side view of the target plane provided by an embodiment of the present invention, such as Figure 3 As shown, in the side view of the plane, there are two corner points, namely and .

[0061] In fact, the theoretical value of the center point can be expressed as:

[0062]

[0063] By expressing the spatial position relationship of the target plane according to the above coordinate system, the subsequent calibration calculation process can be unified.

[0064] Before calibrating the target plane, in addition to setting the theoretical center point coordinates of the target plane, you can also establish the plane function of the target plane. A conventional three-dimensional plane can be expressed by a mathematical formula:

[0065]

[0066] Furthermore, it can be transformed into a mathematical expression about the Z plane:

[0067]

[0068] set up: ; ; .

[0069] Then the plane function of the target plane can be simplified as:

[0070]

[0071] Due to the inevitable characteristic of the calibration plate's fixed Z plane, the planar function of the target plane is converted into an expression related to x and y. Subsequently, it can be quickly determined whether the point is on the plane or converted on the plane based only on the x and y values ​​of the point.

[0072] In some embodiments, calibrating the first sensor with a first target point on a target plane to obtain calibration parameters of the first sensor specifically includes:

[0073] S110, selecting three non-collinear random points to solve the planar function of the target plane.

[0074] In some embodiments, the process of solving the plane function of the target plane specifically includes:

[0075] S1101, initializing the plane function of the target plane with respect to the Z plane.

[0076]

[0077] It should be noted that during the installation of the target plane, there may be physical installation errors (such as the calibration plate is not completely horizontal, the camera viewing angle deviation, etc.), so the parameters need to be adjusted. 、 and Optimize.

[0078] In extreme cases, if the target plane is placed horizontally, that is, the calibration plate is a horizontal chessboard, in the z=0 plane, then the parameter 、 , .

[0079] S1102. Obtain three non - collinear random points on the target plane through the first sensor.

[0080] A three - dimensional plane can be determined by the coordinates of at least three points. Thus, the random points selected by the first sensor are usually 3 non - collinear feature points (such as checkerboard corner points).

[0081] S1103. Solve the parameter coefficients in the plane function according to the obtained random points.

[0082] Specifically: construct a system of linear equations, substitute the coordinates of the obtained random points, and solve the corresponding parameter coefficients. The coordinate of any random point is .

[0083]

[0084] Equivalent to:

[0085]

[0086] where n is a constant, find .

[0087] In some embodiments, after solving the plane function of the target plane, it further includes:

[0088] S1104. Select multiple random calibration points and determine whether the multiple random calibration points are on the plane function.

[0089] For any point N recognizable by the first sensor, the first sensor obtains the parameter coordinates of this point, denoted as (i = 0, 1, …, n - 1, n≥3). Generally, for black - and - white checkerboards, any point N can also be regarded as any grid (the grid size is related to the calibration requirement error).

[0090] In some embodiments, selecting multiple random calibration points and determining whether the multiple random calibration points are on the plane function specifically includes:

[0091] S11041. Determine whether the sum of the squared residuals between the random calibration point and the plane function is less than the first preset threshold.

[0092]

[0093] The first preset threshold is set as M, such as the square of the physical size of the checkerboard. If S < M, it is determined that the random calibration point is on the plane; otherwise, it is discarded.

[0094] S11042. Determine whether the derivative of the sum of the squared residuals with respect to the parameter coefficients of the plane function is less than the second preset threshold.

[0095] The preset target is W. By comparing W with the second preset threshold, if W is less than the second preset threshold (theoretically W tends to zero), it indicates that the random calibration point is on the plane function. It is expressed as:

[0096]

[0097] S11043: If the residual sum of squares is less than a preset threshold and the derivative approaches zero, the multiple random calibration points are determined to be on the plane function; otherwise, the multiple random calibration points are determined not to be on the plane function, and multiple random calibration points are reselected.

[0098] S1105: Delete the random calibration points that are not on the plane function, and use the random calibration points on the plane function as the first target point and the second target point.

[0099] In some embodiments, after solving the plane function of the target plane, the method further includes: determining four corner points of the target plane.

[0100] Assuming the side length of the target plane is L, the coordinates of the four corner points of the target plane are:

[0101]

[0102] Then in the coordinate system, we have:

[0103]

[0104] Then we can obtain .

[0105] S120: Determine a normal vector according to the solved plane function, and select a first target point.

[0106] It should be noted that the first target point is a point that assists the first sensor in calibration. Its coordinate parameters must be on the target plane, and its positional relationship with the first sensor must not be oblique, so as to avoid calibration deviation caused by the calibration point being in the distortion area of ​​the first sensor.

[0107] Among them, the normal vector of the plane function is .

[0108] S130: Solve the rotation matrix and the translation matrix according to the normal vector and the first target point.

[0109] Through the previous article, we can solve the normal vector of the plane function as ;

[0110] The corresponding rotation angle is: ;

[0111] Then we can get the rotation and translation matrix, which is the calibration parameter of the first sensor:

[0112]

[0113] in, is the rotation matrix from the world coordinate system to the camera coordinate system, is the translation matrix from the world coordinate origin to the camera coordinate origin, which is the conventional expression of the calibration parameters. 、 Manufacturing deviations can then be corrected.

[0114] S2: Calibrate the second sensor with a second target point on the target plane to obtain calibration parameters of the second sensor.

[0115] It should be noted that the calibration process of the second sensor is the same as that of the first sensor. However, the first sensor is one sensor, while the second sensor can be one or more sensors, and the second sensor is calibrated using the first sensor as a reference.

[0116] Similarly, the second target point satisfies the coordinate parameters on the target plane and has a non-oblique position relationship with the second sensor.

[0117] Since the second sensor uses the second target point as the calibration point of the target plane, while the first sensor uses the first target point as the calibration point of the target plane, that is, the calibration points of the first sensor and the second sensor are different, resulting in different calibration systems. Therefore, the calibration parameters of the second sensor need to be corrected so that they are in the same coordinate system as the calibration system of the first sensor.

[0118] S3, calibrating the calibration parameters of the second sensor according to the calibration parameters of the first sensor, the first target point, and the second target point.

[0119] Assume that the first target point is That is the point calibrated by the main sensor, the second target point is That is, the point identified by the sensor:

[0120]

[0121] Finally, the calibration parameters of the second sensor can be calculated, and based on this, a joint calibration relationship between the first sensor and the second sensor is established.

[0122] The technical solution in the embodiment of the present invention helps to reduce direct labor costs and indirect costs (such as equipment occupation costs and time costs) in the calibration process through joint calibration by reducing the number of calibration times and simplifying the calibration process. Joint calibration can ensure that the data between multiple sensors are accurately aligned in a unified coordinate system, thereby improving the overall calibration accuracy. It is crucial to achieve high-precision environmental perception and vehicle positioning. Through accurate joint calibration, autonomous driving vehicles can better understand their surroundings and make quick and accurate decisions, thereby improving the safety and reliability of autonomous driving. At the same time, the improvement in calibration accuracy also helps to reduce system failures and false alarms caused by calibration errors. After the joint calibration is completed, subsequent sensor fusion and data processing will become simpler and more efficient. The data from all sensors have been aligned and calibrated in a unified coordinate system, and no additional conversion and adjustment work is required.

[0123] Based on the same inventive concept, an embodiment of the present invention further provides a multi-sensor joint calibration device. Figure 4 A structural block diagram of a multi-sensor joint calibration device provided by an embodiment of the present invention is shown as follows: Figure 4 As shown, the device specifically includes:

[0124] A first acquisition module 100 is configured to calibrate the first sensor with a first target point on a target plane to obtain calibration parameters of the first sensor;

[0125] A second acquisition module 200 is used to calibrate the second sensor with a second target point on the target plane to obtain calibration parameters of the second sensor;

[0126] The parameter correction module 300 is configured to correct the calibration parameters of the second sensor according to the calibration parameters of the first sensor, the first target point, and the second target point.

[0127] The technical solution in the embodiments of the present invention calibrates the joint calibration sensor with the target point on the target, then solves the target plane's function expression based on the target point's coordinates. Other sensors then select their own calibration points from the target plane and establish a relationship with the target plane. At this point, all sensors are calibrated in the same coordinate system as the target. This improves calibration efficiency while ensuring sensor calibration accuracy.

[0128] Based on the same inventive concept, an embodiment of the present invention further provides an electronic device. Figure 5 This is a structural block diagram of an electronic device provided by an embodiment of the present invention. Figure 5As shown, an embodiment of the present invention provides an electronic device including: one or more processors 101, a memory 102, and one or more I / O interfaces 103. The memory 102 stores one or more programs. When the one or more programs are executed by the one or more processors, the one or more processors implement any multi-sensor joint calibration method described in the above embodiments. The one or more I / O interfaces 103 are connected between the processor and the memory and are configured to implement information exchange between the processor and the memory.

[0129] Among them, the processor 101 is a device with data processing capabilities, including but not limited to a central processing unit (CPU); the memory 102 is a device with data storage capabilities, including but not limited to random access memory (RAM, more specifically such as SDRAM, DDR, etc.), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), and flash memory (FLASH); the I / O interface (read-write interface) 103 is connected between the processor 101 and the memory 102, and can realize information interaction between the processor 101 and the memory 102, including but not limited to a data bus (BUS).

[0130] In some embodiments, the processor 101 , the memory 102 , and the I / O interface 103 are connected to each other via a bus 104 , and further connected to other components of the computing device.

[0131] In some embodiments, the one or more processors 101 include a field programmable gate array.

[0132] Embodiments of the present invention also provide a computer-readable medium. The computer-readable medium stores a computer program, wherein, when executed by a processor, the program implements the steps of any of the multi-sensor joint calibration methods described in the above embodiments. The computer-readable storage medium can be volatile or non-volatile.

[0133] An embodiment of the present invention also provides a computer program product, including computer-readable code, or a non-volatile computer-readable storage medium carrying computer-readable code. When the computer-readable code runs in a processor of an electronic device, the processor in the electronic device executes the above-mentioned multi-sensor joint calibration method.

[0134] Those skilled in the art will appreciate that all or some of the steps, systems, and functional modules / units in the methods disclosed above may be implemented as software, firmware, hardware, or appropriate combinations thereof. In hardware implementations, the division between the functional modules / units mentioned in the above description does not necessarily correspond to the division of physical components; for example, a physical component may have multiple functions, or a function or step may be performed collaboratively by several physical components. Some or all of the physical components may be implemented as software executed by a processor, such as a central processing unit, a digital signal processor, or a microprocessor, or as hardware, or as an integrated circuit, such as an application-specific integrated circuit. Such software may be distributed on a computer-readable storage medium, which may include a computer storage medium (or non-transitory medium) and a communication medium (or transient medium).

[0135] As is known to those skilled in the art, the term computer storage media includes volatile and nonvolatile, removable and non-removable media implemented in any method or technology for storage of information, such as computer-readable program instructions, data structures, program modules, or other data. Computer storage media includes, but is not limited to, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM), static random access memory (SRAM), flash memory or other memory technology, portable compact disc read-only memory (CD-ROM), digital versatile disc (DVD) or other optical disk storage, magnetic cassettes, magnetic tape, magnetic disk storage or other magnetic storage devices, or any other medium that can be used to store the desired information and that can be accessed by a computer. Furthermore, as is known to those skilled in the art, communication media typically embodies computer-readable program instructions, data structures, program modules, or other data in a modulated data signal such as a carrier wave or other transport mechanism, and may include any information delivery media.

[0136] The computer-readable program instructions described herein can be downloaded from a computer-readable storage medium to each computing / processing device, or downloaded to an external computer or external storage device via a network, such as the Internet, a local area network, a wide area network, and / or a wireless network. The network can include copper transmission cables, fiber optic transmission, wireless transmission, routers, firewalls, switches, gateway computers, and / or edge servers. The network adapter card or network interface in each computing / processing device receives the computer-readable program instructions from the network and forwards the computer-readable program instructions to be stored in the computer-readable storage medium in each computing / processing device.

[0137] The computer program instructions for performing the operations of the present invention may be assembly instructions, instruction set architecture (ISA) instructions, machine instructions, machine-dependent instructions, microcode, firmware instructions, state setting data, or source code or object code written in any combination of one or more programming languages, including object-oriented programming languages ​​such as Smalltalk, C++, and conventional procedural programming languages ​​such as "C" or similar programming languages. The computer-readable program instructions may be executed entirely on the user's computer, partially on the user's computer, as a stand-alone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In the case of a remote computer, the remote computer may be connected to the user's computer via any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computer (e.g., via the Internet using an Internet service provider). In some embodiments, the state information of the computer-readable program instructions is used to personalize an electronic circuit, such as a programmable logic circuit, a field programmable gate array (FPGA), or a programmable logic array (PLA), so that the electronic circuit can execute the computer-readable program instructions, thereby implementing various aspects of the present invention.

[0138] The computer program product described herein may be implemented in hardware, software, or a combination thereof. In one embodiment, the computer program product is implemented as a computer storage medium. In another embodiment, the computer program product is implemented as a software product, such as a software development kit (SDK).

[0139] Various aspects of the present invention are described herein with reference to flowcharts and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the present invention. It should be understood that each block of the flowcharts and / or block diagrams, and combinations of blocks in the flowcharts and / or block diagrams, can be implemented by computer-readable program instructions.

[0140] These computer-readable program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing device, thereby producing a machine, so that when these instructions are executed by the processor of the computer or other programmable data processing device, a device is generated that implements the functions / actions specified in one or more blocks in the flowchart and / or block diagram. These computer-readable program instructions can also be stored in a computer-readable storage medium, where these instructions cause the computer, programmable data processing device, and / or other device to operate in a specific manner. Thus, the computer-readable medium storing the instructions comprises an article of manufacture that includes instructions for implementing various aspects of the functions / actions specified in one or more blocks in the flowchart and / or block diagram.

[0141] Computer-readable program instructions may also be loaded onto a computer, other programmable data processing apparatus, or other device so that a series of operational steps are performed on the computer, other programmable data processing apparatus, or other device to produce a computer-implemented process, thereby causing the instructions executed on the computer, other programmable data processing apparatus, or other device to implement the functions / actions specified in one or more blocks in the flowchart and / or block diagram.

[0142] The flow charts and block diagrams in the accompanying drawings show the possible architecture, functions and operations of the systems, methods and computer program products according to multiple embodiments of the present invention. In this regard, each box in the flow chart or block diagram can represent a part of a module, program segment or instruction, and the part of the module, program segment or instruction includes one or more executable instructions for realizing the logical function of the specification. In some alternative implementations, the functions marked in the box can also occur in a sequence different from that marked in the accompanying drawings. For example, two consecutive boxes can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the block diagram and / or flow chart, and the combination of the boxes in the block diagram and / or flow chart can be implemented with a dedicated hardware-based system that performs the function or action of the specification, or can be implemented with a combination of dedicated hardware and computer instructions.

[0143] Example embodiments have been disclosed herein, and although specific terms are employed, they are used and should be interpreted only in a general illustrative sense and not for purposes of limitation. In some instances, it will be apparent to those skilled in the art that, unless otherwise expressly indicated, features, characteristics, and / or elements described in conjunction with a particular embodiment may be used alone or in combination with features, characteristics, and / or elements described in conjunction with other embodiments. Therefore, it will be understood by those skilled in the art that various changes in form and detail may be made without departing from the scope of the invention as set forth in the appended claims.

Claims

1. A multi-sensor joint calibration method, characterized in that: include: Calibrate the first sensor with a first target point on the target plane to obtain calibration parameters of the first sensor; Calibrate the second sensor with a second target point on the target plane to obtain calibration parameters of the second sensor; The calibration parameters of the second sensor are calibrated according to the calibration parameters of the first sensor, the first target point, and the second target point.

2. The method according to claim 1, characterized in that The step of calibrating the first sensor and the first target point on the target plane to obtain calibration parameters of the first sensor includes: Select three non-collinear random points to solve the planar function of the target plane; Determine the normal vector based on the solved plane function and select the first target point; Based on the normal vector and the first target point, solve the rotation matrix and translation matrix.

3. The method according to claim 2, characterized in that The first target point satisfies the coordinate parameters on the target plane, and the position relationship with the first sensor is non-oblique.

4. The method according to claim 2, characterized in that The process of solving the planar function of the target plane includes: Initialize the plane function of the target plane with respect to the Z plane; Acquire three non-collinear random points on the target plane by a first sensor; According to the random points obtained, the parameter coefficients of the plane function are solved.

5. The method according to claim 2, characterized in that After solving the plane function of the target plane, it also includes: Selecting a plurality of random calibration points, and determining whether the plurality of random calibration points are on the plane function; The random calibration points that are not on the plane function are deleted, and the random calibration points on the plane function are used as the first target point and the second target point.

6. The method according to claim 5, characterized in that The process of determining whether the plurality of random calibration points are on the plane function includes: Determine whether the sum of squares of the residuals between the random calibration points and the plane function is less than a first preset threshold; Determine whether the derivative of the residual sum of squares with respect to the parameter coefficient of the plane function is less than a second preset threshold; If the residual sum of squares is less than a preset threshold and the derivative approaches zero, the multiple random calibration points are determined to be on the plane function; otherwise, the multiple random calibration points are determined not to be on the plane function, and multiple random calibration points are reselected.

7. The method according to claim 2, characterized in that After solving the plane function of the target plane, the method further includes: determining four corner points of the target plane.

8. A multi-sensor joint calibration device, characterized in that: The device is configured to implement the method according to any one of claims 1 to 7, and includes: a first acquisition module, configured to calibrate the first sensor with a first target point on a target plane, and acquire calibration parameters of the first sensor; a second acquisition module, configured to calibrate the second sensor with a second target point on the target plane, and obtain calibration parameters of the second sensor; A parameter correction module is used to correct the calibration parameters of the second sensor according to the calibration parameters of the first sensor, the first target point and the second target point.

9. An electronic device, characterized in that: The electronic device comprises: at least one processor; and a memory communicatively connected to the at least one processor; wherein, The memory stores a computer program executable by the at least one processor, and the computer program is executed by the at least one processor to enable the at least one processor to perform the steps of the method according to any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that The computer-readable storage medium stores computer instructions, and the computer instructions are used to enable a processor to implement the steps of the method according to any one of claims 1 to 7 when executed.

Images