Calibration method, calibration system, computer equipment and storage medium

Abstract

The invention relates to a calibration method, a calibration system, computer equipment and a storage medium, and the method comprises the steps: obtaining an initial calibration matrix, and first position information and second position information of a target object, the first position information being obtained according to first sensing data, and the second position information being obtained according to second sensing data; the first position information and the second position information are converted to the same coordinate system according to the calibration matrix, and the first position information and the second position information are matched; and adjusting parameters in the calibration matrix until a loss function between the first position information and the second position information which are matched with each other in the same coordinate system meets a convergence condition. By adopting the method, the problem of weak universality of a visual calibration method can be solved.

Description

Technical Field

[0001] The present application relates to the field of calibration, and in particular to a calibration method, a calibration system, a computer device, and a storage medium. Background Art

[0002] With the continuous advancement of science and technology, multi-sensor perception fusion has become an important direction for the development of all industries to provide more accurate and comprehensive perception information. Multi-sensor perception fusion can give full play to the perception advantages of different sensors, and improve the perception capabilities of equipment and products equipped with multi-sensor perception fusion, thus becoming more competitive.

[0003] Traditional manual calibration methods can solve the problem of multi-sensor fusion calibration, but they require a lot of manual participation and a large workload. Automated calibration has become the main development direction of calibration, and the automated calibration methods in related technologies often need to rely on some specific environmental information, or require joint judgment of targets of different sensors to obtain accurate position correspondence, and then realize the solution of calibration relationship.

[0004] With regard to the problem of weak universality of visual calibration methods in related technologies, no effective solution has been proposed so far. Summary of the invention

[0005] Based on this, it is necessary to provide a calibration method, calibration system, computer equipment, and storage medium that can solve the problem of weak versatility of visual calibration methods in response to the above technical problems.

[0006] In a first aspect, a calibration method is provided in this embodiment, the method comprising: Acquire an initial calibration matrix, first position information and second position information of a target object, wherein the first position information is obtained according to first sensing data, and the second position information is obtained according to second sensing data; Converting the first position information and the second position information to the same coordinate system according to the calibration matrix, and matching the first position information and the second position information; The parameters in the calibration matrix are adjusted until the loss function between the first position information and the second position information that match each other in the same coordinate system meets a convergence condition.

[0007] In some embodiments, after matching the first location information with the second location information, the method further includes: The loss function is calculated based on a relative distance and / or relative speed between the first position information and the second position information that match each other.

[0008] In some embodiments, adjusting the parameters in the calibration matrix until a loss function matching the first position information and the second position information in the same coordinate system satisfies a convergence condition, and stopping adjusting the calibration matrix includes: Obtaining a particle swarm corresponding to the parameters of the calibration matrix; Establishing a search space for each particle in the particle swarm according to a value range of a parameter of the calibration matrix; In the search space, adjusting the weight of each particle in the particle swarm according to the loss function; When the loss function satisfies the convergence condition, the calibration matrix after parameter update is obtained according to the particle swarm after weight adjustment.

[0009] In some embodiments, obtaining an initial calibration matrix includes: Acquire an internal parameter matrix of a sensor corresponding to the first sensor data, wherein the internal parameter matrix is ​​constructed according to sensor information corresponding to the first sensor data and / or a first parameter selected within a first value range; Acquire an external parameter matrix of a sensor corresponding to the first sensing data, wherein the external parameter matrix is ​​constructed according to a second parameter within a second value range; The calibration matrix is ​​obtained by combining the intrinsic parameter matrix and the extrinsic parameter matrix.

[0010] In some embodiments, adjusting the parameters in the calibration matrix includes: In a case where the internal parameter matrix is ​​constructed based on sensor information corresponding to the first sensing data, adjusting parameters of the external parameter matrix; In a case where the internal parameter matrix is ​​constructed according to the first parameter, the parameters of the external parameter matrix and the internal parameter matrix are adjusted.

[0011] In some embodiments, the first sensor data is data collected by a camera, and the second sensor data is data collected by a radar.

[0012] In a second aspect, a calibration system is provided in this embodiment, the system comprising a first device, a second device and a processing device; wherein: The first device is used to obtain first sensor data; The second device is used to obtain second sensor data; The processing device is used to obtain an initial calibration matrix, first position information and second position information of a target object, wherein the first position information is obtained based on the first sensing data, and the second position information is obtained based on the second sensing data; convert the first position information and the second position information to the same coordinate system according to the calibration matrix, and match the first position information and the second position information; adjust the parameters in the calibration matrix until the loss function between the first position information and the second position information that match each other in the same coordinate system meets the convergence condition.

[0013] In a third aspect, a computer device is provided in this embodiment, including a memory and a processor, wherein the memory stores a computer program, and the processor implements the calibration method described in the first aspect when executing the computer program.

[0014] In a fourth aspect, a computer-readable storage medium is provided in this embodiment, on which a computer program is stored. When the computer program is executed by a processor, the calibration method described in the first aspect is implemented.

[0015] In a fifth aspect, a computer program product is provided in this embodiment, including a computer program. When the computer program is executed by a processor, the calibration method described in the first aspect is implemented.

[0016] The above-mentioned calibration method, calibration system, computer equipment, and storage medium modify the parameters of the initial calibration matrix according to the loss function between the first position information and the second position information, and convert the calibration process into a process of solving the parameters in the calibration matrix. It can adapt to various scenarios and various devices and solve the problem that the visual calibration method is not universal. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] Figure 1 An application environment diagram of a calibration method in an embodiment; Figure 2 A schematic diagram of a flow chart of a calibration method in an embodiment; Figure 3 is a schematic flow chart of an automatic calibration method in one embodiment; Figure 4 A schematic diagram of a radar vision device acquiring data in a traffic scene in another embodiment; Figure 5 is a structural block diagram of a calibration device in one embodiment; Figure 6 FIG. 4 is a diagram showing the internal structure of a computer device in one embodiment. DETAILED DESCRIPTION

[0018] In order to make the purpose, technical solution and advantages of the present application more clearly understood, the present application is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application.

[0019] The calibration method provided in the embodiment of the present application can be applied to Figure 1 In the application environment shown. Among them, the terminal 102 communicates with the server 104 through the network. The data storage system can store the data that the server 104 needs to process. The data storage system can be integrated on the server 104, or it can be placed on the cloud or other network servers. The first sensor data and the second sensor data are stored by the data storage system, and the calibration method is executed by the terminal to obtain a calibration matrix. Among them, the terminal 102 can be but not limited to various personal computers, laptops, smart phones, and tablet computers. The server 104 can be implemented with an independent server or a server cluster consisting of multiple servers.

[0020] In one embodiment, Figure 2 As shown, a calibration method is provided, which is applied to Figure 1 The terminal in is used as an example to illustrate, including the following steps: Step S202, obtaining an initial calibration matrix, first position information and second position information of the target object, wherein the first position information is obtained according to the first sensing data, and the second position information is obtained according to the second sensing data.

[0021] The calibration matrix is ​​a mapping matrix between the coordinate system of the first sensor data and the coordinate system of the second sensor data, and is used to realize the coordinate conversion between the first position information and the second position information. The initial calibration matrix can be obtained based on a general calibration formula.

[0022] The target object is one or more objects captured by the first sensor data and the second sensor data. The first sensor data and the second sensor data are data captured by different data acquisition devices at the same time. The different data acquisition devices can be radars and cameras, or two different sensors. The types of acquisition devices for acquiring the first sensor data and the second sensor data are not limited here.

[0023] The first position information and the second position information include the coordinates and motion state of the target object in the corresponding coordinate system. Optionally, one or more target objects are identified from the first sensing data and the second sensing data, and the first position information and the second position information are obtained according to the coordinates and motion states such as motion speed and motion direction of the target objects.

[0024] Step S204: convert the first position information and the second position information into the same coordinate system according to the calibration matrix, and match the first position information and the second position information.

[0025] Optionally, the first position information is transformed by a calibration matrix so that the transformed first position information is in the coordinate system where the second position information is located. Alternatively, the second position information is transformed by a calibration matrix so that the transformed second position information is in the coordinate system where the first position information is located.

[0026] Among them, multiple first position information and multiple second position information can be obtained. The multiple first position information can be the position information of multiple different target objects in the first sensing data; correspondingly, the multiple second position information is the position information of multiple different target objects in the second sensing data. Alternatively, the multiple first position information and multiple second position information can also be the position information of the same target object obtained by different data acquisition devices at different times.

[0027] The number of matching pairs of the mutually matching first position information and the second position information is greater than or equal to the number of parameters that need to be adjusted in the calibration matrix. For example, when there are 11 parameters that need to be solved in the calibration matrix, at least 11 pairs of mutually matching first position information and second position information need to be obtained. Optionally, the first position information and the second position information converted to the same coordinate system can be matched by matching algorithms such as the nearest neighbor algorithm and the Hungarian algorithm. By obtaining the matching relationship between the first position information and the second position information, it is ensured that the obtained loss function is calculated based on the two position information of the same target object at the same time, and at the same time, the calibration requirements of the table when there are multiple parameters to be adjusted in the calibration matrix are met.

[0028] Step S206, adjusting the parameters in the calibration matrix until the loss function between the first position information and the second position information that match each other in the same coordinate system meets the convergence condition, and then stopping adjusting the calibration matrix.

[0029] The first position information and the second position information are position information of the same target object captured by different data acquisition devices at the same time. The loss function is used to measure the difference between the first position information and the second position information that match each other. The convergence condition can be that the loss value of the loss function reaches the minimum value, or the number of iterations of the loss function reaches a specified number.

[0030] Optionally, a loss function between the first position information and the second position information is obtained, and the parameters in the calibration matrix are adjusted according to the loss value of the loss function. The first position information and the second position information are converted to the same coordinate system again according to the adjusted calibration matrix, and the steps are repeated: the loss function between the first position information and the second position information is obtained, and the parameters in the calibration matrix are adjusted according to the loss value of the loss function. When the loss value reaches the minimum value, or the loss value no longer decreases significantly, or the number of repeated iterations reaches a specified number, the adjustment of the calibration matrix is ​​stopped, and the calibration matrix obtained by the last adjustment is output.

[0031] The above calibration method modifies the parameters of the initial calibration matrix through the first position information and the second position information, and obtains the calibration matrix by solving the parameters without relying on the environmental information around the first sensor data and the second sensor data. The algorithm has strong versatility and can adapt to various scenarios and various devices.

[0032] In one embodiment, obtaining an initial calibration matrix includes: obtaining an intrinsic parameter matrix of a sensor corresponding to first sensor data, wherein the intrinsic parameter matrix is ​​constructed based on sensor information for collecting the first sensor data and / or a first parameter selected within a first value range; obtaining an extrinsic parameter matrix of the sensor corresponding to the first sensor data, wherein the extrinsic parameter matrix is ​​constructed based on a second parameter within a second value range; and obtaining a calibration matrix by combining the intrinsic parameter matrix and the extrinsic parameter matrix.

[0033] The calibration matrix includes an internal parameter matrix and an external parameter matrix. The internal parameter matrix is ​​used to describe the inherent properties of the sensor for acquiring the sensor data, and the external parameter matrix is ​​used to describe the correlation between the coordinate system where the first sensor data is located and the coordinate system where the second sensor data is located.

[0034] Optionally, when the sensor information of the first sensing data can be obtained, the parameters in one or more internal parameter matrices such as focal length, principal point coordinates, pixel offset angle, etc. are obtained based on the sensor information, and the internal parameter matrix is ​​constructed according to the template of the internal parameter matrix in the calibration matrix. When the sensor information of the first sensing data cannot be obtained or the sensor information of the first sensing data is inaccurate, the first parameter within the corresponding first preset value range is randomly selected, and the internal parameter matrix is ​​constructed according to the first parameter and the internal parameter matrix template. In the case where some parameters of the internal parameter matrix cannot be obtained, the first parameter can also be obtained by combining the sensor information of the first sensing data and the parameters selected within the first value range. Among them, the first value ranges corresponding to different first parameters can be set according to the physical properties or sensor characteristics of the first sensing data acquisition device, respectively, or can be set according to user needs; the first parameter can be data randomly selected within the first value range, or can be data selected by the user based on historical experience.

[0035] Optionally, a corresponding second value range is pre-set for different second parameters, and multiple second parameters are obtained within the second value range. The external parameter matrix is ​​obtained according to the template construction of the second parameter and the external parameter matrix in the calibration matrix. The second value range can be set according to the relative position relationship between the first sensor data and the second sensor data acquisition device or user needs; the second parameter can be data randomly selected within the second value range, or it can be data selected by the user based on historical experience.

[0036] In this embodiment, an initial calibration matrix is ​​constructed by acquiring parameters within a preset range, so that the first position information and the second position information can be transformed based on the initial calibration matrix. In this embodiment, an internal parameter matrix can also be constructed based on the sensor information of the first sensor data, thereby achieving the effect of reducing the number of parameters to be solved in the calibration matrix.

[0037] In one embodiment, adjusting the parameters in the calibration matrix includes: when the internal parameter matrix is ​​constructed based on the sensor information of collecting the first sensor data, adjusting the parameters of the external parameter matrix; when the internal parameter matrix is ​​constructed based on the first parameter, adjusting the parameters of the external parameter matrix and the internal parameter matrix.

[0038] Among them, the first parameters of the intrinsic parameter matrix include focal length, principal point coordinates, pixel offset angle, etc. The second parameters of the extrinsic parameter matrix include rotation parameters in the rotation matrix and translation vectors in the translation matrix. The rotation matrix is ​​a matrix in the intrinsic parameter matrix that indicates the relative rotation relationship between the coordinate system where the first sensor data is located and the coordinate system where the second sensor data is located; the translation matrix is ​​a matrix in the intrinsic parameter matrix that indicates the relative translation relationship between the coordinate system where the first sensor data is located and the coordinate system where the second sensor data is located.

[0039] Optionally, the parameters in the calibration matrix can be modified based on an optimization algorithm such as a particle swarm algorithm or a genetic algorithm, and the convergence of the loss function is taken as the goal of the optimization algorithm. The parameters in the calibration matrix are continuously updated iteratively by the optimization algorithm to solve the parameter values ​​of the calibration matrix. In this embodiment, according to the method of constructing the calibration matrix, the parameters to be adjusted in the calibration matrix are determined, the algorithm is simplified, and the calibration efficiency is improved.

[0040] In one embodiment, after matching the first position information with the second position information, the method further includes: calculating a loss function according to a relative distance and / or relative speed between the first position information and the second position information that match each other.

[0041] The coordinates and moving speed of the target object at the first position can be obtained according to the first position information; the coordinates and moving speed of the target object at the second position can be obtained according to the second position information. The relative distance is the distance between the first position and the second position converted to the same coordinate system. The relative speed is the speed difference between the first position and the second position converted to the same coordinate system.

[0042] Optionally, the Euclidean distance between the positions of the target objects in different data acquisition devices is obtained according to the first position information and the second position information that match each other, and the loss function is calculated based on the Euclidean distance. Alternatively, the loss function is calculated according to the speed difference of the target objects in different data acquisition devices. Alternatively, the loss function is calculated by combining the Euclidean distance and the speed difference, thereby supporting multi-objective optimization in the calibration process. Among them, the loss function can be constructed by using an existing method, and the construction method of the loss function is not limited here.

[0043] In this embodiment, when different visual perception data are obtained to capture the same target by matching the first position information and the second position information, different characteristics and information differences are presented due to their respective data characteristics and perception methods. The loss function is calculated based on the relative distance and / or relative speed in the difference, which can improve the accuracy of calibration.

[0044] In one embodiment, adjusting the parameters in the calibration matrix until the loss function between the first position information and the second position information meets the convergence condition, and stopping adjusting the calibration matrix, includes: obtaining a particle swarm corresponding to the parameters of the calibration matrix; establishing a search space for each particle in the particle swarm according to the value range of the parameters of the calibration matrix; adjusting the weight of each particle in the particle swarm according to the loss function within the search space; and obtaining a calibration matrix after updating the parameters according to the particle swarm after adjusting the weights when the loss function meets the convergence condition.

[0045] Among them, each particle in the particle swarm represents a potential solution for the parameters in the calibration matrix, and the number of parameters to be solved corresponds to the number of particles in the particle swarm. The search space is the value space of each parameter to be solved in the calibration matrix, and the search space is within the value range corresponding to each parameter. The weight of the particle is also called the inertia weight, which is used to limit the movement of the particle in the search space. Optionally, the loss value of the loss function provides a direction for the adjustment of the particle weight, so that the particle approaches the optimal solution of each parameter in the search space. If the loss function meets the convergence condition, the updated particle position can be obtained according to the weight of each particle in the particle swarm, and the particle weight is converted to obtain the parameter value in the calibration matrix.

[0046] In this embodiment, a particle swarm corresponding to the calibration matrix is ​​constructed, and the updated calibration matrix is ​​determined by adjusting the particle weights. The global search capability is strong and can be applied to various scenarios, thereby achieving the effect of improving the calibration accuracy and versatility.

[0047] Radar and video perception fusion is an important branch of multi-sensor perception fusion. In one embodiment, the first sensor data is data collected by a camera, and the second sensor data is data collected by a radar. Optionally, after obtaining the first sensor data, a target frame containing the target object in the video data is determined; and the first position information is obtained according to the position of the target frame. After obtaining the second sensor data, the target object is determined according to radar visual perception, and the second position information of the target object is obtained.

[0048] The fusion of radar and video perception can give full play to the characteristics of radar's long perception distance, accurate perception of position, speed and other characteristics, full-time operation, and less impact of weather. At the same time, it integrates the rich characteristics of video perception environment, making the equipment and products equipped with radar and video fusion have stronger perception capabilities and thus more competitive. Radar and video perception calibration is the basis of radar and video perception fusion, which aims to achieve spatial alignment of the perception data of the two. In this embodiment, by obtaining radar visual perception data and camera perception data, the accuracy and versatility of radar and video calibration can be improved, thereby improving the accuracy of radar and video fusion.

[0049] In one embodiment, Figure 3 A flow chart of another automatic calibration method is provided, such as Figure 3 As shown, the following steps are included: Step S301, constructing initial parameters according to a preset range of parameters.

[0050] The initial parameters are the first parameter and the second parameter in the initial calibration matrix. Optionally, the initial parameters include Z C 、f x 、f y , u0, v0, α, β, γ, dx, dy, dz. C 、f x 、f y , u0, v0 are the first parameters in the above embodiment, and α, β, γ, dx, dy, dz are the second parameters in the above embodiment. Within the value range corresponding to each initial parameter, a value is randomly selected as the initial value of the initial parameter. According to the selected value, an initial calibration matrix is ​​constructed based on a general calibration formula.

[0051] For example, the general calibration formula is as follows: ; is the internal parameter matrix of the sensor used to collect the first sensing data, is the external parameter matrix of the sensor used to collect the first sensing data.

[0052] Among them, Z C is the scale factor. u and v are the position information in the image coordinate system. f x 、f y , u0, v0 are inherent properties of the device collecting the first sensor data. R is a rotation matrix associated with the three rotation angles of pitch β, yaw α, and roll γ. T is a translation matrix associated with the three translation vectors dx, dy, and dz. W , Y W , Z W It is the position information of the target object in the coordinate system where the second sensing data is located.

[0053] Taking the application in radar vision calibration as an example, the first sensor data is the data collected by the camera, and the second sensor data is the data collected by the radar. Based on the camera calibration formula, the radar vision calibration process can be regarded as the process of solving the internal parameter matrix and the external parameter matrix. The parameters in the internal parameter matrix and the external parameter matrix correspond to the value range respectively. Therefore, the radar vision calibration problem can be converted into the problem of solving Z C 、f x 、f y The problem of finding the optimal solution for the 11 parameters u0, v0, α, β, γ, dx, dy, and dz in their respective corresponding spaces.

[0054] Furthermore, in the internal parameter matrix, Z C 、f x 、f y , u0, v0 are unique properties of the camera and are directly related to the relative installation relationship of the sensors for collecting the first sensor data and the second sensor data. When the image sensor information used to collect the first sensor data is known and the intrinsic parameter matrix can be calculated in combination with the information of the image sensor, the optimization and solution process of the above 11-dimensional space can be simplified, and only the parameters related to the extrinsic parameter matrix such as α, β, γ, dx, dy, and dz are solved.

[0055] Step S302: Acquire the first sensor data and the second sensor data, and establish a matching relationship between a plurality of target objects in the first sensor data and the second sensor data.

[0056] Optionally, a plurality of first position information of a plurality of target objects is obtained according to the first sensing data. A plurality of second position information of a plurality of target objects is obtained according to the second sensing data. The first position information is converted by an initial calibration matrix H, or the second position information is converted by an initial calibration matrix H to obtain a plurality of first position information Trg in the same coordinate system. V={P i V | i=1,⋯,n} and multiple second position information Trg R ={P i R | i=1,⋯,n}. Among them, P i R =[x i ,y i ], x, y are the first position information coordinates, P i V =[u i ,v i ], u and v are the coordinates of the second location information. Obtain the matching relationship RVpair between the plurality of first location information and the plurality of second location information. Among them, the matching relationship RVpair can be obtained by using the nearest neighbor, Hungarian algorithm, KM algorithm and other matching algorithms. pair .

[0057] For ease of understanding, Figure 4 A schematic diagram of how radar vision equipment acquires data in a traffic scenario is provided. Figure 4 As shown, the radar vision device perceives the same scene through radar and image sensor, and there is a certain correspondence between the environmental information perceived by the radar and image sensor respectively. Figure 4 The left side of is the visual perception image information, and the video target in the lane uses the center point of the video target frame as the image position information of the video target. Correspondingly, the four vehicle target objects A, B, C, and D can be accurately detected by the visual detection algorithm, and the information of the first position information mentioned above is obtained: Trg V ={P i V | i=1,…,n}. Figure 4 The right side of is the radar perception information. Combined with the radar target detection algorithm, the four radar target objects 1, 2, 3, and 4 can also be accurately detected to obtain the second position information mentioned above: Trg R ={P i R | i=1,…,n}.

[0058] There is a matching relationship between the detection results of radar and vision RV pair That is, vehicle target object A corresponds to radar target object 1, vehicle target object B corresponds to radar target object 2, vehicle target object C corresponds to radar target object 3, and vehicle target object D corresponds to radar target object 4. In the matching relationship RV pair If there are enough, the internal and external parameter matrices in the initial calibration matrix can be accurately solved. For example, if there are 11 parameter requirements, at least 11 sets of matching relationships RV are required. pair .

[0059] Step S303, constructing a loss function.

[0060] Among them, based on the matching relationship RV pair Construct a loss function between the matched first position information and the second position information. Optionally, a suitable loss function can be constructed based on the characteristics of the matching relationship between the first position information and the second position information: the loss function can be constructed based on the distance between the first position information and the second position information, or the loss function can be constructed based on the distance between the first position information and the second position information, the speed of the target object at the first position information, and the speed of the target object at the second position information.

[0061] For example, during the radar-visual calibration process, the target objects in the lane are matched based on the initial calibration matrix, and the radar-visual matching relationship RV is used to match the target objects in the lane. pair Construct the loss function θ between the first position information and the second position information: ; Among them, D 2 (P R ,P V ) is the Euclidean distance between the second position information (radar target object) and the first position information (vehicle target object) corresponding to the radar-visual matching pair.

[0062] Step S304, solving the optimal parameters in the calibration matrix.

[0063] Among them, the optimal parameters in the calibration matrix can be solved by optimization algorithms such as particle swarm optimization and genetic algorithm. Optionally, based on the idea of ​​ICP (Iterative Closest Point) algorithm or NDT (Normal Distribution Transform) algorithm, after obtaining the matching relationship and constructing the loss function, the parameters in the internal parameter matrix and the external parameter matrix are used as unknown quantities, and the loss function is iterated using any of the following algorithms: SVD (Singular Value Decomposition) algorithm, gradient descent algorithm, quasi-Newton method algorithm, BFGS algorithm. Through the above loss function iteration algorithm, the iterated parameter values ​​are obtained and the internal parameter matrix and the external parameter matrix are calculated. The optimal parameter values ​​are solved by the iterative method to obtain the updated calibration matrix.

[0064] Optionally, by Z C 、f x 、f y, u0, v0, α, β, γ, dx, dy, dz constitute 11-dimensional particles, and obtain the state information of each particle. Based on the state of the particle, a calibration matrix is ​​constructed to obtain the matching relationship RV pair , the loss value is calculated according to the first position information and the second position information, and the loss value is used as the weight of the particle, and the weight is used to update the particle population. After iterative updating, the optimal particle of the population is obtained, thereby obtaining the updated calibration matrix.

[0065] Step S305: output the updated calibration matrix as the calibration result.

[0066] In this embodiment, the calibration process of the first sensor data and the second sensor data is converted into an optimization process in a multi-dimensional space, which can realize online automatic calibration without human intervention; and the calibration accuracy is high, which is applicable to calibration requirements of various types and different scenarios.

[0067] Based on the same inventive concept, the embodiment of the present application also provides a calibration device for implementing the calibration method involved above. The implementation scheme for solving the problem provided by the device is similar to the implementation scheme recorded in the above method, so the specific limitations in one or more calibration device embodiments provided below can refer to the limitations on the calibration method above, and will not be repeated here.

[0068] In one embodiment, Figure 5 As shown, a calibration system is provided, the system includes a first device, a second device and a processing device; wherein the first device is used to obtain first sensor data; the second device is used to obtain second sensor data; the processing device is used to obtain an initial calibration matrix, first position information and second position information of the target object, wherein the first position information is obtained according to the first sensor data, and the second position information is obtained according to the second sensor data; the first position information and the second position information are converted to the same coordinate system according to the calibration matrix, and the first position information and the second position information are matched; the parameters in the calibration matrix are adjusted until the loss function between the first position information and the second position information that match each other in the same coordinate system meets the convergence condition.

[0069] Optionally, the processing device obtains an initial calibration matrix, including: obtaining an intrinsic parameter matrix of a sensor corresponding to first sensor data, wherein the intrinsic parameter matrix is ​​constructed based on sensor information corresponding to the first sensor data and / or a first parameter selected within a first value range; obtaining an extrinsic parameter matrix of the sensor corresponding to the first sensor data, wherein the extrinsic parameter matrix is ​​constructed based on a second parameter within a second value range; and obtaining a calibration matrix by combining the intrinsic parameter matrix and the extrinsic parameter matrix.

[0070] Optionally, the processing device adjusts the parameters in the calibration matrix, including: when the intrinsic parameter matrix is ​​constructed based on the sensor information corresponding to the first sensor data, adjusting the parameters of the extrinsic parameter matrix; when the intrinsic parameter matrix is ​​constructed according to the first parameter, adjusting the parameters of the extrinsic parameter matrix and the intrinsic parameter matrix. In some embodiments, the first sensor data is data collected by a camera, and the second sensor data is data collected by a radar.

[0071] In some of the embodiments, after the processing device matches the first position information with the second position information, the execution method further includes: calculating a loss function based on a relative distance and / or relative speed between the first position information and the second position information that match each other.

[0072] In some of the embodiments, the processing device adjusts the parameters in the calibration matrix until the loss function between the first position information and the second position information that match each other in the same coordinate system meets the convergence condition, and stops adjusting the calibration matrix, including: obtaining a particle swarm corresponding to the parameters of the calibration matrix; establishing a search space for each particle in the particle swarm according to the value range of the parameters of the calibration matrix; within the search space, adjusting the weight of each particle in the particle swarm according to the loss function; when the loss function meets the convergence condition, obtaining a calibration matrix after updating the parameters according to the particle swarm after adjusting the weights.

[0073] Each module in the above calibration system can be implemented in whole or in part by software, hardware, or a combination thereof. Each module can be embedded in or independent of a processor in a computer device in the form of hardware, or can be stored in a memory in a computer device in the form of software, so that the processor can call and execute the operations corresponding to each module.

[0074] In one embodiment, a computer device is provided. The computer device may be a terminal, and its internal structure diagram may be as follows: Figure 6As shown. The computer device includes a processor, a memory, an input / output interface, a communication interface, a display unit and an input device. The processor, the memory and the input / output interface are connected through a system bus, and the communication interface, the display unit and the input device are connected to the system bus through the input / output interface. The processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system and a computer program. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The input / output interface of the computer device is used to exchange information between the processor and an external device. The communication interface of the computer device is used to communicate with an external terminal in a wired or wireless manner, and the wireless manner can be implemented through WIFI, a mobile cellular network, NFC (near field communication) or other technologies. When the computer program is executed by the processor, a calibration method is implemented. The display unit of the computer device is used to form a visually visible image, and may be a display screen or a projection device; the input device of the computer device may be a touch layer covered on the display screen, or may be a key, trackball or touchpad provided on the computer device housing, or may be an external keyboard, touchpad or mouse, etc.

[0075] Those skilled in the art will understand that Figure 6 The structure shown in the figure is only a block diagram of a part of the structure related to the solution of the present application, and does not constitute a limitation on the computer device to which the solution of the present application is applied. The specific computer device may include more or fewer components than those shown in the figure, or combine certain components, or have a different arrangement of components.

[0076] In one embodiment, a computer device is further provided, including a memory and a processor, wherein a computer program is stored in the memory, and the processor implements the steps in the above method embodiments when executing the computer program.

[0077] In one embodiment, a computer-readable storage medium is provided, on which a computer program is stored. When the computer program is executed by a processor, the steps in the above-mentioned method embodiments are implemented.

[0078] In one embodiment, a computer program product is provided, including a computer program, which implements the steps in the above method embodiments when executed by a processor.

[0079] Those skilled in the art can understand that all or part of the processes in the above-mentioned embodiment methods can be completed by instructing the relevant hardware through a computer program, and the computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to the memory, database or other medium used in the embodiments provided in this application can include at least one of non-volatile and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetoresistive random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. As an illustration and not limitation, RAM can be in various forms, such as static random access memory (SRAM) or dynamic random access memory (DRAM). The database involved in each embodiment provided in this application may include at least one of a relational database and a non-relational database. Non-relational databases may include distributed databases based on blockchains, etc., but are not limited to this. The processor involved in each embodiment provided in this application may be a general-purpose processor, a central processing unit, a graphics processor, a digital signal processor, a programmable logic device, a data processing logic device based on quantum computing, etc., but are not limited to this.

[0080] The technical features of the above embodiments may be combined arbitrarily. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0081] The above-described embodiments only express several implementation methods of the present application, and the descriptions thereof are relatively specific and detailed, but they cannot be understood as limiting the scope of the present application. It should be pointed out that, for a person of ordinary skill in the art, several variations and improvements can be made without departing from the concept of the present application, and these all belong to the protection scope of the present application. Therefore, the protection scope of the present application shall be subject to the attached claims.

Claims

1. A calibration method, characterized in that: The method comprises: Acquire an initial calibration matrix, first position information and second position information of a target object, wherein the first position information is obtained according to first sensing data, and the second position information is obtained according to second sensing data; Converting the first position information and the second position information to the same coordinate system according to the calibration matrix, and matching the first position information and the second position information; The parameters in the calibration matrix are adjusted based on the particle swarm algorithm until the loss function between the first position information and the second position information that match each other in the same coordinate system meets a convergence condition.

2. The calibration method according to claim 1, characterized in that: After matching the first location information with the second location information, the method further includes: The loss function is calculated based on a relative distance and / or relative speed between the first position information and the second position information that match each other.

3. The calibration method according to claim 1, characterized in that: The step of adjusting the parameters in the calibration matrix based on the particle swarm algorithm until a loss function between the first position information and the second position information that match each other in the same coordinate system meets a convergence condition, and stopping adjusting the calibration matrix, comprises: Obtaining a particle swarm corresponding to the parameters of the calibration matrix; Establishing a search space for each particle in the particle swarm according to a value range of a parameter of the calibration matrix; In the search space, adjusting the weight of each particle in the particle swarm according to the loss function; When the loss function satisfies the convergence condition, the calibration matrix after parameter update is obtained according to the particle swarm after weight adjustment.

4. The calibration method according to any one of claims 1 to 3, characterized in that: The obtaining of the initial calibration matrix comprises: Acquire an internal parameter matrix of a sensor corresponding to the first sensor data, wherein the internal parameter matrix is ​​constructed according to sensor information corresponding to the first sensor data and / or a first parameter selected within a first value range; Acquire an external parameter matrix of a sensor corresponding to the first sensing data, wherein the external parameter matrix is ​​constructed according to a second parameter within a second value range; The calibration matrix is ​​obtained by combining the intrinsic parameter matrix and the extrinsic parameter matrix.

5. The calibration method according to claim 4, characterized in that: The adjusting the parameters in the calibration matrix comprises: In a case where the internal parameter matrix is ​​constructed based on sensor information corresponding to the first sensing data, adjusting parameters of the external parameter matrix; In a case where the internal parameter matrix is ​​constructed according to the first parameter, the parameters of the external parameter matrix and the internal parameter matrix are adjusted.

6. The calibration method according to claim 1, characterized in that: The first sensor data is data collected by a camera, and the second sensor data is data collected by a radar.

7. A calibration system, characterized in that: The system comprises a first device, a second device and a processing device; wherein, The first device is used to obtain first sensor data; The second device is used to obtain second sensor data; The processing device is used to obtain an initial calibration matrix, first position information and second position information of a target object, wherein the first position information is obtained according to the first sensing data, and the second position information is obtained according to the second sensing data; the first position information and the second position information are converted to the same coordinate system according to the calibration matrix, and the first position information and the second position information are matched; the parameters in the calibration matrix are adjusted based on a particle swarm algorithm until a loss function between the first position information and the second position information that match each other in the same coordinate system meets a convergence condition.

8. A computer device comprising a memory and a processor, wherein the memory stores a computer program, wherein: When the processor executes the computer program, the steps of the method according to any one of claims 1 to 6 are implemented.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 6 are implemented.

10. A computer program product, comprising a computer program, characterized in that When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 6 are implemented.

Images