Camera calibration method and device
By establishing the mapping relationship between camera parameters and calibration points and using the least squares method to fit, the problems of complex calculation and low positioning accuracy in the multi-camera calibration method are solved, and more efficient positioning accuracy and lower calculation complexity are achieved.
Patent Information
- Application Number
- CN202310636857.1
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2023-05-31
- Publication Date
- 2024-12-03
AI Technical Summary
The existing multi-camera calibration methods are complex in calculations and have low positioning accuracy, which cannot effectively improve the positioning accuracy of vehicles under vehicle-road collaboration platform.
By obtaining the camera parameters and modeling parameters of multiple cameras, the mapping relationship between the camera parameters and calibration points is established, and the modeling parameters and the target object position are fitted using the least squares method to obtain the fitting function of the optimization target as the positioning accuracy. And calibrate multiple cameras according to the fitting function.
Reduces calculation complexity, improves positioning accuracy, ensures the reliability of calibration parameters and saves calculation costs.
Smart Images

Figure CN119068057A_ABST
Abstract
Description
Technical Field
[0001] The embodiments of the present invention relate to the field of perception and positioning, and specifically, to a camera calibration method and device. Background Art
[0002] There are many existing vehicle positioning methods, but there is still a lot of room for expansion in positioning methods based on vehicle-road collaboration. Under the platform of vehicle-road collaboration, both roadside equipment and vehicle-mounted equipment can locate vehicles, and vehicle positioning based on camera calibration is an important method for roadside equipment. The existing multi-camera calibration method has the problems of complex calculation and low positioning accuracy. Summary of the invention
[0003] The embodiments of the present invention provide a camera calibration method and device to at least solve the problem of complex calculation and low positioning accuracy in the positioning method based on multiple cameras in the related art.
[0004] According to one embodiment of the present invention, a camera calibration method is provided, comprising: obtaining camera parameters and modeling parameters of multiple cameras; obtaining calibration points in a common photographic area of the multiple cameras; establishing a mapping relationship between the camera parameters and the calibration points; based on the mapping relationship, fitting the modeling parameters and the position of the target object using the least squares method to obtain a fitting function with the optimization goal of positioning accuracy; and calibrating the multiple cameras according to the fitting function.
[0005] According to another embodiment of the present invention, a camera calibration device is provided, including: an acquisition module, used to acquire camera parameters and modeling parameters of multiple cameras, and acquire calibration points in a common photographic area of the multiple cameras; a mapping module, used to establish a mapping relationship between the camera parameters and the calibration points; a fitting module, used to fit the modeling parameters and the target object position based on the mapping relationship using the least squares method to obtain a fitting function with the optimization goal of positioning accuracy; a calibration module, used to calibrate the multiple cameras according to the fitting function.
[0006] According to yet another embodiment of the present invention, a computer-readable storage medium is provided, in which a computer program is stored, wherein the computer program is configured to execute the steps of any one of the above method embodiments when run.
[0007] According to yet another embodiment of the present invention, there is provided an electronic device, including a memory and a processor, wherein the memory stores a computer program, and the processor is configured to run the computer program to execute the steps in any one of the above method embodiments.
[0008] Through the embodiment of the present invention, a camera calibration method is provided, which obtains camera parameters and modeling parameters of multiple cameras; obtains calibration points in the common photography area of multiple cameras; establishes a mapping relationship between camera parameters and calibration points; based on the mapping relationship, uses the least squares method to fit the modeling parameters and the position of the target object to obtain a fitting function with the optimization target as positioning accuracy; and calibrates multiple cameras according to the fitting function. The problem of complex calculation and low positioning accuracy in positioning in the related technology is solved, and the effect of reducing calculation complexity and improving positioning accuracy is achieved. BRIEF DESCRIPTION OF THE DRAWINGS
[0009] Figure 1 This is a conceptual diagram of the accuracy range of different fitting methods;
[0010] Figure 2 is a flow chart of a camera calibration method according to an embodiment of the present invention;
[0011] Figure 3 is a structural block diagram of a camera calibration device according to an embodiment of the present invention;
[0012] Figure 4 is a structural block diagram of a camera calibration device according to another embodiment of the present invention;
[0013] Figure 5 is a structural block diagram of an electronic device according to an embodiment of the present invention;
[0014] Figure 6 is a flow chart of camera parameter calibration according to a scenario embodiment of the present invention;
[0015] Figure 7 is a schematic diagram of the conversion principle from the world coordinate system to the camera projection coordinate system according to a scene embodiment of the present invention;
[0016] Figure 8 is a schematic diagram of the coordinate relationship between camera projection coordinate systems according to a scene embodiment of the present invention;
[0017] Fig. 9 is a schematic diagram of the mapping principle of the most distant camera parallel plane to the image coordinate system according to a scene embodiment of the present invention;
[0018] Fig.10 is a schematic diagram of the mapping principle from the image coordinate system to the pixel coordinate system according to a scene embodiment of the present invention;
[0019] Fig.11 It is a structural block diagram of a perception and positioning system according to a scenario embodiment of the present invention. DETAILED DESCRIPTION
[0020] Hereinafter, embodiments of the present invention will be described in detail with reference to the accompanying drawings and in combination with the embodiments.
[0021] It should be noted that the terms "first", "second", etc. in the specification and claims of the present invention and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence.
[0022] Protocol message transmission is a common transmission method in the Internet / Internet of Things, 2G / 3G / 4G communication access network. Protocol messages are exchanged between the receiver and the sender, so that different protocols can achieve different functions. V2X technology, also known as vehicle wireless communication technology, is essentially an Internet of Things technology that can achieve intelligent driving of the vehicle itself through radar and visual technology. In terms of environmental perception, in addition to using the vehicle's own intelligence, it can also use the external environment to obtain information, such as roadside camera equipment.
[0023] The embodiment of the present invention takes into account the characteristics of overlapping camera areas at intersections, and simultaneously establishes the relationship between points in the public camera area and different camera parameters during camera calibration, so that different camera parameters constrain each other, thereby improving the accuracy of camera target positioning and the reliability of calibration parameters. First, a group of calibration points are selected in the public camera area of the intersection camera, and the GPS positions of the calibration points are obtained using a positioning device (such as OBU y9000). At the same time, the image coordinate positions of the group of calibration points are obtained in different camera photography pictures. Then, a mathematical model is established to express the nonlinear mapping relationship between the image coordinate position and its corresponding GPS position. In this way, the same group of calibration points has a group of GPS positions and multiple groups of image coordinate positions, and each group of image coordinate positions can establish a relationship with the GPS position separately, and the relationship is only affected by the parameters of a single camera (camera position, height, focal length, pitch angle, etc.). If only a single camera image is used to establish a fitting relationship with the GPS position, two defects in the scene are likely to occur. One is that it falls into the local optimum during the fitting process, and the other is that if you want to jump out of the local optimum and continue fitting, the calculation cost will increase a lot. Figure 1 This is a conceptual diagram of the accuracy range of different fitting methods. Since the present invention aims to improve the actual positioning accuracy of the camera for the calibration points in the picture at a lower cost, the positioning accuracy is used as the objective function to establish a fitting relationship between each camera parameter and the positioning accuracy, and perform overall fitting. This can not only improve the positioning accuracy, but also, under the premise of high positioning accuracy, the complexity of the fitting process is controllable, saving the fitting cost, and the fitting result naturally meets the actual application scenario. The parameters of each camera constrain each other, and actual position training and verification are used to improve the reliability of the calibration parameters. According to Figure 1 It can be seen that when calibrating camera parameters, the present invention clarifies the calibration range, reduces the calculation cost, and improves the positioning accuracy.
[0024] In this embodiment, a camera calibration method is provided. Figure 2 is a flow chart of a camera calibration method according to an embodiment of the present invention. Figure 2 As shown, the process includes the following steps:
[0025] Step S202, obtaining camera parameters and modeling parameters of multiple cameras;
[0026] In an exemplary embodiment, the modeling parameters include at least one of the following: camera height; camera GPS position coordinates; position coordinates of the center point of the camera's pixel image; the angle between the camera projection coordinate system and the world coordinate system; coordinates of the projection point of the center point of the pixel image on the farthest camera parallel plane.
[0027] It should be understood that the center point of the pixel image of the above-mentioned camera refers to the actual position of the object in the image taken by the camera, and the center point of the pixel image can also be called the center point of the camera pixel.
[0028] Step S204, obtaining calibration points in a common photographing area of multiple cameras;
[0029] Step S206, establishing a mapping relationship between camera parameters and calibration points;
[0030] In an exemplary embodiment, establishing a mapping relationship between camera parameters and calibration points includes: establishing a mapping relationship from a world coordinate system to a pixel coordinate system.
[0031] In an exemplary embodiment, a mapping relationship from a world coordinate system to a pixel coordinate system is established, including: establishing a mapping relationship from a world coordinate system to a camera projection coordinate system; in the camera projection coordinate system, establishing a coordinate correspondence between a projection point of a camera pixel center point on a farthest camera parallel plane and any point in a camera photo; establishing a mapping relationship from the camera projection coordinate system to an image coordinate system; and establishing a mapping relationship from an image coordinate system to a pixel coordinate system.
[0032] Step S208, based on the mapping relationship, using the least squares method to fit the modeling parameters and the position of the target object to obtain a fitting function whose optimization goal is positioning accuracy;
[0033] In an exemplary embodiment, the fitting function is:
[0034] Δd=∑ε i ω i
[0035] Wherein, Δd is the weighted positioning accuracy, ε i is the fitting parameter, ω i is the actual position data of the target object.
[0036] The target object may be a vehicle target.
[0037] In an exemplary embodiment, after establishing a fitting function with the optimization target of positioning accuracy, the method further includes: verifying the accuracy of the fitting function according to the substitute values of the calibration points in the public photography area. Specifically, the accuracy is verified by substituting the substitute values into the fitting function.
[0038] The substitution value refers to substituting a set of data taken each time, that is, a set of actual position data and the GPS position data calculated from the corresponding position of the image, into the least square fitting formula, that is, the fitting function for calculation. x is the actual position data g x采集 The position data g calculated by the image position data and the above modeling equation x计算 The UTM coordinate difference is
[0039] Δg x =g x采集 -g x计算
[0040] Δg y =g y采集 -g y计算
[0041] The difference is equal to the least squares fitting formula, i.e., the right side of the fitting function. For the convenience of explanation, the subsequent formula calculations are based on g. x Replace g x计算 .
[0042] Step S210, calibrating multiple cameras according to the fitting function.
[0043] In the actual implementation process, the process of calibrating multiple cameras according to the fitting function mainly involves adjusting and calibrating the camera parameters according to the fitting function, thereby completing the calibration of the multiple cameras.
[0044] In an exemplary embodiment, after calibrating multiple cameras according to the fitting function, the method further includes: using the calibrated cameras to shoot and locate the target object.
[0045] Through the above steps, a camera calibration method is provided, which obtains camera parameters and modeling parameters of multiple cameras; obtains calibration points in the common photography area of multiple cameras; establishes a mapping relationship between camera parameters and calibration points; based on the mapping relationship, uses the least squares method to fit the modeling parameters and the position of the target object to obtain a fitting function with the optimization target as the positioning accuracy; calibrates multiple cameras according to the fitting function to locate the target. The problem of complex calculation and low positioning accuracy in positioning in the related technology is solved, and the effect of reducing the calculation complexity and improving the positioning accuracy is achieved.
[0046] Through the description of the above implementation methods, those skilled in the art can clearly understand that the method according to the above embodiment can be implemented by means of software plus a necessary general hardware platform, and of course can also be implemented by hardware, but in many cases the former is a better implementation method. Based on such an understanding, the technical solution of the present invention, or the part that contributes to the prior art, can be embodied in the form of a software product, which is stored in a storage medium (such as ROM / RAM, a magnetic disk, or an optical disk), and includes a number of instructions for enabling a terminal device (which can be a mobile phone, a computer, a server, or a network device, etc.) to execute the methods described in each embodiment of the present invention.
[0047] In this embodiment, a camera calibration device is also provided, which is used to implement the above-mentioned embodiments and preferred implementation modes, and will not be repeated here. As used below, the term "module" can be a combination of software and / or hardware that implements a predetermined function. Although the devices described in the following embodiments are preferably implemented in software, the implementation of hardware, or a combination of software and hardware, is also possible and conceivable.
[0048] Figure 3 is a structural block diagram of a camera calibration device according to an embodiment of the present invention. Figure 3 As shown, the calibration device 30 includes: an acquisition module 310, which is used to acquire camera parameters and modeling parameters of multiple cameras, and acquire calibration points in a common photographic area of multiple cameras; a mapping module 320, which is used to establish a mapping relationship between camera parameters and calibration points; a fitting module 330, which is used to fit the modeling parameters and the position of the target object based on the mapping relationship using the least squares method to obtain a fitting function with the optimization goal of positioning accuracy; a calibration module 340, which is used to calibrate multiple cameras according to the fitting function.
[0049] Figure 4 is a structural block diagram of a camera calibration device according to another embodiment of the present invention. Figure 4 As shown, the calibration device 40 includes Figure 3In addition to all the modules shown, the present invention further includes: a verification module 410 for verifying the accuracy of the fitting function according to the substituted values of the calibration points in the common photography area.
[0050] It should be noted that the above modules can be implemented by software or hardware. For the latter, it can be implemented in the following ways, but not limited to: the above modules are all located in the same processor; or the above modules are located in different processors in any combination.
[0051] An embodiment of the present invention further provides a computer-readable storage medium, in which a computer program is stored, wherein the computer program is configured to execute the steps of any of the above method embodiments when running.
[0052] In an exemplary embodiment, the computer-readable storage medium may include, but is not limited to, various media that can store computer programs, such as a USB flash drive, a read-only memory (ROM), a random access memory (RAM), a mobile hard disk, a magnetic disk or an optical disk.
[0053] An embodiment of the present invention further provides an electronic device, Figure 5 is a structural block diagram of an electronic device according to an embodiment of the present invention. Figure 5 As shown, the electronic device 50 includes a memory 510 and a processor 520. The memory 510 stores a computer program, and the processor 520 is configured to run the computer program to execute the steps in any of the above method embodiments.
[0054] In an exemplary embodiment, the electronic device may further include a transmission device and an input / output device, wherein the transmission device is connected to the processor, and the input / output device is connected to the processor.
[0055] For specific examples in this embodiment, reference may be made to the examples described in the above embodiments and exemplary implementation modes, and this embodiment will not be described in detail herein.
[0056] Obviously, those skilled in the art should understand that the above modules or steps of the present invention can be implemented by a general computing device, they can be concentrated on a single computing device, or distributed on a network composed of multiple computing devices, they can be implemented by a program code executable by a computing device, so that they can be stored in a storage device and executed by the computing device, and in some cases, the steps shown or described can be executed in a different order than here, or they can be made into individual integrated circuit modules, or multiple modules or steps therein can be made into a single integrated circuit module for implementation. Thus, the present invention is not limited to any specific combination of hardware and software.
[0057] In order to enable those skilled in the art to better understand the technical solution of the present invention, it is described below in conjunction with specific scenario embodiments.
[0058] Scenario Example 1
[0059] In this scenario embodiment, a multi-camera based positioning method provided by the embodiment of the present invention mainly includes calibration, fitting training phase, and positioning phase. Figure 6 is a flow chart of camera parameter calibration according to a scenario embodiment of the present invention, such as Figure 6 As shown in the figure, mathematical modeling is first used to establish the relationship between different camera parameters and calibration points. Based on the modeling relationship, a least square fitting formula with the optimization goal of positioning accuracy is established, and then the calibration point position proxy values of the public photography area are obtained for fitting calculation until the accuracy requirements are met. If not, the proxy values can be recalculated for fitting calculation or the target optimization accuracy can be reduced.
[0060] The fitting formula and the pixel data in the picture together complete the data support of the roadside equipment, and the mathematical modeling is carried out in a nonlinear mapping way with high accuracy. The modeling process is as follows:
[0061] (1) Establish a mapping from the world coordinate system to the camera projection coordinate system
[0062] Since the world coordinate system is a plane coordinate system and the camera projection coordinate system is a stereo coordinate system, Figure 7 is a schematic diagram of the conversion principle from the world coordinate system to the camera projection coordinate system according to the scene embodiment of the present invention, such as Figure 7 As shown, in the process of constructing the camera projection coordinate system, the xy plane of the camera projection coordinate system (i.e. Figure 7 If the UTMX and UTMY in the image are placed on the ground and coincide with the world coordinate system, then the point of the camera projection coordinate system in the xy plane and the world coordinate system will have a plane coordinate conversion relationship:
[0063] g x =x g cos(t)-y gsin(t)+x0
[0064] g y =x g sin(t)+y g cos(t)+y0
[0065] Among them, (g x , g y ) represents the coordinates of the camera projection coordinate system, (x g ,y g ) represents the coordinates of the world coordinate system, (x0, y0) represents the coordinates of the intersection of the camera projection coordinate system and the world coordinate system, and t represents the angle between the camera projection coordinate system and the world coordinate system.
[0066] (2) Assume the farthest camera parallel plane in the camera projection coordinate system, substitute the special points and general points to obtain the coordinate correspondence between the camera projection coordinate system.
[0067] Figure 8 is a schematic diagram of the coordinate relationship between the camera projection coordinate systems according to a scene embodiment of the present invention, such as Figure 8 As shown, a mathematical model coordinate system is constructed, where point C represents the focal position of the camera (since the focal length is very small, it can be understood as the camera position). The inclined plane in the figure is a plane with the same pixel ratio as the camera pixel image (this method assumes that the position relationship between the plane parallel to the camera lens and the pixel image of the camera imaging is proportional, and this plane is called the farthest camera parallel plane q from the camera to the ground). Plane q intersects with the xy plane of the coordinate system, where the center point of the intersection is S, the projection of the camera pixel center point is M, the projection of the camera pixel center point on the inclined plane is m, and an arbitrary point is P. Figure 8 In the method, the projection M of the center point of the camera pixel and the arbitrary point P are located on the same plane, that is, the ground. This method assumes that the xy plane is the actual ground and the plane q is the farthest camera parallel plane. The projection M of the center point of the camera pixel is taken as a special point, and the arbitrary point P taken by the camera photo is taken as a general point. The coordinate correspondence equation between the special point and the general point is obtained as follows:
[0068]
[0069]
[0070] Among them, (x pg ,y pg ) represents the coordinates of a special point in the camera projection coordinate system, (x pq ,y pq ) represents the coordinates of the intersection of the line connecting the special point and the camera center point and the inclined plane, y mg Indicates the position coordinates corresponding to the center point of the pixel image, zc Indicates the actual camera height in reality, y mq Indicates the y coordinate of the intersection of the ground point corresponding to the center point of the pixel image, the line connecting the center point of the camera and the inclined plane. The x coordinate of the intersection is 0. (y mq , 0) represents the coordinates of the projection point of the camera pixel center point on the farthest camera parallel plane in the above embodiment.
[0071] (3) Mapping from camera projection coordinate system to image coordinate system
[0072] The mapping from the camera projection coordinate system to the image coordinate system can be divided into two steps. The first step is the mapping of the xy plane of the camera projection coordinate system to the most distant camera parallel plane (non-linear mapping, refer to the collinear relationship between the xy plane and plane q based on point C), which is expressed by the following formula:
[0073]
[0074]
[0075] Fig. 9 is a schematic diagram of the mapping principle of the most distant camera parallel plane to the image coordinate system according to the scene embodiment of the present invention, such as Fig. 9 As shown, the second step is the mapping of the most distant camera parallel plane to the image coordinate system (proportional mapping), which is expressed by the formula:
[0076]
[0077]
[0078] Among them, f2 represents the pixel related parameters, y nn 、x nn Represents the position of the point in the image coordinate system.
[0079] (4) Mapping from image coordinate system to pixel coordinate system
[0080] Fig.10 is a schematic diagram of the mapping principle from the image coordinate system to the pixel coordinate system according to a scenario embodiment of the present invention, such as Fig.10 As shown, there is a displacement between the origin of the image coordinate system and the pixel coordinate system, and the straight lines where the coordinate axes are located are consistent. The mapping from the image coordinate system to the pixel coordinate system is expressed by the formula:
[0081]
[0082]
[0083] Among them, the public nn Right now Fig.10 y″ in the middle, x in the middlenn Right now Fig.10 x″ in the middle, y in the middle n Right now Fig.10 y′ in the n Right now Fig.10 x′, f1, and f2 represent pixel-related parameters.
[0084] (5) Determine the optimization objective function as the object positioning accuracy, establish the least squares fitting formula with the modeling parameters as the fitting parameters, and confirm the sensitive parameters related to the positioning accuracy based on the modeling relationship as follows: mq ,y mg 、z c , x0, y0, t. Among them, y mg Indicates the position coordinates corresponding to the center point of the pixel image, z c represents the actual camera height in reality, x0 and y0 represent the GPS position coordinates of the camera, and t represents the angle between the camera projection coordinate system and the world coordinate system.
[0085] Specifically, the above-mentioned sensitive parameters are included in the above-mentioned modeling parameters, that is, the number of modeling parameters is greater than or equal to the number of sensitive parameters. In the actual implementation process, some or all of the modeling parameters related to the positioning accuracy are confirmed as sensitive parameters according to the modeling relationship, and the sensitive parameters involved in fitting can be called fitting parameters. The above-mentioned sensitive parameters can be obtained through the equipment. In the actual implementation process, after confirming the sensitive parameters, the number and type of sensitive parameters involved in fitting can be determined according to different accuracy requirements. For example, if the positioning accuracy requirement is not high, one of the above-mentioned sensitive parameters can be selected for fitting operation; because there are random errors when the staff actually measures the position data, the above-mentioned sensitive parameters can be listed as parameters for fitting when the positioning accuracy requirement is higher.
[0086] The positioning accuracy of the target refers to the distance difference between the actual GPS data and the position obtained through image mapping. Since the fitting target is the above fitting parameters, and the position of the object is a binary (multivariate) parameter, the fitting formula is a multivariate least squares fitting formula. The fitting function is:
[0087] Δd=∑ε i ω i
[0088] Among them, Δd is the weighted positioning accuracy, ε i is the modeling parameter (fitting parameter), ω i is the actual position data of the vehicle target. The above calculation can be completed by python3, numpy library. The specific expansion example of the fitting function can be expressed as follows:
[0089] Δd=0.5Δgx +0.5Δg y
[0090] Δg x =∑(y mq -y′ mq ) x +(tt′)g x +(z c -z′ c ) x
[0091] Δg y =∑(y mq -y′ mq ) y +(tt′)g y +(z c -z′ c ) y
[0092] Among them, y′ mq , t′, z′ c is the constant corresponding to the actual physical value of the corresponding fitting parameter in the modeling model.
[0093] The embodiment of the present invention also provides a sensing and positioning system. Fig.11 is a structural block diagram of a perception and positioning system according to a scenario embodiment of the present invention, such as Fig.11 As shown in the figure, the roadside module (camera, etc.) and the vehicle-mounted module jointly provide data for the V2X data system, providing sufficient and reliable data support for the system to calculate the autonomous driving solution. Fig.11 As shown, the roadside module includes functions such as photographic image information collection, substitution into fitting formulas, and target positioning support. The vehicle-mounted module provides positioning support functions. The V2X data system is independent of the two modules and is used for perception and positioning. In the specific implementation process, the function settings and function divisions of the above modules and systems can be set according to actual conditions and are not limited here.
[0094] In summary, the camera calibration method and device provided by the embodiment of the present invention are applied in the scenario of using the information captured by the intersection camera to provide perception information such as vehicle positioning. When the existing method calibrates multiple cameras, there is still the problem of calibrating the camera images one by one. The method of the present invention combines the parameters of each camera through a fitting formula, and can be calibrated at the same time. The calculation complexity depends more on the sampling data and the optimization target, which is intuitive and controllable. Another existing method, before the vehicle-road data is fused, the position information set obtained from the roadside equipment is the "reference position information" for preliminary perception, which is obtained from the grid map established by it. It is understood as a table-type table lookup to obtain preliminary data, which is relatively rough. The target information obtained by the roadside equipment in the present invention is more accurate and has more reference value in the fusion of vehicle-road information.
[0095] The perception positioning scheme based on the calibration of the camera at the intersection of the vehicle-road cooperative proposed in the embodiment of the present invention has higher accuracy and better reliability. Compared with the classic calibration method, the embodiment of the present invention improves the positioning accuracy by modeling and fitting, verifies with actual values, and ensures the reliability of the scheme. The above embodiment of the present invention is based on the joint calibration of the intersection camera of the vehicle-road cooperative platform. By establishing the relationship between the points in the public photography area and the pictures of different cameras, the least squares fitting formula is established according to the camera parameters, and the camera calibration is completed by substitution fitting. This method uses the actual GPS position as the data basis, and uses multi-camera data fusion verification, and the calculation result has high reliability. The least squares fitting is used to optimize the mathematical model parameters in the camera calibration, and the positioning accuracy is high and the optimization is controllable, which is convenient to reduce the sampling cost and meets the actual production needs. The application scenario mainly involved in the embodiment of the present invention is the vehicle perception support for autonomous driving by the roadside equipment in the V2X platform. The scenario that meets the application in this scenario is the intersection camera, etc., and it can also be a common camera area between other multiple cameras. The present invention does not require the same camera model. According to the characteristics of the camera, each camera can be modeled separately. In the fitting process of the positioning accuracy, the present invention can set the accuracy expectation to control the optimization target and difficulty.
[0096] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. For those skilled in the art, the present invention may have various modifications and variations. Any modification, equivalent replacement, improvement, etc. made within the principle of the present invention shall be included in the protection scope of the present invention.
Claims
1. A camera calibration method, It is characterized in that include: Get camera parameters and modeling parameters of multiple cameras; Acquire calibration points in a common photographing area of the multiple cameras; Establishing a mapping relationship between the camera parameters and the calibration points; Based on the mapping relationship, the least square method is used to fit the modeling parameters and the position of the target object to obtain a fitting function whose optimization goal is positioning accuracy; The multiple cameras are calibrated according to the fitting function.
2. The method according to claim 1, It is characterized in that Establishing a mapping relationship between the camera parameters and the calibration points includes: Establish the mapping relationship from the world coordinate system to the pixel coordinate system.
3. The method according to claim 2, It is characterized in that The establishing of the mapping relationship from the world coordinate system to the pixel coordinate system includes: Establishing a mapping relationship from the world coordinate system to the camera projection coordinate system; In the camera projection coordinate system, a coordinate correspondence between the projection point of the camera pixel center point on the farthest camera parallel plane and any point in the camera photo is established; Establishing a mapping relationship between the camera projection coordinate system and the image coordinate system; A mapping relationship from the image coordinate system to the pixel coordinate system is established.
4. The method according to claim 1, It is characterized in that The modeling parameters include at least one of the following: Camera height; Camera GPS location coordinates; The position coordinates of the center point of the camera's pixel image; The angle between the camera projection coordinate system and the world coordinate system; The coordinates of the projection point of the center point of the pixel image on the most distant camera parallel plane.
5. The method according to claim 4, It is characterized in that in, The fitting function is: Δd=∑ε i oh i Wherein, Δd is the positioning accuracy, ε i is the modeling parameter, ω i is the actual position data of the target object.
6. The method according to claim 1, It is characterized in that After fitting the modeling parameters and the target object position using the least square method based on the mapping relationship to obtain a fitting function whose optimization target is positioning accuracy, the method further includes: The accuracy of the fitting function is verified according to the substituted values of the calibration points in the public photography area.
7. The method according to claim 1, It is characterized in that After calibrating the multiple cameras according to the fitting function, the method further includes: The calibrated camera is used to photograph and locate the target object.
8. A camera calibration device, It is characterized in that include: An acquisition module, used to acquire camera parameters and modeling parameters of multiple cameras, and acquire calibration points in a common photographic area of the multiple cameras; A mapping module, used to establish a mapping relationship between the camera parameters and the calibration points; A fitting module, used for fitting the modeling parameters and the position of the target object using the least square method based on the mapping relationship to obtain a fitting function whose optimization target is positioning accuracy; A calibration module is used to calibrate the multiple cameras according to the fitting function.
9. The device according to claim 8, It is characterized in that Also includes: A verification module is used to verify the accuracy of the fitting function according to the substituted values of the calibration points in the public photography area.
10. A computer-readable storage medium, It is characterized in that The computer-readable storage medium stores a computer program, wherein the computer program implements the method described in any one of claims 1 to 7 when executed by a processor.
11. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, It is characterized in that When the processor executes the computer program, the method described in any one of claims 1 to 7 is implemented.
