Camera external parameter calibration method and device, electronic equipment and storage medium
By identifying the vehicle profile and heading angle in the calibration room, establishing the vehicle coordinate system, and projecting the calibration checkerboard to the camera's vehicle coordinate system, the problems of high cost of camera external parameter calibration and cumbersome process in the existing technology are solved, and convenience is achieved , efficient camera external parameter calibration.
Patent Information
- Application Number
- CN202411072584.3
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-08-06
- Publication Date
- 2024-11-12
- Estimated Expiration
- 2044-08-06
AI Technical Summary
The existing camera external parameter calibration methods have high construction costs and cumbersome calibration process, especially because of the need to use mechanical devices to correct the vehicle movement.
By setting up a camera in the calibration room, obtaining vehicle images, identifying vehicle profiles and heading angles, establishing a vehicle coordinate system, and projecting the calibration checkerboards to the vehicle coordinate system to perform camera external parameter calibration, avoiding the use of mechanical devices to move the vehicle Correction.
It realizes convenient camera external parameter calibration, reduces construction costs, improves calibration efficiency, adapts to various scenarios, and simplifies the calibration process.
Smart Images

Figure CN118941650A_ABST
Abstract
Description
Technical Field
[0001] The present application relates to the field of camera calibration, and in particular to a camera extrinsic calibration method and device, electronic equipment and storage medium. Background Art
[0002] At present, the camera extrinsic calibration system includes a vehicle caliper displacement system and a calibration chessboard, wherein the vehicle caliper displacement system is used to clamp the wheels of the calibrated vehicle through the caliper and move the vehicle to a fixed position, at which the actual vehicle center point coincides with the preset vehicle center point, and the vehicle's heading is consistent with the preset heading. Since the calibration chessboard is fixed, at this time, in the three-dimensional coordinate system with the vehicle center point as the origin and the vehicle heading direction as the positive axis, the exact positions of all chessboards are known. After that, the image containing the calibration chessboard is captured by the camera on the vehicle, and the camera extrinsic parameters can be calculated by identifying the chessboard corner points on the image and forming point pairs corresponding to the coordinates of the chessboard corner points in the actual vehicle coordinate system. In the prior art, due to the need to build a high-standard mechanical automation calibration room and the fixed calibration chessboard, there are problems such as high construction cost, difficult equipment maintenance, and cumbersome calibration adjustment. Summary of the invention
[0003] The present application provides a camera extrinsic parameter calibration method and device, electronic device and storage medium to solve the problem that the camera extrinsic parameter calibration method in the prior art is not only cumbersome in the calibration process but also high in construction cost.
[0004] In a first aspect, the present application provides a camera extrinsic parameter calibration method, comprising: obtaining an image of a vehicle after entering the calibration room through a first camera arranged in the calibration room; obtaining a ground contour of the vehicle from the image, and identifying the ground contour to obtain a center position of the vehicle in an image coordinate system and a first heading angle of the vehicle in the image coordinate system, wherein the image coordinate system is a coordinate system with the center of the vehicle in the image as the coordinate origin; establishing a vehicle coordinate system based on the center position of the vehicle in the calibration room and the first heading angle, wherein the vehicle coordinate system is a coordinate system with the center position of the vehicle as the origin; projecting a calibration checkerboard to a calibration area based on the vehicle coordinate system, and performing extrinsic parameter calibration on a second camera of the vehicle.
[0005] In a second aspect, the present application provides a camera extrinsic parameter calibration device, comprising: an acquisition module, used to acquire an image of a vehicle after entering the calibration room through a first camera arranged in the calibration room; a first processing module, used to acquire a ground contour of the vehicle from the image, and identify the ground contour to obtain a center position of the vehicle in an image coordinate system and a first heading angle of the vehicle in the image coordinate system, wherein the image coordinate system is a coordinate system with the center of the vehicle in the image as the coordinate origin; an establishment module, used to establish a vehicle coordinate system based on the center position of the vehicle in the calibration room and the first heading angle, wherein the vehicle coordinate system is a coordinate system with the center position of the vehicle as the origin; a second processing module, used to project a calibration checkerboard to a calibration area based on the vehicle coordinate system, and perform extrinsic parameter calibration on the second camera of the vehicle.
[0006] In a third aspect, the present application provides an electronic device comprising: at least one communication interface; at least one bus connected to the at least one communication interface; at least one processor connected to the at least one bus; and at least one memory connected to the at least one bus, wherein the processor is configured to execute the camera extrinsic parameter calibration method described in the first aspect of the present application.
[0007] In a fourth aspect, the present application further provides a computer storage medium storing computer executable instructions, wherein the computer executable instructions are used to execute the camera extrinsic parameter calibration method described in the first aspect of the present application.
[0008] The above technical solution provided by the embodiment of the present application has the following advantages over the prior art: in the embodiment of the present application, a calibration room camera is used to capture an image of the vehicle, and then the vehicle outline and its heading angle are identified based on the image data, and then a vehicle coordinate system is established, and the calibration checkerboard is projected to a calibration area based on the vehicle coordinate system, and the external parameter calibration of the second camera of the vehicle is performed. It can be seen that in the embodiment of the present application, after acquiring the image of the vehicle in the calibration room, a corresponding vehicle coordinate system is established, and then the position and angle of the projection are adjusted so that the calibration checkerboard matches the vehicle coordinate system, thereby completing the external parameter calibration of the vehicle camera, which can not only adapt to various scenes more conveniently, but also has a low construction cost and a high calibration efficiency, and does not need to use a mechanical device to move and correct the vehicle as in the prior art, which is not only cumbersome but also has a high construction cost. BRIEF DESCRIPTION OF THE DRAWINGS
[0009] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the invention and, together with the description, serve to explain the principles of the invention.
[0010] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, for ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative labor.
[0011] One or more embodiments are exemplarily described by pictures in the corresponding drawings, and these exemplified descriptions do not constitute limitations on the embodiments. Elements with the same reference numerals in the drawings represent similar elements, and unless otherwise stated, the figures in the drawings do not constitute proportional limitations.
[0012] Figure 1 A flowchart of a camera extrinsic calibration method provided in an embodiment of the present application;
[0013] Figure 2 This is a schematic diagram of the hardware system structure of the calibration room in the specific example of this application;
[0014] Figure 3 This is a schematic diagram of obtaining a vehicle image in a calibration room in a specific example of this application;
[0015] Figure 4 It is a schematic diagram of the projection area of each projector in the calibration room in a specific example of this application;
[0016] Figure 5 A schematic diagram of adjusting a projector in a specific example of this application;
[0017] Figure 6 A schematic diagram of marking a chessboard in a specific example of this application;
[0018] Figure 7 A schematic diagram of a projection chessboard of a projector located at the upper left corner in a specific example of this application;
[0019] Figure 8 A schematic diagram of a chessboard projected by four projectors in a specific example of this application;
[0020] Fig. 9 A schematic diagram of the ground contour of a vehicle in a specific example of this application;
[0021] Fig.10 This is a schematic diagram of a vehicle entering a calibration room in a specific example of this application;
[0022] Fig.11 This is a schematic diagram of adjusting the chessboard based on the vehicle coordinate system in a specific example of this application;
[0023] Fig.12 This is a flow chart of a camera extrinsic calibration method based on image recognition and optical projection in an embodiment of the present application;
[0024] Fig.13 A flow chart of a camera extrinsic calibration device provided in an embodiment of the present application;
[0025] Fig.14 A flowchart of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION
[0026] In order to make the purpose, technical solution and advantages of the embodiments of the present application clearer, the technical solution in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of this application.
[0027] The disclosure below provides many different embodiments or examples to implement different structures of the present invention. In order to simplify the disclosure of the present invention, the parts and settings of specific examples are described below. Of course, they are only examples, and the purpose is not to limit the present invention. In addition, the present invention can repeat reference numbers and / or letters in different examples. This repetition is for the purpose of simplification and clarity, and does not itself indicate the relationship between the various embodiments and / or settings discussed.
[0028] First, the relevant terms involved in the embodiments of the present application are explained;
[0029] Camera extrinsic calibration: Camera extrinsic calibration is the process of determining the position and orientation of the camera in the world coordinate system. Based on the extrinsic parameters, a mapping between the image coordinate system and the world coordinate system can be established. Specifically, it involves calculating two main sets of parameters: the rotation matrix (R) and the translation vector (T). These parameters describe the rotation and translation relationship of the camera coordinate system relative to the world coordinate system. The extrinsic parameters in applications mainly include: 1) Spatial positioning: In many applications, such as robot navigation, augmented reality, 3D reconstruction, etc., knowing the position of the camera in space is crucial to correctly interpreting image data; 2) Object measurement: Through calibration, the two-dimensional points in the image can be converted into three-dimensional points in the world coordinate system, thereby realizing the measurement of the size and shape of the object.
[0030] External parameter calibration process: The basic steps of external parameter calibration include: 1) Select the calibration target: use a calibration plate or object with known geometric features, such as a checkerboard, a dot pattern, or other objects with identifiable features; 2) Capture images: photograph the calibration target from different angles and positions to collect enough data for calculation; 3) Feature detection: detect the feature points of the calibration target in the image, such as corner points or the center of a circle; 4) Establish correspondence: establish a correspondence between the feature points in the image and the corresponding three-dimensional points in the world coordinate system; 5) Parameter estimation: use mathematical models and optimization algorithms (such as the least squares method) to estimate the rotation matrix and translation vector; 6) Verification and optimization: verify the accuracy of the calibration results by comparing the predicted image points with the actually observed points, and make necessary adjustments.
[0031] Camera extrinsic factory calibration: Camera extrinsic factory calibration is different from other types of extrinsic calibration. Its main characteristics are stability, speed and high accuracy. It is usually used for factory calibration during camera assembly, such as extrinsic calibration of cameras on vehicles after assembly.
[0032] In order to solve the problem of camera extrinsic calibration in the prior art, which is not only cumbersome but also has high construction cost, the present application provides a camera extrinsic calibration method, such as Figure 1 As shown, the steps of the method include:
[0033] Step 101, obtaining an image of a vehicle after entering the calibration room by using a first camera arranged in the calibration room;
[0034] In a specific example, the calibration room in the embodiment of the present application is associated with the corresponding hardware system, such as Figure 2 As shown, the hardware system includes a first camera, a displacer, a projector, a computing unit, and a storage unit. Among them, the displacer is mechanically connected to the projector, and the computing unit is data-connected with each other unit. Furthermore, the first camera in the embodiment of the present application can be installed in the middle part of the top of the calibration room, and can completely capture all areas in the calibration room, as shown in Figure 3, and the internal and external parameters of the first camera are known, so the coordinates of any point on the ground in the calibration room can be accurately measured by the internal and external parameters of the first camera. The computing unit of the hardware system is used to process the image captured by the camera, including identifying the vehicle outline, calculating the coordinates of the vehicle center point, calculating the vehicle heading, identifying the ground calibration pattern, and controlling the displacer and projector according to the recognition result and the preset calibration pattern. The storage unit in the hardware system is used to store relevant intermediate data such as the image captured by the camera, the preset projection pattern, the internal and external parameters of the camera, and the necessary recognition program;
[0035] In addition, the displacement device in the embodiment of the present application is installed on the top of the calibration room, and the projector is installed at the intersection of the horizontal and vertical axes through a roller. For example, there are four projectors in the embodiment of the present application, which are respectively used for projector displacement in the four square grid areas of the calibration room. The projectors are named FL (FrontLeft), FR (FrontRight), BL (BackLeft), and BR (BackRight) projectors according to the areas where they are located, and the corresponding areas are the FL, FR, BL, and BR projector activity areas. The displacement device is installed on the top of the calibration room as shown in the figure. Figure 4 As shown, the mounting beam is the infrastructure used to fix the mechanical structure on the top of the calibration room. The mounting beams are arranged in a field shape, dividing the entire top of the calibration room into four squares. A displacement device is installed in each square. Each displacement device consists of two movable beams, horizontal and vertical. The horizontal beam can be translated horizontally by an electrical device, and the vertical beam can be translated vertically by an electrical device. The projector is installed at the intersection of the horizontal and vertical beams through mechanical rollers. Figure 4 The number of projectors in the figure is 4, which is just an example. The number of projectors can be set according to actual needs, for example, the number of projectors can be 2, 6, etc.
[0036] Furthermore, the displacer can drive the projector to move to any position in the grid by moving the horizontal and vertical movable beams, such as Figure 5 The projector is used to project the calibration pattern sent by the computing unit onto the ground, and adjust the projection state according to the feedback control instructions (projection size, projection angle, scaling, keystone correction, etc.). The calibration pattern is calculated by the camera and the computer to obtain the projection parameters, and the projection result that meets the preset requirements is projected, as shown in the figure. Figure 6 The calibration pattern projection consists of the projection patterns of four projection devices, as shown in Figure 7 The projection pattern covering the FL area is projected by the FL projector; the four projection devices are responsible for areas such as Figure 8 As shown, the projection patterns of the projectors partially overlap on the ground, which can assist in the alignment of the calibration patterns.
[0037] Step 102, obtaining a ground contour of the vehicle from the image, and identifying the ground contour to obtain a center position of the vehicle in an image coordinate system and a first heading angle of the vehicle in the image coordinate system, wherein the image coordinate system is a coordinate system with the center of the vehicle in the image as a coordinate origin;
[0038] In the embodiment of the present application, the ground contour recognition algorithm can be set accordingly according to actual needs, and specifically, algorithms such as key point recognition and semantic segmentation can be used. Fig. 9As shown, the ground contour of the vehicle is obtained from different angles in the image. For example, the ground contour on the upper left is identified from the upper left, so only the vehicle contours on the left and above and below can be identified, and the vehicle contour on the right cannot be identified. The ground contours in other directions have similar identification results.
[0039] Step 103, establishing a vehicle coordinate system based on the center position of the vehicle in the calibration room and the first heading angle, wherein the vehicle coordinate system is a coordinate system with the center position of the vehicle as the origin;
[0040] In the specific example, the X-axis of the vehicle coordinate system is consistent with the vehicle's angular direction. Therefore, if the vehicle entering the calibration room stops in the middle of the calibration room and the driving direction is perpendicular to the side of the calibration room, the vehicle coordinate system with the vehicle center position as the origin and the calibration room coordinate system have parallel XYZ axes except for the different origin positions.
[0041] Step 104 , projecting the calibration chessboard to a calibration area based on the vehicle coordinate system, and performing extrinsic calibration on the second camera of the vehicle.
[0042] For the ground contour, in the specific example Figure 8 As shown, taking the number of projectors as 4 as an example, the checkerboard in the calibration room includes FL area, FR area, BL area and BR area, there is an overlapping area between two adjacent areas, and each area has its own independent area.
[0043] Through the above steps 101 to 104, the calibration room camera is used to take an image of the vehicle, and then the vehicle outline and heading angle are identified based on the image data, and then the vehicle coordinate system is established, the calibration chessboard is projected to the calibration area based on the vehicle coordinate system, and the external parameter calibration of the second camera of the vehicle is performed. In a specific example, it can be based on Figure 2 The shifter in the projector moves the projector to the appropriate area, which can be done as follows Figure 5 The method shown is used, and the calibration chessboard is projected to a preset standard calibration area based on the vehicle coordinate system. For example, the heading angle of the current vehicle is not perpendicular to the edge of the calibration room, so not only the position of the projector can be adjusted, but also the angle of the projector can be adjusted, and the calibration chessboard matches the vehicle coordinate system. It can be seen that in the embodiment of the present application, after acquiring the image of the vehicle in the calibration room, a corresponding vehicle coordinate system is established, and then the position and angle of the projection are adjusted so that the calibration chessboard matches the vehicle coordinate system, thereby completing the external parameter calibration of the vehicle camera. It can not only adapt to various scenes more conveniently, but also has a low construction cost and a high calibration efficiency. There is no need to use a mechanical device to move and correct the vehicle as in the prior art. This method is not only cumbersome but also has a high construction cost.
[0044] In the embodiment of the present application, the establishment of the vehicle coordinate system requires obtaining the position of the center of the vehicle and the heading angle of the vehicle. Because the vehicle coordinate system is a coordinate system established with the center position of the vehicle as the origin and the heading angle direction of the vehicle as the X-axis, the calibration checkerboard can be matched with the current position of the vehicle during the subsequent calibration checkerboard adjustment, which is more accurate when calibrating the external parameters of the vehicle camera. Therefore, the method of establishing the vehicle coordinate system based on the center position of the vehicle and the first heading angle in the calibration room involved in the above step 103 can further include the following steps:
[0045] Step 11, based on the intrinsic parameters and extrinsic parameters of the first camera and the first heading angle, obtain the center position of the vehicle in the calibration room coordinate system and the second heading angle of the vehicle in the calibration room coordinate system; wherein the calibration room coordinate system is a coordinate system with a corner point of the calibration room as the origin;
[0046] Step 12, establishing a vehicle coordinate system with the center position of the vehicle in the calibration coordinate system as the origin and the second heading angle.
[0047] In the embodiment of the present application, the calibration room coordinate system is a coordinate system established with the upper left corner of the calibration room as the origin, the left side as the positive X direction, the bottom as the positive Y direction, and the top as the positive Z direction. It can be seen that in the embodiment of the present application, after obtaining the vehicle center position and heading angle of the vehicle in the image through the image, it is necessary to convert it to the calibration room coordinate system to obtain the corresponding center position coordinates and heading angle, because the coordinates in the calibration room coordinate system correspond to the position in the real environment. Therefore, a vehicle coordinate system is established based on the obtained vehicle center position and heading angle, and based on the vehicle coordinate system, the image captured by the vehicle camera can be obtained and compared with the actual environment.
[0048] In the embodiment, the ground contour of the vehicle can be obtained from various angles, so that the vehicle position and heading angle can be determined conveniently and quickly later. Therefore, the method of obtaining the ground contour of the vehicle from the image and identifying the ground contour to obtain the center position of the vehicle in the image coordinate system and the first heading angle of the vehicle in the image coordinate system involved in the above step 102 can further include:
[0049] Step 21, obtaining the ground contact contour of the vehicle at the corresponding angle from the images obtained at different angles in the calibration room;
[0050] Step 22 , identifying the ground contours corresponding to different angles to obtain the center position of the vehicle in the image coordinate system and the first heading angle of the vehicle in the image coordinate system.
[0051] In this regard, in a specific example, Fig. 9As shown, the ground contour of the vehicle is obtained from different angles in the image. For example, the ground contour on the upper left can only be identified from the upper left, and the vehicle contour on the left and above and below cannot be identified. The ground contour in other directions has similar recognition results. The ground contour of the vehicle is obtained from various directions, and the heading angle of the vehicle and the position of the center of the vehicle in the image coordinate system are determined based on this, so that the heading angle of the vehicle and the position of the center of the vehicle can be converted based on the relationship between the image coordinate system and the calibration coordinate system, and then the vehicle coordinate system is established based on this, so as to calibrate the external parameters of the vehicle camera later.
[0052] In the embodiment of the present application, the extrinsic parameters of the vehicle camera are calibrated after the vehicle coordinate system is established, and the establishment of the vehicle coordinate system requires obtaining the position of the center of the vehicle and its heading angle in the real environment; after the vehicle coordinate system is established, the calibration checkerboard needs to be adjusted so that the subsequent calibration of the extrinsic parameters of the vehicle camera is more accurate. Therefore, the method of projecting the calibration checkerboard to the calibration area based on the vehicle coordinate system involved in the above step 104 can further include the following steps:
[0053] Step 31 , based on the center coordinates of the vehicle in the vehicle coordinate system and the second heading angle, adjust the position of the projector in the calibration room so that the projector projects the calibration checkerboard into the calibration area based on the vehicle coordinate system.
[0054] In this regard, in a specific example, for the calibration checkerboard in the calibration coordinate system, Figure 8 However, in actual situations, after the vehicle enters the calibration room, its heading angle is not necessarily perpendicular to the edge of the calibration room. As shown in 10, the vehicle's heading angle is not perpendicular to the edge of the calibration room. Therefore, the calibration checkerboard that matches the calibration room coordinate system cannot be used directly. The calibration checkerboard needs to be adjusted after the vehicle coordinate system is established. The adjusted checkerboard is as shown in Fig.11 As shown, by adjusting the position of the projector, the position and direction of the calibration chessboard are changed, and the change in the position and direction of the calibration chessboard is related to the vehicle coordinate system.
[0055] Furthermore, the method of adjusting the position of the projector in the calibration room based on the center coordinates of the vehicle in the vehicle coordinate system and the second heading angle involved in the above step 31 may further include:
[0056] Step 41, determining the position of the projector in the vehicle coordinate system based on the center coordinates of the vehicle in the vehicle coordinate system and the second heading angle;
[0057] Step 42, adjusting the position of the projector in the calibration room to make it consistent with the position of the projector in the vehicle coordinate system.
[0058] For the above steps 41 and 42, in a specific example, Fig.11 As shown in Figure 2, after establishing the vehicle coordinate system, the calibration chessboard needs to be adjusted. Taking the number of projectors as 4 as an example, the original calibration chessboard is as follows: Figure 8 As shown in Figure 1, each projector corresponds to a calibration chessboard. Therefore, after establishing the vehicle calibration coordinate system, Figure 8 The calibration checkerboard in is adjusted, and the adjusted calibration checkerboard is as follows Fig.11 As shown, the calibration chessboard can be adjusted by adjusting the position and angle of the projection, and then the vehicle camera extrinsic calibration can be performed. Through the present application, there is no need to use a mechanical device to move and correct the vehicle as in the prior art, making the vehicle camera extrinsic calibration more convenient and quick.
[0059] It can be seen that when there are multiple projectors, the method for determining the position of the projector in the vehicle coordinate system based on the center coordinates of the vehicle in the vehicle coordinate system and the second heading angle involved in the above step 41 can further include: determining the position of each projector in the multiple projections in the vehicle coordinate system based on the mapping relationship between the vehicle coordinate system and the calibration coordinate system, as well as the external parameters and internal parameters of the first camera.
[0060] Taking the number of projectors as 4 as an example, it is necessary to determine the position and angle of each projection in the vehicle coordinate system, and then Figure 5 In the manner shown, the position and angle of the projector are adjusted so that the calibration chessboard projected by the projector can be Fig.11 As shown, the camera extrinsic calibration can then be performed, which is simpler and more efficient than the existing technology.
[0061] In the embodiment of the present application, the method of calibrating the external parameters of the second camera of the vehicle involved in the above step 104 may further include:
[0062] Step 51, performing corner point recognition on the image acquired by the second camera to obtain corresponding first corner point coordinates;
[0063] Step 52 , calibrating the extrinsic parameters of the second camera based on the coordinates of the second corner points and the coordinates of the first corner points of the calibration checkerboard in the calibration area.
[0064] In this regard, in a specific example, the vehicle camera is controlled to start taking pictures, and the corner points of the images taken by the vehicle camera are identified. The identified corner points (Pt1 img , Pt2 img , Pt3 img ......) and the corresponding point in the known vehicle coordinate system (Pt1 car , Pt2 car , Pt3 car......) form point pairs, based on which the camera extrinsic parameters can be further calculated.
[0065] The present application is explained below in conjunction with a specific implementation of an embodiment of the present application. The specific implementation provides a camera extrinsic calibration method based on image recognition and optical projection, such as Fig.12 As shown, the steps of the method include:
[0066] Step 1021, vehicle enters the venue;
[0067] Drive the vehicle into the middle position of the calibration field from the outside. The vehicle in the embodiment of the present application will be parked in the middle area of the calibration field with a non-centered and non-vertical heading. In addition, the vehicle parking position must be at a certain distance from the four walls to ensure that the calibration pattern can be fully projected around the vehicle.
[0068] Step 1022, the top camera photographs the vehicles in the factory;
[0069] In the embodiment of the present application, Figure 2 The hardware system in the calibration field is used to calibrate the top high-definition camera (hereinafter referred to as Cam cali ) The captured image is as follows Fig.10 In the embodiment of the present application, Cam cali The known external parameters, internal parameters and distortion coefficients are RT cali , K cali , D cali , in this regard, its internal reference (K cali , D cali ) is provided by the camera manufacturer, and the external reference RT cali Calibration can be performed through the coordinates of the four corner points of the calibration field. The world coordinate system projected by the external parameters is a coordinate system established with the upper left corner of the calibration field as the origin, the left side as the positive X direction, the bottom as the positive Y direction, and the top as the positive Z direction, which is called the calibration coordinate system (Cali).
[0070] Step 1023, identifying the vehicle outline;
[0071] Specifically, the image may be transmitted to the computing unit, and the vehicle ground contour may be identified based on the image recognition algorithm. For the vehicle contour recognition mode under different camera relative positions, Fig. 9 As shown, contour recognition can further calculate the vehicle center position and heading angle (x img ,y img ,θ img ).
[0072] Step 1024, obtaining the center point and heading angle of the vehicle;
[0073] Calibration room camera internal and external parameters RTcali , K cali , D cali And (x obtained in step 1203 img ,y img ,θ img ) can be projected to obtain the vehicle center coordinates and vehicle heading angle (x cali ,y cali , z cali ,θ cali ). After obtaining the vehicle center coordinates and vehicle heading angle in the calibration field coordinate system, the vehicle coordinate system (car) can be established, that is, the vehicle center is the origin, the vehicle heading is the X positive axis, the direction perpendicular to the heading on the left side of the vehicle is the Y positive axis, and the top is the Z positive axis.
[0074] Step 1025, calculating and calibrating the chessboard projection position;
[0075] In this regard, based on the vehicle coordinate system, since the calibration pattern specifications are preset and the relationship between the calibration pattern and the vehicle is preset, the positions of the four projectors can be calculated. Taking the FL projector as an example, Fig.11 As shown in the figure, since the specifications of the calibration pattern are known, the position of the FL projector in the vehicle coordinate system, calibration coordinate system, and image coordinate system can be calculated based on the relationship between the coordinate systems and the internal and external parameters of the camera during calibration. FL ), the same applies to other projectors;
[0076] Step 1026, controlling the movable beam to reach a designated position, and controlling the four regional projectors to project;
[0077] In the specific example, the projector target position Pos can be obtained FL 、Pos FR 、Pos BL 、Pos BR , the calculation unit can control the movement of the movable beams in each area, drive the projector to move, and make the projector in each area reach the target position. After the projector in each area reaches the target position, the calculation unit starts to control the projector to project the calibration pattern, and gradually makes the projection pattern meet the preset by shooting the calibration field image with the camera, identifying the calibration pattern, and adjusting the projector control amount (projection size, projection angle, zoom, trapezoidal correction, etc.). Meeting the preset means that the corner points of the calibration pattern in the captured calibration field are projected to the calibration field coordinate system through the internal and external parameters of the calibration field camera, and further transformed to the vehicle coordinate system. The converted calibration pattern corner point coordinates are consistent with the preset value calibration pattern corner point coordinates or only have a small error.
[0078] Step 1027, vehicle camera calibration.
[0079] Through the above steps 1021 to 1027, in the embodiment of the present application, there is no need to use a mechanical device to correct the movement of the vehicle. The high-definition camera on the top of the calibration room is used to capture images. The computing unit identifies the vehicle outline and its heading, establishes a vehicle coordinate system with the center of the vehicle as the origin, and uses the mechanical equipment projector on the top of the calibration room to move the projector to the appropriate area, and projects the calibration chessboard to the preset standard calibration area based on the vehicle coordinate system, thereby completing the preparation before the camera extrinsic calibration. Through the camera extrinsic calibration method based on image recognition and optical projection in the embodiment of the present application, there is no need for complex and bulky mechanical devices, so the cost is low. In addition, the vehicle coordinate system is actively aligned using camera recognition without adjusting the vehicle alignment preset vehicle coordinate system, and the calibration pattern is easy to modify and adjust, so the calibration is simple and fast and the calibration customization is flexible.
[0080] Corresponding to the above Figure 1 , the embodiment of the present application provides a camera extrinsic calibration device, such as Fig.13 As shown, the device comprises:
[0081] An acquisition module 1302 is used to acquire an image of the vehicle after entering the calibration room through a first camera arranged in the calibration room;
[0082] The first processing module 1304 is used to obtain the ground contour of the vehicle from the image, and identify the ground contour to obtain the center position of the vehicle in the image coordinate system and the first heading angle of the vehicle in the image coordinate system, wherein the image coordinate system is a coordinate system with the center of the vehicle in the image as the coordinate origin;
[0083] An establishing module 1306 is used to establish a vehicle coordinate system based on the center position of the vehicle and the first heading angle in the calibration room, wherein the vehicle coordinate system is a coordinate system with the center position of the vehicle as the origin;
[0084] The second processing module 1308 is used to project the calibration checkerboard to a calibration area based on the vehicle coordinate system, and perform extrinsic calibration on the second camera of the vehicle.
[0085] Through the device of the embodiment of the present application, the calibration room camera is used to take an image of the vehicle, and then the vehicle outline and heading angle are identified based on the image data, and then the vehicle coordinate system is established, the calibration chessboard is projected to the calibration area based on the vehicle coordinate system, and the external parameter calibration of the second camera of the vehicle is performed. In a specific example, it can be based on Figure 2 The shifter in the projector moves the projector to the appropriate area, which can be done as follows Figure 5The method shown is used, and the calibration chessboard is projected to a preset standard calibration area based on the vehicle coordinate system. For example, the heading angle of the current vehicle is not perpendicular to the edge of the calibration room, so not only the position of the projector can be adjusted, but also the angle of the projector can be adjusted, and the calibration chessboard matches the vehicle coordinate system. It can be seen that in the embodiment of the present application, after acquiring the image of the vehicle in the calibration room, a corresponding vehicle coordinate system is established, and then the position and angle of the projection are adjusted so that the calibration chessboard matches the vehicle coordinate system, thereby completing the external parameter calibration of the vehicle camera. It can not only adapt to various scenes more conveniently, but also has a low construction cost and a high calibration efficiency. There is no need to use a mechanical device to move and correct the vehicle as in the prior art. This method is not only cumbersome but also has a high construction cost.
[0086] In an optional implementation manner of an embodiment of the present application, an establishment module in the embodiment of the present application may further include: a first processing unit, used to obtain the center position of the vehicle in the calibration interval coordinate system and the second heading angle of the vehicle in the calibration interval coordinate system based on the intrinsic parameters and extrinsic parameters of the first camera, and the first heading angle; wherein the calibration interval coordinate system is a coordinate system with a corner point of the calibration interval as the origin; an establishment unit, used to establish a vehicle coordinate system with the center position of the vehicle in the calibration interval coordinate system as the origin and the second heading angle.
[0087] In an optional implementation manner of an embodiment of the present application, the first processing module in the embodiment of the present application may further include: an acquisition unit, used to acquire the ground contour of the vehicle at the corresponding angle from images acquired at different angles in the calibration room; a second processing unit, used to identify the ground contours corresponding to different angles to obtain the center position of the vehicle in the image coordinate system and the first heading angle of the vehicle in the image coordinate system.
[0088] In an optional implementation of an embodiment of the present application, the second processing module in the embodiment of the present application may further include: a third processing unit, used to adjust the projector position in the calibration room based on the center coordinates of the vehicle in the vehicle coordinate system and the second heading angle, so that the projector projects the calibration checkerboard to a calibration area based on the vehicle coordinate system.
[0089] In an optional implementation manner of an embodiment of the present application, the third processing unit in the embodiment of the present application may further include: a determination subunit, used to determine the position of the projector in the vehicle coordinate system based on the center coordinates of the vehicle in the vehicle coordinate system and the second heading angle; a processing subunit, used to adjust the position of the projector in the calibration room to make it consistent with the position of the projector in the vehicle coordinate system.
[0090] In an optional implementation manner of an embodiment of the present application, when there are multiple projectors, the determination subunit in the embodiment of the present application is also used to determine the position of each projector in the multiple projections in the vehicle coordinate system based on the mapping relationship between the vehicle coordinate system and the calibration coordinate system, and the external parameters and internal parameters of the first camera.
[0091] In an optional implementation manner of the embodiment of the present application, the second processing module in the embodiment of the present application may further include: an identification unit, used to identify corner points of the image acquired by the second camera to obtain the corresponding first corner point coordinates; a calibration unit, used to perform external parameter calibration of the second camera based on the second corner point coordinates and the first corner point coordinates of the calibration chessboard in the calibration area.
[0092] like Fig.14 As shown, the embodiment of the present application provides an air conditioner control device, including a processor 1411, a communication interface 1412, a memory 1413 and a communication bus 1414, wherein the processor 1411, the communication interface 1412, and the memory 1413 communicate with each other through the communication bus 1414.
[0093] Memory 1413, used for storing computer programs;
[0094] In one embodiment of the present application, the processor 1411 is used to execute the program stored in the memory 1413 to implement the camera extrinsic parameter calibration method provided by any of the aforementioned method embodiments, and the role it plays is similar and will not be repeated here.
[0095] An embodiment of the present application also provides a computer-readable storage medium having a computer program stored thereon. When the computer program is executed by a processor, the steps of the camera extrinsic parameter calibration method provided in any of the aforementioned method embodiments are implemented.
[0096] The device embodiments described above are merely illustrative, wherein the units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the solution of this embodiment.
[0097] Through the description of the above implementation methods, those skilled in the art can clearly understand that each implementation method can be implemented by means of software plus a general hardware platform, and of course, by hardware. Based on this understanding, the above technical solution is essentially or the part that contributes to the relevant technology can be embodied in the form of a software product, and the computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, a disk, an optical disk, etc., including a number of instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to execute the methods described in each embodiment or some parts of the embodiments.
[0098] It should be understood that the terms used herein are only for the purpose of describing specific example embodiments and are not intended to be limiting. Unless the context clearly indicates otherwise, the singular forms "one", "an" and "said" as used herein may also be meant to include plural forms. The terms "include", "comprise", "contain", and "have" are inclusive, and therefore specify the existence of stated features, steps, operations, elements and / or parts, but do not exclude the existence or addition of one or more other features, steps, operations, elements, parts, and / or combinations thereof. The method steps, processes, and operations described herein are not interpreted as necessarily requiring them to be performed in the specific order described or illustrated, unless the execution order is clearly indicated. It should also be understood that additional or alternative steps may be used.
[0099] The foregoing is merely a specific embodiment of the present invention, which enables those skilled in the art to understand or implement the present invention. Various modifications to these embodiments will be apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the present invention. Therefore, the present invention will not be limited to the embodiments shown herein, but rather to the widest scope consistent with the principles and novel features claimed herein.
Claims
1. A camera extrinsic calibration method, characterized in that: include: Acquiring an image of the vehicle after entering the calibration room by a first camera disposed in the calibration room; Acquire a ground contour of the vehicle from the image, and identify the ground contour to obtain a center position of the vehicle in an image coordinate system and a first heading angle of the vehicle in the image coordinate system, wherein the image coordinate system is a coordinate system with the center of the vehicle in the image as a coordinate origin; Establishing a vehicle coordinate system based on the center position of the vehicle and the first heading angle in the calibration room, wherein the vehicle coordinate system is a coordinate system with the center position of the vehicle as the origin; The calibration chessboard is projected onto a calibration area based on the vehicle coordinate system, and an extrinsic calibration is performed on the second camera of the vehicle.
2. The method according to claim 1, characterized in that Establishing a vehicle coordinate system based on the center position of the vehicle and the first heading angle in the calibration room includes: Based on the intrinsic parameters and extrinsic parameters of the first camera and the first heading angle, a center position of the vehicle in a calibration room coordinate system and a second heading angle of the vehicle in the calibration room coordinate system are obtained; wherein the calibration room coordinate system is a coordinate system with a corner point of the calibration room as an origin; The vehicle coordinate system is established with the center position of the vehicle in the calibration coordinate system as the origin and the second heading angle.
3. The method according to claim 1, characterized in that Acquiring a ground contour of the vehicle from the image, and identifying the ground contour to obtain a center position of the vehicle in an image coordinate system and a first heading angle of the vehicle in the image coordinate system includes: Acquire the ground contact contour of the vehicle at a corresponding angle from images acquired at different angles in the calibration room; The ground contours corresponding to different angles are identified to obtain the center position of the vehicle in the image coordinate system and the first heading angle of the vehicle in the image coordinate system.
4. The method according to claim 2, characterized in that: Projecting a calibration checkerboard to a calibration area based on the vehicle coordinate system includes: Based on the center coordinates of the vehicle in the vehicle coordinate system and the second heading angle, the position of the projector in the calibration room is adjusted so that the projector projects the calibration checkerboard into a calibration area based on the vehicle coordinate system.
5. The method according to claim 4, characterized in that Adjusting the position of the projector in the calibration room based on the center coordinates of the vehicle in the vehicle coordinate system and the second heading angle includes: Determine a position of the projector in the vehicle coordinate system based on the center coordinates of the vehicle in the vehicle coordinate system and the second heading angle; The position of the projector in the calibration room is adjusted to be consistent with the position of the projector in the vehicle coordinate system.
6. The method according to claim 5, characterized in that In the case where there are multiple projectors, determining the position of the projector in the vehicle coordinate system based on the center coordinates of the vehicle in the vehicle coordinate system and the second heading angle includes: Based on the mapping relationship between the vehicle coordinate system and the calibration coordinate system, and the external parameters and internal parameters of the first camera, the position of each projector in the multiple projections in the vehicle coordinate system is determined.
7. The method according to claim 1, characterized in that Performing external parameter calibration on the second camera of the vehicle includes: Performing corner point recognition on the image acquired by the second camera to obtain corresponding first corner point coordinates; The extrinsic parameter calibration of the second camera is performed based on the second corner point coordinates and the first corner point coordinates of the calibration chessboard in the calibration area.
8. A camera extrinsic calibration device, characterized in that: include: An acquisition module, used for acquiring an image of the vehicle after entering the calibration room through a first camera arranged in the calibration room; a first processing module, configured to obtain a ground contact contour of the vehicle from the image, and identify the ground contact contour to obtain a center position of the vehicle in an image coordinate system and a first heading angle of the vehicle in the image coordinate system, wherein the image coordinate system is a coordinate system with a center of the vehicle in the image as a coordinate origin; An establishing module, configured to establish a vehicle coordinate system based on the center position of the vehicle and the first heading angle in a calibration room, wherein the vehicle coordinate system is a coordinate system with the center position of the vehicle as an origin; The second processing module is used to project the calibration chessboard to a calibration area based on the vehicle coordinate system and perform extrinsic calibration on the second camera of the vehicle.
9. An electronic device, comprising: at least one communication interface; at least one bus connected to the at least one communication interface; at least one processor coupled to the at least one bus; At least one memory connected to the at least one bus, wherein the processor is configured to execute the camera extrinsic parameter calibration method according to any one of claims 1 to 7.
10. A computer storage medium storing computer executable instructions, wherein the computer executable instructions are used to execute the camera extrinsic parameter calibration method according to any one of claims 1 to 7.
Citation Information
Patent Citations
Target similarity score calculation method and system based on vehicle-mounted camera
CN113792634A
In-situ sensor calibration for autonomous vehicles
US20220179056A1
