Camera extrinsic parameter calibration method and device, electronic device and storage medium

Abstract

The application relates to a camera external parameter calibration method and device, electronic equipment and a storage medium, wherein the method comprises the following steps: acquiring an image of a vehicle after entering a calibration room through a first camera arranged in the calibration room; acquiring a ground wheel contour of the vehicle from the image, and identifying the ground wheel contour to obtain a vehicle center position in an image coordinate system and a first heading angle of the vehicle in the image coordinate system, wherein the image coordinate system is a coordinate system with the center of the vehicle in the image as a coordinate origin; establishing a vehicle coordinate system based on the vehicle center position and the first heading angle in the calibration room, wherein the vehicle coordinate system is a coordinate system with the vehicle center position as an origin; projecting a calibration checkerboard to a calibration area based on the vehicle coordinate system, and performing external parameter calibration on a second camera of the vehicle. Through the application, the problem that the camera external parameter calibration method in the prior art is not only complicated and tedious in the calibration process but also high in construction cost is solved.

Description

Technical Field

[0001] This application relates to the field of camera calibration, and more particularly to a method and apparatus for calibrating camera extrinsic parameters, electronic equipment, and storage medium. Background Technology

[0002] Currently, camera extrinsic parameter calibration systems include a vehicle caliper displacement system and a calibration checkerboard. The vehicle caliper displacement system is used to clamp the wheels of the vehicle to be calibrated and move the vehicle to a fixed position. At this position, the actual vehicle center point coincides with the preset vehicle center point, and the vehicle's heading is consistent with the preset heading. Since the calibration checkerboard remains fixed, the precise positions of all checkerboard points are known in a three-dimensional coordinate system with the vehicle center point as the origin and the vehicle's heading direction as the positive axis. Then, an image containing the calibration checkerboard is captured by a camera on the vehicle. By identifying the checkerboard corner points in the image and matching them with the coordinates of the checkerboard corner points in the actual vehicle coordinate system, point pairs can be formed, allowing the calculation of camera extrinsic parameters. Existing technologies suffer from high construction costs, difficult equipment maintenance, and cumbersome calibration adjustments due to the need for a high-standard, mechanically automated calibration room and the fixed calibration checkerboard. Summary of the Invention

[0003] This application provides a camera extrinsic parameter calibration method, apparatus, electronic device, and storage medium to solve the problem that existing camera extrinsic parameter calibration methods are not only cumbersome in calibration process but also have high construction costs.

[0004] In a first aspect, this application provides a camera extrinsic parameter calibration method, comprising: acquiring an image of a vehicle after it enters the calibration room using a first camera set in the calibration room; acquiring the ground contour of the vehicle from the image, and identifying the ground contour to obtain the vehicle's center position in the image coordinate system and the vehicle's first heading angle in the image coordinate system, wherein the image coordinate system is a coordinate system with the center of the vehicle in the image as the origin; establishing a vehicle coordinate system based on the vehicle's center position and the first heading angle in the calibration room, wherein the vehicle coordinate system is a coordinate system with the vehicle's center position as the origin; projecting a calibration checkerboard onto a calibration area based on the vehicle coordinate system, and calibrating the extrinsic parameters of a second camera of the vehicle.

[0005] Secondly, this application provides a camera extrinsic parameter calibration device, comprising: an acquisition module, configured to acquire an image of a vehicle entering the calibration room using a first camera disposed within the calibration room; a first processing module, configured to acquire the grounding contour of the vehicle from the image, and identify the grounding contour to obtain the vehicle's center position in an image coordinate system and the vehicle's first heading angle in the image coordinate system, wherein the image coordinate system is a coordinate system with the center of the vehicle in the image as the origin; an establishment module, configured to establish a vehicle coordinate system based on the vehicle's center position and the first heading angle within the calibration room, wherein the vehicle coordinate system is a coordinate system with the vehicle's center position as the origin; and a second processing module, configured to project a calibration checkerboard onto a calibration area based on the vehicle coordinate system, and perform extrinsic parameter calibration on a second camera of the vehicle.

[0006] Thirdly, this application provides an electronic device, comprising: at least one communication interface; at least one bus connected to the at least one communication interface; at least one processor connected to the at least one bus; and at least one memory connected to the at least one bus, wherein the processor is configured to perform the camera extrinsic parameter calibration method described in the first aspect of this application.

[0007] Fourthly, this application also provides a computer storage medium storing computer-executable instructions for performing the camera extrinsic parameter calibration method described in the first aspect of this application.

[0008] Compared with the prior art, the technical solution provided in this application has the following advantages: In this application embodiment, a calibration camera is used to capture images of the vehicle, and then the vehicle outline and its heading angle are identified based on the image data. A vehicle coordinate system is then established, and a calibration checkerboard is projected onto the calibration area based on the vehicle coordinate system. The extrinsic parameters of the vehicle's second camera are then calibrated. Therefore, in this application embodiment, by acquiring images of the vehicle in the calibration area, establishing a corresponding vehicle coordinate system, and then adjusting the position and angle of the projection to match the calibration checkerboard with the vehicle coordinate system, the extrinsic parameters of the vehicle camera are calibrated. This not only allows for convenient adaptation to various scenarios but also has lower construction costs and higher calibration efficiency. It eliminates the need for mechanical devices to move and correct the vehicle, as is common in the prior art, which is cumbersome and costly to construct. Attached Figure Description

[0009] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments consistent with the invention and, together with the description, serve to explain the principles of the invention.

[0010] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, for those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0011] One or more embodiments are illustrated by way of example with reference numerals in the accompanying drawings. These illustrations do not constitute a limitation on the embodiments. Elements with the same reference numerals in the drawings are denoted as similar elements. Unless otherwise stated, the figures in the drawings are not to be limited by scale.

[0012] Figure 1 A flowchart illustrating a camera extrinsic parameter calibration method provided in this application embodiment;

[0013] Figure 2 This is a schematic diagram of the hardware system structure of the calibration room in a specific example of this application;

[0014] Figure 3 This is a schematic diagram illustrating the acquisition of vehicle images in a calibration room, as shown in a specific example of this application.

[0015] Figure 4 This is a schematic diagram of the projection areas of each projector in the calibration room, as shown in a specific example of this application.

[0016] Figure 5 This is a schematic diagram illustrating the adjustment of the projector in a specific example of this application;

[0017] Figure 6 This is a schematic diagram of a chessboard marked in a specific example of this application;

[0018] Figure 7 This is a schematic diagram of the projected checkerboard located in the upper left corner of a specific example in this application;

[0019] Figure 8 This is a schematic diagram of a checkerboard projected by four projectors in a specific example of this application;

[0020] Figure 9 This is a schematic diagram of the grounding profile of a vehicle in a specific example of this application;

[0021] Figure 10 This is a schematic diagram of a vehicle entering the calibration chamber, as shown in a specific example of this application.

[0022] Figure 11 This is a schematic diagram illustrating the adjustment of the checkerboard based on the vehicle coordinate system in a specific example of this application;

[0023] Figure 12 This is a flowchart of a camera extrinsic parameter calibration method based on image recognition and optical projection, as described in an embodiment of this application.

[0024] Figure 13 A flowchart of a camera extrinsic parameter calibration device provided in an embodiment of this application;

[0025] Figure 14 A flowchart of an electronic device provided in an embodiment of this application. Detailed Implementation

[0026] To make the objectives, technical solutions, and advantages of the embodiments of this application clearer, the technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0027] The following disclosure provides numerous different embodiments or examples for implementing various structures of the invention. To simplify the disclosure, specific examples of components and arrangements are described below. These are merely examples and are not intended to limit the scope of the invention. Furthermore, reference numerals and / or letters may be repeated in different examples. Such repetition is for simplification and clarity and does not in itself indicate a relationship between the various embodiments and / or arrangements discussed.

[0028] First, the relevant terms involved in the embodiments of this application will be explained;

[0029] Camera extrinsic calibration: Camera extrinsic calibration is the process of determining the camera's position and orientation in the world coordinate system. Based on extrinsic parameters, a mapping between the image coordinate system and the world coordinate system can be established. Specifically, it involves calculating two main sets of parameters: the rotation matrix (R) and the translation vector (T). These parameters describe the rotation and translation relationship between the camera coordinate system and the world coordinate system. Extrinsic parameters are mainly used in the following applications: 1) Spatial positioning: In many applications, such as robot navigation, augmented reality, and 3D reconstruction, understanding the camera's position in space is crucial for correctly interpreting image data; 2) Object measurement: Through calibration, two-dimensional points in an image can be converted into three-dimensional points in the world coordinate system, thereby enabling the measurement of the size and shape of objects.

[0030] External parameter calibration process: The basic steps of external parameter calibration include: 1) Selecting a calibration target: Using a calibration board or object with known geometric features, such as a checkerboard, dot pattern, or other objects with identifiable features; 2) Capturing images: Taking pictures of the calibration target from different angles and positions to collect enough data for calculation; 3) Feature detection: Detecting feature points of the calibration target in the image, such as corner points or center points; 4) Establishing correspondence: Establishing correspondences between feature points in the image and corresponding 3D points in the world coordinate system; 5) Parameter estimation: Using mathematical models and optimization algorithms (such as least squares method) to estimate the rotation matrix and translation vector; 6) Verification and optimization: Verifying the accuracy of the calibration results by comparing the predicted image points with the actually observed points, and making necessary adjustments.

[0031] Camera extrinsic parameter factory calibration: Camera extrinsic parameter factory calibration differs from other types of extrinsic parameter calibration. Its main characteristics are stability, speed, and high accuracy. It is usually used for factory calibration during camera assembly, such as the extrinsic parameter calibration of a camera on a vehicle after assembly.

[0032] To address the shortcomings of existing camera extrinsic parameter calibration methods, which are not only cumbersome but also costly, this application provides a camera extrinsic parameter calibration method, such as... Figure 1 As shown, the steps of this method include:

[0033] Step 101: Use the first camera set in the calibration chamber to acquire an image of the vehicle after it enters the calibration chamber;

[0034] In a specific example, the calibration intervals in this application embodiment are associated with the corresponding hardware systems, such as... Figure 2 As shown, the hardware system includes a first camera, a displacement device, a projector, a computing unit, and a storage unit. The displacement device and the projector are mechanically connected, and the computing unit is data-connected to each of the other units. Further, in this embodiment, the first camera can be installed at the top center of the calibration room, capable of completely capturing all areas within the calibration room, as shown in Figure 3. Since the intrinsic and extrinsic parameters of the first camera are known, the coordinates of any point on the ground within the calibration room can be accurately measured using these parameters. The computing unit of the hardware system processes the images captured by the camera, including recognizing the vehicle outline, calculating the coordinates of the vehicle's center point, calculating the vehicle's heading, recognizing the ground calibration pattern, and controlling the displacement device and the projector based on the recognition results and the preset calibration pattern. The storage unit in the hardware system stores the images captured by the camera, the preset projection pattern, the camera's intrinsic and extrinsic parameters, and necessary intermediate data such as the recognition program.

[0035] Furthermore, in this embodiment, the displacement device is installed on the top of the calibration room, and a projector is mounted on the intersection of its horizontal and vertical axes via rollers. For example, in this embodiment, there are four projectors, each used for projector displacement in one of the four square areas of the calibration room. The projectors are named FL (FrontLeft), FR (FrontRight), BL (BackLeft), and BR (BackRight) projectors according to their location within the area, and the corresponding areas are the active areas of the FL, FR, BL, and BR projectors. The installation of the displacement device on the top of the calibration room is as follows... Figure 4 As shown, the mounting beams are the infrastructure on the ceiling of the calibration room used to fix the mechanical structure. The mounting beams are arranged in a grid pattern, dividing the entire ceiling of the calibration room into four squares. Each square houses a displacement device, and each displacement device consists of two movable beams, one horizontal and one vertical. The horizontal beam can be moved laterally via an electrical device, and the vertical beam can be moved longitudinally via an electrical device. The projector is mounted at the intersection of the horizontal and vertical beams via mechanical rollers. Figure 4 The number of projectors shown is 4, which is just an example. The number of projectors can be set according to actual needs, such as 2 or 6 projectors.

[0036] Furthermore, by moving the horizontal and vertical movable beams, the displacement device can move the projector to any position within the grid, such as... Figure 5 As shown. The projector projects the calibration pattern sent by the computing unit onto the ground and adjusts the projection state according to feedback control commands (projection size, projection angle, scaling, keystone correction, etc.). The calibration pattern obtains projection parameters through joint calculation by the camera and computer, and projects a projection result that conforms to the preset parameters, as shown in the figure. Figure 6 As shown. The calibration pattern projection consists of projection patterns from four projection devices, as follows. Figure 7 The projected pattern covering the central FL area is projected by an FL projector; four projection devices are responsible for areas such as... Figure 8 As shown, the projected patterns of the projector partially overlap on the ground, which can help align the calibration patterns.

[0037] Step 102: Obtain the grounding contour of the vehicle from the image, and identify the grounding contour to obtain the vehicle center position and the vehicle's first heading angle in the image coordinate system. The image coordinate system is a coordinate system with the center of the vehicle in the image as the origin.

[0038] In the embodiments of this application, the algorithm for identifying the grounding contour can be configured according to actual needs, specifically employing algorithms such as key point recognition and semantic segmentation. Figure 9As shown, the ground contours of vehicles are obtained from different angles in the image. For example, the ground contour in the upper left corner can only be identified from the left and the top and bottom vehicle contours because it is identified from the upper left corner. The vehicle contour on the right side cannot be identified. The identification results are similar for ground contours in other directions.

[0039] Step 103: Establish a vehicle coordinate system based on the vehicle center position and the first heading angle within the calibration room, wherein the vehicle coordinate system is a coordinate system with the vehicle center position as the origin;

[0040] In a specific example, the X-axis of the vehicle coordinate system is aligned with the vehicle's orientation. Therefore, if a vehicle entering the calibration chamber stops at the center of the calibration chamber and its driving direction is perpendicular to the edge of the calibration chamber, the XYZ axes of the vehicle coordinate system with the vehicle's center position as the origin are parallel to those of the calibration chamber coordinate system, except for the difference in the origin position.

[0041] Step 104: Project the calibration checkerboard onto the calibration area based on the vehicle coordinate system, and perform extrinsic parameter calibration on the vehicle's second camera.

[0042] For the grounding profile, in a specific example such as Figure 8 As shown, taking a total of 4 projectors as an example, the checkerboard pattern in the calibration room includes the FL area, FR area, BL area and BR area. There are overlapping areas between adjacent areas, and each area has its own independent area.

[0043] Through steps 101 to 104 above, the vehicle is imaged using a calibration camera. Then, based on the image data, the vehicle's outline and heading angle are identified, and a vehicle coordinate system is established. The calibration checkerboard is projected onto the calibration area based on the vehicle coordinate system, and the extrinsic parameters of the vehicle's second camera are calibrated. In a specific example, this can be based on... Figure 2 The displacement device moves the projector to the appropriate area, specifically as follows: Figure 5 The method shown projects a calibration checkerboard onto a preset standard calibration area based on the vehicle coordinate system. Since the vehicle's heading angle is not perpendicular to the edge of the calibration area, not only the projector position but also its angle can be adjusted to match the calibration checkerboard with the vehicle coordinate system. Therefore, in this embodiment, after acquiring an image of the vehicle in the calibration area, a corresponding vehicle coordinate system is established, and the projection position and angle are adjusted to match the calibration checkerboard with the vehicle coordinate system, thus completing the extrinsic parameter calibration of the vehicle camera. This method is not only conveniently adaptable to various scenarios but also has low construction costs and high calibration efficiency. It eliminates the need for mechanical devices to move and correct the vehicle, as is common in existing technologies, which are cumbersome and costly to implement.

[0044] In this embodiment, establishing the vehicle coordinate system requires obtaining the vehicle's center position and heading angle. Since the vehicle coordinate system is established with the vehicle's center position as the origin and the heading angle direction as the X-axis, this ensures the calibration checkerboard matches the vehicle's current position during subsequent calibration, leading to more accurate vehicle camera extrinsic parameter calibration. Therefore, the method of establishing the vehicle coordinate system based on the vehicle's center position and the first heading angle within the calibration interval, as mentioned in step 103, can further include the following steps:

[0045] Step 11: Based on the intrinsic and extrinsic parameters of the first camera and the first heading angle, obtain the vehicle center position in the calibration coordinate system and the vehicle's second heading angle in the calibration coordinate system; wherein, the calibration coordinate system is a coordinate system with the corner point of the calibration space as the origin;

[0046] Step 12: Establish a vehicle coordinate system with the vehicle center position in the calibration coordinate system as the origin and the second heading angle.

[0047] In this embodiment, the calibration coordinate system is established with the upper left corner of the calibration space as the origin, the left side as the positive X direction, the lower side as the positive Y direction, and the upper side as the positive Z direction. It is evident that in this embodiment, after obtaining the vehicle's center position and heading angle from the image, it is necessary to transform them to the calibration coordinate system to obtain the corresponding center position coordinates and heading angle, because the coordinates in the calibration coordinate system correspond to the position in the real environment. Therefore, a vehicle coordinate system is established based on the obtained vehicle center position and heading angle. Based on this vehicle coordinate system, the image captured by the vehicle camera can be compared with the actual environment.

[0048] In this embodiment, the ground contact profile of the vehicle can be obtained from various angles to facilitate quick and easy determination of the vehicle's position and heading angle. Therefore, the method of obtaining the ground contact profile of the vehicle from the image and identifying the ground contact profile to obtain the vehicle's center position and first heading angle in the image coordinate system in step 102 can further include:

[0049] Step 21: Obtain the grounding profile of the vehicle at the corresponding angle from the images obtained from different angles within the calibration room;

[0050] Step 22: Identify the grounding contours corresponding to different angles to obtain the vehicle center position and the vehicle's first heading angle in the image coordinate system.

[0051] In this regard, in specific examples such as Figure 9As shown, the vehicle's ground contact contour is acquired from various angles in the image. For example, the ground contact contour in the upper left corner can only be identified from the left and top / bottom sides, not the right side. Similar results are obtained for ground contact contours in other directions. By acquiring the vehicle's ground contact contour from various directions, the vehicle's heading angle and center position in the image coordinate system are determined. This allows for subsequent transformation of the vehicle's heading angle and center position based on the relationship between the image coordinate system and the calibration coordinate system. Then, a vehicle coordinate system is established based on this transformation for subsequent calibration of the vehicle's camera extrinsic parameters.

[0052] In this embodiment, the extrinsic parameters of the vehicle camera are calibrated after establishing a vehicle coordinate system. Establishing the vehicle coordinate system requires obtaining the position of the vehicle's center and its heading angle in the real environment. After establishing the vehicle coordinate system, the calibration checkerboard needs to be adjusted to ensure more accurate subsequent calibration of the vehicle camera's extrinsic parameters. Therefore, the method of projecting the calibration checkerboard onto the calibration area based on the vehicle coordinate system, as mentioned in step 104, can further include the following steps:

[0053] Step 31: Based on the vehicle's center coordinates and the second heading angle in the vehicle coordinate system, adjust the position of the projector in the calibration room so that the projector projects the calibration checkerboard onto the calibration area based on the vehicle coordinate system.

[0054] In this specific example, for the calibration chessboard grid in the calibration coordinate system, such as... Figure 8 As shown in Figure 10, however, in reality, after a vehicle enters the calibration bay, its heading angle is not necessarily perpendicular to the edge of the calibration bay. Therefore, the calibration checkerboard pattern that matches the calibration bay's coordinate system cannot be directly used. The calibration checkerboard pattern needs to be adjusted after establishing the vehicle coordinate system. The adjusted checkerboard pattern is shown below. Figure 11 As shown, by adjusting the position of the projector, the position and orientation of the calibration checkerboard grid are changed. This change in the position and orientation of the calibration checkerboard grid is related to the vehicle coordinate system.

[0055] Furthermore, the method of adjusting the projector position in the calibration room based on the vehicle's center coordinates and the second heading angle in the vehicle coordinate system, as mentioned in step 31 above, may further include:

[0056] Step 41: Determine the position of the projector in the vehicle coordinate system based on the center coordinates of the vehicle and the second heading angle.

[0057] Step 42: Adjust the position of the projector in the calibration room to make it consistent with the position of the projector in the vehicle coordinate system.

[0058] Regarding steps 41 and 42 above, in a specific example, such as Figure 11 As shown, after establishing the vehicle coordinate system, the calibration checkerboard needs to be adjusted. Taking a set of 4 projectors as an example, the original calibration checkerboard is as follows: Figure 8 As shown, each projector corresponds to a calibration checkerboard grid. Therefore, after establishing the vehicle coordinate system, it is necessary to... Figure 8 The calibration chessboard grid in the image is adjusted, and the adjusted calibration chessboard grid is as follows: Figure 11 As shown, the calibration checkerboard can be adjusted by adjusting the position and angle of the projection, and then the vehicle camera extrinsic parameter calibration can be performed. With this application, there is no need to use mechanical devices to move and correct the vehicle as in the prior art, making the vehicle camera extrinsic parameter calibration more convenient and faster.

[0059] As can be seen, when there are multiple projectors, the method of determining the position of the projector in the vehicle coordinate system based on the center coordinates of the vehicle and the second heading angle in the vehicle coordinate system involved in step 41 above can further include: determining the position of each projector in the vehicle coordinate system based on the mapping relationship between the vehicle coordinate system and the calibration coordinate system, as well as the extrinsic and intrinsic parameters of the first camera.

[0060] Taking a scenario with 4 projectors as an example, the position and angle of each projector need to be determined in the vehicle coordinate system, and then... Figure 5 The position and angle of the projector are adjusted as shown to ensure that the calibration checkerboard projected by the projector is as shown. Figure 11 As shown, camera extrinsic parameter calibration can then be performed, which is simpler and more efficient than existing technologies.

[0061] In this embodiment of the application, the method for calibrating the extrinsic parameters of the vehicle's second camera in step 104 above may further include:

[0062] Step 51: Perform corner point recognition on the image acquired by the second camera to obtain the coordinates of the corresponding first corner point;

[0063] Step 52: Perform extrinsic parameter calibration of the second camera based on the coordinates of the second corner point and the first corner point within the calibration area of ​​the calibration checkerboard.

[0064] In this specific example, the vehicle's camera is controlled to start capturing images, and corner point recognition is performed on the images captured by the vehicle's camera. The identified corner points (Pt1) on the image are then... img Pt2 img Pt3 img ...) and the corresponding point (Pt1) in the known vehicle coordinate system. car Pt2 car Pt3 car...) form point pairs, and based on these point pairs, camera extrinsic parameters can be further calculated.

[0065] The present application will now be explained in detail with reference to specific embodiments of the present application. These specific embodiments provide a camera extrinsic parameter calibration method based on image recognition and optical projection, such as... Figure 12 As shown, the steps of this method include:

[0066] Step 1021, Vehicle enters the site;

[0067] The vehicle is driven from the outside into the middle of the calibration field. In this embodiment, the vehicle will be parked in the middle area of ​​the calibration field with a non-centered and non-perpendicular heading. In addition, the vehicle must be parked at a certain distance from the four walls to ensure that the calibration pattern can be completely projected around the vehicle.

[0068] Step 1022: The top camera takes pictures of vehicles inside the factory;

[0069] In the embodiments of this application, combined with Figure 2 The hardware system in the calibration field uses a high-definition camera (hereinafter referred to as Cam) at the top of the calibration field. cali The footage captured is as follows Figure 10 As shown. In the embodiments of this application, Cam cali The known external parameters, internal parameters, and distortion coefficients are RT. cali K cali D cali In response, its internal reference (K) cali D cali Provided by the camera manufacturer, external parameters RT cali The calibration can be performed using the coordinates of the four corner points of the calibration field. The world coordinate system projected by the external parameters is a coordinate system established with the upper left corner of the calibration field as the origin, the left side as the positive X direction, the lower side as the positive Y direction, and the upper side as the positive Z direction. This is called the calibration coordinate system (Cali).

[0070] Step 1023: Identify the vehicle outline;

[0071] Specifically, this could involve inputting an image into a computing unit, which could then identify the vehicle's ground contour based on an image recognition algorithm. This would allow for vehicle contour recognition patterns under different camera relative positions, such as... Figure 9 As shown, contour recognition can further calculate the vehicle's center position and heading angle (x) in the image coordinate system. img y img θ img ).

[0072] Step 1024: Obtain the vehicle's center point and heading angle;

[0073] Calibration of camera internal and external parameters RTcali K cali D cali and the (x) obtained in step 1203 img y img θ img The vehicle's center coordinates and heading angle (x) can be projected into the calibration field coordinate system. cali y cali , z cali θ cali After obtaining the vehicle center coordinates and vehicle heading angle in the calibration field coordinate system, a vehicle coordinate system (car) can be established, with the vehicle center as the origin, the vehicle heading as the positive X-axis, the left side of the vehicle perpendicular to the heading as the positive Y-axis, and the top as the positive Z-axis.

[0074] Step 1025: Calculate the projected position of the calibration chessboard grid;

[0075] Based on the vehicle coordinate system, and given the preset calibration pattern specifications and the preset relationship between the calibration pattern and the vehicle, the positions of the four projectors can be calculated. Taking an FL projector as an example... Figure 11 As shown, since the calibration pattern specifications are known, the position (Pos) of the FL projector in the vehicle coordinate system, calibration coordinate system, and image coordinate system can be determined based on the relationships between various coordinate systems and the camera's intrinsic and extrinsic parameters during calibration. FL The same applies to other projectors;

[0076] Step 1026: Control the movable beam to reach the designated position and control the four area projectors to project;

[0077] In a specific example, the target position Pos of the projector can be obtained. FL Pos FR Pos BL Pos BR The computing unit controls the movement of the movable beams in each area, which in turn moves the projectors, ensuring that each projector reaches its target position. After the projectors in each area reach their target positions, the computing unit begins to control the projectors to project calibration patterns. This is achieved by capturing calibration field images with a camera, identifying the calibration patterns, and adjusting the projector control parameters (projection size, projection angle, scaling, keystone correction, etc.) to gradually make the projected pattern conform to the preset values. Conforming to the preset values ​​means that the corner points of the calibration patterns in the captured calibration field are projected onto the calibration field coordinate system through the intrinsic and extrinsic parameters of the calibration field camera, and further transformed into the vehicle coordinate system. The resulting corner point coordinates of the calibration patterns are consistent with or have only minor errors compared to the preset corner point coordinates.

[0078] Step 1027, vehicle camera calibration.

[0079] Through steps 1021 to 1027 above, in this embodiment of the application, there is no need to use mechanical devices to move and correct the vehicle. A high-definition camera on top of the calibration room captures images, a computing unit identifies the vehicle's outline and heading, and establishes a vehicle coordinate system with the vehicle's center as the origin. A mechanical device on top of the calibration room projects the image onto an appropriate area, projecting the calibration checkerboard pattern onto a preset standard calibration area based on the vehicle coordinate system, thus completing the preparation for camera extrinsic parameter calibration. The camera extrinsic parameter calibration method based on image recognition and optical projection in this embodiment of the application eliminates the need for complex and cumbersome mechanical devices, resulting in low cost. Furthermore, the camera actively aligns with the vehicle coordinate system without needing to adjust the vehicle to a preset vehicle coordinate system, and the calibration pattern is easily modified and adjusted; therefore, calibration is simple, fast, and flexible in customization.

[0080] Corresponding to the above Figure 1 This application provides a camera extrinsic parameter calibration device, such as... Figure 13 As shown, the device includes:

[0081] The acquisition module 1302 is used to acquire images of the vehicle after it enters the calibration room using a first camera set in the calibration room;

[0082] The first processing module 1304 is used to obtain the grounding contour of the vehicle from the image, and to identify the grounding contour to obtain the vehicle center position and the vehicle's first heading angle in the image coordinate system. The image coordinate system is a coordinate system with the center of the vehicle in the image as the origin.

[0083] Module 1306 is used to establish a vehicle coordinate system based on the vehicle center position and the first heading angle within the calibration space, wherein the vehicle coordinate system is a coordinate system with the vehicle center position as the origin;

[0084] The second processing module 1308 is used to project the calibration checkerboard onto the calibration area based on the vehicle coordinate system and to perform extrinsic parameter calibration on the vehicle's second camera.

[0085] The apparatus of this application uses a calibration camera to capture images of a vehicle, then identifies the vehicle's outline and heading angle based on the image data, thereby establishing a vehicle coordinate system. A calibration checkerboard pattern is projected onto a calibration area based on the vehicle coordinate system, and extrinsic parameter calibration is performed on the vehicle's second camera. In a specific example, it can be based on... Figure 2 The displacement device moves the projector to the appropriate area, specifically as follows: Figure 5The method shown projects a calibration checkerboard onto a preset standard calibration area based on the vehicle coordinate system. Since the vehicle's heading angle is not perpendicular to the edge of the calibration area, not only the projector position but also its angle can be adjusted to match the calibration checkerboard with the vehicle coordinate system. Therefore, in this embodiment, after acquiring an image of the vehicle in the calibration area, a corresponding vehicle coordinate system is established, and the projection position and angle are adjusted to match the calibration checkerboard with the vehicle coordinate system, thus completing the extrinsic parameter calibration of the vehicle camera. This method is not only conveniently adaptable to various scenarios but also has low construction costs and high calibration efficiency. It eliminates the need for mechanical devices to move and correct the vehicle, as is common in existing technologies, which are cumbersome and costly to implement.

[0086] In an optional embodiment of this application, the establishment module may further include: a first processing unit, used to obtain the vehicle center position in the calibration coordinate system and the vehicle's second heading angle in the calibration coordinate system based on the intrinsic and extrinsic parameters of the first camera and the first heading angle; wherein the calibration coordinate system is a coordinate system with the corner point of the calibration as the origin; and an establishment unit, used to establish a vehicle coordinate system with the vehicle center position in the calibration coordinate system as the origin and the second heading angle.

[0087] In an optional embodiment of this application, the first processing module may further include: an acquisition unit, used to acquire the grounding profile of the vehicle at a corresponding angle from images acquired at different angles within the calibration room; and a second processing unit, used to identify the grounding profiles corresponding to different angles to obtain the vehicle center position in the image coordinate system and the vehicle's first heading angle in the image coordinate system.

[0088] In an optional embodiment of this application, the second processing module in this application embodiment may further include: a third processing unit, used to adjust the position of the projector in the calibration room based on the center coordinates of the vehicle in the vehicle coordinate system and the second heading angle, so that the projector projects the calibration checkerboard onto the calibration area based on the vehicle coordinate system.

[0089] In an optional embodiment of this application, the third processing unit in this application embodiment may further include: a determining subunit, used to determine the position of the projector in the vehicle coordinate system based on the center coordinates of the vehicle and the second heading angle; and a processing subunit, used to adjust the position of the projector in the calibration room to make it consistent with the position of the projector in the vehicle coordinate system.

[0090] In an optional embodiment of this application, when there are multiple projectors, the determining subunit in this application embodiment is further used to determine the position of each projector in the vehicle coordinate system based on the mapping relationship between the vehicle coordinate system and the calibration coordinate system, as well as the extrinsic and intrinsic parameters of the first camera.

[0091] In an optional embodiment of this application, the second processing module may further include: an identification unit, used to identify corner points in the image acquired by the second camera to obtain the corresponding first corner point coordinates; and a calibration unit, used to calibrate the extrinsic parameters of the second camera based on the second corner point coordinates and the first corner point coordinates within the calibration area of ​​the calibration checkerboard.

[0092] like Figure 14 As shown in the figure, this application provides an air conditioner control device, including a processor 1411, a communication interface 1412, a memory 1413, and a communication bus 1414. The processor 1411, communication interface 1412, and memory 1413 communicate with each other via the communication bus 1414.

[0093] Memory 1413 is used to store computer programs;

[0094] In one embodiment of this application, when the processor 1411 executes the program stored in the memory 1413, it implements the camera extrinsic parameter calibration method provided in any of the aforementioned method embodiments, and its function is similar, so it will not be described again here.

[0095] This application also provides a computer-readable storage medium storing a computer program thereon, which, when executed by a processor, implements the steps of the camera extrinsic parameter calibration method provided in any of the foregoing method embodiments.

[0096] The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs.

[0097] Through the above description of the embodiments, those skilled in the art can clearly understand that each embodiment can be implemented using software plus a general-purpose hardware platform, or of course, using hardware. Based on this understanding, the above technical solutions, in essence or the parts that contribute to the related technology, can be embodied in the form of a software product. This computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, magnetic disk, optical disk, etc., and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute the methods described in the various embodiments or some parts of the embodiments.

[0098] It should be understood that the terminology used herein is for the purpose of describing particular exemplary embodiments only and is not intended to be limiting. Unless the context clearly indicates otherwise, the singular forms “a,” “an,” and “described” as used herein may also include the plural forms. The terms “comprising,” “including,” “containing,” and “having” are inclusive and therefore indicate the presence of the stated features, steps, operations, elements, and / or components, but do not exclude the presence or addition of one or more other features, steps, operations, elements, components, and / or combinations thereof. The method steps, processes, and operations described herein are not construed as requiring them to be performed in a particular order described or illustrated unless the order of performance is explicitly indicated. It should also be understood that additional or alternative steps may be used.

[0099] The above description is merely a specific embodiment of the present invention, enabling those skilled in the art to understand or implement the invention. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the invention. Therefore, the present invention is not to be limited to the embodiments shown herein, but is to be accorded the widest scope consistent with the principles and novel features claimed herein.

Claims

1. A method for calibrating camera extrinsic parameters, characterized in that, include: The first camera installed in the calibration chamber acquires images of the vehicle after it enters the calibration chamber. The grounding contour of the vehicle is obtained from the image, and the grounding contour is identified to obtain the vehicle center position in the image coordinate system and the first heading angle of the vehicle in the image coordinate system, wherein the image coordinate system is a coordinate system with the center of the vehicle in the image as the origin. A vehicle coordinate system is established based on the vehicle center position and the first heading angle within the calibration space, wherein the vehicle coordinate system is a coordinate system with the vehicle center position as the origin; The calibration checkerboard is projected onto the calibration area based on the vehicle coordinate system, and the extrinsic parameters of the vehicle's second camera are calibrated. The process of establishing a vehicle coordinate system based on the vehicle center position within the calibration interval and the first heading angle includes: obtaining the vehicle center position in the calibration interval coordinate system and the second heading angle of the vehicle in the calibration interval coordinate system based on the intrinsic and extrinsic parameters of the first camera and the first heading angle; wherein the calibration interval coordinate system is a coordinate system with the corner point of the calibration interval as the origin; and the vehicle coordinate system is established with the vehicle center position in the calibration interval coordinate system as the origin and the second heading angle. The process of projecting the calibration checkerboard onto a calibration area based on the vehicle coordinate system includes: adjusting the position of the projector in the calibration room based on the center coordinates of the vehicle in the vehicle coordinate system and the second heading angle, so that the projector projects the calibration checkerboard onto the calibration area based on the vehicle coordinate system.

2. The method according to claim 1, characterized in that, Obtaining the ground contact profile of the vehicle from the image, and identifying the ground contact profile to obtain the vehicle's center position in the image coordinate system and the vehicle's first heading angle in the image coordinate system includes: The grounding profile of the vehicle at the corresponding angle is obtained from images acquired from different angles within the calibration room; The grounding profiles corresponding to different angles are identified to obtain the vehicle center position in the image coordinate system and the vehicle's first heading angle in the image coordinate system.

3. The method according to claim 1, characterized in that, Adjusting the position of the projector in the calibration room based on the center coordinates of the vehicle in the vehicle coordinate system and the second heading angle includes: Based on the center coordinates of the vehicle in the vehicle coordinate system and the second heading angle, the position of the projector in the vehicle coordinate system is determined; Adjust the position of the projector within the calibration room to align it with the position of the projector in the vehicle coordinate system.

4. The method according to claim 3, characterized in that, When there are multiple projectors, determining the position of the projector in the vehicle coordinate system based on the center coordinates of the vehicle in the vehicle coordinate system and the second heading angle includes: Based on the mapping relationship between the vehicle coordinate system and the calibration coordinate system, as well as the extrinsic and intrinsic parameters of the first camera, the position of each projector in the plurality of projections in the vehicle coordinate system is determined.

5. The method according to claim 1, characterized in that, The external parameter calibration of the second camera of the vehicle includes: Corner point recognition is performed on the image acquired by the second camera to obtain the coordinates of the corresponding first corner point; The extrinsic parameters of the second camera are calibrated based on the coordinates of the second corner point and the first corner point within the calibration area of ​​the calibration chessboard.

6. A camera extrinsic parameter calibration device, characterized in that, include: The acquisition module is used to acquire images of the vehicle after it enters the calibration room using a first camera set in the calibration room; The first processing module is used to obtain the grounding contour of the vehicle from the image, and to identify the grounding contour to obtain the vehicle center position in the image coordinate system and the first heading angle of the vehicle in the image coordinate system, wherein the image coordinate system is a coordinate system with the center of the vehicle in the image as the origin. A module is established to establish a vehicle coordinate system based on the vehicle center position and the first heading angle within the calibration space, wherein the vehicle coordinate system is a coordinate system with the vehicle center position as the origin; The second processing module is used to project the calibration checkerboard onto the calibration area based on the vehicle coordinate system and to perform extrinsic parameter calibration on the vehicle's second camera. The establishment module includes: a first processing unit, used to obtain the vehicle center position in the calibration coordinate system and the vehicle's second heading angle in the calibration coordinate system based on the intrinsic and extrinsic parameters of the first camera and the first heading angle; wherein the calibration coordinate system is a coordinate system with the corner point of the calibration space as the origin; and an establishment unit, used to establish a vehicle coordinate system with the vehicle center position in the calibration coordinate system as the origin and the second heading angle. The second processing module includes a third processing unit, which adjusts the position of the projector in the calibration room based on the center coordinates of the vehicle in the vehicle coordinate system and the second heading angle, so that the projector projects the calibration checkerboard onto the calibration area based on the vehicle coordinate system.

7. An electronic device, comprising: At least one communication interface; At least one bus connected to the at least one communication interface; At least one processor connected to the at least one bus; At least one memory connected to the at least one bus, wherein the processor is configured to perform the camera extrinsic calibration method according to any one of claims 1 to 5.

8. A computer storage medium storing computer-executable instructions for performing the camera extrinsic calibration method according to any one of claims 1 to 5.

Images