A camera and vehicle extrinsic parameter calibration method and device, electronic equipment, and storage medium

Abstract

The application provides a camera and vehicle extrinsic parameter calibration method and device, electronic equipment and storage medium, wherein the camera and vehicle extrinsic parameter calibration method comprises the following steps: acquiring the intrinsic parameter of the camera and the initial extrinsic parameter between the camera and the vehicle, and acquiring a continuous original image through the camera; processing the original image to obtain pixel coordinate points of lane lines, converting the pixel coordinate points into camera coordinate points in a camera coordinate system according to the intrinsic parameter matrix; converting the original image into a bird's eye view image according to the initial extrinsic parameter, extracting the lane lines on the bird's eye view image, and converting the lane lines into a cubic curve equation in a vehicle coordinate system; establishing data correlation between the camera coordinate points and the cubic curve equation, and calculating the target extrinsic parameter between the camera and the vehicle. In the scheme, after the target extrinsic parameter between the camera and the vehicle is calculated, the target extrinsic parameter is calibrated to meet the recognition requirement of the sensor in the intelligent driving at present, and the intelligent driving of the vehicle is facilitated.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of camera and vehicle exterior parameter calibration in intelligent driving, in particular to a camera and vehicle exterior parameter calibration method and device, electronic equipment and computer readable storage medium. BACKGROUND

[0002] With the development of intelligent driving technology, the intelligent driving function of the vehicle is increasingly perfect, and the sensors installed on the vehicle are also increasing. The camera is a very common sensor on the vehicle at present. In order to ensure the normal operation of the intelligent driving function, the camera and vehicle exterior parameter calibration is a necessary and important part in intelligent driving.

[0003] At present, the method of calibrating the camera and vehicle exterior parameters according to the image and lane line has a high requirement for the calibration environment, needs to assume that the lane line is a parallel straight line, which will cause calibration error. The calculation through the feature points of the lane line in the image will discretize the lane line, which ignores the continuous characteristics of the lane line. In the field of intelligent driving, the lane line is often represented by a cubic curve. After discretizing the lane line, the constraints and physical meaning between the lane line points are lost. SUMMARY

[0004] In view of the above-mentioned shortcomings of the prior art, the present application provides a camera and vehicle exterior parameter calibration method and device, electronic equipment and storage medium to solve the technical problem that the camera exterior parameters can only be calibrated based on the assumption that the lane line is parallel or the lane line is discretized.

[0005] In a first aspect, the present application provides a camera and vehicle exterior parameter calibration method, comprising:

[0006] obtaining the intrinsic parameters of the camera and the initial exterior parameters between the camera and the vehicle, and obtaining a continuous original image through the camera;

[0007] processing the original image to obtain a plurality of pixel coordinate points of the lane line, and converting each pixel coordinate point into a camera coordinate point in the camera coordinate system according to the intrinsic parameters;

[0008] converting the original image into a bird's eye view image according to the initial exterior parameters, extracting the lane line on the bird's eye view image, and converting the lane line into a cubic curve equation in the vehicle coordinate system;

[0009] establishing a data association between each camera coordinate point and the cubic curve equation, and calculating the target exterior parameters between the camera and the vehicle.

[0010] Optionally, the internal parameters of the camera and initial extrinsic parameters between the camera and the vehicle are acquired, and a continuous original image is acquired through the camera, including:

[0011] The original image is continuous and contains lane lines.

[0012] Optionally, the original image is processed to obtain a plurality of pixel coordinate points of the lane lines, including:

[0013] The original image is input into a neural network model for lane line segmentation and identification, and a plurality of pixel points are randomly extracted on each lane line, the pixel coordinate points of the pixel points on the original image are recorded, and each pixel coordinate point is sequentially stored in a container V.

[0014] Optionally, the original image is converted into a bird's-eye view image according to the initial extrinsic parameters, including:

[0015] Assuming that the lane lines are distributed on a plane in the vehicle coordinate system, the original image is converted into the vehicle coordinate system by inverse perspective transformation to eliminate the perspective effect, and the bird's-eye view image with lane distribution is obtained.

[0016] Optionally, the data correlation between each camera coordinate point and the cubic curve equation is established, and the target extrinsic parameters between the camera and the vehicle are calculated, including:

[0017] The container V of the original image is traversed, the constraint relationship between the coordinate points on the lane lines in the vehicle coordinate system and the camera coordinate points is established, then the constraint relationship is substituted into the cubic curve equation, and the target extrinsic parameters between the camera and the vehicle are calculated.

[0018] Optionally, the data correlation between each camera coordinate point and the cubic curve equation is established, and the target extrinsic parameters between the camera and the vehicle are calculated, including:

[0019] The data correlation between each camera coordinate point and the cubic curve equation is established, and the target extrinsic parameters between the camera and the vehicle are calculated.

[0020] Optionally, after the data correlation between each camera coordinate point and the cubic curve equation is established, and the target extrinsic parameters between the camera and the vehicle are calculated, including:

[0021] An optimization equation is established, and the target extrinsic parameters between the camera and the vehicle are calculated through an optimization algorithm and are calibrated, the coordinate points on the lane lines in the vehicle coordinate system are projected onto the original image by using the internal parameters and the target extrinsic parameters to obtain projection coordinate points, and if the re-projection error between the projection coordinate points and the corresponding camera coordinate points is within a threshold range, the target extrinsic parameters are valid.

[0022] In a second aspect, the present application provides a camera and vehicle extrinsic parameter calibration device, comprising:

[0023] An acquisition module is configured to acquire intrinsic parameters of the camera, initial extrinsic parameters between the camera and the vehicle, and a continuous original image through the camera.

[0024] A conversion module is configured to process the original image to obtain a plurality of pixel coordinate points of lane lines, and convert each pixel coordinate point into a camera coordinate point in a camera coordinate system according to the intrinsic parameters.

[0025] An extraction module is configured to convert the original image into a bird's eye view image according to the initial extrinsic parameters, extract lane lines on the bird's eye view image, and convert the lane lines into a cubic curve equation in a vehicle coordinate system.

[0026] A calculation module is configured to establish a data association between each camera coordinate point and the cubic curve equation, and calculate target extrinsic parameters between the camera and the vehicle.

[0027] In a third aspect, the present application provides an electronic device, comprising:

[0028] One or more processors;

[0029] A storage device is configured to store one or more programs, which, when executed by the one or more processors, cause the electronic device to implement the camera and vehicle extrinsic parameter calibration method according to any one of the above aspects.

[0030] In a fourth aspect, the present application provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor of a computer, causes the computer to perform the camera and vehicle extrinsic parameter calibration method according to any one of the above aspects.

[0031] In the above-mentioned camera and vehicle extrinsic parameter calibration method and device, electronic device, and storage medium, the lane lines are expressed in each coordinate system and described by a cubic curve equation, the relationship between the lane lines in the vehicle coordinate system and the camera coordinate system is established, the intrinsic parameters and the initial extrinsic parameters of the camera are substituted to calculate the target extrinsic parameters between the camera and the vehicle. In this scheme, the lane lines are not assumed to be parallel and discretized, the obtained target extrinsic parameter value is accurate, and after the target extrinsic parameters are calibrated and applied to intelligent driving, the intelligent driving can be more effectively assisted. In this scheme, after the target extrinsic parameters between the camera and the vehicle are calculated, they are calibrated to meet the recognition requirements of sensors in intelligent driving, which is convenient for intelligent driving of vehicles.

[0032] It is to be understood that both the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the application, as claimed. BRIEF DESCRIPTION OF DRAWINGS

[0033] The accompanying drawings, which are incorporated herein and form a part of the specification, illustrate embodiments consistent with the present application and, together with the description, further serve to explain the principles of the application. It is to be understood that the drawings are designed solely for purposes of illustration to be used in conjunction with the following detailed description.

[0034] Figure 1 is a schematic diagram of an implementation environment of a camera and vehicle extrinsic parameter calibration method according to an example embodiment of the present application;

[0035] Figure 2 is a flowchart of a camera and vehicle extrinsic parameter calibration method according to an example embodiment of the present application;

[0036] Figure 3 is a schematic diagram of a vehicle coordinate system according to an example embodiment of the present application;

[0037] Figure 4 is a block diagram of a camera and vehicle extrinsic parameter calibration apparatus according to an example embodiment of the present application;

[0038] Figure 5 shows a schematic diagram of a computer system of an electronic device suitable for implementing embodiments of the present application. DETAILED DESCRIPTION

[0039] Other advantages and novel features of the present application will become apparent from the following detailed description of the application when considered in conjunction with the drawings. The application can be put into practice / embodied in various ways and is not limited to the specific embodiments described in this specification. The specific embodiments described in this specification are merely illustrative of the principles of the application and are not intended to limit the scope of the application.

[0040] It should be noted that the drawings included in the following embodiments are only schematic and are not drawn to scale. They are provided to aid in understanding the present application and are not provided to limit the present application. The specific instrumentalities and materials disclosed in the detailed description section merely provide example embodiments of the present application. Other embodiments of the present application will occur to those skilled in the art upon consideration of the specification and may be practiced without departing from the spirit of the application.

[0041] In the following description, numerous specific details are discussed in order to provide a thorough understanding of embodiments of the present application. However, it will be apparent to one skilled in the art that the embodiments of the present application can be practiced without these specific details. In other instances, well-known structures and devices are not described in detail in order to avoid obscuring embodiments of the present application.

[0042] First of all, it needs to be pointed out that the parameters calibrated in the present scheme refer to the external parameters of the camera and the vehicle, which will cooperate with the image of the camera to judge the actual position in intelligent driving. The camera in the present scheme refers to a sensor capable of acquiring an image, and also includes a camera.

[0043] Among them, the internal and external parameters are the parameters corresponding to the camera to different coordinate systems, the internal parameter refers to the parameter corresponding to the point in the image of the camera coordinate system, and the external parameter refers to the parameter corresponding to the world coordinate system in the camera coordinate system. The world coordinate system in the present scheme corresponds to the vehicle coordinate system. The internal parameter is obtained by camera calibration, and after the camera is installed on the vehicle, the initial external parameter is calibrated. The external parameter of the camera needs to be recalibrated according to the actual application scene after the vehicle is used. The calibration of the external parameter of the camera in the present scheme is after the vehicle is used.

[0044] Figure 1 It is an embodiment of an exemplary embodiment of the present application, which shows an embodiment environment schematic diagram of a camera and vehicle external parameter calibration method. Among them, the vehicle drives on the road, identifies the lane line through the camera installed on the vehicle, and calculates through the calibrated external parameter, which can provide accurate road information for intelligent driving.

[0045] Among them, the camera is installed inside or outside the vehicle and connected with the intelligent terminal. The intelligent terminal can be any terminal device supporting the installation of navigation map software, such as smart phone, vehicle computer, tablet computer, notebook computer or wearable device, but is not limited to this. The intelligent terminal can communicate with the navigation server 220 through 3G (third generation mobile information technology), 4G (fourth generation mobile information technology), 5G (fifth generation mobile information technology) and other wireless networks, and this is not limited here. Figure 1 The server device shown is a server, which can be a standalone physical server, a server cluster or a distributed system composed of multiple physical servers, or a cloud server providing cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communication, middleware services, domain name services, security services, CDN (Content Delivery Network), and basic cloud computing services such as big data and artificial intelligence platforms, and this is not limited here.

[0046] Reference is made to Figure 2 , Figure 2 is a flow chart of a camera and vehicle extrinsic parameter calibration method according to an example embodiment of the present application. The method can be applied to Figure 1 the implementation environment shown in FIG. 1 and is specifically executed by the vehicle in the implementation environment. It should be understood that the method can also be applied to other example implementation environments and is specifically executed by devices in other implementation environments, and the present embodiment does not limit the implementation environment to which the method is applied.

[0047] As shown in Figure 2 , in an example embodiment, the camera and vehicle extrinsic parameter calibration method includes at least steps S210 to S240, which are described in detail as follows:

[0048] Step S210: Obtain the intrinsic parameters of the camera and the initial extrinsic parameters between the camera and the vehicle, and obtain a continuous raw image through the camera.

[0049] In some embodiments, multiple frames of the raw image are continuously obtained, and the raw image contains lane lines.

[0050] According to the installation requirements, the initial value of the extrinsic parameters between the camera and the vehicle can be easily obtained. After the camera processes the original data of the external environment through the image sensor, the raw image output with lane lines is obtained.

[0051] Step S220: Process the raw image to obtain multiple pixel coordinate points of the lane lines, and convert each pixel coordinate point into a camera coordinate point in the camera coordinate system according to the intrinsic parameters.

[0052] In some embodiments, the raw image is sent to a neural network model for lane line segmentation and recognition, and multiple pixel points are randomly extracted on each lane line. The pixel coordinate points of the pixel points on the raw image are recorded, and each pixel coordinate point is sequentially stored in a container V.

[0053] In the specific implementation process, after M frames of image data are obtained, the raw image data is sent to a deep learning neural network model for lane line segmentation and recognition, and N points are randomly extracted on each lane line. The pixel coordinate values of the points on the corresponding image are recorded and sequentially stored in a container V.

[0054] Without loss of generality, it is assumed that the camera coordinate system is a three-dimensional space coordinate system with the camera optical center as the coordinate origin, the z-axis pointing forward, and the right-hand rule. According to the intrinsic parameters of the camera The pixel coordinates (u,v) of the image are converted into points (x,y,1) in the camera coordinate system. Since the pixels on the image only retain two-dimensional information and lose scale information, their third-dimensional coordinate values ​​can be normalized to 1.

[0055] Step S230: Convert the original image into a bird's-eye view image based on the initial extrinsic parameters, extract the lane lines on the bird's-eye view image, and convert the lane lines into a cubic curve equation in the vehicle coordinate system;

[0056] In some embodiments, assuming the lane lines are distributed in a plane of the vehicle coordinate system, the original image is transformed to the vehicle coordinate system by inverse perspective transformation to eliminate perspective effects, thereby obtaining the bird's-eye view image with lane distribution.

[0057] Inverse perspective transformation (IPM) is a technique used in images captured by a forward-facing camera to eliminate perspective distortion. In such images, parallel objects appear to intersect due to perspective effects. IPM is thus called inverse perspective transformation because it removes this perspective distortion.

[0058] Specifically, since the camera's initial extrinsic parameters relative to the vehicle are relatively easy to obtain after installation, and under the ground assumption, lane lines are distributed on a plane in the vehicle coordinate system, the original image and the actual lane line plane undergo a 2D projective transformation. The original image is then transformed to the vehicle coordinate system using inverse perspective transformation (IPM), and the BEV image is fed into a deep learning neural network for lane line recognition and extraction from a bird's-eye view (BEV). A cubic continuous curve is used to describe the lane line equation, such as: Y = c³X. 3 +c2X 2 +c1X+c0, where X is the pixel coordinate of the lane line on the X-axis in the vehicle coordinate system, and Y is the pixel coordinate of the lane line on the Y-axis in the vehicle coordinate system. The vehicle coordinate system is as follows: Figure 3 As shown.

[0059] Among them, the BEV image can realize a single-road environment-level distribution map, which can accurately estimate the future trajectory of moving vehicles.

[0060] Step S240: Establish the data association between each of the camera coordinate points and the cubic curve equation, and calculate the target extrinsic parameters between the camera and the vehicle.

[0061] In some embodiments, the container V of the original image is traversed to establish the constraint relationship between the coordinate points on the lane line in the vehicle coordinate system and the camera coordinate points. Then, the constraint relationship is substituted into the cubic curve equation to calculate the target extrinsic parameters between the camera and the vehicle.

[0062] In some embodiments, a target function is established to express the correlation between the camera coordinate points and the data of the cubic curve equation, and the target function is solved to obtain the target external parameter between the camera and the vehicle. The original image is continuously acquired in multiple frames, and the original image contains lane lines.

[0063] Specifically, a lane line pixel point container V on the original image is traversed, and the correlation between the lane line and the vehicle coordinate system is established according to different lane line IDs. After the 3D point in the vehicle coordinate system is converted to the camera coordinate system through rotation and translation, the 3D point is projected to the image to obtain the corresponding pixel point by using the known camera intrinsic parameter K. The conversion relationship is as follows:

[0064]

[0065] The constraint relationship between the 3D point X and Y coordinates of the lane line in the vehicle coordinate system and the pixel point (u, v) with the established matching relationship can be established by using the above expression, and then the obtained constraint relationship is brought into the optimization equation of the point to the cubic curve, so as to solve the rotation and translation relationship between the camera and the vehicle. Although the scale in the projection expression is unknown in each image, the current scale can be calculated by using the obtained rotation matrix R and translation vector t after each iteration under the constraint of the ground assumption, and an initial rotation and translation can be obtained before optimization. Therefore, the rotation matrix R, the translation vector t, and the scale can be solved in two steps, so that the scale can not be included in the optimization variables when the optimization target function is set, thereby reducing the dimension of the optimization variables.

[0066]

[0067] In some embodiments, an optimization equation is established, the target external parameter between the camera and the vehicle is calculated by using an optimization algorithm to iteratively calculate, and the coordinate point on the lane line in the vehicle coordinate system is projected to the original image by using the intrinsic parameter and the target external parameter to obtain a projection coordinate point. If the re-projection error between the projection coordinate point and the corresponding camera coordinate point is within a threshold range, the target external parameter calibration is effective.

[0068] Specifically, after the rotation matrix R and the translation vector t between the camera and the vehicle are calculated by using an optimization algorithm to iteratively calculate, some 3D points in the vehicle coordinate system can be projected to the image plane by using the intrinsic and extrinsic parameters of the camera, and the corresponding pixel points on the image are detected, and the re-projection error is calculated. It is judged whether the re-projection error exceeds a certain threshold value. If it exceeds, it is considered that this calibration fails, and the calibration result is not updated or adopted; if the re-projection error is less than a certain threshold value, it is considered that this calibration is successful.

[0069] In an embodiment, a camera and vehicle extrinsic parameter calibration device is provided, which corresponds to the camera and vehicle extrinsic parameter calibration method in the above embodiment, as shown in Figure 4 Figure 4 is a structural schematic diagram of a camera and vehicle extrinsic parameter calibration device according to an exemplary embodiment of the present application, which comprises an acquisition module 401, a conversion module 402, an extraction module 403, and a calculation module 404, and the functions of each module are described as follows:

[0070] The acquisition module 401 is configured to acquire the intrinsic parameters of the camera and the initial extrinsic parameters between the camera and the vehicle, and acquire a continuous original image through the camera;

[0071] The conversion module 402 is configured to process the original image to obtain a plurality of pixel coordinate points of the lane line, and convert each pixel coordinate point into a camera coordinate point in a camera coordinate system according to the intrinsic parameters;

[0072] The extraction module 403 is configured to convert the original image into a bird's eye view image according to the initial extrinsic parameters, extract the lane line on the bird's eye view image, and convert the lane line into a cubic curve equation in a vehicle coordinate system;

[0073] The calculation module 404 is configured to establish a data association between each camera coordinate point and the cubic curve equation, and calculate the target extrinsic parameters between the camera and the vehicle.

[0074] It should be noted that the camera and vehicle extrinsic parameter calibration device provided in the above embodiment and the camera and vehicle extrinsic parameter calibration method provided in the above embodiment belong to the same concept, and the specific operation modes of each module and unit have been described in detail in the method embodiments, which will not be repeated here. The camera and vehicle extrinsic parameter calibration device provided in the above embodiment can be used to complete the above-described functions by different functional modules according to the needs in actual application, i.e., the internal structure of the device is divided into different functional modules to complete all or part of the above-described functions, and this is not limited herein.

[0075] The embodiments of the present application also provide an electronic device, which comprises one or more processors and a storage device configured to store one or more programs, when the one or more programs are executed by the one or more processors, the electronic device implements the camera and vehicle extrinsic parameter calibration method provided in each of the above embodiments.

[0076] Figure 5 A structural schematic diagram of a computer system of an electronic device suitable for implementing the embodiments of the present application is shown. It should be noted that, Figure 5 ​The computer system 500 of the electronic device shown is merely one example, and should not be taken as limiting the functionality or use of embodiments of the present application.

[0077] As shown in Figure 5 The computer system 500 includes a central processing unit (CPU) 501 that can perform various suitable actions and processes in accordance with a program stored in read-only memory (ROM) 502 or a program loaded from the storage section 508 into random access memory (RAM) 503, such as performing the methods described in the embodiments above. Various programs and data required for system operation are also stored in the RAM 503. The CPU 501, ROM 502, and RAM 503 are connected to each other through a bus 504. An input / output (I / O) interface 505 is also connected to the bus 504.

[0078] The following components are connected to the I / O interface 505: an input section 506 including a keyboard, a mouse, etc.; an output section 507 including a display such as a Cathode Ray Tube (CRT), a Liquid Crystal Display (LCD), etc., and a speaker, etc.; a storage section 508 including a hard disk, etc.; and a communication section 509 including a network interface card such as a LAN (Local Area Network) card, a modem, etc. The communication section 509 performs communication processing via a network such as the Internet. A drive 510 is also connected to the I / O interface 505 as necessary. A removable recording medium 511 such as a magnetic disk, an optical disk, a magneto-optical disk, a semiconductor memory, etc. is attached to the drive 510 as necessary, so that a computer program read therefrom is installed into the storage section 508 as necessary.

[0079] In particular, in accordance with embodiments of the present application, the processes described above with reference to the flowcharts can be implemented as a computer software program. For example, embodiments of the present application include a computer program product comprising a computer program carried on a computer readable medium, the computer program containing computer programs for performing the methods shown in the flowcharts. In such embodiments, the computer program can be downloaded and installed from a network via the communication section 509, and / or installed from the removable recording medium 511. When the computer program is executed by the central processing unit (CPU) 501, various functions defined in the system of the present application are performed.

[0080] It should be noted that the computer-readable medium in the embodiments of the present application can be a computer-readable signal medium or a computer-readable storage medium or any combination thereof. The computer-readable storage medium may, for example, be an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or apparatus, or any combination thereof. More specific examples of the computer-readable storage medium can include, but are not limited to, an electrical connection having one or more wires, a portable computer diskette, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM), a flash memory, an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination thereof. In this application, the computer-readable signal medium can include a data signal propagated in a baseband or as a carrier wave in a propagated data signal, in which the computer-readable computer program is carried. Such a propagated data signal can take on many forms, including but not limited to an electromagnetic signal, an optical signal, or any suitable combination thereof. The computer-readable signal medium can also be any computer-readable medium other than the computer-readable storage medium, which can send, propagate, or transmit the program for use by or in connection with an instruction execution system, apparatus, or device. The computer program contained on the computer-readable medium can be transmitted in any suitable medium, including but not limited to wireless, wired, or the like, or any suitable combination thereof.

[0081] The flowcharts and block diagrams in the drawings illustrate the possible architectures, functions, and operations of systems, methods, and computer program products according to various embodiments of the present application. Each block in the flowcharts or block diagrams can represent a module, a program segment, or a portion of code, which contains one or more executable instructions for implementing the specified logical functions. It should also be noted that in some alternative implementations, the functions noted in the blocks can occur in a different order than that shown in the figures. For example, two blocks noted in succession can actually be executed substantially concurrently, or they can sometimes be executed in reverse order, depending on the functionality involved. It should also be noted that each block in the flowcharts or block diagrams, and combinations of blocks in the flowcharts or block diagrams, can be implemented by special-purpose hardware-based systems, which perform the specified functions or operations, or can be implemented by a combination of special-purpose hardware and computer instructions.

[0082] The units described in the embodiments of the present application can be implemented in the form of software, or can be implemented in the form of hardware, and the described units can also be arranged in a processor. In some cases, the names of the units do not constitute a limitation on the units themselves.

[0083] Another aspect of the present application also provides a computer readable storage medium, which stores a computer program. When the computer program is executed by a processor of a computer, the computer executes the camera and vehicle extrinsic parameter calibration method as described above. The computer readable storage medium can be included in the electronic device described in the above embodiments, or can exist separately and not be assembled into the electronic device.

[0084] Another aspect of the present application also provides a computer program product or computer program, which includes computer instructions stored in a computer readable storage medium. A processor of a computer device reads the computer instructions from the computer readable storage medium, and the processor executes the computer instructions, so that the computer device executes the camera and vehicle extrinsic parameter calibration method provided in the above embodiments.

[0085] The above embodiments only exemplarily illustrate the principles and effects of the present application, and are not used to limit the present application. Any person skilled in the art can modify or change the above embodiments without departing from the spirit and scope of the present application. Therefore, all equivalent modifications or changes completed by those skilled in the art without departing from the spirit and technical thought of the present application should be covered by the claims of the present application.

Claims

1. A method for calibrating extrinsic parameters of a camera and a vehicle, characterized in that, The method comprises: obtaining the internal parameters of a camera, and initial external parameters between the camera and a vehicle, and obtaining a continuous original image through the camera; processing the original image to obtain a plurality of pixel coordinate points of lane lines, and converting each pixel coordinate point into a camera coordinate point in a camera coordinate system according to the internal parameters; converting the original image into a bird's-eye view image according to the initial external parameters, extracting lane lines on the bird's-eye view image, and converting the lane lines into a cubic curve equation in a vehicle coordinate system; establishing data association between each camera coordinate point and the cubic curve equation, and calculating target external parameters between the camera and the vehicle.

2. The camera and vehicle extrinsic calibration method of claim 1, wherein: obtaining the internal parameters of a camera, and initial external parameters between the camera and a vehicle, and obtaining a continuous original image through the camera, comprising: the original image is continuously obtained in multiple frames, and the original image contains lane lines.

3. The camera and vehicle extrinsic calibration method of claim 2, wherein, processing the original image to obtain a plurality of pixel coordinate points of lane lines, comprising: feeding the original image into a neural network model for lane line segmentation and identification, and randomly extracting a plurality of pixel points on each lane line, recording the pixel coordinate points of the pixel points on the original image, and storing each pixel coordinate point in a container V in sequence.

4. The camera and vehicle extrinsic calibration method of claim 3, wherein: converting the original image into a bird's-eye view image according to the initial external parameters, comprising: assuming that the lane lines are distributed on a plane in the vehicle coordinate system, eliminating the perspective effect by inverse perspective transformation to convert the original image into the vehicle coordinate system, and obtaining the bird's-eye view image with lane distribution.

5. The camera and vehicle extrinsic calibration method of claim 3, wherein: establishing data association between each camera coordinate point and the cubic curve equation, and calculating target external parameters between the camera and the vehicle, comprising: traversing the container V of the original image, establishing the constraint relationship between the coordinate points on the lane lines in the vehicle coordinate system and the camera coordinate points, then substituting the constraint relationship into the cubic curve equation, and calculating the target external parameters between the camera and the vehicle.

6. The camera and vehicle extrinsic calibration method of claim 4, wherein: establishing data association between each camera coordinate point and the cubic curve equation, and calculating target external parameters between the camera and the vehicle, comprising: establishing a target function to express the data association between each camera coordinate point and the cubic curve equation, and solving the target function to obtain the target external parameters between the camera and the vehicle.

7. The camera and vehicle extrinsic calibration method of claim 5, wherein: after establishing data association between each camera coordinate point and the cubic curve equation, and calculating target external parameters between the camera and the vehicle, comprising: establishing an optimization equation, and obtaining the target external parameters between the camera and the vehicle through iterative calculation by an optimization algorithm and performing calibration, projecting the coordinate points on the lane lines in the vehicle coordinate system onto the original image to obtain projection coordinate points by using the internal parameters and the target external parameters, and if the re-projection error between the projection coordinate points and the corresponding camera coordinate points is within a threshold range, the target external parameter calibration is effective.

8. An apparatus for calibrating extrinsic parameters of a camera and a vehicle, characterized in that, The device comprises: an acquisition module, configured to obtain the internal parameters of a camera, and initial external parameters between the camera and a vehicle, and obtain a continuous original image through the camera; a conversion module, configured to process the original image to obtain a plurality of pixel coordinate points of lane lines, and convert each pixel coordinate point into a camera coordinate point in a camera coordinate system according to the internal parameters; An extraction module is configured to convert the original image into a bird's eye view image according to the initial extrinsic parameter, extract lane lines on the bird's eye view image, and convert the lane lines into a cubic curve equation in a vehicle coordinate system; A calculation module is configured to establish a data association between each of the camera coordinate points and the cubic curve equation, and calculate a target extrinsic parameter between the camera and the vehicle.

9. An electronic device, comprising: The electronic device includes: one or more processors; a storage device configured to store one or more programs that, when executed by the one or more processors, cause the electronic device to implement the camera and vehicle extrinsic parameter calibration method according to any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that, A computer program is stored thereon, and when the computer program is executed by a processor of a computer, the computer is caused to perform the camera and vehicle extrinsic parameter calibration method according to any one of claims 1 to 7.

Images