Vehicle-mounted camera calibration method and device, electronic equipment, storage medium and computer program product

Abstract

The invention discloses a vehicle-mounted camera calibration method and device, electronic equipment, a storage medium and a computer program product. The method comprises the following steps: constructing a vehicle-mounted camera two-dimensional model; wherein the vehicle-mounted camera two-dimensional model is used for representing a space mapping relation between the vehicle-mounted camera and a target; according to different installation positions of the vehicle-mounted camera on the whole vehicle, target existence areas and target candidate positions of targets of the vehicle-mounted camera at different installation positions are determined in the two-dimensional model of the vehicle-mounted camera; determining a first target position of the target in the two-dimensional model of the vehicle-mounted camera according to the target existence area and the target candidate position; and according to the target installation position of the vehicle-mounted camera on the whole vehicle, a second target position of the target under the whole vehicle coordinates is determined, and the target at the second target position is adopted to calibrate the vehicle-mounted camera. According to the technical scheme of the embodiment of the invention, by constructing the two-dimensional model of the vehicle-mounted camera, the design difficulty of the calibration work station is simplified, and the calibration efficiency of the vehicle-mounted camera is improved.

Description

Technical Field

[0001] This invention relates to the field of autonomous driving technology, and in particular to a method, apparatus, electronic device, storage medium, and computer program product for calibrating an in-vehicle camera. Background Technology

[0002] The automotive industry has now entered the era of intelligent vehicles, and the most important manifestation of intelligence in intelligent vehicles is the Advanced Driver Assistance System (ADAS). In-vehicle cameras are an essential component for realizing intelligent driving. Due to factors such as the installation process and individual vehicle differences, the installation angle of the in-vehicle camera may have errors. To better realize the ADAS warning function, the system needs to accurately correct the installation angle error of the in-vehicle camera. Therefore, the installation angle of the in-vehicle camera needs to be calibrated before the vehicle leaves the production line or before after-sales replacement.

[0003] Currently, the calibration of vehicle-mounted cameras typically employs a combination of calibration boards (such as checkerboard calibration boards) and multi-view image acquisition. Camera parameters, such as intrinsic parameters, extrinsic parameters, and distortion coefficients, are calculated through feature point matching and spatial geometric transformation. However, the different camera placements in different vehicle models and locations result in varying requirements for the target's position.

[0004] Therefore, how to use the same calibration station equipment to calibrate vehicle cameras for different vehicle models and camera locations is a problem that urgently needs to be solved by those skilled in the art. Summary of the Invention

[0005] This invention provides a method, apparatus, electronic device, and storage medium for calibrating vehicle-mounted cameras, in order to solve the problem that existing calibration workstation equipment cannot meet the requirements for calibrating vehicle-mounted cameras using targets in different vehicle models and at different vehicle-mounted camera deployment locations.

[0006] According to one aspect of the present invention, a method for calibrating a vehicle-mounted camera is provided, the method comprising:

[0007] Construct a two-dimensional model of the vehicle-mounted camera; wherein, the two-dimensional model of the vehicle-mounted camera is used to represent the spatial mapping relationship between the vehicle-mounted camera and the target;

[0008] Based on the different installation positions of the vehicle-mounted camera on the vehicle, the target existence area and target candidate position of the vehicle-mounted camera under different installation positions are determined in the two-dimensional model of the vehicle-mounted camera.

[0009] Based on the target's location and candidate target positions, the first target position of the target in the vehicle-mounted camera's two-dimensional model is determined;

[0010] Based on the target installation position of the vehicle-mounted camera on the vehicle, the second target position of the target in the vehicle coordinate system is determined, and the vehicle-mounted camera is calibrated using the target at the second target position.

[0011] According to another aspect of the present invention, a vehicle-mounted camera calibration device is provided, the device comprising:

[0012] The model building module is used to build a two-dimensional model of the vehicle-mounted camera; wherein, the two-dimensional model of the vehicle-mounted camera is used to represent the spatial mapping relationship between the vehicle-mounted camera and the target;

[0013] The target information determination module is used to determine the target presence area and target candidate position of the vehicle-mounted camera in the two-dimensional model of the vehicle-mounted camera based on the different installation positions of the vehicle-mounted camera on the whole vehicle.

[0014] The first target position determination module is used to determine the first target position of the target in the two-dimensional model of the vehicle-mounted camera based on the target existence area and the target candidate position;

[0015] The second target position determination module is used to determine the second target position of the target in the vehicle coordinate system based on the target installation position of the vehicle-mounted camera on the vehicle, and to use the target at the second target position to calibrate the vehicle-mounted camera.

[0016] According to another aspect of the present invention, an electronic device is provided, the electronic device comprising:

[0017] At least one processor; and

[0018] A memory communicatively connected to the at least one processor; wherein,

[0019] The memory stores a computer program that can be executed by the at least one processor, the computer program being executed by the at least one processor to enable the at least one processor to perform the vehicle camera calibration method according to any embodiment of the present invention.

[0020] According to another aspect of the present invention, a computer-readable storage medium is provided, the computer-readable storage medium storing computer instructions for causing a processor to execute and implement the vehicle camera calibration method according to any embodiment of the present invention.

[0021] According to another aspect of the present invention, a computer program product is provided, the computer program product comprising a computer program that, when executed by a processor, implements the vehicle camera calibration method as described in any embodiment of the present invention.

[0022] The technical solution of this invention involves constructing a two-dimensional model of an onboard camera. This model represents the spatial mapping relationship between the onboard camera and a target. Based on the different installation positions of the onboard camera on the vehicle, the target presence area and candidate target positions are determined in the two-dimensional model at different installation positions. Based on the target presence area and candidate target positions, a first target position of the target in the two-dimensional model is determined. Based on the target installation position of the onboard camera on the vehicle, a second target position of the target in the vehicle coordinate system is determined, and the onboard camera is calibrated using the target at the second target position. By constructing a two-dimensional model of the onboard camera, this invention enables the use of the same calibration station for calibration under different vehicle models and different onboard camera deployment positions, simplifying the design of the calibration station and improving the efficiency of onboard camera calibration.

[0023] It should be understood that the description in this section is not intended to identify key or essential features of the embodiments of the present invention, nor is it intended to limit the scope of the invention. Other features of the invention will become readily apparent from the following description. Attached Figure Description

[0024] To more clearly illustrate the technical solutions in the embodiments of the present invention, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0025] Figure 1 This is a flowchart of a vehicle-mounted camera calibration method according to Embodiment 1 of the present invention;

[0026] Figure 2 This is a flowchart of a vehicle-mounted camera calibration method according to Embodiment 2 of the present invention;

[0027] Figure 3 This is a schematic diagram of a two-dimensional model of a vehicle-mounted camera according to Embodiment 2 of the present invention;

[0028] Figure 4 This is a schematic diagram of target position determination according to Embodiment 2 of the present invention;

[0029] Figure 5 This is a schematic diagram of the structure of a vehicle-mounted camera calibration device according to Embodiment 3 of the present invention;

[0030] Figure 6 This is a schematic diagram of the structure of an electronic device provided in Embodiment 4 of the present invention. Detailed Implementation

[0031] To enable those skilled in the art to better understand the present invention, the technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings of the embodiments of the present invention. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort should fall within the scope of protection of the present invention.

[0032] The acquisition, storage, use, and processing of data in the technical solution of this application all comply with relevant laws and regulations. It should be noted that the terms "first," "second," "target," and "original," etc., in the specification, claims, and accompanying drawings of this invention are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments of the invention described herein can be implemented in orders other than those illustrated or described herein. Furthermore, the terms "comprising," "etc.," and "having," and any variations thereof, are intended to cover non-exclusive inclusion; for example, a process, method, system, product, or apparatus that includes a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus.

[0033] Example 1

[0034] Figure 1 This is a flowchart illustrating a vehicle-mounted camera calibration method according to Embodiment 1 of the present invention. This embodiment is applicable to situations where vehicle-mounted cameras are calibrated at different deployment locations and on different vehicles within the same calibration station. The method can be executed by a vehicle-mounted camera calibration device, which can be implemented in hardware and / or software. This device can be configured in any electronic device with network communication capabilities. Figure 1 As shown, the method includes:

[0035] S110. Construct a two-dimensional model of the vehicle-mounted camera.

[0036] The two-dimensional model of the vehicle-mounted camera is used to represent the spatial mapping relationship between the vehicle-mounted camera and the target. The two-dimensional model refers to mapping the vehicle-mounted camera and the target into the same two-dimensional space to construct the relative positional relationship between them. For example, the relative position between the vehicle-mounted camera and the target is determined using the location of the vehicle-mounted camera as the origin.

[0037] S120. Based on the different installation positions of the vehicle-mounted camera on the vehicle, determine the target existence area and target candidate position of the vehicle-mounted camera in the two-dimensional model of the vehicle-mounted camera under different installation positions.

[0038] Among them, the target presence area is the area where the target may exist within the field of view of the vehicle camera; the target candidate position is the position of the target in the target presence area that is more conducive to the calibration of the vehicle camera.

[0039] The installation location of the vehicle-mounted camera varies depending on the vehicle model; and this difference in installation location results in different target areas and candidate locations. For example, see... Figure 3 The red dot indicates the location of the vehicle-mounted camera. The area formed by the near boundary, far boundary, and side boundary is the target presence area, and the blue recommended location is the target candidate location. The target presence area and target candidate location will vary depending on the location of the vehicle-mounted camera.

[0040] S130. Based on the target's location and candidate target positions, determine the first target position of the target in the vehicle-mounted camera's two-dimensional model.

[0041] Here, the first target position is the location of the target within the two-dimensional model of the vehicle-mounted camera; it refers to the target position in the two-dimensional model of the vehicle-mounted camera that uniquely represents the calibration of the vehicle-mounted camera, determined from the target presence areas and candidate target positions under different installation positions. Since different vehicle models result in different installation positions of the vehicle-mounted cameras, the corresponding target presence areas and candidate target positions also differ. Therefore, a first target position needs to be determined such that the target at the first target position can satisfy all vehicle models and vehicle-mounted camera installation positions, so that camera calibration can be completed using the target at the first target position.

[0042] In this embodiment of the invention, the first target position of the target in the two-dimensional model of the vehicle-mounted camera is determined based on the target presence area and target candidate position of the target under different installation positions of the vehicle-mounted camera.

[0043] S140. Based on the target installation position of the vehicle-mounted camera on the vehicle, determine the second target position of the target in the vehicle coordinate system, and use the target at the second target position to calibrate the vehicle-mounted camera.

[0044] The second target position refers to the position of the target in the vehicle coordinate system, specifically the position uniquely representing the target in the vehicle coordinate system. After determining the first target position of the target in the 2D model of the vehicle-mounted camera, the first target position needs to be mapped to the second target position in the vehicle coordinate system, and the vehicle-mounted camera is calibrated based on the target at the second target position. Optionally, based on the first target position of the target in the 2D model of the vehicle-mounted camera, the relative position between the target and the vehicle-mounted camera is determined. Based on the relative position and the target installation position of the vehicle-mounted camera on the vehicle, the second target position of the target in the vehicle coordinate system is determined, and the vehicle-mounted camera is calibrated using the target at the second target position.

[0045] This invention provides a method for calibrating a vehicle-mounted camera. The method involves constructing a two-dimensional model of the vehicle-mounted camera. This model represents the spatial mapping relationship between the camera and a target. Based on the different installation positions of the camera on the vehicle, the method determines the target presence area and candidate target positions in the two-dimensional model for each installation position. Based on these target presence areas and candidate target positions, a first target position of the target in the two-dimensional model is determined. Finally, based on the target installation position of the camera on the vehicle, a second target position of the target in the vehicle coordinate system is determined, and the camera is calibrated using the target at the second target position. By constructing a two-dimensional model of the vehicle-mounted camera, this method allows for calibration using the same calibration station across different vehicle models and camera deployment positions, simplifying the design of the calibration station and improving calibration efficiency.

[0046] Example 2

[0047] Figure 2 This is a flowchart of a vehicle-mounted camera calibration method provided in Embodiment 2 of the present invention. The present invention further optimizes the aforementioned embodiments based on the above embodiments, and can be combined with various optional solutions from one or more of the above embodiments. For example... Figure 2 As shown, the method includes:

[0048] S210. Simplify the vehicle-mounted camera as the origin in a two-dimensional coordinate system.

[0049] See Figure 3 The vehicle-mounted camera is simplified to the origin in a two-dimensional coordinate system, and its corresponding field of view is defined. The field of view of the vehicle-mounted camera is a fan-shaped region.

[0050] S220: The target is simplified to a straight line segment parallel to the plane where the vehicle-mounted camera is located.

[0051] The target used for vehicle-mounted camera calibration is simplified to a straight line segment parallel to the plane where the vehicle-mounted camera is located; the target length L is determined by projecting the actual target size onto the two-dimensional model, and the target width W is determined by the actual target size.

[0052] S230. Based on the target size, vehicle-mounted camera configuration parameters, and vehicle-mounted camera field of view, determine the target's location area and candidate target positions.

[0053] The target dimensions include the target length and width, while the vehicle-mounted camera configuration parameters include the vehicle-mounted camera focal length, pixel count, and resolution. Based on the target dimensions, vehicle-mounted camera configuration parameters, and vehicle-mounted camera field of view, the target's location area and candidate target positions are determined.

[0054] As an optional but non-limiting implementation, the target presence area and candidate target positions are determined based on the target size, vehicle-mounted camera configuration parameters, and vehicle-mounted camera field of view, including but not limited to steps A1-A2:

[0055] Step A1: Based on the target size, vehicle camera configuration parameters, and vehicle camera field of view, determine the near boundary, far boundary, and side boundary of the target within the vehicle camera field of view;

[0056] Step A2: Determine the target's location and candidate positions based on the target's near boundary, far boundary, and lateral boundary within the vehicle-mounted camera's field of view.

[0057] The near boundary refers to the boundary region closer to the vehicle-mounted camera, the far boundary refers to the boundary region farther from the vehicle-mounted camera, and the lateral boundary refers to the boundary region located on the side of the vehicle-mounted camera's field of view. All three boundaries are within the field of view of the vehicle-mounted camera. Based on the target size, vehicle-mounted camera configuration parameters, and the vehicle-mounted camera's field of view, the near, far, and lateral boundaries of the target within the vehicle-mounted camera's field of view are determined. The area enclosed by these three boundaries is defined as the target's presence area, and the recommended position of the target within this area is designated as the target candidate position.

[0058] As an optional but non-limiting implementation, determining the near boundary of the target within the field of view of the vehicle-mounted camera based on the target size, the configuration parameters of the vehicle-mounted camera, and the field of view of the vehicle-mounted camera includes, but is not limited to, steps B1-B4:

[0059] Step B1: Determine the first near boundary of the target within the field of view of the vehicle camera based on the focal length of the vehicle camera, the length of the target, and the threshold number of pixels at the edge of the target.

[0060] Step B2: Based on the vertical field of view of the vehicle-mounted camera, the target width, the number of targets, the vertical installation error of the vehicle-mounted camera, and the cropping angle of the vehicle-mounted camera algorithm, determine the second near boundary of the target within the field of view of the vehicle-mounted camera;

[0061] Step B3: Based on the horizontal field of view of the vehicle-mounted camera, the target width, the number of targets, the horizontal installation error of the vehicle-mounted camera, and the cropping angle of the vehicle-mounted camera algorithm, determine the third near boundary of the target within the field of view of the vehicle-mounted camera;

[0062] Step B4: Select the largest near boundary from the first near boundary, the second near boundary, and the third near boundary as the target near boundary within the field of view of the vehicle-mounted camera.

[0063] The first near boundary is determined by the focal length of the vehicle-mounted camera, the target length, and the target edge pixel count threshold. The first near boundary can be expressed as:

[0064] ;

[0065] in, The first near-boundary is indicated by L, the target length by L, and the focal length by f of the vehicle-mounted camera. This represents the threshold number of pixels at the target edge.

[0066] The second near boundary is determined by the vertical field of view of the vehicle-mounted camera, the target width, the number of targets, the vertical installation error of the vehicle-mounted camera, and the cropping angle of the vehicle-mounted camera algorithm. The second near boundary can be expressed as:

[0067] ;

[0068] in, The second near boundary is indicated, and W represents the target length. Indicates the number of targets. Vertical field of view of the vehicle-mounted camera This indicates the vertical installation error of the vehicle-mounted camera and the cropping angle of the vehicle-mounted camera algorithm.

[0069] The third near boundary is determined by the horizontal field of view of the vehicle-mounted camera, the target width, the number of targets, the horizontal installation error of the vehicle-mounted camera, and the cropping angle of the vehicle-mounted camera algorithm. The third near boundary can be expressed as:

[0070] ;

[0071] in, Indicates the third nearest boundary. Indicates the number of targets. Horizontal field of view of the vehicle-mounted camera This indicates the horizontal installation error of the vehicle-mounted camera and the cropping angle of the vehicle-mounted camera algorithm.

[0072] After determining the first, second, and third near boundaries, the largest near boundary is selected as the target's near boundary within the vehicle-mounted camera's field of view, which can be expressed as:

[0073] ;

[0074] in, It refers to the near-boundary of the target within the field of view of the vehicle-mounted camera.

[0075] As an optional but non-limiting implementation, determining the far boundary of the target within the field of view of the vehicle-mounted camera based on the target size, the configuration parameters of the vehicle-mounted camera, and the field of view of the vehicle-mounted camera includes, but is not limited to, steps C1-C3:

[0076] Step C1: Select the shorter side from the target length and target width, and determine the size of the shorter side of the target; the detection distance of the vehicle-mounted camera is limited by the size of the shorter side of the target;

[0077] Step C2: Select the minimum resolution from the horizontal and vertical resolutions of the vehicle camera;

[0078] Step C3: Based on the target's short side dimensions, the vehicle-mounted camera's minimum resolution, the vehicle-mounted camera's detection accuracy requirements, and the target's minimum module size, determine the target's far boundary within the vehicle-mounted camera's field of view.

[0079] The far boundary of the target within the field of view of the vehicle-mounted camera is determined by the maximum detection distance of the vehicle-mounted camera and the target accuracy. The shorter side is selected from the target length and width, and its size is determined; this shorter side size limits the maximum detection distance of the vehicle-mounted camera. The minimum resolution is selected from the horizontal and vertical resolutions of the vehicle-mounted camera, where the minimum resolution determines the overall resolution of the camera. The minimum target module size refers to the physical size of the minimum target module; the smaller the minimum target module size, the higher the required imaging accuracy and the shorter the maximum detection distance. Based on the determined target shorter side size, the minimum resolution of the vehicle-mounted camera, the required detection accuracy of the vehicle-mounted camera, and the minimum target module size, the far boundary of the target within the field of view of the vehicle-mounted camera is determined; whereby the far boundary is represented as:

[0080] ;

[0081] in, L represents the far boundary, W represents the target length, m represents the horizontal resolution of the vehicle-mounted camera, n represents the vertical resolution of the vehicle-mounted camera, p represents the detection accuracy requirement of the vehicle-mounted camera, and P represents the minimum module size of the target.

[0082] As an optional but non-limiting implementation, determining the side boundary of the target within the field of view of the vehicle-mounted camera based on the target size, the configuration parameters of the vehicle-mounted camera, and the field of view of the vehicle-mounted camera includes:

[0083] Based on the horizontal field of view of the vehicle-mounted camera, the horizontal installation error of the vehicle-mounted camera, and the cropping angle of the camera algorithm, the left and right boundaries of the target within the field of view of the vehicle-mounted camera are determined.

[0084] The left and right side boundaries of the target are determined by the horizontal field of view of the vehicle-mounted camera, the horizontal installation error of the vehicle-mounted camera, and the cropping angle of the camera algorithm. The side boundaries can be represented as:

[0085] ;

[0086] in, This indicates the left and right boundaries of the target within the field of view of the vehicle-mounted camera.

[0087] Specifically, after determining the near boundary, far boundary, and lateral boundary of the target within the field of view of the vehicle-mounted camera, the area enclosed by the near boundary, far boundary, and lateral boundary is defined as the target presence area, and the recommended position of the target within this area is designated as the target candidate position. See also... Figure 3 The blue area represents the candidate target location, which can be represented as:

[0088] ;

[0089] in, This represents the candidate position of the target, and k is the experimental correction coefficient, which is usually taken as 0.8-1.2.

[0090] S240. Based on the origin position of the vehicle-mounted camera, the target area, and the target candidate positions, construct a two-dimensional model of the vehicle-mounted camera.

[0091] In this process, after determining the origin position of the vehicle-mounted camera, the target area, and the candidate target positions, the relative positional relationship between the vehicle-mounted camera and the target is determined, and a two-dimensional model of the vehicle-mounted camera is constructed.

[0092] S250. Based on the different installation positions of the vehicle-mounted camera on the vehicle, determine the target existence area and target candidate position of the vehicle-mounted camera in the two-dimensional model of the vehicle-mounted camera under different installation positions.

[0093] This involves obtaining the target presence area and candidate target positions for the vehicle-mounted cameras at different installation locations, depending on the specific mounting position of the cameras on the vehicle. (See also...) Figure 4 The red dots represent vehicle-mounted cameras at two different installation locations. The target areas and candidate locations corresponding to the vehicle-mounted cameras at different installation locations are different.

[0094] S260. Based on the target's location and candidate target positions, determine the first target position of the target in the vehicle-mounted camera's two-dimensional model.

[0095] After determining the target presence area and candidate target positions for the vehicle-mounted camera in different installation locations, it is necessary to select a target position that can uniquely characterize the vehicle-mounted camera calibration so as to be applicable to vehicle-mounted camera calibration under different vehicle models and vehicle-mounted camera installation locations.

[0096] As an optional but non-limiting implementation, determining the first target position of the target in the 2D model of the vehicle-mounted camera based on the target's location area and candidate target positions includes:

[0097] If the target areas of the vehicle-mounted camera at different installation positions overlap, and each candidate target position is within the overlapping area, then the average of each candidate target position is taken as the first target position of the target in the two-dimensional model of the vehicle-mounted camera.

[0098] Alternatively, if the target areas of the vehicle-mounted camera at different installation positions overlap, and at least one target candidate position is not within the overlapping area, then the average of the target candidate positions within the overlapping area will be used as the first target position of the target in the two-dimensional model of the vehicle-mounted camera.

[0099] Alternatively, if the target areas of the vehicle-mounted camera at different installation positions do not overlap, then it is determined that the target needs to be mobile; a mobile range that can include all target candidate positions is constructed based on each target candidate position, and the mobile range is used as the overall mobile area of ​​the target in the two-dimensional model of the vehicle-mounted camera.

[0100] If the target areas of the vehicle-mounted camera at different installation positions overlap, and all candidate target positions are within the overlap area, then the average of the candidate target positions is used as the first target position of the target in the 2D model of the vehicle-mounted camera. If the target areas of the vehicle-mounted camera at different installation positions overlap, and at least one candidate target position is not within the overlap area, then the average of the candidate target positions within the overlap area is used as the first target position of the target in the 2D model of the vehicle-mounted camera. If the target areas of the vehicle-mounted camera at different installation positions do not overlap, then the target needs to have mobility. A mobility range that can include all candidate target positions is constructed based on each candidate target position, and this mobility range is used as the overall mobility area of ​​the target in the 2D model of the vehicle-mounted camera. Specifically, a minimum bounding box is used to select each candidate target position, and the range of the minimum bounding box on the horizontal plane is used as the horizontal mobility range. The highest and lowest positions of the vehicle-mounted camera at different installation positions determine the vertical mobility range.

[0101] S270. Based on the target installation position of the vehicle-mounted camera on the vehicle, determine the second target position of the target in the vehicle coordinate system, and use the target at the second target position to calibrate the vehicle-mounted camera.

[0102] After determining the first target position of the target in the two-dimensional model of the vehicle camera, the first target position needs to be mapped to the second target position in the vehicle coordinate system, and the vehicle camera needs to be calibrated based on the target at the second target position.

[0103] As an optional but non-limiting implementation, the step of determining the second target position of the target in the vehicle coordinate system based on the target installation position of the vehicle-mounted camera on the vehicle, and using the target at the second target position for vehicle-mounted camera calibration, includes but is not limited to steps D1-D2:

[0104] Step D1: Determine the relative position between the target and the vehicle-mounted camera based on the first target position of the target in the two-dimensional model of the vehicle-mounted camera;

[0105] Step D2: Based on the relative position and the target installation position of the vehicle-mounted camera on the vehicle, determine the second target position of the target in the vehicle coordinate system, and use the target at the second target position to calibrate the vehicle-mounted camera.

[0106] Specifically, after determining the first target position of the target in the two-dimensional model of the vehicle-mounted camera, the relative position of the target and the vehicle-mounted camera is determined based on the spatial mapping relationship between the vehicle-mounted camera and the target; the target installation position of the vehicle-mounted camera on the whole vehicle is determined, and the second target position of the target in the coordinate system of the whole vehicle is determined based on the relative position of the target and the vehicle-mounted camera, and the vehicle-mounted camera is calibrated using the target at the second target position.

[0107] In this embodiment of the invention, a two-dimensional model of the vehicle-mounted camera is constructed to determine the spatial mapping relationship between the vehicle-mounted camera and the target. The target presence area and candidate target positions for different installation locations of the vehicle-mounted camera are determined within the two-dimensional model of the vehicle-mounted camera, and a first target position applicable to different vehicle models and camera installation locations is selected. Based on the spatial mapping relationship between the vehicle-mounted camera and the target, the first target position is mapped to the vehicle coordinate system to obtain a second target position. Using the technical solution of this embodiment, a vehicle-mounted camera calibration station applicable to different vehicle models and camera installation locations is constructed to meet the calibration requirements of multiple vehicle models and multiple camera installation locations, simplifying the design difficulty of the calibration station and improving the calibration efficiency of the vehicle-mounted camera.

[0108] Example 3

[0109] Figure 5 This is a schematic diagram of a vehicle-mounted camera calibration device provided in Embodiment 3 of the present invention. Figure 5 As shown, the device includes:

[0110] The model building module 510 is used to build a two-dimensional model of the vehicle-mounted camera; wherein, the two-dimensional model of the vehicle-mounted camera is used to represent the spatial mapping relationship between the vehicle-mounted camera and the target;

[0111] The target information determination module 520 is used to determine the target presence area and target candidate position of the vehicle-mounted camera in the two-dimensional model of the vehicle-mounted camera based on the different installation positions of the vehicle-mounted camera on the vehicle.

[0112] The first target position determination module 530 is used to determine the first target position of the target in the two-dimensional model of the vehicle-mounted camera based on the target existence area and the target candidate position.

[0113] The second target position determination module 540 is used to determine the second target position of the target in the vehicle coordinate system based on the target installation position of the vehicle-mounted camera on the vehicle, and to use the target at the second target position to calibrate the vehicle-mounted camera.

[0114] Optional, the model building module, specifically used for:

[0115] The vehicle-mounted camera is simplified to the origin in a two-dimensional coordinate system; wherein, the field of view of the vehicle-mounted camera is a fan-shaped region;

[0116] The target is simplified to a straight line segment parallel to the plane where the vehicle-mounted camera is located;

[0117] Based on the target size, vehicle-mounted camera configuration parameters, and vehicle-mounted camera field of view, the target presence area and target candidate positions are determined; where the target size includes the target length and target width, and the vehicle-mounted camera configuration parameters include the vehicle-mounted camera focal length, vehicle-mounted camera pixels, and vehicle-mounted camera resolution.

[0118] A two-dimensional model of the vehicle-mounted camera is constructed based on the origin position of the camera, the area where the target exists, and the candidate positions of the target.

[0119] Optionally, the model building module is also specifically used for:

[0120] Based on the target size, the configuration parameters of the vehicle-mounted camera, and the field of view of the vehicle-mounted camera, determine the near boundary, far boundary, and lateral boundary of the target within the field of view of the vehicle-mounted camera;

[0121] Based on the near boundary, far boundary, and lateral boundary of the target within the field of view of the vehicle-mounted camera, the area where the target exists and the candidate location of the target are determined.

[0122] Optionally, the model building module is also specifically used for:

[0123] Based on the focal length of the vehicle-mounted camera, the length of the target, and the threshold of the number of pixels at the edge of the target, the first near boundary of the target within the field of view of the vehicle-mounted camera is determined;

[0124] Based on the vertical field of view of the vehicle-mounted camera, the target width, the number of targets, the vertical installation error of the vehicle-mounted camera, and the cropping angle of the vehicle-mounted camera algorithm, the second near boundary of the target within the field of view of the vehicle-mounted camera is determined.

[0125] Based on the horizontal field of view of the vehicle-mounted camera, the target width, the number of targets, the horizontal installation error of the vehicle-mounted camera, and the cropping angle of the vehicle-mounted camera algorithm, the third near boundary of the target within the field of view of the vehicle-mounted camera is determined.

[0126] The largest near boundary among the first, second, and third near boundaries is selected as the target near boundary within the field of view of the vehicle-mounted camera.

[0127] Optionally, the model building module is also specifically used for:

[0128] The shorter side is selected from the target length and the target width, and the size of the shorter side of the target is determined; the detection distance of the vehicle-mounted camera is limited by the size of the shorter side of the target.

[0129] Choose the minimum resolution from the horizontal and vertical resolutions of the vehicle camera;

[0130] Based on the target's short side dimensions, the vehicle-mounted camera's minimum resolution, the vehicle-mounted camera's detection accuracy requirements, and the target's minimum module size, the target's far boundary within the vehicle-mounted camera's field of view is determined.

[0131] The far boundary is represented as:

[0132] ;

[0133] in, L represents the far boundary, W represents the target length, m represents the horizontal resolution of the vehicle-mounted camera, n represents the vertical resolution of the vehicle-mounted camera, p represents the detection accuracy requirement of the vehicle-mounted camera, and P represents the minimum module size of the target.

[0134] Optionally, the model building module is also specifically used for:

[0135] Based on the horizontal field of view of the vehicle-mounted camera, the horizontal installation error of the vehicle-mounted camera, and the cropping angle of the camera algorithm, the left and right boundaries of the target within the field of view of the vehicle-mounted camera are determined.

[0136] Optionally, the target first position determination module is specifically used for:

[0137] If the target areas of the vehicle-mounted camera at different installation positions overlap, and each candidate target position is within the overlapping area, then the average of each candidate target position is taken as the first target position of the target in the two-dimensional model of the vehicle-mounted camera.

[0138] Alternatively, if the target areas of the vehicle-mounted camera at different installation positions overlap, and at least one target candidate position is not within the overlapping area, then the average of the target candidate positions within the overlapping area will be used as the first target position of the target in the two-dimensional model of the vehicle-mounted camera.

[0139] Alternatively, if the target areas of the vehicle-mounted camera at different installation positions do not overlap, then it is determined that the target needs to be mobile; a mobile range that can include all target candidate positions is constructed based on each target candidate position, and the mobile range is used as the overall mobile area of ​​the target in the two-dimensional model of the vehicle-mounted camera.

[0140] Optionally, the target second position determination module is specifically used for:

[0141] Based on the first target position of the target in the two-dimensional model of the vehicle-mounted camera, determine the relative position of the target and the vehicle-mounted camera;

[0142] Based on the relative position and the target installation position of the vehicle-mounted camera on the vehicle, the second target position of the target in the vehicle coordinate system is determined, and the vehicle-mounted camera is calibrated using the target at the second target position.

[0143] The vehicle-mounted camera calibration device provided in the embodiments of the present invention can execute the vehicle-mounted camera calibration method provided in any of the embodiments of the present invention, and has the corresponding functions and beneficial effects of executing the vehicle-mounted camera calibration method. For details, please refer to the relevant operations of the vehicle-mounted camera calibration method in the foregoing embodiments.

[0144] Example 4

[0145] Figure 6 A schematic diagram of an electronic device 10, which can be used to implement embodiments of the present invention, is shown. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device can also represent various forms of mobile devices, such as personal digital processors, cellular phones, smartphones, wearable devices (e.g., helmets, glasses, watches, etc.), and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely illustrative and are not intended to limit the implementation of the invention described and / or claimed herein.

[0146] like Figure 6As shown, the electronic device 10 includes at least one processor 11 and a memory, such as a read-only memory (ROM) 12 or a random access memory (RAM) 13, communicatively connected to the at least one processor 11. The memory stores computer programs executable by the at least one processor. The processor 11 can perform various appropriate actions and processes based on the computer program stored in the ROM 12 or loaded from storage unit 18 into the RAM 13. The RAM 13 can also store various programs and data required for the operation of the electronic device 10. The processor 11, ROM 12, and RAM 13 are interconnected via a bus 14. An input / output (I / O) interface 15 is also connected to the bus 14.

[0147] Multiple components in electronic device 10 are connected to I / O interface 15, including: input unit 16, such as keyboard, mouse, etc.; output unit 17, such as various types of displays, speakers, etc.; storage unit 18, such as disk, optical disk, etc.; and communication unit 19, such as network card, modem, wireless transceiver, etc. Communication unit 19 allows electronic device 10 to exchange information / data with other devices through computer networks such as the Internet and / or various telecommunications networks.

[0148] Processor 11 can be a variety of general-purpose and / or special-purpose processing components with processing and computing capabilities. Some examples of processor 11 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various special-purpose artificial intelligence (AI) computing chips, various processors running machine learning model algorithms, a digital signal processor (DSP), and any suitable processor, controller, microcontroller, etc. Processor 11 performs the various methods and processes described above, such as the vehicle camera calibration method.

[0149] In particular, according to embodiments of the present invention, the processes described above with reference to the flowcharts can be implemented as computer software programs. For example, embodiments of the present invention include a computer program product comprising a computer program carried on a non-transitory computer-readable medium, the computer program containing program code for performing the methods shown in the flowcharts. In such embodiments, the computer program can be downloaded and installed from a network via communication unit 19, or installed from storage unit 18, or installed from ROM 12. When the computer program is executed by processor 11, it performs the functions defined in the methods of the embodiments of the present invention.

[0150] In some embodiments, the vehicle camera calibration method may be implemented as a computer program tangibly contained in a computer-readable storage medium, such as storage unit 18. In some embodiments, part or all of the computer program may be loaded and / or installed on electronic device 10 via ROM 12 and / or communication unit 19. When the computer program is loaded into RAM 13 and executed by processor 11, one or more steps of the vehicle camera calibration method described above may be performed. Alternatively, in other embodiments, processor 11 may be configured to perform the vehicle camera calibration method by any other suitable means (e.g., by means of firmware).

[0151] Various embodiments of the systems and techniques described above herein can be implemented in digital electronic circuit systems, integrated circuit systems, field-programmable gate arrays (FPGAs), application-specific integrated circuits (ASICs), application-specific standard products (ASSPs), system-on-a-chip (SoCs), complex programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various embodiments may include implementations in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which may be a dedicated or general-purpose programmable processor, capable of receiving data and instructions from a storage system, at least one input device, and at least one output device, and transferring data and instructions to the storage system, the at least one input device, and the at least one output device.

[0152] Computer programs used to implement the methods of the present invention may be written in any combination of one or more programming languages. These computer programs may be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing device, such that when executed by the processor, the computer programs cause the functions / operations specified in the flowcharts and / or block diagrams to be performed. The computer programs may be executed entirely on a machine, partially on a machine, or as a standalone software package, partially on a machine and partially on a remote machine, or entirely on a remote machine or server.

[0153] In the context of this invention, a computer-readable storage medium can be a tangible medium that may contain or store a computer program for use by or in conjunction with an instruction execution system, apparatus, or device. A computer-readable storage medium may include, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatus, or devices, or any suitable combination thereof. Alternatively, a computer-readable storage medium may be a machine-readable signal medium. More specific examples of machine-readable storage media include electrical connections based on one or more wires, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fibers, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination thereof.

[0154] To provide interaction with a user, the systems and techniques described herein can be implemented on an electronic device having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user; and a keyboard and pointing device (e.g., a mouse or trackball) through which the user provides input to the electronic device. Other types of devices can also be used to provide interaction with the user; for example, feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including sound input, voice input, or tactile input).

[0155] The systems and technologies described herein can be implemented in computing systems that include backend components (e.g., as data servers), or middleware components (e.g., application servers), or frontend components (e.g., user computers with graphical user interfaces or web browsers through which users can interact with implementations of the systems and technologies described herein), or any combination of such backend, middleware, or frontend components. The components of the system can be interconnected via digital data communication of any form or medium (e.g., communication networks). Examples of communication networks include local area networks (LANs), wide area networks (WANs), blockchain networks, and the Internet.

[0156] A computing system can include clients and servers. Clients and servers are generally located far apart and typically interact through communication networks. The client-server relationship is created by computer programs running on the respective computers and having a client-server relationship with each other. The server can be a cloud server, also known as a cloud computing server or cloud host, which is a hosting product within the cloud computing service system to address the shortcomings of traditional physical hosts and VPS services, such as high management difficulty and weak business scalability.

[0157] It should be understood that the various forms of processes shown above can be used, with steps reordered, added, or deleted. For example, the steps described in this invention can be executed in parallel, sequentially, or in different orders, as long as the desired result of the technical solution of this invention can be achieved, and this is not limited herein.

[0158] The specific embodiments described above do not constitute a limitation on the scope of protection of this invention. Those skilled in the art should understand that various modifications, combinations, sub-combinations, and substitutions can be made according to design requirements and other factors. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of this invention should be included within the scope of protection of this invention.

Claims

1. A vehicle camera calibration method, characterized by, The method comprises: constructing a vehicle-mounted camera two-dimensional model, wherein the vehicle-mounted camera two-dimensional model is used to represent a spatial mapping relationship between the vehicle-mounted camera and a target; determining a target existing area and a target candidate position of the target in the vehicle-mounted camera two-dimensional model according to different installation positions of the vehicle-mounted camera on the whole vehicle; determining a first target position of the target in the vehicle-mounted camera two-dimensional model according to the target existing area and the target candidate position; determining a second target position of the target in the whole vehicle coordinate according to a target installation position of the vehicle-mounted camera on the whole vehicle, and performing vehicle-mounted camera calibration by using the target at the second target position.

2. The method of claim 1, wherein, The construction of the vehicle-mounted camera two-dimensional model comprises: simplifying the vehicle-mounted camera as an origin in a two-dimensional coordinate system, wherein the field of view angle of the vehicle-mounted camera is a sector area; simplifying the target as a straight line segment parallel to the plane where the vehicle-mounted camera is located; determining the target existing area and the target candidate position of the target according to the target size, the vehicle-mounted camera configuration parameters and the vehicle-mounted camera field of view angle, wherein the target size comprises a target length and a target width, and the vehicle-mounted camera configuration parameters comprise a vehicle-mounted camera focal length, a vehicle-mounted camera pixel and a vehicle-mounted camera resolution; constructing the vehicle-mounted camera two-dimensional model according to the origin position of the vehicle-mounted camera, the target existing area and the target candidate position.

3. The method of claim 2, wherein, The determination of the target existing area and the target candidate position of the target according to the target size, the vehicle-mounted camera configuration parameters and the vehicle-mounted camera field of view angle comprises: determining a near boundary, a far boundary and a side boundary of the target in the vehicle-mounted camera field of view angle according to the target size, the vehicle-mounted camera configuration parameters and the vehicle-mounted camera field of view angle; determining the target existing area and the target candidate position according to the near boundary, the far boundary and the side boundary of the target in the vehicle-mounted camera field of view angle.

4. The method of claim 3, wherein, The determination of the near boundary of the target in the vehicle-mounted camera field of view angle according to the target size, the vehicle-mounted camera configuration parameters and the vehicle-mounted camera field of view angle comprises: determining a first near boundary of the target in the vehicle-mounted camera field of view angle according to the vehicle-mounted camera focal length, the target length and a target edge pixel number threshold value; determining a second near boundary of the target in the vehicle-mounted camera field of view angle according to a vertical field of view angle of the vehicle-mounted camera, the target width, a target number, a vertical installation error of the vehicle-mounted camera and an algorithm clipping angle of the vehicle-mounted camera; determining a third near boundary of the target in the vehicle-mounted camera field of view angle according to a horizontal field of view angle of the vehicle-mounted camera, the target width, the target number, a horizontal installation error of the vehicle-mounted camera and the algorithm clipping angle of the vehicle-mounted camera; selecting a maximum near boundary from the first near boundary, the second near boundary and the third near boundary as a target near boundary of the target in the vehicle-mounted camera field of view angle.

5. The method of claim 3, wherein, The determination of the far boundary of the target in the vehicle-mounted camera field of view angle according to the target size, the vehicle-mounted camera configuration parameters and the vehicle-mounted camera field of view angle comprises: selecting a short side from the target length and the target width and determining a target short side size, wherein a detection distance of the vehicle-mounted camera is limited by the target short side size; selecting a minimum resolution from a horizontal resolution and a vertical resolution of the vehicle-mounted camera; According to a target short side size, a vehicle-mounted camera minimum resolution, a vehicle-mounted camera detection accuracy requirement, and a target minimum module size, a far boundary of the target in a vehicle-mounted camera field of view angle is determined; The far boundary is expressed as: ； wherein, wherein L denotes the target length, W denotes the target width, m denotes the horizontal resolution of the vehicle-mounted camera, n denotes the vertical resolution of the vehicle-mounted camera, p denotes the detection accuracy requirement of the vehicle-mounted camera, and P denotes the minimum module size of the target.

6. The method of claim 3, wherein, According to a target size, a vehicle-mounted camera configuration parameter, and a vehicle-mounted camera field of view angle, a side boundary of the target in the vehicle-mounted camera field of view angle is determined, including: According to a vehicle-mounted camera horizontal field of view angle, a vehicle-mounted camera horizontal installation error, and a camera algorithm clipping angle, left and right two side boundaries of the target in the vehicle-mounted camera field of view angle are determined.

7. The method of claim 1, wherein, The first target position of the target in the vehicle-mounted camera two-dimensional model is determined according to the target existing area and the target candidate position, including: If the target existing areas of the target under different installation positions of the vehicle-mounted camera have an intersection, and each target candidate position is in the intersection area, an average of each target candidate position is taken as the first target position of the target in the vehicle-mounted camera two-dimensional model; Or, if the target existing areas of the target under different installation positions of the vehicle-mounted camera have an intersection, and there is at least one target candidate position not in the intersection area, an average of the target candidate positions in the intersection area is taken as the first target position of the target in the vehicle-mounted camera two-dimensional model; Or, if the target existing areas of the target under different installation positions of the vehicle-mounted camera do not have an intersection, it is determined that the target needs to have a moving ability; a moving range capable of containing all the target candidate positions is constructed according to each target candidate position, and the moving range is taken as an overall moving area of the target in the vehicle-mounted camera two-dimensional model.

8. The method of claim 1, wherein, The second target position of the target in the vehicle coordinate is determined according to the target installation position on the vehicle, and the target at the second target position is used for vehicle-mounted camera calibration, including: A relative position of the target and the vehicle-mounted camera is determined according to the first target position of the target in the vehicle-mounted camera two-dimensional model; The second target position of the target in the vehicle coordinate is determined according to the relative position and the target installation position of the vehicle-mounted camera on the vehicle, and the target at the second target position is used for vehicle-mounted camera calibration.

9. A vehicle-mounted camera calibration device, characterized by comprising: The device includes: A model construction module is configured to construct a vehicle-mounted camera two-dimensional model; wherein the vehicle-mounted camera two-dimensional model is used to represent a spatial mapping relationship between the vehicle-mounted camera and the target; A target information determination module is configured to determine, according to different installation positions of the vehicle-mounted camera on the vehicle, a target existing area and a target candidate position of the target under different installation positions of the vehicle-mounted camera in the vehicle-mounted camera two-dimensional model; A target first position determination module is configured to determine, according to the target existing area and the target candidate position, a first target position of the target in the vehicle-mounted camera two-dimensional model; A target second position determination module is configured to determine, according to a target installation position of the vehicle-mounted camera on the vehicle, a second target position of the target in the vehicle coordinate, and use the target at the second target position for vehicle-mounted camera calibration.

10. An electronic device, comprising: The electronic device includes: at least one processor; and a memory connected with the at least one processor in communication; wherein, The memory stores a computer program executable by the at least one processor, and the computer program is executed by the at least one processor to enable the at least one processor to perform the vehicle-mounted camera calibration method in any one of claims 1-8.

11. A computer readable storage medium, characterized in that, The computer readable storage medium stores computer instructions for enabling the processor to implement the vehicle-mounted camera calibration method in any one of claims 1-8 when executed.

12. A computer program product, characterised in that, The computer program product comprises a computer program which, when executed by a processor, implements the vehicle-mounted camera calibration method according to any one of claims 1-8.

Images