Camera external parameter calibration method and related equipment

Abstract

The invention relates to the technical field of camera external parameter calibration, in particular to a camera external parameter calibration method and related equipment. The method comprises the following steps: acquiring image data about a target area acquired by a camera; a plurality of local black and white lattice patterns are arranged in the target area; identifying at least one local black and white lattice pattern from the image data; determining the position of an angular point in each local black and white lattice pattern; determining a two-dimensional coordinate of the angular point in a camera coordinate system according to the position of the angular point in each local black and white lattice pattern; acquiring a corresponding three-dimensional coordinate of each angular point in a world coordinate system; and external parameter information of the camera is determined based on the two-dimensional coordinates and the three-dimensional coordinates of all the angular points.

Description

[Technical field]

[0001] The present invention relates to the technical field of camera extrinsic parameter calibration, and in particular to a camera extrinsic parameter calibration method and related equipment. [Background technology]

[0002] At present, in order to ensure the safety of vehicle driving, multiple cameras are often set inside the vehicle to collect image data of the interior space of the vehicle, which is used to detect fatigue and distracted driving of the driver in the driving seat. Image acquisition of the driving seat is often achieved by an infrared camera. In addition to detecting the driver in the driving seat, the behavior of the passengers in the car is also identified. For example, whether there is a baby in the car, whether there is a pet, whether the pet is in the cabin, etc. Image acquisition of passengers in the car is often achieved by a wide-angle camera. For example, a wide-angle infrared camera set near the rearview mirror in the car is used to collect images of the interior space. In the process of implementing the above detection, it is necessary to calibrate the external parameters of each camera. That is, calculate the position and posture relationship between the camera coordinate system and the world coordinate system. When calibrating the external parameters of the camera, it is necessary to input the 2D coordinates of multiple points in the camera coordinate system and the 3D coordinates of these points in the world coordinate system, and then calculate the camera external parameters. In the prior art, a large black and white chessboard pattern is often set in the car, and then the calibration is completed according to the 2D coordinates and 3D coordinates of the corner points of the black and white chessboard captured in the camera. However, this method requires the use of a large black and white chessboard, which is prone to occlusion problems. In addition, in the image captured by the wide-angle camera, the black and white chessboard only occupies a very small part of the imaging result and cannot reflect the actual situation of the entire interior space of the vehicle, thus leading to inaccurate calibration results. [Summary of the invention]

[0003] In order to solve the above problems, the embodiments of the present invention provide a camera extrinsic calibration method and related equipment, which completes the camera extrinsic calibration by setting multiple local black and white grid patterns at different locations in the car. This overcomes the occlusion problem caused by the black and white grid pattern being too large and the problem of inaccurate wide-angle camera calibration in the prior art.

[0004] In a first aspect, an embodiment of the present invention provides a camera extrinsic calibration method, comprising:

[0005] Acquire image data of a target area collected by a camera; a plurality of local black and white grid patterns are arranged in the target area;

[0006] Identify at least one of the local black-and-white checkered patterns from the image data;

[0007] Determine the position of the corner point in each of the local black and white grid patterns;

[0008] Determine the two-dimensional coordinates of the corner points in the camera coordinate system according to the positions of the corner points in each local black and white grid pattern;

[0009] Obtain the three-dimensional coordinates corresponding to each corner point in the world coordinate system;

[0010] The external parameter information of the camera is determined based on the two-dimensional coordinates and the three-dimensional coordinates of each of the corner points.

[0011] In a possible implementation manner, determining the position of the corner point in each of the local black-and-white checkered patterns includes:

[0012] Identify the border position of each of the local black and white grid patterns based on a first preset algorithm;

[0013] Determine the area where the local black and white grid pattern is located according to the position of the frame;

[0014] Based on a second preset algorithm, identifying a boundary between black and white grids in the area where the local black and white grid pattern is located;

[0015] The positions of the corner points in the local black-and-white grid pattern are determined according to the boundaries between the black grid and the white grid.

[0016] In a possible implementation manner, before obtaining the three-dimensional coordinates corresponding to each corner point in the world coordinate system, the method further includes:

[0017] Scanning the target area to obtain scanning result information;

[0018] Determine edge contour information of the plurality of local black and white grid patterns according to the scanning result information;

[0019] The three-dimensional coordinates of the corresponding multiple corner points are obtained based on the edge contour information of each local black and white grid pattern.

[0020] In a possible implementation, obtaining the three-dimensional coordinates corresponding to each corner point in the world coordinate system includes:

[0021] A target three-dimensional coordinate matching the corner point is determined from the multiple three-dimensional coordinates to serve as the three-dimensional coordinate corresponding to the corner point.

[0022] In a possible implementation, identification information is further provided on the local black and white checkered pattern, and determining a target three-dimensional coordinate matching the corner point from the plurality of three-dimensional coordinates includes:

[0023] Marking the three-dimensional coordinates with a corresponding first label according to the identification information of the local black and white checkered pattern;

[0024] Identifying identification information corresponding to each of the local black and white grid patterns from the image data;

[0025] According to the identification information, the two-dimensional coordinates of the corner points identified in each local black and white grid pattern are marked with corresponding second labels;

[0026] Determine a target first tag matching the second tag from the plurality of first tags;

[0027] The three-dimensional coordinates corresponding to the target first label are determined as the target three-dimensional coordinates matching the corner point.

[0028] In a possible implementation manner, after determining the extrinsic parameter information of the camera based on the two-dimensional coordinates and the three-dimensional coordinates of each of the corner points, the method further includes:

[0029] Determine the projection coordinates of each corner point after projecting it onto the camera coordinate system according to the external parameter information and the three-dimensional coordinates of each corner point;

[0030] respectively determining the difference between each of the projection coordinates and the corresponding two-dimensional coordinates;

[0031] determining an average of the differences;

[0032] If the average value is greater than or equal to a preset threshold, determining the corner point corresponding to the maximum difference among the differences as an abnormal corner point;

[0033] The external parameter information of the camera is recalculated according to the three-dimensional coordinates and the two-dimensional coordinates of the remaining corner points except the abnormal corner point, until the average value is less than the preset threshold or satisfies the preset condition.

[0034] In a possible implementation, determining the projection coordinates of each corner point after being projected onto a camera coordinate system according to the extrinsic parameter information and the three-dimensional coordinates of each corner point includes:

[0035] Converting the world coordinate system into a projection coordinate system corresponding to the camera coordinate system according to the external parameter information;

[0036] The projection coordinates of the corner point in the projection coordinate system are determined according to the intrinsic parameter information of the camera and the three-dimensional coordinates of the corner point.

[0037] In a second aspect, an embodiment of the present invention provides a camera extrinsic calibration device, comprising:

[0038] An acquisition module, used to acquire image data of a target area collected by a camera; a plurality of local black and white grid patterns are arranged in the target area;

[0039] A recognition module, used to recognize at least one of the local black and white checkered patterns from the image data;

[0040] A processing module, used to determine the position of the corner point in each of the local black and white grid patterns;

[0041] The processing module is further used to determine the two-dimensional coordinates of the corner point in the camera coordinate system according to the position of the corner point in each local black and white grid pattern;

[0042] The acquisition module is further used to acquire the three-dimensional coordinates corresponding to each corner point in the world coordinate system;

[0043] The processing module is further used to determine the external parameter information of the camera based on the two-dimensional coordinates and the three-dimensional coordinates of each of the corner points.

[0044] In a third aspect, an embodiment of the present invention provides an electronic device, comprising: at least one processor; the processor is coupled to a memory, wherein: the memory stores program instructions executable by the processor, and the processor calls the program instructions to execute the method described in the first aspect.

[0045] In a fourth aspect, an embodiment of the present invention provides a computer-readable storage medium, wherein the computer-readable storage medium stores computer instructions, and the computer instructions enable the computer to execute the method described in the first aspect.

[0046] In a fifth aspect, an embodiment of the present invention provides a computer program product comprising instructions, which, when executed on a computer or at least one processor, enables the computer to execute the method described in the first aspect.

[0047] In a sixth aspect, an embodiment of the present invention provides a vehicle, comprising a plurality of cameras, wherein the plurality of cameras are calibrated with external parameters based on the method described in the first aspect.

[0048] It should be understood that the second to sixth aspects of the embodiments of the present invention are consistent with the technical solutions of the first aspect of the embodiments of the present invention, and the beneficial effects achieved by each aspect and the corresponding feasible implementation methods are similar and will not be described in detail.

[0049] In the embodiment of the present invention, multiple local black and white checkered patterns are set in the target area and image data of the target area collected by the camera is obtained. Then, the local black and white checkered patterns are identified from the image data to complete the external parameter information of the camera, so that more in-vehicle information can be obtained without interfering with the driver's driving, thereby improving the accuracy of external parameter calibration.

Brief Description of the Drawings

[0050] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings required for use in the embodiments are briefly introduced below. Obviously, the drawings described below are only some embodiments of the embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0051] Figure 1 A flowchart of a camera extrinsic calibration method provided by an embodiment of the present invention;

[0052] Figure 2 A schematic diagram of a local black and white checkered pattern provided by an embodiment of the present invention;

[0053] Figure 3 A flowchart of another camera extrinsic calibration method provided by an embodiment of the present invention;

[0054] Figure 4 A schematic structural diagram of a camera extrinsic calibration device provided by an embodiment of the present invention;

[0055] Figure 5 A schematic structural diagram of another camera extrinsic calibration device provided by an embodiment of the present invention;

[0056] Figure 6 A schematic diagram of the structure of an electronic device provided by an embodiment of the present invention. [Specific implementation method]

[0057] In order to better understand the technical solution of this specification, the embodiments of the present invention are described in detail below with reference to the accompanying drawings.

[0058] It should be clear that the described embodiments are only part of the embodiments of this specification, not all of the embodiments. Based on the embodiments in this specification, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0059] The terms used in the embodiments of the present invention are only for the purpose of describing specific embodiments, and are not intended to limit this specification. The singular forms "a", "said" and "the" used in the embodiments of the present invention and the appended claims are also intended to include plural forms, unless the context clearly indicates other meanings.

[0060] Figure 1 A schematic diagram of a camera extrinsic calibration method provided by an embodiment of the present invention. Figure 1 As shown in , the method processing steps include:

[0061] Step 101, obtaining image data of a target area collected by a camera. A plurality of local black and white grid patterns are set in the target area. Figure 2 Schematic diagram of a partial black and white checkered pattern provided by an embodiment of the present invention. Figure 2 As shown in , the local black and white grid is a 2×2 black and white grid pattern, with two groups of squares of pure white and pure black at the diagonal corners. Therefore, multiple local black and white grids can be set in various places in the interior space of the vehicle, so that the camera can capture more black and white grid patterns, thereby increasing the number of black and white grid patterns involved in external parameter settlement and improving the accuracy of external parameter calibration.

[0062] Specifically, multiple local black and white checkered patterns can be set from the main driver's seat near the B-pillar to the area on the roof. Then, it can be extended in the direction of the co-pilot seat, and multiple local black and white checkered patterns can also be set in the area on the co-pilot seat near the B-pillar. Optionally, multiple local black and white checkered patterns can also be set near the roof handrail. By setting the local black and white checkered pattern as described above, the calibration result has better robustness, and since only part of the black and white checkered pattern is used, the occlusion of the interior space is small, and more effective in-vehicle information can be obtained when performing fatigue driving detection or distracted driving detection and other detection systems. And it will not interfere with the driver, which is convenient for the expansion of dynamic acquisition. Furthermore, it is also friendly to wide-angle cameras. For wide-angle cameras, since the edges of the camera will produce large distortion, the calculated external parameters can be made more accurate.

[0063] Step 102, identifying at least one local black-and-white checkered pattern from the image data, wherein an existing target detection algorithm or other intelligent algorithm or detection model may be used to identify the local black-and-white checkered pattern from the image data based on the pixel characteristics of the local black-and-white checkered pattern.

[0064] Step 103, determine the position of the corner point in each local black and white checkered pattern. Taking the above-mentioned 2×2 black and white checkered pattern as an example, the corner point is the point where the four squares in the middle of the pattern intersect. Each 2×2 local black and white checkered pattern has only one corner point. Since the deployment positions of each camera are different and the shooting angles are also different, the number and position of the local black and white checkered patterns that can be captured by each camera are also different. Therefore, for each camera, the image data captured by each camera can be roughly positioned for the local black and white checkered pattern, and then the precise position of the corner point can be identified based on the rough positioning result.

[0065] Specifically, the border position of each local black and white grid pattern can be first identified based on the first preset algorithm. The first preset algorithm can be a target detection algorithm such as Faster-rcnn, FCOS, and YOLO. The border position of the local black and white grid pattern can be specifically in the form of coordinates of the four vertices of the local black and white grid. Since the local black and white grid is a rectangular shape. Therefore, it can also be implemented in the form of coordinates of the upper left vertex + coordinates of the lower right vertex. Or, the coordinates of the upper right vertex + the coordinates of the lower left vertex.

[0066] Secondly, the area where the local black and white checkered pattern is located is determined according to the second frame position. That is, the frame coordinates of the local black and white checkered pattern are first determined from the image data, and then the coordinates of the corner points are determined from the area where the frame coordinates are located. Compared with directly identifying the corner points from the image data, this method has lower data processing pressure and higher efficiency. Alternatively, the rough positioning result of the local black and white checkered pattern can be determined by matching the image data with multiple templates prepared in advance through template matching, so as to more robustly match the local black and white checkered pattern of various illumination, angle, and scale. Alternatively, template matching can be combined with a target detection algorithm. When training the target detection algorithm, the output of template matching can be used to pre-label the training data, thereby reducing the workload of manual labeling.

[0067] Afterwards, the boundary between the black grid and the white grid can be identified from the area where the black and white grid pattern is located based on the second preset algorithm. Finally, the position of the corner point in the local black and white grid pattern is determined according to the boundary between the black grid and the white grid. Specifically, the second preset algorithm can be a convolutional neural network model for key point positioning. Alternatively, after preprocessing the image data, an edge detection algorithm such as the canny operator is used to extract the edge of the local black and white grid pattern. Afterwards, the rectangular outline of the local black and white grid pattern is extracted by polygonal approximation and the like, and after determining the two rectangular centers, the two rectangular centers are intercepted and corner detection is performed using a corner detection algorithm (Shi-Tomasi and goodFeaturesToTrack, etc.) to find the corner points. Afterwards, a more accurate corner point position is found by a sub-pixel corner detection algorithm (cornerSubPix, etc.). Optionally, the canny operator edge detection method can also be used to annotate the training data, and the annotated training data can be used to train the convolutional neural network model for key point positioning, thereby reducing the cost of manual annotation and improving the training efficiency of the network model.

[0068] Step 104 , determining the two-dimensional coordinates of the corner points in the camera coordinate system according to the positions of the corner points in each local black-and-white checkered pattern.

[0069] Step 105, obtain the three-dimensional coordinates corresponding to each corner point in the world coordinate system. The purpose of this step is to match the three-dimensional coordinates with the two-dimensional coordinates, so as to determine which three-dimensional coordinates the two-dimensional coordinates of the identified corner point correspond to. Among them, the three-dimensional coordinates can be obtained by scanning the target area with a scanning device such as a laser scanner or a total station to obtain scanning result information. Among them, the scanning result information can be point cloud information or laser reflection information, etc. Then, the edge contour information of multiple local black and white grid patterns is determined according to the scanning result information. The three-dimensional coordinates of multiple corner points are obtained based on the edge contour information of each local black and white grid pattern. Afterwards, when the three-dimensional coordinates corresponding to each corner point are obtained according to the two-dimensional coordinates, the target three-dimensional coordinates matching the corner point can be determined from the multiple three-dimensional coordinates obtained in the above steps as the three-dimensional coordinates corresponding to the corner point.

[0070] Specifically, each local black and white grid group is also provided with identification information, and the three-dimensional coordinates corresponding to each corner point can be determined based on the identification information. Optionally, a black-bottomed cardboard that is not easy to reflect light can be set next to the local chessboard pattern, and then identification information such as numbers or letters can be printed on it, so as to identify each local chessboard. First, the three-dimensional coordinates can be marked with a corresponding first label according to the identification information of the local black and white grid pattern. Then, the identification information corresponding to each local black and white grid pattern is identified from the image data. According to the identification information, the two-dimensional coordinates of the corner points identified in each local black and white grid pattern are marked with a corresponding second label. Then, the target first label matching the second label is determined from multiple first labels. Finally, the three-dimensional coordinates corresponding to the target first label are determined as the target three-dimensional coordinates matching the corner point. For example, the identification information is a number, and each three-dimensional coordinate has been marked. Then, the number corresponding to the local black and white grid pattern is identified from the image data using a recognition algorithm or the like. Then, the three-dimensional coordinates marked with the same number are searched from multiple three-dimensional coordinates as the three-dimensional coordinates of the corner point of the local black and white grid pattern.

[0071] Step 106, determine the extrinsic parameter information of the camera based on the two-dimensional coordinates and three-dimensional coordinates of each corner point. After obtaining multiple sets of corresponding three-dimensional coordinates and two-dimensional coordinates through the above steps, the extrinsic parameters of each camera can be calculated using algorithms such as solvepnp.

[0072] In some embodiments, in order to prevent partial coordinate inaccuracies from affecting the accuracy of the final external parameter calibration, the abnormal corner points with inaccurate coordinates can be first found and removed, and then new external parameter information can be recalculated. Figure 3 Flow chart of another camera extrinsic calibration method provided by an embodiment of the present invention. Figure 3 As shown in , the method includes:

[0073] Step 301, based on the external parameter information and the three-dimensional coordinates of each corner point, determine the projection coordinates of each corner point after being projected to the camera coordinate system. Specifically, the world coordinate system can be first converted into a projection coordinate system corresponding to the camera coordinate system based on the external parameter information. Then, based on the intrinsic parameter information of the camera and the three-dimensional coordinates of the corner point, determine the projection coordinates of the corner point in the projection coordinate system.

[0074] Step 302, respectively determine the difference between each projection coordinate and the corresponding two-dimensional coordinate, wherein the two-dimensional coordinate corresponding to each projection coordinate is the two-dimensional coordinate of the corner point corresponding to the projection coordinate.

[0075] Step 303: determine the average value of each difference.

[0076] Step 304: If the average value is greater than or equal to the preset threshold, the corner point corresponding to the maximum difference in the differences is determined as an abnormal corner point.

[0077] Step 305, recalculate the camera's external parameter information based on the three-dimensional coordinates and two-dimensional coordinates of the remaining corner points except the abnormal corner points, until the average value is less than a preset threshold or meets a preset condition. Through this method, the calculation is repeated multiple times, and the point with the largest difference is eliminated each time, until the average value is less than the preset threshold or the number of cycles reaches the target number of times, the cycle is stopped, so as to obtain more accurate external parameter information.

[0078] Corresponding to the above-mentioned camera extrinsic parameter calibration method, an embodiment of the present invention provides a camera extrinsic parameter calibration device. Figure 4 The present invention provides a schematic diagram of the structure of a camera extrinsic calibration device. Figure 4 As shown in , the camera extrinsic parameter calibration device includes: an acquisition module 401, an identification module 402 and a processing module 403.

[0079] The acquisition module 401 is used to acquire image data of a target area collected by a camera. A plurality of local black and white grid patterns are arranged in the target area.

[0080] The recognition module 402 is used to recognize at least one local black and white checkered pattern from the image data.

[0081] The processing module 403 is used to determine the position of the corner point in each local black and white grid pattern.

[0082] The processing module 403 is further configured to determine the two-dimensional coordinates of the corner points in the camera coordinate system according to the positions of the corner points in each local black and white checkered pattern.

[0083] The acquisition module 401 is further used to acquire the three-dimensional coordinates corresponding to each corner point in the world coordinate system.

[0084] The processing module 403 is further configured to determine the external parameter information of the camera based on the two-dimensional coordinates and the three-dimensional coordinates of each corner point.

[0085] In some embodiments, the processing module 403 is specifically used to:

[0086] Based on the first preset algorithm, the frame position of each local black and white checkered pattern is identified. According to the frame position, the area where the local black and white checkered pattern is located is determined. Based on the second preset algorithm, the boundary between the black and white grids is identified from the area where the local black and white checkered pattern is located. According to the boundary between the black and white grids, the position of the corner point in the local black and white checkered pattern is determined.

[0087] In some embodiments, the camera extrinsic calibration device also includes a scanning module. Figure 5 A schematic diagram of the structure of another camera extrinsic calibration device provided by an embodiment of the present invention. Figure 5 As shown in , the camera extrinsic parameter calibration device also includes a scanning module 404.

[0088] The scanning module 404 is used to scan the target area and obtain scanning result information.

[0089] The processing module 403 is further used to determine edge contour information of multiple local black and white checkered patterns according to the scanning result information, and obtain the three-dimensional coordinates of multiple corresponding corner points based on the edge contour information of each local black and white checkered pattern.

[0090] In some embodiments, the acquisition module 401 is specifically used to:

[0091] A target three-dimensional coordinate matching the corner point is determined from the multiple three-dimensional coordinates to serve as the three-dimensional coordinate corresponding to the corner point.

[0092] In some embodiments, identification information is also provided on the partial black and white checkered pattern, and the processing module 403 is specifically used to:

[0093] The three-dimensional coordinates are marked with a corresponding first label according to the identification information of the local black and white grid pattern.

[0094] The recognition module 402 is further used to recognize identification information corresponding to each local black and white grid pattern from the image data.

[0095] The processing module 403 is further configured to annotate the two-dimensional coordinates of the corner points identified in each local black and white checkered pattern with corresponding second labels according to the identification information, determine a target first label matching the second label from the plurality of first labels, and determine the three-dimensional coordinates corresponding to the target first label as the target three-dimensional coordinates matching the corner point.

[0096] The processing module 403 is further used for:

[0097] According to the external parameter information and the three-dimensional coordinates of each corner point, the projection coordinates of each corner point after being projected to the camera coordinate system are determined. The difference between each projection coordinate and the corresponding two-dimensional coordinate is determined respectively. The average value of each difference is determined. If the average value is greater than or equal to the preset threshold, the corner point corresponding to the maximum difference in the difference is determined as an abnormal corner point. The external parameter information of the camera is recalculated according to the three-dimensional coordinates and two-dimensional coordinates of the remaining corner points except the abnormal corner point until the average value is less than the preset threshold or the preset condition is met.

[0098] In some embodiments, the processing module 403 is specifically used to:

[0099] The world coordinate system is converted into a projection coordinate system corresponding to the camera coordinate system according to the external parameter information. The projection coordinates of the corner point in the projection coordinate system are determined according to the camera's internal parameter information and the three-dimensional coordinates of the corner point.

[0100] Figure 4 and Figure 5 The camera extrinsic calibration device provided in the embodiment shown can be used to implement the present invention. Figure 1 to Figure 3 The technical solution of the method embodiment shown, its implementation principle and technical effects can be further referred to the relevant description in the method embodiment.

[0101] Figure 6 A schematic diagram of the structure of another electronic device provided by an embodiment of the present invention, such as Figure 6 As shown, the electronic device may include a processor, a memory, and a computer program, wherein the computer program is stored in the memory, and the computer program includes instructions, and when the instructions are executed, the electronic device executes the instructions of this specification. Figure 1 to Figure 3 The illustrated embodiment provides a camera extrinsic parameter calibration method.

[0102] like Figure 6 As shown, the electronic device is in the form of a general computing device. The components of the electronic device may include but are not limited to: one or more processors 610, communication interface 620 and memory 630, and a communication bus 640 connecting different system components (including memory 630, communication interface 620 and processor 610).

[0103] The communication bus 640 represents one or more of several types of bus structures, including a memory bus or memory controller, a peripheral bus, a graphics acceleration port, a processor or a local bus using any of a variety of bus structures. For example, these architectures include but are not limited to Industry Standard Architecture (ISA) bus, Micro Channel Architecture (MAC) bus, Enhanced ISA bus, Video Electronics Standards Association (VESA) local bus and Peripheral Component Interconnection (PCI) bus.

[0104] Electronic devices typically include a variety of computer system readable media. These media can be any available media that can be accessed by the electronic device, including volatile and non-volatile media, removable and non-removable media.

[0105] The memory 630 may include a computer system readable medium in the form of a volatile memory, such as a random access memory (RAM) and / or a cache memory. The electronic device may further include other removable / non-removable, volatile / non-volatile computer system storage media. The memory 630 may include at least one program product having a set (e.g., at least one) of program modules configured to perform the functions of each embodiment of the present specification.

[0106] A program / utility having a set (at least one) of program modules may be stored in memory 630, such program modules including, but not limited to, an operating system, one or more application programs, other program modules, and program data, each of which or some combination may include an implementation of a network environment. The program modules generally perform the functions and / or methods of the embodiments described in this specification.

[0107] The processor 610 executes various functional applications and data processing by running the programs stored in the memory 630, such as implementing the Figure 1 to Figure 3 The illustrated embodiment provides a camera extrinsic parameter calibration method.

[0108] The embodiment of the present specification provides a computer-readable storage medium, wherein the computer-readable storage medium stores computer instructions, wherein the computer instructions enable the computer to execute the present specification. Figure 1 to Figure 3The illustrated embodiment provides a camera extrinsic parameter calibration method.

[0109] The above-mentioned computer-readable storage medium can adopt any combination of one or more computer-readable media. The computer-readable medium can be a computer-readable signal medium or a computer-readable storage medium. The computer-readable storage medium can be, for example, - but not limited to - an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or device, or any combination of the above. More specific examples (non-exhaustive list) of computer-readable storage media include: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ReadOnly Memory; hereinafter referred to as: ROM), an erasable programmable read-only memory (Erasable Programmable ReadOnly Memory; hereinafter referred to as: EPROM) or flash memory, optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In this document, a computer-readable storage medium can be any tangible medium containing or storing a program, which can be used by an instruction execution system, device or device or used in combination with it.

[0110] The above is a description of a specific embodiment of the specification. Other embodiments are within the scope of the appended claims. In some cases, the actions or steps recorded in the claims can be performed in an order different from that in the embodiments and still achieve the desired results. In addition, the processes depicted in the drawings do not necessarily require the specific order or continuous order shown to achieve the desired results. In some embodiments, multitasking and parallel processing are also possible or may be advantageous.

[0111] In the description of this specification, the description with reference to the terms "one embodiment", "some embodiments", "example", "specific example", or "some examples" etc. means that the specific features, structures, materials or characteristics described in conjunction with the embodiment or example are included in at least one embodiment or example of this specification. In this specification, the schematic representations of the above terms do not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials or characteristics described may be combined in any one or more embodiments or examples in a suitable manner. In addition, those skilled in the art may combine and combine the different embodiments or examples described in this specification and the features of the different embodiments or examples, unless they are contradictory.

[0112] In addition, the terms "first" and "second" are used for descriptive purposes only and should not be understood as indicating or implying relative importance or implicitly indicating the number of the indicated technical features. Therefore, the features defined as "first" and "second" may explicitly or implicitly include at least one of the features. In the description of this specification, the meaning of "plurality" is at least two, such as two, three, etc., unless otherwise clearly and specifically defined.

[0113] Any process or method description in a flowchart or otherwise described herein may be understood to represent a module, segment or portion of code that includes one or more executable instructions for implementing the steps of a custom logical function or process, and the scope of the preferred embodiments of this specification includes alternative implementations in which functions may not be performed in the order shown or discussed, including performing functions in a substantially simultaneous manner or in reverse order depending on the functions involved, which should be understood by technicians in the technical field to which the embodiments of this specification belong.

[0114] The word "if" as used herein may be interpreted as "at the time of" or "when" or "in response to determining" or "in response to detecting", depending on the context. Similarly, the phrases "if it is determined" or "if (stated condition or event) is detected" may be interpreted as "when it is determined" or "in response to determining" or "when detecting (stated condition or event)" or "in response to detecting (stated condition or event)", depending on the context.

[0115] It should be noted that the devices involved in the embodiments of this specification may include but are not limited to personal computers (Personal Computer; hereinafter referred to as: PC), personal digital assistants (Personal Digital Assistant; hereinafter referred to as: PDA), wireless handheld devices, tablet computers (Tablet Computer), mobile phones, MP3 displays, MP4 displays, etc.

[0116] In the several embodiments provided in this specification, it should be understood that the disclosed systems, devices and methods can be implemented in other ways. For example, the device embodiments described above are only schematic. For example, the division of the units is only a logical function division. There may be other division methods in actual implementation. For example, multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms.

[0117] In addition, each functional unit in each embodiment of this specification may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above integrated unit may be implemented in the form of hardware or in the form of hardware plus software functional units.

[0118] The above-mentioned integrated unit implemented in the form of a software functional unit can be stored in a computer-readable storage medium. The above-mentioned software functional unit is stored in a storage medium, including a number of instructions for a computer device (which can be a personal computer, a connector, or a network device, etc.) or a processor (Processor) to perform some steps of the method described in each embodiment of this specification. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (Read-Only Memory; hereinafter referred to as: ROM), random access memory (Random Access Memory; hereinafter referred to as: RAM), disk or optical disk and other media that can store program codes.

[0119] The above description is only a preferred embodiment of this specification and is not intended to limit this specification. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of this specification should be included in the scope of protection of this specification.

Claims

1. A camera extrinsic calibration method, characterized in that: The method comprises: Acquire image data of a target area collected by a camera; a plurality of local black and white grid patterns are arranged in the target area; Identify at least one of the local black-and-white checkered patterns from the image data; Determine the position of the corner point in each of the local black and white grid patterns; Determine the two-dimensional coordinates of the corner points in the camera coordinate system according to the positions of the corner points in each local black and white grid pattern; Obtain the three-dimensional coordinates corresponding to each corner point in the world coordinate system; The external parameter information of the camera is determined based on the two-dimensional coordinates and the three-dimensional coordinates of each of the corner points.

2. The method according to claim 1, characterized in that The step of determining the position of each corner point in the local black and white grid pattern comprises: Identify the border position of each of the local black and white grid patterns based on a first preset algorithm; Determine the area where the local black and white grid pattern is located according to the position of the frame; Based on a second preset algorithm, identifying a boundary between black and white grids in the area where the local black and white grid pattern is located; The positions of the corner points in the local black-and-white grid pattern are determined according to the boundaries between the black grid and the white grid.

3. The method according to claim 1, characterized in that Before obtaining the three-dimensional coordinates corresponding to each corner point in the world coordinate system, the method further includes: Scanning the target area to obtain scanning result information; Determine edge contour information of the plurality of local black and white grid patterns according to the scanning result information; The three-dimensional coordinates of the corresponding multiple corner points are obtained based on the edge contour information of each local black and white grid pattern.

4. The method according to claim 3, characterized in that Obtaining the three-dimensional coordinates corresponding to each corner point in the world coordinate system, including: A target three-dimensional coordinate matching the corner point is determined from the multiple three-dimensional coordinates to serve as the three-dimensional coordinate corresponding to the corner point.

5. The method according to claim 4, characterized in that The local black and white checkered pattern is also provided with identification information, and the target three-dimensional coordinates matching the corner point are determined from the plurality of three-dimensional coordinates, including: Marking the three-dimensional coordinates with a corresponding first label according to the identification information of the local black and white checkered pattern; Identifying identification information corresponding to each of the local black and white grid patterns from the image data; According to the identification information, the two-dimensional coordinates of the corner points identified in each local black and white grid pattern are marked with corresponding second labels; Determine a target first tag matching the second tag from the plurality of first tags; The three-dimensional coordinates corresponding to the target first label are determined as the target three-dimensional coordinates matching the corner point.

6. The method according to claim 1, characterized in that After determining the extrinsic parameter information of the camera based on the two-dimensional coordinates and the three-dimensional coordinates of each of the corner points, the method further includes: Determine the projection coordinates of each corner point after projecting it onto the camera coordinate system according to the external parameter information and the three-dimensional coordinates of each corner point; respectively determining the difference between each of the projection coordinates and the corresponding two-dimensional coordinates; determining an average of the differences; If the average value is greater than or equal to a preset threshold, determining the corner point corresponding to the maximum difference among the differences as an abnormal corner point; The external parameter information of the camera is recalculated according to the three-dimensional coordinates and the two-dimensional coordinates of the remaining corner points except the abnormal corner point, until the average value is less than the preset threshold or satisfies the preset condition.

7. The method according to claim 6, characterized in that Determining the projection coordinates of each corner point after being projected onto the camera coordinate system according to the external parameter information and the three-dimensional coordinates of each corner point includes: Converting the world coordinate system into a projection coordinate system corresponding to the camera coordinate system according to the external parameter information; The projection coordinates of the corner point in the projection coordinate system are determined according to the intrinsic parameter information of the camera and the three-dimensional coordinates of the corner point.

8. A camera extrinsic calibration device, characterized in that: include: An acquisition module, used to acquire image data about a target area collected by a camera; A plurality of local black and white checkered patterns are arranged in the target area; A recognition module, used to recognize at least one of the local black and white checkered patterns from the image data; A processing module, used to determine the position of the corner point in each of the local black and white grid patterns; The processing module is further used to determine the two-dimensional coordinates of the corner points in the camera coordinate system according to the positions of the corner points in each local black and white grid pattern; The acquisition module is further used to acquire the three-dimensional coordinates corresponding to each corner point in the world coordinate system; The processing module is further used to determine the external parameter information of the camera based on the two-dimensional coordinates and the three-dimensional coordinates of each of the corner points.

9. An electronic device, characterized in that: include: at least one processor; The processor is coupled to a memory, wherein: The memory stores program instructions executable by the processor, and the processor can execute the method according to any one of claims 1 to 7 by calling the program instructions.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores computer instructions, and the computer instructions enable the computer to execute the method according to any one of claims 1 to 7.

Images