Around view calibration method and system supporting multiple types of templates, medium and equipment

Abstract

The invention discloses a look-around calibration method and system supporting multiple types of templates, a medium and equipment. The look-around calibration method comprises the steps that original calibration images shot by all cameras are acquired; screening out a plurality of target calibration points from each original calibration image according to a preset calibration point distribution condition, and obtaining corresponding actual physical coordinates; mapping the actual physical coordinates of the target calibration points to a preset theoretical calibration cloth space according to a size proportion relation between the actual calibration cloth and the preset theoretical calibration cloth, and obtaining corresponding theoretical physical coordinates; according to the theoretical physical coordinates of the target calibration points and the pixel coordinates in the original calibration image, performing parameter calibration to obtain first internal and external parameters; adjusting the first internal and external parameters to an actual calibration distribution space according to the size proportion relation to obtain second internal and external parameters; and fusing the original calibration images according to the second internal and external parameters to obtain a panorama. According to the invention, the problem of poor reusability of a cross-scene adaptation algorithm during traditional look-around calibration is solved.

Description

Technical Field

[0001] The present invention belongs to the field of computer vision and relates to a surround view calibration method, system, medium and equipment supporting multiple types of templates. Background Art

[0002] Traditional methods for surround view system calibration rely on a calibration grid with a fixed geometric structure (such as a checkerboard or circular array) to extract feature points and calibrate parameters. The core process includes calibration point detection, calculation of internal and external parameters, and image projection mapping. This approach relies on the strong correlation between the geometric features of the calibration grid and the physical positions of the cameras, achieving multi-camera image alignment through homography.

[0003] Existing calibration algorithms require the design of unique feature point screening rules and parameter mapping logic for different calibration cloth types (sizes, patterns), resulting in the need to redevelop adaptive algorithm modules when changing calibration cloths. For example, checkerboard patterns rely on corner point detection, while circular arrays require a center location algorithm. The parameter mapping between the two requires hard-coded proportional relationships, making it impossible to adaptively switch between calibration cloth types and sizes through a unified framework. This results in algorithm redundancy and increased adaptation costs. Summary of the Invention

[0004] In response to the shortcomings of the existing technology, this application provides a surround view calibration method, system, medium and equipment that supports multiple types of templates, solving the problems of high cross-scene adaptation cost and poor algorithm reusability of traditional surround view calibration algorithms.

[0005] To achieve the above objectives, in a first aspect, the present invention provides a surround view calibration method supporting multiple types of templates, comprising:

[0006] Get the original calibration images taken by each camera;

[0007] According to the preset calibration point distribution conditions, a number of target calibration points are screened out in each of the original calibration images, and the corresponding actual physical coordinates are obtained;

[0008] According to the size ratio relationship between the actual calibration cloth and the preset theoretical calibration cloth, the actual physical coordinates of each target calibration point are mapped to the preset theoretical calibration cloth space to obtain the corresponding theoretical physical coordinates;

[0009] Performing parameter calibration according to the theoretical physical coordinates of each target calibration point and the pixel coordinates in the original calibration image to obtain first intrinsic and extrinsic parameters;

[0010] According to the size ratio relationship, the first internal and external parameters are adjusted to the actual calibration cloth space to obtain second internal and external parameters;

[0011] The original calibration images are fused according to the second internal and external parameters to obtain a panoramic image.

[0012] Compared with the existing technology, the embodiments of the present application have the following beneficial effects: by obtaining the original calibration images of each camera, a basis for data fusion is provided; based on the preset calibration point distribution conditions, the target calibration points are screened and their actual physical coordinates are obtained, and based on the preset conditions, the calibration points with reasonable distribution are dynamically selected, so that the algorithm no longer relies on the calibration cloth template of a specific shape or size, and can directly adapt to various calibration cloth types such as checkerboards and circular arrays; by mapping the actual physical coordinates to the theoretical calibration cloth space, the size difference of the actual calibration cloth is converted into a unified theoretical space ratio relationship, so that the algorithm only needs to adjust the parameters according to the size ratio between the actual calibration cloth and the theoretical calibration cloth, without the need to redevelop the algorithm due to changes in the size or type of the calibration cloth; based on the theoretical space coordinates and pixel coordinates, the internal and external parameters are calibrated, and finally restored to the actual space through the size ratio relationship, so that the calibration parameters are adaptive; finally, the panoramic image is generated by fusion to ensure the integrity and consistency of the image stitching. The entire solution not only ensures the calibration accuracy, but also realizes the versatility of the algorithm framework, significantly reducing the adaptation and development cost of different calibration cloths.

[0013] In some embodiments of the first aspect of the present application, screening out a plurality of target calibration points from each of the original calibration images according to a preset calibration point distribution condition includes:

[0014] Detecting all calibration points in each of the original calibration images to generate a set of candidate calibration points;

[0015] Selecting first candidate calibration points from the candidate calibration point set according to a preset spatial distribution uniformity criterion;

[0016] With the goal of maximizing geometric independence, each of the first candidate calibration points is iteratively screened according to a preset heuristic search algorithm until a preset optimization goal is met, and a number of target calibration points are screened out.

[0017] Compared with the prior art, the above embodiment has the following beneficial effects: generating a candidate set through full-image calibration point detection to avoid missing valid features; screening the first candidate calibration point through the spatial distribution uniformity criterion, forcing the calibration points to cover different areas of the image, and improving the spatial sampling balance; driving the heuristic search algorithm with the goal of maximizing geometric independence, iteratively optimizing the calibration point combination, and breaking the collinear / coplanar distribution restrictions; terminating the screening process through the preset optimization target to ensure the dispersion and independence of the calibration points in space, avoid calibration failure problems caused by collinearity or aggregation, improve the adaptability to irregular or non-standard calibration distribution, and provide a reliable data basis for high-precision calibration.

[0018] In some embodiments of the first aspect of the present application, fusing the original calibration images according to the second internal and external parameters to obtain a panoramic image includes:

[0019] Calculating the sub-area field of view corresponding to each camera's field of view based on the second internal and external parameters;

[0020] Calculate the theoretical projection position coordinates of each camera image based on the field of view of each sub-area and the size information of the actual calibration cloth;

[0021] Calculating the visual range of each camera according to the second internal and external parameters and the actual shooting range of each camera in the original calibration image;

[0022] According to the visual range and the theoretical projection position coordinates, each original calibration image is mapped to the corresponding sub-region field of view for fusion to generate a panoramic image.

[0023] Compared with the existing technology, the above embodiment has the following beneficial effects: the sub-area field of view is calculated by internal and external parameters to clarify the responsibility area of each camera; the theoretical projection position coordinates are calculated based on the actual calibration cloth size, and the physical space constraints are integrated into the projection model; the visible range is dynamically calculated based on the actual shooting range extracted from the original image to adapt to the camera installation position deviation and viewing angle limitation; the image is collaboratively mapped by theoretical projection coordinates and dynamic visible range to reduce stitching gaps or overlapping areas, eliminate blind spots and redundant coverage, and achieve efficient stitching and fusion of multi-field images.

[0024] In some embodiments of the first aspect of the present application, calculating the sub-region field of view corresponding to each camera field of view based on the second internal and external parameters includes:

[0025] Constructing a perspective projection model of each camera according to the second internal and external parameters;

[0026] According to each perspective projection model, the independent field of view of each camera and the overlapping field of view between the cameras are calculated, and the union of the field of view ranges of all cameras is calculated to obtain the global field of view;

[0027] According to the preset master-slave field of view allocation rule, and the independent fields of view and the overlapping fields of view, the global field of view is divided to obtain the sub-region fields of view corresponding to each camera.

[0028] Compared with the existing technology, the above embodiment has the following beneficial effects: quantifying the geometric coverage relationship of each camera's field of view through a perspective projection model; separating independent fields of view from overlapping fields of view, and clarifying the boundaries of the main camera's responsibility area and the collaborative area; generating a global field of view through a union operation to ensure the integrity of the scene without omissions; allocating overlapping areas through a master-slave rule, giving priority to ensuring the continuity of the main camera's field of view, reducing stitching gaps, and optimizing the efficiency of computing resource allocation.

[0029] In some embodiments of the first aspect of the present application, calculating the theoretical projection position coordinates of each camera image based on the field of view of each sub-region and the size information of the actual calibration cloth includes:

[0030] Calculating the original theoretical projection position coordinates of each camera image according to the field of view of each sub-region and the size information of the actual calibration cloth;

[0031] Calculate the theoretical coordinates of the same first auxiliary point in the corresponding camera images based on the actual physical coordinates of the same first auxiliary point in the original calibration images corresponding to the adjacent cameras and the corresponding second intrinsic and extrinsic parameters;

[0032] Calculating and generating a corresponding compensation factor based on each of the theoretical coordinates; wherein the compensation factor is used to minimize the Euclidean distance between the theoretical coordinates;

[0033] According to the compensation factor, the original theoretical projection position coordinates of each camera image are adjusted to obtain the theoretical projection position coordinates.

[0034] Compared with the existing technology, the above embodiment has the following beneficial effects: the original theoretical projection position coordinates are calculated by the actual calibration cloth size, and a reference mapping from physical space to image plane is established; the theoretical coordinate difference across cameras is calculated by the actual physical coordinates of the first auxiliary point, and the source of installation error is located; the compensation factor is generated with the goal of minimizing the Euclidean distance, and the vertical / horizontal alignment deviation is quantified; the original projection coordinates are adjusted by the compensation factor, the theoretical position offset between multiple cameras is eliminated, and the geometric consistency of stitching is improved.

[0035] In some embodiments of the first aspect of the present application, calculating the visible range of each camera based on the second intrinsic and extrinsic parameters and the actual shooting range of each camera in the original calibration image includes:

[0036] Calculating the initial visual range of each camera based on the second intrinsic and extrinsic parameters and the actual shooting range of each camera in the original calibration image;

[0037] Extracting a second auxiliary point from each of the original calibration images, and calculating a corresponding reprojection error according to pixel coordinates of the second auxiliary point and theoretical projection position coordinates within the initial visible range;

[0038] According to each of the reprojection errors, a corresponding incremental ratio is generated, and according to each of the incremental ratios and a preset boundary constraint, each of the initial visible ranges is adjusted to obtain a corresponding visible range.

[0039] Compared with the existing technology, the above embodiment has the following beneficial effects: the initial visible range is calculated by internal and external parameters and the actual shooting range to establish a physical coverage benchmark; the pixel coordinates of the second auxiliary point are extracted and compared with the theoretical projection position to quantify the impact of the calibration error on the visible range; the incremental ratio is generated based on the reprojection error to dynamically expand or shrink the boundary of the visible area; the adjustment range is limited by preset boundary constraints to prevent scene fragmentation caused by overfitting, thereby enhancing the fault tolerance and robustness of the system.

[0040] In a second aspect, the present invention further provides a surround view calibration system supporting multiple types of templates, comprising: a data acquisition module, a screening module, a mapping module, a parameter calibration module, a parameter adjustment module, and a fusion module;

[0041] The data acquisition module is used to acquire the original calibration images taken by each camera;

[0042] The screening module is used to screen out a number of target calibration points in each of the original calibration images according to a preset calibration point distribution condition, and obtain the corresponding actual physical coordinates;

[0043] The mapping module is used to map the actual physical coordinates of each target calibration point to a preset theoretical calibration cloth space according to the size ratio relationship between the actual calibration cloth and the preset theoretical calibration cloth, so as to obtain the corresponding theoretical physical coordinates;

[0044] The parameter calibration module is used to perform parameter calibration according to the theoretical physical coordinates of each target calibration point and the pixel coordinates in the original calibration image to obtain the first intrinsic and extrinsic parameters;

[0045] The parameter adjustment module is used to adjust the first internal and external parameters to the actual calibration cloth space according to the size ratio relationship to obtain second internal and external parameters;

[0046] The fusion module is used to fuse the original calibration images according to the second internal and external parameters to obtain a panoramic image.

[0047] Compared with the prior art, the above embodiments of the present application have the following beneficial effects: by obtaining the original calibration images of each camera, a basis for data fusion is provided; based on the preset calibration point distribution conditions, target calibration points are screened and their actual physical coordinates are obtained, and based on the preset conditions, calibration points with reasonable distribution are dynamically selected, so that the algorithm no longer relies on calibration cloth templates of specific shapes or sizes, and can directly adapt to various calibration cloth types such as checkerboards and circular arrays; by mapping the actual physical coordinates to the theoretical calibration cloth space, the size difference of the actual calibration cloth is converted into a unified theoretical space ratio relationship, so that the algorithm only needs to adjust the parameters according to the size ratio between the actual calibration cloth and the theoretical calibration cloth, without the need to redevelop the algorithm due to changes in the size or type of the calibration cloth; based on the theoretical space coordinates and pixel coordinates, internal and external parameters are calibrated, and finally restored to the actual space through the size ratio relationship, so that the calibration parameters are adaptive; finally, a panoramic image is generated by fusion to ensure the integrity and consistency of the image stitching. The entire solution not only ensures calibration accuracy, but also realizes the versatility of the algorithm framework, significantly reducing the adaptation and development cost of different calibration cloths.

[0048] In some embodiments of the second aspect of the present application, the screening module includes: a detection unit, a first screening unit, and a second screening unit;

[0049] The detection unit is configured to detect all calibration points in each of the original calibration images and generate a set of candidate calibration points;

[0050] The first screening unit is configured to screen out first candidate calibration points from the candidate calibration point set according to a preset spatial distribution uniformity criterion;

[0051] The second screening unit is configured to iteratively screen each of the first candidate calibration points according to a preset heuristic search algorithm with the goal of maximizing geometric independence until a preset optimization goal is met, thereby screening out a plurality of target calibration points.

[0052] Compared with the prior art, the above embodiment has the following beneficial effects: generating a candidate set through full-image calibration point detection to avoid missing valid features; screening the first candidate calibration point through the spatial distribution uniformity criterion, forcing the calibration points to cover different areas of the image, and improving the spatial sampling balance; driving the heuristic search algorithm with the goal of maximizing geometric independence, iteratively optimizing the calibration point combination, and breaking the collinear / coplanar distribution restrictions; terminating the screening process through the preset optimization target to ensure the dispersion and independence of the calibration points in space, avoid calibration failure problems caused by collinearity or aggregation, improve the adaptability to irregular or non-standard calibration distribution, and provide a reliable data basis for high-precision calibration.

[0053] In the third aspect, the present invention also provides a surround view calibration device that supports multiple types of templates, including a memory, a processor, and a computer program stored in the memory and runnable on the processor. When the computer program is loaded into the processor, the steps of the surround view calibration method that supports multiple types of templates are implemented.

[0054] In a fourth aspect, an embodiment of the present application further provides a computer-readable storage medium, wherein the computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the steps of the surround view calibration method supporting multiple types of templates are implemented. BRIEF DESCRIPTION OF THE DRAWINGS

[0055] Figure 1 : A flowchart of a surround view calibration method supporting multiple types of templates provided in some embodiments of the present invention.

[0056] Figure 2 : A structural diagram of a surround view calibration system supporting multiple types of templates provided in some embodiments of the present invention.

[0057] Figure 3 : A structural diagram of a surround view calibration device that supports multiple types of templates provided in some embodiments of the present invention. DETAILED DESCRIPTION

[0058] The following will clearly and completely describe the technical solutions in the embodiments of the present invention in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. All other embodiments obtained by ordinary technicians in this field based on the embodiments of the present invention without making any creative efforts shall fall within the scope of protection of the present invention.

[0059] Example 1:

[0060] Please refer to Figure 1 , a surround view calibration method supporting multiple types of templates provided by an embodiment of the present invention, including steps S1 to S6:

[0061] Step S1: Obtain the original calibration images taken by each camera.

[0062] In this embodiment, step S1 provides a basis for data fusion by acquiring the original calibration images of each camera.

[0063] Step S2: According to the preset calibration point distribution conditions, a number of target calibration points are screened out in each of the original calibration images, and the corresponding actual physical coordinates are obtained.

[0064] Furthermore, step S2 can be implemented by the following preferred implementation, including steps S21-S23, as follows:

[0065] S21: Detecting all calibration points in each of the original calibration images to generate a set of candidate calibration points;

[0066] S22: Filtering first candidate calibration points from the candidate calibration point set according to a preset spatial distribution uniformity criterion;

[0067] S23: with the goal of maximizing geometric independence, iteratively screening each of the first candidate calibration points according to a preset heuristic search algorithm until a preset optimization goal is met, and a plurality of target calibration points are screened.

[0068] For example, in a specific implementation, all available calibration points can first be detected in the captured image, either automatically or manually. This step typically uses computer vision techniques, such as corner detection algorithms (such as Harris corner detection and Shi-Tomasi corner detection) or template matching-based methods.

[0069] Then, the first candidate calibration points are selected from all the detected calibration points. For example, the geometric relationship between the calibration points can be used to ensure that the selected points are distributed in different areas of the image as much as possible to ensure that they can provide sufficient constraints for accurate calibration.

[0070] After selecting a series of first candidate calibration points, an optimization strategy can be applied to select the best combination of calibration points (for example, we can select 4 calibration points). If the number of calibration points is small, this step can be achieved through exhaustive search or by using a heuristic search algorithm.

[0071] For example, a genetic algorithm or a particle swarm optimization algorithm can be used to implement it: first, multiple groups of candidate calibration point combinations (each group contains 4 points) are randomly generated as the initial solution set; the objective function is set to minimize the collinearity or coplanarity error (such as calculating the degree of collinearity of three points or the coplanarity residual of four points) to maximize geometric independence; then the error of each group of candidate points is calculated and scored, and the combination with the smallest error is given priority; new combinations are iteratively generated through algorithmic mechanisms (such as crossover and mutation of genetic algorithms or position update of particle swarms), high error combinations are eliminated, and the optimal solution of geometric independence is gradually approached, and finally the target calibration points that meet the error threshold are output.

[0072] After selecting the target calibration points, you need to verify them. This can be done by initially estimating the internal and external parameters and checking the reprojection error. If the error is large, you may need to return to the previous step and reselect another set of calibration points.

[0073] In this preferred embodiment, steps S21-S23 generate a candidate set through full-image calibration point detection to avoid missing valid features; the first candidate calibration point is screened by the spatial distribution uniformity criterion, and the calibration points are forced to cover different areas of the image to improve the spatial sampling balance; the heuristic search algorithm is driven by the goal of maximizing geometric independence, and the calibration point combination is iteratively optimized to break the collinear / coplanar distribution restrictions; the screening process is terminated by pre-setting the optimization target to ensure the dispersion and independence of the calibration points in space, avoid calibration failure problems caused by collinearity or aggregation, and improve the adaptability to irregular or non-standard calibration cloths, so that the algorithm no longer relies on calibration cloth templates of specific shapes or sizes, and can directly adapt to various calibration cloth types such as checkerboards and circular arrays, or even if the number and layout of calibration points on the calibration cloth change, the algorithm can still work stably, providing a reliable data basis for high-precision calibration.

[0074] Step S3: mapping the actual physical coordinates of each target calibration point to the preset theoretical calibration cloth space according to the size ratio relationship between the actual calibration cloth and the preset theoretical calibration cloth, to obtain the corresponding theoretical physical coordinates.

[0075] In this embodiment, step S3 normalizes the calibration data by mapping the physical coordinates of the actual calibration cloth to the preset theoretical calibration cloth space based on the scale. This operation uniformly converts actual calibration cloths of different sizes or types into a standardized theoretical coordinate system, allowing subsequent parameter calibration (such as internal and external parameter calculations and projection model construction) to be performed entirely based on the unified scale of the theoretical space, eliminating the need to modify the algorithm logic due to differences in actual calibration cloth size.

[0076] Step S4: performing parameter calibration according to the theoretical physical coordinates of each target calibration point and the pixel coordinates in the original calibration image to obtain first intrinsic and extrinsic parameters.

[0077] In this embodiment, step S4 calibrates the internal and external parameters based on the theoretical space coordinates and pixel coordinates, decoupling the calculation of these parameters from the actual size of the calibration cloth. This operation unifies the calibration logic within the theoretical space framework, ensuring that parameter calibration relies solely on the standardized coordinates of the theoretical space rather than the actual physical size of the calibration cloth, thereby eliminating parameter adaptation issues caused by variations in calibration cloth size.

[0078] Step S5: According to the size ratio relationship, the first internal and external parameters are adjusted to the actual calibration space to obtain second internal and external parameters.

[0079] In this embodiment, step S5 dynamically adjusts the calibration parameters based on the size ratio by mapping parameters from the theoretical space to the actual calibration cloth space, achieving "real-size-insensitive adaptation" of the calibration algorithm. Specifically, the theoretical internal and external parameters (first internal and external parameters) are directly scaled to the space corresponding to the actual calibration cloth (second internal and external parameters) using a proportional factor. This allows the same theoretical calibration logic to automatically adapt to actual calibration cloths of any size, eliminating the need to re-derive the calibration model or modify the core algorithm module due to changes in the calibration cloth size.

[0080] For example, in the specific implementation, the standardized theoretical calibration cloth size (such as 5×5 unit length) is first set as a reference benchmark, and then the scale factor (2 in this example) is calculated based on the proportional relationship between the actual calibration cloth size (such as 10×10 unit length) and the theoretical size, and the physical coordinates of the feature points on the actual calibration cloth are mapped to the theoretical space according to this ratio; then, after the preliminary calibration of the internal and external parameters is completed in the theoretical space, the theoretical parameters (i.e., the first internal and external parameters) are reversely scaled to the physical space corresponding to the actual calibration cloth through the scale factor to ensure that the calibration results are consistent with the scale of the actual scene; finally, based on the adjusted actual parameters (i.e., the second internal and external parameters), the precise fusion of multi-camera images is achieved to form a panoramic image.

[0081] Here, the theoretical calibration cloth size establishes a universal reference system, freeing the algorithm from dependence on specific physical dimensions. Its core value lies in building a flexible and robust calibration system. This design first implements a standardized processing flow, allowing calibration cloths of different sizes to be processed using a unified algorithm logic, significantly improving code reuse and maintenance efficiency. At the same time, it dynamically adapts to multi-size scenarios through scale factors, eliminating the need to write independent algorithms or adjust code logic for different sizes. In engineering practice, this mechanism also enhances the system's fault tolerance. Even when actual size data is missing or erroneous, an approximate calibration can still be completed based on the theoretical size and scale relationship. This layered design is particularly suitable for applications that require compatibility with calibration cloths of multiple specifications and have high system reliability requirements.

[0082] Step S6: fusing the original calibration images according to the second internal and external parameters to obtain a panoramic image.

[0083] Furthermore, step S6 can be implemented by the following preferred implementation, including steps S61-S64, as follows:

[0084] S61: Calculating the sub-area field of view corresponding to each camera's field of view according to the second internal and external parameters.

[0085] Furthermore, step S61 can be implemented by the following preferred implementation, including steps S611-S613, as follows:

[0086] S611: Constructing a perspective projection model of each camera according to the second internal and external parameters;

[0087] S612: Calculate the independent field of view of each camera and the overlapping field of view between the cameras according to each perspective projection model, and calculate the union of the field of view ranges of all cameras to obtain a global field of view;

[0088] S613: Divide the global field of view according to a preset master-slave field of view allocation rule and the independent fields of view and the overlapping fields of view to obtain sub-region fields of view corresponding to each camera.

[0089] In the specific implementation, the perspective projection model first constructed describes the mapping relationship from 3D world coordinates to 2D image coordinates. Based on this model, the field of view of all cameras can be identified and calculated. The perspective projection transformation algorithm when constructing the model is: ′ =K[R|t]X; where X ′ represents a two-dimensional point on the image plane, [R|t] is the extrinsic parameter matrix, which consists of the rotation matrix R and the translation vector t, describing the transformation from the world coordinate system to the camera coordinate system; K is the camera intrinsic parameter matrix, which contains the internal parameters of the camera, such as focal length, principal point coordinates, etc.; X is a three-dimensional point in the world coordinate system.

[0090] When dividing the global field of view, a rectangular grid or a content-based adaptive segmentation method can be used to obtain the responsibility area (i.e., sub-area field of view) corresponding to each camera, ensuring that only one camera is the main contributor in each sub-area field of view, and the information provided by other cameras is used as auxiliary reference.

[0091] In this preferred embodiment, steps S611-S613 quantify the geometric coverage relationship of each camera's field of view through a perspective projection model; separate independent fields of view from overlapping fields of view, and clarify the boundaries of the main camera's responsibility area and the collaborative area; generate a global field of view through a union operation to ensure the integrity of the scene without omissions; allocate overlapping areas through a master-slave rule, give priority to ensuring the continuity of the main camera's field of view, reduce stitching gaps, and optimize the efficiency of computing resource allocation.

[0092] S62: Calculate the theoretical projection position coordinates of each camera image according to the field of view of each sub-region and the size information of the actual calibration cloth.

[0093] Furthermore, step S62 can be implemented by the following preferred implementation, including steps S621-S624, as follows:

[0094] S621: Calculating the original theoretical projection position coordinates of each camera image according to the field of view of each sub-region and the size information of the actual calibration cloth;

[0095] S622: Calculating theoretical coordinates of the same first auxiliary point in the corresponding camera images based on the actual physical coordinates of the same first auxiliary point in the original calibration images corresponding to the adjacent cameras and the corresponding second intrinsic and extrinsic parameters;

[0096] S623: Calculate and generate corresponding compensation factors according to the theoretical coordinates; wherein the compensation factors are used to minimize the Euclidean distance between the theoretical coordinates;

[0097] S624: Adjusting the original theoretical projection position coordinates of each camera image according to the compensation factor to obtain each theoretical projection position coordinate.

[0098] In specific implementation, the original theoretical projection position coordinates of each camera image are first calculated using the sub-region field of view and the size information of the actual calibration cloth. This original theoretical projection position coordinate represents the position where each camera image should be placed under ideal conditions (i.e., assuming an ideal state without installation errors and environmental interference). However, considering the differences between the actual situation and the ideal state, such as the angle deviation of the camera installation and lens distortion, a compensation factor is introduced to fine-tune the original theoretical projection position coordinates of each camera image. This compensation factor can be obtained by comparing known features (such as fixed points on the calibration cloth) in the same scene captured by adjacent cameras, as shown in steps S622 and S623, for example:

[0099] When two adjacent cameras A and B capture the same calibration point P (i.e., the first auxiliary point mentioned above), first obtain the external parameter matrix of cameras A and B from the second internal and external parameters [R A ∣t A ] and [R B ∣t B ], and the internal parameter matrix K A and K B Then, using the internal and external parameters of cameras A and B and the above perspective projection transformation algorithm, point P in the world coordinate system is transformed and projected onto the plane of each 2D camera image to obtain the theoretical coordinates x of point P in the camera image of camera A. A and the theoretical coordinates x in the camera image of camera B B In an ideal situation, if there is no error, x A and x B should be on the same line (i.e. their y coordinates are the same). Therefore, the compensation factor can be calculated by minimizing the x A and x B The Euclidean distance between them is used to determine the distance, especially along the vertical direction (y-axis).

[0100] When calculating the compensation factor to adjust the original theoretical projection position coordinates, this process can be implemented using functions provided by computer vision libraries (such as OpenCV). For example, the cv::stereoRectify function can be used to calculate the rotation matrix for correction, and the cv::initUndistortRectifyMap function can be used to generate the mapping, and finally cv::remap can be used to apply these mappings to complete the adjustment. In addition, for the adjustment of each camera image, a nonlinear least squares method such as Levenberg-Marquardt or other gradient descent methods can be used for iterative optimization to minimize the discontinuity of the boundaries between adjacent images.

[0101] In this preferred embodiment, steps S621-S624 calculate the original theoretical projection position coordinates through the actual calibration cloth size to establish a reference mapping from physical space to image plane; calculate the theoretical coordinate difference across cameras through the actual physical coordinates of the first auxiliary point to locate the source of installation error; generate a compensation factor with the goal of minimizing the Euclidean distance to quantify the vertical / horizontal alignment deviation; adjust the original projection coordinates through the compensation factor to eliminate the theoretical position offset between multiple cameras and improve the geometric consistency of stitching.

[0102] S63: Calculate the visual range of each camera according to the second internal and external parameters and the actual shooting range of each camera in the original calibration image.

[0103] Furthermore, step S63 can be implemented by the following preferred implementation, including steps S631-S633, as follows:

[0104] S631: Calculating the initial visual range of each camera according to the second internal and external parameters and the actual shooting range of each camera in the original calibration image;

[0105] S632: Extracting a second auxiliary point from each of the original calibration images, and calculating a corresponding reprojection error according to the pixel coordinates of the second auxiliary point and the theoretical projection position coordinates within the initial visible range;

[0106] S633: Generate corresponding incremental ratios according to the reprojection errors, and adjust the initial visible ranges according to the incremental ratios and preset boundary constraints to obtain corresponding visible ranges.

[0107] When calibrating the surround view system, due to the differences in the installation angles, field of view coverage, or placement of the calibration cloth of different cameras, there may be deviations between the actual positions of the calibration points captured by each camera in the image and the expected positions. This deviation may cause excessive overlap, large gaps, or other discontinuities in the stitched image, affecting the quality of the final panoramic image. In this solution, the visual range can be adjusted by incremental ratio for optimization.

[0108] In specific implementation, first, as in step S631, the initial visual range of each camera is calculated, and then the feature point detection algorithm (such as SIFT, SURF, etc.) is used to extract feature points (i.e., second auxiliary points) from each original calibration image, and a matching relationship is established between different images of the same scene. Then, the feature points in the world coordinate system are projected onto the image plane using the second internal and external parameters and the perspective projection transformation algorithm to obtain the corresponding theoretical projection position coordinates, and the Euclidean distance between these theoretical projection position coordinates and the corresponding actual detected pixel coordinates is calculated, i.e., the reprojection error. Then, the reprojection errors of all second auxiliary points are statistically analyzed, such as calculating the mean and standard deviation. If the error exceeds the preset threshold, it is considered that there is a significant position difference and the initial visual range needs to be adjusted.

[0109] When calculating the incremental ratio and adjusting the initial visible range, the following method can be used: first, based on the empirical value or experimental data, set an initial incremental ratio α (for example, 5%) of the visible range. This initial incremental ratio is used to expand or shrink the initial visible range to adapt to different application scenarios. Then, according to the size and direction of the position difference obtained by the above statistical analysis of the reprojection error, the initial incremental ratio is dynamically adjusted. For example, if the difference in a certain direction is large, the increment in that direction can be increased accordingly; otherwise, the increment can be reduced. The purpose is to make the transition between adjacent camera images as smooth as possible. The formula is as follows: Δα=f(e); where Δα represents the adjustment amount of the initial incremental ratio, and f(e) is a function of the position difference e. Finally, the adjusted incremental ratio is applied to the initial visible range to obtain the adjusted visible range. The adjustment formula is as follows: AVR=IVR×(1+α A ); AVR represents the adjusted visual range, IVR represents the initial visual range, α A represents the incremental ratio after adjustment. Furthermore, during adjustment, boundary constraints are required to ensure that the adjusted visual range does not exceed the physical limitations of the system (such as the maximum viewing angle of the camera) and that the overlap between cameras remains reasonable. If a single adjustment cannot completely eliminate the discrepancy, an iterative optimization approach can be used until the predetermined accuracy requirements are met.

[0110] In this preferred embodiment, steps S631-S633 calculate the initial visible range through internal and external parameters and the actual shooting range to establish a physical coverage benchmark; extract the pixel coordinates of the second auxiliary point and compare them with the theoretical projection position to quantify the impact of the calibration error on the visible range; generate an incremental ratio based on the reprojection error to dynamically expand or shrink the boundary of the visible area; limit the adjustment range through preset boundary constraints to prevent scene fragmentation caused by overfitting, thereby enhancing the system's fault tolerance and robustness.

[0111] S64: According to the visual ranges and theoretical projection position coordinates, each original calibration image is mapped to a corresponding sub-region field of view for fusion to generate a panoramic image.

[0112] In specific implementation, weighted averaging, feathering processing and other operations can be used to fuse images to improve the quality of fusion.

[0113] In this preferred embodiment, steps S61-S64 calculate the sub-area field of view through internal and external parameters to clarify the responsibility area of each camera; calculate the theoretical projection position coordinates based on the actual calibration cloth size, and integrate the physical space constraints into the projection model; dynamically calculate the visible range based on the actual shooting range extracted from the original image, and adapt to the camera installation position deviation and viewing angle limitation; collaboratively map the image through theoretical projection coordinates and dynamic visible range to reduce stitching gaps or overlapping areas, eliminate blind spots and redundant coverage, and achieve efficient stitching and fusion of multi-field images.

[0114] In addition, when implementing a surround view calibration method that supports multiple types of templates in this solution, all parameters in the algorithm can be saved in the form of a configuration file. When adapting to different calibration layouts, only the corresponding parameters need to be modified without modifying the algorithm itself.

[0115] In summary, compared with the prior art, the above embodiments of the present application have the following beneficial effects: by obtaining the original calibration images of each camera, a basis for data fusion is provided; based on the preset calibration point distribution conditions, the target calibration points are screened and their actual physical coordinates are obtained, and based on the preset conditions, the calibration points with reasonable distribution are dynamically selected, so that the algorithm no longer relies on the calibration cloth template of a specific shape or size, and can directly adapt to various calibration cloth types such as checkerboard and circular array; by mapping the actual physical coordinates to the theoretical calibration cloth space, the size difference of the actual calibration cloth is converted into a unified theoretical space ratio relationship, so that the algorithm only needs to adjust the parameters according to the size ratio between the actual calibration cloth and the theoretical calibration cloth, without the need to redevelop the algorithm due to changes in the size or type of the calibration cloth; based on the theoretical space coordinates and pixel coordinates, the internal and external parameters are calibrated, and finally restored to the actual space through the size ratio relationship, so as to achieve the adaptation of the calibration parameters; finally, the panoramic image is generated by fusion to ensure the integrity and consistency of the image stitching. The entire solution not only ensures the calibration accuracy, but also realizes the versatility of the algorithm framework, significantly reducing the adaptation and development cost of different calibration cloths.

[0116] Example 2:

[0117] Please refer to Figure 2 Based on the same inventive concept, an embodiment of the present invention discloses a surround view calibration system supporting multiple types of templates, including: a data acquisition module M1, a screening module M2, a mapping module M3, a parameter calibration module M4, a parameter adjustment module M5 and a fusion module M6;

[0118] The data acquisition module M1 is used to acquire the original calibration images taken by each camera.

[0119] In this embodiment, the data acquisition module M1 provides a basis for data fusion by acquiring the original calibration images of each camera.

[0120] The screening module M2 is used to screen out a number of target calibration points in each of the original calibration images according to a preset calibration point distribution condition, and obtain the corresponding actual physical coordinates.

[0121] Furthermore, the screening module M2 includes: a detection unit, a first screening unit and a second screening unit;

[0122] The detection unit is configured to detect all calibration points in each of the original calibration images and generate a set of candidate calibration points;

[0123] The first screening unit is configured to screen out first candidate calibration points from the candidate calibration point set according to a preset spatial distribution uniformity criterion;

[0124] The second screening unit is configured to iteratively screen each of the first candidate calibration points according to a preset heuristic search algorithm with the goal of maximizing geometric independence until a preset optimization goal is met, thereby screening out a plurality of target calibration points.

[0125] In this preferred embodiment, the screening module M2 generates a candidate set through full-image calibration point detection to avoid missing valid features; screens the first candidate calibration point through the spatial distribution uniformity criterion, forces the calibration points to cover different areas of the image, and improves the spatial sampling balance; drives the heuristic search algorithm with the goal of maximizing geometric independence, iteratively optimizes the calibration point combination, and breaks the collinear / coplanar distribution restrictions; terminates the screening process through the preset optimization target to ensure the dispersion and independence of the calibration points in space, avoid calibration failure problems caused by collinearity or aggregation, and improve the adaptability to irregular or non-standard calibration cloths, so that the algorithm no longer relies on calibration cloth templates of specific shapes or sizes, and can directly adapt to various calibration cloth types such as checkerboards and circular arrays, providing a reliable data basis for high-precision calibration.

[0126] The mapping module M3 is used to map the actual physical coordinates of each target calibration point to the preset theoretical calibration cloth space according to the size ratio relationship between the actual calibration cloth and the preset theoretical calibration cloth, so as to obtain the corresponding theoretical physical coordinates.

[0127] In this embodiment, mapping module M3 normalizes the calibration data by mapping the physical coordinates of the actual calibration cloth to a pre-set theoretical calibration cloth space based on its size ratio. This operation uniformly converts actual calibration cloths of varying sizes or types into a standardized theoretical coordinate system, allowing subsequent parameter calibration (such as internal and external parameter calculations and projection model construction) to be performed entirely within the unified scale of the theoretical space, eliminating the need to modify algorithm logic due to differences in actual calibration cloth size.

[0128] The parameter calibration module M4 is configured to perform parameter calibration according to the theoretical physical coordinates of each target calibration point and the pixel coordinates in the original calibration image to obtain first intrinsic and extrinsic parameters.

[0129] In this embodiment, parameter calibration module M4 calibrates internal and external parameters based on theoretical spatial coordinates and pixel coordinates, decoupling the calculation of these parameters from the actual dimensions of the calibration cloth. This operation unifies the calibration logic within the theoretical spatial framework, ensuring that parameter calibration relies solely on the standardized coordinates of the theoretical space rather than the physical dimensions of the actual calibration cloth, thereby eliminating parameter adaptation issues caused by variations in calibration cloth dimensions.

[0130] The parameter adjustment module M5 is used to adjust the first internal and external parameters to the actual calibration space according to the size ratio relationship to obtain second internal and external parameters.

[0131] In this embodiment, the parameter adjustment module M5 dynamically adjusts the calibration parameters based on the size ratio by mapping parameters from the theoretical space to the actual calibration cloth space, achieving "real-size-insensitive adaptation" of the calibration algorithm. Specifically, the theoretical internal and external parameters (the first internal and external parameters) are directly scaled to the space corresponding to the actual calibration cloth (the second internal and external parameters) using a proportional factor. This allows the same theoretical calibration logic to automatically adapt to actual calibration cloths of any size, eliminating the need to re-derive the calibration model or modify the core algorithm module due to changes in the calibration cloth size.

[0132] The fusion module M6 is configured to fuse the original calibration images according to the second internal and external parameters to obtain a panoramic image.

[0133] Furthermore, the fusion module M6 includes: a sub-area field of view calculation unit, a theoretical coordinate calculation unit, a visual range calculation unit and an image fusion unit;

[0134] The sub-area field of view calculation unit is configured to calculate the sub-area field of view corresponding to the field of view of each camera according to the second internal and external parameters.

[0135] The theoretical coordinate calculation unit is used to calculate the theoretical projection position coordinates of each camera image according to the field of view of each sub-area and the size information of the actual calibration cloth.

[0136] The visual range calculation unit is used to calculate the visual range of each camera according to the second internal and external parameters and the actual shooting range of each camera in the original calibration image.

[0137] The image fusion unit is used to map each original calibration image to the corresponding sub-region field of view according to each visual range and theoretical projection position coordinates, and fuse them to generate a panoramic image.

[0138] In this preferred embodiment, the fusion module M6 calculates the sub-area field of view through internal and external parameters to clarify the responsibility area of each camera; calculates the theoretical projection position coordinates based on the actual calibration cloth size, and integrates the physical space constraints into the projection model; dynamically calculates the visible range based on the actual shooting range extracted from the original image, and adapts to the camera installation position deviation and viewing angle limitation; collaboratively maps images through theoretical projection coordinates and dynamic visible range to reduce stitching gaps or overlapping areas, eliminate blind spots and redundant coverage, and achieve efficient stitching and fusion of multi-field images.

[0139] Furthermore, the sub-region field of view calculation unit includes: a projection model construction sub-unit, a field of view calculation sub-unit and a field of view division sub-unit;

[0140] The projection model construction subunit is configured to construct a perspective projection model of each camera according to the second internal and external parameters;

[0141] The field of view calculation subunit is used to calculate the independent field of view of each camera, the overlapping field of view between the cameras, and calculate the union of the field of view ranges of all cameras to obtain the global field of view according to each perspective projection model;

[0142] The field of view division subunit is used to divide the global field of view according to the preset master-slave field of view allocation rule and the independent fields of view and the overlapping fields of view to obtain the sub-region fields of view corresponding to each camera.

[0143] In this preferred embodiment, the sub-area field of view calculation unit quantifies the geometric coverage relationship of each camera's field of view through a perspective projection model; separates independent fields of view from overlapping fields of view, and clarifies the boundaries of the main camera's responsibility area and the collaborative area; generates a global field of view through a union operation to ensure the integrity of the scene without omissions; allocates overlapping areas through a master-slave rule, giving priority to ensuring the continuity of the main camera's field of view, reducing stitching gaps, and optimizing the efficiency of computing resource allocation.

[0144] Furthermore, the theoretical coordinate calculation unit includes: an original theoretical coordinate calculation subunit, an auxiliary point coordinate calculation subunit, a compensation factor calculation subunit and an original coordinate adjustment subunit;

[0145] The original theoretical coordinate calculation subunit is used to calculate the original theoretical projection position coordinates of each camera image according to the field of view of each sub-area and the size information of the actual calibration cloth;

[0146] The auxiliary point coordinate calculation subunit is used to calculate the theoretical coordinates of the same first auxiliary point in the corresponding camera images based on the actual physical coordinates of the same first auxiliary point in the original calibration images corresponding to the adjacent cameras and the corresponding second intrinsic and extrinsic parameters;

[0147] The compensation factor calculation subunit is used to calculate and generate a corresponding compensation factor based on each of the theoretical coordinates; wherein the compensation factor is used to minimize the Euclidean distance between the theoretical coordinates;

[0148] The original coordinate adjustment subunit is used to adjust the original theoretical projection position coordinates of each camera image according to the compensation factor to obtain each theoretical projection position coordinate.

[0149] In this preferred embodiment, the theoretical coordinate calculation unit calculates the original theoretical projection position coordinates through the actual calibration cloth size, and establishes a reference mapping from physical space to image plane; calculates the theoretical coordinate difference across cameras through the actual physical coordinates of the first auxiliary point, and locates the source of installation error; generates a compensation factor with the goal of minimizing the Euclidean distance, and quantifies the vertical / horizontal alignment deviation; adjusts the original projection coordinates through the compensation factor, eliminates the theoretical position offset between multiple cameras, and improves the geometric consistency of stitching.

[0150] Furthermore, the visible range calculation unit includes: an initial visible range calculation subunit, a reprojection error calculation subunit and a visible range adjustment subunit;

[0151] The initial visual range calculation subunit is configured to calculate the initial visual range of each camera according to the second internal and external parameters and the actual shooting range of each camera in the original calibration image;

[0152] The reprojection error calculation subunit is used to extract the second auxiliary point in each of the original calibration images, and calculate the corresponding reprojection error according to the pixel coordinates of the second auxiliary point and the theoretical projection position coordinates within the initial visible range;

[0153] The visible range adjustment subunit is configured to generate a corresponding incremental ratio according to each reprojection error, and adjust each initial visible range according to each incremental ratio and a preset boundary constraint to obtain a corresponding visible range.

[0154] In this preferred embodiment, the visible range calculation unit calculates the initial visible range through internal and external parameters and the actual shooting range to establish a physical coverage benchmark; extracts the pixel coordinates of the second auxiliary point and compares them with the theoretical projection position to quantify the impact of the calibration error on the visible range; generates an incremental ratio based on the reprojection error to dynamically expand or shrink the visible area boundary; limits the adjustment range through preset boundary constraints to prevent scene fragmentation caused by overfitting, thereby enhancing the system's fault tolerance and robustness.

[0155] In summary, compared with the existing technology, the embodiments of the present application have the following beneficial effects: by obtaining the original calibration images of each camera, a basis for data fusion is provided; based on the preset calibration point distribution conditions, the target calibration points are screened and their actual physical coordinates are obtained, and based on the preset conditions, the calibration points with reasonable distribution are dynamically selected, so that the algorithm no longer relies on the calibration cloth template of a specific shape or size, and can directly adapt to various calibration cloth types such as checkerboard and circular array; by mapping the actual physical coordinates to the theoretical calibration cloth space, the size difference of the actual calibration cloth is converted into a unified theoretical space ratio relationship, so that the algorithm only needs to adjust the parameters according to the size ratio between the actual calibration cloth and the theoretical calibration cloth, without the need to redevelop the algorithm due to changes in the size or type of the calibration cloth; based on the theoretical space coordinates and pixel coordinates, the internal and external parameters are calibrated, and finally restored to the actual space through the size ratio relationship to achieve self-adaptation of the calibration parameters; finally, the panoramic image is generated by fusion to ensure the integrity and consistency of the image stitching. The entire solution not only ensures the calibration accuracy, but also realizes the versatility of the algorithm framework, significantly reducing the adaptation and development cost of different calibration cloths.

[0156] Example 3:

[0157] Figure 3 The structure diagram of a surround view calibration device supporting multiple types of templates is presented. Figure 3 As shown, the surround view calibration device supporting multiple types of templates may include: a processor N1, a memory N2, a data interface N3 and a communication bus N4.

[0158] Among them: the processor N1, the memory N2, and the data interface N3 communicate with each other through the communication bus N4; the data interface N3 is used for data communication with other devices such as input devices or output devices; the processor N1 is used to execute the program N5, which can specifically execute the relevant steps in the above-mentioned embodiment of the surround view calibration method supporting multiple types of templates.

[0159] Specifically, the program N5 may include program code, which includes computer-executable instructions.

[0160] The processor N1 may be a central processing unit (CPU), an application-specific integrated circuit (ASIC), or one or more integrated circuits configured to implement the embodiments of the present application. The one or more processors included in the surround view calibration device supporting multiple types of templates may be processors of the same type, such as one or more CPUs, or may be processors of different types, such as one or more CPUs and one or more ASICs.

[0161] The memory N2 is used to store the program N5. The memory N2 may include a high-speed RAM memory, and may also include a non-volatile memory (non-volatile memory), such as at least one disk memory.

[0162] The algorithms or displays provided herein are not inherently related to any particular computer, virtual system, or other device. In addition, the embodiments of the present application are not directed to any particular programming language.

[0163] Example 4:

[0164] An embodiment of the present invention also provides a computer-readable storage medium, which stores at least one executable instruction. When the executable instruction is run on a surround view calibration device / system that supports multiple types of templates, the surround view calibration device / system that supports multiple types of templates executes a surround view calibration method that supports multiple types of templates in any of the above method embodiments.

[0165] In the description provided herein, a large number of specific details are described. However, it is understood that the embodiments of the present application can be practiced without these specific details. Similarly, in order to streamline the application and help understand one or more of the various inventive aspects, in the above description of the exemplary embodiments of the application, the various features of the embodiments of the application are sometimes grouped together into a single embodiment, figure, or description thereof. Wherein, the claims that follow the specific embodiment are hereby clearly incorporated into the specific embodiment, wherein each claim itself serves as a separate embodiment of the application.

[0166] Those skilled in the art will appreciate that the modules in the devices of the embodiments can be adaptively changed and installed in one or more devices different from the embodiments. The modules, units, or components in the embodiments can be combined into one module, unit, or component, and furthermore, they can be divided into multiple submodules, subunits, or subcomponents. Except that at least some of such features and / or processes or units are mutually exclusive.

Claims

1. A surround view calibration method supporting multiple template types, characterized in that: include: Get the original calibration images taken by each camera; According to the preset calibration point distribution conditions, a number of target calibration points are screened out in each of the original calibration images, and the corresponding actual physical coordinates are obtained; According to the size ratio relationship between the actual calibration cloth and the preset theoretical calibration cloth, the actual physical coordinates of each target calibration point are mapped to the preset theoretical calibration cloth space to obtain the corresponding theoretical physical coordinates; Performing parameter calibration according to the theoretical physical coordinates of each target calibration point and the pixel coordinates in the original calibration image to obtain first intrinsic and extrinsic parameters; According to the size ratio relationship, the first internal and external parameters are adjusted to the actual calibration cloth space to obtain second internal and external parameters; The original calibration images are fused according to the second internal and external parameters to obtain a panoramic image.

2. The surround view calibration method supporting multiple template types according to claim 1, wherein: The method of selecting a plurality of target calibration points from each of the original calibration images according to the preset calibration point distribution conditions includes: Detecting all calibration points in each of the original calibration images to generate a set of candidate calibration points; Selecting first candidate calibration points from the candidate calibration point set according to a preset spatial distribution uniformity criterion; With the goal of maximizing geometric independence, each of the first candidate calibration points is iteratively screened according to a preset heuristic search algorithm until a preset optimization goal is met, and a number of target calibration points are screened out.

3. The surround view calibration method supporting multiple template types according to claim 1, wherein: The step of fusing the original calibration images according to the second internal and external parameters to obtain a panoramic image includes: Calculating the sub-area field of view corresponding to each camera's field of view based on the second internal and external parameters; Calculate the theoretical projection position coordinates of each camera image based on the field of view of each sub-area and the size information of the actual calibration cloth; Calculating the visual range of each camera according to the second internal and external parameters and the actual shooting range of each camera in the original calibration image; According to the visual range and the theoretical projection position coordinates, each original calibration image is mapped to the corresponding sub-region field of view for fusion to generate a panoramic image.

4. The surround view calibration method supporting multiple template types according to claim 3, wherein: The calculating, based on the second internal and external parameters, the sub-area field of view corresponding to the field of view of each camera includes: Constructing a perspective projection model of each camera according to the second internal and external parameters; According to each perspective projection model, the independent field of view of each camera and the overlapping field of view between the cameras are calculated, and the union of the field of view ranges of all cameras is calculated to obtain the global field of view; According to the preset master-slave field of view allocation rule, and the independent fields of view and the overlapping fields of view, the global field of view is divided to obtain the sub-region fields of view corresponding to each camera.

5. The surround view calibration method supporting multiple template types according to claim 3, wherein: The method of calculating the theoretical projection position coordinates of each camera image according to the field of view of each sub-region and the size information of the actual calibration cloth includes: Calculating the original theoretical projection position coordinates of each camera image according to the field of view of each sub-area and the size information of the actual calibration cloth; Calculate the theoretical coordinates of the same first auxiliary point in the corresponding camera images based on the actual physical coordinates of the same first auxiliary point in the original calibration images corresponding to the adjacent cameras and the corresponding second intrinsic and extrinsic parameters; Calculating and generating a corresponding compensation factor based on each of the theoretical coordinates; wherein the compensation factor is used to minimize the Euclidean distance between the theoretical coordinates; According to the compensation factor, the original theoretical projection position coordinates of each camera image are adjusted to obtain the theoretical projection position coordinates.

6. The surround view calibration method supporting multiple template types according to claim 3, wherein: The calculating the visual range of each camera according to the second internal and external parameters and the actual shooting range of each camera in the original calibration image includes: Calculating the initial visual range of each camera based on the second intrinsic and extrinsic parameters and the actual shooting range of each camera in the original calibration image; Extracting a second auxiliary point from each of the original calibration images, and calculating a corresponding reprojection error according to pixel coordinates of the second auxiliary point and theoretical projection position coordinates within the initial visible range; According to each of the reprojection errors, a corresponding incremental ratio is generated, and according to each of the incremental ratios and a preset boundary constraint, each of the initial visible ranges is adjusted to obtain a corresponding visible range.

7. A surround view calibration system supporting multiple types of templates, characterized in that: include: Data acquisition module, screening module, mapping module, parameter calibration module, parameter adjustment module and fusion module; The data acquisition module is used to acquire the original calibration images taken by each camera; The screening module is used to screen out a number of target calibration points in each of the original calibration images according to a preset calibration point distribution condition, and obtain the corresponding actual physical coordinates; The mapping module is used to map the actual physical coordinates of each target calibration point to a preset theoretical calibration cloth space according to the size ratio relationship between the actual calibration cloth and the preset theoretical calibration cloth, so as to obtain the corresponding theoretical physical coordinates; The parameter calibration module is used to perform parameter calibration according to the theoretical physical coordinates of each target calibration point and the pixel coordinates in the original calibration image to obtain the first intrinsic and extrinsic parameters; The parameter adjustment module is used to adjust the first internal and external parameters to the actual calibration cloth space according to the size ratio relationship to obtain second internal and external parameters; The fusion module is used to fuse the original calibration images according to the second internal and external parameters to obtain a panoramic image.

8. The surround view calibration system supporting multiple template types according to claim 7, characterized in that: The screening module includes: a detection unit, a first screening unit and a second screening unit; The detection unit is configured to detect all calibration points in each of the original calibration images and generate a set of candidate calibration points; The first screening unit is configured to screen out first candidate calibration points from the candidate calibration point set according to a preset spatial distribution uniformity criterion; The second screening unit is configured to iteratively screen each of the first candidate calibration points according to a preset heuristic search algorithm with the goal of maximizing geometric independence until a preset optimization goal is met, thereby screening out a plurality of target calibration points.

9. A surround view calibration device supporting multiple types of templates, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that: When the computer program is loaded into a processor, the steps of a surround view calibration method supporting multiple types of templates according to any one of claims 1 to 6 are implemented.

10. A computer-readable storage medium storing a computer program, characterized in that: When the computer program is executed by a processor, the steps of the surround view calibration method supporting multiple types of templates according to any one of claims 1 to 6 are implemented.

Images