A method, system, device and medium for online correction of external parameters of surround view system

Abstract

The present invention relates to an online correction method, system, device and medium for external parameters of a surround view system, the method comprising: step S1: using a surround view system to collect image data; step S2: based on ORB feature matching and pixel gradient value threshold setting, frame selection and pixel selection are performed on the image data collected by the surround view system to determine the image frame and the pixel of the image for subsequent optimization; step S3: using the corresponding pixel points on the selected image frame, a dual-camera model is established, the pixel value of the common view area is scaled by calculating the pixel mean ratio of the common view area of ​​adjacent cameras, and an optimized graph structure is constructed based on the dual-camera loss term error corresponding to each pixel; step S4: using the Levenberg-Marquardt LM algorithm to optimize the optimized graph structure to obtain the optimal camera pose, thereby completing the correction of the external parameters of the surround view system. Compared with the prior art, the present invention relaxes the restrictions on the requirements for ground lane lines and has no special requirements for ground texture.

Description

Technical Field

[0001] The present invention relates to the field of surround view system parameter correction, and in particular to an online correction method, system, equipment and medium for external parameters of a surround view system. Background Art

[0002] As an indispensable component of advanced driver assistance systems, more and more vehicles are equipped with surround view systems. The surround view system usually consists of four to six wide-angle fisheye cameras, which are installed on the vehicle facing different directions to achieve 360-degree full coverage perception of the vehicle's surroundings. By calibrating the internal and external parameters of the surround view system, we can determine the relative pose between the cameras in the surround view system, and then synthesize a high-quality surround view in real time. The surround view can not only broaden the driver's field of view and eliminate blind spots, but can also be used in computer vision tasks such as parking space detection, pedestrian detection, and autonomous parking, which plays an important role in promoting the development of assisted driving.

[0003] After the extrinsic parameters are calibrated, the cameras in the surround view system will be fixed on the vehicle to ensure that the extrinsic parameters (i.e. the relative position between the cameras) will not change significantly under normal circumstances. However, collisions and bumps during vehicle driving, or thermal expansion and contraction caused by temperature changes may destroy the original spatial structure between the cameras in the surround view camera system, causing changes in the extrinsic parameters. At this time, if the original extrinsic parameters are still used to synthesize the surround view without corresponding adjustments, obvious geometric cracks will appear in the generated surround view, reflecting that the perception of the surround view system is abnormal.

[0004] When obvious geometric cracks appear in the synthesized surround view, timely correction of the extrinsic parameters of the surround view system is of great significance to ensure driving safety. Unfortunately, so far, academia and industry still lack solutions for the design of surround view systems that can effectively and online correct the extrinsic parameters of the surround view system. At present, the existing research solutions in related fields have the following two limitations:

[0005] a) Most existing methods for online correction of extrinsic parameters are designed for ordinary multi-camera systems (such as binocular cameras). Although the surround view system is also a special multi-camera system, due to the particularity of its structure, these methods are usually not easy to be extended to adapt to the characteristics of the surround view system. Specifically, most of the existing related schemes extract feature points in the common view area of ​​different cameras and calculate the corresponding descriptors, then match the features, and finally use the feature point pairs obtained by matching to restore accurate extrinsic parameters. However, in the surround view system, the common view area between adjacent cameras is much narrower than that of ordinary binocular cameras, and the distortion is more serious, which makes it difficult for us to extract high-quality feature pairs from these areas.

[0006] b) Most of the existing solutions for surround view systems are based on some specific landmarks such as lane lines, which means they rely on relatively ideal application environments. For example, there must be two parallel lane lines on the ground that can be clearly detected. After searching, Chinese Patent Publication No. CN112529966A discloses an online calibration method for a vehicle-mounted surround view system and its vehicle-mounted surround view system. This method requires feature extraction based on lane line information and has restrictions on the ground environment. Therefore, these methods have obvious limitations in terms of usability and generalization ability. Summary of the invention

[0007] The purpose of the present invention is to overcome the defects of the above-mentioned prior art and to provide a method, system, device and medium for online correction of external parameters of a surround view system with low ground texture limitation and high precision.

[0008] The purpose of the present invention can be achieved by the following technical solutions:

[0009] According to a first aspect of the present invention, there is provided a method for online correction of external parameters of a surround view system, the method comprising the following steps:

[0010] Step S1: using a surround view system to collect image data;

[0011] Step S2: Based on ORB feature matching and pixel gradient value threshold setting, frame selection and pixel selection are performed on the image data collected by the surround view system to determine the image frame and image pixels for subsequent optimization;

[0012] Step S3: using the corresponding pixel points on the selected image frame, a dual-camera model is established, the pixel values ​​of the common view area are scaled by calculating the pixel mean ratio of the common view area of ​​adjacent cameras, and an optimized graph structure is constructed based on the dual-camera loss term error corresponding to each pixel;

[0013] Step S4: Utilize the Levenberg-Marquardt LM algorithm to optimize the optimization graph structure to obtain the optimal camera pose, thereby completing the correction of the external parameters of the surround view system.

[0014] Preferably, step S2 comprises the following steps:

[0015] Step S21: Check whether the current frame is at least P frames away from the previous frame in the sliding window. If yes, continue to execute the subsequent steps; otherwise, discard the current frame and wait for a new frame to arrive.

[0016] Step S22: Detect and match ORB features on the ring view of the current frame and the previous frame, then perform homography estimation between the two frames and check the isometry of the homography matrix. When the transformation corresponding to the homography matrix satisfies the isometry transformation within the approximate range, the ground is considered to be relatively flat, and the subsequent steps are continued. Otherwise, the current frame is discarded and the arrival of a new frame is waited for.

[0017] Step S23: selecting pixel points whose pixel gradient values ​​are greater than a preset value in the common viewing area of ​​adjacent cameras of the current frame;

[0018] Step S24: based on color matching, remove the pixel points that are not located on the ground plane from the selected points whose gradient is greater than a preset value;

[0019] Step S25: Check whether the number of pixels selected on the current frame reaches a predetermined number. If so, store the current frame and the selected pixels together in the local sliding window; otherwise, discard the current frame and wait for the arrival of a new frame.

[0020] Preferably, the isometry of the homography matrix in step S22 is determined by checking whether the determinant of the 2x2 matrix block in the upper left corner of the homography matrix is ​​within an error range of 1. If so, it is an isometric transformation.

[0021] Preferably, the step S24 is specifically as follows:

[0022] First, define the variable color ratio, for the three-channel color map I t with I t′ For example, the color of point p in channel c is greater than r c (p) is defined as:

[0023]

[0024] Among them, the value of c is {1,2,3}, corresponding to R, G, and B channels respectively;

[0025] Further, I t with I t′ The color difference D corresponding to position p color (p) is:

[0026]

[0027] Among them, P p is the set consisting of p and all its adjacent pixels, r μ (p w ) is p w The mean of the color ratios over all channels.

[0028] Preferably, step S3 comprises the following steps:

[0029] Step S31: Check whether the number of frames in the sliding window meets the set value, if so, continue to execute the subsequent steps, otherwise return to step S2;

[0030] Step S32: calculating the pixel average ratio of the common viewing area of ​​adjacent cameras in the surround viewing system, and scaling the pixel values ​​in the common viewing area according to the ratio to correct the camera exposure time;

[0031] Step S33: Based on the dual-camera model and qualified pixels on all frames in the sliding window, a least squares dual-camera loss term corresponding to each pixel is constructed, and all loss terms are summed to obtain the final error of the system to be optimized;

[0032] Step S34: The camera pose to be optimized and the inverse depth information of the point are constructed as vertices, and the corresponding loss items are added as edges to the g2o optimizer to complete the construction of the optimization graph structure.

[0033] Preferably, the dual-camera loss term in step S33 is:

[0034] For camera C i With Camera C j For example, a point p on the ring graph G The corresponding dual-camera loss term The form is as follows:

[0035]

[0036] in, and Camera C i With Camera C j Images captured at the same time, and They are p G exist and The projection on γ ij is the exposure time correction factor.

[0037] Preferably, step S4 comprises the following steps:

[0038] Step S41: performing pure motion optimization, that is, optimizing only the camera pose, and performing M iterations of optimization on the camera pose using the Levenberg-Marquardt LM algorithm;

[0039] Step S42: perform dual optimization of structure and motion, that is, optimize the inverse depth and camera pose of the point at the same time, and continue N iterations;

[0040] Step S43: Read the vertex information corresponding to the camera posture from the optimized graph structure and update the external parameters.

[0041] According to a second aspect of the present invention, a system based on the above-mentioned method for online automatic correction of external parameters of a surround view system is provided, and the system comprises the following modules:

[0042] An image data acquisition module, used to acquire image data collected by the surround view system;

[0043] A data preprocessing module, used for performing frame selection and pixel selection on the collected image data;

[0044] A model building module, for building an optimized graph structure based on the frames and pixels selected by the data preprocessing module;

[0045] The external parameter correction module is used to optimize the optimization graph structure constructed by the model building module to obtain the optimal value of the camera pose, thereby correcting the external parameters of the surround view system.

[0046] According to a third aspect of the present invention, there is provided an electronic device, comprising a memory and a processor, wherein a computer program is stored in the memory, and the processor implements the above method when executing the program.

[0047] According to a fourth aspect of the present invention, there is provided a computer-readable storage medium having a computer program stored thereon, wherein the program implements the above method when executed by a processor.

[0048] Compared with the prior art, the present invention has the following advantages:

[0049] 1) The present invention is based on the optimization framework of the direct method. The dual-camera model essentially follows the idea of ​​the direct method, that is, directly projecting a point onto another image, and taking the pixel value of the point and the projection point as the error, and relying on the image gradient to guide the optimization; the indirect method, on the other hand, mainly requires determining a set of feature pairs in advance, and then taking the projection of a point on another image and the position of its paired features as the error; compared with the indirect method, the direct method does not require feature matching to determine the feature point pairs, so it does not require the presence of special markers on the ground. As long as the ground is relatively flat and has a certain amount of texture, the correction can be completed, which relaxes the restrictions on ground conditions;

[0050] 2) The present invention integrates the proposed pixel selection strategy and frame selection strategy, and can automatically select the frames and corresponding pixels suitable for optimization to avoid the failure of posture correction;

[0051] 3) The present invention can simultaneously utilize multi-frame information for joint optimization;

[0052] 4) The correction method of the present invention is designed for a surround view system and is specifically used for optimizing the external parameters of the surround view system. BRIEF DESCRIPTION OF THE DRAWINGS

[0053] Figure 1It is a flow chart of the method for automatically correcting the external parameters of the surround view system on-line according to the present invention;

[0054] Figure 2 is a flow chart of the method in the embodiment;

[0055] Figure 3 Schematic diagram of the dual-camera model.

[0056] Figure 4a This is the result graph corresponding to data set A;

[0057] Figure 4b This is the result graph corresponding to data set B;

[0058] Figure 4c This is the result graph corresponding to data set C;

[0059] Figure 4d This is the result graph corresponding to the dataset D. DETAILED DESCRIPTION

[0060] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work should fall within the scope of protection of the present invention.

[0061] The present invention discloses a method for automatically correcting the external parameters of a surround view system online. The method can be divided into three parts: 1) performing frame selection and pixel selection on image data collected by the surround view system to find image frames and suitable pixels on the images that can be used for subsequent optimization; 2) using the corresponding pixel points on the selected frames to construct an optimization structure based on a dual-camera model; 3) using the Levenberg-Marquardt method to perform graph optimization to obtain the optimal camera pose, thereby completing the correction of the external parameters of the surround view system;

[0062] Part 1: The steps of frame selection and pixel selection for the image data collected by the surround view system include:

[0063] (1) Check whether the current frame is at least five frames away from the previous frame in the sliding window. If so, continue to execute the subsequent steps; otherwise, discard the current frame and wait for the arrival of a new frame.

[0064] (2) Detect and match ORB features on the ring view of the current frame and the previous frame, then estimate the homography between the two frames and check the isometry of the homography matrix, that is, whether the determinant of the 2x2 matrix block in the upper left corner of the homography matrix is ​​close to 1. When the homography transformation is sufficiently close to the isometry transformation, the ground can be considered relatively flat, and the subsequent steps are continued. Otherwise, the current frame is discarded and the arrival of a new frame is waited for.

[0065] (3) Select pixels with larger pixel gradient values ​​in the common view area of ​​adjacent cameras in the current frame.

[0066] (4) Eliminate pixels that are not on the ground plane from the selected points with larger gradients based on color matching.

[0067] (5) Check whether the number of pixels selected on the current frame is sufficient. If so, store the current frame and the selected pixels together in the local sliding window. Otherwise, discard the current frame and wait for the arrival of a new frame.

[0068] In the second part, the steps to build the optimized structure include:

[0069] (1) Check whether the number of frames in the sliding window is sufficient. If so, continue to perform the subsequent steps; otherwise, return to the first part of the process.

[0070] (2) Calculate the pixel mean ratio of the common viewing area of ​​adjacent cameras in the surround view system, and scale the pixel values ​​in the common viewing area according to the ratio to correct the camera exposure time.

[0071] (3) Based on the dual-camera model and the qualified pixels on all frames in the sliding window, the least squares loss term corresponding to each pixel is constructed, and all loss terms are summed to obtain the final error of the system to be optimized.

[0072] (4) The camera pose and inverse depth information of the points to be optimized are constructed as vertices, and the corresponding loss items are added as edges to the g2o optimizer to complete the construction of the optimization graph structure.

[0073] In the third part, the steps of camera pose correction include:

[0074] (1) Perform pure motion optimization, only optimize the camera pose, and use the Levenberg-Marquardt method to optimize the camera pose for 100 iterations.

[0075] (2) Perform dual optimization of structure and motion, and optimize the inverse depth and camera pose of the point at the same time, for 30 iterations.

[0076] (3) Read the vertex information corresponding to the camera posture from the optimized graph structure and complete the update of external parameters. Specific embodiments

[0078] Figure 1 The figure shows a specific flow chart of an online correction method for external parameters of a surround view system according to the present invention. Figure 2 The following steps are described in detail:

[0079] During data preprocessing, first check whether the time interval between the current set of surround view frames collected by the surround view system and the previous frame in the local window is sufficient. The time sequence between the two frames should be at least five frames apart.

[0080] If the time interval between the current frame and the last frame in the local window meets the requirements, the homography transformation matrix between the current frame and the previous frame is calculated, and the determinant of the 2x2 matrix block in the upper left corner of the estimated homography matrix is ​​checked to see if it is close to 1, thereby determining the isometry of the homography matrix. When the determinant is close enough to 1, that is, between 0.8 and 1.2, it is considered that the vehicle is approximately moving in a plane, that is, the ground is relatively flat;

[0081] When both of the above checks have passed, pixel selection is then performed on the current frame. First, the area of ​​common view of adjacent cameras is selected, and pixels with relatively large gradients are selected from them. The gradient of the selected pixel point should be greater than the average gradient of the image and greater than 70;

[0082] Then, the homography matrix obtained between the current frame and the previous frame is used to align the two adjacent frames, and the color difference of the corresponding pixels between the two frames is checked, and the pixels whose difference is greater than the average value are eliminated;

[0083] The specific calculation method of color difference is as follows:

[0084] First, define the variable color ratio; for the three-channel color map I t with I t′ For example, the color ratio of point p in channel c (from 1 to 3 corresponds to R, G, B respectively) is r c (p) is defined as:

[0085]

[0086] Further, I t with I t′ The color difference D corresponding to position p color (p) is

[0087]

[0088] Among them, P p is the set consisting of p and all its adjacent pixels, r μ (p w ) is p w The mean of the color ratios over all channels.

[0089] After completing the pixel selection, check whether the number of qualified pixels in the current frame is not less than 6000. If this condition is met, it is considered that the texture on the ground is relatively rich, and the frame and related pixel selection information are stored in the local window. Otherwise, the frame is discarded;

[0090] During the optimization structure construction process, when the number of frames in the local window reaches 10, the optimization structure construction begins. First, calculate the adjacent camera C i With C j The ratio of the mean pixel values ​​in the common view area γ ij , and scale the pixel values ​​in the corresponding common view area according to the ratio to reduce the impact of camera exposure time on system performance;

[0091] Based on the dual-camera model, each qualified pixel point on each frame in the local window can be used to construct a dual-camera error term. The final least squares error to be optimized is obtained by adding up all the error terms.

[0092] like Figure 3 As shown, for camera C i With Camera C j For example, a point p on the ring graph G The corresponding dual camera error The form is as follows:

[0093]

[0094] in, and Camera C i With Camera C j Images captured at the same time, and They are p G exist and The projection on γ ij is the exposure time correction factor.

[0095] It can be seen that the dual-camera model essentially follows the idea of ​​the direct method, that is, directly projecting a point onto another image, taking the pixel value of the point and the projection point as the error, and relying on the image gradient to guide the optimization. In contrast, the indirect method mainly requires determining a set of feature pairs in advance, and then taking the projection of a point on another image and the position of its paired features as the error. Compared with the indirect method, the direct method does not require feature matching to determine the feature point pairs, so it does not require special landmarks on the ground, and can work as long as there is a certain texture on the ground.

[0096] Based on the g2o graph optimization library, all camera poses to be optimized and the inverse depths of all qualified points are constructed as vertices, all error terms are constructed as edges, and all vertices and edges are registered to the optimizer, and then the optimization process will begin;

[0097] During the optimization process, the Levenberg-Marquardt method was first used to perform 100 iterations of pure motion optimization, that is, all inverse depth values ​​were fixed and only the camera pose was optimized;

[0098] Subsequently, the Levenberg-Marquardt LM algorithm was used to perform 30 iterations of structure-motion dual optimization, that is, to simultaneously optimize the camera pose and the inverse depth value of the point;

[0099] After the optimization is completed, the optimized values ​​are read from the camera pose vertices and the camera extrinsic parameters are updated.

[0100] This example collects four test data sets A to D, each of which contains 100 images. The characteristics of each data set from A to D are as follows:

[0101] A) Very rich texture;

[0102] B) Richer texture;

[0103] C) texture is relatively sparse;

[0104] D) There is an object with obvious height in the field of vision;

[0105] The method of the present invention, the method in the document "Online Camera Pose Optimization for the Surround-view System" (Xiao Liu, Lin Zhang, Ying Shen et al., in Proc. ACM Conf. Multimedia, 2019) and the method in the document "OECS: Towards Online Extrinsics Correction for The Surround-view System" (Tianjun Zhang, Lin Zhang, Ying Shen et al., in Proc. IEEE Int'l Conf. Multimedia and Expo, 2020) were tested respectively on these four sets of data, and the average photometric error of the synthesized ring view using the corrected extrinsic parameters for each set of data was recorded. In addition, the offline calibration scheme "Revisit surround-view camera system calibration" (Xuan Shao, Xiao Liu, Lin Zhang et al., in Proc. IEEE Int'l Conf. Multimedia and Expo, 2019.) was used to obtain the extrinsic parameters, and the corresponding average photometric error was recorded as a reference. Specific experimental results Figure 4a As shown in Figure 2, it can be seen from the result diagram that the method of the present invention is significantly better than the method of the comparative literature.

[0106] Those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working process of the described module can refer to the corresponding process in the aforementioned method embodiment, and will not be repeated here.

[0107] The electronic device of the present invention includes a central processing unit (CPU), which can perform various appropriate actions and processes according to computer program instructions stored in a read-only memory (ROM) or loaded from a storage unit into a random access memory (RAM). In the RAM, various programs and data required for device operation can also be stored. The CPU, ROM and RAM are connected to each other via a bus. An input / output (I / O) interface is also connected to the bus.

[0108] Multiple components in the device are connected to the I / O interface, including: input units, such as keyboards, mice, etc.; output units, such as various types of displays, speakers, etc.; storage units, such as disks, optical disks, etc.; and communication units, such as network cards, modems, wireless communication transceivers, etc. The communication unit allows the device to exchange information / data with other devices through computer networks such as the Internet and / or various telecommunication networks.

[0109] The processing unit performs the various methods and processes described above, such as methods S1 to S4. For example, in some embodiments, methods S1 to S4 may be implemented as a computer software program, which is tangibly contained in a machine-readable medium, such as a storage unit. In some embodiments, part or all of the computer program may be loaded and / or installed on the device via a ROM and / or a communication unit. When the computer program is loaded into the RAM and executed by the CPU, one or more steps of methods S1 to S4 described above may be performed. Alternatively, in other embodiments, the CPU may be configured to execute methods S1 to S4 in any other appropriate manner (e.g., by means of firmware).

[0110] The functions described above herein may be performed at least in part by one or more hardware logic components. For example, without limitation, exemplary types of hardware logic components that may be used include: field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), systems on chip systems (SOCs), load programmable logic devices (CPLDs), and the like.

[0111] The program code for implementing the method of the present invention can be written in any combination of one or more programming languages. These program codes can be provided to a processor or controller of a general-purpose computer, a special-purpose computer or other programmable data processing device, so that the program code, when executed by the processor or controller, enables the functions / operations specified in the flow chart and / or block diagram to be implemented. The program code can be executed entirely on the machine, partially on the machine, partially on the machine as a stand-alone software package and partially on a remote machine, or entirely on a remote machine or server.

[0112] In the context of the present invention, a machine-readable medium may be a tangible medium that may contain or store a program for use by or in conjunction with an instruction execution system, device, or equipment. A machine-readable medium may be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium may include, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, devices, or equipment, or any suitable combination of the foregoing. A more specific example of a machine-readable storage medium may include an electrical connection based on one or more lines, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

[0113] The above is only a specific embodiment of the present invention, but the protection scope of the present invention is not limited thereto. Any technician familiar with the technical field can easily think of various equivalent modifications or replacements within the technical scope disclosed by the present invention, and these modifications or replacements should be included in the protection scope of the present invention. Therefore, the protection scope of the present invention shall be based on the protection scope of the claims.

Claims

1. A method for online correction of external parameters of a surround view system, characterized in that: The method comprises the following steps: Step S1: using a surround view system to collect image data; Step S2: Based on ORB feature matching and pixel gradient value threshold setting, frame selection and pixel selection are performed on the image data collected by the surround view system to determine the image frame and image pixels for subsequent optimization; Step S3: using the corresponding pixel points on the selected image frame, a dual-camera model is established, the pixel values ​​of the common view area are scaled by calculating the pixel mean ratio of the common view area of ​​adjacent cameras, and an optimized graph structure is constructed based on the dual-camera loss term error corresponding to each pixel; Step S4: Optimizing the optimization graph structure using the Levenberg-Marquardt LM algorithm to obtain the optimal camera pose, thereby completing the correction of the external parameters of the surround view system; The step S2 comprises the following steps: Step S21: Check whether the current frame is at least P frames away from the previous frame in the sliding window. If yes, continue to execute the subsequent steps; otherwise, discard the current frame and wait for a new frame to arrive. Step S22: Detect and match ORB features on the ring view of the current frame and the previous frame, then perform homography estimation between the two frames and check the isometry of the homography matrix. When the transformation corresponding to the homography matrix satisfies the isometry transformation within the approximate range, the ground is considered to be relatively flat, and the subsequent steps are continued. Otherwise, the current frame is discarded and the arrival of a new frame is waited for. Step S23: selecting pixel points whose pixel gradient values ​​are greater than a preset value in the common viewing area of ​​adjacent cameras of the current frame; Step S24: based on color matching, remove the pixel points that are not located on the ground plane from the selected points whose gradient is greater than a preset value; Step S25: Check whether the number of pixels selected on the current frame reaches a predetermined number. If so, store the current frame and the selected pixels together in the local sliding window. Otherwise, discard the current frame and wait for the arrival of a new frame. The step S3 comprises the following steps: Step S31: Check whether the number of frames in the sliding window meets the set value, if so, continue to execute the subsequent steps, otherwise return to step S2; Step S32: calculating the pixel average ratio of the common viewing area of ​​adjacent cameras in the surround viewing system, and scaling the pixel values ​​in the common viewing area according to the ratio to correct the camera exposure time; Step S33: Based on the dual-camera model and qualified pixels on all frames in the sliding window, a least squares dual-camera loss term corresponding to each pixel is constructed, and all loss terms are summed to obtain the final error of the system to be optimized; Step S34: construct the camera pose to be optimized and the inverse depth information of the point as vertices, and the corresponding loss items as edges, and add them to the g2o optimizer to complete the construction of the optimization graph structure; The dual-camera loss term in step S33 is: For camera C i With Camera C j For example, a point p on the ring graph G The corresponding dual-camera loss term The form is as follows: in, and Camera C i With Camera C j Images captured at the same time. and They are p G exist and The projection on γ ij is the exposure time correction factor.

2. The method for online correction of external parameters of a surround view system according to claim 1, characterized in that: The isometry determination of the homography matrix in step S22 is as follows: checking whether the determinant of the 2x2 matrix block in the upper left corner of the homography matrix is ​​within a preset error range of 1. If so, it is an isometric transformation.

3. The method for online correction of external parameters of a surround view system according to claim 1, characterized in that: The step S24 is specifically as follows: First, define the variable color ratio, for the three-channel color map I t with I t′ For example, the color of point p in channel c is greater than r c (p) is defined as: Among them, the value of c is {1,2,3}, corresponding to R, G, and B channels respectively; Further, I t with I t′ The color difference D corresponding to position p color (p) is: Among them, P p is the set consisting of p and all its adjacent pixels, r μ (p w ) is p w The mean of the color ratios over all channels.

4. The method for online correction of external parameters of a surround view system according to claim 1, characterized in that: The step S4 comprises the following steps: Step S41: performing pure motion optimization, that is, optimizing only the camera pose, and performing M iterations of optimization on the camera pose using the Levenberg-Marquardt LM algorithm; Step S42: perform dual optimization of structure and motion, that is, optimize the inverse depth and camera pose of the point at the same time, and continue N iterations; Step S43: Read the vertex information corresponding to the camera posture from the optimized graph structure and update the external parameters.

5. A system based on the method for online correction of external parameters of a surround view system according to claim 1, characterized in that: The system includes the following modules: An image data acquisition module, used to acquire image data collected by the surround view system; A data preprocessing module, used for performing frame selection and pixel selection on the collected image data; A model building module, for building an optimized graph structure based on the frames and pixels selected by the data preprocessing module; The external parameter correction module is used to optimize the optimization graph structure and obtain the optimal value of the camera pose, thereby correcting the external parameters of the surround view system.

6. An electronic device comprising a memory and a processor, wherein a computer program is stored in the memory, wherein: When the processor executes the program, the method according to any one of claims 1 to 4 is implemented.

7. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the program is executed by a processor, the method according to any one of claims 1 to 4 is implemented.

Images