Camera parameter adjustment method and device, vehicle, medium and program product

Abstract

The application relates to a camera parameter adjustment method and device, a vehicle, a medium and a program product, which are applied to a vehicle. The adjustment method comprises the following steps: determining a target optimization mode based on road information where the vehicle is located; determining a detection result based on current driving information and current lane line information of the vehicle, wherein the detection result represents whether a parameter adjustment is triggered; in the case that the detection result is a first detection result, determining current adjustment information of a first surround-view camera based on a target image containing a dashed lane collected by the first surround-view camera and last adjustment information of the first surround-view camera according to the target optimization mode; and adjusting the external parameter of the first surround-view camera according to the current adjustment information of the first surround-view camera. The application can improve the adaptability and accuracy of parameter adjustment of the surround-view camera of the vehicle.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of vehicles, in particular to a camera parameter adjustment method and device, vehicle, medium and program product. BACKGROUND

[0002] In an intelligent driving system, as an important perception device, the accuracy of the parameters of a surround-view camera directly affects key functions such as image recognition and lane line detection. During vehicle driving, parameter drift may be caused by factors such as vibration and temperature change, so dynamic adjustment or calibration of the parameters of the surround-view camera is needed to ensure that the surround-view camera can continuously provide reliable visual information.

[0003] In related technologies, the adjustment of the parameters of the surround-view camera is basically dependent on a physical target, which has problems such as poor adaptability and low accuracy. SUMMARY

[0004] One of the purposes of the present application is to provide a camera parameter adjustment method to solve the problems of poor adaptability and low accuracy in the adjustment of the parameters of the surround-view camera in related technologies; the second purpose is to provide a camera parameter adjustment device; the third purpose is to provide a vehicle; the fourth purpose is to provide a computer-readable storage medium; and the fifth purpose is to provide a computer program product.

[0005] To achieve the above purposes, the present application provides a camera parameter adjustment method applied to a vehicle, which adopts the following technical solutions:

[0006] Based on road information in which the vehicle is located, a target optimization mode is determined; wherein the target optimization mode includes one of the following: a first optimization mode, a second optimization mode, and the adjustment order of the second optimization mode is different from that of the first optimization mode;

[0007] Based on current driving information and current lane line information of the vehicle, a detection result is determined; wherein the detection result represents whether to trigger parameter adjustment;

[0008] In the case of the detection result being a first detection result, current adjustment information of a first surround-view camera is determined based on a target image containing a dashed lane collected by the first surround-view camera and previous adjustment information of the first surround-view camera according to the target optimization mode; wherein the first detection result represents triggering parameter adjustment; the first surround-view camera includes at least one of the following: a front surround-view camera installed at the front of the vehicle body, a rear surround-view camera installed at the rear of the vehicle body, a left surround-view camera installed at the left side of the vehicle body, and a right surround-view camera installed at the right side of the vehicle body; the previous adjustment information includes a previous pitch angle, a previous yaw angle, and a previous roll angle;

[0009] Adjust the extrinsic parameter of the first surround-view camera according to the current adjustment information of the first surround-view camera.

[0010] According to the above technical means, first, the target optimization mode is determined according to the road information, which can dynamically change the adjustment strategy according to different road conditions, improving the flexibility and adaptability of the adjustment. Second, whether to trigger parameter adjustment is comprehensively determined according to the current driving information and lane line information, so as to ensure that the adjustment process is started under appropriate conditions, improving the stability and reliability of the adjustment. Finally, by adjusting based on historical adjustment information and current image features, dynamic parameter adjustment can be realized without physical targets, which can effectively deal with the problem of extrinsic parameter offset caused by mechanical deformation in long-term use, while avoiding the shortcomings of traditional calibration relying on fixed scenes or manual intervention, improving the accuracy and efficiency of adjustment, thereby reducing the re-calibration process caused by panoramic stitching effect or automatic parking performance problems, and further reducing maintenance costs.

[0011] Further, based on the road information of the vehicle, the target optimization mode is determined, including: in the case that the road information represents that the vehicle is in a single lane or the side lane of the vehicle is a solid line lane, the first optimization mode is taken as the target optimization mode; in the case that the road information represents that the side lanes on both sides of the vehicle are virtual line lanes, the second optimization mode is taken as the target optimization mode.

[0012] According to the above technical means, by distinguishing different road structure characteristics such as single solid line and double virtual line, different optimization strategies are selected and corresponding adjustment priorities are set, so as to optimize the adjustment efficiency and accuracy in different scenes. In particular, in complex and variable road conditions, adjustment resources can be more reasonably allocated, invalid adjustment can be reduced, and overall response speed and accuracy can be improved.

[0013] Further, based on the current driving information and the current lane line information of the vehicle, the detection result is determined, including: in the case that the current driving information of the vehicle represents that the vehicle is in a uniform straight driving state, and the current lane line information includes a virtual line lane, the first detection result is taken as the detection result; in the case that the current driving information of the vehicle represents that the vehicle is not in a uniform straight driving state, and / or the current lane line information does not include a virtual line lane, the second detection result is taken as the detection result; wherein the second detection result represents that the parameter adjustment is not triggered.

[0014] According to the above technical means, by jointly judging the driving state of the vehicle and the type of lane line, it is determined whether to trigger parameter adjustment, so that the adjustment is triggered only under certain conditions, thereby avoiding false adjustment or invalid adjustment in unstable driving environment, and improving the effectiveness and robustness of the adjustment.

[0015] Further, according to the target tuning mode, based on the target image containing the dashed lane collected by the first surround-view camera and the last adjustment information of the first surround-view camera, the current adjustment information of the first surround-view camera is determined, comprising: according to the target tuning mode, the first surround-view camera is determined from a plurality of surround-view cameras; wherein the plurality of surround-view cameras include a front surround-view camera, a rear surround-view camera, a left surround-view camera and a right surround-view camera; based on the target image containing the dashed lane collected by the first surround-view camera, a lane line feature point is determined; based on the lane line feature point, current deviation information of the first surround-view camera is determined; based on the current deviation information of the first surround-view camera and the last adjustment information of the first surround-view camera, the current adjustment information of the first surround-view camera is determined.

[0016] According to the above technical means, by selecting appropriate cameras, extracting lane line key points, calculating deviations and combining historical adjustment data, a closed-loop feedback mechanism is constructed to realize accurate control of the current adjustment information, thereby improving the convergence speed and stability of the adjustment, avoiding problems such as excessive adjustment and insufficient adjustment, and ensuring that a high adjustment accuracy can be maintained under different driving conditions.

[0017] Further, the current deviation information includes pitch angle deviation information and heading angle deviation information; based on the lane line feature point, the current deviation information of the first surround-view camera is determined, comprising: based on the lane line feature point, at least two lane lines in the target image are determined to have current vanishing points; based on the current vanishing points and the reference vanishing points, a current deviation vector is determined; wherein the current deviation vector includes a current lateral deviation vector and a current longitudinal deviation vector, and the reference vanishing points are determined based on the parameters of the first surround-view camera; based on the current lateral deviation vector, the pitch angle deviation information is determined; based on the current longitudinal deviation vector, the heading angle deviation information is determined.

[0018] According to the above technical means, by analyzing the relative positional relationship between the lane line vanishing points and the reference vanishing points, the deviation values of the pitch angle and the heading angle are derived, which is suitable for adaptive calibration requirements under various road conditions, and the accuracy of the adjustment is improved.

[0019] Further, the current deviation information includes roll angle deviation information; based on the lane line feature point, the current deviation information of the first surround-view camera is determined, comprising: based on the lane line feature point, a slope of a target line segment is determined; wherein the target line segment is composed of one endpoint of two dashed line segments in the dashed lane included in the target image; in the case that the slope of the target line segment is not equal to zero, based on the deviation angle between the target line segment and the lower edge of the target image, the roll angle deviation information is determined.

[0020] According to the above technical means, by analyzing the included angle between the auxiliary line formed by the lane line end point and the bottom edge of the image, the roll angle deviation is evaluated, thereby further expanding the parameter adjustment dimension, making the adjustment not limited to the pitch direction and the heading direction, but also covering the roll direction, and comprehensively improving the calibration accuracy. At the same time, it can also avoid the problem of relying on physical calibration board or fixed scene in related technologies, so as to adapt to the dynamic changes in the vehicle driving process, improve the real-time and stability of the calibration, and then effectively cope with the external parameter offset problem caused by mechanical deformation in the long-term use of the vehicle, and reduce the after-sales maintenance cost.

[0021] Further, in the case where the target tuning mode is the first tuning mode, the adjustment method further comprises: generating a first inverse perspective transformation graph based on the image collected by the second surround view camera; wherein the second surround view camera includes a front surround view camera and a rear surround view camera; performing geometric constraint verification on the lane line in the first inverse perspective transformation graph to obtain a parameter verification result of the second surround view camera; wherein the geometric constraint verification includes at least one of the following: lane line parallel line constraint verification, lane line equal length constraint verification, and lane line endpoint connecting line parallel constraint; the lane line parallel line constraint verification is used to verify whether the same lane line collected by two surround view cameras is parallel; the lane line equal length constraint verification is used to verify whether the connecting line between the two endpoints of the two dashed line segments collected by two surround view cameras is equal; and the lane line endpoint connecting line parallel constraint is used to verify whether the connecting line between the two endpoints of the two dashed line segments collected by two surround view cameras is parallel; in the case where the parameter verification result of the second surround view camera represents that the verification passes, readjusting the external parameters of the third surround view camera; wherein the third surround view camera includes a left surround view camera and a right surround view camera.

[0022] According to the above technical means, by inverse perspective transformation graph and geometric constraint for re-examination verification, it is ensured that the adjusted parameters meet the global constraint condition, thereby introducing a global verification mechanism on the basis of local adjustment, improving the accuracy and reliability of the adjustment.

[0023] Further, after adjusting the external parameters of the third surround view camera, the adjustment method further comprises: generating a second inverse perspective transformation graph based on the image collected by the fourth surround view camera; wherein the fourth surround view camera includes a front surround view camera, a rear surround view camera, a left surround view camera and a right surround view camera; performing geometric constraint verification on the lane line in the second inverse perspective transformation graph to obtain a parameter verification result of the third surround view camera; in the case where the parameter verification results of the third surround view camera all represent that the verification passes, fixing the external parameters of the fourth surround view camera.

[0024] According to the above technical means, in a specific scene, the previous and then left and right step-by-step rechecking verification is adopted to ensure the consistency and rationality of each camera parameter after each round of adjustment, effectively eliminate the error caused by mis-detection or noise, thereby improving the calibration accuracy, and further enhancing the panoramic image stitching quality and the stability of the automatic driving system.

[0025] Further, in the case where the target optimization mode is the second optimization mode, the adjustment method further comprises: generating a third inverse perspective transformation map based on the image collected by the fourth surround-view camera; performing geometric constraint verification on the lane lines in the third inverse perspective transformation map to obtain a parameter verification result of the fourth surround-view camera; and fixing the extrinsic parameter of the fourth surround-view camera in the case where the parameter verification result of the fourth surround-view camera represents that the verification is passed.

[0026] According to the above technical means, in a specific scene, a global adjustment strategy is adopted to complete parameter optimization with more stringent standards, ensuring that all camera parameters remain consistent in high-precision requirement scenarios.

[0027] Further, the adjustment method further comprises: determining stitching misalignment information and slope deviation information based on the image collected by the fourth surround-view camera; wherein the stitching misalignment information includes whether each intersection point overlaps in the images collected by adjacent surround-view cameras, and the intersection point is the intersection point between the lane line and the preset target ray; the slope deviation information includes the slope difference of the same lane line collected by adjacent surround-view cameras; and the adjustment result is determined based on the stitching misalignment information and the slope deviation information.

[0028] According to the above technical means, the stitching misalignment and slope difference statistical indicators are used to evaluate the re-projection error, which is particularly suitable for road scenes lacking real-world coordinate system information, and realizes quantitative evaluation and continuous optimization of the adjustment effect.

[0029] An adjustment device for camera parameters is applied to a vehicle, and the adjustment device comprises:

[0030] A first determination module is configured to determine a target optimization mode based on road information where the vehicle is located; wherein the target optimization mode comprises one of the following: a first optimization mode and a second optimization mode, and the adjustment order of the second optimization mode is different from that of the first optimization mode;

[0031] A second determination module is configured to determine a detection result based on current driving information and current lane line information of the vehicle; wherein the detection result represents whether to trigger parameter adjustment;

[0032] The third determining module is configured to, when the detection result is a first detection result, determine the current adjustment information of the first surround-view camera according to a target tuning manner based on a target image containing a dashed lane collected by the first surround-view camera and previous adjustment information of the first surround-view camera, wherein the first detection result represents a trigger parameter adjustment; the first surround-view camera includes at least one of a front surround-view camera installed at the front of the vehicle body, a rear surround-view camera installed at the rear of the vehicle body, a left surround-view camera installed at the left side of the vehicle body, and a right surround-view camera installed at the right side of the vehicle body; and the previous adjustment information includes a previous pitch angle, a previous yaw angle, and a previous roll angle.

[0033] The adjusting module is configured to adjust the extrinsic parameter of the first surround-view camera according to the current adjustment information of the first surround-view camera.

[0034] A vehicle includes a processor and a memory, the memory storing a computer program capable of running on the processor, and the processor implements the method of any one of the above when executing the computer program.

[0035] A computer-readable storage medium stores a computer program, and the computer program is executed by a processor to implement the method of any one of the above.

[0036] A computer program product includes a computer program or instructions, and the computer program or instructions are executed by a processor to implement the method of any one of the above.

[0037] The beneficial effects of the present application are as follows:

[0038] (1) By distinguishing different road structure features such as single solid line, double dashed line, etc., different tuning strategies are selected and corresponding adjustment priorities are set, so as to optimize the adjustment efficiency and accuracy in different scenarios, especially in complex and variable road conditions, the adjustment resources can be more reasonably allocated.

[0039] (2) By jointly judging the vehicle driving state and the lane line type, it is determined whether to trigger parameter adjustment, so that the adjustment is only triggered under certain conditions, thereby avoiding false adjustment or invalid adjustment in unstable driving environment, and improving the effectiveness and robustness of the adjustment.

[0040] (3) By selecting appropriate cameras, extracting lane line key points, calculating deviations, and combining historical adjustment data, a closed-loop feedback mechanism is constructed to realize precise control of the current adjustment information, thereby improving the convergence speed and stability of the adjustment, avoiding problems such as excessive adjustment and insufficient adjustment, to ensure that a high adjustment accuracy can be maintained under different driving conditions.

[0041] (4) By analyzing the angle between the auxiliary line formed by the lane line endpoint and the bottom edge of the image, the roll angle deviation is evaluated, and dynamic parameter adjustment can be performed without physical targets, which can effectively deal with the problem of external parameter offset caused by mechanical deformation in long-term use, and avoid the shortcomings of traditional calibration relying on fixed scene or manual intervention, improve the accuracy and efficiency of adjustment, thereby reducing the after-sales recalibration process caused by panoramic stitching effect or automatic parking performance problems, and further reducing maintenance costs.

[0042] (5) Through the inverse perspective transformation graph and geometric constraints, the consistency and rationality of each camera parameter after each round of adjustment are verified, effectively excluding errors caused by false detection or noise.

[0043] (6) The reprojection error is evaluated by stitching misplacement and slope difference statistical indicators, which is especially suitable for road scenes lacking real-world coordinate system information, and realizes the quantitative evaluation and continuous optimization of the adjustment effect. BRIEF DESCRIPTION OF DRAWINGS

[0044] Figure 1 An implementation process schematic of a camera parameter adjustment method provided by an embodiment of the present application Figure One ;

[0045] Figure 2 A schematic of a lane line provided by an embodiment of the present application Figure One ;

[0046] Figure 3 A schematic of a lane line provided by an embodiment of the present application Figure Two ;

[0047] Figure 4 A schematic of a lane line provided by an embodiment of the present application Figure Three ;

[0048] Figure 5 A schematic diagram of a vanishing point provided by an embodiment of the present application

[0049] Figure 6 A schematic of a lane line provided by an embodiment of the present application Figure Four ;

[0050] Figure 7 A schematic of a lane line provided by an embodiment of the present application Figure Five ;

[0051] Figure 8 A schematic of a lane line provided by an embodiment of the present application Figure Six ;

[0052] Figure 9 An implementation process schematic of a camera parameter adjustment method provided by an embodiment of the present applicationFigure Two ;

[0053] Figure 10 A schematic diagram of a camera parameter adjustment device according to an embodiment of the present application;

[0054] Figure 11 A schematic diagram of a hardware entity of an electronic device according to an embodiment of the present application. DETAILED DESCRIPTION

[0055] Other advantages and effects of the present application can be easily understood by those skilled in the art from the description of the present application. The present application can also be implemented or applied by different specific embodiments, and the details in the description can be modified or changed based on different views and applications without departing from the spirit of the present application. It should be understood that the preferred embodiments are only for illustrating the present application, but not for limiting the protection scope of the present application.

[0056] It should be noted that the diagrams provided in the following embodiments only schematically illustrate the basic concept of the present application, and only the components related to the present application are shown in the diagrams, but not the number, shape and size of the components in actual implementation. The shape, number and proportion of the components in actual implementation can be randomly changed, and the layout pattern of the components can be more complex.

[0057] In the following description, "some embodiments" are described, which describe a subset of all possible embodiments, but it can be understood that "some embodiments" can be the same subset or different subset of all possible embodiments, and can be combined with each other without conflict.

[0058] In the following description, the terms "first\second\third" are only to distinguish similar objects, and do not represent the specific order of the objects. It can be understood that "first\second\third" can be interchanged with the specific order or sequence as allowed, so that the embodiments of the present application described herein can be implemented in an order other than that illustrated or described herein.

[0059] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which the present application belongs. The terms used herein are only for the purpose of describing the embodiments of the present application, and are not intended to limit the present application.

[0060] The method provided in this application can be executed by an electronic device, which can be a laptop, tablet, desktop computer, vehicle, set-top box, mobile device (e.g., mobile phone, portable music player, personal digital assistant, dedicated messaging device, portable gaming device), or a server. The server can be a standalone physical server, a server cluster or distributed system composed of multiple physical servers, or a cloud server providing basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communication, middleware services, domain name services, security services, content delivery networks (CDN), and big data and artificial intelligence platforms.

[0061] The technical solutions in the embodiments of this application will now be clearly and completely described with reference to the accompanying drawings.

[0062] Figure 1 A schematic diagram of the implementation process of a camera parameter adjustment method provided in this application embodiment. Figure One ,like Figure 1 As shown, the adjustment method includes steps S11 to S14, wherein:

[0063] Step S11: Determine the target optimization method based on the road information where the vehicle is located.

[0064] Here, road information refers to the structural characteristics of the road on which the vehicle travels, such as single-lane, three-lane, or side lanes consisting of dashed lines. A single-lane road refers to a road with only one lane available for travel, commonly seen on narrow roads or temporarily controlled traffic sections. A three-lane road is a common road structure, meaning there are three lanes on the road, usually the middle one as the main lane and the left and right auxiliary lanes. Side lanes consisting of dashed lines on both sides mean that there are dashed lane lines on both sides of the main lane where the vehicle is located, indicating the presence of more than three lanes. Road information determines the parameter optimization method; different road information can correspond to the same or different optimization methods.

[0065] Optimization methods may include, but are not limited to, adjusting the target and the order of adjustments. The target can include at least one surround-view camera. A surround-view camera is a wide-angle lens installed around the vehicle (including the front, rear, left, and right sides) to capture images of the surrounding environment, achieving 360° field of view coverage. That is, the vehicle includes a front surround-view camera installed at the front of the vehicle, a rear surround-view camera installed at the rear of the vehicle, a left surround-view camera installed on the left side of the vehicle, and a right surround-view camera installed on the right side of the vehicle. The adjustment order refers to the priority of parameter adjustments for each target.

[0066] The tuning mode can include, but is not limited to, a first tuning mode, a second tuning mode, and the like. The adjustment objects and / or adjustment sequences of the first tuning mode and the second tuning mode are different. For example, the adjustment objects of the first tuning mode and the second tuning mode are the same, but the adjustment sequences are different. For example, the first tuning mode is to tune in the order of the rear surround view camera, the right surround view camera, the left surround view camera, and the front surround view camera; and the second tuning mode is to adjust the front and rear surround view cameras first, and then adjust the left and right surround view cameras.

[0067] The target tuning mode is a tuning mode selected under a specific road condition to ensure the accuracy and stability of the camera calibration result. The determination manner of the target tuning mode can be any suitable manner.

[0068] In some embodiments, a correspondence between each road information and each tuning mode can be established in advance, and according to the correspondence, the target tuning mode that is adapted to the road information where the vehicle is located can be obtained.

[0069] In some embodiments, the road information can be input into a pre-established rule base or a machine learning model to obtain the target tuning mode.

[0070] In some embodiments, the step S11 includes a step S111 and / or a step S112, wherein:

[0071] The step S111 is to select the first tuning mode as the target tuning mode when the road information indicates that the vehicle is in a single lane or at least one side of the adjacent lane is a solid line lane.

[0072] The step S112 is to select the second tuning mode as the target tuning mode when the road information indicates that the adjacent lanes on both sides of the vehicle are dotted lanes.

[0073] Here, when the vehicle is driving in a single lane or at least one side of the adjacent lane is a solid line lane, it indicates that in this scenario, the surround view cameras on both sides cannot construct a regular rectangle for independent tuning, so the first tuning mode is selected as the target tuning mode. When the vehicle is driving in the adjacent lanes on both sides are dotted lanes, it indicates that this scenario is more suitable for constructing a regular rectangle for independent tuning, so the second tuning mode is selected as the target tuning mode. The first tuning mode is limited in more applicable scenarios, and the surround view cameras on both sides are not conducive to independent tuning, and focuses on high-precision matching of local features; while the second tuning mode is applicable to more open scenarios, such as multiple dotted lanes, and the second tuning mode can use the information of multiple common areas for independent optimization, thereby improving the overall consistency of the calibration.

[0074] In some embodiments, considering the common vertical parking, the rear surround view camera has the highest accuracy requirement, while for horizontal parking and diagonal parking, the rear surround view camera works together with the left and right surround view cameras, and the accuracy requirement is higher than that of the front surround view camera. Therefore, in the first tuning mode, the rear surround view camera can be adjusted first, then the front surround view camera, and then the left and right surround view cameras are adjusted jointly; and in the second tuning mode, the rear surround view camera can be adjusted first, then the left and right surround view cameras, and finally the front surround view camera.

[0075] In the embodiments of the present application, by distinguishing different road structure features such as single solid line, double dashed line, etc., different tuning strategies are selected and corresponding adjustment priorities are set, so as to optimize the adjustment efficiency and accuracy in different scenarios, especially in complex and variable road conditions, the adjustment resources can be more reasonably allocated, the invalid adjustment is reduced, and the overall response speed and accuracy are improved.

[0076] Step S12, determining a detection result based on the current driving information and the current lane line information of the vehicle; wherein the detection result represents whether to trigger parameter adjustment.

[0077] Here, the driving information can include but is not limited to chassis signals of the vehicle such as steering angle, acceleration, etc. The driving information is used to judge whether the vehicle is in a suitable calibration condition. For example, only when the vehicle is in a uniform straight-line driving state, parameter adjustment is suitable.

[0078] The current lane line information includes the detected lane line type (solid line or dashed line), position, length, etc.

[0079] The detection result is a logical judgment result, which is used to decide whether to enter the subsequent parameter adjustment process. The detection result can include but is not limited to a first detection result, a second detection result, etc. The first detection result represents triggering parameter adjustment, and the second detection result represents not triggering parameter adjustment. It can be understood that when the detection result is the first detection result, this situation indicates that the current environment meets the calibration condition, so the subsequent parameter adjustment process can be continued; when the detection result is not the first detection result, this situation indicates that the current environment does not meet the calibration condition, and the subsequent parameter adjustment process will not be executed, and enters a waiting state until the calibration condition is met again to execute the parameter adjustment process.

[0080] The determination method of the detection result can be any suitable method. In some embodiments, a corresponding relationship between each driving information, each lane line information and each detection result can be established in advance, and according to the corresponding relationship, a detection result that is suitable for the current driving information and the current lane line information can be obtained.

[0081] In some embodiments, the current driving information and the current lane line information can be input into a pre-established rule base or a machine learning model to obtain the detection result.

[0082] In some embodiments, the step S12 comprises a step S121 and / or a step S122, wherein:

[0083] The step S121 comprises: in a case where the current driving information of the vehicle indicates that the vehicle is in a uniform straight driving state, and the current lane line information comprises a dashed lane, taking the first detection result as the detection result.

[0084] The step S122 comprises: in a case where the current driving information of the vehicle indicates that the vehicle is not in a uniform straight driving state, and / or the current lane line information does not comprise a dashed lane, taking the second detection result as the detection result; wherein the second detection result indicates that the parameter adjustment is not triggered.

[0085] Here, the current driving information is used to determine whether the vehicle is in a stable driving state. The current driving information usually comprises a steering angle, an acceleration, etc. The acceleration can comprise, but is not limited to, a lateral acceleration, a longitudinal acceleration, etc. In implementation, when the steering angle is approximately zero, the lateral acceleration is approximately zero, and the longitudinal acceleration is approximately zero, it indicates that the vehicle is in a uniform straight driving state. At this time, the motion trajectory of the vehicle is stable, which is conducive to accurate parameter calibration optimization.

[0086] The dashed lane refers to a dashed lane line. The dashed lane can refer to a current lane and / or a side lane. The dashed lane line comprises a plurality of dashed segments. The length of the dashed segment is fixed, and the interval between two dashed segments is also fixed. For example, the length of a white dashed segment on a highway is 6 meters, and the interval between two white dashed segments is 9 meters; the length of a white dashed segment on an urban road is 2 meters, and the interval between two white dashed segments is 4 meters.

[0087] In implementation, if the vehicle is in a uniform straight driving state and the surrounding environment comprises a dashed lane, the environmental condition is suitable for performing the external parameter optimization, and therefore the first detection result is output. If the vehicle is not in a uniform driving state (such as turning, accelerating, or decelerating), or the dashed lane is not identified, it is considered that the current condition is not sufficient to support accurate calibration calculation, and therefore the second detection result is output, and the parameter adjustment is not performed.

[0088] In the embodiments of the present application, by jointly judging the driving state of the vehicle and the type of the lane line, it is determined whether to trigger the parameter adjustment, so that the adjustment is triggered only under specific conditions, thereby avoiding false adjustment or invalid adjustment in an unstable driving environment, and improving the effectiveness and robustness of the adjustment.

[0089] In implementation, step S11 can be performed before step S12, or step S12 can be performed before step S11, or steps S11 and S12 can be performed simultaneously.

[0090] Step S13, in a case where the detection result is a first detection result, determining, according to a target tuning manner, current adjustment information of the first surround-view camera based on a target image containing a dashed lane collected by the first surround-view camera and last adjustment information of the first surround-view camera; wherein the first detection result represents a trigger parameter adjustment; the first surround-view camera includes at least one of a front surround-view camera installed at a front of a vehicle body, a rear surround-view camera installed at a rear of the vehicle body, a left surround-view camera installed at a left side of the vehicle body, and a right surround-view camera installed at a right side of the vehicle body; the last adjustment information includes a last pitch angle, a last yaw angle, and a last roll angle.

[0091] Here, the parameters of the surround-view camera can include but are not limited to external parameters and internal parameters. The external parameters refer to the rotation angles of the surround-view camera relative to the vehicle body coordinate system, which can include but are not limited to a pitch angle (Pitch), a yaw angle (Yaw), and a roll angle (Roll). The Pitch describes the up-down inclination angle of the surround-view camera. The Yaw describes the horizontal rotation angle of the surround-view camera. The Roll describes the rotation angle of the surround-view camera around its optical axis. These angles directly affect the perspective effect of the image collected by the surround-view camera and the accuracy of image stitching.

[0092] The target tuning manner determines the adjustment object (i.e., the first surround-view camera) and the adjustment sequence. Different tuning manners correspond to different adjustment objects and / or adjustment sequences.

[0093] The first surround-view camera refers to the surround-view camera participating in this parameter adjustment. The number of the first surround-view camera can be at least one, and usually can be part or all of the four directions (front, rear, left, and right). Since the dependence on the surround-view camera is different in different road scenes, each adjustment can be performed only on part of the surround-view cameras, rather than on all surround-view cameras at one time.

[0094] The target image includes at least a dashed lane, which can be the current lane and / or the adjacent lane. The target image can be an image collected at the current time, and the acquisition manner of the target image can be any suitable manner. In some embodiments, each surround-view camera can collect images at a certain frame rate (such as 10 FPS) in a timely manner, and then the target image can be obtained from the first surround-view camera. In some embodiments, an image collection instruction can be sent to the first surround-view camera to enable the first surround-view camera to collect the target image based on the image collection instruction.

[0095] The last adjustment information refers to the extrinsic parameter information obtained in the last adjustment process, i.e., the last pitch angle, the last heading angle, and the last roll angle. The last adjustment information is used as a basic reference for the current adjustment, so as to calculate more accurate new adjustment information in the current adjustment. The manner of obtaining the last adjustment information can be any suitable manner. In some embodiments, the last adjustment information can be read from a configuration file or a database. In implementation, the adjustment information of each time can be stored in the configuration file or the database. In some embodiments, the last adjustment information sent by other devices can be received. It can be understood that for the first adjustment, the last adjustment information can be initial adjustment information, which can be zero or preset adjustment information.

[0096] The current adjustment information can include, but is not limited to, at least one of the current pitch angle, the current heading angle, and the current roll angle. The manner of determining the current adjustment information can be any suitable manner. In some embodiments, the target image and the last adjustment information can be input into a prediction model, and the current adjustment information can be obtained. The prediction model can be any suitable neural network model capable of achieving the function. In some embodiments, feature extraction can be performed on the lane line in the target image to obtain lane line feature points, the current deviation information can be determined according to the lane line feature points, and finally the current adjustment information can be determined according to the current deviation information and the last adjustment information.

[0097] Step S14: adjusting the extrinsic parameter of the first surround-view camera according to the current adjustment information of the first surround-view camera.

[0098] Here, the adjustment process of the extrinsic parameter is to apply the current adjustment information to the hardware or software configuration of the first surround-view camera, so that the extrinsic parameter of the first surround-view camera changes to improve the image acquisition effect. The specific manner of adjustment can be to change the physical angle of the surround-view camera through a motor, or to simulate the effect of angle change through a software algorithm.

[0099] In implementation, when the number of the first surround-view cameras is at least two, the extrinsic parameters of the first surround-view cameras are adjusted in sequence according to the adjustment order. For example, when the first surround-view cameras include four surround-view cameras, i.e., a front surround-view camera, a rear surround-view camera, a left surround-view camera, and a right surround-view camera, and the adjustment order is: the rear surround-view camera, the right surround-view camera, the left surround-view camera, and the front surround-view camera, the extrinsic parameters of the first surround-view cameras are adjusted in sequence according to the adjustment order, i.e., the current adjustment information of the first surround-view camera can be determined according to the target image collected by the first surround-view camera and the last adjustment information of the first surround-view camera, and then the extrinsic parameter of the first surround-view camera is adjusted according to the current adjustment information of the first surround-view camera.

[0100] In the embodiments of the present application, first, the target optimization mode is determined according to the road information, which can dynamically change the adjustment strategy according to different road conditions, improving the flexibility and adaptability of the adjustment; second, whether to trigger parameter adjustment is comprehensively determined according to the current driving information and lane line information, so as to ensure that the adjustment process is started under appropriate conditions, improving the stability and reliability of the adjustment; finally, the adjustment is realized based on historical adjustment information and current image features, which realizes dynamic parameter adjustment without physical target, so as to effectively cope with the problem of external parameter offset caused by mechanical deformation in long-term use, avoids the shortcomings of traditional calibration relying on fixed scene or manual intervention, improves the accuracy and efficiency of the adjustment, thereby reducing the re-calibration process caused by panoramic stitching effect or automatic parking performance problems, and further reducing the maintenance cost.

[0101] In some embodiments, the step S13 includes steps S131 to S134, wherein:

[0102] In step S131, the first surround view camera is determined from the multiple surround view cameras according to the target optimization mode; wherein the multiple surround view cameras include a front surround view camera, a rear surround view camera, a left surround view camera and a right surround view camera.

[0103] Here, the front surround view camera refers to a surround view camera installed on the front of the vehicle, which is used to capture image information of the environment in front of the vehicle, and is particularly suitable for identifying front lane lines and road boundaries. The front surround view camera usually has a wide field of view angle to cover a wider range of view.

[0104] The rear surround view camera refers to a surround view camera installed on the rear of the vehicle, which is used to capture image information of the environment behind the vehicle. The rear surround view camera also has a wide field of view angle, which plays an important role in identifying rear lane lines and providing panoramic assistance in parking scenarios.

[0105] The left surround view camera refers to a surround view camera installed on the left side of the vehicle, which is used to capture image information of the environment on the left side of the vehicle, and can effectively capture side lane lines, especially when changing lanes or diagonally parking, the left surround view camera plays an important role.

[0106] The right surround view camera refers to a surround view camera installed on the right side of the vehicle, which is used to capture image information of the environment on the right side of the vehicle. The right surround view camera and the left surround view camera together construct a complete side view and improve the stitching accuracy of the panoramic image through cooperative work.

[0107] The target tuning mode determines the adjustment object and the adjustment order. In implementation, the first surround-view camera is selected according to the target tuning mode. For example, when the target tuning mode is the second tuning mode, the rear surround-view camera is first selected as the first surround-view camera, then the right surround-view camera is selected as the first surround-view camera, then the left surround-view camera is selected as the first surround-view camera, and finally the front surround-view camera is selected as the first surround-view camera.

[0108] In step S132, lane line feature points are determined based on the target image containing the dashed lane collected by the first surround-view camera.

[0109] Here, the lane line feature points are key geometric points extracted from the target image by an image processing algorithm, and are used to describe the shape and position of the lane line. The lane line feature points can include but are not limited to the endpoints of the dashed segment, the starting point of the lane line, the ending point of the lane line, etc.

[0110] In some embodiments, the target image can be pre-processed (such as grayscale, denoising, edge detection), and then the lane line feature points are obtained from the pre-processed target image.

[0111] The accuracy of the lane line feature points directly affects the effect of subsequent calibration optimization. Therefore, in some embodiments, a multi-frame image fusion method can be used to smooth the lane line feature points to reduce errors caused by single-frame image noise or occlusion.

[0112] The lane line feature points can be obtained in any suitable manner. In some embodiments, the lane line feature points can be extracted by using Hough transform, Canny edge detection, etc. In some embodiments, the lane line feature points can be extracted by a deep learning model.

[0113] In step S133, the current deviation information of the first surround-view camera is determined based on the lane line feature points.

[0114] Here, the current deviation information is used to analyze the difference between the actual value and the ideal value. The current deviation information can include, but is not limited to, at least one of pitch angle deviation information, heading angle deviation information, roll angle deviation information, and the like. The pitch angle deviation information refers to the angular offset of the first surround-view camera in the vertical direction due to factors such as changes in installation position or vehicle vibration. The pitch angle deviation information will affect the vertical distribution of the lane line in the panoramic image, thereby affecting the parallelism of the lane line in the inverse perspective mapping (IPM). In some embodiments, the pitch angle deviation information can be determined by comparing the vertical distance between the vanishing point of the lane line in the image and the reference vanishing point. The heading angle deviation information refers to the angular offset of the first surround-view camera in the horizontal direction, i.e., the rotational angular deviation of the first surround-view camera relative to the forward direction of the vehicle. The heading angle deviation information will cause the horizontal distribution of the lane line in the panoramic image to be misaligned, thereby affecting the continuity of the image stitching of the front and rear cameras. In some embodiments, the heading angle deviation information can be determined by comparing the horizontal distance between the vanishing point of the lane line and the reference vanishing point. The roll angle deviation information refers to a parameter that describes the degree of rotation of the first surround-view camera about the X-axis relative to the horizontal plane, which is used to measure whether the first surround-view camera is tilted due to vehicle bumps or installation errors. The roll angle deviation information is usually represented by an angle value, a positive value indicating a right tilt and a negative value indicating a left tilt. The roll angle deviation information is one of the important indicators for evaluating the quality of panoramic image stitching.

[0115] The determination method of the current deviation information can be any suitable method. In some embodiments, a corresponding relationship between each lane line feature point and each deviation information can be established in advance, and according to the corresponding relationship, the current deviation information adapted to the lane line feature point can be obtained. In some embodiments, the lane line feature point can be input into a pre-established neural network model, and the current deviation information can be obtained. The neural network model can be any suitable model that can achieve this function. In some embodiments, the current vanishing point can be determined according to the lane line feature point, the current deviation vector can be determined according to the current vanishing point and the reference vanishing point, and finally the pitch angle deviation information and the heading angle deviation information can be determined according to the current deviation vector. In some embodiments, the slope of the target line segment can be determined based on the lane line feature point, and the roll angle deviation information can be determined according to the slope of the target line segment.

[0116] In step S134, the current adjustment information of the first surround-view camera is determined based on the current deviation information of the first surround-view camera and the last adjustment information of the first surround-view camera.

[0117] Here, the current adjustment information is used to update the calibration parameters of the first surround-view camera, so that the calibration parameters of the first surround-view camera are closer to the true value. The last adjustment information refers to a calibration result before this adjustment. There is a progressive relationship between the current adjustment information and the last adjustment information. Each adjustment is based on the last result for fine-tuning, each adjustment further approaches the optimal solution, while each adjustment avoids the instability problem caused by over-adjustment. In implementation, the current deviation information can be used to determine whether further adjustment is needed to form an iterative optimization process, which can ensure that the calibration accuracy is maintained at a high level in complex or dynamic driving environments.

[0118] The current adjustment information can be a vector, i.e., including the current adjustment direction and the current adjustment amount. The determination method of the current adjustment information can be any suitable method.

[0119] In some embodiments, a corresponding relationship between each deviation information, each last adjustment information and each current adjustment information can be established in advance, and according to the corresponding relationship, the current adjustment information that is adapted to the current deviation information and the last adjustment information can be obtained. It can be understood that the corresponding relationship established by different surround-view cameras can be the same or different.

[0120] In some embodiments, the current adjustment information can be determined based on the sum of the last adjustment information and the current deviation information. For example, the sum or the weighted sum is taken as the current adjustment amount.

[0121] In the embodiments of the present application, by selecting a suitable camera, extracting lane line key points, calculating deviations and combining historical adjustment data, a closed-loop feedback mechanism is constructed to realize precise control of the current adjustment information, thereby improving the convergence speed and stability of the adjustment, avoiding over-adjustment, insufficient adjustment and other problems, and ensuring that the adjustment accuracy is maintained at a high level under different driving conditions.

[0122] In some embodiments, the current deviation information includes pitch angle deviation information and heading angle deviation information; and the step S133 includes steps S1331-S1334, wherein:

[0123] In step S1331, based on the lane line feature points, current vanishing points of at least two lane lines in the target image are determined.

[0124] Here, the vanishing point refers to the point where multiple parallel lines intersect in an image. Lane lines are usually parallel lines, so the intersection of lane lines in an image can be used as a vanishing point. The position of the vanishing point reflects the perspective characteristics of the first surround view camera, and the position of the vanishing point is closely related to the rotation angle of the first surround view camera. When the first surround view camera has a pitch angle or a heading angle deviation, the vanishing point will deviate from the reference vanishing point. Therefore, by detecting the position of the vanishing point, the rotation state of the first surround view camera can be indirectly inferred.

[0125] The determination method of the vanishing point can be any suitable method. In some embodiments, the straight line equations of the two lane lines of the current lane can be fitted, and the intersection of the two lane lines can be taken as the current vanishing point. In some embodiments, in order to avoid the intersection of the two lane lines being deviated due to errors in the parameters of the straight line equations, the lane lines of the adjacent lane are usually introduced for fitting of multiple lane lines, so that the intersection of these lane lines is taken as the current vanishing point.

[0126] In some embodiments, the lane line feature points in multiple frames of images can be fused to reduce the influence of noise in a single frame of image on the noise source, thereby improving the accuracy of the vanishing point calculation.

[0127] In step S1332, a current deviation vector is determined based on the current vanishing point and the reference vanishing point; the current deviation vector includes a current lateral deviation vector and a current longitudinal deviation vector, and the reference vanishing point is determined based on the parameters of the first surround view camera.

[0128] Here, the deviation vector is used to quantify the relative deviation degree between the vanishing point in the image and the reference vanishing point (or the theoretical vanishing point). The reference vanishing point is the position where the lane line vanishing point should be in an ideal case. In implementation, it can be calculated according to the intrinsic parameters and the current extrinsic parameters of the first surround view camera. For example, by substituting the current rotation matrix (i.e., the current extrinsic parameter) into the camera projection model, the reference vanishing point of the current vanishing point in the pixel coordinate system can be obtained. The camera projection model is a mathematical model in computer vision and graphics that describes the mapping relationship from three-dimensional space to two-dimensional image plane. Its core meaning is to simulate the imaging process of a real camera through mathematical methods, and to convert the object points (3D coordinates) in the real world into pixel points (2D coordinates) on the image.

[0129] The lateral deviation vector refers to the distance vector between the actual vanishing point and the reference vanishing point in the horizontal direction of the image, which reflects the heading angle deviation of the first surround view camera.

[0130] The longitudinal deviation vector is the distance vector between the vanishing point and the reference vanishing point in the vertical direction of the image, which reflects the pitch angle deviation of the first surround view camera.

[0131] The deviation vector is a vector, that is, it includes a deviation direction and a deviation value. The determination manner of the current deviation vector can be any suitable manner. In some embodiments, a correspondence between each actual vanishing point, each reference vanishing point and each deviation vector can be established in advance, and according to the correspondence, the current deviation vector that is adapted to the current vanishing point and the reference vanishing point can be obtained. In some embodiments, mathematical operations can be performed on the current vanishing point and the reference vanishing point to obtain the current deviation vector.

[0132] In step S1333, the pitch angle deviation information is determined based on the current lateral deviation vector.

[0133] Here, the pitch angle deviation information can include a pitch angle deviation value, a pitch angle deviation direction, etc. Specifically, the larger the lateral deviation vector, the more significant the heading angle deviation.

[0134] The determination manner of the pitch angle deviation information can be any suitable manner. In some embodiments, a correspondence between each lateral deviation vector and each pitch angle deviation information can be established in advance, and according to the correspondence, the pitch angle deviation information adapted to the current lateral deviation vector can be obtained. For example, a pitch angle correction decision table shown in Table 1 below is established in advance, and according to the pitch angle correction decision table, the pitch angle deviation information adapted to the current lateral deviation vector can be obtained, that is, the value of the current lateral deviation vector is a pixel value deviation, the pitch angle deviation value is an adjustment angle size, and the pitch angle deviation direction is an adjustment angle sign.

[0135] Table 1 Pitch angle correction decision table

[0136]

[0137] In some embodiments, the current lateral deviation vector can be input into a pre-set rule base or model, and the pitch angle deviation information can be obtained. The model can be any suitable neural network model or mathematical conversion formula that can realize the function.

[0138] In implementation, by establishing a mapping relationship between the lateral deviation vector and the pitch angle, the pitch angle of the first surround-view camera can be automatically adjusted, and the extrinsic matrix of the first surround-view camera can be returned to the best state.

[0139] In step S1334, the heading angle deviation information is determined based on the current longitudinal deviation vector.

[0140] Here, the heading angle deviation information can include a heading angle deviation value, a heading angle deviation direction, etc. Specifically, the larger the longitudinal deviation vector, the more obvious the pitch angle deviation.

[0141] The determination manner of the heading angle deviation information can be any suitable manner. In some embodiments, a corresponding relationship between each longitudinal deviation vector and each heading angle deviation information can be established in advance, and the heading angle deviation information adapted to the current longitudinal deviation vector can be obtained according to the corresponding relationship. For example, a heading angle correction decision table shown in Table 2 below can be established in advance, and the heading angle deviation information adapted to the current longitudinal deviation vector can be obtained according to the heading angle correction decision table, that is, the value of the current longitudinal deviation vector is a pixel value deviation, the value of the heading angle deviation is an adjustment angle size, and the direction of the heading angle deviation is an adjustment angle sign.

[0142] Table 2 Heading angle correction decision table

[0143]

[0144] In some embodiments, the current longitudinal deviation vector can be input into a preset rule base or model, and the heading angle deviation information can be obtained. The model can be any suitable neural network model or mathematical conversion formula capable of achieving the function.

[0145] In implementation, by establishing the mapping relationship between the longitudinal deviation vector and the heading angle, the heading angle of the first surround-view camera can be automatically adjusted, and the extrinsic matrix of the first surround-view camera can be returned to the best state.

[0146] In the embodiments of the present application, by analyzing the relative position relationship between the lane line vanishing point and the image center, the deviation values of the pitch angle and the heading angle are derived, which is suitable for adaptive calibration requirements under various road conditions, and the adjustment accuracy is improved.

[0147] In some embodiments, the current deviation information includes the roll angle deviation information; and the step S133 includes a step S1335 and a step S1336, wherein:

[0148] The step S1335 includes determining a slope of a target line segment based on the lane line feature point; and the target line segment is composed of an end point of two dashed line segments in a dashed lane included in the target image.

[0149] Here, the target line segment is a straight line segment composed of end points of two dashed line segments extracted from the image, which is used to assist in determining the roll angle deviation information. For example, in a dashed lane, a plurality of intermittent line segments are usually included, each line segment has a start point and an end point, and the line segment is referred to as a dashed line segment.

[0150] The target line segment is a line segment taking the start point or the end point of the two adjacent dashed line segments as two end points of the target line segment, or a representative line segment selected according to a specific rule. The number of the target line segments can include at least one. For example, in the case where the target image includes two complete dashed line segments, the target line segment can be composed of the start points of the two dashed line segments and / or the end points of the two dashed line segments. For another example, in the case where the target image includes two incomplete dashed line segments, the target line segment can be composed of the start points of the two dashed line segments or the end points of the two dashed line segments.

[0151] The slope of the target line segment reflects the degree of inclination of the target line segment in the pixel coordinate system, and the calculation of the slope is the basis for subsequent judgment of the roll angle deviation information. By analyzing the slope distribution of the target line segment, the abnormal inclination can be more accurately identified.

[0152] In actual implementation, there is a close relationship between the target line segment and the roll angle deviation information. Since the roll angle deviation information will cause the change of the inclination direction of the lane line in the image, the size and direction of the roll angle deviation information can be indirectly derived by analyzing the slope change of the target line segment, so as to realize the non-contact calibration mode.

[0153] In step S1336, in the case where the slope of the target line segment is not equal to zero, the roll angle deviation information is determined based on the deviation angle between the target line segment and the lower edge of the target image.

[0154] Here, the lower edge of the target image refers to the horizontal boundary line at the bottom of the image, which usually corresponds to the ground plane below the wheels.

[0155] The deviation angle is the included angle between the target line segment and the lower edge of the target image, which is used to reflect the degree of the roll angle deviation information. When the target line segment is parallel to the lower edge of the target image, the deviation angle is zero, indicating that the first surround-view camera is in an ideal state; when the target line segment is inclined, the deviation angle increases, indicating that there is roll angle deviation information.

[0156] The roll angle deviation information can include a roll angle deviation value, a roll angle deviation direction, etc. Specifically, the larger the deviation angle, the more obvious the roll angle deviation.

[0157] The determination manner of the roll angle deviation information can be any suitable manner. In some embodiments, a corresponding relationship between each deviation angle and each roll angle deviation information can be established in advance, and according to the corresponding relationship, the roll angle deviation information matched with the deviation angle can be obtained.

[0158] For example, a roll angle correction decision table shown in Table 3 below is established in advance, and according to the roll angle correction decision table, the roll angle deviation information adapted to the deviation angle can be obtained, that is, the deviation angle is the slope difference, the roll angle deviation value is the adjustment angle size, and the roll angle deviation direction is the adjustment angle sign.

[0159] Table 3 Roll angle correction decision table

[0160]

[0161] In some embodiments, the deviation angle can be input into a preset rule base or model to obtain the roll angle deviation information. The model can be any suitable neural network model or mathematical conversion formula that can achieve this function.

[0162] In the embodiments of the present application, by analyzing the included angle between the auxiliary line formed by the lane line endpoints and the bottom edge of the image, the roll angle deviation situation is evaluated, thereby further expanding the parameter adjustment dimension, making the adjustment not limited to the pitch direction and the heading direction, but also covering the roll direction, and comprehensively improving the calibration accuracy. At the same time, it can also avoid the problem of relying on physical calibration board or fixed scene in related technologies, so as to adapt to the dynamic changes in the vehicle driving process, improve the real-time and stability of the calibration, and then effectively cope with the problem of external parameter deviation caused by mechanical deformation during long-term use of the vehicle, and reduce the after-sales maintenance cost.

[0163] In some embodiments, in the case where the target tuning mode is the first tuning mode, the adjustment method further includes steps S151 to S153, wherein:

[0164] Step S151, generating a first inverse perspective transformation image based on the image collected by the second surround view camera; wherein the second surround view camera includes a front surround view camera and a rear surround view camera.

[0165] Here, after adjusting the external parameters of the surround view camera, the adjusted surround view camera can be rechecked and verified to verify the accuracy and reliability of the adjustment. The first tuning mode includes adjusting the front and rear surround view cameras first and then adjusting the left and right surround view cameras. Then, after adjusting the front and rear surround view cameras, the front and rear surround view cameras can be rechecked and verified first, and then the left and right surround view cameras can be adjusted after the verification is passed.

[0166] The second surround view camera refers to a combination of cameras used to collect images of the environment around the vehicle, specifically including a front surround view camera and a rear surround view camera. The front surround view camera is used to capture the front road situation, and the rear surround view camera is used to capture the rear road situation. The image data of the front surround view camera and the rear surround view camera are used for synthesis to generate more complete road scene information.

[0167] The process of generating an IPM map is to convert a two-dimensional image into a bird's eye view form, thereby facilitating the analysis of geometric features such as lane lines. The generation process of the IPM map is based on known camera parameters and a projection model, and the points in the image are back-projected to a virtual bird's eye plane to form an image representation from a top-down perspective. The generation process of the IPM map helps to eliminate perspective distortion, making features such as lane lines more intuitive and easier to analyze in the IPM map. It can be understood that the first IPM map is a semi-panoramic IPM map.

[0168] The generation of the IPM map can be in any suitable manner. In some embodiments, the image captured by the front surround view camera and the image captured by the rear surround view camera can be input into a preset generation model, and the first IPM map can be obtained. The generation model can be any suitable neural network model or mathematical model that can achieve this function.

[0169] In some embodiments, the pixel points in the image can be projected into three-dimensional space points by the following formula (1-1), and the IPM map can be generated according to the three-dimensional space points, i.e.:

[0170] (1-1);

[0171] wherein, is a pixel point in the image, is an intrinsic matrix of the surround view camera, is a rotation matrix of the surround view camera, is a translation matrix of the surround view camera.

[0172] In step S152, the lane lines in the first inverse perspective transformation map are subjected to geometric constraint verification to obtain a parameter verification result of the second surround view camera. The geometric constraint verification includes at least one of the following: lane line parallel line constraint verification, lane line equal length constraint verification, and lane line endpoint connecting line parallel constraint. The lane line parallel line constraint verification is used to verify whether the same lane line captured by the two surround view cameras is parallel. The lane line equal length constraint verification is used to verify whether the connecting line between the two endpoints of the two dashed line segments captured by the two surround view cameras is equal. The lane line endpoint connecting line parallel constraint is used to verify whether the connecting line between the two endpoints of the two dashed line segments captured by the two surround view cameras is parallel.

[0173] Here, the geometric constraint verification is a method of judging whether the surround view camera parameters are reasonable by checking the geometric characteristics of the lane lines in the image.

[0174] The lane line parallel line constraint verification aims to ensure that the same lane line captured by the two surround view cameras remains parallel in the first IPM map, which is an important basis for judging whether the rotation angle (such as the heading angle) of the surround view camera is accurate. If the two sets of lane lines are not parallel, it indicates that there is an angle deviation, which needs to be adjusted.

[0175] The lane line equal length constraint verification is an operation for confirming whether the lengths of two lines (i.e., respectively composed of the end points of two dashed line segments) are consistent, which helps to evaluate whether the lateral offset or roll angle of the surround view camera is abnormal. If the lengths of the two lines differ too much, it may indicate that the surround view camera is not symmetrically installed or the surround view camera has a roll angle problem.

[0176] The lane line end point line parallel constraint further verifies whether the two lines collected by the two surround view cameras are parallel, which helps to detect whether there is a relative displacement or rotation deviation between the surround view cameras.

[0177] Through the comprehensive judgment of these geometric constraints, the parameter verification result of the surround view camera can be obtained as the basis for subsequent adjustment. In some embodiments, multi-dimensional geometric constraint conditions can be used to comprehensively evaluate the rationality of the parameters of the surround view camera to improve the calibration accuracy, thereby effectively avoiding the misjudgment problem caused by insufficient single indicators.

[0178] The parameter verification result refers to the effectiveness conclusion of the calibration parameter after the geometric constraint verification, which is usually represented in the form of verification passed or verification failed. The parameter verification result can include but is not limited to a first parameter verification result, a second parameter verification result, etc. The first parameter verification result represents verification passed, and the second parameter verification result represents verification failed.

[0179] The determination method of the parameter verification result can be any suitable method. In some embodiments, when each kind of geometric constraint verification passes, the first parameter verification result is taken as the parameter verification result; when at least one kind of geometric constraint verification fails, the second parameter verification result is taken as the parameter verification result.

[0180] It can be understood that when the parameter verification result is the first parameter verification result, it indicates that the parameter after the current external parameter adjustment meets the geometric constraint condition and can be used for subsequent calibration process; when the parameter verification result is the second parameter verification result, it indicates that the external parameter optimization of the surround view camera does not reach the best, and the external parameter needs to be continuously adjusted until the parameter verification result is the first parameter verification result or the maximum number of adjustments is reached. The parameter verification result is a key basis for judging whether to enter the next operation.

[0181] Step S153, in the case where the parameter verification result of the second surround view camera represents verification passed, the external parameter of the third surround view camera is adjusted again; wherein the third surround view camera includes the left surround view camera and the right surround view camera.

[0182] Here, when the parameter verification of the second surround view camera is passed, it indicates that the image data collected by the second surround view camera and the generated first IPM map meet the geometric constraint conditions, so as to determine that the parameters of the second surround view camera are reasonable, and at this time, the next stage of adjustment process can be entered, that is, the optimization of the external parameters of the third surround view camera. The external parameter adjustment refers to the correction of the external parameters (such as the rotation matrix and the translation vector) so as to make the image collected by the surround view camera more matched with the actual road environment.

[0183] In the embodiments of the present application, the re-examination verification is performed through the inverse perspective transformation map and the geometric constraint, so as to ensure that the adjusted parameters meet the global constraint conditions, thereby introducing the global verification mechanism on the basis of the local adjustment, and improving the accuracy and reliability of the adjustment.

[0184] In some embodiments, after adjusting the external parameters of the third surround view camera, the adjustment method further includes steps S154 to S156, wherein:

[0185] Step S154, generating a second inverse perspective transformation map based on the image collected by the fourth surround view camera; wherein the fourth surround view camera includes a front surround view camera, a rear surround view camera, a left surround view camera and a right surround view camera.

[0186] Here, the fourth surround view camera refers to four surround view cameras installed on the vehicle, which are respectively located at the front, rear, left and right of the vehicle, and are used to collect image information of the surrounding environment of the vehicle. The four surround view cameras usually have wide-angle lenses and can cover a larger field of view, so as to generate a panoramic image by splicing.

[0187] The purpose of the second IPM map is to map the ground features under the camera view to a unified top-down coordinate system. The image processed by the second IPM map can more intuitively show the relationship between the vehicle and the surrounding road, and is convenient for lane line detection, external parameter correction and global optimization operation. The generation of the second IPM map depends on the current camera internal and external parameters and the known road model, so it needs to be regenerated after each external parameter adjustment to ensure the consistency and accuracy of the calibration results.

[0188] The generation method of the second IPM map is similar to that of the first IPM map, and in the implementation, the generation of the second IPM map can refer to the generation of the first IPM map in the aforementioned step S151.

[0189] Step S155, performing geometric constraint verification on the lane lines in the second inverse perspective transformation map to obtain a parameter verification result of the third surround view camera.

[0190] Here, the geometric constraint verification refers to judging whether the current calibration parameter meets the expected condition by using the geometric characteristics (such as parallelism, equal length, horizontal degree of the endpoint connecting line, etc.) of the lane line in the second IPM graph. For example, in the second IPM graph, the left and right lane lines should remain parallel, and the endpoint connecting line of the dashed line segment should be parallel to the lower edge of the image. If these conditions are not met, it indicates that the current calibration parameter has deviation and needs to be further adjusted.

[0191] The geometric constraint verification of the second IPM graph is similar to the geometric constraint verification of the first IPM. In implementation, reference can be made to the geometric constraint verification of the first IPM in the aforementioned step S152.

[0192] Step S156, in the case where the parameter verification results of the third surround-view camera all represent that the verification is passed, fixing the extrinsic parameter of the fourth surround-view camera.

[0193] Here, when the parameter verification results of the third surround-view camera all pass, it indicates that the current extrinsic parameter adjustment has reached the optimal state and meets the geometric constraint condition. At this time, the extrinsic parameter of the fourth surround-view camera can be set as a fixed value to prevent the extrinsic parameter of the fourth surround-view camera from being offset again due to factors such as vibration and impact in the subsequent use process.

[0194] The process of fixing the extrinsic parameter of the fourth surround-view camera can be realized by software configuration update or hardware locking, etc., to ensure the long-term stability of the extrinsic parameter of the fourth surround-view camera. In this way, not only the calibration frequency can be reduced, but also the stitching accuracy of the panoramic image and the reliability of the automatic parking function can be improved. In addition, the fixed extrinsic parameter of the fourth surround-view camera also provides a stable input basis for other function modules (such as obstacle detection, path planning, etc.) that rely on calibration parameters, thereby improving the overall running efficiency and safety.

[0195] In some embodiments, in order to ensure the effectiveness and reliability of the extrinsic parameter, before fixing the extrinsic parameter of the fourth surround-view camera, the re-projection error can be counted based on the image collected by the fourth surround-view camera to evaluate the adjustment result (i.e., the calibration effect), and the extrinsic parameter of the fourth surround-view camera is fixed again in the case where the evaluation adjustment result is better. The adjustment result can be determined according to the stitching displacement information and the slope deviation information.

[0196] In the embodiments of the present application, the step-by-step re-examination and verification of front and back and then left and right is adopted in specific scenarios to ensure the consistency and rationality of the camera parameters after each round of adjustment, effectively eliminate errors caused by false detection or noise, thereby improving the calibration accuracy, and further enhancing the panoramic image stitching quality and the stability of the automatic driving system.

[0197] In some embodiments, in the case that the target tuning mode is the second tuning mode, the adjustment method further comprises steps S161 to S163, wherein:

[0198] Step S161, generating a third inverse perspective transformation map based on the image collected by the fourth surround view camera.

[0199] Here, the fourth surround view camera refers to a four-way surround view camera installed on the vehicle, respectively located in the front, rear, left and right directions of the vehicle, for collecting image information of the environment around the vehicle.

[0200] The third IPM map refers to a bird's eye view generated by the image collected by the fourth surround view camera after inverse perspective transformation processing, for further extracting road features such as lane lines. The third IPM map can intuitively show the road structure around the vehicle, facilitating subsequent lane line recognition and geometric verification. The generation method of the third IPM map is similar to that of the second IPM map. In implementation, the generation of the third IPM map can refer to the generation of the second IPM map in the aforementioned step S154.

[0201] Step S162, performing geometric constraint verification on the lane lines in the third inverse perspective transformation map to obtain a parameter verification result of the fourth surround view camera.

[0202] Here, the geometric constraint verification refers to verifying the lane lines in the third IPM map according to known geometric rules (such as lane line parallelism, horizontal degree of virtual line endpoint connecting line, length consistency, etc.), to determine whether the geometric constraint verification process meets the expected road features of the lane lines in the third IPM map.

[0203] The parameter verification result is a conclusion obtained after geometric constraint verification, for evaluating whether the extrinsic parameters (rotation angle and position) of the currently used fourth surround view camera are accurate. If the verification result shows that the lane lines meet all the constraint conditions, it is considered that the extrinsic parameters of the fourth surround view camera are valid; otherwise, there is a deviation and adjustment is needed.

[0204] The geometric constraint verification of the third IPM map is similar to that of the second IPM. In implementation, it can refer to the geometric constraint verification of the second IPM in the aforementioned step S155.

[0205] Step S163, fixing the extrinsic parameters of the fourth surround view camera in the case that the parameter verification result of the fourth surround view camera represents that the verification is passed.

[0206] Here, when the parameter verification result of the fourth surround view camera is passed, it means that the extrinsic parameter configuration of the fourth surround view camera is reasonable, and frequent adjustment of the extrinsic parameter configuration of the fourth surround view camera will not be performed, but the extrinsic parameter configuration of the fourth surround view camera will be fixed, to reduce unnecessary computational overhead and parameter fluctuation.

[0207] Fixing the extrinsic parameters of the fourth surround view camera means that in the subsequent running process, unless the optimization condition is triggered again (such as significant deviation after long-time driving), the extrinsic parameters of the fourth surround view camera will not be updated. Fixing the extrinsic parameters of the fourth surround view camera helps to maintain the stability of the calibration results and avoid false optimization of the fourth surround view camera due to short-term noise interference.

[0208] In some embodiments, in order to ensure the effectiveness and reliability of the extrinsic parameters, before fixing the extrinsic parameters of the fourth surround view camera, the re-projection error can be statistically analyzed based on the images collected by the fourth surround view camera to evaluate the adjustment result (i.e., the calibration effect), and the extrinsic parameters of the fourth surround view camera can be fixed again in the case that the adjustment result is good. The adjustment result can be determined according to the stitching misplacement information and the slope deviation information.

[0209] In the embodiments of the present application, a global adjustment strategy is adopted in a specific scenario to complete parameter optimization with stricter standards and ensure that all camera parameters remain consistent in high-precision scenarios.

[0210] In some embodiments, the adjustment method further includes steps S171 and S172, wherein:

[0211] Step S171, based on the images collected by the fourth surround view camera, determine the stitching misplacement information and the slope deviation information; wherein the stitching misplacement information includes whether each intersection point overlaps in the images collected by adjacent surround view cameras, and the intersection point is the intersection point between the lane line and the preset target ray; the slope deviation information includes the slope difference of the same lane line collected by adjacent surround view cameras.

[0212] Here, the target ray is a ray preset in advance. The number of target rays can be at least one. As shown in Figure 2 The target ray includes 4 rays, i.e., ray 211, ray 212, ray 213, and ray 214, and the current lane is a dashed lane, including multiple dashed segments, i.e., dashed segment 221, dashed segment 222, dashed segment 223, dashed segment 224, dashed segment 225, and dashed segment 226.

[0213] Stitching misplacement information refers to judging whether the intersection points of the lane lines and the target rays in the images of adjacent surround view cameras overlap or misplace in different images by detecting the intersection points. is used to evaluate whether there is a visual break or ghosting phenomenon in the panoramic stitching of multiple images. For example, if the intersection points of the same lane line displayed in two images are inconsistent, this phenomenon indicates that there is a stitching misplacement, which needs to be adjusted. can provide a quantitative basis for subsequent extrinsic parameter optimization and improve the consistency of stitching. With As the value increases, the consistency of image stitching decreases. For example, in images from left and right surround view cameras, when the distance between the intersection points of two lane lines exceeds a set threshold, The value increases accordingly, indicating a significant misalignment between the images, requiring adjustments to external parameters to improve the stitching effect.

[0214] In some implementations, the splicing misalignment information can be the misalignment information of a single intersection point, or it can be the sum of the misalignment information of all intersection points. Misalignment information refers to the pixel deviation of the intersection point in the two images.

[0215] In some implementations, the splicing misalignment information can be determined using the following formula (1-2):

[0216] (1-2);

[0217] in, Indicates the number of intersections; Indicates the first The pixel coordinates of the intersection points in an image; Indicates the first The pixel coordinates of the intersection point in another image.

[0218] For example, in Figure 2 In the image, ray 211 intersects with dashed line segment 221. Therefore, the difference between the coordinates of this intersection point in the image captured by the front surround view camera and the coordinates of this intersection point in the image captured by the left surround view camera can be calculated, and this difference can be used as the misalignment information of the intersection point.

[0219] Slope deviation information This refers to the difference in slope between the straight line equations fitted by adjacent surround-view cameras for the same lane line. This is used to measure the consistency of lane line geometry from different surround-view camera perspectives. If there is a significant difference in the slope of the same lane line captured by adjacent surround-view cameras, it indicates that there may be rotation or translation errors in the external parameters, requiring further correction. It helps to identify image distortion problems caused by extrinsic parameter offset, thereby improving calibration accuracy. The larger the value, the more inconsistent the geometric features of the lane lines are from different viewpoints. For example, if the slope of a lane line is 0.8 in the image captured by the left surround-view camera, but 0.5 in the image captured by the front surround-view camera, then... A value of 0.3 indicates a significant geometric deviation, requiring adjustment of the extrinsic parameters to reduce the error.

[0220] In some implementations, the It can be the slope deviation of a single lane line (including solid and / or dashed lines), or the sum of the slope deviations of all lane lines (including solid and / or dashed lines).

[0221] In some implementations, the slope deviation information can be determined using the following formulas (1-3):

[0222] (1-3);

[0223] in, Indicates the number of lane lines; Indicates the first The slope of a lane line in an image; Indicates the first The slope of the lane lines in another image.

[0224] For example, in Figure 2 In the process, for the dashed line segment 221, the difference between the slope of the dashed line segment 221 in the image captured by the front surround view camera and the slope of the dashed line segment 221 in the image captured by the left surround view camera can be calculated, and this difference can be used as the slope deviation of the dashed line segment 221.

[0225] and They are complementary. More attention is paid to the spatial alignment of the image, while More attention is paid to the geometric consistency of lane lines from different viewpoints. Therefore, and By combining evaluation metrics for the quality of panoramic image stitching, we can more comprehensively assess the status of extrinsic parameters and help identify and correct extrinsic parameter offset issues, thereby improving the overall calibration effect.

[0226] Step S172: Determine the adjustment result based on splicing error information and slope deviation information.

[0227] Here, the adjustment result is obtained by combining splicing misalignment information and slope deviation information. Specifically, the continuity and consistency of image splicing are judged based on splicing misalignment information, and the geometric matching degree of lane lines is judged based on slope deviation information, thereby determining the adjustment result. This adjustment result may include, but is not limited to, a first adjustment result and a second adjustment result. The first adjustment result indicates a better calibration effect, while the second adjustment result indicates a poorer calibration effect.

[0228] The determination manner of the adjustment result can be any suitable manner. In some embodiments, a correspondence between the splicing error information, the slope deviation information, and the adjustment result can be established in advance, and according to the correspondence, the adjustment result that is adapted to both the splicing error information and the slope deviation information can be obtained. In some embodiments, it can be determined whether the splicing error information and the slope deviation information are within the corresponding threshold range, and if both the splicing error information and the slope deviation information are within the corresponding threshold range, the first adjustment result is taken as the adjustment result; otherwise, if at least one of the splicing error information and the slope deviation information is not within the corresponding threshold range, the second adjustment result is taken as the adjustment result.

[0229] In the embodiments of the present application, the re-projection error is evaluated by the splicing misplacement and the slope difference statistical indicators, which is particularly suitable for road scenes lacking real-world coordinate system information, and realizes quantitative evaluation and continuous optimization of the adjustment effect.

[0230] The technical solutions provided by the embodiments of the present application will be described in detail below in combination with specific application scenarios.

[0231] The external parameter adjustment in the related art has the following defects:

[0232] Dependence on physical targets or fixed scenes, unable to adapt to dynamic changes during driving.

[0233] External parameter solidification leads to an increase in cumulative error after long-term use.

[0234] Vehicle motion state and multi-modal sensor data are not fused.

[0235] The present scheme provides a panoramic camera dynamic optimization method without physical targets and capable of real-time correction of external parameter offset, solving the technical bottleneck that the traditional calibration scheme cannot adapt to long-term mechanical deformation of the vehicle. The present scheme does not need to rely on site target equipment, and can automatically perform the optimization process of calibrating the external parameter angle when the vehicle motion characteristics and the lane line characteristics in the image meet the requirements during driving of the vehicle.

[0236] The present scheme includes the following parts:

[0237] I. Initialization setting

[0238] In the initialization phase, the chassis signals of the vehicle (corresponding to the aforementioned current driving information) and the camera internal and external parameter parameters need to be read. The present scheme relies on stable driving of the vehicle in the lane, and does not require it to drive in the center.

[0239] Steering angle reading: the steering angle can be used to determine whether the vehicle is in a straight driving state, to ensure that the calibration conditions are appropriate. If the steering angle is 0, the vehicle is in a straight driving state.

[0240] Acceleration reading: Determine if the vehicle is in a steady state by checking the acceleration. If both longitudinal and lateral acceleration signals are 0, the vehicle is in a steady state.

[0241] By checking if the steering angle is zero and the longitudinal and lateral acceleration are close to zero, we can determine if the vehicle is in a steady straight-line steady state. In this state, the lane line conforms to the single vanishing point model, and the vehicle body attitude meets the requirement of being level with the ground, providing a stable foundation for subsequent calibration optimization.

[0242] Parameter loading: Load the current camera intrinsic and extrinsic parameters for subsequent calibration optimization. In implementation, read the intrinsic matrix K, rotation matrix R, and translation vector t of the currently used camera from the system configuration file.

[0243] II. Lane line detection

[0244] During vehicle driving, the camera captures images and sends them to the optimization algorithm module. The optimization algorithm module processes the images and extracts lane line feature points. The lane lines of the current lane and the adjacent lane can be described using a single vanishing point (or single vanishing point) model.

[0245] The processing process of the optimization algorithm module is as follows:

[0246] 1) Data preprocessing: The image can be grayscale , Gaussian filtering , etc. Preprocessing operations can reduce image noise and improve lane line detection accuracy.

[0247] 2) Lane line detection: Use Canny edge detection and Hough transform to detect lane lines in the image and extract feature points. It can be a pre-set value, for example, 50. It can be a pre-set value, for example, 150. is the i-th feature point, and N represents the number of feature points. In this process, Canny is used for image binarization, and Hough transform is used to detect lane line data.

[0248] 3) Dotted line recognition: Detect the end points of the lane line and determine if it is a dotted line to provide accurate feature points for subsequent calibration optimization. If the lane line is a dotted line, extract its end points (ustart, vstart) and (uend, vend).

[0249] 4) Noise suppression: Use median filtering or Gaussian filtering Smooth the extracted lane line feature points to reduce the lane line endpoint detection error caused by noise. is a pre-set window size, for example, 3*3, 5*5, etc.

[0250] 5) Error reduction: Lane line feature points of multiple frames are fused , and lane line endpoints of multiple frames are averaged. The accuracy of lane line endpoint detection is improved, and the error caused by noise in a single frame image is reduced.

[0251] III. Vanishing point calculation and external parameter optimization

[0252] The vanishing point is calculated by the extracted lane line feature points, and the camera external parameters, i.e. pitch and yaw, are optimized. Whether it is a virtual-real lane line scene or a single lane scene, the pitch and yaw can be optimized.

[0253] Vanishing point calculation: fitting lane line equation , the intersection of the lane lines is obtained . Among them, u and v are the width and height of the image coordinate system, m is the slope in the image coordinate system, and c is the intercept. , . Among them, (m1, c1) is the slope and intercept of one lane line, and (m2, c2) is the slope and intercept of the other lane line. By calculating the vanishing point of the lane line in the image, the rotation angle of the camera is determined. As shown in Figure 3 , when the side lane is a dashed lane, the vanishing point 31 is calculated by fitting multiple lane line equations.

[0254] Deviation calculation: compare the vanishing point with the reference vanishing point to calculate the deviation vector , wherein . Among them, the reference vanishing point , is calculated according to the internal and external parameters of the camera. By calculating the deviation between the vanishing point and the reference vanishing point, the rotation angle of the camera is adjusted. For example, when the horizontal pixel coordinate of the vanishing point is less than , the yaw is adjusted to the left.

[0255] External parameter optimization: adjust the pitch and yaw of the camera according to the deviation. Among them, is the last pitch, is the pitch deviation information, is the last yaw, is the yaw deviation information. By adjusting the pitch and yaw of the camera, the external parameters are optimized. As shown in Figure 4As shown, according to the vanishing point 41 and the reference vanishing point 42, the horizontal pixel deviation and the vertical pixel deviation can be obtained, and by looking up table 1, the pitch angle deviation information adapted to the horizontal pixel deviation can be obtained, and by looking up table 2, the heading angle deviation information adapted to the vertical pixel deviation can be obtained.

[0256] IV. Roll angle adjustment

[0257] Adjust the roll angle by connecting the endpoints of the dashed lane line.

[0258] Endpoint connection: extract the endpoints of the dashed lane line, and calculate the slope of the auxiliary line (corresponding to the target line segment) described above. Wherein, (a , ) and (b , ) are two endpoints of an auxiliary line. Determine whether the auxiliary line is parallel to the lower edge of the image to determine whether the roll angle needs to be adjusted. If the slope of the auxiliary line is not zero, the roll angle needs to be adjusted, and the roll angle is adjusted according to the deviation . Wherein, is the last roll angle, is the roll angle deviation information. Adjust the roll angle so that the auxiliary line is parallel to the lower edge of the image. As Figure 5 shown, when the side lane is a solid lane, the vanishing point 51 is calculated by fitting a plurality of lane line equations, and the auxiliary line 52 is constructed to determine the roll angle deviation.

[0259] V. Recheck verification

[0260] After each adjustment of the external parameters, introduce the IPM graph for recheck verification.

[0261] 1) IPM: Project the points in the image back to the points in the three-dimensional space to generate a bird's eye view. In implementation, the points in the image can be converted to the points in the IPM graph by the foregoing formula (1-1).

[0262] 2) Geometric constraint verification

[0263] Lane line parallelism constraint: Calculate the slope difference of the lane lines, and determine whether it is less than the threshold to ensure that the lane lines are parallel in the IPM graph. is a set deviation threshold, for example, 0.01°. and are the slopes of two lane lines.

[0264] Lane line length constraint: Calculate the length difference of the lane lines, and determine whether it is less than the threshold to ensure that the lane lines are equal in length in the IPM graph. It is the set deviation threshold, for example, 2 pixels. and These are the lengths of the two lane lines, respectively.

[0265] Parallel constraint of lane endpoints: Calculate the slope difference of the lane endpoints. Determine if it is less than the threshold. This is to ensure that the lane line endpoints are parallel in the IPM diagram. It is the set deviation threshold, for example, 0.006. and These are the slopes of the two lines connecting them.

[0266] For example, such as Figure 6 As shown, dashed line segments 611 and 612 are captured by the front surround-view camera, dashed line segments 621 and 622 are captured by the left surround-view camera, dashed line segments 631 and 632 are captured by the right surround-view camera, and dashed line segments 641 and 642 are captured by the rear surround-view camera. Therefore:

[0267] Lane line parallelism constraints are achieved by calculating the slope differences between dashed line segments 611 and 621, 612 and 631, 622 and 641, and 632 and 642.

[0268] Lane length equalization constraints are performed by calculating whether the first length and the second length are equal, and / or whether the third length and the fourth length are equal.

[0269] Parallelism constraints on lane line endpoint connections are performed by calculating whether the first and second connecting lines are parallel, and / or whether the third and fourth connecting lines are parallel.

[0270] The first length can be one of the following: the length between the upper endpoint of dashed line segment 611 and the upper endpoint of dashed line segment 612, the length between the lower endpoint of dashed line segment 611 and the lower endpoint of dashed line segment 612, the length between the upper endpoint of dashed line segment 611 and the lower endpoint of dashed line segment 612, and the length between the lower endpoint of dashed line segment 611 and the upper endpoint of dashed line segment 612.

[0271] The second length can be one of the following: the length between the upper endpoint of dashed line segment 641 and the upper endpoint of dashed line segment 642, the length between the lower endpoint of dashed line segment 641 and the lower endpoint of dashed line segment 642, the length between the upper endpoint of dashed line segment 641 and the lower endpoint of dashed line segment 642, and the length between the lower endpoint of dashed line segment 641 and the upper endpoint of dashed line segment 642.

[0272] The third length can be one of the following: a length between the upper end point of the dashed segment 621 and the upper end point of the dashed segment 622, a length between the lower end point of the dashed segment 621 and the lower end point of the dashed segment 622, a length between the upper end point of the dashed segment 621 and the lower end point of the dashed segment 622, and a length between the lower end point of the dashed segment 621 and the upper end point of the dashed segment 622.

[0273] The fourth length can be one of the following: a length between the upper end point of the dashed segment 631 and the upper end point of the dashed segment 632, a length between the lower end point of the dashed segment 631 and the lower end point of the dashed segment 632, a length between the upper end point of the dashed segment 631 and the lower end point of the dashed segment 632, and a length between the lower end point of the dashed segment 631 and the upper end point of the dashed segment 632.

[0274] The first connecting line can be one of the following: a connecting line between the upper end point of the dashed segment 611 and the upper end point of the dashed segment 612, a connecting line between the lower end point of the dashed segment 611 and the lower end point of the dashed segment 612, a connecting line between the upper end point of the dashed segment 611 and the lower end point of the dashed segment 612, and a connecting line between the lower end point of the dashed segment 611 and the upper end point of the dashed segment 612.

[0275] The second connecting line can be one of the following: a connecting line between the upper end point of the dashed segment 641 and the upper end point of the dashed segment 642, a connecting line between the lower end point of the dashed segment 641 and the lower end point of the dashed segment 642, a connecting line between the upper end point of the dashed segment 641 and the lower end point of the dashed segment 642, and a connecting line between the lower end point of the dashed segment 641 and the upper end point of the dashed segment 642.

[0276] The third connecting line can be one of the following: a connecting line between the upper end point of the dashed segment 621 and the upper end point of the dashed segment 622, a connecting line between the lower end point of the dashed segment 621 and the lower end point of the dashed segment 622, a connecting line between the upper end point of the dashed segment 621 and the lower end point of the dashed segment 622, and a connecting line between the lower end point of the dashed segment 621 and the upper end point of the dashed segment 622.

[0277] The fourth connecting line can be one of the following: a connecting line between the upper end point of the dashed segment 631 and the upper end point of the dashed segment 632, a connecting line between the lower end point of the dashed segment 631 and the lower end point of the dashed segment 632, a connecting line between the upper end point of the dashed segment 631 and the lower end point of the dashed segment 632, and a connecting line between the lower end point of the dashed segment 631 and the upper end point of the dashed segment 632.

[0278] In some embodiments, in order to reduce the problem of irregular construction of rectangles caused by lane line endpoint recognition errors, the lane line endpoints of multiple frames of images can be fused Wherein, N represents the frame number of the image, and the error caused by single-frame image noise is reduced. When generating the IPM map, the lane is smoothed to ensure its consistency and continuity in the bird's-eye view.

[0279] Six, global optimization mode based on different scenarios

[0280] Considering the common vertical parking, the rear surround view camera has the highest accuracy requirement, while for horizontal parking and diagonal parking, the rear surround view camera works together with the left and right surround view cameras, and its accuracy requirement is higher than that of the front surround view camera. However, in a single lane scenario, the left and right surround view cameras are not conducive to independent optimization. Therefore, different global optimization modes are proposed for different road conditions.

[0281] 1) When the left and right side lanes are composed of dashed lanes: use the four common areas of the vehicle left front, left rear, right front and right rear to construct regular rectangles, check the splicing ghosting results (i.e. adjustment results), and in the case of poor splicing ghosting results, perform external parameter optimization of the surround view camera, and the optimization sequence is rear, right, left, and front; otherwise, no external parameter optimization is performed. For example, when the four rectangles are as shown in Figure 7 , external parameter optimization of the surround view camera is required; when the four rectangles are as shown in Figure 8 , no external parameter optimization of the surround view camera is required.

[0282] 2) In a common three-lane scenario: regular rectangles cannot be constructed on the left and right sides, therefore, a semi-panoramic IPM map is constructed by combining the front and rear camera images, the slopes of the lane lines on the left and right sides are determined, the external parameters of the front and rear cameras are adjusted according to the deviation, and then the external parameters of the left and right cameras are adjusted. If the roll angle of the right surround view camera is incorrect, the slope of the lane line collected by the right surround view camera will be misaligned with the lane line in the IPM map constructed by the front and rear surround view cameras. If the slope of the lane line collected by the right surround view camera is smaller than that collected by the front / rear surround view camera, the heading angle needs to be adjusted counterclockwise.

[0283] Seven, re-projection error statistics

[0284] After completing global optimization, the re-projection error needs to be counted to evaluate the calibration effect. A new statistical method is proposed for the re-projection error in a road scenario, i.e. using the splicing misalignment degree (see formula (1-2) above) and the slope difference of the lane line in the multi-camera image (see formula (1-3) above) for statistics.

[0285] Eight, application effect

[0286] Through the above steps, the scheme can correct the external parameter offset of the panoramic camera in real time, and solve the technical bottleneck that the traditional calibration scheme cannot adapt to long-term mechanical deformation of the vehicle. After optimization according to the application, the optimized calibration parameters are used for subsequent image processing and calibration, so as to ensure the calibration accuracy and stability of the vehicle in different road environments.

[0287] As Figure 9 shown, the adjustment method includes steps S301 to S315, wherein:

[0288] Step S301, judging whether the vehicle is in a straight driving state through steering angle information;

[0289] Step S302, judging whether the vehicle is in a uniform speed driving state through acceleration information;

[0290] Step S303, capturing images by the surround-view cameras;

[0291] Step S304, detecting lane line feature points in the images;

[0292] Step S305, adjusting the roll angle according to the constraint that the angle between the connecting line of the left and right virtual lane line endpoints and the lower edge of the image is about 90 degrees;

[0293] Step S306, adjusting the pitch angle and the heading angle based on the single vanishing point model and taking the lane line vanishing point as a constraint;

[0294] Step S307, generating an IPM image using new parameters after optimizing the external parameters of the front and rear surround-view cameras;

[0295] Step S308, verifying whether the lane lines in the images collected by the optimized front and rear surround-view cameras are smooth by using rectangular constraints and combining the images collected by the front and rear surround-view cameras;

[0296] Step S309, adjusting the roll angle of the left surround-view camera by combining the three images collected by the front, left and rear surround-view cameras;

[0297] In this way, the lane line width in the image is uniform, and the inner and outer edges of the lane line are parallel to the lower edge of the image. In implementation, for the left side vehicle, the left surround-view camera can also be adjusted in combination with multi-lane line information.

[0298] Step S310, adjusting the pitch angle and the heading angle of the left surround-view camera by combining the three images collected by the front, left and rear surround-view cameras;

[0299] Here, the fitted lane line is equal in width, parallel and continuous to the lane lines collected by the front and rear surround-view cameras.

[0300] Step S311, adjusting the roll angle of the right surround-view camera by combining the three images collected by the front, right and rear surround-view cameras;

[0301] In this way, the lane line width in the image is uniform, and the inner and outer edges of the lane line are parallel to the lower edge of the image. In practice, for the right side lane, the multi-lane line information can also be combined for adjustment.

[0302] In step S312, the three images captured by the front-right and rear-looking surround-view cameras are combined, and the horizontal roll angle of the right surround-view camera is adjusted.

[0303] Here, the fitted lane line is made to have the same width, parallelism and continuity as the lane line captured by the front and rear surround-view cameras.

[0304] In step S313, the IPM image is generated using the new parameters after optimizing the left and right surround-view camera extrinsic parameters.

[0305] In step S314, the images captured by the four surround-view cameras are combined, and the lane line in the images captured by the left and right surround-view cameras after optimization is verified for smoothness using rectangular constraints.

[0306] In step S315, the corrected parameters are saved.

[0307] The present scheme realizes dynamic optimization of panoramic camera extrinsic parameters by combining vehicle motion characteristics and lane line characteristics in images. This scheme not only can correct the extrinsic parameter offset in real time, but also can adapt to various road environments, significantly improving the calibration accuracy and stability of vehicles in different driving scenarios. Through the above implementation steps, the technical bottleneck that traditional calibration methods cannot adapt to long-term mechanical deformation of vehicles can be effectively solved, providing a new solution for the field of autonomous driving and computer vision technology. At the same time, it can also reduce the process of re-calibration caused by panoramic stitching effect or automatic parking performance problems, thereby reducing maintenance costs.

[0308] Based on the above embodiment, the present application also provides a camera parameter adjustment device, Figure 10 The composition structure diagram of a camera parameter adjustment device provided by the present application is applied to a vehicle, as shown in Figure 10 The adjustment device 100 includes a first determination module 101, a second determination module 102, a third determination module 103, and an adjustment module 104, wherein:

[0309] The first determination module 101 is configured to determine a target optimization mode based on road information where the vehicle is located; wherein the target optimization mode includes one of the following: a first optimization mode, a second optimization mode, and the adjustment order of the second optimization mode is different from that of the first optimization mode.

[0310] The second determination module 102 is configured to determine a detection result based on current driving information and current lane line information of the vehicle; wherein the detection result represents whether to trigger parameter adjustment.

[0311] The third determination module 103 is configured to, when the detection result is a first detection result, determine current adjustment information of the first surround-view camera according to a target tuning manner based on a target image containing a dashed lane collected by the first surround-view camera and previous adjustment information of the first surround-view camera; the first detection result indicates that a parameter adjustment is triggered; the first surround-view camera includes at least one of a front surround-view camera installed at the front of the vehicle body, a rear surround-view camera installed at the rear of the vehicle body, a left surround-view camera installed at the left side of the vehicle body, and a right surround-view camera installed at the right side of the vehicle body; the previous adjustment information includes a previous pitch angle, a previous yaw angle, and a previous roll angle.

[0312] The adjustment module 104 is configured to adjust the external parameter of the first surround-view camera according to the current adjustment information of the first surround-view camera.

[0313] In some embodiments, the first determination module 101 is further configured to, when the road information indicates that the vehicle is in a single lane or at least one side of the vehicle is in a solid lane, take the first tuning manner as the target tuning manner; and when the road information indicates that both sides of the vehicle are in a dashed lane, take the second tuning manner as the target tuning manner.

[0314] In some embodiments, the second determination module 102 is further configured to, when the current driving information of the vehicle indicates that the vehicle is in a constant-speed straight driving state and the current lane line information includes a dashed lane, take the first detection result as the detection result; and when the current driving information of the vehicle indicates that the vehicle is not in a constant-speed straight driving state and / or the current lane line information does not include a dashed lane, take a second detection result as the detection result; the second detection result indicates that a parameter adjustment is not triggered.

[0315] In some embodiments, the third determination module 103 is further configured to, according to the target tuning manner, determine the first surround-view camera from a plurality of surround-view cameras; the plurality of surround-view cameras include the front surround-view camera, the rear surround-view camera, the left surround-view camera, and the right surround-view camera; determine lane line feature points based on the target image containing the dashed lane collected by the first surround-view camera; determine current deviation information of the first surround-view camera based on the lane line feature points; and determine the current adjustment information of the first surround-view camera based on the current deviation information of the first surround-view camera and the previous adjustment information of the first surround-view camera.

[0316] In some embodiments, the current deviation information comprises pitch angle deviation information and heading angle deviation information; the third determining module 43 is further configured to determine, based on the lane line feature points, current vanishing points of the at least two lane lines in the target image; determine, based on the current vanishing points and reference vanishing points, a current deviation vector; wherein the current deviation vector comprises a current lateral deviation vector and a current longitudinal deviation vector, and the reference vanishing points are determined based on parameters of the first surround-view camera; determine, based on the current lateral deviation vector, the pitch angle deviation information; and determine, based on the current longitudinal deviation vector, the heading angle deviation information.

[0317] In some embodiments, the current deviation information comprises roll angle deviation information; the third determining module 103 is further configured to determine, based on the lane line feature points, a slope of a target line segment; wherein the target line segment is composed of one end point of two dashed line segments in a dashed lane included in the target image; and in a case where the slope of the target line segment is not equal to zero, determine, based on a deviation angle between the target line segment and a lower edge of the target image, the roll angle deviation information.

[0318] In some embodiments, in a case where the target tuning mode is the first tuning mode, the adjusting apparatus further comprises a verification module configured to generate a first inverse perspective transformation map based on images captured by a second surround-view camera; wherein the second surround-view camera comprises a front surround-view camera and a rear surround-view camera; perform geometric constraint verification on lane lines in the first inverse perspective transformation map to obtain a parameter verification result of the second surround-view camera; wherein the geometric constraint verification comprises at least one of the following: lane line parallel line constraint verification, lane line equal length constraint verification, and lane line end point connecting line parallel constraint verification; the lane line parallel line constraint verification is used to verify whether lane lines on the same side captured by two surround-view cameras are parallel; the lane line equal length constraint verification is used to verify whether connecting lines between two end points of dashed line segments on left and right sides captured by two surround-view cameras are equal; and the lane line end point connecting line parallel constraint verification is used to verify whether the connecting lines between the two end points of the dashed line segments on the left and right sides captured by the two surround-view cameras are parallel; and in a case where the parameter verification result of the second surround-view camera indicates that the verification passes, adjust the extrinsic parameters of a third surround-view camera; wherein the third surround-view camera comprises a left surround-view camera and a right surround-view camera.

[0319] In some embodiments, after the extrinsic parameters of the third surround-view camera are adjusted, the verification module is further configured to generate a second inverse perspective transformation map based on images captured by a fourth surround-view camera; wherein the fourth surround-view camera comprises the front surround-view camera, the rear surround-view camera, the left surround-view camera, and the right surround-view camera; perform geometric constraint verification on lane lines in the second inverse perspective transformation map to obtain a parameter verification result of the third surround-view camera; and in a case where the parameter verification results of the third surround-view camera all indicate that the verification passes, fix the extrinsic parameters of the fourth surround-view camera.

[0320] In some embodiments, when the target optimization mode is the second optimization mode, the verification module is further configured to generate a third inverse perspective transformation image based on an image captured by the fourth surround-view camera; perform geometric constraint verification on lane lines in the third inverse perspective transformation image to obtain a parameter verification result of the fourth surround-view camera; and fix the extrinsic parameters of the fourth surround-view camera when the parameter verification result of the fourth surround-view camera indicates that the verification is passed.

[0321] In some embodiments, the adjustment device further includes a fourth determination module configured to determine stitching dislocation information and slope deviation information based on an image captured by the fourth surround-view camera; wherein the stitching dislocation information includes whether each intersection point is overlapped in images captured by adjacent surround-view cameras, and the intersection point is a point of intersection between a lane line and a preset target ray; the slope deviation information includes a slope difference of a same lane line captured by adjacent surround-view cameras; and the adjustment result is determined based on the stitching dislocation information and the slope deviation information.

[0322] The above device embodiments are similar to the descriptions of the above method embodiments, and have similar beneficial effects to the method embodiments. For technical details not disclosed in the device embodiments of the present application, please refer to the description of the method embodiments of the present application for understanding.

[0323] It should be noted that, in the embodiments of the present application, if the above method is implemented in the form of a software function module and sold or used as an independent product, it can also be stored in a computer readable storage medium. Based on such understanding, the technical solutions of the embodiments of the present application can be embodied in the form of a software product, which is stored in a storage medium and includes a plurality of instructions for causing an electronic device (which can be a personal computer, a server, or a network device, etc.) to execute all or part of the methods described in the embodiments of the present application. The aforementioned storage medium includes: a U disk, a mobile hard disk, a read-only memory (ROM), a magnetic disk or an optical disk, and various other media capable of storing program codes. Thus, the embodiments of the present application are not limited to any specific combination of hardware and software.

[0324] The present application also provides a vehicle including a memory and a processor, the memory storing a computer program capable of running on the processor, and the processor implements any of the above methods when executing the computer program.

[0325] The present application also provides a computer readable storage medium having a computer program stored thereon, and the computer program is executed by a processor to implement the above method. The computer readable storage medium can be transitory or non-transitory.

[0326] The present application also provides a computer program product, which comprises a computer program or instructions, and the computer program or instructions are executed by a processor to realize some or all steps in any of the above methods. The computer program product can be realized by hardware, software or a combination thereof. In an optional embodiment, the computer program product is embodied as a computer storage medium, and in another optional embodiment, the computer program product is embodied as a software product, such as a software development kit (SDK) and the like.

[0327] It should be noted that, Figure 11 A hardware entity diagram of an electronic device provided in the embodiments of the present application is shown in FIG. 1, which comprises a processor 111, a communication interface 112 and a memory 113. Figure 11 The hardware entity of the electronic device 110 comprises a processor 111, a communication interface 112 and a memory 113, wherein:

[0328] The processor 111 generally controls the overall operation of the electronic device 110.

[0329] The communication interface 112 can enable the electronic device to communicate with other terminals or servers through a network.

[0330] The memory 113 is configured to store instructions and applications executable by the processor 111, and can also cache data (for example, image data, audio data, voice communication data and video communication data) to be processed by the processor 111 and modules in the electronic device 110, which can be realized by a FLASH or a random access memory (RAM). The processor 111, the communication interface 112 and the memory 113 can transmit data through a bus 114.

[0331] It should be noted that: the above description of the storage medium and the device embodiments is similar to the description of the above method embodiments, and has similar beneficial effects as the method embodiments. For technical details not disclosed in the storage medium and device embodiments of the present application, please refer to the description of the method embodiments of the present application for understanding.

[0332] The above embodiments are only preferred embodiments of the present application, and the protection scope of the present application is not limited thereto. Any equivalent replacement or transformation of the present application based on the present application is within the protection scope of the present application.

Claims

1. A method for adjusting camera parameters, characterized in that, When applied to vehicles, the adjustment method includes: Based on the road information where the vehicle is located, a target optimization method is determined; wherein, the road information represents the lane line information of the vehicle, and the target optimization method includes one of the following: a first optimization method and a second optimization method, wherein the adjustment order of the second optimization method is different from the adjustment order of the first optimization method, and the adjustment order represents the adjustment priority of at least one surround view camera in the vehicle. Based on the vehicle's current driving information and current lane line information, a detection result is determined; wherein, the detection result indicates whether parameter adjustment is triggered; If the detection result is the first detection result, then according to the target optimization method, based on the target image containing the dashed lane captured by the first surround-view camera and the previous adjustment information of the first surround-view camera, the current adjustment information of the first surround-view camera is determined; wherein, the first detection result represents the trigger parameter adjustment; the first surround-view camera includes at least one of the following: a front surround-view camera installed at the front of the vehicle body, a rear surround-view camera installed at the rear of the vehicle body, a left surround-view camera installed on the left side of the vehicle body, and a right surround-view camera installed on the right side of the vehicle body; the previous adjustment information includes the previous pitch angle, the previous yaw angle, and the previous roll angle; Adjust the extrinsic parameters of the first surround view camera according to the current adjustment information of the first surround view camera.

2. The adjustment method according to claim 1, characterized in that, The step of determining the target optimization method based on the road information where the vehicle is located includes: When the road information indicates that the vehicle is in a single lane or that at least one of the adjacent lanes of the vehicle is a solid line lane, the first optimization method is taken as the target optimization method. When the road information indicates that both sides of the vehicle are dashed lanes, the second optimization method is used as the target optimization method.

3. The adjustment method according to claim 1, characterized in that, The determination of the detection result based on the vehicle's current driving information and current lane line information includes: If the current driving information of the vehicle indicates that the vehicle is in a constant speed straight-line driving state and the current lane line information includes a dashed lane, the first detection result shall be used as the detection result. If the current driving information of the vehicle indicates that the vehicle is not in a constant speed straight driving state, and / or the current lane line information does not include the dashed lane, the second detection result shall be used as the detection result; wherein, the second detection result indicates that parameter adjustment is not triggered.

4. The adjustment method according to claim 1, characterized in that, The step of determining the current adjustment information of the first surround-view camera according to the target optimization method, based on the target image containing the dashed lane captured by the first surround-view camera and the previous adjustment information of the first surround-view camera, includes: According to the target optimization method, the first surround view camera is determined from a plurality of surround view cameras; wherein, the plurality of surround view cameras includes the front surround view camera, the rear surround view camera, the left surround view camera, and the right surround view camera; Based on the target image containing the dashed lane captured by the first surround-view camera, lane line feature points are determined. Based on the lane line feature points, the current deviation information of the first surround view camera is determined; Based on the current deviation information of the first surround view camera and the previous adjustment information of the first surround view camera, the current adjustment information of the first surround view camera is determined.

5. The adjustment method according to claim 4, characterized in that, The current deviation information includes pitch angle deviation information and heading angle deviation information; Determining the current deviation information of the first surround-view camera based on the lane line feature points includes: Based on the lane line feature points, determine the current vanishing points of at least two lane lines in the target image; Based on the current vanishing point and the reference vanishing point, the current deviation vector is determined; wherein, the current deviation vector includes the current lateral deviation vector and the current longitudinal deviation vector, and the reference vanishing point is determined based on the parameters of the first surround-view camera; Based on the current lateral deviation vector, the pitch angle deviation information is determined; Based on the current longitudinal deviation vector, the heading angle deviation information is determined.

6. The adjustment method according to claim 4, characterized in that, The current deviation information includes roll angle deviation information; Determining the current deviation information of the first surround-view camera based on the lane line feature points includes: Based on the lane line feature points, the slope of the target line segment is determined; wherein, the target line segment is formed by one endpoint of two dashed line segments in the dashed lane included in the target image; If the slope of the target line segment is not equal to zero, the roll angle deviation information is determined based on the deviation angle between the target line segment and the lower edge of the target image.

7. The adjustment method according to claim 1, characterized in that, When the target optimization method is the first optimization method, the adjustment method further includes: A first inverse perspective transformation image is generated based on the images captured by the second surround-view camera; wherein, the second surround-view camera includes the front surround-view camera and the rear surround-view camera; Geometric constraint verification is performed on the lane lines in the first inverse perspective transformation image to obtain the parameter verification results of the second surround view camera; wherein, the geometric constraint verification includes at least one of the following: lane line parallel line constraint verification, lane line equal length constraint verification, and lane line endpoint connection parallel constraint. The lane line parallel line constraint verification is used to verify whether the same lane line captured by the two surround view cameras is parallel. The lane line equal length constraint verification is used to verify whether the connection between the two endpoints of two dashed line segments captured by the two surround view cameras is equal. The lane line endpoint connection parallel constraint is used to verify whether the connection between the two endpoints of two dashed line segments captured by the two surround view cameras is parallel. If the parameter verification results of the second surround view camera pass the verification, the external parameters of the third surround view camera are then adjusted; wherein, the third surround view camera includes the left surround view camera and the right surround view camera.

8. The adjustment method according to claim 7, characterized in that, After adjusting the extrinsic parameters of the third surround-view camera, the adjustment method further includes: A second inverse perspective transformation image is generated based on the image captured by the fourth surround-view camera; wherein, the fourth surround-view camera includes the front surround-view camera, the rear surround-view camera, the left surround-view camera, and the right surround-view camera; Geometric constraint verification is performed on the lane lines in the second inverse perspective transformation image to obtain the parameter verification results of the third surround view camera; If the parameter verification results of the third surround view camera all indicate that the verification is successful, the extrinsic parameters of the fourth surround view camera are fixed.

9. The adjustment method according to claim 1, characterized in that, When the target optimization method is the second optimization method, the adjustment method further includes: A third inverse perspective transformation image is generated based on the images captured by the fourth surround-view camera; wherein, the fourth surround-view camera includes the front surround-view camera, the rear surround-view camera, the left surround-view camera, and the right surround-view camera; Geometric constraint verification is performed on the lane lines in the third inverse perspective transformation diagram to obtain the parameter verification results of the fourth surround view camera; If the parameter verification result of the fourth surround view camera passes the verification, the extrinsic parameters of the fourth surround view camera are fixed.

10. The adjustment method according to any one of claims 1 to 9, characterized in that, The adjustment method further includes: Based on the images captured by the fourth surround-view camera, stitching misalignment information and slope deviation information are determined; wherein, the stitching misalignment information includes whether each intersection point overlaps in the images captured by adjacent surround-view cameras, and the intersection point is the point where the lane line and the preset target ray intersect; the slope deviation information includes the slope difference of the same lane line captured by adjacent surround-view cameras; Based on the splicing misalignment information and the slope deviation information, the adjustment result is determined.

11. A device for adjusting camera parameters, characterized in that, When used in vehicles, the adjustment device includes: The first determining module is used to determine a target optimization method based on the road information where the vehicle is located; wherein the road information represents the lane line information of the vehicle, and the target optimization method includes one of the following: a first optimization method and a second optimization method, wherein the adjustment order of the second optimization method is different from the adjustment order of the first optimization method, and the adjustment order represents the adjustment priority of at least one surround view camera in the vehicle. The second determining module is used to determine the detection result based on the vehicle's current driving information and current lane line information; wherein the detection result indicates whether parameter adjustment is triggered; The third determining module is used to determine the current adjustment information of the first surround-view camera according to the target optimization method, based on the target image containing the dashed lane collected by the first surround-view camera and the previous adjustment information of the first surround-view camera, when the detection result is the first detection result; wherein, the first detection result represents the trigger parameter adjustment; the first surround-view camera includes at least one of the following: a front surround-view camera installed at the front of the vehicle body, a rear surround-view camera installed at the rear of the vehicle body, a left surround-view camera installed on the left side of the vehicle body, and a right surround-view camera installed on the right side of the vehicle body; the previous adjustment information includes the previous pitch angle, the previous yaw angle, and the previous roll angle; The adjustment module is used to adjust the extrinsic parameters of the first surround-view camera according to the current adjustment information of the first surround-view camera.

12. A vehicle, characterized in that, It includes a processor and a memory, the memory storing a computer program that can run on the processor, the processor executing the computer program to implement the method of any one of claims 1 to 10.

13. A computer-readable storage medium, characterized in that, It stores a computer program that, when executed by a processor, implements the method described in any one of claims 1 to 10.

14. A computer program product comprising a computer program or instructions, characterized in that, When the computer program or instructions are executed by a processor, they implement the method of any one of claims 1 to 10.

Images