Lane line detection loss compensation method and device based on look-around camera

Abstract

The invention relates to a lane line detection loss compensation method and device based on an all-round camera, and the method comprises the steps: judging whether a lane signal detection result of an FCM algorithm meets a lane line loss condition or not, starting the all-round camera when the detection result meets the lane line loss condition, and obtaining the lane line information in combination with a lane line detection model; and acquiring driving data of the target vehicle, fusing the lane line information and the driving data to obtain fusion information, and performing data conversion on the fusion information to obtain target format fusion data, so that the ADAS system controls the target vehicle to execute corresponding driving operation according to the target format fusion data. According to the method, when the lane line is lost, the lane line information is acquired by adopting the surround view camera, data processing is performed on the lane line information, and the lane line information is converted into the lane line information required by the ADAS through the image fusion technology, so that the track of the current vehicle is predicted, the adaptive capacity of the ADAS vehicle to the driving environment is improved, and meanwhile, the safety and reliability of the driving process are ensured.

Description

Technical Field

[0001] This application relates to the technical field of lane line loss compensation, and particularly to a lane line detection loss compensation method and device based on a surround view camera. Background Art

[0002] With the wide application of ADAS technology, the research on predicting and planning the current vehicle trajectory has become a hot topic. Currently, the cameras of ADAS vehicles are usually installed behind the windshield. However, in complex traffic scenarios of low-speed congestion, it usually blocks the detection of the target lane line by the vehicle's own camera, resulting in problems such as incomplete road information collection by autonomous driving vehicles and low accuracy of preview planning, which urgently need to be solved. Summary of the Invention

[0003] This application provides a lane line detection loss compensation method and device based on a surround view camera to solve problems such as the front lane line being blocked and lost due to the too-close distance between the leading vehicle and the own vehicle in low-speed congestion scenarios, and the inability to effectively predict and plan the driving trajectory.

[0004] In a first aspect embodiment of this application, a lane line detection loss compensation method based on a surround view camera is provided, including the following steps: Detect the lane information in front of the target vehicle based on a preset ADAS system and FCM algorithm to obtain corresponding detection results; Determine whether the detection results meet the lane line loss condition. Wherein, when the detection results meet the lane line loss condition, start the preset surround view camera, and obtain the lane line information in front of the target vehicle through the surround view camera and a pre-constructed lane line detection model; Collect the driving data of the target vehicle, and fuse the lane line information and the driving data to obtain corresponding fusion information, and perform data conversion on the fusion information to obtain fusion data in a target format, so that the ADAS system controls the target vehicle to perform corresponding driving operations according to the fusion data in the target format.

[0005] Optionally, in an embodiment of this application, it further includes: If the detection results do not meet the lane line loss condition, control the target vehicle to perform corresponding driving operations according to the detection results.

[0006] Optionally, in an embodiment of the present application, the obtaining of the lane line information in front of the target vehicle through the surround view camera and a pre-constructed lane line detection model includes: collecting road image data corresponding to the target vehicle through the surround view camera, and inputting the road image data into the lane line detection model to extract lane line features corresponding to the road image data; performing curve fitting on the lane line features to obtain fitted lane line features, and performing classification and regression processing on the fitted lane line features to generate the lane line information.

[0007] Optionally, in an embodiment of the present application, the collecting of the driving data of the target vehicle, and fusing the lane line information and the driving data to obtain corresponding fusion information, and performing data conversion on the fusion information to obtain fusion data in a target format includes: based on a preset extended Kalman filtering algorithm, fusing the lane line information and the driving data to obtain the fusion information; performing coordinate system conversion on the fusion information to convert the fusion information into a target coordinate system, and performing time synchronization on the fusion information in the target coordinate system to generate the fusion data in the target format.

[0008] Optionally, in an embodiment of the present application, the enabling of the ADAS system to control the target vehicle to perform corresponding driving operations according to the fusion data in the target format includes: predicting the driving trajectory of the target vehicle based on the fusion data in the target format and a preset ADAS system, and determining a corresponding driving path through the driving trajectory, so that the target user controls the target vehicle to perform corresponding driving operations according to the driving trajectory and the driving path.

[0009] An embodiment of the second aspect of the present application provides a lane line detection loss compensation device based on a surround view camera, including: a detection module, configured to detect lane information in front of a target vehicle based on a preset ADAS system and an FCM algorithm to obtain a corresponding detection result; a switching module, configured to determine whether the detection result meets a lane line loss condition, wherein when the detection result meets the lane line loss condition, a preset surround view camera is started, and lane line information in front of the target vehicle is obtained through the surround view camera and a pre-constructed lane line detection model; a compensation module, configured to collect driving data of the target vehicle, and fuse the lane line information and the driving data to obtain corresponding fusion information, and perform data conversion on the fusion information to obtain fusion data in a target format, so that the ADAS system controls the target vehicle to perform corresponding driving operations according to the fusion data in the target format.

[0010] Optionally, in an embodiment of the present application, it further includes: an execution module, configured to, if the detection result does not meet the lane line loss condition, control the target vehicle to perform corresponding driving operations according to the detection result.

[0011] Optionally, in an embodiment of the present application, the switching module includes: an extraction unit, configured to collect road image data corresponding to the target vehicle through the surround view camera, and input the road image data into the lane line detection model to extract lane line features corresponding to the road image data; a fitting unit, configured to perform curve fitting on the lane line features to obtain fitted lane line features, and perform classification and regression processing on the fitted lane line features to generate the lane line information.

[0012] Optionally, in an embodiment of the present application, the compensation module includes: a fusion unit, configured to fuse the lane line information and the driving data based on a preset extended Kalman filter algorithm to obtain the fusion information; a conversion unit, configured to perform coordinate system conversion on the fusion information to convert the fusion information into a target coordinate system, and perform time synchronization on the fusion information in the target coordinate system to generate the fusion data in the target format.

[0013] Optionally, in an embodiment of the present application, the compensation module further includes: a prediction unit, configured to predict the driving trajectory of the target vehicle based on the fusion data in the target format and a preset ADAS system, and determine a corresponding driving path through the driving trajectory, so that the target user controls the target vehicle to perform corresponding driving operations according to the driving trajectory and the driving path.

[0014] An embodiment of the third aspect of the present application provides a vehicle, including: a memory, a processor, and a computer program stored on the memory and executable on the processor, where the processor executes the program to implement the method for detecting and compensating for lane line loss based on a surround view camera as described in the above embodiment.

[0015] An embodiment of the fourth aspect of the present application provides a computer-readable storage medium storing a computer program, and when the program is executed by a processor, it implements the method for detecting and compensating for lane line loss based on a surround view camera as described above.

[0016] An embodiment of the fifth aspect of the present application provides a computer program product including a computer program, and when the computer program is executed, it is used to implement the method for detecting and compensating for lane line loss based on a surround view camera as described above.

[0017] Therefore, the embodiments of the present application have the following beneficial effects:

[0018] Embodiments of the present application can detect lane information in front of the target vehicle based on a preset ADAS system and FCM algorithm to obtain corresponding detection results; determine whether the detection results meet the lane line loss condition. Among them, when the detection results meet the lane line loss condition, a preset surround view camera is activated, and lane line information in front of the target vehicle is obtained through the surround view camera and a pre-constructed lane line detection model; the driving data of the target vehicle is collected, and the lane line information and driving data are fused to obtain corresponding fusion information, and the fusion information is subjected to data conversion to obtain fusion data in a target format, so that the ADAS system controls the target vehicle to perform corresponding driving operations according to the fusion data in the target format. The present application uses a surround view camera to obtain lane line information when the lane lines are lost, processes the data, and converts it into lane line information required by the ADAS through image fusion technology, thereby predicting and planning the trajectory of the current vehicle, improving the adaptability of the ADAS vehicle to the driving environment, and ensuring the safety and reliability of the driving process. Thus, the problems that in a low-speed congestion scenario, the distance between the vehicle in front and the own vehicle is too close, resulting in the occlusion and loss of the front lane lines, and the inability to effectively predict and plan the driving trajectory are solved.

[0019] Additional aspects and advantages of the present application will be given in part in the following description, become apparent in part from the following description, or be understood through the practice of the present application. BRIEF DESCRIPTION OF THE DRAWINGS

[0020] The above and / or additional aspects and advantages of the present application will become apparent and be readily understood from the following description of the embodiments in conjunction with the drawings, in which:

[0021] Figure 1 is a flowchart of a method for compensating for lane line detection loss based on a surround view camera according to an embodiment of the present application;

[0022] Figure 2 is a schematic diagram of the logical architecture of a method for compensating for lane line detection loss based on a surround view camera provided by an embodiment of the present application;

[0023] Figure 3 is an example diagram of a device for compensating for lane line detection loss based on a surround view camera according to an embodiment of the present application;

[0024] Figure 4 is a schematic diagram of the structure of a vehicle provided by an embodiment of the present application.

[0025] Among them, 10 - a device for compensating for lane line detection loss based on a surround view camera; 100 - a detection module, 200 - a switching module, 300 - a compensation module; 401 - a memory, 402 - a processor, 403 - a communication interface. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0026] Embodiments of the present application will be described in detail below. Examples of the embodiments are shown in the accompanying drawings, where the same or similar reference numerals denote the same or similar elements or elements having the same or similar functions throughout. The embodiments described below by referring to the accompanying drawings are exemplary and are intended to explain the present application and should not be construed as limiting the present application.

[0027] A method and device for compensating for lost lane line detection based on a surround-view camera according to an embodiment of the present application will be described below with reference to the accompanying drawings. In view of the problems mentioned in the above background art, the present application provides a method for compensating for lost lane line detection based on a surround-view camera. In this method, based on a preset ADAS system and FCM algorithm, lane information in front of the target vehicle is detected to obtain corresponding detection results; it is determined whether the detection results meet the lane line loss condition, where when the detection results meet the lane line loss condition, a preset surround-view camera is activated, and lane line information in front of the target vehicle is obtained through the surround-view camera and a pre-constructed lane line detection model; driving data of the target vehicle is collected, and the lane line information and the driving data are fused to obtain corresponding fusion information, and the fusion information is subjected to data conversion to obtain target format fusion data, so that the ADAS system controls the target vehicle to perform corresponding driving operations according to the target format fusion data. By using a surround-view camera to obtain lane line information when the lane line is lost, processing the data, and converting it into lane line information required by the ADAS through image fusion technology, the present application predicts and plans the trajectory of the current vehicle, improves the adaptability of the ADAS vehicle to the driving environment, and at the same time ensures the safety and reliability of the driving process. Thus, problems such as the front lane line being blocked and lost due to the too-close distance between the vehicle in front and the self-vehicle in a low-speed congestion scenario, and the inability to effectively predict and plan the driving trajectory are solved.

[0028] Specifically, Figure 1 is a flowchart of a method for compensating for lost lane line detection based on a surround-view camera provided by an embodiment of the present application.

[0029] As Figure 1 shown, the method for compensating for lost lane line detection based on a surround-view camera includes the following steps:

[0030] In step S101, based on a preset ADAS system and FCM algorithm, lane information in front of the target vehicle is detected to obtain corresponding detection results.

[0031] In step S102, it is determined whether the detection results meet the lane line loss condition, where when the detection results meet the lane line loss condition, a preset surround-view camera is activated, and lane line information in front of the target vehicle is obtained through the surround-view camera and a pre-constructed lane line detection model.

[0032] In the embodiments of the present application, when the Advanced Driver Assistance System (ADAS) detects the loss of lane lines by the FCM algorithm, the surround view camera is immediately activated to ensure data continuity and seamless system switching. A lane line detection model is used to accurately identify lane line features from the images obtained by the surround view camera, including straight lines, dotted lines, curves, etc.

[0033] Thus, when the ADAS system fails to detect lane lines, the embodiments of the present application automatically switch to the surround view camera to process the images provided by the surround view camera, and use image recognition technology to extract and identify lane line information, thereby providing reliable data support for subsequent loss compensation of lane line detection.

[0034] Optionally, in an embodiment of the present application, lane line information in front of the target vehicle is obtained through a surround view camera and a pre-constructed lane line detection model, including: collecting road image data corresponding to the target vehicle through the surround view camera, and inputting the road image data into the lane line detection model to extract lane line features corresponding to the road image data; performing curve fitting on the lane line features to obtain fitted lane line features, and performing classification and regression processing on the fitted lane line features to generate lane line information.

[0035] It should be noted that the embodiments of the present application can use lightweight networks such as MobileNet and ShuffleNet to construct a lane line detection model, and input the road image data collected by the surround view camera into the lane line detection model to extract lane line features (or lane line point sets) corresponding to the road image data.

[0036] Secondly, the embodiments of the present application can use a spline curve (such as a B-spline) to fit the lane line features to obtain fitted lane line features, and use a polynomial or spline curve to fit the lane line points, and map the lane lines to a bird's-eye view through inverse perspective transformation, and apply geometric constraints such as parallelism and width to optimize the lane lines, and perform classification and regression processing on them to generate lane line information.

[0037] During the actual execution process, as Figure 2 shown, the deployment of the lane line detection model can be selectively configured in either one according to the computing power resources of the ADAS controller and the FAPA controller.

[0038] Thus, the embodiments of the present application use a lightweight deep learning model to detect lane lines, which can not only meet the real-time requirements, but also further improve the accuracy and continuity of lane lines through operations such as curve fitting.

[0039] Optionally, in an embodiment of the present application, it further includes: if the detection result does not meet the lane line loss condition, controlling the target vehicle to perform corresponding driving operations according to the detection result.

[0040] During the actual execution process, when the lane information detection results obtained by the ADAS system and the FCM algorithm do not meet the lane line loss condition, the embodiments of the present application can control the vehicle to perform corresponding overtaking and other driving operations according to the detection results, that is, specific lane line information such as lane line position and type.

[0041] In step S103, the driving data of the target vehicle is collected, and the lane line information and the driving data are fused to obtain corresponding fused information, and the fused information is subjected to data conversion to obtain fused data in the target format, so that the ADAS system controls the target vehicle to perform corresponding driving operations according to the fused data in the target format.

[0042] Furthermore, the embodiments of the present application also need to collect the driving data of the vehicle (that is, the current state data of the vehicle), and fuse the identified lane line information with the current state data of the vehicle, and convert it into a data format applicable to the ADAS (that is, the fused data in the target format), so that the ADAS system predicts and plans the current driving trajectory of the vehicle according to the fused data in the target format to ensure the safety and reliability of the driving process.

[0043] Optionally, in an embodiment of the present application, collecting the driving data of the target vehicle, and fusing the lane line information and the driving data to obtain corresponding fused information, and performing data conversion on the fused information to obtain fused data in the target format, includes: based on a preset extended Kalman filter algorithm, fusing the lane line information and the driving data to obtain fused information; performing coordinate system conversion on the fused information to convert the fused information into the target coordinate system, and synchronizing the time of the fused information in the target coordinate system to generate fused data in the target format.

[0044] It should be noted that the embodiments of the present application can fuse the identified lane line information with data such as vehicle speed and direction, and convert it into a format suitable for ADAS processing. The specific process is as follows:

[0045] 1. Obtain lane line information (such as the position, type, and curvature of the lane line) and driving data (such as vehicle speed, vehicle direction, vehicle position, acceleration, and yaw rate);

[0046] 2. Use the extended Kalman filter to fuse the lane line information and the driving data to obtain fused information, so as to better process the vehicle motion model and sensor data;

[0047] 3. Convert the lane line information from the image coordinate system to the vehicle coordinate system or the world coordinate system, which is convenient for fusing with the vehicle dynamic data, and ensure that the time stamps of the lane line information and the vehicle dynamic data are consistent to avoid errors caused by time asynchronization;

[0048] 4. Convert the fused data into a format supported by the ADAS system, such as CAN bus messages, ROS messages, or other protocols.

[0049] Thus, the embodiments of the present application fuse lane line information with data such as vehicle speed and direction, and convert it into a format suitable for the ADAS system to process, thereby improving the accuracy of data fusion and conversion, and ensuring the smooth implementation of advanced driving assistance functions such as lane keeping and path planning.

[0050] Optionally, in an embodiment of the present application, the ADAS system controls the target vehicle to perform corresponding driving operations according to the data fused in the target format, including: predicting the driving trajectory of the target vehicle based on the data fused in the target format and the preset ADAS system, and determining the corresponding driving path through the driving trajectory, so that the target user controls the target vehicle to perform corresponding driving operations according to the driving trajectory and the driving path.

[0051] After that, the embodiments of the present application also need to predict the driving trajectory of the vehicle in real time according to the data fused in the target format, plan a safe driving path, improve the vehicle's adaptability to complex driving environments, and feedback the predicted trajectory and the planned results to the driver, so that the ADAS system controls the vehicle to perform operations such as lane keeping assistance and adaptive cruise control according to the data fused in the target format, and dynamically adjusts according to the actual driving situation to ensure the safety of the driving process.

[0052] Thus, the embodiments of the present application improve the robustness of road information collection and the reliability of the autonomous driving vehicle system, not only solve the technical challenges of the ADAS system in low-speed congestion scenarios, but also enhance the ADAS vehicle's perception and processing capabilities of the environment, providing a safer and more reliable driving assistance for the driver.

[0053] The lane line detection loss compensation method based on a surround view camera according to an embodiment of the present application detects the lane information in front of the target vehicle based on a preset ADAS system and the FCM algorithm to obtain corresponding detection results; determines whether the detection results meet the lane line loss condition, where when the detection results meet the lane line loss condition, the preset surround view camera is activated, and the lane line information in front of the target vehicle is obtained through the surround view camera and a pre-constructed lane line detection model; collects the driving data of the target vehicle, fuses the lane line information and the driving data to obtain corresponding fusion information, and performs data conversion on the fusion information to obtain fusion data in a target format, so that the ADAS system controls the target vehicle to perform corresponding driving operations according to the fusion data in the target format. The present application uses a surround view camera to obtain lane line information when the lane lines are lost, processes the data, and converts it into the lane line information required by the ADAS through image fusion technology, thereby predicting and planning the trajectory of the current vehicle, improving the adaptability of the ADAS vehicle to the driving environment, and ensuring the safety and reliability of the driving process.

[0054] Secondly, a lane line detection loss compensation device based on a surround view camera according to an embodiment of the present application is described with reference to the accompanying drawings.

[0055] Figure 3 It is a block diagram of a lane line detection loss compensation device based on a surround view camera according to an embodiment of the present application.

[0056] As Figure 3 shown, the lane line detection loss compensation device 10 based on a surround view camera includes: a detection module 100, a switching module 200, and a compensation module 300.

[0057] Among them, the detection module 100 is used to detect the lane information in front of the target vehicle based on a preset ADAS system and the FCM algorithm to obtain corresponding detection results.

[0058] The switching module 200 is used to determine whether the detection results meet the lane line loss condition, where when the detection results meet the lane line loss condition, the preset surround view camera is activated, and the lane line information in front of the target vehicle is obtained through the surround view camera and a pre-constructed lane line detection model.

[0059] The compensation module 300 is used to collect the driving data of the target vehicle, fuse the lane line information and the driving data to obtain corresponding fusion information, and perform data conversion on the fusion information to obtain fusion data in a target format, so that the ADAS system controls the target vehicle to perform corresponding driving operations according to the fusion data in the target format.

[0060] Optionally, in an embodiment of the present application, the lane line detection loss compensation device 10 based on surround cameras in the embodiment of the present application further includes: an execution module, configured to, if the detection result does not meet the lane line loss condition, control the target vehicle to perform corresponding driving operations according to the detection result.

[0061] Optionally, in an embodiment of the present application, the switching module 200 includes: an extraction unit and a fitting unit.

[0062] Among them, the extraction unit is configured to collect road image data corresponding to the target vehicle through surround cameras, and input the road image data into the lane line detection model to extract lane line features corresponding to the road image data.

[0063] The fitting unit is configured to perform curve fitting on the lane line features to obtain fitted lane line features, and perform classification and regression processing on the fitted lane line features to generate lane line information.

[0064] Optionally, in an embodiment of the present application, the compensation module 300 includes: a fusion unit and a conversion unit.

[0065] Among them, the fusion unit is configured to fuse the lane line information and the driving data based on a preset extended Kalman filter algorithm to obtain fusion information.

[0066] The conversion unit is configured to perform coordinate system conversion on the fusion information to convert the fusion information into a target coordinate system, and perform time synchronization on the fusion information in the target coordinate system to generate target format fusion data.

[0067] Optionally, in an embodiment of the present application, the compensation module 300 further includes: a prediction unit, configured to predict the driving trajectory of the target vehicle based on the target format fusion data and a preset ADAS system, and determine a corresponding driving path through the driving trajectory, so that the target user controls the target vehicle to perform corresponding driving operations according to the driving trajectory and the driving path.

[0068] It should be noted that the foregoing explanation of the embodiment of the lane line detection loss compensation method based on surround cameras also applies to the lane line detection loss compensation device based on surround cameras in this embodiment, and will not be elaborated here.

[0069] The lane line detection loss compensation device based on an omnidirectional camera according to an embodiment of the present application includes a detection module 100, configured to detect lane information in front of a target vehicle based on a preset ADAS system and an FCM algorithm to obtain corresponding detection results; a switching module 200, configured to determine whether the detection results meet the lane line loss condition, where when the detection results meet the lane line loss condition, a preset omnidirectional camera is activated, and lane line information in front of the target vehicle is obtained through the omnidirectional camera and a pre-constructed lane line detection model; a compensation module 300, configured to collect driving data of the target vehicle, fuse the lane line information and the driving data to obtain corresponding fused information, and perform data conversion on the fused information to obtain fused data in a target format, so that the ADAS system controls the target vehicle to perform corresponding driving operations according to the fused data in the target format. The present application uses an omnidirectional camera to obtain lane line information when the lane lines are lost, processes the data, and converts it into lane line information required by the ADAS through image fusion technology, thereby predicting and planning the trajectory of the current vehicle, improving the adaptability of the ADAS vehicle to the driving environment, and ensuring the safety and reliability of the driving process.

[0070] Figure 4 It is a schematic structural diagram of a vehicle provided by an embodiment of the present application. The vehicle may include:

[0071] A memory 401, a processor 402, and a computer program stored on the memory 401 and executable on the processor 402.

[0072] When the processor 402 executes the program, it implements the lane line detection loss compensation method based on an omnidirectional camera provided in the above embodiment.

[0073] Further, the vehicle further includes:

[0074] A communication interface 403, configured for communication between the memory 401 and the processor 402.

[0075] The memory 401 is used to store a computer program executable on the processor 402.

[0076] The memory 401 may include a high-speed RAM memory, and may also include a non-volatile memory, such as at least one disk memory.

[0077] If the memory 401, the processor 402, and the communication interface 403 are implemented independently, the communication interface 403, the memory 401, and the processor 402 can be interconnected through a bus and communicate with each other. The bus can be an Industry Standard Architecture (ISA) bus, a Peripheral Component Interconnect (PCI) bus, an Extended Industry Standard Architecture (EISA) bus, or the like. The bus can be divided into an address bus, a data bus, a control bus, etc. For ease of representation, Figure 4 only a thick line is used to represent it in Figure 4 , but it does not mean that there is only one bus or one type of bus.

[0078] Optionally, in a specific implementation, if the memory 401, the processor 402, and the communication interface 403 are integrated on a single chip, the memory 401, the processor 402, and the communication interface 403 can communicate with each other through an internal interface.

[0079] The processor 402 may be a Central Processing Unit (CPU), or an Application Specific Integrated Circuit (ASIC), or one or more integrated circuits configured to implement the embodiments of the present application.

[0080] The embodiments of the present application also provide a computer-readable storage medium, on which a computer program is stored. When the program is executed by a processor, the above method for compensating for the loss of lane line detection based on a surround camera is implemented.

[0081] The embodiments of the present application also provide a computer program product, including a computer program, which is used to implement the above method for compensating for the loss of lane line detection based on a surround camera when the computer program is executed.

[0082] In the description of this specification, the descriptions referring to terms such as "one embodiment", "some embodiments", "example", "specific example", or "some examples", etc., mean that the specific features, structures, materials, or characteristics described in connection with the embodiment or example are included in at least one embodiment or example of this application. In this specification, the schematic representations of the above terms do not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials, or characteristics described may be combined in any one or N embodiments or examples in a suitable manner. In addition, without contradiction, those skilled in the art can combine and combine the different embodiments or examples described in this specification and the features of different embodiments or examples.

[0083] In addition, the terms "first" and "second" are used only for descriptive purposes and cannot be construed as indicating or implying relative importance or implicitly specifying the quantity of the indicated technical features. Thus, the features defined with "first" and "second" may explicitly or implicitly include at least one of such features. In the description of this application, the meaning of "N" is at least two, such as two, three, etc., unless otherwise specifically and clearly defined.

[0084] Any process or method description depicted in a flowchart or otherwise described herein may be understood to represent a module, segment, or portion of code including one or N executable instructions for implementing a customized logic function or process, and the scope of the preferred embodiments of this application includes additional implementations, where the functions may be executed in a substantially simultaneous manner or in a reverse order according to the functions involved, rather than in the order shown or discussed, which should be understood by those skilled in the art to which the embodiments of this application pertain.

[0085] The logic and / or steps represented in the flowchart or otherwise described herein, for example, can be considered as a definable list of executable instructions for implementing logical functions, and can be embodied specifically in any computer-readable medium for use by an instruction execution system, apparatus, or device (such as a computer-based system, a system including a processor, or other systems that can fetch and execute instructions from the instruction execution system, apparatus, or device), or used in conjunction with these instruction execution systems, apparatuses, or devices. For the purposes of this specification, a "computer-readable medium" can be any device that can contain, store, communicate, propagate, or transport a program for use by or in conjunction with an instruction execution system, apparatus, or device. More specific examples (a non-exhaustive list) of the computer-readable medium include the following: an electrical connection part (electronic device) having one or N wirings, a portable computer disk cartridge (magnetic device), a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber device, and a portable compact disc read-only memory (CDROM). Additionally, the computer-readable medium can even be paper or other suitable media on which the program can be printed, because the program can be obtained electronically by optically scanning the paper or other media, followed by editing, interpretation, or otherwise processing as appropriate, and then stored in a computer memory.

[0086] It should be understood that various parts of the present application can be implemented by hardware, software, firmware, or a combination thereof. In the above-described embodiments, the N steps or methods can be implemented by software or firmware stored in a memory and executed by a suitable instruction execution system. If implemented in hardware, as in another embodiment, any one or a combination of the following techniques well known in the art can be used: discrete logic circuits having logic gate circuits for implementing logical functions on data signals, application specific integrated circuits having appropriate combinational logic gate circuits, programmable gate arrays (PGAs), field programmable gate arrays (FPGAs), etc.

[0087] Those of ordinary skill in the art of this technology can understand that all or part of the steps carried by the method of implementing the above embodiments can be completed by a program instructing relevant hardware, and the program can be stored in a computer-readable storage medium. When the program is executed, it includes one or a combination of the steps of the method embodiments.

[0088] In addition, each functional unit in various embodiments of the present application may be integrated into a processing module, may exist physically alone for each unit, or two or more units may be integrated into one module. The above-mentioned integrated module may be implemented in the form of hardware or in the form of a software functional module. When the integrated module is implemented in the form of a software functional module and sold or used as an independent product, it may also be stored in a computer-readable storage medium.

[0089] The above-mentioned storage medium may be a read-only memory, a magnetic disk, an optical disc, etc. Although the embodiments of the present application have been shown and described above, it can be understood that the above embodiments are exemplary and should not be construed as limiting the present application. Those of ordinary skill in the art can make changes, modifications, substitutions, and variations to the above embodiments within the scope of the present application.

Claims

1. A lane line detection loss compensation method based on a surround camera, characterized in that, It includes the following steps: Based on a preset ADAS system and FCM algorithm, detect the lane information in front of the target vehicle to obtain corresponding detection results; Determine whether the detection results meet the lane line loss condition. Among them, when the detection results meet the lane line loss condition, start a preset surround-view camera, and obtain the lane line information in front of the target vehicle through the surround-view camera and a pre-constructed lane line detection model; Collect the driving data of the target vehicle, and fuse the lane line information and the driving data to obtain corresponding fused information, and perform data conversion on the fused information to obtain fused data in a target format, so that the ADAS system controls the target vehicle to perform corresponding driving operations according to the fused data in the target format.

2. The method according to claim 1, wherein It also includes: If the detection results do not meet the lane line loss condition, control the target vehicle to perform corresponding driving operations according to the detection results.

3. The method according to claim 1, characterized in that The obtaining the lane line information in front of the target vehicle through the surround-view camera and a pre-constructed lane line detection model includes: Collect the road image data corresponding to the target vehicle through the surround-view camera, and input the road image data into the lane line detection model to extract the lane line features corresponding to the road image data; Perform curve fitting on the lane line features to obtain fitted lane line features, and perform classification and regression processing on the fitted lane line features to generate the lane line information.

4. The method according to claim 3, wherein The collecting the driving data of the target vehicle, and fusing the lane line information and the driving data to obtain corresponding fused information, and performing data conversion on the fused information to obtain fused data in a target format includes: Based on a preset extended Kalman filter algorithm, fuse the lane line information and the driving data to obtain the fused information; Perform coordinate system conversion on the fused information to convert the fused information into a target coordinate system, and perform time synchronization on the fused information in the target coordinate system to generate the fused data in the target format.

5. The method according to claim 4, wherein The making the ADAS system control the target vehicle to perform corresponding driving operations according to the fused data in the target format includes: Based on the fused data in the target format and a preset ADAS system, predict the driving trajectory of the target vehicle, and determine a corresponding driving path through the driving trajectory, so that the target user controls the target vehicle to perform corresponding driving operations according to the driving trajectory and the driving path.

6. A lane line detection loss compensation device based on an omnidirectional camera, characterized in that, It includes: A detection module, used to detect the lane information in front of the target vehicle based on a preset ADAS system and FCM algorithm to obtain corresponding detection results; A switching module, used to determine whether the detection results meet the lane line loss condition. Among them, when the detection results meet the lane line loss condition, start a preset surround-view camera, and obtain the lane line information in front of the target vehicle through the surround-view camera and a pre-constructed lane line detection model; A compensation module, configured to collect driving data of the target vehicle, fuse the lane line information and the driving data to obtain corresponding fused information, and perform data conversion on the fused information to obtain fused data in a target format, so that the ADAS system controls the target vehicle to perform corresponding driving operations according to the fused data in the target format.

7. The device according to claim 6, characterized in that, It further includes: An execution module, configured to, if the detection result does not meet the lane line loss condition, control the target vehicle to perform corresponding driving operations according to the detection result.

8. A vehicle, characterized in that, It includes: A memory, a processor, and a computer program stored on the memory and executable on the processor, where the processor executes the program to implement the method for lane line detection loss compensation based on a surround view camera according to any one of claims 1-5.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that, The program is executed by the processor to implement the method for lane line detection loss compensation based on a surround view camera according to any one of claims 1-5.

10. A computer program product, comprising a computer program, characterized in that, The computer program is executed to implement the method for lane line detection loss compensation based on a surround view camera according to any one of claims 1-5.

Images