Lane line correction method and device, equipment, storage medium and chip
By obtaining vehicle condition information in unmanned vehicles and applying preset correction rules, lane line information is corrected, which solves the problem that unmanned vehicles cannot obtain correct and continuous lane lines during driving, and improves the vehicle's driving accuracy and safety.
Patent Information
- Application Number
- CN202311240758.8
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2023-09-22
- Publication Date
- 2025-03-25
AI Technical Summary
During driving, due to lane lines blocking and wear problems, the correct and continuous lane lines information cannot be obtained, affecting the vehicle's accurate driving.
By acquiring vehicle condition information, including target lane line information and vehicle lane change information, the abnormal target lane line information is corrected using the preset first correction rules and second correction rules. The specific method includes using the first correction rule when the vehicle has not changed lanes, using the second correction rule when the vehicle has changed lanes, and correcting left and right lane line information or lane line information.
It realizes the target lane line information that intelligently corrects abnormalities in different vehicle conditions, improves the accuracy and safety of vehicle driving, and solves the problem that unmanned vehicles cannot obtain correct and continuous lane lines.
Smart Images
Figure CN119693893A_ABST
Abstract
Description
Technical Field
[0001] The present disclosure relates to the field of vehicle technology, and in particular to a lane line correction method, device, equipment, storage medium and chip. Background Art
[0002] With the continuous development of artificial intelligence and intelligent transportation technology, unmanned vehicles based on autonomous driving technology have begun to enter people's lives. In order to ensure that unmanned vehicles can comply with traffic rules and ensure driving safety during driving, unmanned vehicles need to identify lane lines on the road during driving.
[0003] However, in the related technologies, lane lines may be unclear or broken due to obstruction, wear and tear, or lane lines may be missing at intersections. This in turn causes the unmanned vehicle to be unable to obtain correct and continuous lane lines, further affecting its accurate driving. Summary of the invention
[0004] In order to overcome the problems existing in the related art, the present disclosure provides a lane line correction method, device, equipment, storage medium and chip to solve the technical problems existing in the above-mentioned related art such as the inability to obtain correct and continuous lane lines, thereby affecting the accurate driving of the vehicle.
[0005] According to a first aspect of an embodiment of the present disclosure, a lane line correction method is provided, comprising:
[0006] Acquiring vehicle condition information, wherein the vehicle condition information includes target lane line information of the vehicle and vehicle lane change information;
[0007] When there is an abnormality in the target lane line information and the vehicle lane change information is used to indicate that the vehicle is not currently changing lanes, correcting the target lane line information based on a preset first correction rule;
[0008] When there is an abnormality in the target lane line information and the vehicle lane change information is used to indicate that the vehicle is currently changing lanes, the target lane line information is corrected based on a preset second correction rule, and the first correction rule and the second correction rule are different.
[0009] In some embodiments, the target lane line information includes the main lane line information and / or lane change lane line information of the vehicle currently traveling, and the target lane line information includes the left lane line information and the right lane line information of the target lane, and the correcting the target lane line information based on the preset first correction rule includes any one of the following:
[0010] When there is an abnormality in the left lane line information of the target lane, the left lane line information of the target lane is corrected using the right lane line information of the target lane;
[0011] When there is an abnormality in the right lane line information of the target lane, the right lane line information of the target lane is corrected using the left lane line information of the target lane;
[0012] When both the left lane line information and the right lane line information of the target lane are abnormal, the left lane line information and the right lane line information of the target lane are respectively corrected using the inferred lane line information of the target lane, where the inferred lane line information is obtained by predicting and inferring the lane line information of the target lane at a historical moment.
[0013] In some embodiments, the target lane line information includes lane change lane line information, and the correcting the target lane line information based on a preset second correction rule includes:
[0014] Acquiring sensor data of the vehicle, wherein the sensor data includes a lane change angular velocity and / or a lane change acceleration of the vehicle;
[0015] The lane change lane line information is corrected using the sensor data of the vehicle.
[0016] In some embodiments, the modifying the lane change lane line information using the sensor data of the vehicle includes:
[0017] Perform lane line prediction calculation on historical information of a lane change lane to obtain predicted lane line information of the lane change lane, wherein the historical information includes lane line information at historical moments and sensor data at historical moments;
[0018] The predicted lane line information is used to correct the lane change lane line information.
[0019] In some embodiments, before obtaining the vehicle condition information, the method further includes:
[0020] Acquire a road image by a camera device, wherein the road image includes current lane line information of the vehicle;
[0021] Performing lane line processing on the road image to obtain the current lane line information;
[0022] When there is an abnormality in the current lane line information, the camera parameters of the camera device are adjusted, and the step of acquiring the road image through the camera device is re-executed according to the adjusted camera parameters until there is no abnormality in the current lane line information.
[0023] In some embodiments, performing lane line processing on the road image to obtain the current lane line information includes:
[0024] Extracting lane lines from the road image to obtain initial lane line information;
[0025] The initial lane line information is subjected to coordinate transformation to transform the initial lane line information in the image coordinate system into the current lane line information in the real world coordinate system.
[0026] According to a second aspect of an embodiment of the present disclosure, a lane correction device is provided, comprising:
[0027] An acquisition module is configured to acquire vehicle condition information, wherein the vehicle condition information includes target lane line information of the vehicle and vehicle lane change information;
[0028] a processing module configured to correct the target lane line information based on a preset first correction rule when there is an abnormality in the target lane line information and the vehicle lane change information is used to indicate that the vehicle is not currently changing lanes;
[0029] The processing module is further configured to correct the target lane line information based on a preset second correction rule when there is an abnormality in the target lane line information and the vehicle lane change information is used to indicate that the vehicle is currently changing lanes, and the first correction rule and the second correction rule are different.
[0030] For the contents not introduced or described in the embodiments of the present disclosure, please refer to the relevant introduction in the aforementioned method embodiments, and the embodiments of the present disclosure are not limited thereto.
[0031] According to a third aspect of an embodiment of the present disclosure, there is provided an electronic device, comprising: a processor; and a memory for storing processor executable instructions; wherein the processor is configured to execute the executable instructions to implement the steps of the above-mentioned lane line correction method.
[0032] According to a fourth aspect of an embodiment of the present disclosure, a computer-readable storage medium is provided, on which computer program instructions are stored. When the program instructions are executed by a processor, the steps of the lane correction method provided in the first aspect of the present disclosure are implemented.
[0033] According to a fifth aspect of an embodiment of the present disclosure, a chip is provided, comprising: a processor and an interface; the processor is used to read instructions to execute the steps of the above-mentioned lane correction method.
[0034] The technical solution provided by the embodiment of the present disclosure may include the following beneficial effects: the electronic device obtains vehicle condition information, the vehicle condition information includes the target lane line information of the vehicle and the vehicle lane change information; when there is an abnormality in the target lane line information and the vehicle lane change information is used to indicate that the vehicle is not currently changing lanes, the target lane line information is corrected based on a preset first correction rule; when there is an abnormality in the target lane line information and the vehicle lane change information is used to indicate that the vehicle is currently changing lanes, the target lane line information is corrected based on a preset second correction rule, and the first correction rule and the second correction rule are different. It can be seen that the electronic device can intelligently correct the target lane line information with abnormalities using different correction rules under different vehicle conditions to obtain a more accurate corrected lane line, which is conducive to improving the accuracy and safety of vehicle driving. At the same time, it also solves the technical problems existing in the above-mentioned related technologies such as the inability to obtain correct and continuous lane lines, thereby affecting the accurate driving of vehicles.
[0035] It is to be understood that the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the present disclosure. BRIEF DESCRIPTION OF THE DRAWINGS
[0036] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the present disclosure and, together with the description, serve to explain the principles of the present disclosure.
[0037] Figure 1 The diagram is a schematic diagram of a system structure according to an exemplary embodiment.
[0038] Figure 2 The figure is a flowchart of a lane correction method according to an exemplary embodiment.
[0039] Figure 3 The figure is a schematic diagram of a process of obtaining lane line information according to an exemplary embodiment.
[0040] Figure 4 The figure is a schematic diagram of a process of lane calibration according to an exemplary embodiment.
[0041] Figure 5 The diagram is a schematic diagram of lane lines according to an exemplary embodiment.
[0042] FIG. 6( a ) and FIG. 6( b ) are schematic diagrams showing two types of lane line reasoning according to an exemplary embodiment.
[0043] Figure 7 The figure is a schematic diagram of an IMU lane change according to an exemplary embodiment.
[0044] Figure 8 The figure is a schematic structural diagram of a lane correction device according to an exemplary embodiment.
[0045] Fig. 9 The diagram is a schematic structural diagram of an electronic device according to an exemplary embodiment.
[0046] Fig.10 The figure is a schematic diagram showing the structure of a chip according to an exemplary embodiment. DETAILED DESCRIPTION
[0047] Exemplary embodiments will be described in detail herein, examples of which are shown in the accompanying drawings. When the following description refers to the drawings, the same numbers in different drawings represent the same or similar elements unless otherwise indicated. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with the present disclosure. Instead, they are merely examples of devices and methods consistent with some aspects of the present disclosure as detailed in the appended claims.
[0048] It should be noted that all actions of acquiring signals, information or data in the present disclosure are carried out in compliance with the relevant data protection laws and policies of the country where the device is located and with the authorization given by the owner of the corresponding device.
[0049] In the field of autonomous driving, accurate information about the surrounding environment plays a vital role in safe driving. Real-time lane line information can help the vehicle determine its real-time position and make timely adjustments to the driving state of lane deviation to ensure safe driving of the vehicle. It can be seen that accurate lane line information is also very important for safe driving.
[0050] At present, the existing lane information solutions usually include: using laser radar to obtain the vehicle's environmental point cloud data, then identifying the lane line pixels in the road image, matching the lane line point cloud data from the environmental point cloud data based on the lane line pixels, and then fitting the target lane line; or using cameras and high-precision maps to collect the target lane line. However, the application scenarios of the above two solutions have certain limitations, such as high requirements for equipment, specifically the need to install laser radar or obtain high-precision maps, etc.; and due to the equipment's own factors (such as the laser radar has blind spots, the equipment cannot obtain high-precision maps, etc.), the precision or accuracy of the target lane line will also be affected.
[0051] To solve the above problems, the present disclosure provides a lane correction method, device, equipment, storage medium and chip. Figure 1 FIG. 1 is a schematic diagram of a system structure according to an exemplary embodiment. Figure 1The system shown includes a lane calibration module 100, a lane detection module 200, a lane correction module 300 and a lane reasoning module 400. The lane calibration module 100 is mainly used to calibrate the shooting parameters of the shooting device, so that the lane information contained in the lane image shot by the calibrated shooting device does not have abnormalities, that is, the lane information is accurately and normally shot.
[0052] The lane line detection module 200 is mainly used to detect and identify the corresponding lane line information. For example, the present disclosure below can detect and identify the corresponding lane line information from the road image or the lane line image. Optionally, the lane line detection module 200 can also detect whether there is an abnormality in the above lane line information and whether compensation correction is required. If there is an abnormality in the above lane line information, it is necessary to notify the lane line correction module 300 to perform corresponding correction processing on the lane line information. The above lane line correction module 300 is mainly used to correct the corresponding lane line information when the corresponding lane line information needs to be corrected, so as to ensure the accuracy of the corresponding lane line information, thereby ensuring the safety of vehicle driving.
[0053] The lane line inference module 400 (also referred to as the lane line filtering and tracking module) is mainly used to perform inference calculations based on historical lane line information to predict future lane line information. For example, the lane line inference module 400 can predict and infer the lane line information at the next moment based on the lane line information at some historical moments, or the lane line information at the current moment. Optionally, the lane line inference module 400 can combine the vehicle sensor data and the image sensor data to predict and infer the lane line information at the current moment. The vehicle sensor data can also be referred to as the sensor data of the vehicle, which can include but is not limited to data such as angular velocity and / or acceleration collected by an inertial measurement unit (IMU). Specifically, the lane line inference module 400 can predict, infer and calculate the lane line information at the current moment based on the lane line information at some historical moments and the IMU acceleration and / or angular velocity information corresponding to the historical moment. The functions of the lane calibration module 100 , lane detection module 200 , lane correction module 300 and lane reasoning module 400 will be described in detail in the following embodiments of the present disclosure and will not be introduced here.
[0054] It should be noted that if Figure 1The system shown can be applied to electronic devices, including but not limited to smart phones (such as Android phones, iOS phones, etc.), tablet computers, PDAs, laptop computers, mobile Internet devices (Mobile Internet Devices, MID), wearable devices (such as smart watches, smart bracelets, etc.) or other devices with communication functions.
[0055] See also Figure 2 FIG. 1 is a flow chart of a lane correction method according to an exemplary embodiment. Figure 2 The method shown is applied to an electronic device, and the method may include the following implementation steps:
[0056] S201. Acquire vehicle condition information, where the vehicle condition information includes target lane line information of the vehicle and vehicle lane change information.
[0057] The above-mentioned vehicle condition information disclosed in the present invention may include, but is not limited to, for example, the target lane line information of the vehicle, the vehicle lane change information, the vehicle driving parameters or other custom information. The above-mentioned vehicle lane change information may refer to the relevant information used to describe the vehicle lane change, which can be used to indicate whether the vehicle is currently changing lanes. The above-mentioned vehicle driving parameters may refer to the working parameters of the vehicle's current driving, such as vehicle speed, steering wheel angle, etc. The above-mentioned target lane line information may refer to the lane line information on which the vehicle can travel, which may include, but is not limited to, for example, any one or more of the following combinations: the lane line information of the main lane where the vehicle is currently traveling (referred to as the main lane line information), the lane line information of the lane where the vehicle is traveling after changing lanes (referred to as the lane change lane line information), or the lane line information of other lanes, etc. The present disclosure does not make too many restrictions and details on this.
[0058] For example, see Figure 5 FIG. 1 is a schematic diagram of a lane line according to an exemplary embodiment. Figure 5 There are three lanes shown, where the lane the vehicle is currently traveling in is called the main lane, the lane on the left side of the main lane is called the left lane, and the lane on the right side of the main lane is called the right lane. For each lane, its corresponding lane line information includes left lane line information and right lane line information, where the left lane line information may refer to the lane line information on the left side of the current lane, and the right lane line information may refer to the lane line information on the right side of the current lane, etc. For example, in the figure, for the main lane, the left lane line information of the main lane may refer to the left lane line information in the figure, and the right lane line information of the main lane may refer to the right lane line information in the figure.
[0059] The lane line information involved in the present disclosure refers to information used to describe a road lane, which may include but is not limited to, for example, lane width information, lane line slope information, curvature information or other information used to describe a road lane.
[0060] The present disclosure does not limit the implementation method for obtaining the above vehicle condition information, for example, it can be obtained from other devices (such as servers or other terminals) through the network, etc. For another example, taking the above vehicle condition information including the above vehicle lane change information as an example, the present disclosure can detect in real time whether the vehicle is currently changing lanes through sensors, or detect whether the vehicle is currently changing lanes through information such as maps, etc.
[0061] For another example, the vehicle condition information includes target lane line information, and the target lane line information includes main lane line information, and the specific implementation method of obtaining the target lane line information is introduced. Figure 3 FIG. 1 is a flow chart showing a process of obtaining lane line information according to an exemplary embodiment. Figure 3 The process shown may include the following implementation steps:
[0062] S301. Acquire a lane line image, where the lane line image at least includes main lane line information on which the vehicle is traveling.
[0063] The present disclosure does not limit the implementation method of acquiring the lane line image. For example, the present disclosure can obtain the lane line image by using a calibrated camera device. The lane line image includes lane line information of the main lane where the vehicle is currently traveling, referred to as main lane line information. Optionally, it can also include other lane line information on the current road, which is not limited by the present disclosure.
[0064] S302: Perform lane line processing on the target image to obtain the target lane line information, where the target image includes the lane line image.
[0065] The present disclosure does not limit the specific implementation of the lane line processing mentioned above. For example, the present disclosure may first perform lane line extraction on the target image to extract the lane line information of the corresponding lane from the target image, which may be referred to as the initial lane line information. Then, coordinate transformation is performed on the initial lane line information to transform the initial lane line information in the image coordinate system to the target lane line information in the real world coordinate system. The present disclosure does not limit the specific implementation of the lane line extraction mentioned above. For example, a preset image recognition algorithm may be used to extract the initial lane line information of the corresponding lane from the target image.
[0066] In some embodiments, the present disclosure may also perform corresponding lane line calibration before using a camera device to obtain the lane line image. Figure 4 FIG. 1 is a schematic diagram of a process flow of lane calibration according to an exemplary embodiment. Figure 4 The process shown may include the following implementation steps:
[0067] S401. Acquire a road image through a camera device, where the road image includes current lane line information of the vehicle.
[0068] The present disclosure can obtain corresponding road images by photographing through a camera device, and the road image can include lane line information of the lane where the vehicle is currently traveling, referred to as current lane line information for short.
[0069] S402: Perform lane line processing on a target image to obtain current lane line information, wherein the target image includes the road image.
[0070] The present disclosure can perform lane line processing on the target image to extract the current lane line information from the target image. For the lane line processing, please refer to the aforementioned Figure 3 The relevant introduction in the embodiments will not be repeated here.
[0071] After obtaining the above-mentioned current lane line information, it can be determined whether the current lane line information has an abnormality. If there is an abnormality, step S403 can be continued; otherwise, the process can be terminated to complete the corresponding lane line calibration process. The present disclosure does not limit the implementation method for determining whether the above-mentioned lane line information has an abnormality. For example, in actual applications, the lane line information includes left lane line information and right lane line information, and each lane line information can include a lot of coordinate lattice information. The present disclosure can determine whether the above-mentioned lane line information is abnormal by determining whether the left and right lane lines are parallel, whether the lane line is missing, and whether the number of coordinate lattice information in the lane line information is too small (less than a preset number of points, etc.).
[0072] S403: When there is an abnormality in the current lane line information, adjust the camera parameters of the camera device, and re-execute the step of acquiring the road image through the camera device according to the adjusted camera parameters until there is no abnormality in the current lane line information.
[0073] When the present disclosure determines that the above-mentioned current lane line information is abnormal, the camera parameters of the above-mentioned camera device can be adjusted, such as adjusting the pitch angle of the camera device, etc. Then, the above-mentioned steps S401-S403 are re-executed using the adjusted camera device until it is determined that the above-mentioned current lane information is not abnormal, and the process can be ended after completing the lane line calibration.
[0074] S202: When there is an abnormality in the target lane line information and the vehicle lane change information is used to indicate that the vehicle is not currently changing lanes, correct the target lane line information based on a preset first correction rule.
[0075] S203: When there is an abnormality in the target lane line information and the vehicle lane change information is used to indicate that the vehicle is currently changing lanes, the target lane line information is corrected based on a preset second correction rule, and the first correction rule is different from the second correction rule.
[0076] After obtaining the above vehicle condition information (specifically, the target lane line information and the vehicle lane change information), the present disclosure can determine whether the above target lane line information is abnormal, determine whether the above vehicle lane information is used to indicate that the vehicle is currently changing lanes, and then perform corresponding lane line correction based on the above judgment results. Among them, the judgment implementation method of whether the above target lane line information is abnormal can be referred to the above Figure 4 The relevant introduction in the embodiment is not repeated here. The specific implementation order of the above two judgments is not limited. For example, the present disclosure can first determine whether the target lane line information is abnormal, and then determine whether the above vehicle lane information is used to indicate the current lane change of the vehicle, etc., and the present disclosure does not limit or elaborate on this. The following is an exemplary introduction to two specific embodiments of the above lane line correction.
[0077] In one embodiment, when there is no abnormality in the target lane line information, no lane line correction may be performed and the process may be terminated; or the target lane line information may be output for the user to view.
[0078] In another embodiment, when the target lane line information is abnormal and the vehicle lane change information indicates that the vehicle is not currently changing lanes, the target lane line information may be corrected based on a preset first correction rule. In this case, the target lane line information may include the main lane line information that the vehicle is currently traveling on before the lane change and / or the lane change lane line information that the vehicle is currently traveling on after the lane change.
[0079] The above-mentioned first correction rule may be a lane line correction rule customized by an electronic device or a user, which may be customized according to actual needs, etc. The present disclosure does not limit the specific implementation method of correcting the target lane line information based on the above-mentioned first correction rule. The following takes a single target lane line information (such as main lane line information or lane change lane line information) as an example to introduce the specific implementation method of its correction, but it does not constitute a limitation. When there are multiple target lane line information, each target lane line information can be corrected and processed according to the following implementation method. Among them, for a single target lane line information, it may include the left lane line information of the target lane and the right lane line information of the target lane, and the specific implementation method of the lane line correction involved may include any of the following:
[0080] In one embodiment, when the left lane line information of the target lane is abnormal, the present disclosure can use the right lane line information of the target lane to correct the left lane line information. In a specific implementation, the present disclosure can use the lane width, slope information and right lane line information of the target lane to compensate and correct the left lane line information of the target lane, thereby obtaining more accurate left lane line information after correction.
[0081] In another embodiment, when the right lane line information of the target lane is abnormal, the present disclosure can use the left lane line information of the target lane to correct the right lane line information. In a specific implementation, for example, the present disclosure can use the lane width, slope information and left lane line information of the target lane to compensate and correct the right lane line information of the target lane, thereby obtaining more accurate right lane line information after correction.
[0082] In another embodiment, when both the left lane line information and the right lane line information of the target lane are abnormal, the present disclosure can use the inferred lane line information of the target lane to respectively correct the left lane line information and the right lane line information of the target lane, wherein the inferred lane line information is obtained by prediction and inference calculation based on the lane line information of the target lane at a historical moment. The present disclosure does not limit the number of lane line information at the historical moment, and generally, the more lane line information at the historical moment, the higher the accuracy of the inferred lane line information obtained by inference.
[0083] For example, please refer to FIG. 6(a), which is a possible lane line reasoning schematic diagram according to an exemplary embodiment. As shown in FIG. 6(a), the present disclosure can obtain the lane line information of the target lane at time t0 to time t30, respectively, and use the lane line information at these times, and use a preset reasoning algorithm for prediction and reasoning calculation to obtain the lane line information at the next time (time t31). The present disclosure does not limit the reasoning algorithm, which may include but is not limited to, for example, a neural network algorithm, a regression analysis algorithm, a gray prediction algorithm or other algorithms for lane line reasoning and prediction. It should be noted that the lane line information of the target lane at each historical moment shown in FIG. 6(a) is only an example, but does not constitute a limitation. In actual applications, it can be updated or replaced according to the actual application situation.
[0084] In practical applications, the present disclosure can use the lane line information of the target lane at each historical moment to continuously / multiple times predict and infer the inferred lane line information at the next moment. When the above inference times (also called prediction times) are less than the preset times, the present disclosure can use the inference lane line information obtained at the current moment to compensate the target lane line information at the current moment; however, when the above inference times are greater than or equal to the preset times, the inference lane line information obtained by inference may have a large error. At this time, the present disclosure can output a warning prompt information, which is used to prompt the vehicle that it is not suitable for automatic driving at present, and prompt the user to perform manual intervention / manual driving, etc.
[0085] In another embodiment, when there is an abnormality in the target lane line information, and the vehicle lane change information is used to indicate that the vehicle is currently changing lanes, the target lane line information can be corrected based on a preset second correction rule. In this case, the target lane line information can include the lane change lane line information that the vehicle is changing lanes into. Among them, the detection / acquisition implementation method of the lane change lane line information is not limited by the present disclosure. For example, it can detect whether there is other lane change lane line information that supports lane change around the main lane line information by taking the corresponding road image by a camera device; or it can detect whether there is other lane change lane line information around the main lane line information by using sensors such as laser radar. The second correction rule can be a lane line correction rule set by an electronic device or a user, which can be customized according to actual needs, etc.; and the first correction rule is different from the second correction rule.
[0086] During implementation, the present disclosure can detect in real time or periodically whether the lane change lane line information needs to be corrected, that is, detect whether there is any abnormality in the lane change lane line information, during the time period corresponding to the lane change of the above-mentioned vehicle. In a specific implementation, during the lane change process of the above-mentioned vehicle, since the vehicle body may be offset at a certain angle relative to the lane line, the present disclosure can further detect whether the lane change lane line information is complete. If it is incomplete, the present disclosure needs to correct the lane change lane line information; otherwise, it can be determined that the lane change lane line does not need to be corrected. When the lane change lane line information needs to be corrected (that is, the lane change lane line has an abnormality), the present disclosure can correct the lane change lane line information based on the second correction rule. Specifically, for example, the lane change lane line information can be corrected using the sensor data of the above-mentioned vehicle. The sensor data includes but is not limited to information such as the lane change angular velocity and / or lane change acceleration of the IMU.
[0087] See also Figure 7 FIG. 1 is a schematic diagram of an IMU lane change according to an exemplary embodiment. Figure 7As shown, the vehicle changes lanes from the position at time T0 to the position at time Tn. The distance between the positions at time T0 and Tn is shown as P, and the angle between the straight line and the horizontal line is θ. The present disclosure can use the following formula (1) to compensate and correct the offset of the vehicle, and then compensate and correct the lane line information of the lane change.
[0088]
[0089] Among them, v t w represents the speed at time t; θ t w represents the angle at time t; P t w represents the position at time t; α and β represent the corrected weight values, which can be determined according to the actual situation, and usually α = 0.3, β = 0.7 by default. As can be seen from the above formula (1), the present disclosure can combine the IMU sensor data and the image sensor of the camera device to compensate the above lane line information to improve the accuracy or precision of the lane line during the lane change process.
[0090] During implementation, the present disclosure does not limit the specific implementation method for correcting the lane line information of the lane change. For example, the present disclosure can also use a lane line reasoning module to perform reasoning correction, etc. In a specific implementation, the present disclosure can obtain historical information of the lane change lane, which includes the lane line information of the lane change lane at a historical moment and the sensor data corresponding to the historical moment (such as IMU angular velocity and / or acceleration information). Then, a preset reasoning algorithm is used to perform lane line reasoning and prediction calculation on the historical information of the lane change lane, so as to obtain the predicted lane line information of the lane change lane. Then, the predicted lane line information is used to correct the lane change lane line information, for example, the lane width and lane slope in the predicted lane line information are used to compensate and correct the lane change lane line information. For the above-mentioned reasoning algorithm, please refer to the relevant introduction in the aforementioned embodiment, which will not be repeated here.
[0091] For example, please refer to FIG6(b), which is another lane line reasoning schematic diagram according to an exemplary embodiment. As shown in FIG6(b), the present disclosure can obtain the lane line information of the lane change lane at time t0 to time t30 and the IMU sensor data at these times, and then use the lane line information and IMU sensor data at these times, and use the preset reasoning algorithm to perform prediction and reasoning calculations to obtain the lane line information at the next moment (time t31). It should be noted that the lane line information and IMU sensor data of the lane change lane at each historical moment shown in FIG6(b) are only examples, but do not constitute a limitation. In actual applications, they can be updated or replaced according to actual application conditions.
[0092] By implementing the disclosed embodiment, the electronic device obtains vehicle condition information, the vehicle condition information includes the target lane line information of the vehicle and the vehicle lane change information; when there is an abnormality in the target lane line information and the vehicle lane change information is used to indicate that the vehicle is not currently changing lanes, the target lane line information is corrected based on a preset first correction rule; when there is an abnormality in the target lane line information and the vehicle lane change information is used to indicate that the vehicle is currently changing lanes, the target lane line information is corrected based on a preset second correction rule, and the first correction rule and the second correction rule are different. It can be seen that the electronic device can intelligently correct the target lane line information with abnormalities using different correction rules under different vehicle conditions to obtain a more accurate corrected lane line, which is conducive to improving the accuracy and safety of vehicle driving. At the same time, it also solves the technical problems existing in the above-mentioned related technologies such as the inability to obtain correct and continuous lane lines, thereby affecting the accurate driving of the vehicle.
[0093] Based on the above examples, see Figure 8 FIG. 1 is a schematic diagram showing the structure of a lane correction device according to an exemplary embodiment. Figure 8 The device shown can be applied to an electronic device, and the device may include an acquisition module 801 and a processing module 802. Among them:
[0094] The acquisition module 801 is configured to acquire vehicle condition information, wherein the vehicle condition information includes target lane line information of the vehicle and vehicle lane change information;
[0095] The processing module 802 is configured to correct the target lane line information based on a preset first correction rule when there is an abnormality in the target lane line information and the vehicle lane change information is used to indicate that the vehicle is not currently changing lanes;
[0096] The processing module 802 is further configured to correct the target lane line information based on a preset second correction rule when there is an abnormality in the target lane line information and the vehicle lane change information is used to indicate that the vehicle is currently changing lanes, and the first correction rule and the second correction rule are different.
[0097] In some embodiments, the target lane line information includes the main lane line information and / or lane change lane line information of the vehicle currently traveling, and the target lane line information includes the left lane line information and the right lane line information of the target lane, and the processing module 802 is configured to perform any one of the following:
[0098] When there is an abnormality in the left lane line information of the target lane, the left lane line information of the target lane is corrected using the right lane line information of the target lane;
[0099] When there is an abnormality in the right lane line information of the target lane, the right lane line information of the target lane is corrected using the left lane line information of the target lane;
[0100] When both the left lane line information and the right lane line information of the target lane are abnormal, the left lane line information and the right lane line information of the target lane are respectively corrected using the inferred lane line information of the target lane, where the inferred lane line information is obtained by predicting and inferring the lane line information of the target lane at a historical moment.
[0101] In some embodiments, the target lane line information includes lane change lane line information, and the processing module 802 is configured to obtain sensor data of the vehicle, wherein the sensor data includes the lane change angular velocity and / or lane change acceleration of the vehicle; and use the sensor data of the vehicle to correct the lane change lane line information.
[0102] In some embodiments, the processing module 802 is configured to perform lane line prediction calculation on historical information of the lane changing lane to obtain predicted lane line information of the lane changing lane, wherein the historical information includes lane line information at historical moments and sensor data at historical moments; and use the predicted lane line information to correct the lane changing lane line information.
[0103] In some embodiments, before obtaining the vehicle condition information, the processing module 802 is also configured to obtain a road image through a camera device, wherein the road image includes current lane line information of the vehicle; perform lane line processing on the road image to obtain the current lane line information; when there is an abnormality in the current lane line information, adjust the camera parameters of the camera device, and re-execute the step of obtaining the road image through the camera device according to the adjusted camera parameters, until there is no abnormality in the current lane line information.
[0104] In some embodiments, the processing module 802 is configured to perform lane line extraction on the road image to obtain initial lane line information; and perform coordinate transformation on the initial lane line information to transform the initial lane line information in the image coordinate system into the current lane line information in the real-world coordinate system.
[0105] Regarding the device in the above embodiment, the specific manner in which each module performs operations has been described in detail in the embodiment of the method, and will not be elaborated here.
[0106] The present disclosure also provides a computer-readable storage medium having computer program instructions stored thereon, and when the program instructions are executed by a processor, the steps of the lane correction method provided by the present disclosure are implemented.
[0107] Fig. 9 9 is a schematic diagram of the structure of an electronic device according to an exemplary embodiment. For example, the electronic device 900 may be a mobile phone, a computer, a digital broadcast terminal, a message transceiver, a game console, a tablet device, a medical device, a fitness device, a personal digital assistant, or other electronic device.
[0108] Reference Fig. 9 , the electronic device 900 may include one or more of the following components: a processing component 902 , a memory 904 , a power component 906 , a multimedia component 908 , an audio component 910 , an input / output interface 912 , a sensor component 914 , and a communication component 916 .
[0109] The processing component 902 generally controls the overall operation of the electronic device 900, such as operations associated with display, phone calls, data communications, camera operations, and recording operations. The processing component 902 may include one or more processors 920 to execute instructions to complete all or part of the steps of the lane correction method described above. In addition, the processing component 902 may include one or more modules to facilitate the interaction between the processing component 902 and other components. For example, the processing component 902 may include a multimedia module to facilitate the interaction between the multimedia component 908 and the processing component 902.
[0110] The memory 904 is configured to store various types of data to support operations on the device 900. Examples of such data include instructions for any application or method operating on the device 900, contact data, phone book data, messages, pictures, videos, etc. The memory 904 can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic memory, flash memory, magnetic disk or optical disk.
[0111] The power supply component 906 provides power to the various components of the device 900. The power supply component 906 can include a power management system, one or more power supplies, and other components associated with generating, managing, and distributing power for the device 900.
[0112] The multimedia component 908 includes a screen that provides an output interface between the device 900 and the user. In some embodiments, the screen may include a liquid crystal display (LCD) and a touch panel (TP). If the screen includes a touch panel, the screen may be implemented as a touch screen to receive input signals from the user. The touch panel includes one or more touch sensors to sense touch, slide, and gestures on the touch panel. The touch sensor may not only sense the boundaries of the touch or slide action, but also detect the duration and pressure associated with the touch or slide operation. In some embodiments, the multimedia component 908 includes a front camera and / or a rear camera. When the electronic device 900 is in an operating mode, such as a shooting mode or a video mode, the front camera and / or the rear camera may receive external multimedia data. Each front camera and rear camera may be a fixed optical lens system or have a focal length and optical zoom capability.
[0113] The audio component 910 is configured to output and / or input audio signals. For example, the audio component 910 includes a microphone (MIC), and when the device 900 is in an operating mode, such as a call mode, a recording mode, and a speech recognition mode, the microphone is configured to receive an external audio signal. The received audio signal can be further stored in the memory 904 or sent via the communication component 916. In some embodiments, the audio component 910 also includes a speaker for outputting audio signals.
[0114] The input / output interface 912 provides an interface between the processing component 902 and the peripheral interface modules, which may be keyboards, click wheels, buttons, etc. These buttons may include but are not limited to: a home button, a volume button, a start button, and a lock button.
[0115] The sensor assembly 914 includes one or more sensors for providing various aspects of status assessment for the device 900. For example, the sensor assembly 914 can detect the open / closed state of the device 900, the relative positioning of components, such as the display and keypad of the device 900, and the sensor assembly 914 can also detect the position change of the device 900 or a component of the device 900, the presence or absence of user contact with the device 900, the orientation or acceleration / deceleration of the device 900, and the temperature change of the device 900. The sensor assembly 914 may include a proximity sensor configured to detect the presence of nearby objects without any physical contact. The sensor assembly 914 may also include an optical sensor, such as a CMOS or CCD image sensor, for use in imaging applications. In some embodiments, the sensor assembly 914 may also include an accelerometer, a gyroscope sensor, a magnetic sensor, a pressure sensor, or a temperature sensor.
[0116] The communication component 916 is configured to facilitate wired or wireless communication between the device 900 and other devices. The device 900 can access a wireless network based on a communication standard, such as WiFi, 2G or 3G, or a combination thereof. In an exemplary embodiment, the communication component 916 receives a broadcast signal or broadcast-related information from an external broadcast management system via a broadcast channel. In an exemplary embodiment, the communication component 916 also includes a near field communication (NFC) module to facilitate short-range communication. For example, the NFC module can be implemented based on radio frequency identification (RFID) technology, infrared data association (IrDA) technology, ultra-wideband (UWB) technology, Bluetooth (BT) technology and other technologies.
[0117] In an exemplary embodiment, device 900 may be implemented by one or more application-specific integrated circuits (ASICs), digital signal processors (DSPs), digital signal processing devices (DSPDs), programmable logic devices (PLDs), field programmable gate arrays (FPGAs), controllers, microcontrollers, microprocessors, or other electronic components to perform the lane correction method described above.
[0118] In an exemplary embodiment, a non-transitory computer-readable storage medium including instructions is also provided, such as a memory 904 including instructions, and the instructions can be executed by a processor 920 of the device 900 to complete the above-mentioned upper lane line correction method. For example, the non-transitory computer-readable storage medium can be a ROM, a random access memory (RAM), a CD-ROM, a magnetic tape, a floppy disk, an optical data storage device, etc.
[0119] In addition to being an independent electronic device, the above-mentioned device can also be a part of an independent electronic device. For example, in one embodiment, the device can be an integrated circuit (IC) or a chip, wherein the integrated circuit can be an IC or a collection of multiple ICs; the chip can include but is not limited to the following types: GPU (Graphics Processing Unit), CPU (Central Processing Unit), FPGA (Field Programmable Gate Array), DSP (Digital Signal Processor), ASIC (Application Specific Integrated Circuit), SOC (System on Chip, SoC), etc. The above-mentioned integrated circuit or chip can be used to execute executable instructions (or codes) to implement the above-mentioned lane correction method. The executable instructions can be stored in the integrated circuit or chip, or can be obtained from other devices or equipment, such as the integrated circuit or chip including a processor, a memory, and an interface for communicating with other devices. The executable instruction may be stored in the memory, and when the executable instruction is executed by the processor, the above-mentioned lane line correction method is implemented; alternatively, the integrated circuit or chip may receive the executable instruction through the interface and transmit it to the processor for execution, so as to implement the above-mentioned lane line correction method.
[0120] In another exemplary embodiment, a computer program product is also provided. The computer program product includes a computer program executable by a programmable device. The computer program has a code portion for executing the above lane correction method when executed by the programmable device.
[0121] See also Fig.10 FIG. 1 is a schematic diagram showing the structure of a chip according to an exemplary embodiment. Fig.10 The chip 1000 shown includes a processor 1001 and an interface 1002. Optionally, it may also include a memory 1003. The number of the processors 1001 may be one or more, and the number of the interfaces 1002 may be multiple.
[0122] In one embodiment, for a chip used to implement the method embodiment of the present disclosure:
[0123] The interface 1002 is used to receive or output signals;
[0124] The processor 1001 is used to execute part or all of the contents of the lane correction method embodiment.
[0125] Understandably, the processor in the embodiment of the present disclosure may be an integrated circuit chip having signal processing capabilities. In the implementation process, each step of the above method embodiment may be completed by an integrated logic circuit of hardware in the processor or an instruction in the form of software. The above processor may be a general-purpose processor, a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field programmable gate array (FPGA) or other programmable logic devices, discrete gates or transistor logic devices, discrete hardware components.
[0126] It is understandable that the memory in the embodiments of the present disclosure may be a volatile memory or a non-volatile memory, or may include both volatile and non-volatile memories. Among them, the non-volatile memory may be a read-only memory (ROM), a programmable read-only memory (PROM), an erasable programmable read-only memory (EPROM), an electrically erasable programmable read-only memory (EEPROM), or a flash memory. The volatile memory may be a random access memory (RAM), which is used as an external cache. By way of example and not limitation, many forms of RAM are available, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), synchronous link DRAM (SLDRAM), and direct RAM (DR RAM). It should be noted that the memory of the systems and methods described herein is intended to include, but is not limited to, these and any other suitable types of memory.
[0127] It should be noted here that the description of the above storage medium, device and chip embodiments is similar to the description of the above method embodiments, and has similar beneficial effects as the method embodiments. For technical details not disclosed in the storage medium, storage medium and device embodiments of the present disclosure, please refer to the description of the method embodiments of the present disclosure for understanding.
[0128] Those skilled in the art will readily appreciate other embodiments of the present disclosure after considering the specification and practicing the present disclosure. The present disclosure is intended to cover any variations, uses or adaptations of the present disclosure that follow the general principles of the present disclosure and include common knowledge or customary techniques in the art that are not disclosed in the present disclosure. The description and examples are to be considered as exemplary only, and the true scope and spirit of the present disclosure are indicated by the following claims.
[0129] It should be understood that the present disclosure is not limited to the exact structures that have been described above and shown in the drawings, and that various modifications and changes may be made without departing from the scope thereof. The scope of the present disclosure is limited only by the appended claims.
Claims
1. A lane correction method, characterized in that: include: Acquiring vehicle condition information, wherein the vehicle condition information includes target lane line information of the vehicle and vehicle lane change information; When there is an abnormality in the target lane line information and the vehicle lane change information is used to indicate that the vehicle is not currently changing lanes, correcting the target lane line information based on a preset first correction rule; When there is an abnormality in the target lane line information and the vehicle lane change information is used to indicate that the vehicle is currently changing lanes, the target lane line information is corrected based on a preset second correction rule, and the first correction rule and the second correction rule are different.
2. The method according to claim 1, characterized in that The target lane line information includes the main lane line information and / or lane line information of the lane change lane currently traveled by the vehicle, and the target lane line information includes the left lane line information and the right lane line information of the target lane, and the correction of the target lane line information based on the preset first correction rule includes any one of the following: When there is an abnormality in the left lane line information of the target lane, the left lane line information of the target lane is corrected using the right lane line information of the target lane; When there is an abnormality in the right lane line information of the target lane, the right lane line information of the target lane is corrected using the left lane line information of the target lane; When both the left lane line information and the right lane line information of the target lane are abnormal, the left lane line information and the right lane line information of the target lane are respectively corrected using the inferred lane line information of the target lane, where the inferred lane line information is obtained by predicting and inferring the lane line information of the target lane at a historical moment.
3. The method according to claim 1, characterized in that The target lane line information includes lane change lane line information, and the correcting the target lane line information based on a preset second correction rule includes: Acquiring sensor data of the vehicle, wherein the sensor data includes a lane change angular velocity and / or a lane change acceleration of the vehicle; The lane change lane line information is corrected using the sensor data of the vehicle.
4. The method according to claim 3, characterized in that The modifying the lane change lane line information by using the sensor data of the vehicle includes: Perform lane line prediction calculation on historical information of a lane change lane to obtain predicted lane line information of the lane change lane, wherein the historical information includes lane line information at historical moments and sensor data at historical moments; The predicted lane line information is used to correct the lane change lane line information.
5. The method according to claim 1, characterized in that: Before obtaining the vehicle condition information, the method further includes: Acquire a road image by a camera device, wherein the road image includes current lane line information of the vehicle; Performing lane line processing on the road image to obtain the current lane line information; When there is an abnormality in the current lane line information, the camera parameters of the camera device are adjusted, and the step of acquiring the road image through the camera device is re-executed according to the adjusted camera parameters until there is no abnormality in the current lane line information.
6. The method according to claim 5, characterized in that The performing lane line processing on the road image to obtain the current lane line information includes: Extracting lane lines from the road image to obtain initial lane line information; The initial lane line information is subjected to coordinate transformation to transform the initial lane line information in the image coordinate system into the current lane line information in the real world coordinate system.
7. A lane correction device, characterized in that: include: An acquisition module is configured to acquire vehicle condition information, wherein the vehicle condition information includes target lane line information of the vehicle and vehicle lane change information; a processing module configured to correct the target lane line information based on a preset first correction rule when there is an abnormality in the target lane line information and the vehicle lane change information is used to indicate that the vehicle is not currently changing lanes; The processing module is further configured to correct the target lane line information based on a preset second correction rule when there is an abnormality in the target lane line information and the vehicle lane change information is used to indicate that the vehicle is currently changing lanes, and the first correction rule and the second correction rule are different.
8. An electronic device, characterized in that: include: processor; a memory for storing processor-executable instructions; The processor is configured to execute the executable instructions to implement the steps of the method according to any one of claims 1 to 6.
9. A computer-readable storage medium having computer program instructions stored thereon, characterized in that: When the computer program instructions are executed by a processor, the steps of the method according to any one of claims 1 to 6 are implemented.
10. A chip, characterized in that: The method comprises a processor and an interface; the processor is used to read instructions to execute the method according to any one of claims 1 to 6.