Hybrid sensing type lane departure assisting method and system for intelligent networked automobile

Abstract

The invention discloses a hybrid sensing type lane departure assisting method and system for an intelligent connected automobile, and the method comprises the steps: obtaining various types of road information of a road through various types of sensors disposed on a road lane line, and extracting the feature vectors of the various types of road information; performing data fusion operation on the feature vectors to generate fused new feature vectors; a lane departure judgment model is set, whether the vehicle deviates from the lane where the vehicle is located or not when the vehicle is driven on the road is judged in combination with the new feature vector, when the vehicle deviates, early warning information is generated, early warning is conducted, related data such as the early warning information are uploaded to a cloud end, and data such as historical early warning information are updated; the information is pushed to the edge computing gateway through OTA upgrading and is issued to the vehicle-mounted terminal; in order to enable the system to be more energy-saving, a piezoelectric power generation module and a photovoltaic module are arranged as power generation modules.

Description

Technical Field

[0001] The present invention belongs to the technical field of assisted driving, and more specifically, relates to a hybrid sensing lane departure assistance method and system for intelligent connected vehicles. Background Technique

[0002] With the development of intelligent connected vehicles towards higher-level autonomous driving, lane safety control in mixed driving scenarios has become a key bottleneck for technology implementation. Currently, there are significant differences in decision-making logic and control accuracy between autonomous driving vehicles and human-driven vehicles, which poses a dual challenge to lane keeping and departure warning systems: they need to adapt to the uncertainties of human drivers and achieve high-precision lane positioning in complex traffic environments. However, the accuracy of existing lane departure warning systems on highways or in complex environments cannot meet the sub-meter requirement. Especially under low visibility conditions such as haze and snow days, the perception accuracy of the system is greatly reduced. At this time, a single sensor (such as a camera or radar) is easily affected by the environment, resulting in increased errors in lane line recognition and vehicle positioning. Haze weather will seriously hinder the viewing distance of the camera, and snow days may cover road signs and lane lines, greatly reducing the reliability and robustness of the perception system. Under these extreme weather conditions, the existing lane departure warning systems cannot effectively suppress perception errors, seriously affecting the accuracy and safety of the system. Summary of the Invention

[0003] To solve the above technical problems, the present invention proposes a hybrid sensing lane departure assistance method for intelligent connected vehicles, including:

[0004] Obtaining various road information of the road through various sensors arranged on the road lane lines, and extracting feature vectors of the various road information;

[0005] Performing a data fusion operation on the feature vectors to generate a new fused feature vector;

[0006] Setting a lane departure judgment model, and combining with the new feature vector to judge whether the vehicle deviates from the lane where it is traveling when the vehicle is driving on the road. When the vehicle deviates, generating a warning message and giving a warning.

[0007] Further, performing the data fusion operation on the feature vectors further includes: allocating attention weights to each feature vector through an attention mechanism, and then performing the data fusion operation.

[0008] Further, the lane departure judgment model is a deep learning model, which is a convolutional neural network or a recurrent neural network:

[0009] Further, generating a warning message and giving a warning includes: sending the warning message to a terminal device on the vehicle, where the terminal device includes a car machine and / or a HUD display device.

[0010] Further, the lane departure judgment model is:

[0011] Δd(t)=exp(-λ′(S fused (t)-v(t)-a(t)-(θ1·tan(θ(t))+θ2·sin 2 (θ(t)))))

[0012] where Δd(t) is the lane departure index at time t, λ′ is the first adjustment factor of the lane departure judgment model, S fused (t) is the new feature vector at time t, v(t) is the speed of the vehicle at time t, a(t) is the acceleration of the vehicle at time t, θ1 is the second adjustment factor of the lane departure judgment model, θ(t) is the steering angle of the vehicle at time t, and θ2 is the third adjustment factor of the lane departure judgment model.

[0013] Further, the new feature vector S fused (t) at time t is:

[0014]

[0015] where n is the number of sensors, λ i is the weight of the i-th sensor, T is the value of the time window, S i (k) is the feature vector of the i-th sensor at time k, f i (S i (t)) is the change function of the feature vector S i (t) of the i-th sensor at time t.

[0016] Further, the change function f i (S i (t)) of the feature vector S i (t) of the i-th sensor at time t includes:

[0017]

[0018] where τ i is the feature scale of the i-th sensor, λ" i′ is the weight of the i′-th weather feature vector, weather i′ (k) is the i′-th weather feature vector at time k, weaher i′ (t) is the i′-th weather feature vector at time t.

[0019] Further, the lane departure index Δd(t) at time t is normalized to finally obtain the lane departure distance. Among them, the average deviation distance of the vehicle is obtained, and the result of multiplying it by the lane departure index Δd(t) at time t is used as the lane departure distance.

[0020] Further, when the lane departure distance exceeds a preset deviation distance, it is determined that the vehicle has deviated, and a warning message is generated and a warning is issued.

[0021] The present invention also proposes a hybrid sensing lane departure assistance system for intelligent connected vehicles, including:

[0022] A feature vector extraction module, configured to obtain various road information of the road through a variety of sensors arranged on the road lane lines, and extract the feature vectors of the various road information;

[0023] A fusion module, configured to perform data fusion operations on the feature vectors to generate a new feature vector after fusion;

[0024] A warning module, configured to set a lane departure judgment model, and combine the new feature vector to judge whether the vehicle deviates from the lane where it is traveling when the vehicle is driving on the road. When the vehicle deviates, a warning message is generated and a warning is issued.

[0025] Generally speaking, compared with the prior art through the above technical solutions conceived by the present invention, the following beneficial effects are obtained:

[0026] Through the feature weighted fusion algorithm and the lane departure judgment model of the present invention, the dynamic trajectory of the vehicle can be identified in real time. This algorithm can effectively suppress noise signals in complex environments and improve the accuracy and robustness of lane departure warnings. Description of the Drawings

[0027] Figure 1 is the system structure diagram of Embodiment 2 of the present invention;

[0028] Figure 2 is the overall schematic diagram of the present invention;

[0029] Figure 3 is the flow schematic diagram of Embodiment 1 of the present invention;

[0030] Figure 4 is the module schematic diagram of Embodiment 2 of the present invention. Detailed Embodiments

[0031] In order to better understand the above technical solutions, the above technical solutions will be described in detail below in conjunction with the accompanying drawings of the specification and specific embodiments.

[0032] The method provided by the present invention can be implemented in the following terminal environment. The terminal may include one or more of the following components: a processor, a storage medium, and a display screen. Among them, at least one instruction is stored in the storage medium, and the instruction is loaded and executed by the processor to implement the method described in the following embodiments.

[0033] The processor may include one or more processing cores. The processor uses various interfaces and lines to connect various parts within the entire terminal, and by running or executing instructions, programs, code sets, or instruction sets stored in the storage medium, and calling data stored in the storage medium, it executes various functions of the terminal and processes data.

[0034] The storage medium may include a random access memory (RAM), and may also include a read-only memory (ROM). The storage medium can be used to store instructions, programs, codes, code sets, or instructions.

[0035] The display screen is used to display the user interfaces of various application programs.

[0036] In addition, those skilled in the art can understand that the structure of the above terminal does not constitute a limitation on the terminal. The terminal may include more or fewer components, or combine certain components, or have different component arrangements. For example, the terminal may also include components such as a radio frequency circuit, an input unit, a sensor, an audio circuit, and a power supply, which will not be elaborated here.

[0037] Embodiment 1

[0038] Such as Figure 3 , this embodiment proposes a hybrid sensing-based lane departure assistance method for intelligent connected vehicles, including:

[0039] Step 1: Install an intelligent road detection unit on the road lane line and install an edge computing gateway above the road;

[0040] Step 2: Obtain various road information of the road through the hybrid sensor array of the on-road detection unit and upload it to the edge computing gateway. Among them, the hybrid sensor array includes various sensors arranged on the road lane line, and the various road information includes road data such as geomagnetic data, road pressure data, road vibration frequency, road slope data, etc., and environmental data such as humidity data and temperature data;

[0041] Step 3: The edge computing gateway respectively extracts the features of different sensor data, including geomagnetic field changes, vibration frequency, pressure changes, road slope changes, humidity changes, and temperature fluctuations, etc.;

[0042] Step 4: Using an algorithm based on the attention mechanism, weight each feature according to its importance, and perform a data fusion operation on the feature vectors to generate a new fused feature vector;

[0043] Step 5: Set up a lane departure judgment model, and combine the new feature vector to determine whether the vehicle deviates from its lane during road driving. When the vehicle deviates, generate a warning message and issue a warning;

[0044] Specifically, setting up the lane departure judgment model includes:

[0045] Analyze the fused feature vector through a deep learning model or a rule engine to determine whether a lane departure occurs and generate a warning.

[0046] To make the lane departure judgment model more accurate, this embodiment also designs the following lane departure judgment model:

[0047] Specifically, the lane departure judgment model is:

[0048] Δd(t) = exp(-λ′(S fused (t) - v(t) - a(t) - (θ1·tan(θ(t)) + θ2·sin 2 (θ(t)))))

[0049] where Δd(t) is the lane departure index at time t, λ′ is the first adjustment factor of the lane departure judgment model, S fused (t) is the new feature vector at time t, v(t) is the speed of the vehicle at time t, a(t) is the acceleration of the vehicle at time t, θ1 is the second adjustment factor of the lane departure judgment model, θ(t) is the steering angle of the vehicle at time t, and θ2 is the third adjustment factor of the lane departure judgment model. Among them, the lane departure index Δd(t) at time t is a dimensionless value, and the dimensions can be removed by normalizing the physical parameters on the right side of the formula.

[0050] Specifically, the new feature vector S fused (t) at time t is:

[0051]

[0052] where n is the number of sensors, λ i is the weight of the i-th sensor, T is the value of the time window, S i (k) is the feature vector of the i-th sensor at time k, f i (S i (t)) is the change function of the feature vector S i (t) of the i-th sensor at time t.

[0053] Specifically, the feature vector S of the i-th sensor at time t i (t) has a variation function f i (S i (t)) includes:

[0054]

[0055] Where τ i is the feature scale of the i-th sensor (this feature scale represents different precisions of the same sensor. For example, if a geomagnetic sensor acquires data once every 1 second, once every 3 seconds, and once every 10 seconds, then this geomagnetic sensor has 3 scales), and λ″ i′ is the weight of the i'-th weather feature vector, weather i′ (k) is the i'-th weather feature vector at time k, and weather i′ (t) is the i'-th weather feature vector at time t.

[0056] Specifically, the lane departure index Δd(t) at time t is normalized to finally obtain the lane departure distance. Among them, the average deviation distance of the vehicle is obtained, and the result of multiplying it by the lane departure index Δd(t) at time t is used as the lane departure distance.

[0057] Specifically, when the lane departure distance exceeds the preset departure distance, it is determined that the vehicle has deviated, and a warning message is generated and a warning is issued.

[0058] Step Six: Generate a warning message according to the decision result of Step Five, and issue a warning to the driver through the in-vehicle terminal's car machine and / or augmented reality HUD. At the same time, the data is transmitted back to the cloud platform for optimization learning.

[0059] Step Seven: The cloud platform analyzes the received real-time data, optimizes the decision-making algorithm, and pushes the optimization result to the edge computing gateway through OTA upgrade. The edge computing gateway distributes the system optimization strategy to each device.

[0060] Embodiment 2

[0061] As Figure 1 and Figure 2As shown in the figure, this embodiment proposes a hybrid sensing lane departure assistance system for intelligent connected vehicles. This system is deployed in the cloud management platform and obtains various road information through intelligent road detection units. The intelligent road detection unit also includes a piezoelectric power generation module (piezoelectric sensor) and a photovoltaic module (solar panel) for providing electric energy, uploading relevant data such as warning information to the cloud, updating data such as historical warning information, and pushing it to the edge computing gateway through OTA upgrade and then sending it to the in-vehicle terminal.

[0062] As Figure 4 shown, the following modules are specifically set and deployed in the intelligent road detection unit, edge computing gateway, in-vehicle terminal or cloud management platform, as follows:

[0063] Feature vector extraction module, which is used to obtain various road information of the road through various sensors set on the road lane lines and extract the feature vectors of the various road information;

[0064] Fusion module, which is used to perform data fusion operations on the feature vectors to generate new feature vectors after fusion;

[0065] Specifically, performing data fusion operations on the feature vectors further includes: assigning attention weights to each of the feature vectors through an attention mechanism and then performing data fusion operations.

[0066] Warning module, which is used to set a lane departure judgment model and, in combination with the new feature vectors, judge whether the vehicle deviates from its lane when driving on the road. When the vehicle deviates, generate a warning message and issue a warning.

[0067] Specifically, the lane departure judgment model is a deep learning model, which is a convolutional neural network or a recurrent neural network.

[0068] Specifically, generating a warning message and issuing a warning includes: sending the warning message to the terminal device on the vehicle, where the terminal device includes a car machine and / or a HUD display device.

[0069] In order to make the lane departure judgment model more accurate, this embodiment also designs the following lane departure judgment model:

[0070] Specifically, the lane departure judgment model is:

[0071] Δd(t) = exp(-λ′(S fused (t) - v(t) - a(t) - (θ1·tan(θ(t)) + θ2·sin 2 (θ(t))))))

[0072] Among them, Δd(t) is the lane departure index at time t, λ′ is the first adjustment factor of the lane departure judgment model, S fused (t) is the new feature vector at time t, v(t) is the speed of the vehicle at time t, a(t) is the acceleration of the vehicle at time t, θ1 is the second adjustment factor of the lane departure judgment model, θ(t) is the steering angle of the vehicle at time t, and θ2 is the third adjustment factor of the lane departure judgment model.

[0073] Specifically, the new feature vector S fused (t) at time t is:

[0074]

[0075] Among them, n is the number of sensors, λ i is the weight of the i-th sensor, T is the value of the time window, S i (k) is the feature vector of the i-th sensor at time k, f i (S i (t)) is the change function of the feature vector S i (t) of the i-th sensor at time t.

[0076] Specifically, the change function f i of the feature vector S i (t) of the i-th sensor at time t i includes:

[0077]

[0078] Among them, τ i is the feature scale of the i-th sensor, λ" i′ is the weight of the i′-th weather feature vector, weather i′ (k) is the i′-th weather feature vector at time k, weather i′ (t) is the i′-th weather feature vector at time t.

[0079] Specifically, the lane departure index Δd(t) at time t is normalized to finally obtain the lane departure distance. Among them, the average deviation distance of the vehicle is obtained, and the result of multiplying it by the lane departure index Δd(t) at time t is used as the lane departure distance.

[0080] Specifically, when the lane departure distance exceeds the preset deviation distance, it is determined that the vehicle has deviated, and a warning message is generated and a warning is issued.

[0081] Embodiment 3

[0082] An embodiment of the present invention also provides a storage medium storing multiple instructions for implementing the hybrid sensing lane departure assistance method for intelligent connected vehicles as described above.

[0083] Optionally, in this embodiment, the above storage medium may be located in any computer terminal in a computer terminal group in a computer network or in any mobile terminal in a mobile terminal group.

[0084] Optionally, in this embodiment, the storage medium is configured to store program code for performing the following steps: Step 1: Install an intelligent road detection unit on the road lane line and install an edge computing gateway above the road;

[0085] Step 2: Obtain various road information of the road through the hybrid sensor array of the on-road detection unit and upload it to the edge computing gateway. Among them, the hybrid sensor array includes various sensors arranged on the road lane line, and the various road information includes road data such as geomagnetic data, road pressure data, road vibration frequency, road slope data, etc., and environmental data such as humidity data and temperature data;

[0086] Step 3: The edge computing gateway extracts the features of different sensor data respectively, including geomagnetic field changes, vibration frequency, pressure changes, road slope changes, humidity changes, and temperature fluctuations, etc.;

[0087] Step 4: Use an algorithm based on the attention mechanism to weight each feature according to its importance and perform a data fusion operation on the feature vectors to generate a new feature vector after fusion;

[0088] Step 5: Set a lane departure judgment model and, in combination with the new feature vector, judge whether the vehicle deviates from the lane where it is traveling when driving on the road. When the vehicle deviates, generate a warning message and give a warning;

[0089] Specifically, setting the lane departure judgment model includes:

[0090] Analyze the fused feature vector through a deep learning model or a rule engine to judge whether a lane departure occurs and generate a warning.

[0091] In order to make the lane departure judgment model more accurate, this embodiment also designs the following lane departure judgment model:

[0092] Specifically, the lane departure judgment model is:

[0093] Δd(t)=exp(-λ′(S fused (t)-v(t)-a(t)-(θ1·tan(θ(t))+θ2·sin 2 (θ(t)))))

[0094] Among them, Δd(t) is the lane departure index at time t, λ′ is the first adjustment factor of the lane departure judgment model, S fused (t) is the new feature vector at time t, v(t) is the speed of the vehicle at time t, a(t) is the acceleration of the vehicle at time t, θ1 is the second adjustment factor of the lane departure judgment model, θ(t) is the steering angle of the vehicle at time t, θ2 is the third adjustment factor of the lane departure judgment model. Among them, the lane departure index Δd(t) at time t is a dimensionless value, and the dimension can be removed by normalizing the physical parameters on the right side of the formula.

[0095] Specifically, the new feature vector S fused (t) at time t is:

[0096]

[0097] Among them, n is the number of sensors, λ i is the weight of the i-th sensor, T is the value of the time window, S i (k) is the feature vector of the i-th sensor at time k, f i (S i (t)) is the change function of the feature vector S i (t) of the i-th sensor at time t.

[0098] Specifically, the change function f i (S i (t)) of the feature vector S i (t) of the i-th sensor at time t includes:

[0099]

[0100] Among them, τ i is the feature scale of the i-th sensor, λ″ i′ is the weight of the i′-th weather feature vector, weather i′ (k) is the i′-th weather feature vector at time k, weather i′ (t) is the i′-th weather feature vector at time t.

[0101] Specifically, the lane departure index Δd(t) at time t is normalized to finally obtain the lane departure distance. Among them, the average deviation distance of the vehicle is obtained, and the result of multiplying it by the lane departure index Δd(t) at time t is used as the lane departure distance.

[0102] Specifically, when the lane departure distance exceeds a preset deviation distance, it is determined that the vehicle has deviated, and a warning message is generated and a warning is issued.

[0103] Step Six: Generate a warning message according to the decision result in Step Five, and issue a warning to the driver through the in-vehicle terminal's car computer and / or augmented reality HUD. At the same time, the data is transmitted back to the cloud platform for optimization learning.

[0104] Step Seven: The cloud platform analyzes the received real-time data, optimizes the decision-making algorithm, and pushes the optimization result to the edge computing gateway through OTA upgrade. The edge computing gateway distributes the system optimization strategy to each device.

[0105] Embodiment 4

[0106] The embodiment of the present invention also proposes an electronic device, including a processor and a storage medium connected to the processor. The storage medium stores multiple instructions, and the instructions can be loaded and executed by the processor so that the processor can execute the hybrid sensing-based lane departure assistance method for intelligent connected vehicles.

[0107] Specifically, the electronic equipment in this embodiment can be a computer terminal, and the computer terminal can include: one or more processors, and a storage medium.

[0108] Among them, the storage medium can be used to store software programs and modules, such as the hybrid sensing-based lane departure assistance method for intelligent connected vehicles in the embodiment of the present invention, the corresponding program instructions / modules. The processor executes various functional applications and data processing by running the software programs and modules stored in the storage medium, that is, implements the above-mentioned hybrid sensing-based lane departure assistance method for intelligent connected vehicles. The storage medium can include a high-speed random access storage medium, and can also include a non-volatile storage medium, such as one or more magnetic storage systems, flash memory, or other non-volatile solid-state storage media. In some instances, the storage medium can further include a storage medium remotely set relative to the processor, and these remote storage media can be connected to the terminal through a network. Examples of the above network include but are not limited to the Internet, enterprise intranet, local area network, mobile communication network and their combinations.

[0109] The processor can call the information and application programs stored in the storage medium through the transmission system to execute the following steps: Step One: Install an intelligent road detection unit on the road lane line and install an edge computing gateway above the road;

[0110] Step 2: Obtain various road information of the road through the hybrid sensor array of the on-road detection unit and upload it to the edge computing gateway. Among them, the hybrid sensor array includes various sensors set on the road lane lines, and the various road information includes road data such as geomagnetic data, road pressure data, road vibration frequency, road slope data, etc., and environmental data such as humidity data and temperature data;

[0111] Step 3: The edge computing gateway extracts the features of different sensor data respectively, including geomagnetic field changes, vibration frequency, pressure changes, road slope changes, humidity changes, and temperature fluctuations, etc.;

[0112] Step 4: Use an algorithm based on the attention mechanism to weight each feature according to its importance and perform a data fusion operation on the feature vectors to generate a new fused feature vector;

[0113] Step 5: Set up a lane departure judgment model, and combine the new feature vector to judge whether the vehicle deviates from its lane when driving on the road. When the vehicle deviates, generate a warning message and give a warning;

[0114] Specifically, setting up the lane departure judgment model includes:

[0115] Analyze the fused feature vector through a deep learning model or a rule engine to judge whether a lane departure occurs and generate a warning.

[0116] In order to make the lane departure judgment model more accurate, this embodiment also designs the following lane departure judgment model:

[0117] Specifically, the lane departure judgment model is:

[0118] Δd(t)=exp(-λ′(S fused (t)-v(t)-a(t)-(θ1·tan(θ(t))+θ2·sin 2 (θ(t)))))

[0119] Where, Δd(t) is the lane departure index at time t, λ′ is the first adjustment factor of the lane departure judgment model, S fused (t) is the new feature vector at time t, v(t) is the vehicle speed at time t, a(t) is the vehicle acceleration at time t, θ1 is the second adjustment factor of the lane departure judgment model, θ(t) is the vehicle steering angle at time t, θ2 is the third adjustment factor of the lane departure judgment model. Among them, the lane departure index Δd(t) at time t is a dimensionless value, and the dimension can be removed by normalizing the physical parameters on the right side of the formula.

[0120] Specifically, the new feature vector S at time tfused (t) is:

[0121]

[0122] where n is the number of sensors, λ i is the weight of the i-th sensor, T is the value of the time window, and S i (k) is the feature vector of the i-th sensor at time k, and f i (S i (t)) is the change function of the feature vector S of the i-th sensor at time t i (t).

[0123] Specifically, the change function f i (S i (t)) of the feature vector S of the i-th sensor at time t includes: i

[0124]

[0125] where τ i is the feature scale of the i-th sensor, λ" i′ is the weight of the i'-th weather feature vector, and weather i′ (k) is the i'-th weather feature vector at time k, and weather i′ (t) is the i'-th weather feature vector at time t.

[0126] Specifically, the lane departure index Δd(t) at time t is normalized to finally obtain the lane departure distance. Among them, the average deviation distance of the vehicle is obtained, and the result of multiplying it by the lane departure index Δd(t) at time t is used as the lane departure distance.

[0127] Specifically, when the lane departure distance exceeds the preset deviation distance, it is determined that the vehicle has deviated, and a warning message is generated and a warning is issued.

[0128] Step Six: Generate a warning message according to the decision result of Step Five, and issue a warning to the driver through the in-vehicle terminal's car machine and / or the augmented reality HUD. At the same time, the data is transmitted back to the cloud platform for optimization learning.

[0129] Step Seven: The cloud platform analyzes the received real-time data, optimizes the decision-making algorithm, and pushes the optimization result to the edge computing gateway through OTA upgrade. The edge computing gateway distributes the system optimization strategy to each device.

[0130] The serial numbers of the above embodiments of the present invention are only for description and do not represent the advantages and disadvantages of the embodiments.

[0131] In the above embodiments of the present invention, the descriptions of the various embodiments each have their own focuses. For the parts not detailed in a certain embodiment, reference may be made to the relevant descriptions of other embodiments.

[0132] In several embodiments provided by the present invention, it should be understood that the disclosed technical content can be implemented in other ways. Among them, the system embodiments described above are merely illustrative. For example, the division of the units is only a logical function division. In actual implementation, there may be other division methods. For example, multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the displayed or discussed couplings or direct couplings or communication connections to each other can be through some interfaces. The indirect couplings or communication connections of the units or modules can be in an electrical or other form.

[0133] The units described as separate components may or may not be physically separated. The components displayed as units may or may not be physical units, that is, they can be located in one place or distributed to multiple network units. Some or all of the units can be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0134] In addition, the functional units in the various embodiments of the present invention can be integrated into one processing unit, or each unit can exist physically alone, or two or more units can be integrated into one unit. The above-mentioned integrated units can be implemented in the form of hardware or in the form of software functional units.

[0135] If the above-mentioned integrated units are implemented in the form of software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. The computer software product is stored in a storage medium and includes several instructions for causing a computer device (which can be a personal computer, a server, or a network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of the present invention. The foregoing storage medium includes: USB flash drives, read-only storage media (ROM, Read-Only Memory), random access storage media (RAM, Random Access Memory), mobile hard disks, magnetic disks, or optical discs and other various media that can store program codes.

[0136] Obviously, the above embodiments are merely examples given for clear illustration and not limitations on the implementation manners. For those of ordinary skill in the art, other different forms of changes or modifications can be made based on the above description. It is not necessary and impossible to list all implementation manners here. And the obvious changes or modifications derived therefrom still fall within the protection scope of the present invention.

Claims

1. A hybrid sensing-based lane departure assistance method for intelligent connected vehicles, characterized in that Including: Obtain various road information of the road through a variety of sensors set on the road lane lines, and extract the feature vectors of the various road information; Perform a data fusion operation on the feature vectors to generate a new fused feature vector; Set a lane departure judgment model, and combine the new feature vector to judge whether the vehicle deviates from the lane when driving on the road. When the vehicle deviates, generate a warning message and give a warning.

2. The hybrid sensing-based lane departure assistance method for intelligent connected vehicles according to claim 1, wherein, Performing the data fusion operation on the feature vectors further includes: assigning attention weights to each of the feature vectors through an attention mechanism, and then performing the data fusion operation.

3. The hybrid sensing-based lane departure assistance method for intelligent connected vehicles according to claim 1, characterized in that The lane departure judgment model is a deep learning model, which is a convolutional neural network or a recurrent neural network.

4. The hybrid sensing-based lane departure assistance method for intelligent connected vehicles according to claim 1, wherein, Generating a warning message and giving a warning includes: sending the warning message to a terminal device on the vehicle, where the terminal device includes a car machine and / or a HUD display device.

5. A hybrid sensing-based lane departure assistance method for intelligent connected vehicles according to claim 1, characterized in that, The lane departure judgment model is: Δd(t) = exp(-λ′(S fused (t) - v(t) - a(t) -(θ1·tan(θ(t))+θ2·sin 2 (θ(t))))) Among them, Δd(t) is the lane departure index at time t, λ′ is the first adjustment factor of the lane departure judgment model, S fused (t) is the new feature vector at time t, v(t) is the speed of the vehicle at time t, a(t) is the acceleration of the vehicle at time t, θ1 is the second adjustment factor of the lane departure judgment model, θ(t) is the steering angle of the vehicle at time t, and θ2 is the third adjustment factor of the lane departure judgment model.

6. The hybrid sensing lane departure assistance method for intelligent connected vehicles according to claim 5, characterized in that, The new feature vector S at time t fused (t) is as follows: where n is the number of sensors, λ i is the weight of the i-th sensor, T is the value of the time window, S i (k) is the feature vector of the i-th sensor at time k, f i (S i (t)) is the change function of the feature vector S i (t) at time t.

7. The hybrid sensing-based lane departure assistance method for intelligent connected vehicles according to claim 6, characterized in that, The feature vector S of the i-th sensor at time t i The variation function f of (t) i (S i (t)) includes: Among them, τ i is the characteristic scale of the i-th sensor, λ″ i′ is the weight of the i'-th weather feature vector, weather i′ (k) is the i'-th weather feature vector at time k, weather i′ (t) is the i'-th weather feature vector at time t.

8. The hybrid sensing lane departure assistance method for intelligent connected vehicles according to claim 5, characterized in that Normalize the lane departure index Δd(t) at time t to finally obtain the lane departure distance, where the average deviation distance of the vehicle is obtained, and the result of multiplying it by the lane departure index Δd(t) at time t is used as the lane departure distance.

9. The hybrid sensing-based lane departure assistance method for intelligent connected vehicles according to claim 8, wherein When the lane departure distance exceeds a preset departure distance, it is determined that the vehicle deviates, and a warning message is generated and a warning is given.

10. A hybrid sensing lane departure assist system for intelligent connected vehicles, characterized in that Including: A feature vector extraction module, configured to obtain various road information of the road through a variety of sensors set on the road lane lines, and extract the feature vectors of the various road information; A fusion module, configured to perform a data fusion operation on the feature vectors to generate a new fused feature vector; A warning module, configured to set a lane departure judgment model, and combine the new feature vector to judge whether the vehicle deviates from the lane when driving on the road. When the vehicle deviates, generate a warning message and give a warning.

Images