Vision and high-precision map lane line fusion method and system and medium

Abstract

The invention relates to a vision and high-precision map lane line fusion method and system and a medium, and the method comprises the steps: S1, enabling a vehicle to run on a road, obtaining the image data information of the road in real time based on a vehicle-mounted camera, and obtaining the map data information of a current position in real time based on a vehicle-mounted high-precision map; and S2, based on the image data information of the road, identifying and classifying the image of the road by using an improved convolutional neural network algorithm based on regional analysis to obtain image data information of the lane line of the road. According to the method, the stability of lane line output is improved, vision is combined with a map, the redundancy is greatly improved, and more reliable input is provided for vehicle positioning and planning decision making.

Description

Technical Field

[0001] The present invention relates to the field of visual recognition technology, and in particular to a method, system and medium for fusing vision with lane lines of a high-precision map. Background Art

[0002] With the development of the modern automobile industry, intelligent driving systems have become the development trend of the current automobile industry, and more and more fields have begun to use unmanned driving. Lane detection, as a key technology in the field of intelligent driving, plays an important role in advanced driver assistance systems (ADAS) such as lane departure warning (LDW) and lane keeping (LK), lane change (LC) and forward collision warning (FCW), adaptive cruise control (ACC), etc. Vision-based vehicle navigation uses cameras to detect lane markings for lane tracking. This method will deviate in bad weather or when the lane lines are very complex. Radar and lidar can cooperate with the above technologies, but can only decompose road boundaries, which is less efficient under multi-lane conditions. Summary of the invention

[0003] In view of the above problems, the present invention provides a method, system and medium for fusing lane lines of vision and high-precision maps, which not only improves the stability of lane line output, combines vision with maps, greatly improves redundancy, but also provides a more reliable input for vehicle positioning and planning decisions.

[0004] In order to achieve the above-mentioned purpose and other related purposes, the technical solution provided by the present invention is as follows: a method for fusing lane lines of a vision map with a high-precision map, the method comprising:

[0005] S1. The vehicle is driving on the road, and the image data information of the road is obtained in real time based on the on-board camera, and the map data information of the current position is obtained in real time based on the on-board high-precision map;

[0006] S2. Based on the image data information of the road, an improved convolutional neural network algorithm based on regional analysis is used to identify and classify the image of the road to obtain the image data information of the lane line of the road;

[0007] S3. Based on the image data information of the lane line of the road and the map data information of the current position, a multimodal feature fusion algorithm based on a recurrent neural network is used to fuse the image and map data of the lane line to obtain fused data information of the lane line;

[0008] S4. Based on the fused lane line data information, the lane line is optimized using an adaptive particle swarm optimization algorithm based on chaotic mapping to obtain optimized lane line data information.

[0009] Furthermore, in step S2, the use of an improved convolutional neural network algorithm based on regional analysis to identify and classify road images includes:

[0010] S21. Based on the image data information of the road, establish an image area division function Q,

[0011] Wherein, x is the image data information of the road, α1, α2 and α3 are adaptive adjustment factors of road image segmentation, and the image of the road is segmented to obtain the image data information of the segmented road;

[0012] S22. Input the image data information of the divided road into a convolutional neural network for training and learning, and determine the neuron function W,

[0013]

[0014] Among them, y is the image data information of the divided road, β1, β2 and β3 are the weight factors of the model, and the trained convolutional neural network model is obtained;

[0015] S23. Based on the trained convolutional neural network model, the image data information of the road is input, the image of the road is identified and classified, and the image data information of the lane lines of the road is obtained.

[0016] Furthermore, the adaptive adjustment factors α1, α2 and α3 of the road image segmentation are:

[0017]

[0018]

[0019]

[0020] Among them, x is the image data information of the road, and x0 is the mean point of the road image.

[0021] Furthermore, the constraints of the weight factors β1, β2 and β3 of the model are:

[0022]

[0023] Furthermore, in step S3, the adopting of a multimodal feature fusion algorithm based on a recurrent neural network to fuse the lane line image and map data includes:

[0024] S31. Inputting the image data information of the lane line of the road into a recurrent neural network for training and learning, constructing an image feature extraction function R of the network,

[0025]

[0026] Wherein, z is the image data information of the lane line of the road, δ1, δ2 and δ3 are image feature extraction factors, and feature extraction is performed on the image of the lane line of the road to obtain feature data information of the image of the lane line of the road;

[0027] S32. Inputting the map data information of the current location into a recurrent neural network for training and learning, constructing a network map feature extraction function P,

[0028]

[0029] Wherein, a is the map data information of the current location, γ1, γ2 and γ3 are map feature extraction factors, and feature extraction is performed on the map data of the current location to obtain feature data information of the map of the current location;

[0030] S33. Based on the feature data information of the current position map and the feature data information of the lane line image of the road, a multimodal feature fusion function G of the lane line is established,

[0031]

[0032] Among them, b is the feature data information of the current location map, c is the feature data information of the image of the lane line of the road, η1, η2 and η3 are feature fusion factors, and the lane line image and map data are fused to obtain the fused lane line data information.

[0033] Furthermore, the constraint function f1 of the map feature extraction factors γ1, γ2 and γ3 is,

[0034]

[0035] The constraint function f2 of the image feature extraction factors δ1, δ2 and δ3 is,

[0036]

[0037] Among them, the value range of constraint function f1 is (1,2), and the value range of constraint function f2 is (2,3).

[0038] Furthermore, in step S4, the optimization of the lane line using the adaptive particle swarm optimization algorithm based on chaotic mapping includes:

[0039] S41. Based on the fused lane line data information, establish a chaotic mapping function H of the lane line,

[0040]

[0041] Among them, g is the data information of the fused lane line, θ and ρ are the chaos adjustment factors of the lane line, and the chaotic value of the lane line is characterized to obtain the chaotic value data information of the lane line;

[0042] S42. Based on the chaotic value data information of the lane line, the particle population is initialized, the maximum number of iterations is determined, and the initialized particle population data information is obtained;

[0043] S43. Based on the initialized particle population data information, establish the fitness function L of the population individuals,

[0044]

[0045] Among them, h is the data information of the particle population after initialization, ρ1, ρ2 and ρ3 are the fitness control parameters of the individual particle population, and the fitness value of the individual particle population is calculated to obtain the fitness value data information of the individual particle population;

[0046] S44. Based on the fitness value data information of the individual particle population, establish the target optimization function Q of the particle population,

[0047]

[0048] Among them, v is the fitness value data information of the individual particle population, ω1, ω2 and ω3 are optimization factors, and the lane line is optimized to obtain the data information of the optimized lane line.

[0049] Furthermore, the constraints of the fitness control parameters ρ1, ρ2 and ρ3 of the individual particle population are:

[0050]

[0051] In order to achieve the above-mentioned purpose and other related purposes, the present invention also provides a system for fusing lane lines of a visual map with a high-precision map, comprising a computer device, which is programmed or configured to execute the steps of any one of the methods for fusing lane lines of a visual map with a high-precision map.

[0052] In order to achieve the above-mentioned purpose and other related purposes, the present invention also provides a computer-readable storage medium, which stores a computer program programmed or configured to execute any one of the methods for fusing lane lines of visual and high-precision maps.

[0053] The present invention has the following positive effects:

[0054] 1. The present invention adopts an improved convolutional neural network algorithm based on regional analysis to identify and classify road images, and combines a multimodal feature fusion algorithm based on a recurrent neural network to fuse lane line images and map data. It can not only fuse visual data with high-precision map data, but also improve the stability of lane line output. The combination of vision and map greatly improves redundancy and provides a more reliable input for vehicle positioning and planning decisions.

[0055] 2. The present invention optimizes lane lines by adopting an adaptive particle swarm optimization algorithm based on chaotic mapping, which can not only fuse high-precision map lane data with smart camera lane line data to improve the redundancy of output lane lines, but also improve the stability and safety of vehicle driving. BRIEF DESCRIPTION OF THE DRAWINGS

[0056] Figure 1 It is a schematic diagram of the method flow of the present invention;

[0057] Figure 2 A schematic diagram of the flow of the improved convolutional neural network algorithm based on regional analysis of the present invention;

[0058] Figure 3 It is a flow chart of a multimodal feature fusion algorithm based on a recurrent neural network of the present invention;

[0059] Figure 4 It is a schematic diagram of the flow of the adaptive particle swarm optimization algorithm based on chaotic mapping of the present invention. DETAILED DESCRIPTION

[0060] The following is a description of exemplary embodiments of the present disclosure in conjunction with the accompanying drawings, including various details of the embodiments of the present disclosure to facilitate understanding, which should be considered as merely exemplary. Therefore, it should be recognized by those of ordinary skill in the art that various changes and modifications may be made to the embodiments described herein without departing from the scope and spirit of the present disclosure. Similarly, for the sake of clarity and conciseness, descriptions of well-known functions and structures are omitted in the following description.

[0061] Example 1: Figure 1 As shown, a method for fusing lane lines of a vision image with a high-precision map comprises:

[0062] S1. The vehicle is driving on the road, and the image data information of the road is obtained in real time based on the on-board camera, and the map data information of the current position is obtained in real time based on the on-board high-precision map;

[0063] S2. Based on the image data information of the road, an improved convolutional neural network algorithm based on regional analysis is used to identify and classify the image of the road to obtain the image data information of the lane line of the road;

[0064] S3. Based on the image data information of the lane line of the road and the map data information of the current position, a multimodal feature fusion algorithm based on a recurrent neural network is used to fuse the image and map data of the lane line to obtain fused data information of the lane line;

[0065] S4. Based on the fused lane line data information, the lane line is optimized using an adaptive particle swarm optimization algorithm based on chaotic mapping to obtain optimized lane line data information.

[0066] In this embodiment, if Figure 2 As shown, in step S2, the use of the improved convolutional neural network algorithm based on regional analysis to identify and classify the road image includes:

[0067] S21. Based on the image data information of the road, establish an image area division function Q,

[0068] Wherein, x is the image data information of the road, α1, α2 and α3 are adaptive adjustment factors of road image segmentation, and the image of the road is segmented to obtain the image data information of the segmented road;

[0069] S22. Input the image data information of the divided road into a convolutional neural network for training and learning, and determine the neuron function W,

[0070]

[0071] Among them, y is the image data information of the divided road, β1, β2 and β3 are the weight factors of the model, and the trained convolutional neural network model is obtained;

[0072] S23. Based on the trained convolutional neural network model, the image data information of the road is input, the image of the road is identified and classified, and the image data information of the lane lines of the road is obtained.

[0073] In this embodiment, the adaptive adjustment factors α1, α2 and α3 of the road image division are:

[0074]

[0075]

[0076]

[0077] Among them, x is the image data information of the road, and x0 is the mean point of the road image.

[0078] In this embodiment, the constraints of the weight factors β1, β2 and β3 of the model are:

[0079] In this embodiment, the high-precision map lane data and the smart camera lane line data are identified and classified through an improved convolutional neural network algorithm based on regional analysis to improve the redundancy of the output lane line. The output visual lane line data is fused. Under complex driving conditions, the smart camera and the high-precision map locator have some complementary redundancy, which improves the safety and reliability of autonomous driving.

[0080] Example 2: Based on the method for fusing vision with lane lines of a high-precision map in Example 1, the present invention is further illustrated and described below.

[0081] like Figure 1 As shown, a method for fusing lane lines of a vision image with a high-precision map comprises:

[0082] S1. The vehicle is driving on the road, and the image data information of the road is obtained in real time based on the on-board camera, and the map data information of the current position is obtained in real time based on the on-board high-precision map;

[0083] S2. Based on the image data information of the road, an improved convolutional neural network algorithm based on regional analysis is used to identify and classify the image of the road to obtain the image data information of the lane line of the road;

[0084] S3. Based on the image data information of the lane line of the road and the map data information of the current position, a multimodal feature fusion algorithm based on a recurrent neural network is used to fuse the image and map data of the lane line to obtain fused data information of the lane line;

[0085] S4. Based on the fused lane line data information, the lane line is optimized using an adaptive particle swarm optimization algorithm based on chaotic mapping to obtain optimized lane line data information.

[0086] In this embodiment, if Figure 3 As shown, in step S3, the use of a multimodal feature fusion algorithm based on a recurrent neural network to fuse the lane line image and map data includes:

[0087] S31. Inputting the image data information of the lane line of the road into a recurrent neural network for training and learning, constructing an image feature extraction function R of the network,

[0088]

[0089] Wherein, z is the image data information of the lane line of the road, δ1, δ2 and δ3 are image feature extraction factors, and feature extraction is performed on the image of the lane line of the road to obtain feature data information of the image of the lane line of the road;

[0090] S32. Inputting the map data information of the current location into a recurrent neural network for training and learning, constructing a network map feature extraction function P,

[0091]

[0092] Wherein, a is the map data information of the current location, γ1, γ2 and γ3 are map feature extraction factors, and feature extraction is performed on the map data of the current location to obtain feature data information of the map of the current location;

[0093] S33. Based on the feature data information of the current position map and the feature data information of the lane line image of the road, a multimodal feature fusion function G of the lane line is established,

[0094]

[0095] Among them, b is the feature data information of the current location map, c is the feature data information of the image of the lane line of the road, η1, η2 and η3 are feature fusion factors, and the lane line image and map data are fused to obtain the fused lane line data information.

[0096] In this embodiment, the constraint function f1 of the map feature extraction factors γ1, γ2 and γ3 is,

[0097]

[0098] The constraint function f2 of the image feature extraction factors δ1, δ2 and δ3 is,

[0099]

[0100] Among them, the value range of constraint function f1 is (1,2), and the value range of constraint function f2 is (2,3).

[0101] In this embodiment, if Figure 4 As shown, in step S4, the optimization of the lane line using the adaptive particle swarm optimization algorithm based on chaotic mapping includes:

[0102] S41. Based on the fused lane line data information, establish a chaotic mapping function H of the lane line,

[0103]

[0104] Among them, g is the data information of the fused lane line, θ and ρ are the chaos adjustment factors of the lane line, and the chaotic value of the lane line is characterized to obtain the chaotic value data information of the lane line;

[0105] S42. Based on the chaotic value data information of the lane line, the particle population is initialized, the maximum number of iterations is determined, and the initialized particle population data information is obtained;

[0106] S43. Based on the initialized particle population data information, establish the fitness function L of the population individuals,

[0107]

[0108] Among them, h is the data information of the particle population after initialization, ρ1, ρ2 and ρ3 are the fitness control parameters of the individual particle population, and the fitness value of the individual particle population is calculated to obtain the fitness value data information of the individual particle population;

[0109] S44. Based on the fitness value data information of the individual particle population, establish the target optimization function Q of the particle population,

[0110]

[0111] Among them, v is the fitness value data information of the individual particle population, ω1, ω2 and ω3 are optimization factors, and the lane line is optimized to obtain the data information of the optimized lane line.

[0112] In this embodiment, the constraints of the fitness control parameters ρ1, ρ2 and ρ3 of the individual particle population are:

[0113]

[0114] In this embodiment, the present invention provides a system for fusing lane lines of a visual map with a high-precision map, including a computer device that is programmed or configured to execute the steps of any one of the methods for fusing lane lines of a visual map with a high-precision map.

[0115] In this embodiment, the present invention provides a computer-readable storage medium, on which is stored a computer program programmed or configured to execute any one of the methods for fusing vision with lane lines of a high-precision map.

[0116] Any reference to memory, storage, database or other media used in the embodiments provided in the present application may include non-volatile and / or volatile memory. Non-volatile memory may include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM) or flash memory. Volatile memory may include random access memory (RAM) or external cache memory. As an illustration and not limitation, RAM is available in various forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), synchronous link (Synchlink) DRAM (SLDRAM), memory bus (Rambus) direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM) and memory bus dynamic RAM (RDRAM).

[0117] In summary, the present invention not only improves the stability of lane line output, combines vision with maps, greatly improves redundancy, but also provides a more reliable input for vehicle positioning and planning decisions.

[0118] The above specific implementations do not constitute a limitation on the protection scope of the present disclosure. It should be understood by those skilled in the art that various modifications, combinations, sub-combinations and substitutions can be made according to design requirements and other factors. Any modification, equivalent substitution and improvement made within the spirit and principle of the present disclosure shall be included in the protection scope of the present disclosure.

Claims

1. A method for fusing lane lines of vision and high-precision maps, characterized in that: The method comprises: S1. The vehicle is driving on the road, and the image data information of the road is obtained in real time based on the on-board camera, and the map data information of the current position is obtained in real time based on the on-board high-precision map; S2. Based on the image data information of the road, an improved convolutional neural network algorithm based on regional analysis is used to identify and classify the image of the road to obtain the image data information of the lane line of the road; S3. Based on the image data information of the lane line of the road and the map data information of the current position, a multimodal feature fusion algorithm based on a recurrent neural network is used to fuse the image and map data of the lane line to obtain fused data information of the lane line; S4. Based on the fused lane line data information, the lane line is optimized using an adaptive particle swarm optimization algorithm based on chaotic mapping to obtain optimized lane line data information.

2. The method for fusing lane lines of vision and high-precision maps according to claim 1, characterized in that: In step S2, the use of an improved convolutional neural network algorithm based on regional analysis to identify and classify road images includes: S21. Based on the image data information of the road, establish an image area division function Q, Wherein, x is the image data information of the road, α1, α2 and α3 are adaptive adjustment factors of road image segmentation, and the image of the road is segmented to obtain the image data information of the segmented road; S22. Input the image data information of the divided road into a convolutional neural network for training and learning, and determine the neuron function W, Among them, y is the image data information of the divided road, β1, β2 and β3 are the weight factors of the model, and the trained convolutional neural network model is obtained; S23. Based on the trained convolutional neural network model, the image data information of the road is input, the image of the road is identified and classified, and the image data information of the lane lines of the road is obtained.

3. The method for fusing lane lines of a vision map with a high-precision map according to claim 2, characterized in that: The adaptive adjustment factors α1, α2 and α3 of the road image division are: Among them, x is the image data information of the road, and x0 is the mean point of the road image.

4. The method for fusing lane lines of a vision map with a high-precision map according to claim 1, characterized in that: The constraints of the weight factors β1, β2 and β3 of the model are:

5. The method for fusing lane lines of vision and high-precision maps according to claim 1, characterized in that: In step S3, the method of fusing the lane line image and map data using a multimodal feature fusion algorithm based on a recurrent neural network includes: S31. Inputting the image data information of the lane line of the road into a recurrent neural network for training and learning, constructing an image feature extraction function R of the network, Wherein, z is the image data information of the lane line of the road, δ1, δ2 and δ3 are image feature extraction factors, and feature extraction is performed on the image of the lane line of the road to obtain feature data information of the image of the lane line of the road; S32. Inputting the map data information of the current location into a recurrent neural network for training and learning, constructing a network map feature extraction function P, Wherein, a is the map data information of the current location, γ1, γ2 and γ3 are map feature extraction factors, and feature extraction is performed on the map data of the current location to obtain feature data information of the map of the current location; S33. Based on the feature data information of the current position map and the feature data information of the lane line image of the road, a multimodal feature fusion function G of the lane line is established, Among them, b is the feature data information of the current location map, c is the feature data information of the image of the lane line of the road, η1, η2 and η3 are feature fusion factors, and the lane line image and map data are fused to obtain the fused lane line data information.

6. The method for fusing lane lines of vision and high-precision maps according to claim 5 is characterized by: The constraint function f1 of the map feature extraction factors γ1, γ2 and γ3 is: The constraint function f2 of the image feature extraction factors δ1, δ2 and δ3 is, Among them, the value range of constraint function f1 is (1,2), and the value range of constraint function f2 is (2,3).

7. The method for fusing lane lines of vision and high-precision maps according to claim 1, characterized in that: In step S4, the optimization of the lane line by using the adaptive particle swarm optimization algorithm based on chaotic mapping includes: S41. Based on the fused lane line data information, establish a chaotic mapping function H of the lane line, Among them, g is the data information of the fused lane line, θ and ρ are the chaos adjustment factors of the lane line, and the chaotic value of the lane line is characterized to obtain the chaotic value data information of the lane line; S42. Based on the chaotic value data information of the lane line, the particle population is initialized, the maximum number of iterations is determined, and the initialized particle population data information is obtained; S43. Based on the initialized particle population data information, establish the fitness function L of the population individuals, Among them, h is the data information of the particle population after initialization, ρ1, ρ2 and ρ3 are the fitness control parameters of the individual particle population, and the fitness value of the individual particle population is calculated to obtain the fitness value data information of the individual particle population; S44. Based on the fitness value data information of the individual particle population, establish the target optimization function Q of the particle population, Among them, v is the fitness value data information of the individual particle population, ω1, ω2 and ω3 are optimization factors, and the lane line is optimized to obtain the data information of the optimized lane line.

8. The method for fusing lane lines of a vision map with a high-precision map according to claim 7, characterized in that: The constraints of the fitness control parameters ρ1, ρ2 and ρ3 of the individual particle population are:

9. A system for integrating lane lines of vision and high-precision maps, comprising a computer device, characterized in that: The computer device is programmed or configured to execute the steps of the method for fusing the lane lines of the vision and high-precision map as described in any one of claims 1 to 8.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program that is programmed or configured to execute the method for fusing lane lines of a visual map with a high-precision map as described in any one of claims 1 to 8.

Images