A road area recognition method, automatic driving control method and device

Through the integration of real-time acquisition and historical frame data, combined with deep learning model and data dislocation translation technology, the accuracy and robustness of road area identification in complex scenarios in the existing technology are solved, and efficient and accurate road area is achieved. Identify and plan autonomous driving paths.

CN114419592BActive Publication Date: 2024-11-29CHANGSHA HUILIAN INTELLIGENT TECH CO LTD
View PDF 5 Cites 0 Cited by

Patent Information

Application Number
CN202210057452.8
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2022-01-18
Publication Date
2024-11-29
Estimated Expiration
2042-01-18

AI Technical Summary

Technical Problem

In the prior art, the road area identification method has low recognition accuracy when facing scenes such as flat grass or mud, and is easily affected by light changes, resulting in missed identification problems, making it difficult to take into account efficiency, accuracy and robustness.

Method used

By acquiring the on-board camera images in real time, obtaining the road area data of the current frame and historical frames, performing coordinate conversion and fusion, combining the deep learning model for road area recognition, and performing data misalignment and translation when necessary to fill the void area, improving identification Accuracy and robustness.

Benefits of technology

It effectively avoids missing recognition caused by road area shadows, improves adaptability to scene changes and lighting changes, improves the accuracy and integrity of road area identification, and is suitable for path planning in complex road environments.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN114419592B_ABST
    Figure CN114419592B_ABST
Patent Text Reader

Abstract

The present invention discloses a road area recognition method, an automatic driving control method and a device, and the recognition method comprises the following steps: S1. Real-time acquisition of a road environment image captured by a vehicle-mounted camera for image segmentation to segment a road area; S2. Acquiring current frame road area data and a specified number of historical frame road area data from the segmented road area; S3. Converting the historical frame road area data to the coordinate space of the current frame road area data to obtain converted historical frame road data; S4. Fusing the converted historical frame road data and the current frame road area data to obtain a final road area recognition result. The present invention has the advantages of simple implementation method, high detection efficiency and accuracy, strong environmental adaptability and robustness, etc.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention relates to the field of image processing technology, and in particular to a road area recognition method, an automatic driving control method and a device. Background Art

[0002] Road area recognition is to identify the area on the road that is a road. In situations such as autonomous driving of unmanned vehicles, it is necessary to identify the road area in real time to provide it to the vehicle for path planning. The accuracy of road area recognition will directly affect the effectiveness of path planning.

[0003] For the recognition of road areas, the following methods are usually used in the prior art:

[0004] 1. Recognition method based on laser point cloud

[0005] This method uses laser radar to collect laser point clouds of the road environment and uses three-dimensional information to identify the road. For example, an area with little height change and a certain height above the ground is calculated as the road area. However, this method is only applicable to scenes where the road and the surrounding environment are very different. When there is flat grass or mud near the road area, the road area cannot be accurately identified. The applicable scenes are very limited and the recognition accuracy is not high.

[0006] 2. Recognition method based on image processing

[0007] This method collects images of the road environment and analyzes the road area based on the image texture and color features. However, this method has poor adaptability to the environment and is greatly affected by lighting. Light changes have a great impact on the texture and color of the image. When the light is strong, the color of the road surface will deviate, and when the light is strong, the road surface will produce shadows and uneven brightness. Therefore, the actual recognition accuracy and robustness are poor.

[0008] Some practitioners have proposed to realize road area recognition based on deep learning methods, such as training a road area recognition network model based on a deep convolutional network for road area images, and inputting the real-time collected images into the trained network model to realize road area recognition. However, since the road area approaches the vehicle from far to near during driving, the road area at the same position in the road area will change in appearance in the image taken by the on-board camera. For example, the road area at the same position is smaller in the image when it is far away, and it will be larger in the image as the vehicle drives closer. The road area at the same position will also have different other characteristics at different distances. For example, the contrast between the shadow and non-shadow area of ​​the road area is smaller when it is far away than when it is close. Therefore, the deep convolutional network will have different responses to the difference factors in the image, that is, the road area at the same position will get different recognition results in the deep convolutional network, which is not only poorly adapted to the environment and easily affected by light, but also leads to missed recognition of the shadow part of the road area.

[0009] In summary, the recognition method based on laser point cloud in the existing technology is suitable for scenes where there are large differences between the road and the surrounding environment. It has large limitations on applicable scenes and low recognition accuracy. The recognition method based on image processing has poor adaptability to the environment and is greatly affected by lighting, and is prone to missed recognition. Therefore, there is an urgent need to provide a road area recognition method that can take into account the efficiency, accuracy, environmental adaptability and robustness of recognition. Summary of the invention

[0010] The technical problem to be solved by the present invention is: in response to the technical problems existing in the prior art, the present invention provides a road area recognition method and device with simple implementation method, high detection efficiency and accuracy, and strong robustness, as well as an automatic driving control method with accurate and reasonable path planning.

[0011] In order to solve the above technical problems, the technical solution proposed by the present invention is:

[0012] A road area recognition method, comprising the steps of:

[0013] S1. Real-time acquisition of road environment images collected by the vehicle camera to perform image segmentation and segment the road area;

[0014] S2. Obtaining the current frame road area data and a specified number of historical frame road area data from the segmented road area;

[0015] S3. Converting the historical frame road area data to the coordinate space of the current frame road area data to obtain the converted historical frame road data;

[0016] S4. Fusing the converted historical frame road data and current frame road area data to obtain a final road area recognition result.

[0017] Furthermore, in the step S3, the historical frame road area data is converted into the coordinate space of the current frame road area data according to the displacement and angle deflection of the historical frame road area data relative to the current frame road area data.

[0018] Furthermore, the historical frame road area data is converted into the coordinate space of the current frame road area data according to the following formula:

[0019]

[0020]

[0021] Among them, x b ,y b are the lateral and longitudinal displacements of the road area data of the historical frame relative to the road area data of the current frame in the vehicle body coordinate system, θ is the angular change of the road area data of the historical frame relative to the road area data of the current frame in the vehicle body coordinate system, and x i h ,y i h are the horizontal and vertical coordinates of a data point in the historical frame road area data, x i c ,y i c They are respectively the horizontal and vertical coordinate values ​​of converting the coordinates of the historical frame road area data to the coordinate space of the current frame road area data.

[0022] Furthermore, the steps of step S4 include:

[0023] S401. Performing a dislocation translation on the converted historical frame road data to obtain translated historical frame road data, wherein the dislocation translation is to translate each frame data in an alternating manner toward the front and rear directions of the road;

[0024] S402. The converted historical frame road data and the translated historical frame road data are initially fused, and the result obtained by the initial fusion is secondarily fused with the current frame road area data to obtain a final road area recognition result.

[0025] Furthermore, the step S402 includes:

[0026] S421. Find out the different areas between the translated historical frame road area data and the converted historical frame road area data, merge the translated historical frame road area data with the found different areas, and obtain the initial fusion result;

[0027] S422. Merge the initial fusion result with the road area data of the current frame to achieve secondary fusion, and obtain the final road area recognition result.

[0028] Furthermore, in the step S1, the image to be processed including the road area is collected in real time by a vehicle-mounted camera arranged on the vehicle, and after the step S2 and before the step S3, the step of projecting the segmented road area onto the vehicle body coordinate system is also included, including: by using the radar arranged on the vehicle as the reference coordinate system, the vehicle body and the radar, as well as the radar and the vehicle-mounted camera are jointly calibrated respectively, and the conversion from the vehicle-mounted camera coordinate system to the radar coordinate system is completed according to the joint calibration parameters, and then the radar coordinate system is converted to the vehicle body coordinate system to realize the projection of the road area onto the vehicle body coordinate system.

[0029] Furthermore, in step S01, a pre-built deep learning model is used to perform image segmentation.

[0030] A method for controlling an autonomous driving vehicle, comprising:

[0031] When the vehicle is driving, the on-board camera collects images of the road environment in front of the vehicle;

[0032] Using the road area recognition method as described above to identify the road area;

[0033] The vehicle's path planning is carried out according to the identified road area control.

[0034] Furthermore, the vehicle path planning according to the identified road area control includes: preferentially planning the driving path according to the road area recognition result corresponding to the current frame road area data; when it is determined that there is an obstacle on the planned driving path, planning the driving path according to the historical frame recognition result obtained by the converted historical frame road area data and the converted historical frame road area data.

[0035] A road area recognition device comprises a processor and a memory, wherein the memory is used to store a computer program, the processor is used to execute the computer program, and the processor is used to execute the computer program to perform the above method.

[0036] Compared with the prior art, the advantages of the present invention are:

[0037] 1. The present invention segments the road area from the road environment image, obtains the current frame road area data, and obtains part of the historical frame road area data at the same time. After the coordinates of the historical frame road area data are converted, it is merged with the current frame road area data to obtain the final road area recognition result. The historical frame road area data can be combined to effectively avoid missed recognition due to road area shadows, etc., improve the accuracy of road area recognition, and at the same time improve the robustness to scene changes and lighting changes. It can be suitable for scenes with grass or muddy ground beside the road, etc., to avoid missed recognition due to strong shadows and color deviation of some road sections caused by scene changes and lighting changes.

[0038] 2. The present invention further shifts the historical frame road data along the front-to-back direction of the road so that the historical frame road data is staggered along the road direction, and the staggered distribution data is integrated with the current frame data and the historical frame data. This can effectively fill the void areas in the road area, improve the integrity of road area recognition, further improve the accuracy of road area recognition, and at the same time improve the robustness to scene changes and illumination changes.

[0039] 3. The present invention further identifies the road environment in which the vehicle is traveling in real time. If it is identified that the vehicle is currently in a complex road environment that meets preset conditions, the current frame road area data is used preferentially for path planning. Only when there are obstacles in the current frame road area data and the vehicle cannot pass through, the historical frame road area data is used. The road environment characteristics can be combined to ensure the stability and reliability of path planning under different road environments. BRIEF DESCRIPTION OF THE DRAWINGS

[0040] Figure 1 It is a schematic diagram of the implementation flow of the road area recognition method of this embodiment.

[0041] Figure 2 It is a schematic diagram of the result of road area segmentation in a specific application embodiment of the present invention.

[0042] Figure 3 It is a schematic diagram of the process of realizing the conversion from the image coordinate system to the vehicle body coordinate system in this embodiment.

[0043] Figure 4 It is a schematic diagram of the road area effect projected onto the vehicle body coordinate system in a specific application embodiment of the present invention.

[0044] Figure 5 Schematic diagram of the steps for implementing data fusion in this embodiment.

[0045] Figure 6 It is a schematic diagram of the road area recognition result obtained in a specific application embodiment of the present invention. DETAILED DESCRIPTION

[0046] The present invention is further described below in conjunction with the accompanying drawings and specific preferred embodiments, but the protection scope of the present invention is not limited thereby.

[0047] like Figure 1 As shown, the steps of the road area recognition method in this embodiment include:

[0048] S1. Real-time acquisition of the image to be processed containing the road area for image segmentation to segment the road area;

[0049] S2. Obtain the current frame road area data and a specified number of historical frame road area data from the segmented road area;

[0050] S3. Convert the historical frame road area data to the coordinate space of the current frame road area data to obtain the converted historical frame road area data;

[0051] S4. Fusing the converted historical frame road data and the current frame road area data to obtain the final road area recognition result.

[0052] Considering that the road area in the images continuously captured by the vehicle camera changes continuously, the features of the same area when close to the road vanishing point (i.e., features at a long distance) and in front of the vehicle (i.e., features at a short distance) are obviously different. The road area at the same position is smaller in the image when it is far away, and becomes larger in the image as the vehicle moves closer. The shadows of the road area at the same position in the image at different distances will also be different. The contrast between the shadow and non-shadow area of ​​the road area is smaller when it is far away than when it is close. The shadow of the road area is easy to be missed and cause holes. This embodiment takes the above-mentioned problem into consideration, and after segmenting the road area from the image, obtains the current frame road area data while obtaining part of the historical frame road area data, and after coordinate conversion of the historical frame road area data, merges them with the current frame road area data to obtain the final road area recognition result. It can combine the historical frame road area data to effectively avoid missed recognition due to road area shadows, etc., and improve the accuracy of road area recognition. At the same time, it can also improve the robustness to scene changes and lighting changes. It can be suitable for scenes with grass or mud beside the road, and avoid missed recognition due to strong shadows and color deviations of some road sections caused by scene changes and lighting changes.

[0053] In step S1 of this embodiment, a pre-built deep learning model is specifically used for image segmentation. First, images containing road areas are collected and annotated in advance to form a road area image data set, and the road area image data set is used to train the deep network model; when the video containing road information collected by the camera is obtained in real time in step S1, the deep learning model ENET is used to segment the road area in the image coordinate system. This embodiment makes full use of the advantages of the deep learning network model. By collecting and annotating road area pictures of specific scenes, the deep network model "learns" the characteristics of the road area, and the road area can be quickly segmented after the road environment image is obtained in real time.

[0054] In a specific application embodiment, the collected original image of the road area and the road area segmented using the deep learning model are as follows: Figure 2 As shown, Figure 2 (a) corresponds to the original image of the road area collected. Figure 2 (b) corresponds to the road area segmented in the image coordinate system.

[0055] It is understandable that the image segmentation of the above road area may also be implemented using other models or segmentation methods according to actual needs.

[0056] In step S1 of the present embodiment, the image to be processed including the road area is collected in real time by the on-board camera arranged on the vehicle. It is further necessary to convert the image coordinate system where the image collected by the on-board camera is located into the vehicle body coordinate system. After step S2 and before step S3, the step of projecting the segmented road area into the vehicle body coordinate system is also included, including: using the radar arranged on the vehicle as the reference coordinate system, jointly calibrating the vehicle body and the radar, and the radar and the on-board camera respectively, completing the conversion from the on-board camera coordinate system to the radar coordinate system according to the joint calibration parameters, and then converting the radar coordinate system to the vehicle body coordinate system to realize the projection of the road area into the vehicle body coordinate system.

[0057] like Figure 3 As shown, when the segmented road area is projected to the vehicle body coordinate system, the present embodiment first calibrates the intrinsic parameters of the camera and the conversion matrix parameters of the vehicle front radar (the radar arranged at the front end of the vehicle) to the vehicle body coordinate system. Then, the camera and the radar are jointly calibrated through a joint calibration tool to calibrate the joint calibration conversion matrix parameters of the camera and the front radar. The road area is projected from the image coordinate system to the vehicle body coordinate system through the transformation relationship between the calibrated image and the vehicle body coordinate system.

[0058] In a specific application embodiment, the vehicle-mounted laser radar is used as a reference coordinate system, and the vehicle body and laser radar, and the laser radar and the vehicle-mounted camera are calibrated twice respectively to determine the transformation relationship between the vehicle body coordinate system and the laser radar coordinate system, and the laser radar coordinate system and the vehicle-mounted camera. The external parameter matrix from the laser radar to the vehicle body and the external parameter matrix from the laser radar to the vehicle-mounted camera are obtained by joint calibration. The external parameter matrix includes a rotation matrix and a translation matrix. The external parameter matrix from the laser radar to the vehicle body obtained by the joint calibration is inverted and multiplied with the external parameter matrix from the laser radar to the vehicle-mounted camera to obtain the external parameter transformation matrix from the vehicle body to the vehicle-mounted camera, that is, the projection transformation matrix between the vehicle body coordinates and the image pixel points is obtained. The external parameter transformation matrix from the vehicle body to the vehicle-mounted camera is calculated based on the internal parameter matrix of the vehicle-mounted camera and the two external parameter transformation matrices obtained by the joint calibration, and the coordinate transformation relationship between the image pixels and the vehicle body is constructed to achieve the coordinate transformation between the image pixels and the vehicle body. In a specific application embodiment, the segmented road area is projected onto the vehicle body coordinate system. The result is as follows: Figure 4 shown.

[0059] Taking into account the fact that the road width and road attributes of adjacent road sections are most likely consistent, when taking out a specified number of historical road area data in step S2 of this embodiment, it also includes performing a closing operation on each frame of historical road area data to enhance the connectivity of the road area.

[0060] In step S3 of this embodiment, the historical frame road area data is converted into the coordinate space of the current frame road area data according to the displacement and angular deflection of the historical frame road area data relative to the current frame road area data, so that the historical frame road area data and the current frame road area data are synchronized in time and space, which facilitates the subsequent data fusion.

[0061] This embodiment specifically converts the historical frame road area data into the coordinate space of the current frame road area data according to the following formula:

[0062]

[0063]

[0064] Among them, x b ,y b are the lateral and longitudinal displacements of the road area data of the historical frame relative to the road area data of the current frame in the vehicle body coordinate system, θ is the angular change of the road area data of the historical frame relative to the road area data of the current frame in the vehicle body coordinate system, and x i h ,y i h are the horizontal and vertical coordinates of a data point in the historical frame road area data, x i c,y i c They are respectively the horizontal and vertical coordinate values ​​of converting the coordinates of the historical frame road area data to the coordinate space of the current frame road area data.

[0065] In a specific application embodiment, 10 frames (the specific number can be configured according to actual needs) of historical frame road area data are taken, and the closed operation algorithm is first processed, and then the displacement (x b ,y b ) and the angle deflection θ, the road area data of the historical frames are unified into the coordinate space of the current frame according to the above formula (1), so as to ensure that the road area data of different historical frames are synchronized in time and space.

[0066] In this embodiment, step S3 converts the historical frame road area data to the same coordinate space as the current frame road area data, and then the converted historical frame road area data can be directly merged with the current frame road area data, such as by performing an OR operation to obtain the final road area recognition result. Since the images generated by observing the same area under different distances of the historical frame data are different, such as the road area at the same position is smaller in the image when it is far away, and larger in the image when it is close with the movement of the vehicle, the contrast between the shadow and non-shadow area of ​​the road area is also different at different distances. After the deep convolution network, the difference factors in the above images will lead to different responses. By merging the historical frame road area data with the current frame road area data, the integration of long and short distance features can be achieved, and the problem of missed recognition caused by the shadow part of the road area can be improved.

[0067] Directly fusing the historical frame road area data with the current frame road area data, the obtained fusion result may still have some holes caused by shadows, etc. In this embodiment, step S4 further introduces the dislocation translation historical frame data to achieve full fusion, and the specific steps include:

[0068] S401. Performing a shift translation on the converted historical frame road data to obtain the shifted historical frame road data, wherein the shift translation is to shift each frame data alternately in the front and rear directions of the road;

[0069] S402. The converted historical frame road data and the translated historical frame road data are initially fused, and the result obtained by the initial fusion is secondarily fused with the current frame road area data to obtain the final road area recognition result.

[0070] Since the width and attributes of a similar road section are usually roughly the same, this embodiment shifts the historical frame road data along the front-to-back direction of the road so that the historical frame road data is staggered along the front-to-back direction of the road. The staggered distribution data is fused on the basis of the current frame data and the historical frame data. This can effectively fill the void areas in the road area, further improve the integrity of the road area recognition results, improve the accuracy of road area recognition, and at the same time improve the robustness to scene changes and lighting changes.

[0071] In this embodiment, in the above step S401, the odd-numbered frame data in the historical frame road area data is specifically translated spatially toward the front of the road, and the even-numbered frame data is translated spatially toward the rear of the road, that is, the offset translation is performed according to the odd and even frames to obtain the translated historical frame road data. The above-mentioned method of performing offset translation according to the odd and even frames can fully ensure that the historical frame road data is staggered along the front and back directions of the road, thereby ensuring that various types of hole areas in the road area can be filled, and finally a complete and accurate road area recognition result can be obtained. Of course, the specific offset translation method can also adopt other methods according to actual needs, such as taking data of more than two adjacent frames each time and spatially translating them toward the front of the road, and spatially translating data of more than two adjacent frames toward the rear of the road, or even adopting a method of randomly taking data frames.

[0072] like Figure 5 As shown, step S402 in this embodiment includes:

[0073] S421. Find out the different areas of the translated historical frame road area data and the converted historical frame road area data, merge the translated historical frame road area data with the found different areas, and obtain the initial fusion result;

[0074] S422. Merge the initial fusion result with the road area data of the current frame to achieve secondary fusion and obtain the final road area recognition result.

[0075] If the original historical frame road area data converted in step S3 and the translated historical frame road area data have the same area, it is highly likely that the same area is repeatedly identified. If there are different areas, it may be a new identification area. This embodiment initially merges the translated historical frame road area data with the original converted historical frame road area data to remove the repeated identification areas, screen out all the identification areas, and then performs a secondary fusion with the current frame road area data. This can make full use of the historical frame road area data to avoid missed identification caused by strong shadows and color deviations of some sections due to scene changes and lighting changes, and avoid misidentification of grass or mud next to the road. At the same time, it can also eliminate the empty areas in the road area, so as to obtain a road area identification result that is as complete and accurate as possible. In a specific application embodiment, the final road area identification result obtained after fusion is as follows. Figure 6 shown.

[0076] In the above steps S421 and S422, all the roads in the historical frames and the areas identified in the current frame are integrated into one road area through an OR operation.

[0077] In step S4 of this embodiment, the historical frame road area data and the current frame road area data can be directly merged, or the fusion can be achieved by fusing the historical frame road area data, the historical frame road area data after offset translation, and the current frame road area data twice according to the above steps. The specific fusion method can be determined based on the actual road environment scene, real-time and complexity requirements, etc.

[0078] After the fusion in step S4 of this embodiment, a closing operation is also performed to obtain the final road area, so that the connectivity of the road area is enhanced, and the characteristics that the road width and road attributes of adjacent road sections are roughly consistent are fully utilized, which can further improve the accuracy and robustness of road area recognition.

[0079] After step S4, this embodiment also includes configuring the first priority of the road area recognition result corresponding to the current frame road area data in the final road area recognition result, and configuring the second priority of the road area recognition result corresponding to the historical frame road area data, the first priority being higher than the second priority, so that the current frame road area data is used first, and when the current frame road area data cannot be used, the road area recognition result corresponding to the historical frame road area data is started. In complex scenes or road sections with varying road widths, the road area recognition result corresponding to the historical frame road area data may have errors. This embodiment can further ensure the reliability of road area recognition in complex environment road conditions by setting priorities to give priority to the current frame road area data in special scenes.

[0080] The steps of the automatic driving vehicle control method of this embodiment include:

[0081] During the driving process of the vehicle, an environmental image of the road ahead of the vehicle is collected and used as an image to be processed;

[0082] Using the road area recognition method to identify the road area;

[0083] The vehicle's path planning is carried out according to the identified road area control.

[0084] The present embodiment performs path planning of the vehicle according to the identified road area control, including: if it is identified that the vehicle is currently in a complex road environment that meets preset conditions, the driving path is planned according to the road area recognition result corresponding to the current frame road area data; when it is determined that there are obstacles on the planned driving path, the driving path is planned according to the historical frame recognition result obtained from the converted historical frame road area data and the converted historical frame road area data.

[0085] In this embodiment, step S4 also includes a priority division step, in which the road area recognition result corresponding to the current frame road area data in the final road area recognition result is configured with a first priority, and the historical frame recognition result is configured with a second priority, and the first priority is higher than the second priority.

[0086] After fusing the historical frame road area data with the current frame road area data, in ordinary road environments, such as straight roads or ordinary curves with small curvature, path planning can be performed directly according to the final road area recognition results obtained after fusion. However, in complex scenes or road sections with varying road widths, the fusion of historical frame road area data may cause a small amount of error. This embodiment identifies the road environment in which the vehicle is traveling in real time. If it is identified that the vehicle is currently in a complex road environment that meets preset conditions, such as a curve with a curvature greater than a preset threshold, the current frame road area data is used first for path planning. Only when the current frame road area data is impassable due to obstacles, the historical frame road area data is used. The road environment characteristics can be combined to ensure the stability and reliability of path planning in different road environments.

[0087] In a specific application embodiment, after the road area is identified by the above-mentioned road area identification method, the identification area pixel obtained by the current frame road area data is identified as 100, the identification area obtained by the historical frame road area data is identified as 101, and the non-road area is identified as 0. Among them, the identification area corresponding to the road area data of the previous frame is configured with the highest priority, that is, the identification number 100 has the highest priority. When planning the path, the area identified as 100 is searched first. When there is an obstacle in the area identified as 100 and it cannot be driven, the area identified as 101 is searched.

[0088] This embodiment also provides a road area recognition device, including a processor and a memory, the memory is used to store a computer program, the processor is used to execute the computer program, and the processor is used to execute the computer program to execute the above method.

[0089] The above is only a preferred embodiment of the present invention, and does not limit the present invention in any form. Although the present invention has been disclosed as a preferred embodiment, it is not intended to limit the present invention. Therefore, any simple modification, equivalent change and modification made to the above embodiment according to the technical essence of the present invention without departing from the content of the technical solution of the present invention shall fall within the scope of protection of the technical solution of the present invention.

Claims

1. A road area recognition method, characterized in that the steps include: S1. Real-time acquisition of road environment images collected by the vehicle camera to perform image segmentation and segment the road area; S2. Obtaining the current frame road area data and a specified number of historical frame road area data from the segmented road area; S3. Converting the historical frame road area data to the coordinate space of the current frame road area data to obtain the converted historical frame road data; S4. Fusing the converted historical frame road data and the current frame road area data to obtain the final road area recognition result; The steps of step S4 include: S401. Performing a dislocation translation on the converted historical frame road data to obtain translated historical frame road data, wherein the dislocation translation is to translate each frame data in an alternating manner toward the front and rear directions of the road; S402. The converted historical frame road data and the translated historical frame road data are initially fused, and the result obtained by the initial fusion is secondary fused with the current frame road area data to obtain a final road area recognition result; The step S402 includes: S421. Find out the different areas between the translated historical frame road area data and the converted historical frame road data, and merge the translated historical frame road area data with the found different areas to obtain the initial fusion result; S422. Merge the initial fusion result with the road area data of the current frame to achieve secondary fusion, and obtain the final road area recognition result.

2. The road area recognition method according to claim 1, characterized in that: In the step S3, the historical frame road region data is converted into the coordinate space of the current frame road region data according to the displacement and angle deflection of the historical frame road region data relative to the current frame road region data.

3. The road area recognition method according to claim 2, characterized in that: The historical frame road area data is converted into the coordinate space of the current frame road area data according to the following formula: in ,x b , y b are the lateral and longitudinal displacements of the road area data of the historical frame relative to the road area data of the current frame in the vehicle body coordinate system, θ is the angular change of the road area data of the historical frame relative to the road area data of the current frame in the vehicle body coordinate system, x i h , y i h are the horizontal and vertical coordinates of a data point in the historical frame road area data, x i c , y i c They are respectively the horizontal and vertical coordinate values ​​of converting the coordinates of the historical frame road area data to the coordinate space of the current frame road area data.

4. The road area recognition method according to any one of claims 1 to 3, characterized in that: In the step S1, a to-be-processed image including a road area is collected in real time by a vehicle-mounted camera arranged on the vehicle. After the step S2 and before the step S3, the step of projecting the segmented road area onto a vehicle body coordinate system is also included, including: by using a radar arranged on the vehicle as a reference coordinate system, the vehicle body and the radar, as well as the radar and the vehicle-mounted camera are jointly calibrated respectively, and the conversion from the vehicle-mounted camera coordinate system to the radar coordinate system is completed according to the joint calibration parameters, and then the radar coordinate system is converted to the vehicle body coordinate system, so as to realize the projection of the road area onto the vehicle body coordinate system.

5. The road area recognition method according to any one of claims 1 to 3, characterized in that: In step S1, a pre-built deep learning model is used to perform image segmentation.

6. A method for controlling an automatic driving vehicle, characterized in that the steps include: When the vehicle is driving, the on-board camera collects images of the road environment in front of the vehicle; Identify the road area using the road area identification method according to any one of claims 1 to 5; The vehicle's path planning is carried out according to the identified road area control.

7. The automatic driving vehicle control method according to claim 6, characterized in that: The vehicle path planning according to the identified road area control includes: preferentially planning the driving path according to the road area recognition result corresponding to the current frame road area data; when it is determined that there is an obstacle on the planned driving path, planning the driving path according to the converted historical frame road data and the historical frame recognition result obtained from the converted historical frame road data.

8. A road area recognition device, comprising a processor and a memory, wherein the memory is used to store a computer program, and the processor is used to execute the computer program, characterized in that: The processor is configured to execute the computer program to perform the method according to any one of claims 1 to 5.

Citation Information

Patent Citations

  • Passable area detection method and device, storage medium and electronic equipment

    CN108256413A

  • An intelligent vehicle passable area detection method based on multi-source information fusion

    CN109829386A

  • Barrier-free driving area detection method and device, vehicle and storage medium

    CN110516621A

  • Point cloud data and image data fusion method and storage medium

    CN113762413A

  • Method for extracting road structure information, unmanned aerial vehicle, and automatic driving system

    WO2020172875A1