Decision information acquisition method, device, terminal and storage medium

Abstract

The present application provides a decision information acquisition method, device, terminal and storage medium, which belongs to the field of autonomous driving. The method includes: obtaining a road image taken by the current vehicle during driving; determining a first target area and lane line information in the road image, the first target area being a non-drivable area in the road image, and the lane line information being used to indicate each lane line in the road image; and determining decision information based on the lane line information and the first target area. The above scheme can determine the position of the lane line on the road in the road image by determining the first target area and lane line information from the road image taken by the current vehicle during driving, and can also determine the driving environment around the current vehicle based on the lane line and the non-drivable area, and then determine decision information, so as to control the current vehicle to effectively avoid obstacles on the road according to the decision information, thereby improving the control efficiency of the vehicle.

Description

Technical Field

[0001] The present application relates to the field of autonomous driving, and in particular to a method, device, terminal and storage medium for acquiring decision information. Background Art

[0002] With the development of autonomous driving technology, self-driving cars are now capable of handling most traffic scenarios. However, during autonomous driving, the vehicle is constantly engaged in a struggle with obstacles in the lane. Therefore, obstacle detection and prediction to ensure safe vehicle movement is a crucial research topic.

[0003] Currently, obstacle detection is typically simplified to a rectangular bounding box. This method ignores the actual shape of the obstacle, leading to inaccurate detection results and missed detections, which in turn affects vehicle control decisions and prevents the vehicle from effectively avoiding obstacles. Summary of the Invention

[0004] The embodiments of the present application provide a decision information acquisition method, device, terminal, and storage medium that can determine the driving environment around the current vehicle based on lane lines and non-driving areas, and then determine decision information, thereby controlling the current vehicle to effectively avoid obstacles on the road based on the decision information, thereby improving vehicle control efficiency. The technical solution is as follows:

[0005] In one aspect, a method for obtaining decision information is provided, the method comprising:

[0006] Obtain road images taken by the current vehicle during driving;

[0007] In the road image, determining a first target area and lane line information, wherein the first target area is a non-drivable area in the road image, and the lane line information is used to indicate each lane line in the road image;

[0008] Based on the lane line information and the first target area, decision information is determined, where the decision information is used to indicate how to control the current vehicle so that the current vehicle avoids the first target area.

[0009] In some embodiments, determining the decision information based on the lane line information and the first target area includes:

[0010] Obtaining a first lane line and a second lane line from the lane line information, where the first lane line is a lane line on the left side of the lane where the current vehicle is located, and the second lane line is a lane line on the right side of the lane where the current vehicle is located;

[0011] The decision information is determined based on the first lane line, the second lane line, and the first target area.

[0012] In some embodiments, determining the decision information based on the first lane line, the second lane line, and the first target area includes:

[0013] Determine a first lane area based on the first lane line, where the first lane area is a rectangular area centered on the first lane line;

[0014] Determine a second lane area based on the second lane line, where the second lane area is a rectangular area centered on the second lane line;

[0015] Based on the positional relationship between at least one of the first lane area and the second lane area and the first target area, first decision information is determined, where the first decision information is used to indicate that a vehicle is detected merging into the current lane, and the current vehicle is controlled to execute at least one of reducing speed, switching lanes, and avoiding to the left or right.

[0016] In some embodiments, determining the decision information based on the first lane line, the second lane line, and the first target area includes:

[0017] Determine a lane centerline based on the first lane line and the second lane line, where the lane centerline is the centerline of the lane where the current vehicle is located;

[0018] Second decision information is determined based on a positional relationship between the lane centerline and a boundary of the first target area. The second decision information is used to indicate that an obstacle exists in the current lane and control the current vehicle to reduce speed or switch lanes.

[0019] In some embodiments, determining the second decision information based on the positional relationship between the lane centerline and the boundary of the first target area includes:

[0020] When the first lane line or the second lane line is partially missing, determining the obstacle type of the obstacle based on a change trend of the relative distance between the lane centerline and the boundary of the first target area, the obstacle type including static obstacles and dynamic obstacles;

[0021] When the obstacle type is a static obstacle, the second decision information is determined.

[0022] In some embodiments, the method further comprises:

[0023] In a case where the obstacle type is a dynamic obstacle, third decision information is determined, where the third decision information is used to indicate that the vehicle is currently in a slow driving state, and the current vehicle is controlled to maintain driving in the current lane.

[0024] In some embodiments, the method further comprises:

[0025] In a case where the lane line information is not obtained, fourth decision information is determined, where the fourth decision information is used to prompt the driver to control the current vehicle.

[0026] In some embodiments, the method further comprises:

[0027] Constructing a road coordinate system with the current vehicle position as the origin;

[0028] Based on the road coordinate system, the position of each lane line in the road image is determined to obtain the lane line information.

[0029] In some embodiments, the method further comprises:

[0030] Detecting the road image based on a region detection model to obtain an image Boolean matrix corresponding to the road image, wherein elements in the image Boolean matrix are used to represent Boolean values ​​of corresponding pixels in the road image, and the region detection model is used to detect a drivable area in the input image;

[0031] An image region corresponding to pixels having a value of 0 in the image Boolean matrix is ​​determined as the first target region.

[0032] In some embodiments, the method further comprises:

[0033] Acquire a sample image and annotation information of the sample image, where the annotation information is used to indicate a drivable area and a non-drivable area in the sample image;

[0034] The region detection model of the i-th iteration is trained based on the sample image using the labeling information as supervision information, where i is a positive integer.

[0035] In another aspect, a decision information acquisition device is provided, the device comprising:

[0036] The first acquisition module is used to acquire the road image taken by the current vehicle during driving;

[0037] a perception module, configured to determine, in the road image, a first target area and lane line information, wherein the first target area is a non-drivable area in the road image, and the lane line information is used to indicate each lane line in the road image;

[0038] A decision module is used to determine decision information based on the lane line information and the first target area, wherein the decision information is used to indicate how to control the current vehicle to enable the current vehicle to avoid the first target area.

[0039] In some embodiments,

[0040] The decision module includes:

[0041] an acquiring unit, configured to acquire a first lane line and a second lane line from the lane line information, wherein the first lane line is a lane line on the left side of the lane where the current vehicle is located, and the second lane line is a lane line on the right side of the lane where the current vehicle is located;

[0042] A decision unit is configured to determine the decision information based on the first lane line, the second lane line, and the first target area.

[0043] In some embodiments, the decision unit is used to determine a first lane area based on the first lane line, where the first lane area is a rectangular area centered on the first lane line; determine a second lane area based on the second lane line, where the second lane area is a rectangular area centered on the second lane line; and determine first decision information based on a positional relationship between at least one of the first lane area and the second lane area and the first target area, where the first decision information is used to indicate that a vehicle is detected merging into the current lane, and to control the current vehicle to execute at least one of reducing speed, switching lanes, and avoiding to the left or right.

[0044] In some embodiments, the decision unit is used to determine a lane centerline based on the first lane line and the second lane line, where the lane centerline is the centerline of the lane where the current vehicle is located; and determine second decision information based on a positional relationship between the lane centerline and the boundary of the first target area, where the second decision information is used to indicate the presence of an obstacle in the current lane and control the current vehicle to reduce speed or switch lanes.

[0045] In some embodiments, the decision unit is used to determine the obstacle type of the obstacle based on the changing trend of the relative distance between the lane centerline and the boundary of the first target area when the first lane line or the second lane line is partially missing, and the obstacle type includes static obstacles and dynamic obstacles; when the obstacle type is a static obstacle, determine the second decision information.

[0046] In some embodiments, the decision unit is further used to determine third decision information when the obstacle type is a dynamic obstacle, and the third decision information is used to indicate that the current vehicle is in a slow driving state and control the current vehicle to maintain driving in the current lane.

[0047] In some embodiments, the decision module is further used to determine fourth decision information when the lane line information is not obtained, and the fourth decision information is used to prompt the driver to control the current vehicle.

[0048] In some embodiments, the perception module is further used to construct a road coordinate system with the current vehicle position as the origin; based on the road coordinate system, determine the position of each lane line in the road image to obtain the lane line information.

[0049] In some embodiments, the perception module is also used to detect the road image based on the region detection model to obtain an image Boolean matrix corresponding to the road image, where the elements in the image Boolean matrix are used to represent the Boolean values ​​of the corresponding pixels in the road image, and the region detection model is used to detect the drivable area in the input image; the image area corresponding to the pixels with a value of 0 in the image Boolean matrix is ​​determined as the first target area.

[0050] In some embodiments, the apparatus further comprises:

[0051] a second acquisition module, configured to acquire a sample image and annotation information of the sample image, wherein the annotation information is used to indicate a drivable area and a non-drivable area in the sample image;

[0052] The training module is used to train the region detection model of the i-th iteration based on the sample image using the labeling information as supervision information, where i is a positive integer.

[0053] On the other hand, a terminal is provided, comprising a processor and a memory, wherein the memory is used to store at least one computer program, and the at least one computer program is loaded and executed by the processor to implement the decision information acquisition method in the embodiment of the present application.

[0054] On the other hand, a computer-readable storage medium is provided, in which at least one computer program is stored. The at least one computer program is loaded and executed by a processor to implement the decision information acquisition method as in the embodiment of the present application.

[0055] In another aspect, a computer program product is provided, comprising computer program code stored in a computer-readable storage medium. A processor of a terminal reads the computer program code from the computer-readable storage medium and executes the computer program code, causing the terminal to perform the decision information acquisition method provided in various optional implementations of the above aspects.

[0056] The technical solution provided in the embodiment of the present application can determine the position of the lane lines on the road in the road image by determining the first target area and lane line information from the road image taken during the current vehicle's driving process. It can also determine the non-drivable areas corresponding to obstacles on the road in the road image. Based on the lane lines and non-drivable areas, the driving environment around the current vehicle can be determined, and decision information can be determined. According to the decision information, the current vehicle can be controlled to effectively avoid obstacles on the road, thereby improving the control efficiency of the vehicle. BRIEF DESCRIPTION OF THE DRAWINGS

[0057] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the following briefly introduces the drawings required for use in the description of the embodiments. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without any creative work.

[0058] Figure 1 is a structural block diagram of an autonomous driving system provided according to an embodiment of the present application;

[0059] Figure 2 This is a flowchart of a method for obtaining decision information according to an embodiment of the present application;

[0060] Figure 3 is a flowchart of another method for obtaining decision information provided according to an embodiment of the present application;

[0061] Figure 4 is a schematic diagram of a road coordinate system provided according to an embodiment of the present application;

[0062] Figure 5 is a flowchart of another method for obtaining decision information provided according to an embodiment of the present application;

[0063] Figure 6 This is a block diagram of a decision information acquisition device provided according to an embodiment of the present application;

[0064] Figure 7 is a block diagram of another decision information acquisition device provided according to an embodiment of the present application;

[0065] Figure 8This is a structural block diagram of a terminal provided according to an embodiment of the present application. DETAILED DESCRIPTION

[0066] In order to make the objectives, technical solutions and advantages of this application clearer, the implementation methods of this application will be further described in detail below with reference to the accompanying drawings.

[0067] In this application, the terms "first", "second", etc. are used to distinguish identical or similar items with substantially the same effects and functions. It should be understood that there is no logical or temporal dependency between "first", "second", and "nth", nor is there any limitation on the quantity and execution order.

[0068] In the present application, the term "at least one" means one or more, and the term "plurality" means two or more.

[0069] In the specific implementation of this application, related data such as sample images, test images and road images are involved. When the embodiments of this application are applied to specific products or technologies, user permission or consent is required, and the collection, use and processing of relevant data must comply with relevant laws, regulations and standards of relevant countries and regions.

[0070] The following describes the terms used in the embodiments of this application:

[0071] RepVGG: Making VGG-style ConvNets Great Again (RepVGG: Making VGG Networks Great Again, an academic paper) is named RepVGG because the architecture of the training and inference time structures is achieved through a technique called architectural reparameterization. The VGG network is a new deep convolutional neural network developed by researchers from the Visual Geometry Group at the University of Oxford and Google DeepMind.

[0072] U-Net: Convolutional Networks for Biomedical Image Segmentation (U-Net: Convolutional Networks for Biomedical Image Segmentation, is an academic paper). U-Net is one of the earliest algorithms using fully convolutional networks for semantic segmentation. The symmetrical U-shaped structure containing compression and expansion paths used in the paper was very innovative at the time and to some extent influenced the design of several subsequent segmentation networks. The network is also named after its U-shaped shape.

[0073] ERFNet: Efficient Residual Factorized ConvNet for Real-Time Semantic Segmentation (ERFNet: Effective Residual Factorized ConvNet for Real-Time Semantic Segmentation, is an academic paper), ERFNet is one of the earlier real-time semantic segmentation networks.

[0074] PSPNet (Pyramid Scene Parsing Network), the core module is the pyramid pooling module, which can aggregate contextual information from different regions, thereby improving the ability to obtain global information.

[0075] Figure 1 1 is a block diagram of an autonomous driving system 100 according to an embodiment of the present application. The autonomous driving system is deployed in an unmanned vehicle, and the current vehicle in the embodiment of the present application is an unmanned vehicle equipped with the autonomous driving system. The autonomous driving system includes a camera 101, a controller 102, and a terminal 103. The camera 101 is used to perceive the environment surrounding the current vehicle, the controller 102 is used to control the current vehicle, and the terminal 103 is used to predict the driving trajectories of other vehicles and plan the current vehicle's driving strategy.

[0076] The terminal 103 is connected to the camera 101 and the controller 102 via a wireless network or a wired network. The terminal 103 is used to process the data collected by the data acquisition module such as the camera 101, and then generate a control signal based on the processing result, and the controller 102 controls the vehicle based on the control signal. The terminal 103 can be a vehicle-mounted terminal or an external terminal mounted based on a data interface, and this embodiment of the application is not limited to this. The decision information acquisition method provided in the embodiment of the application can be executed by the automatic driving system or by the terminal 103.

[0077] In some embodiments, the camera 101 includes a binocular camera, a trinocular camera, or a multi-camera, and is used to capture images while the vehicle is traveling. It should be noted that the terminal 103 can determine the vehicle information of the current vehicle and the vehicle information of other vehicles on the road based on the data collected by the unmanned vehicle during driving.

[0078] Figure 2 is a flow chart of a method for obtaining decision information according to an embodiment of the present application. Figure 2 As shown, in the embodiment of the present application, the vehicle terminal is used as an example for explanation. The decision information acquisition method includes the following steps:

[0079] 201. Obtain a road image captured by the current vehicle while it is traveling.

[0080] In the embodiment of the present application, the vehicle terminal is Figure 1 The terminal 103 is deployed on the current vehicle. When the current vehicle is driving, it can capture road images through a camera, such as the camera Figure 1 The road image includes information in front of the current vehicle, information on the side of the vehicle, and information on the rear of the vehicle.

[0081] 202. In a road image, determine a first target area and lane line information, where the first target area is a non-drivable area in the road image, and the lane line information is used to indicate each lane line in the road image.

[0082] In an embodiment of the present application, the vehicle-mounted terminal is capable of performing image segmentation on a road image, thereby segmenting the road image into a first target area and a second target area. Specifically, the first target area is determined to be a non-drivable area in the road image, and the second target area is determined to be a drivable area in the road image. The terminal is also capable of recognizing the road image, identifying lane lines in the road image, and thereby determining the position of each lane line in the road image.

[0083] 203. Determine decision information based on the lane line information and the first target area.

[0084] In an embodiment of the present application, after the vehicle-mounted terminal determines the first target area in the road image, it can determine whether the current vehicle will pass through the non-drivable area based on the lane line information, and then determine the decision information based on the judgment result. The decision information is used to indicate how to control the current vehicle, such as slowing down or switching lanes, so that the current vehicle avoids the first target area.

[0085] The technical solution provided by the embodiment of the present application can determine the position of the lane lines on the road in the road image by determining the first target area and lane line information from the road image taken during the current vehicle's driving process. It can also determine the non-drivable areas corresponding to obstacles on the road in the road image, and then determine the driving environment around the current vehicle based on the lane lines and non-drivable areas, and then determine decision information, so as to control the current vehicle to effectively avoid obstacles on the road according to the decision information, thereby improving the control efficiency of the vehicle.

[0086] above Figure 2 The main process of the decision information acquisition method provided in the embodiment of the present application is exemplified. The decision information acquisition method is further introduced below based on an application scenario. Figure 3 is a flowchart of another method for obtaining decision information according to an embodiment of the present application. Figure 3 As shown, in the embodiment of the present application, the vehicle terminal is used as an example for explanation. The decision information acquisition method includes the following steps:

[0087] 301. Obtain a sample image and annotation information of the sample image, where the annotation information is used to indicate a drivable area and a non-drivable area in the sample image.

[0088] In an embodiment of the present application, the vehicle-mounted terminal is able to obtain a sample data set, which includes a plurality of labeled sample images. The sample images included in the sample data set are road images involving various driving scenarios, such as daytime scenes, nighttime scenes, rainy scenes, traffic jam scenes, and non-traffic jam scenes. The sample image obtained by the vehicle-mounted terminal is any sample image in the sample data set, and the annotation information of the sample image can indicate which positions in the sample image belong to the drivable area and which positions belong to the non-drivable area. Among them, the drivable area refers to the area where the vehicle can drive normally, such as the road surface; the non-drivable area refers to the area where the vehicle cannot drive, such as the sky, the lawn on the roadside, and the road surface blocked by other vehicles, which are all non-drivable areas in the sample image. It should be noted that the area where the lane lines and the text on the lanes are located also belongs to the drivable area because they do not affect the driving of the vehicle. The areas other than the drivable area in the sample image belong to the non-drivable area.

[0089] 302. Using the labeled information as supervision information, the region detection model of the i-th iteration is trained based on the sample image, where i is a positive integer.

[0090] In an embodiment of the present application, the vehicle-mounted terminal can use a supervised learning method to train the region detection model. During the training process, the annotation information of the sample image is used as the supervision information to train the region detection model of this iteration. Among them, if the i-th iteration is the first iteration, the region detection model of the i-th iteration is the initial model; if the i-th iteration is not the first iteration, the region detection model of the i-th iteration is the region detection model with the model parameters adjusted after the i-1-th iteration is completed. The region detection model is used to detect the drivable area in the input image. Since the areas other than the drivable area in the input image are all non-drivable areas, the region detection model can also be used to detect the non-drivable areas in the input image.

[0091] It should be noted that the embodiment of the present application does not limit the structure of the region detection model. The region detection model can be a RepVGG model, or a segmentation model such as U-Net, ERFNet, and PSPNet.

[0092] For example, the neural network used by the region detection model is a codec network with RepVGG as the skeleton. The region detection model performs semantic segmentation on the input sample image based on the neural network, and then outputs a Boolean value matrix corresponding to each pixel in the sample image. The elements of the Boolean value matrix indicate whether the corresponding pixel point is a drivable area. Then, based on the difference between the Boolean value matrix and the annotation information, the parameters of the region detection model are adjusted.

[0093] It should be noted that after the region detection model is trained, the vehicle terminal can also test the region detection model based on an evaluation dataset to determine whether the model's performance meets expectations. For scenes where the model performs poorly, such as rainy days, sunny scenes, traffic jams, and non-traffic jams, the number of sample images of the corresponding scenes can be increased, and the region detection model can be further trained. The evaluation dataset includes multiple labeled test images.

[0094] It should be noted that the embodiments of the present application are described using the example of a region detection model obtained by training and testing by a vehicle-mounted terminal. In some embodiments, the region detection model is obtained by training and testing by a server, and the vehicle-mounted terminal can obtain the trained and tested region detection model from the server, and then the vehicle-mounted terminal implements the decision information acquisition method provided in the embodiments of the present application based on the region detection model.

[0095] 303. Obtain a road image captured by the current vehicle while it is traveling.

[0096] In the embodiment of the present application, the vehicle can capture a road image in real time while driving. The vehicle-mounted terminal can obtain the captured road image, which can be any road image, and then input the road image into the above-mentioned area detection model to determine the non-drivable area and the drivable area in the road image.

[0097] 304. Determine a first target area in the road image based on the area detection model, where the first target area is a non-drivable area in the road image.

[0098] In an embodiment of the present application, the area detection model can divide the road image into a first target area and a second target area. The first target area is the non-drivable area of ​​the current vehicle in the road image, such as the sky, the lawn on the roadside, and the road surface where other vehicles are located; the second target area is the drivable area of ​​the current vehicle in the road image, such as a road surface without obstacles, and the obstacle is an object that prevents the vehicle from passing.

[0099] In some embodiments, the step of determining a first target area in a road image by the vehicle terminal includes: the vehicle terminal detecting the road image based on a region detection model to obtain an image Boolean matrix corresponding to the road image, wherein the elements in the image Boolean matrix represent the Boolean values ​​of corresponding pixels in the road image. The number of rows in the image Boolean matrix is ​​equal to the number of pixels in the road image in the vertical direction, and the number of columns in the image Boolean matrix is ​​equal to the number of pixels in the road image in the horizontal direction. That is, the elements in the image Boolean matrix correspond one-to-one with the pixels in the road image. After obtaining the image Boolean matrix, the vehicle terminal determines the image area corresponding to pixels with a value of 0 in the image Boolean matrix as the first target area. That is, the Boolean value corresponding to pixels in the first target area of ​​the road image in the image Boolean matrix is ​​0, and the Boolean value corresponding to pixels in the second target area of ​​the road image in the image Boolean matrix is ​​1. By inputting the road image into the region detection model, the region detection model performs semantic segmentation on the road image, and outputs the image Boolean matrix, the vehicle terminal can quickly determine whether the area where each pixel in the road image is located is a non-drivable area based on the image Boolean matrix, thereby improving processing efficiency.

[0100] 305 . Determine lane line information in the road image. The lane line information is used to indicate each lane line in the road image.

[0101] In an embodiment of the present application, the vehicle-mounted terminal can determine the position of lane lines in a road image by establishing a coordinate system. The vehicle-mounted terminal determines lane line information in a road image by constructing a road coordinate system with the current vehicle position as the origin. The vehicle-mounted terminal then determines the position of each lane line in the road image based on this road coordinate system. By establishing the road coordinate system, the vehicle-mounted terminal can accurately represent each lane line in the road image using a function.

[0102] For example, Figure 4 is a schematic diagram of a road coordinate system provided according to an embodiment of the present application. Figure 4 As shown, the road image includes three lanes and four lane markings. The current vehicle is traveling in the middle lane. The left side of the left lane and the right side of the right lane are both flower beds, which are not drivable areas. The road image also includes the sky, which is not drivable areas. The vehicle-mounted terminal constructs a road coordinate system based on the road image, with the origin of the road coordinate system being the current vehicle's location. The horizontal axis of the road coordinate system is the Y-axis, and the vertical axis is the X-axis. Each lane marking can be represented by a function between x and y.

[0103] In some embodiments, the vehicle-mounted terminal can recognize the road image based on an image recognition model, obtain the position information of each lane line in the road image, and then determine the position information of each lane line as the lane line information of the road image. The image recognition model is used to recognize the input image and output the lane line information. The embodiments of this application do not limit the image recognition model.

[0104] It should be noted that the vehicle currently operates autonomously under the control of the onboard terminal based on lane information. If no lane information is detected, the onboard terminal cannot safely control the vehicle. In this case, it is necessary to prompt the driver to take over to avoid a safety accident. Therefore, if the onboard terminal does not obtain lane information, it determines the fourth decision information, which prompts the driver to take control of the current vehicle. By prompting the driver to take control of the current vehicle based on the fourth decision information, the safety factor of autonomous driving can be improved.

[0105] 306. Determine decision information based on the lane line information and the first target area.

[0106] In an embodiment of the present application, the first target area in the road image may include static obstacles, such as roadside flower beds, utility poles, and mobile traffic lights; it may also include dynamic obstacles, such as other moving vehicles or pedestrians. Detecting static obstacles may be referred to as lane occupation detection, and detecting dynamic obstacles may be referred to as merging detection or lane change detection. The vehicle-mounted terminal can combine the lane markings with the first target area to determine whether there are obstacles affecting the current vehicle's travel, and thus determine decision information.

[0107] In some embodiments, the vehicle-mounted terminal determines decision information based on the two lane lines of the current vehicle's lane and the first target area. The vehicle-mounted terminal can obtain a first lane line and a second lane line from the lane line information, where the first lane line is the lane line to the left of the current vehicle's lane, and the second lane line is the lane line to the right of the current vehicle's lane. The vehicle-mounted terminal then determines decision information based on the first lane line, the second lane line, and the first target area. By obtaining the lane line corresponding to the current lane, the amount of computation required to determine whether an obstacle is affecting the current vehicle's driving can be reduced, and the judgment logic is more consistent with actual driving scenarios, which can improve the accuracy of decision information.

[0108] In some embodiments, the vehicle-mounted terminal is capable of determining whether a vehicle in an adjacent lane is about to change lanes, i.e., change lanes from an adjacent lane to the lane where the current vehicle is located. The vehicle-mounted terminal determines decision information based on a first lane line, a second lane line, and a first target area, including: determining a first lane area based on the first lane line, wherein the first lane area is a rectangular area centered on the first lane line. Determining a second lane area based on the second lane line, wherein the second lane area is a rectangular area centered on the second lane line, wherein the width of the rectangular area is no greater than the width of the lane, and the length of the rectangular area is set based on actual conditions, which is not limited in this embodiment of the present application. The vehicle-mounted terminal determines first decision information based on a positional relationship between at least one of the first lane area and the second lane area and the first target area. The first decision information indicates that a vehicle has been detected merging into the current lane, and controls the current vehicle to at least one of reduce speed, switch lanes, or steer to the left or right. If the first lane area overlaps with the first target area, the vehicle-mounted terminal controls the current vehicle to reduce speed or steer to the right. Alternatively, if the vehicle-mounted terminal overlaps with the first target area in the second lane area, the current vehicle is controlled to reduce speed or evade to the left. Alternatively, if the vehicle-mounted terminal overlaps with the first target area in both the first lane area and the second lane area, the current vehicle is controlled to reduce speed or switch lanes. By determining whether the first target area overlaps with at least one of the first lane area and the second lane area, it is possible to determine whether at least one of the first lane line and the second lane line is blocked, or at least one is about to be blocked, thereby determining whether there is a vehicle about to change lanes and cut in from an adjacent lane. Further, when it is determined that a vehicle is about to change lanes and cut in from an adjacent lane, the first decision information is determined so that the first decision information can control the current vehicle to respond to the lane change and cut in situation.

[0109] For example, the current vehicle is in the middle lane. The first lane area is a rectangular area centered on the first lane line. The length of the rectangular area is 20 meters, the width is 1 meter, and the long side of the rectangular area is parallel to the first lane line. If a vehicle changes lanes from the left lane to the middle lane, the vehicle will inevitably pass through the first lane line. Since the road surface where the vehicle is located will be determined as the first target area, the first target area and the first lane area will overlap. During the lane change process, the overlapping area will increase from small to large and then decrease, indicating that the vehicle has started the lane change, is passing the lane line, and has completed the lane change. If at least one of the first lane area and the second lane area overlaps with the first target area, the vehicle-mounted terminal determines the first decision information based on the change in the overlapping area, and then controls the current vehicle to reduce speed or avoid to the left or right.

[0110] In some embodiments, the vehicle-mounted terminal can determine whether there is an obstacle in the lane in which the vehicle is currently located. The vehicle-mounted terminal determines decision information based on a first lane line, a second lane line, and a first target area, including: determining a lane centerline based on the first lane line and the second lane line, where the lane centerline is the centerline of the lane in which the vehicle is currently located. Then, the vehicle-mounted terminal determines second decision information based on the positional relationship between the lane centerline and the boundary of the first target area. This second decision information indicates the presence of an obstacle in the current lane and controls the vehicle to reduce speed or switch lanes. If the distance between the lane centerline and the boundary of the first target area is less than a target distance, the vehicle-mounted terminal determines that there is an obstacle in the current lane and controls the vehicle to switch lanes or reduce speed. The target distance is a lateral distance that is less than half the lane width. By determining the distance between the lane centerline and the boundary of the first target area, it is possible to determine whether there is an obstacle in the lane in which the vehicle is currently located that would impede the vehicle's travel, thereby determining the second decision information. If an obstacle is determined to be present in the lane, the vehicle-mounted terminal controls the vehicle to switch lanes to avoid it.

[0111] It should be noted that when the first lane line or the second lane line is partially missing, it may be caused by traffic jam ahead or following a vehicle. If the on-board terminal prompts the driver to take over the vehicle, it is a false alarm, which affects the driver's driving experience. At this time, the on-board terminal can obtain the change information of the relative distance between the center line of the lane and the boundary of the first target area, and determine whether there is an obstacle based on the change information. The change information is used to indicate the relative distance between the center line of the lane and the boundary of the first target area, and the change trend within the target time before taking the road image. The target time can be 3 seconds, 5 seconds, and 1 minute, etc., and the embodiment of the present application does not limit this. The on-board terminal can determine the obstacle type of the obstacle based on the change trend of the relative distance between the center line of the lane and the boundary of the first target area. The obstacle type includes static obstacles and dynamic obstacles. If the trend of the change in the relative distance between the centerline of the lane and the boundary of the first target area is that the relative distance is getting smaller and smaller, the on-board terminal determines that there is a static obstacle in the lane where the current vehicle is located. That is, when the obstacle type is a static obstacle, the on-board terminal determines the second decision information. The second decision information is used to indicate that the current lane has a static obstacle and control the current vehicle to switch lanes. If the trend of the change in the relative distance between the centerline of the lane and the boundary of the first target area is that the relative distance is getting larger, fluctuating, or basically flat, the on-board terminal determines that there is a dynamic obstacle in the lane where the current vehicle is located. That is, when the obstacle type is a dynamic obstacle, the on-board terminal determines the third decision information based on the driving condition of the dynamic obstacle. The third decision information is used to indicate that there is a dynamic obstacle in the current lane and is currently in a slow driving state. The current vehicle is controlled to maintain driving in the current lane and perform one of acceleration, deceleration, and lane switching. It should be noted that the embodiment of the present application does not limit the method by which the on-board terminal obtains the change information of the first target area.

[0112] For example, if the vehicle is traveling in the middle lane and there is an overturned truck 100 meters ahead of the vehicle, occupying two-thirds of the middle lane and one-third of the left lane, the onboard terminal determines that the distance between the centerline of the middle lane and the boundary of the first target area is a negative value, meaning that the centerline passes through the first target area. Furthermore, based on the changes in the non-drivable area over a period of 5 seconds, the onboard terminal determines that the obstacle in the first target area is a static obstacle. The onboard terminal then determines the second decision information for controlling the vehicle to switch to the right lane. If the vehicle is traveling in the middle lane and there is an overturned truck 100 meters ahead of the vehicle, occupying one-third of the middle lane and two-thirds of the right lane, the lane width is 3.6 meters. After determining the centerline of the middle lane, the on-board terminal determines that the distance between the lane centerline and the boundary of the first target area is 0.6 meters, which is less than half the lane width. Furthermore, based on the changes in the no-drivable area over a period of 5 seconds, the on-board terminal determines that the obstacle in the first target area is a static obstacle. The on-board terminal then determines second decision information for controlling the current vehicle to switch to the left lane. If the current vehicle is traveling in the middle lane and a normally traveling truck is 50 meters ahead of the current vehicle, and the on-board terminal determines that the distance between the lane centerline and the boundary of the first target area is a negative value (i.e., the lane centerline passes through the first target area), and based on the changes in the no-drivable area over a period of 5 seconds, the on-board terminal determines that the obstacle in the first target area is a dynamic obstacle, the on-board terminal then determines second decision information for controlling the current vehicle to perform one of acceleration, deceleration, and lane switching based on the driving condition of the dynamic obstacle.

[0113] It should be noted that, in order to make the solution described in steps 301 to 306 easier to understand, see Figure 5 As shown, Figure 5 This is a flowchart of another method for obtaining decision information according to an embodiment of the present application. The process includes the following steps: 501. Obtaining sample data, which is labeled data; 502. Model training; 503. Model evaluation and iteration; 504. Importing and lane occupation detection; 505. Determining decision information.

[0114] It should be noted that the above embodiment exemplarily shows that the vehicle can control the current vehicle based on the positional relationship between the first lane line, the second lane line, the lane center line and the first target area. In actual application, the vehicle terminal can also, based on the first target area, combine radar obstacle recognition, relative speed detection, intention prediction, and change trend statistics and other technical means to comprehensively judge the actual road conditions, plan the current vehicle's driving path, and then control the current vehicle according to the driving path.

[0115] The technical solution provided by the embodiment of the present application can determine the position of the lane line on the road in the road image by determining the first target area and lane line information from the road image taken during the current vehicle's driving process, and can also determine the non-drivable area corresponding to the obstacles on the road in the road image, and then can determine the driving environment around the current vehicle based on the lane line and the non-drivable area, and then determine the decision information, so as to control the current vehicle to effectively avoid obstacles on the road according to the decision information, thereby improving the control efficiency of the vehicle. In addition, by identifying the drivable area and the non-drivable area based on the road image, the recall of extreme situations is improved. Compared with the defect that the traditional obstacle detection method can only identify objects existing in the database but cannot identify unknown objects, the drivable area detection provided by the embodiment of the present application can improve the effect of road surface recognition. In addition, on the basis of the solution of the present application, combined with the classic merging detection method, the accuracy of merging detection, that is, lane change and cut-in detection, can be significantly improved.

[0116] Figure 6 This is a block diagram of a decision information acquisition device provided according to an embodiment of the present application. The device is used to perform the steps in the above-mentioned decision information acquisition method, see Figure 6 The device includes: a first acquisition module 601, a perception module 602 and a decision module 603.

[0117] The first acquisition module 601 is used to acquire a road image captured by the current vehicle during driving;

[0118] A perception module 602 is configured to determine, in the road image, a first target area and lane line information, wherein the first target area is a non-drivable area in the road image, and the lane line information is configured to indicate each lane line in the road image;

[0119] The decision module 603 is configured to determine decision information based on the lane line information and the first target area, where the decision information is used to indicate how to control the current vehicle so that the current vehicle avoids the first target area.

[0120] In some embodiments, Figure 7 This is a block diagram of another decision information acquisition device provided according to an embodiment of the present application, see Figure 7 As shown, the decision module 603 includes:

[0121] An acquisition unit 6031 is configured to acquire a first lane line and a second lane line from the lane line information, wherein the first lane line is a lane line on the left side of the lane where the current vehicle is located, and the second lane line is a lane line on the right side of the lane where the current vehicle is located;

[0122] The decision unit 6032 is configured to determine decision information based on the first lane marking, the second lane marking, and the first target area.

[0123] In some embodiments, the decision unit 6032 is used to determine a first lane area based on the first lane line, where the first lane area is a rectangular area centered on the first lane line; determine a second lane area based on the second lane line, where the second lane area is a rectangular area centered on the second lane line; and determine first decision information based on a positional relationship between at least one of the first lane area and the second lane area and the first target area, where the first decision information is used to indicate that a vehicle is detected merging into the current lane, and to control the current vehicle to execute at least one of reducing speed, switching lanes, and avoiding to the left or right.

[0124] In some embodiments, the decision unit 6032 is used to determine the lane centerline based on the first lane line and the second lane line, where the lane centerline is the centerline of the lane where the current vehicle is located; and determine second decision information based on the positional relationship between the lane centerline and the boundary of the first target area, where the second decision information is used to indicate the presence of an obstacle in the current lane and control the current vehicle to reduce speed or switch lanes.

[0125] In some embodiments, the decision unit 6032 is used to determine the obstacle type of the obstacle based on the changing trend of the relative distance between the lane centerline and the boundary of the first target area when the first lane line or the second lane line is partially missing, and the obstacle type includes static obstacles and dynamic obstacles; when the obstacle type is a static obstacle, determine the second decision information.

[0126] In some embodiments, the decision unit 6032 is further used to determine third decision information when the obstacle type is a dynamic obstacle. The third decision information is used to indicate that the vehicle is currently in a slow driving state and to control the vehicle to maintain driving in the current lane.

[0127] In some embodiments, the decision module 603 is further configured to determine fourth decision information when the lane line information is not obtained, and the fourth decision information is configured to prompt the driver to control the current vehicle.

[0128] In some embodiments, the perception module 602 is further used to construct a road coordinate system with the current vehicle position as the origin; based on the road coordinate system, determine the position of each lane line in the road image to obtain the lane line information.

[0129] In some embodiments, the perception module 602 is also used to detect the road image based on the area detection model to obtain an image Boolean matrix corresponding to the road image, where the elements in the image Boolean matrix are used to represent the Boolean values ​​of the corresponding pixels in the road image, and the area detection model is used to detect the drivable area in the input image; the image area corresponding to the pixels with a value of 0 in the image Boolean matrix is ​​determined as the first target area.

[0130] In some embodiments, see Figure 7 As shown, the device also includes:

[0131] A second acquisition module 604 is configured to acquire a sample image and annotation information of the sample image, where the annotation information is used to indicate a drivable area and a non-drivable area in the sample image;

[0132] The training module 605 is configured to train the region detection model of the i-th iteration based on the sample image using the annotation information as supervision information, where i is a positive integer.

[0133] The technical solution provided in the embodiment of the present application can determine the position of the lane lines on the road in the road image by determining the first target area and lane line information from the road image taken during the current vehicle's driving process. It can also determine the non-drivable areas corresponding to obstacles on the road in the road image. Based on the lane lines and non-drivable areas, the driving environment around the current vehicle can be determined, and decision information can be determined. According to the decision information, the current vehicle can be controlled to effectively avoid obstacles on the road, thereby improving the control efficiency of the vehicle.

[0134] It should be noted that the decision information acquisition device provided in the above embodiment is only illustrated by the division of the above functional modules when performing vehicle control. In actual applications, the above functions can be assigned to different functional modules as needed, that is, the internal structure of the device can be divided into different functional modules to complete all or part of the functions described above. In addition, the decision information acquisition device provided in the above embodiment and the decision information acquisition method embodiment are based on the same concept. The specific implementation process is detailed in the method embodiment and will not be repeated here.

[0135] Figure 8The following is a block diagram of a terminal 800 according to an embodiment of the present application. Terminal 800 may be a portable mobile terminal, such as a smartphone, tablet computer, MP3 player (Moving Picture Experts Group Audio Layer III), MP4 player (Moving Picture Experts Group Audio Layer IV), laptop computer, or desktop computer. Terminal 800 may also be referred to as user equipment, portable terminal, laptop terminal, desktop terminal, or other similar names.

[0136] Typically, the terminal 800 includes a processor 801 and a memory 802 .

[0137] The processor 801 may include one or more processing cores, such as a 4-core processor, an 8-core processor, etc. The processor 801 may be implemented in at least one hardware form of DSP (Digital Signal Processing), FPGA (Field-Programmable Gate Array), or PLA (Programmable Logic Array). The processor 801 may also include a main processor and a coprocessor. The main processor is a processor for processing data in the awake state, also known as a CPU (Central Processing Unit); the coprocessor is a low-power processor for processing data in the standby state. In some embodiments, the processor 801 may be integrated with a GPU (Graphics Processing Unit), which is responsible for rendering and drawing the content to be displayed on the display screen. In some embodiments, the processor 801 may also include an AI (Artificial Intelligence) processor, which is used to process computing operations related to machine learning.

[0138] The memory 802 may include one or more computer-readable storage media, which may be non-transitory. The memory 802 may also include high-speed random access memory and non-volatile memory, such as one or more disk storage devices and flash memory storage devices. In some embodiments, the non-transitory computer-readable storage medium in the memory 802 is used to store at least one computer program, which is executed by the processor 801 to implement the decision information acquisition method provided in the method embodiment of the present application.

[0139] In some embodiments, terminal 800 may optionally include a peripheral device interface 803 and at least one peripheral device. The processor 801, memory 802, and peripheral device interface 803 may be connected via a bus or signal lines. Each peripheral device may be connected to peripheral device interface 803 via a bus, signal lines, or circuit boards. Specifically, the peripheral device may include at least one of a radio frequency circuit 804, a display screen 805, a camera assembly 806, an audio circuit 807, a positioning assembly 808, and a power supply 809.

[0140] The peripheral device interface 803 can be used to connect at least one I / O (Input / Output)-related peripheral device to the processor 801 and the memory 802. In some embodiments, the processor 801, the memory 802, and the peripheral device interface 803 are integrated on the same chip or circuit board; in some other embodiments, any one or two of the processor 801, the memory 802, and the peripheral device interface 803 can be implemented on separate chips or circuit boards, which is not limited in this embodiment.

[0141] The radio frequency circuit 804 is used to receive and transmit RF (Radio Frequency) signals, also known as electromagnetic signals. The radio frequency circuit 804 communicates with communication networks and other communication devices via electromagnetic signals. The radio frequency circuit 804 converts electrical signals into electromagnetic signals for transmission, or converts received electromagnetic signals into electrical signals. Optionally, the radio frequency circuit 804 includes: an antenna system, an RF transceiver, one or more amplifiers, a tuner, an oscillator, a digital signal processor, a codec chipset, a user identity module card, etc. The radio frequency circuit 804 can communicate with other terminals via at least one wireless communication protocol. Such wireless communication protocols include, but are not limited to, the World Wide Web, a metropolitan area network, an intranet, various generations of mobile communication networks (2G, 3G, 4G, and 5G), a wireless local area network, and / or a WiFi (Wireless Fidelity) network. In some embodiments, the radio frequency circuit 804 may also include circuits related to NFC (Near Field Communication), which is not limited in this application.

[0142] The display screen 805 is used to display a user interface (UI). This UI may include graphics, text, icons, videos, or any combination thereof. When the display screen 805 is a touch screen display, it is also capable of collecting touch signals on or above the surface of the display screen 805. These touch signals can be input as control signals to the processor 801 for processing. In this case, the display screen 805 can also be used to provide virtual buttons and / or a virtual keyboard, also known as soft buttons and / or a soft keyboard. In some embodiments, there can be one display screen 805, disposed on the front panel of the terminal 800; in other embodiments, there can be at least two display screens 805, disposed on different surfaces of the terminal 800 or in a foldable design; in still other embodiments, the display screen 805 can be a flexible display, disposed on a curved or foldable surface of the terminal 800. Furthermore, the display screen 805 can be configured as a non-rectangular irregular shape, i.e., a special-shaped screen. The display screen 805 can be made of materials such as LCD (Liquid Crystal Display) and OLED (Organic Light-Emitting Diode).

[0143] The camera assembly 806 is used to capture images or videos. Optionally, the camera assembly 806 includes a front camera and a rear camera. Typically, the front camera is arranged on the front panel of the terminal, and the rear camera is arranged on the back of the terminal. In some embodiments, there are at least two rear cameras, which are any one of a main camera, a depth of field camera, a wide-angle camera, and a telephoto camera, so as to realize the fusion of the main camera and the depth of field camera to realize the background blur function, the fusion of the main camera and the wide-angle camera to realize panoramic shooting and VR (Virtual Reality) shooting function or other fusion shooting functions. In some embodiments, the camera assembly 806 may also include a flash. The flash can be a monochrome temperature flash or a dual-color temperature flash. A dual-color temperature flash refers to a combination of a warm light flash and a cold light flash, which can be used for light compensation at different color temperatures.

[0144] The audio circuit 807 may include a microphone and a speaker. The microphone is used to collect sound waves from the user and the environment, and convert the sound waves into electrical signals that are input into the processor 801 for processing, or input into the radio frequency circuit 804 to achieve voice communication. For the purpose of stereo sound collection or noise reduction, there may be multiple microphones, each located in different parts of the terminal 800. The microphone may also be an array microphone or an omnidirectional collection microphone. The speaker is used to convert electrical signals from the processor 801 or the radio frequency circuit 804 into sound waves. The speaker may be a traditional thin film speaker or a piezoelectric ceramic speaker. When the speaker is a piezoelectric ceramic speaker, it can not only convert electrical signals into sound waves audible to humans, but also convert electrical signals into sound waves inaudible to humans for purposes such as ranging. In some embodiments, the audio circuit 807 may also include a headphone jack.

[0145] Positioning component 808 is used to locate the current geographic location of terminal 800 to implement navigation or LBS (Location Based Service). Positioning component 808 can be a positioning component based on the US GPS (Global Positioning System), China's Beidou system, or Russia's Galileo system.

[0146] Power supply 809 is used to power various components in terminal 800. Power supply 809 can be AC ​​power, DC power, a disposable battery, or a rechargeable battery. When power supply 809 includes a rechargeable battery, the rechargeable battery can be a wired rechargeable battery or a wireless rechargeable battery. A wired rechargeable battery is a battery that is charged via a wired line, while a wireless rechargeable battery is a battery that is charged via a wireless coil. The rechargeable battery can also be used to support fast charging technology.

[0147] In some embodiments, the terminal 800 further includes one or more sensors 810 , including but not limited to: an acceleration sensor 811 , a gyroscope sensor 812 , a pressure sensor 813 , a fingerprint sensor 814 , an optical sensor 815 , and a proximity sensor 816 .

[0148] The accelerometer 811 can detect the magnitude of acceleration along the three coordinate axes of the coordinate system established by the terminal 800. For example, the accelerometer 811 can be used to detect the components of gravity acceleration along the three coordinate axes. The processor 801 can control the display screen 805 to display the user interface in a landscape or portrait view based on the gravity acceleration signal collected by the accelerometer 811. The accelerometer 811 can also be used to collect game or user motion data.

[0149] The gyroscope sensor 812 can detect the orientation and rotation angle of the terminal 800. It can work in conjunction with the accelerometer 811 to collect the user's 3D movements of the terminal 800. Based on the data collected by the gyroscope sensor 812, the processor 801 can implement the following functions: motion sensing (for example, changing the UI based on the user's tilt operation), image stabilization during shooting, game control, and inertial navigation.

[0150] The pressure sensor 813 can be set on the side frame of the terminal 800 and / or the lower layer of the display screen 805. When the pressure sensor 813 is set on the side frame of the terminal 800, it can detect the user's grip signal of the terminal 800, and the processor 801 performs left and right hand recognition or shortcut operations based on the grip signal collected by the pressure sensor 813. When the pressure sensor 813 is set on the lower layer of the display screen 805, the processor 801 controls the operational controls on the UI interface based on the user's pressure operation on the display screen 805. The operational controls include at least one of a button control, a scroll bar control, an icon control, and a menu control.

[0151] The fingerprint sensor 814 is used to collect the user's fingerprint. The processor 801 identifies the user's identity based on the fingerprint collected by the fingerprint sensor 814, or the fingerprint sensor 814 identifies the user's identity based on the collected fingerprint. When the user's identity is recognized as a trusted identity, the processor 801 authorizes the user to perform relevant sensitive operations, such as unlocking the screen, viewing encrypted information, downloading software, making payments, and changing settings. The fingerprint sensor 814 can be set on the front, back, or side of the terminal 800. When a physical button or manufacturer logo is provided on the terminal 800, the fingerprint sensor 814 can be integrated with the physical button or manufacturer logo.

[0152] The optical sensor 815 is used to detect ambient light intensity. In one embodiment, the processor 801 can control the display brightness of the display screen 805 based on the ambient light intensity detected by the optical sensor 815. Specifically, when the ambient light intensity is high, the display brightness of the display screen 805 is increased; when the ambient light intensity is low, the display brightness of the display screen 805 is decreased. In another embodiment, the processor 801 can also dynamically adjust the shooting parameters of the camera assembly 806 based on the ambient light intensity detected by the optical sensor 815.

[0153] Proximity sensor 816, also known as a distance sensor, is typically located on the front panel of terminal 800. Proximity sensor 816 is used to detect the distance between the user and the front of terminal 800. In one embodiment, when proximity sensor 816 detects that the distance between the user and the front of terminal 800 is gradually decreasing, processor 801 controls display screen 805 to switch from the screen-on state to the screen-off state. When proximity sensor 816 detects that the distance between the user and the front of terminal 800 is gradually increasing, processor 801 controls display screen 805 to switch from the screen-off state to the screen-on state.

[0154] Those skilled in the art will understand that Figure 8 The structure shown in the figure does not constitute a limitation on the terminal 800, and the terminal 800 may include more or fewer components than shown in the figure, or combine certain components, or adopt a different component arrangement.

[0155] The present application also provides a computer-readable storage medium that stores at least one computer program. The at least one computer program is loaded and executed by a processor of a terminal to implement the operations performed by the terminal in the decision information acquisition method of the above embodiment. For example, the computer-readable storage medium can be a read-only memory (ROM), a random access memory (RAM), a compact disc read-only memory (CD-ROM), a magnetic tape, a floppy disk, an optical data storage device, etc.

[0156] The present application also provides a computer program product, including computer program code stored in a computer-readable storage medium. A processor of a terminal reads the computer program code from the computer-readable storage medium and executes the computer program code, causing the terminal to perform the decision information acquisition method provided in the various optional implementations described above.

[0157] Those skilled in the art will understand that all or part of the steps to implement the above embodiments may be accomplished by hardware, or by a program to instruct the relevant hardware, and the program may be stored in a computer-readable storage medium, which may be a read-only memory, a disk, or an optical disk, etc.

[0158] The above description is merely an optional embodiment of the present application and is not intended to limit the present application. Any modifications, equivalent replacements, improvements, etc. made within the spirit and principles of the present application shall be included in the scope of protection of the present application.

Claims

1. A decision information acquisition method, characterized in that: The method comprises: Obtain road images taken by the current vehicle during driving; In the road image, determining a first target area, where the first target area is a non-drivable area in the road image, and the first target area includes static obstacles and dynamic obstacles; Constructing a road coordinate system with the current vehicle position as an origin; determining the position of each lane line in the road image based on the road coordinate system to obtain lane line information, wherein the lane line information is used to indicate each lane line in the road image; Obtaining a first lane line and a second lane line from the lane line information, where the first lane line is a lane line on the left side of the lane where the current vehicle is located, and the second lane line is a lane line on the right side of the lane where the current vehicle is located; Determine a first lane region based on the first lane line, where the first lane region is a rectangular region centered on the first lane line; determine a second lane region based on the second lane line, where the second lane region is a rectangular region centered on the second lane line; the width of the rectangular region is no greater than the width of the lane; When the first lane area overlaps with the first target area, it is determined that a vehicle is detected merging into the current lane, and the current vehicle is controlled to slow down or steer to the right; when the second lane area overlaps with the first target area, it is determined that a vehicle is detected merging into the current lane, and the current vehicle is controlled to slow down or steer to the left; when both the first lane area and the second lane area overlap with the first target area, it is determined that a vehicle is detected merging into the current lane, and the current vehicle is controlled to slow down or switch lanes.

2. The method according to claim 1, characterized in that The method further comprises: Determine a lane centerline based on the first lane line and the second lane line, where the lane centerline is the centerline of the lane where the current vehicle is located; Second decision information is determined based on a positional relationship between the lane centerline and a boundary of the first target area. The second decision information is used to indicate that an obstacle exists in the current lane and control the current vehicle to reduce speed or switch lanes.

3. The method according to claim 2, characterized in that The determining of the second decision information based on the positional relationship between the lane centerline and the boundary of the first target area includes: When the first lane line or the second lane line is partially missing, determining the obstacle type of the obstacle based on a change trend of the relative distance between the lane centerline and the boundary of the first target area, the obstacle type including static obstacles and dynamic obstacles; When the obstacle type is a static obstacle, the second decision information is determined.

4. The method according to claim 3, characterized in that The method further comprises: In a case where the obstacle type is a dynamic obstacle, third decision information is determined, where the third decision information is used to indicate that the vehicle is currently in a slow driving state, and the current vehicle is controlled to maintain driving in the current lane.

5. The method according to any one of claims 1 to 4, characterized in that The method further comprises: In a case where the lane line information is not obtained, fourth decision information is determined, where the fourth decision information is used to prompt the driver to control the current vehicle.

6. The method according to any one of claims 1 to 4, characterized in that The method further comprises: Detecting the road image based on a region detection model to obtain an image Boolean matrix corresponding to the road image, wherein elements in the image Boolean matrix are used to represent Boolean values ​​of corresponding pixels in the road image, and the region detection model is used to detect a drivable area in the input image; An image region corresponding to pixels having a value of 0 in the image Boolean matrix is ​​determined as the first target region.

7. The method according to claim 6, characterized in that The method further comprises: Acquire a sample image and annotation information of the sample image, where the annotation information is used to indicate a drivable area and a non-drivable area in the sample image; The region detection model of the i-th iteration is trained based on the sample image using the labeling information as supervision information, where i is a positive integer.

8. A decision information acquisition device, characterized in that: The device comprises: The first acquisition module is used to acquire the road image taken by the current vehicle during driving; a perception module configured to determine a first target area in the road image, the first target area being a non-drivable area in the road image, the first target area including static obstacles and dynamic obstacles; construct a road coordinate system with the current vehicle position as an origin; and determine a position of each lane line in the road image based on the road coordinate system to obtain lane line information, the lane line information being used to indicate each lane line in the road image; Decision module, including acquisition unit and decision unit; The acquiring unit is configured to acquire a first lane line and a second lane line from the lane line information, wherein the first lane line is a lane line on the left side of the lane where the current vehicle is located, and the second lane line is a lane line on the right side of the lane where the current vehicle is located; The decision unit is configured to determine a first lane area based on the first lane line, the first lane area being a rectangular area centered on the first lane line; determine a second lane area based on the second lane line, the second lane area being a rectangular area centered on the second lane line; the width of the rectangular area being no greater than the width of the lane; when the first lane area overlaps with the first target area, determine that a vehicle is detected merging into the current lane, and control the current vehicle to slow down or evade to the right; when the second lane area overlaps with the first target area, determine that a vehicle is detected merging into the current lane, and control the current vehicle to slow down or evade to the left; when both the first lane area and the second lane area overlap with the first target area, determine that a vehicle is detected merging into the current lane, and control the current vehicle to slow down or switch lanes.

9. The device according to claim 8, characterized in that The decision unit is further used to determine a lane centerline based on the first lane line and the second lane line, where the lane centerline is the centerline of the lane where the current vehicle is located; and determine second decision information based on a positional relationship between the lane centerline and the boundary of the first target area, where the second decision information is used to indicate the presence of an obstacle in the current lane and control the current vehicle to reduce speed or switch lanes.

10. The device according to claim 9, characterized in that The decision unit is further configured to, when the first lane line or the second lane line is partially missing, determine the obstacle type of the obstacle based on a trend of change in the relative distance between the lane centerline and the boundary of the first target area, the obstacle types including static obstacles and dynamic obstacles; and determine the second decision information when the obstacle type is a static obstacle.

11. The device according to claim 10, characterized in that The decision unit is further configured to, when the obstacle type is a dynamic obstacle, determine third decision information, wherein the third decision information is configured to indicate that the vehicle is currently in a slow driving state and control the vehicle to maintain driving in the current lane.

12. The device according to any one of claims 8 to 11, characterized in that The decision module is further configured to determine fourth decision information when the lane line information is not obtained, and the fourth decision information is configured to prompt the driver to control the current vehicle.

13. The device according to any one of claims 8 to 11, characterized in that The perception module is further configured to detect the road image based on a region detection model to obtain an image Boolean matrix corresponding to the road image, wherein elements in the image Boolean matrix represent Boolean values ​​of corresponding pixels in the road image, and the region detection model is configured to detect drivable areas in the input image; An image region corresponding to pixels having a value of 0 in the image Boolean matrix is ​​determined as the first target region.

14. The device according to claim 13, characterized in that The device further comprises: a second acquisition module, configured to acquire a sample image and annotation information of the sample image, wherein the annotation information is used to indicate a drivable area and a non-drivable area in the sample image; The training module is used to train the region detection model of the i-th iteration based on the sample image using the labeling information as supervision information, where i is a positive integer.

15. A terminal, characterized in that: The terminal includes a processor and a memory, the memory is used to store at least one computer program, and the at least one computer program is loaded and executed by the processor to implement the decision information acquisition method according to any one of claims 1 to 7.

16. A computer-readable storage medium, characterized in that The computer-readable storage medium is used to store at least one computer program, and the at least one computer program is loaded and executed by a processor to implement the decision information acquisition method according to any one of claims 1 to 7.

17. A computer program product comprising a computer program, characterized in that When the computer program is executed by a processor, the decision information acquisition method according to any one of claims 1 to 7 is implemented.

Images