Target area determination method and device, equipment and computer storage medium

Abstract

The invention discloses a target area determination method and device, equipment and a computer storage medium, and belongs to the technical field of computer vision. And performing feature extraction on the driving scene image through the feature extraction model to obtain edge features and semantic features. And fusing the edge features with the semantic features to obtain a travelable boundary with a first granularity. And refining the travelable boundary to obtain a travelable boundary of a second granularity. And finally, determining a target area according to the travelable boundary. According to the embodiment of the invention, the target area in which the unmanned equipment can drive can be identified based on the driving scene image. In the recognition process, by determining the edge features and the semantic features of the driving scene image, the determination of the drivable boundary line for dividing the drivable area and the non-drivable area is realized, so that the target area is determined. The occurrence of holes is avoided, and the accuracy of identifying the target area is improved.

Description

Technical Field

[0001] The present application belongs to the field of computer vision technology, and in particular, relates to a method, device, equipment and computer storage medium for determining a target area. Background Art

[0002] When driving, unmanned vehicles need to determine the distance between themselves and obstacles, so as to plan driving decisions and achieve unmanned driving. Usually, before determining the distance between unmanned vehicles and obstacles, they need to obtain the target area where the unmanned vehicle can drive based on the driving scene images collected by the unmanned driving equipment.

[0003] In the prior art, after obtaining the driving scene image collected by the unmanned vehicle, each pixel in the driving scene image is usually subjected to binary classification processing based on the semantic segmentation model to determine whether the area corresponding to each pixel point is drivable, thereby determining the drivable target area in the driving scene image.

[0004] However, when faced with some scenes, the semantic segmentation model often finds it difficult to determine whether the areas corresponding to certain pixels are drivable, resulting in holes in the areas corresponding to these pixels. The determined drivable areas are inaccurate and deviate from the actual ones. Summary of the invention

[0005] The embodiments of the present application provide a target area determination method, apparatus, device and computer storage medium, which can improve the accuracy of identifying target areas where unmanned driving equipment can travel.

[0006] On the one hand, an embodiment of the present application provides a method for determining a target area, the method comprising:

[0007] Acquire the driving scene image of the unmanned driving device;

[0008] Extracting features from the driving scene image using a feature extraction model to obtain edge features and semantic features of objects in the driving scene image;

[0009] By slicing and splicing, the edge feature and the semantic feature are merged to obtain a drivable boundary line with a first granularity in the driving scene image;

[0010] Refining the drivable dividing line according to the distance between the actual area corresponding to the drivable dividing line and the unmanned driving device, and determining a drivable dividing line of a second granularity in the driving scene image, wherein the first granularity is greater than the second granularity;

[0011] A target area in which the unmanned driving device can drive is determined according to a drivable boundary line of a second granularity in the driving scene image.

[0012] Optionally, the feature extraction model includes N residual modules, and at least some of the residual modules do not include a pooling layer, and N is a positive integer greater than or equal to 2;

[0013] The step of extracting features from the driving scene image using a feature extraction model to obtain edge features and semantic features of objects in the driving scene image includes:

[0014] Inputting the driving scene image into a feature extraction model;

[0015] Based on the first M residual modules of the feature extraction model, feature extraction is performed on the driving scene image to obtain edge features of objects in the driving scene image, where M is a positive integer less than N;

[0016] Based on the N residual modules of the feature extraction model, feature extraction is performed on the driving scene image to obtain semantic features of objects in the driving scene image.

[0017] Optionally, the merging of the edge feature and the semantic feature by slicing and splicing to obtain a drivable boundary line with a first granularity in the driving scene image includes:

[0018] Determining the category to which the object in the driving scene image belongs according to a plurality of preset categories and the semantic features;

[0019] Slicing the semantic features according to different categories to obtain a number of slice features;

[0020] respectively splicing the slice features with the edge features to obtain several categories of splicing features;

[0021] The category splicing features of each category are respectively fused by means of group convolution to obtain a number of sub-drivable dividing lines carrying category identifiers, wherein the sub-drivable dividing lines are the first granularity;

[0022] According to the plurality of sub-drivable dividing lines, a drivable dividing line of a first granularity is determined in the driving scene image.

[0023] Optionally, before determining the target area in which the unmanned driving device can drive according to the drivable boundary line of the second granularity in the driving scene image, the method further includes:

[0024] According to the drivable dividing line, determining the category identification of each particle point constituting the drivable dividing line, wherein the particle point is the smallest element constituting the drivable dividing line;

[0025] Starting from the first particle point of the drivable dividing line, a sliding window filtering is performed on the category identification of each particle point in the drivable dividing line through a sliding window of preset width.

[0026] Optionally, determining a target area in which the unmanned driving device can drive according to a drivable boundary line of a second granularity in the driving scene image includes:

[0027] According to the drivable boundary lines of the second granularity composed of sub-drivable boundary lines carrying category identifiers, a target area in which the unmanned driving device can be driven and a category to which objects at the boundary of the target area belong are determined.

[0028] Optionally, the refining of the drivable boundary line according to the distance between the actual area corresponding to the drivable boundary line and the unmanned driving device, and determining the drivable boundary line of the second granularity in the driving scene image includes:

[0029] For each column of pixels in the driving scene image, the pixels in the column that coincide with the drivable boundary line are determined as boundary pixels;

[0030] According to the driving scene image, the pixel point at the end of the end closer to the unmanned driving device at both ends of the column of pixel points is determined as an endpoint pixel point;

[0031] Determine the boundary pixel point closest to the endpoint pixel point as the target pixel point;

[0032] In the driving scene image, a line segment composed of the target pixel points is determined as a drivable boundary line of the second granularity that has been refined.

[0033] Optionally, the refining of the drivable boundary line according to the distance between the actual area corresponding to the drivable boundary line and the unmanned driving device, and determining the drivable boundary line of the second granularity in the driving scene image includes:

[0034] The driving scene image marked with a drivable boundary line of the first granularity is input into the edge refinement module, and the convolution layer and the pooling layer of the edge refinement module are used for calculation to refine the drivable boundary line and determine the drivable boundary line of the second granularity.

[0035] Optionally, determining a target area in which the unmanned driving device can drive according to a drivable boundary line of a second granularity in the driving scene image includes:

[0036] Selecting a plurality of target demarcation points from the drivable demarcation lines of the second granularity;

[0037] Based on a monocular vision algorithm, respectively determine the target distance between each target demarcation point and the unmanned driving device;

[0038] A target area in which the unmanned driving device can travel is determined based on the plurality of target distances.

[0039] Optionally, the selecting of a plurality of target demarcation points from the drivable demarcation lines of the second granularity includes:

[0040] Acquiring the acquisition posture of the driving scene image;

[0041] According to the collected posture, determining a first line segment, a second line segment, and a third line segment from the driving scene image, wherein a midpoint of the first line segment, a first endpoint of the second line segment, and a first endpoint of the third line segment intersect at a line segment intersection point;

[0042] In the angle formed by the first line segment and the second line segment, with the intersection of the line segments as the center and at intervals of the first angle, auxiliary lines are drawn toward the drivable dividing line;

[0043] In the angle formed by the second line segment and the third line segment, an auxiliary line is drawn toward the drivable dividing line at intervals of the second angle with the intersection of the line segments as the center;

[0044] In the angle formed by the first line segment and the third line segment, an auxiliary line is drawn toward the drivable dividing line with the intersection of the line segments as the center and the third angle as the interval;

[0045] The intersection point of the auxiliary line and the drivable dividing line is determined as the target dividing point.

[0046] On the other hand, an embodiment of the present application provides a target area determination device, the device comprising:

[0047] An acquisition unit, used to acquire an image of a driving scene in which the unmanned driving device is located;

[0048] An extraction unit, used to extract features from the driving scene image through a feature extraction model to obtain edge features and semantic features of objects in the driving scene image;

[0049] A fusion unit, configured to fuse the edge feature with the semantic feature by slicing and splicing to obtain a drivable boundary line with a first granularity in the driving scene image;

[0050] a refinement unit, configured to refine the drivable boundary line according to a distance between an actual area corresponding to the drivable boundary line and the unmanned driving device, and determine a drivable boundary line of a second granularity in the driving scene image, wherein the first granularity is greater than the second granularity;

[0051] A determination unit is used to determine a target area in which the unmanned driving device can drive according to a drivable boundary line of a second granularity in the driving scene image.

[0052] On the other hand, an embodiment of the present application provides a target area determination device, the device comprising:

[0053] a processor and a memory storing computer program instructions;

[0054] When the processor executes the computer program instructions, the target area determination method as described in the above aspect is implemented.

[0055] On the other hand, an embodiment of the present application provides a computer storage medium, on which computer program instructions are stored. When the computer program instructions are executed by a processor, the target area determination method as described in any one of the above aspects is implemented.

[0056] On the other hand, an embodiment of the present application provides a computer program product. When instructions in the computer program product are executed by a processor of an electronic device, the electronic device implements the target area determination method as described in any one of the above aspects.

[0057] The target area determination method, device, equipment and computer storage medium of the embodiment of the present application can identify the target area where the unmanned driving equipment can drive based on the driving scene image. In the recognition process, by determining the edge features and semantic features of the driving scene image, the drivable boundary line that divides the drivable area and the non-drivable area is determined, thereby determining the target area. There is no need to perform semantic segmentation on each pixel in the driving scene image, which avoids the occurrence of holes and improves the accuracy of identifying the target area. BRIEF DESCRIPTION OF THE DRAWINGS

[0058] In order to more clearly illustrate the technical solution of the embodiments of the present application, the following is a brief introduction to the drawings required for use in the embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without any creative work.

[0059] Figure 1 is a flow chart of a method for determining a target area provided by an embodiment of the present application;

[0060] Figure 2 This is a schematic diagram of a feature extraction process provided by an embodiment of the present application;

[0061] Figure 3 is a schematic diagram of a drivable boundary line of the first granularity provided by an embodiment of the present application;

[0062] Figure 4 This is a schematic diagram of an edge thinning process provided by an embodiment of the present application;

[0063] Figure 5 is a schematic diagram of edge thinning effect provided by an embodiment of the present application;

[0064] Figure 6 This is a schematic diagram of an ideal situation of the refinement effect of the drivable dividing line provided by an embodiment of the present application;

[0065] Figure 7 This is a schematic diagram of determining a target demarcation point provided by an embodiment of the present application;

[0066] Figure 8 is a flow chart of a method for determining a target area provided by an embodiment of the present application;

[0067] Fig. 9 is a flowchart of a method for determining a target area provided by another embodiment of the present application;

[0068] Fig.10 This is a schematic diagram of a process of slice splicing provided by an embodiment of the present application;

[0069] Fig.11 This is a schematic diagram of a fusion process provided by an embodiment of the present application;

[0070] Fig.12 This is a schematic diagram of sliding window filtering provided by an embodiment of the present application;

[0071] Fig.13 It is a schematic diagram of the post-processing process provided by an embodiment of the present application;

[0072] Fig.14 is a schematic diagram of the structure of a target area determination device provided by an embodiment of the present application;

[0073] Fig.15 It is a structural diagram of a target area determination device provided by an embodiment of the present application. DETAILED DESCRIPTION

[0074] The features and exemplary embodiments of various aspects of the present application will be described in detail below. In order to make the purpose, technical solutions and advantages of the present application clearer, the present application will be further described in detail below in conjunction with the accompanying drawings and specific embodiments. It should be understood that the specific embodiments described herein are only intended to explain the present application, rather than to limit the present application. For those skilled in the art, the present application can be implemented without the need for some of these specific details. The following description of the embodiments is only to provide a better understanding of the present application by illustrating the examples of the present application.

[0075] It should be noted that, in this article, relational terms such as first and second, etc. are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, the elements defined by the statement "include..." do not exclude the presence of other identical elements in the process, method, article or device including the elements.

[0076] In order to solve the problems in the prior art, the embodiments of the present application provide a method, device, equipment and computer storage medium for determining a target area. The method for determining a target area provided by the embodiments of the present application is first introduced below.

[0077] Figure 1 FIG. 1 is a flow chart of a method for determining a target area according to an embodiment of the present application. Figure 1 As shown, the target area determination method provided in the embodiment of the present application includes the following steps: S101 to S105.

[0078] S101: Acquire a driving scene image of the unmanned driving device.

[0079] In one or more embodiments of the present application, the target area determination method may be performed by an electronic device. Of course, the electronic device may be a computer, a server, a mobile phone, a tablet computer, etc. The present application does not limit the specific device and may set the device as needed.

[0080] Specifically, the electronic device can obtain a driving scene image of the area where the unmanned driving device is located. The unmanned driving device can be an unmanned vehicle, a drone, or other device. The specific device is not limited in this application and can be set as needed. In one or more embodiments of this application, for the sake of convenience, the unmanned driving device is an unmanned vehicle as an example for description.

[0081] The driving scene image may be an image collected by a camera, image sensor or other device configured by the unmanned driving device. The application does not limit how the driving scene image is collected, and it may be set as needed. It should be noted that since the driving scene image is used to determine the target area where the unmanned driving device can drive, the subject of the driving scene image is the objects in the area around the unmanned driving device.

[0082] By adopting the above method, the electronic device can obtain a driving scene image of the scene where the unmanned driving device is located, so as to determine a target area where the unmanned driving device can drive based on the driving scene image.

[0083] S102: extracting features from the driving scene image using a feature extraction model to obtain edge features and semantic features of objects in the driving scene image.

[0084] Since the driving scene image may contain pedestrians, other driving equipment (vehicles, drones, etc.), traffic lights, green belts, road surfaces, and other objects, and only the area corresponding to the road surface is the area where the unmanned vehicle can drive. Therefore, the electronic device can perform feature extraction on the driving scene image based on the feature extraction model to determine the semantic features and edge features of the objects contained in the driving scene image. Among them, the semantic features characterize what kind of objects certain objects in the driving scene image are. The edge features are the edge lines of each object in the driving scene image.

[0085] Specifically, the electronic device may input the driving scene image into a feature extraction model. The feature extraction model may be a deep residual network (ResNet) model, a bidirectional long short-term memory (BILSTM) model, or the like. The application does not limit the specific model of the feature extraction model, and the model may be set as required.

[0086] In general, deep learning models usually include multiple layers, where the features output by the deep layers often have a larger receptive field, contain features at a higher semantic level, and are suitable for extracting semantic information. On the other hand, the features output by the shallow layers often have a smaller receptive field, are more sensitive to details and textures in the image, can provide detailed edge location information, and are suitable for extracting edge information.

[0087] Therefore, in one or more embodiments of the present application, the feature extraction model includes N residual modules (ie, the output layer in the model), where N is a positive integer greater than or equal to 2.

[0088] The electronic device can perform feature extraction on the driving scene image based on the first M residual modules in the feature extraction model to obtain edge features of several objects in the driving scene image, where M is a positive integer less than N. That is to say, the electronic device can determine the outputs of the first M residual modules of the feature extraction model as edge features of the driving scene image. In addition, since the pooling layer has the functions of increasing the receptive field and reducing information redundancy, a smaller receptive field is required when outputting edge features. Therefore, in one or more embodiments of the present application, when determining the feature extraction model, the electronic device can determine that some residual modules of the feature extraction model do not include a pooling layer.

[0089] The electronic device can also perform feature extraction on the driving scene image based on the N residual modules of the feature extraction model to obtain semantic features of objects in the driving scene image.

[0090] It should be noted that, in general, when extracting features from the driving scene image, the feature extraction model needs to process the driving scene image separately through several cascaded residual modules, and the residual module usually includes a convolution layer. Therefore, the sizes of edge features output by different residual modules are often different. Therefore, in one or more embodiments of the present application, the electronic device may upsample the edge features and semantic features output by the residual module so that the edge features extracted from the driving scene image are consistent in size with the semantic features.

[0091] Figure 2 A schematic diagram of a feature extraction process provided for an embodiment of the present application is shown in FIG. Figure 2 As shown, after the driving scene image is input according to step 201, the driving scene image passes through the first residual module 202, the second residual module 203, the third residual module 204, the fourth residual module 205 and the fifth residual module 206 in sequence to realize feature extraction. The first residual module 202 outputs a first sub-edge feature 210, the data output by the second residual module 203 is up-sampled by the up-sampling module 207 to obtain a second sub-edge feature 211, the data output by the third residual module 204 is up-sampled by the up-sampling module 208 to obtain a third sub-edge feature 212, and the data output by the fifth residual module 206 is up-sampled by the up-sampling module 209 to obtain a semantic feature 213.

[0092] By adopting the above method, the electronic device can extract edge features and semantic features of the driving scene image through the feature extraction model, so as to identify the target area where the unmanned vehicle can drive based on the driving scene image.

[0093] S103: The edge feature and the semantic feature are merged by slicing and splicing to obtain a drivable boundary line with a first granularity in the driving scene image.

[0094] In one or more embodiments of the present application, after the electronic device completes the extraction of edge features and semantic features of the driving scene image, it can fuse the edge features and the semantic features to determine a drivable dividing line that divides a drivable area and a non-drivable area.

[0095] Specifically, the electronic device can process the edge feature and the semantic feature by slicing and splicing to obtain the feature to be fused. Then, the feature to be fused is fused by group convolution to determine the drivable boundary line of the first granularity in the driving scene image. The value of the first granularity is large, indicating that the drivable boundary line is relatively rough. Figure 3 A schematic diagram of a drivable boundary line of the first granularity provided in an embodiment of the present application is as follows: Figure 3 As shown, in the driving scene image 300 , the white area is a drivable boundary line 301 of the first granularity.

[0096] Among them, since the technologies of slice splicing and group convolution are relatively mature, for the sake of brevity, this application will not go into details about how to process the edge feature and the semantic feature through slice splicing, and how to fuse the features to be fused through group convolution.

[0097] By adopting the above method, the electronic device can determine a drivable boundary line of the first granularity, so as to determine the target area where the unmanned vehicle can drive.

[0098] S104: Refining the drivable boundary line according to the distance between the actual area corresponding to the drivable boundary line and the unmanned driving device, and determining a drivable boundary line of a second granularity in the driving scene image, wherein the first granularity is greater than the second granularity.

[0099] Since the edge feature and the semantic feature are processed by slicing, splicing and fusion, the first granularity of the drivable boundary line is relatively rough. In other words, the drivable boundary line covers a large position in the driving scene image, and the accuracy of the target area obtained based on the drivable boundary line of the first granularity is low. Therefore, in one or more embodiments of the present application, the electronic device can refine the drivable boundary line of the first granularity, so as to determine a more accurate target area based on the refined drivable boundary line.

[0100] Specifically, the electronic device may input a driving scene image marked with a drivable boundary line of a first granularity into an edge refinement module, and perform calculations through the convolution layer and the pooling layer of the edge refinement module to refine the drivable boundary line, and determine a drivable boundary line of a second granularity in the driving scene image. The second granularity is smaller than the first granularity.

[0101] Figure 4 A schematic diagram of an edge refinement process provided by an embodiment of the present application is shown in FIG. Figure 4 As shown, sub-driving dividing line 401, sub-driving dividing line 402 and sub-driving dividing line 403 are processed by edge refinement module 404 including convolution layer and pooling layer to obtain sub-driving dividing line 405, sub-driving dividing line 406 and sub-driving dividing line 407.

[0102] Figure 5 A schematic diagram of edge refinement effect provided by an embodiment of the present application is shown in FIG. Figure 5 As shown, in the driving scene image 500, line segment A and line segments constitute a drivable boundary line of the second granularity, specifically, line segment A is a boundary line composed of pixel points of obstacle category information, and line segment B is a boundary line composed of pixel points of the drivable area.

[0103] In one or more embodiments of the present application, the electronic device may also determine, for each column of pixels in the driving scene image, pixels in the column that coincide with the drivable dividing line as boundary pixels.

[0104] Figure 6 A schematic diagram of an ideal situation of the refinement effect of the drivable dividing line provided in one embodiment of the present application, such as Figure 6 In the driving scene image 600 without edge refinement, the drivable boundary line includes pixel points 602 filled with the first gray value (in Figure 6 The vertical lines in the figure represent the first grayscale value) and the pixel points 601 filled with the second grayscale value (in Figure 6 The second gray value is represented by a horizontal line in the figure). The edge-thinned driving scene image 603 includes pixel points 602 filled with the first gray value. After the pixel points 602 filled with the first gray value included in the edge-thinned driving scene image 603 are extracted, a complete row of pixel points 604 is obtained.

[0105] According to the driving scene image, the acquisition posture of the driving scene image is determined. And based on the acquisition posture, a direction closer to the unmanned vehicle is determined from the four directions (up, down, left, and right) of the driving scene image. Figure 2 As shown, obviously, Figure 2The lower part of the middle image (bottom of the image) is closer to the unmanned driving device. Based on the direction closer to the unmanned driving device, the electronic device can determine the pixel at the end of the end closer to the unmanned driving device from the two ends of the pixel point as the endpoint pixel point. Then, the boundary pixel point closest to the endpoint pixel point is determined as the target pixel point. And in the driving scene image, the line segment composed of the target pixel point is determined as the drivable boundary line of the second granularity that has been refined.

[0106] By adopting the above method, the electronic device can refine the drivable dividing line so as to more accurately determine the target area based on the refined drivable dividing line.

[0107] S105: Determine a target area in which the unmanned driving device can drive according to a drivable boundary line of a second granularity in the driving scene image.

[0108] In one or more embodiments of the present application, after determining a drivable decomposition line that divides a drivable area from a non-drivable area and determining that the drivable dividing line is a second granularity, the electronic device can determine a target area in which the unmanned vehicle can drive based on the drivable dividing line.

[0109] Specifically, the electronic device may determine the direction of the driving scene image that is closer to the unmanned vehicle as the target direction based on the content of step S104. Then, based on the drivable boundary line of the second granularity in the driving scene image and the target edge corresponding to the target direction in the driving scene image, determine the area enclosed by the drivable boundary line and the target edge as the image target area. Finally, the electronic device may determine the target area based on the corresponding relationship between the driving scene image and the actual map and according to the image target area. Figure 5 As shown, line segment A, line segment B and the area below the vehicle in the image are the image target area.

[0110] In some cases, after determining the target area, the unmanned vehicle needs to determine the distance between the unmanned device and the obstacles at the boundary of the target area in order to drive normally. Therefore, in one or more embodiments of the present application, the electronic device can also determine the target distance between the unmanned device and several target demarcation points on the drivable boundary line before determining the target area.

[0111] Specifically, the electronic device can determine several target demarcation points from the drivable demarcation line of the second granularity. Then based on the monocular vision algorithm, the target distance between each target demarcation point and the unmanned vehicle is determined respectively. Of course, the electronic device can also determine the distance between the target demarcation point and the unmanned vehicle by other methods, for example, in combination with a laser radar, determine the distance between the obstacle corresponding to the target demarcation point and the unmanned vehicle, or determine the distance between the target demarcation point and the unmanned vehicle by an algorithm such as a distance transform (DT) algorithm. The specific method of determining the distance between the target demarcation point and the unmanned vehicle is not limited by this application and can be set as needed. Finally, the electronic device can determine the target area where the unmanned device can be driven from the driving scene image based on the target distance between the unmanned vehicle and several target demarcation points, and the drivable demarcation line of the second granularity.

[0112] Among them, since the chip configured in the unmanned vehicle is often required to perform multiple tasks such as target detection, lane detection, and determination of driving instructions at the same time during driving, the computing power allocated to the target area identification is limited. Therefore, in one or more embodiments of the present application, when determining the target demarcation point, the electronic device can randomly select a preset number of particle points from the several particle points that constitute the drivable demarcation line and determine them as the target demarcation point. Thereby reducing the target distance that needs to be determined, improving efficiency, and reducing the occupation of chip computing resources. Among them, the particle point is the smallest element (smallest unit) that constitutes the drivable demarcation line.

[0113] Furthermore, because usually, the driving scene image is collected by the image sensor during the driving process of the unmanned vehicle, and when configuring the image sensor, the unmanned vehicle usually uses the direction that needs more attention as the orientation of the image sensor, so that the image collected by the image sensor can better meet the needs of the unmanned vehicle. For example, because the unmanned vehicle pays more attention to the information in front of the vehicle (the direction of the front of the vehicle) during driving, the unmanned vehicle is usually configured with an image sensor facing the front of the vehicle. Therefore, in one or more embodiments of the present application, when the electronic device determines the target demarcation point from the drivable demarcation line, it can also determine more target demarcation points at the center position of the driving scene image.

[0114] The electronic device can obtain the acquisition posture of the driving scene image, and according to the acquisition posture, determine a direction closer to the unmanned vehicle from the four directions (up, down, left, and right) of the driving scene image. Then, according to the edge of the direction, determine the first line segment from the driving scene image, and the first line segment coincides with the edge of the direction. Then, take the midpoint of the first line segment as the starting point, and make the second line segment and the third line segment respectively. The midpoint of the first line segment is also the first endpoint of the second line segment and the third line segment. Among them, the angle between the second line segment and the first line segment is the first angle, and the angle between the third line segment and the first line segment is the second angle. The first angle is not equal to the second angle, and the sum of the first angle and the second angle is not equal to the straight angle. That is to say, the midpoint of the first line segment, the first endpoint of the second line segment, and the first endpoint of the third line segment intersect at the line segment intersection.

[0115] Therefore, the electronic device can make auxiliary lines toward the drivable dividing line in the angle formed by the first line segment and the second line segment, with the intersection of the line segments as the center and the first angle as the interval. In the angle formed by the second line segment and the third line segment, with the intersection of the line segments as the center and the second angle as the interval, auxiliary lines can be made toward the drivable dividing line. In the angle formed by the first line segment and the third line segment, with the intersection of the line segments as the center and the third angle as the interval, auxiliary lines can be made toward the drivable dividing line. The second angle is smaller than the first angle, and the second angle is smaller than the third angle. Preferably, the angle between the second line segment and the third line segment is a right angle.

[0116] Figure 7 A schematic diagram for determining a target demarcation point is provided for an embodiment of the present application, such as Figure 7 As shown. A plurality of target dividing lines are set in the driving scene image 700. Among them, in the driving scene image 701, the first line segment 702, the second line segment 703 and the third line segment 704 intersect at the midpoint 708 of the first line segment. The auxiliary line angle between the first line segment 702 and the second line segment 703 is a first angle 705, the auxiliary line angle between the second line segment 703 and the third line segment 704 is a second angle 706, and the auxiliary line angle between the first line segment 702 and the third line segment 704 is a third angle 707. The intersection of the auxiliary line and the drivable boundary line 709 is the target boundary point.

[0117] Finally, the electronic device can determine the intersection points of the auxiliary line and the drivable dividing line as target dividing points.

[0118] It should be noted that Figure 7 The target dividing lines in the center of the image are denser than the target dividing lines on both sides. The denser the target dividing lines are, the smaller the angle between the two target dividing lines is. Figure 7The relationship among the first angle 705 , the second angle 706 and the third angle 707 may be: first angle 705 < second angle 706 < third angle 707 .

[0119] In addition, it should be noted that Figure 7 In the embodiment, multiple target dividing lines are divided, and the angle between two dividing lines defines the angle. In another embodiment, Figure 7 The target dividing lines in the image can also be divided into multiple target dividing lines with equal angle intervals, that is, the angles between two target dividing lines are the same, and the specific degree of the angle is not limited.

[0120] Figure 8 A flowchart of a method for determining a target area provided in another embodiment of the present application is shown in FIG. Figure 8 As shown. After inputting the driving scene image to the feature extraction module 802 according to step 801, the edge feature 804 and the semantic feature 803 can be obtained based on the feature extraction module 802, and the edge feature 804 and the semantic feature 803 are input into the semantic edge detection module 805. After fusion, the drivable boundary line of the first granularity is obtained. The drivable boundary line of the first granularity is input into the edge refinement module 806 to obtain the drivable boundary line of the second granularity. Finally, based on the drivable boundary line of the second granularity, the drivable area is determined, and step 807 is executed to output the drivable area.

[0121] By adopting the above method, the electronic device can recognize the target area where the unmanned driving device can travel, and there is no need to complete the recognition through semantic segmentation, thus avoiding the occurrence of voids and improving the accuracy.

[0122] The above is a specific implementation of the target area determination method provided in the embodiment of the present application. It can be seen that in the above embodiment, the electronic device can identify the target area where the unmanned driving device can drive based on the driving scene image. In the recognition process, by determining the edge features and semantic features of the driving scene image, the determination of the drivable boundary line that divides the drivable area and the non-drivable area is achieved, thereby determining the target area. There is no need to perform semantic segmentation on each pixel in the driving scene image, which avoids the appearance of holes and improves the accuracy of identifying the target area.

[0123] In order to further improve the accuracy of identifying the target area in which the unmanned driving equipment can travel, as another implementation method of the present application, the present application also provides another implementation method of the target area determination method, please refer to the following embodiments for details.

[0124] See also Fig. 9 Another implementation of the target area determination method provided in the present application includes the following steps:

[0125] S901: Acquire a driving scene image of the unmanned driving device.

[0126] S902: extracting features from the driving scene image using a feature extraction model to obtain edge features and semantic features of objects in the driving scene image.

[0127] S903: Determine the category to which the object in the driving scene image belongs according to a plurality of preset categories and the semantic features.

[0128] Specifically, since the semantic features characterize what kind of objects certain objects in the driving scene image are, the edges of different objects may have different properties. For example, the edges of objects such as vehicles and pedestrians may change over time, while objects such as traffic lights and green belts are fixed and will not change over time. Therefore, in one or more embodiments of the present application, the electronic device can classify different objects. And when determining the drivable dividing line, determine which category the edge lines of each object that constitutes the drivable dividing line correspond to.

[0129] Specifically, the electronic device can determine the category to which each object in the driving scene image belongs according to a plurality of preset categories and the semantic feature.

[0130] S904: Slice the semantic features according to different categories to obtain a number of slice features.

[0131] Specifically, the electronic device may slice the semantic feature according to different categories.

[0132] S905: respectively splicing the slice features and the edge features to obtain several categories of splicing features.

[0133] Specifically, the electronic device may splice each slice feature with the edge feature according to the following formula to obtain several categories of splicing features:

[0134]

[0135] Among them, A f is the category concatenation feature, as well as They are all edge features. is the semantic feature corresponding to the first category, is the semantic feature corresponding to the K-th category.

[0136] Fig.10 A schematic diagram of a process of slice splicing provided in an embodiment of the present application is shown in FIG. Fig.10As shown, after slicing the semantic feature 1001, a first slice feature 1002, a second slice feature 1003 and a third slice feature 1004 are obtained, and the first slice feature 1002, the second slice feature 1003 and the third slice feature 1004 are fused with the first sub-edge feature 1005, the second sub-edge feature 1006 and the third sub-edge feature 1007 respectively to obtain a first category splicing feature 1008, a first category splicing feature 1009 and a first category splicing feature 1010.

[0137] S906: The category concatenation features of each category are respectively fused by means of group convolution to obtain a plurality of sub-drivable boundaries carrying category identifiers, where the sub-drivable boundaries are the first granularity.

[0138] Specifically, the electronic device may fuse the category splicing features by group convolution according to the following formula to obtain a category edge feature map corresponding to each category:

[0139]

[0140] Among them, A k is the category edge feature map corresponding to the K-th category, W k is the parameter of group convolution, is the edge feature of the K-th category, is the semantic feature corresponding to the K-th category.

[0141] Fig.11 A schematic diagram of the fusion process provided for an embodiment of the present application is shown in FIG. Fig.11 As shown, after the first category splicing feature 1101 is processed by group convolution 1102, a sub-drivable boundary line 1103 is obtained. After the first category splicing feature 1104 is processed by group convolution 1105, a sub-drivable boundary line 1106 is obtained. After the first category splicing feature 1107 is processed by group convolution 1108, a sub-drivable boundary line 1109 is obtained.

[0142] S907: Determine a drivable boundary line of a first granularity in the driving scene image according to the plurality of sub-drivable boundary lines.

[0143] Specifically, the electronic device may splice the plurality of sub-drivable dividing lines, and determine a drivable dividing line of a first granularity in the driving scene image.

[0144] S908: Refining the drivable dividing line according to the distance between the actual area corresponding to the drivable dividing line and the unmanned driving device, and determining a drivable dividing line of a second granularity in the driving scene image.

[0145] S909: Determine a target area in which the unmanned driving device can drive according to a drivable boundary line of a second granularity in the driving scene image.

[0146] When determining the sub-drivable dividing lines that constitute the drivable dividing line, errors may occur, resulting in errors in the category identification of the smallest element (unit) that constitutes the drivable dividing line. Therefore, in one or more embodiments of the present application, the electronic device may also perform sliding filtering on the drivable dividing line before executing step S909.

[0147] Specifically, the electronic device can determine the category identifier of each minimum element constituting the drivable dividing line according to the drivable dividing line, and the minimum element is a particle point.

[0148] Then, starting from the first particle point of the drivable dividing line, or starting from the particle point at the end of one end of the drivable dividing line, based on a sliding window of a preset width, the class identification of each particle point in the drivable dividing line is subjected to sliding window filtering.

[0149] Fig.12 A schematic diagram of sliding window filtering is provided for an embodiment of the present application, such as Fig.12 As shown, the drivable dividing line 1200 includes particle points 1201 filled with a first color (in Fig.12 The first color is represented by a vertical line in the figure), and the particle points 1202 filled with the second color (in Fig.12 The horizontal line in the figure indicates the second color) and the particle points 1203 filled with the third color (in Fig.12 The third color is indicated by a diagonal line in the figure). The category of the particle point 1201 filled with the first color is identified as A, the category of the particle point 1202 filled with the second color is identified as B, and the category of the particle point 1203 filled with the third color is identified as C. Obviously, the category identification corresponding to the particle point 1202 filled with the second color is wrong. After sliding window filtering, a drivable dividing line 1204 is obtained. In the drivable dividing line 1204, the particle point 1202 filled with the second color is changed to the particle point 1201 filled with the first color.

[0150] In addition, in one or more embodiments of the present application, the electronic device can also determine the target area in which the unmanned driving device can be driven, and the category to which the object at the boundary of the target area belongs, based on the second-granularity drivable boundary composed of sub-drivable boundaries carrying category identifiers.

[0151] in addition, Fig.13 A schematic diagram of a post-processing process provided for an embodiment of the present application is shown in FIG. Fig.13As shown. Post-processing mainly includes the steps of sliding filtering the drivable dividing line, determining the target distance, and outputting the target area. After executing step 1301 to input the particle points into the post-processing module, the post-processing module can execute step 1302 to detect and eliminate the erroneous points in the particle points. Then execute step 1303 to select the dividing point from the particle points of the drivable dividing line. Then execute step 1304 to determine the target distance based on the selected dividing point. Finally, execute step 1305 to output the target area.

[0152] S901 to S902 are the same as S101 to S102 in the above embodiment, and S908 to S909 are the same as S104 to S105 in the above embodiment, and for the sake of brevity, they will not be described in detail here.

[0153] In the above embodiment, the electronic device determines the category identifiers of the sub-drivable dividing lines that constitute the drivable dividing line, so that when determining the target area in which the unmanned driving device can drive, it determines the category of the objects at the boundary of the target area, thereby more accurately and comprehensively determining the information of the target area.

[0154] In addition, the acquisition, storage, use, and processing of data in the technical solution of this application comply with the relevant provisions of national laws and regulations.

[0155] Based on the target area determination method provided in the above embodiment, the present application also provides a specific implementation of a target area determination device. Please refer to the following embodiment.

[0156] See first Fig.14 , the target area determination device provided in the embodiment of the present application includes the following units:

[0157] The acquisition unit 1401 is used to acquire an image of a driving scene in which the unmanned driving device is located;

[0158] An extraction unit 1402 is used to extract features from the driving scene image using a feature extraction model to obtain edge features and semantic features of objects in the driving scene image;

[0159] A fusion unit 1403 is used to fuse the edge feature with the semantic feature by slicing and splicing to obtain a drivable boundary line with a first granularity in the driving scene image;

[0160] A refinement unit 1404 is configured to refine the drivable boundary line according to the distance between the actual area corresponding to the drivable boundary line and the unmanned driving device, and determine a drivable boundary line of a second granularity in the driving scene image, wherein the first granularity is greater than the second granularity;

[0161] The determination unit 1405 is used to determine a target area in which the unmanned driving device can drive according to a drivable boundary line of a second granularity in the driving scene image.

[0162] The target area determination device of the embodiment of the present application can identify the target area where the unmanned driving device can drive based on the driving scene image. During the recognition process, the extraction unit 1402 determines the edge features and semantic features of the driving scene image to determine the drivable boundary line that divides the drivable area and the non-drivable area, thereby determining the target area based on the determination unit 1405. There is no need to perform semantic segmentation on each pixel in the driving scene image, which avoids the appearance of holes and improves the accuracy of identifying the target area.

[0163] As another implementation of the present application, in order to further improve the accuracy of identifying the target area where the unmanned driving equipment can travel, the above-mentioned device may further specifically include: an extraction subunit.

[0164] The extraction subunit is used to input the driving scene image into a feature extraction model, perform feature extraction on the driving scene image based on the first M residual modules of the feature extraction model, and obtain edge features of objects in the driving scene image, where M is a positive integer less than N, and perform feature extraction on the driving scene image based on the N residual modules of the feature extraction model to obtain semantic features of objects in the driving scene image.

[0165] As another implementation of the present application, in order to further improve the accuracy of identifying the target area where the unmanned driving equipment can travel, the above-mentioned device may also specifically include: a fusion subunit.

[0166] The fusion subunit is used to determine the category to which the object in the driving scene image belongs according to several preset categories and the semantic features, slice the semantic features according to different categories to obtain several slice features, splice the slice features with the edge features respectively to obtain several category splicing features, fuse the category splicing features of each category respectively by group convolution to obtain several sub-drivable dividing lines carrying category identifications, the sub-drivable dividing lines are of the first granularity, and determine the drivable dividing lines of the first granularity in the driving scene image according to the several sub-drivable dividing lines.

[0167] As another implementation of the present application, in order to further improve the accuracy of identifying the target area where the unmanned driving equipment can travel, the above-mentioned device may also specifically include: a determination subunit.

[0168] The determination subunit is used to determine the category identification of each particle point constituting the drivable dividing line according to the drivable dividing line, the particle point is the smallest element constituting the drivable dividing line, and starting from the first particle point of the drivable dividing line, the category identification of each particle point in the drivable dividing line is subjected to sliding window filtering through a sliding window of a preset width.

[0169] As another implementation of the present application, in order to further improve the accuracy of identifying the target area where the unmanned driving equipment can travel, the above-mentioned device may also specifically include: a determination subunit.

[0170] The determination subunit is used to determine the target area where the unmanned driving device can drive and the category to which the object at the boundary of the target area belongs based on the second granularity of the drivable boundary composed of the sub-drivable boundary lines carrying the category identifier.

[0171] As another implementation of the present application, in order to further improve the accuracy of identifying the target area where the unmanned driving equipment can travel, the above-mentioned device may further specifically include: a refinement subunit.

[0172] The refinement subunit is used to determine, for each column of pixels in the driving scene image, the pixel points in the column that coincide with the drivable dividing line as boundary pixels, and according to the driving scene image, determine the pixel points at the ends of the column of pixels that are closer to the unmanned driving device as endpoint pixels, and determine the boundary pixel point closest to the endpoint pixel point as the target pixel point, and in the driving scene image, determine the line segment composed of the target pixel points as the drivable dividing line of the second granularity that has been refined.

[0173] As another implementation of the present application, in order to further improve the accuracy of identifying the target area where the unmanned driving equipment can travel, the above-mentioned device may further specifically include: a refinement subunit.

[0174] The refinement subunit is used to input the driving scene image marked with the drivable dividing line of the first granularity into the edge refinement module, and perform calculations through the convolution layer and the pooling layer of the edge refinement module to refine the drivable dividing line and determine the drivable dividing line of the second granularity.

[0175] As another implementation method of the present application, in order to further improve the accuracy of identifying the target area where the unmanned driving equipment can travel, the above-mentioned device can also specifically include: a determination subunit 14053.

[0176] The determination subunit is used to select a number of target dividing points from the drivable dividing line of the second granularity, determine the target distance between each target dividing point and the unmanned driving device based on a monocular vision algorithm, and determine the target area in which the unmanned driving device can be driven according to the several target distances.

[0177] As another implementation of the present application, in order to further improve the accuracy of identifying the target area where the unmanned driving equipment can travel, the above-mentioned device may also specifically include: a determination subunit.

[0178] The determination subunit is used to obtain the acquisition posture of the driving scene image, and determine a first line segment, a second line segment and a third line segment from the driving scene image according to the acquisition posture, wherein the midpoint of the first line segment, the first endpoint of the second line segment and the first endpoint of the third line segment intersect at a line segment intersection point, and in an angle formed by the first line segment and the second line segment, with the line segment intersection point as the center and the first angle as the interval, an auxiliary line is made toward the drivable dividing line, in an angle formed by the second line segment and the third line segment, with the line segment intersection point as the center and the second angle as the interval, an auxiliary line is made toward the drivable dividing line, in an angle formed by the first line segment and the third line segment, with the line segment intersection point as the center and the third angle as the interval, and an auxiliary line is made toward the drivable dividing line, in an angle formed by the first line segment and the third line segment, with the line segment intersection point as the center and the third angle as the interval, and the intersection point of the auxiliary line with the drivable dividing line is determined as the target dividing point.

[0179] Fig.15 A schematic diagram of the hardware structure of a target area determination device provided in an embodiment of the present application is shown.

[0180] The target area determination device may include a processor 1501 and a memory 1502 storing computer program instructions.

[0181] Specifically, the processor 1501 may include a central processing unit (CPU), or an application specific integrated circuit (ASIC), or may be configured to implement one or more integrated circuits of the embodiments of the present application.

[0182] Memory 1502 may include a large capacity memory for data or instructions. By way of example and not limitation, memory 1502 may include a hard disk drive (HDD), a floppy disk drive, a flash memory, an optical disk, a magneto-optical disk, a magnetic tape, or a universal serial bus (USB) drive or a combination of two or more of these. In appropriate cases, memory 1502 may include removable or non-removable (or fixed) media. In appropriate cases, memory 1502 may be inside or outside the integrated gateway disaster recovery device. In a specific embodiment, memory 1502 is a non-volatile solid-state memory.

[0183] In certain embodiments, the memory 1502 may include a read-only memory (ROM), a random access memory (RAM), a magnetic disk storage media device, an optical storage media device, a flash memory device, an electrical, optical or other physical / tangible memory storage device. Thus, in general, the memory includes one or more tangible (non-transitory) computer-readable storage media (e.g., a memory device) encoded with software including computer-executable instructions, and when the software is executed (e.g., by one or more processors), it is operable to perform the operations described with reference to the method according to an aspect of the present disclosure.

[0184] The processor 1501 implements any one of the target area determination methods in the above embodiments by reading and executing computer program instructions stored in the memory 1502 .

[0185] In one example, the target area determination device may further include a communication interface 1503 and a bus 1510. Fig.15 As shown, the processor 1501, the memory 1502, and the communication interface 1503 are connected via a bus 1510 and communicate with each other.

[0186] The communication interface 1503 is mainly used to implement communication between various modules, devices, units and / or equipment in the embodiments of the present application.

[0187] Bus 1510 includes hardware, software or both, and the parts of target area determination equipment are coupled to each other.For example, but not limitation, bus may include accelerated graphics port (AGP) or other graphics bus, enhanced industrial standard architecture (EISA) bus, front side bus (FSB), hypertransport (HT) interconnection, industrial standard architecture (ISA) bus, infinite bandwidth interconnection, low pin count (LPC) bus, memory bus, micro channel architecture (MCA) bus, peripheral component interconnection (PCI) bus, PCI-Express (PCI-X) bus, serial advanced technology attachment (SATA) bus, video electronics standard association local (VLB) bus or other suitable bus or two or more of these combinations. In appropriate cases, bus 1510 may include one or more buses. Although the present application embodiment describes and shows a specific bus, the application considers any suitable bus or interconnection.

[0188] The target area determination device can execute the target area determination method in the embodiment of the present application, thereby realizing the combination Figure 1 and Fig.14 The target area determination method and device are described.

[0189] In addition, in combination with the target area determination method in the above embodiments, the present application embodiment can provide a computer storage medium for implementation. The computer storage medium stores computer program instructions; when the computer program instructions are executed by a processor, any one of the target area determination methods in the above embodiments is implemented.

[0190] It should be clear that the present application is not limited to the specific configuration and processing described above and shown in the figures. For the sake of simplicity, a detailed description of the known method is omitted here. In the above embodiments, several specific steps are described and shown as examples. However, the method process of the present application is not limited to the specific steps described and shown, and those skilled in the art can make various changes, modifications and additions, or change the order between the steps after understanding the spirit of the present application.

[0191] The functional blocks shown in the above-described block diagram can be implemented as hardware, software, firmware or a combination thereof. When implemented in hardware, it can be, for example, an electronic circuit, an application specific integrated circuit (ASIC), appropriate firmware, a plug-in, a function card, etc. When implemented in software, the elements of the present application are programs or code segments that are used to perform the required tasks. The program or code segment can be stored in a machine-readable medium, or transmitted on a transmission medium or a communication link by a data signal carried in a carrier wave. "Machine-readable medium" can include any medium capable of storing or transmitting information. Examples of machine-readable media include electronic circuits, semiconductor memory devices, ROM, flash memory, erasable ROM (EROM), floppy disks, CD-ROMs, optical disks, hard disks, optical fiber media, radio frequency (RF) links, etc. The code segment can be downloaded via a computer network such as the Internet, an intranet, etc.

[0192] It should also be noted that the exemplary embodiments mentioned in this application describe some methods or systems based on a series of steps or devices. However, this application is not limited to the order of the above steps, that is, the steps can be performed in the order mentioned in the embodiment, or in a different order from the embodiment, or several steps can be performed simultaneously.

[0193] Aspects of the present disclosure are described above with reference to the flowchart and / or block diagram of the method, device and computer program product according to the embodiment of the present disclosure. It should be understood that each box in the flowchart and / or block diagram and the combination of each box in the flowchart and / or block diagram can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing device to produce a machine so that these instructions executed by the processor of the computer or other programmable data processing device enable the implementation of the function / action specified in one or more boxes of the flowchart and / or block diagram. Such a processor can be, but is not limited to, a general-purpose processor, a special-purpose processor, a special application processor, or a field programmable logic circuit. It can also be understood that each box in the block diagram and / or flowchart and the combination of boxes in the block diagram and / or flowchart can also be implemented by dedicated hardware that performs a specified function or action, or can be implemented by a combination of dedicated hardware and computer instructions.

[0194] The above is only a specific implementation of the present application. Those skilled in the art can clearly understand that for the convenience and simplicity of description, the specific working processes of the systems, modules and units described above can refer to the corresponding processes in the aforementioned method embodiments, and will not be repeated here. It should be understood that the protection scope of the present application is not limited to this. Any technician familiar with the technical field can easily think of various equivalent modifications or replacements within the technical scope disclosed in this application, and these modifications or replacements should be included in the protection scope of this application.

Claims

1. A method for determining a target area, characterized in that: include: Acquire the driving scene image of the unmanned driving device; Extracting features from the driving scene image using a feature extraction model to obtain edge features and semantic features of objects in the driving scene image; By slicing and splicing, the edge feature and the semantic feature are merged to obtain a drivable boundary line with a first granularity in the driving scene image; Refining the drivable dividing line according to the distance between the actual area corresponding to the drivable dividing line and the unmanned driving device, and determining a drivable dividing line of a second granularity in the driving scene image, wherein the first granularity is greater than the second granularity; A target area in which the unmanned driving device can drive is determined according to a drivable boundary line of a second granularity in the driving scene image.

2. The method according to claim 1, characterized in that The feature extraction model includes N residual modules, and at least some of the residual modules do not include a pooling layer, and N is a positive integer greater than or equal to 2; The step of extracting features from the driving scene image using a feature extraction model to obtain edge features and semantic features of objects in the driving scene image includes: Inputting the driving scene image into a feature extraction model; Based on the first M residual modules of the feature extraction model, feature extraction is performed on the driving scene image to obtain edge features of objects in the driving scene image, where M is a positive integer less than N; Based on the N residual modules of the feature extraction model, feature extraction is performed on the driving scene image to obtain semantic features of objects in the driving scene image.

3. The method according to claim 1, characterized in that The step of fusing the edge feature with the semantic feature by slicing and splicing to obtain a drivable boundary line with a first granularity in the driving scene image includes: Determining the category to which the object in the driving scene image belongs according to a plurality of preset categories and the semantic features; Slicing the semantic features according to different categories to obtain a number of slice features; respectively splicing the slice features with the edge features to obtain several categories of splicing features; The category splicing features of each category are respectively fused by means of group convolution to obtain a number of sub-drivable dividing lines carrying category identifiers, wherein the sub-drivable dividing lines are the first granularity; According to the plurality of sub-drivable dividing lines, a drivable dividing line of a first granularity is determined in the driving scene image.

4. The method according to claim 3, characterized in that Before determining the target area in which the unmanned driving device can drive according to the drivable boundary line of the second granularity in the driving scene image, the method further includes: According to the drivable dividing line, determining the category identification of each particle point constituting the drivable dividing line, wherein the particle point is the smallest element constituting the drivable dividing line; Starting from the first particle point of the drivable dividing line, a sliding window filtering is performed on the category identification of each particle point in the drivable dividing line through a sliding window of preset width.

5. The method according to claim 3, characterized in that: The determining, according to the drivable boundary line of the second granularity in the driving scene image, a target area in which the unmanned driving device can drive, includes: According to the drivable boundary lines of the second granularity composed of sub-drivable boundary lines carrying category identifiers, a target area in which the unmanned driving device can be driven and a category to which objects at the boundary of the target area belong are determined.

6. The method according to claim 1, characterized in that The method of refining the drivable boundary line according to the distance between the actual area corresponding to the drivable boundary line and the unmanned driving device, and determining the drivable boundary line of the second granularity in the driving scene image, comprises: For each column of pixels in the driving scene image, the pixels in the column that coincide with the drivable boundary line are determined as boundary pixels; According to the driving scene image, the pixel point at the end of the end closer to the unmanned driving device at both ends of the column of pixel points is determined as an endpoint pixel point; Determine the boundary pixel point closest to the endpoint pixel point as the target pixel point; In the driving scene image, a line segment composed of the target pixel points is determined as a drivable boundary line of the second granularity that has been refined.

7. The method according to claim 1, characterized in that The method of refining the drivable boundary line according to the distance between the actual area corresponding to the drivable boundary line and the unmanned driving device, and determining the drivable boundary line of the second granularity in the driving scene image, comprises: The driving scene image marked with a drivable boundary line of the first granularity is input into the edge refinement module, and the convolution layer and the pooling layer of the edge refinement module are used for calculation to refine the drivable boundary line and determine the drivable boundary line of the second granularity.

8. The method according to claim 1, characterized in that The determining, according to the drivable boundary line of the second granularity in the driving scene image, a target area in which the unmanned driving device can drive, includes: Selecting a plurality of target demarcation points from the drivable demarcation lines of the second granularity; Based on a monocular vision algorithm, respectively determine the target distance between each target demarcation point and the unmanned driving device; A target area in which the unmanned driving device can travel is determined based on the plurality of target distances.

9. The method according to claim 8, characterized in that The selecting of a plurality of target demarcation points from the drivable demarcation lines of the second granularity includes: Acquiring the acquisition posture of the driving scene image; According to the collected posture, determining a first line segment, a second line segment, and a third line segment from the driving scene image, wherein a midpoint of the first line segment, a first endpoint of the second line segment, and a first endpoint of the third line segment intersect at a line segment intersection point; In the angle formed by the first line segment and the second line segment, with the intersection of the line segments as the center and at intervals of the first angle, auxiliary lines are drawn toward the drivable dividing line; In the angle formed by the second line segment and the third line segment, an auxiliary line is drawn toward the drivable dividing line at intervals of the second angle with the intersection of the line segments as the center; In the angle formed by the first line segment and the third line segment, an auxiliary line is drawn toward the drivable dividing line with the intersection of the line segments as the center and the third angle as the interval; The intersection point of the auxiliary line and the drivable dividing line is determined as the target dividing point.

10. A target area determination device, characterized in that: The device comprises: An acquisition unit, used to acquire an image of a driving scene in which the unmanned driving device is located; An extraction unit, used to extract features from the driving scene image through a feature extraction model to obtain edge features and semantic features of objects in the driving scene image; A fusion unit, configured to fuse the edge feature with the semantic feature by slicing and splicing to obtain a drivable boundary line with a first granularity in the driving scene image; a refinement unit, configured to refine the drivable boundary line according to a distance between an actual area corresponding to the drivable boundary line and the unmanned driving device, and determine a drivable boundary line of a second granularity in the driving scene image, wherein the first granularity is greater than the second granularity; A determination unit is used to determine a target area in which the unmanned driving device can drive according to a drivable boundary line of a second granularity in the driving scene image.

Images