Vehicle drivable area detection method and device, storage medium and electronic equipment
By building a semantic segmentation model that includes semantic segmentation networks and generative adversarial networks, the problem of high computing resources consumption in vehicle driving area detection is solved, efficient driving area detection is achieved, and the computing burden of vehicles is reduced.
Patent Information
- Application Number
- CN202510503213.4
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2025-04-21
- Publication Date
- 2025-07-18
AI Technical Summary
The existing vehicle driving area detection method requires a large amount of vehicle-side computing resources, resulting in high computing pressure and affecting vehicle operation.
The semantic segmentation model composed of a trained semantic segmentation network and a generative adversarial network is adopted to perform semantic segmentation and hole repair on the image to generate a travelable area mask, reducing post-processing needs.
Shorten the detection time, reduce the consumption of computing resources, reduce the computing burden on the vehicle side, and avoid adverse effects on the vehicle operation.
Smart Images
Figure CN120339987A_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the technical field of autonomous driving, and particularly to a method and device for detecting a drivable area of a vehicle, a storage medium, and an electronic device. Background Art
[0002] With the development of autonomous driving technology, the autonomous driving system has become an important part of vehicle design. In the autonomous driving scenario of a vehicle, it is necessary to collect image data for the surrounding environment, and use the collected images to detect the drivable area of the vehicle, so as to use the detected drivable area as the decision-making basis for autonomous driving control.
[0003] Currently, for the detection of the drivable area of a vehicle, usually a semantic segmentation network is first used to perform semantic segmentation on the collected images to obtain a mask marking the drivable area. Due to factors such as sensor noise in the real environment, semantic segmentation is often interfered, resulting in holes in the mask of the drivable area generated by the semantic segmentation network. Therefore, generally, image post-processing technologies such as image morphological operations are used to repair the holes in the mask of the drivable area. For example, a dilation operation is applied to fill the holes, and then the drivable area of the vehicle is determined based on the repaired mask.
[0004] In an actual application scenario, the detection task of the drivable area of the vehicle is completed by the computing resources on the vehicle side. Based on the existing detection method for the drivable area of the vehicle, it is necessary to apply an image post-processing algorithm on the vehicle side to perform image repair to achieve the detection of the drivable area. However, the processing process of the image post-processing algorithm is usually very cumbersome, with a long operation time and a large consumption of computing resources. And the computing resources on the vehicle side are usually very limited. The application of the image post-processing algorithm is likely to cause a computing burden on the vehicle side, affecting the response of the vehicle side to other tasks and having an adverse impact on vehicle operation. Summary of the Invention
[0005] In view of this, embodiments of the present invention provide a method for detecting a drivable area of a vehicle to solve the problem that the existing detection method for the drivable area of the vehicle requires a large amount of computing resources on the vehicle side, resulting in a large computing pressure on the vehicle side and being likely to have an adverse impact on vehicle operation.
[0006] Embodiments of the present invention also provide a device for detecting a drivable area of a vehicle to ensure the implementation and application of the above method in practice.
[0007] To achieve the above object, embodiments of the present invention provide the following technical solutions:
[0008] A method for detecting a drivable area of a vehicle, comprising:
[0009] When it is necessary to detect the drivable area of a vehicle in a target area, determine the area image corresponding to the target area;
[0010] Input the area image into a pre-constructed semantic segmentation model to obtain a drivable area mask output by the semantic segmentation model for the area image, and use the drivable area mask output by the semantic segmentation model as the target drivable area mask; the semantic segmentation model is composed of a trained semantic segmentation network and a trained generative adversarial network; the semantic segmentation network is used to perform semantic segmentation on the image input into the semantic segmentation model to generate a drivable area mask corresponding to the image, and the generative adversarial network is used to utilize the image input into the semantic segmentation model to perform hole repair on the drivable area mask generated by the semantic segmentation network, generate a repaired drivable area mask, and use the repaired drivable area mask as the output data of the semantic segmentation model;
[0011] Use the drivable area marked by the target drivable area mask as the drivable area of the vehicle.
[0012] For the above method, optionally, the construction process of the semantic segmentation model includes:
[0013] Construct an original semantic segmentation network and an original generative adversarial network;
[0014] Construct an original model; the original model is composed of the original semantic segmentation network and the original generative adversarial network, the input end of the original model is the input end of the original semantic segmentation network, the output end of the original semantic segmentation network is connected to the input end of the original generative adversarial network, and the output end of the original generative adversarial network is the output end of the original model;
[0015] Determine a first training dataset; the first training dataset includes multiple first training samples, and the sample feature of each first training sample is the area sample image corresponding to the sample, and the sample label is the true drivable area mask corresponding to the sample;
[0016] Based on the first training dataset, independently train the original semantic segmentation network, and use the independently trained original semantic segmentation network as the first network;
[0017] Determine a second training dataset; the second training dataset includes multiple second training samples, and the sample feature of each second training sample is the area sample image corresponding to the sample and the mask to be repaired corresponding to the sample, and the sample label is the true drivable area mask corresponding to the sample, and the mask to be repaired corresponding to the sample is a drivable area mask with a hole area;
[0018] Based on the second training dataset, independently train the original generative adversarial network, and use the independently trained original generative adversarial network as the second network;
[0019] Use the original model composed of the first network and the second network as the target model;
[0020] Based on the first training dataset, train the target model, and use the trained target model as the semantic segmentation model.
[0021] For the above method, optionally, the determination of the second training dataset includes:
[0022] For each of the first training samples, input the sample features of the first training sample into the first network, obtain the drivable area mask output by the first network for the first training sample, and use the drivable area mask as the semantic segmentation mask corresponding to the first training sample;
[0023] For the semantic segmentation mask corresponding to each of the first training samples, determine whether there are hole regions in the semantic segmentation mask. If there are hole regions in the semantic segmentation mask, use the first training sample as a sample to be repaired;
[0024] For each of the samples to be repaired, construct a hole sample corresponding to the sample to be repaired. The sample features of the hole sample are the regional sample image corresponding to the hole sample and the mask to be repaired corresponding to the hole sample, and the sample label is the true drivable area mask corresponding to the hole sample; the regional sample image corresponding to the hole sample is the regional sample image corresponding to the sample to be repaired, the mask to be repaired corresponding to the hole sample is the semantic segmentation mask corresponding to the sample to be repaired, and the true drivable area mask corresponding to the hole sample is the true drivable area mask corresponding to the sample to be repaired;
[0025] Use each of the hole samples as the second training sample, and form the second training dataset from each of the second training samples.
[0026] For the above method, optionally, the determination of whether there are hole regions in the semantic segmentation mask includes:
[0027] Perform connected region detection on the semantic segmentation mask to obtain each connected region corresponding to the semantic segmentation mask;
[0028] Determine the area of each connected region;
[0029] For each of the connected regions, compare the area of the connected region with a preset area threshold. If the area of the connected region is less than the area threshold, mark the connected region as a hole;
[0030] If there is at least one connected region marked as a hole among the connected regions, it is determined that there is a hole region in the semantic segmentation mask.
[0031] In the above method, optionally, the semantic segmentation network is a network constructed based on the U-net network.
[0032] In the above method, optionally, the loss function in the training process of the semantic segmentation network includes: a cross-entropy loss function and a Dice loss function;
[0033] The loss function of the generator network in the training process of the generative adversarial network includes: an adversarial loss function and a reconstruction loss function.
[0034] In the above method, optionally, the process of taking the drivable region marked by the target drivable region mask as the drivable region of the vehicle includes:
[0035] Determining a drivable region image in the region image according to the pixel mapping relationship between the region image and the target drivable region mask;
[0036] Taking the region represented by the drivable region image as the drivable region marked by the target drivable region mask to obtain the drivable region of the vehicle.
[0037] A detection device for the drivable region of a vehicle, comprising:
[0038] An image determination unit, configured to determine a region image corresponding to the target region when it is necessary to detect the drivable region of the vehicle in the target region;
[0039] A semantic segmentation unit, configured to input the region image into a pre-constructed semantic segmentation model, obtain a drivable region mask output by the semantic segmentation model for the region image, and use the drivable region mask output by the semantic segmentation model as a target drivable region mask; the semantic segmentation model is composed of a trained semantic segmentation network and a trained generative adversarial network; the semantic segmentation network is configured to perform semantic segmentation on the image input into the semantic segmentation model to generate a drivable region mask corresponding to the image, and the generative adversarial network is configured to use the image input into the semantic segmentation model to repair holes in the drivable region mask generated by the semantic segmentation network, generate a repaired drivable region mask, and use the repaired drivable region mask as the output data of the semantic segmentation model;
[0040] A region extraction unit, configured to take the drivable region marked by the target drivable region mask as the drivable region of the vehicle.
[0041] A storage medium, the storage medium including stored instructions, wherein when the instructions are running, the device where the storage medium is located is controlled to execute the detection method of the drivable area of the vehicle as described above.
[0042] An electronic device, including a memory, and one or more instructions, wherein the one or more instructions are stored in the memory and are configured to be executed by one or more processors to execute the detection method of the drivable area of the vehicle as described above.
[0043] Based on the detection method of the drivable area of the vehicle provided by the embodiments of the present invention, it includes: when it is necessary to detect the drivable area of the vehicle in the target area, determining the area image corresponding to the target area; inputting the area image into a pre-constructed semantic segmentation model to obtain the drivable area mask output by the semantic segmentation model for the area image, and using the drivable area mask output by the semantic segmentation model as the target drivable area mask; the semantic segmentation model is composed of a trained semantic segmentation network and a trained generative adversarial network; the semantic segmentation network is used to perform semantic segmentation on the image input into the semantic segmentation model to generate the drivable area mask corresponding to the image, and the generative adversarial network is used to use the image input into the semantic segmentation model to perform hole repair on the drivable area mask generated by the semantic segmentation network, generate the repaired drivable area mask, and use the repaired drivable area mask as the output data of the semantic segmentation model; using the drivable area marked by the target drivable area mask as the drivable area of the vehicle. Applying the method provided by the embodiments of the present invention, by processing the area image of the target area through the pre-constructed semantic segmentation model, using the drivable area mask output by the semantic segmentation model as the identification basis for the drivable area. The semantic segmentation model performs operations by integrating the generative adversarial network on the basis of the semantic segmentation network, and the operations of the generative adversarial network are relatively simple. The operation time of the semantic segmentation model and the occupation of computing resources are not much different from the operations of the semantic segmentation network. When applying the semantic segmentation model to generate the drivable area mask, the generative adversarial network can be used to perform hole repair on the drivable area mask generated by the semantic segmentation network, so that the drivable area mask output by the semantic segmentation model is a mask after hole repair, and there is no need to apply image post-processing technology to repair the mask during the detection process, which can effectively shorten the time-consuming of the drivable area detection process, reduce the consumption of computing resources during the detection process, is beneficial to reducing the computing burden on the vehicle side, and avoids having an adverse impact on vehicle operation. Description of the Drawings
[0044] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the following will briefly introduce the drawings required for the description of the embodiments or the prior art. Obviously, the drawings in the following description are only the embodiments of the present invention. For those of ordinary skill in the art, without creative efforts, other drawings can be obtained based on the provided drawings.
[0045] Figure 1 It is a flowchart of a method for detecting a drivable area of a vehicle provided by an embodiment of the present invention;
[0046] Figure 2 It is a flowchart of a construction process of a semantic segmentation model provided by an embodiment of the present invention;
[0047] Figure 3 It is a schematic diagram of a training process of a semantic segmentation model provided by an embodiment of the present invention;
[0048] Figure 4 It is a schematic structural diagram of a device for detecting a drivable area of a vehicle provided by an embodiment of the present invention;
[0049] Figure 5 It is a schematic structural diagram of an electronic device provided by an embodiment of the present invention. Detailed implementation manners
[0050] The following will clearly and completely describe the technical solutions in the embodiments of the present invention with reference to the drawings in the embodiments of the present invention. Obviously, the described embodiments are only a part of the embodiments of the present invention, rather than all the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those of ordinary skill in the art without creative efforts belong to the scope of protection of the present invention.
[0051] In this application, the term "including", "comprising" or any other variant thereof is intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements not only includes those elements, but also includes other elements not expressly listed, or also includes elements inherent to such process, method, article or device. Without further limitations, the element defined by the statement "including one..." does not exclude the existence of additional identical elements in the process, method, article or device including the said element.
[0052] An embodiment of the present invention provides a method for detecting a drivable area of a vehicle. The method can be applied to the detection of the drivable area of an autonomous driving system, and its execution subject can be the processor of the vehicle. The flowchart of the method is as Figure 1 shown and includes:
[0053] S101: When it is necessary to detect the drivable area of a vehicle in a target area, determine the area image corresponding to the target area;
[0054] In the method provided by the embodiments of the present invention, in the scenario of autonomous driving, when it is necessary to detect the current drivable area of a vehicle, the processor, in response to the detection requirement, may use the area to be detected as the target area and detect the drivable area of the vehicle in the target area. During the operation of the vehicle, the vehicle-mounted camera can be used to collect images of the areas around the vehicle to obtain real-time images of each area. When the processor needs to detect the drivable area of the vehicle in the target area, it can obtain the image matching the target area from the collected image data and use this image as the area image corresponding to the target area.
[0055] S102: Input the area image into a pre-constructed semantic segmentation model to obtain the drivable area mask output by the semantic segmentation model for the area image, and use the drivable area mask output by the semantic segmentation model as the target drivable area mask; the semantic segmentation model is composed of a trained semantic segmentation network and a trained generative adversarial network; the semantic segmentation network is used to perform semantic segmentation on the image input into the semantic segmentation model to generate the drivable area mask corresponding to the image, and the generative adversarial network is used to use the image input into the semantic segmentation model to perform hole repair on the drivable area mask generated by the semantic segmentation network, generate the repaired drivable area mask, and use the repaired drivable area mask as the output data of the semantic segmentation model;
[0056] In the method provided by the embodiments of the present invention, a semantic segmentation model can be pre-constructed. The semantic segmentation model is obtained by integrating a semantic segmentation network and a generative adversarial network. Through training, when the semantic segmentation model receives an input image, the input image can enter the semantic segmentation network and the generative adversarial network respectively. The semantic segmentation network can perform semantic segmentation on the input image and output a mask of the drivable area corresponding to the image. The mask of the drivable area output by the semantic segmentation network then enters the generative adversarial network, and the generative adversarial network performs hole repair on the mask of the drivable area output by the semantic segmentation network based on the input image, and outputs a mask of the drivable area after hole repair. The output of the generative adversarial network is the output of the semantic segmentation model, that is, the output of the semantic segmentation model is the mask of the drivable area generated by the generative adversarial network, which is the mask after hole repair based on the mask of the drivable area generated by the semantic segmentation network. The semantic segmentation network in the semantic segmentation model refers to a network used to implement semantic segmentation, which can be constructed based on existing semantic segmentation algorithms, such as semantic segmentation algorithms based on U-Net, semantic segmentation algorithms based on Fully Convolutional Networks (FCN), semantic segmentation algorithms based on SegNet, etc. U-Net is a U-shaped network based on Convolutional Neural Networks (CNN), and SegNet is a deep convolutional encoder-decoder architecture for image segmentation. The structure of the generative adversarial network (GAN) in the semantic segmentation model is an existing network structure, which includes a generator network (generative model) and a discriminator network (discriminative model), and will not be introduced in detail here. It can be understood that the mask of the drivable area refers to a mask that marks the drivable area (Mask), and the mask is also called a mask, which is usually a binary image or a boolean image with the same size as the original image.
[0057] In the method provided by the embodiments of the present invention, the processor can input the regional image of the target area into the semantic segmentation model. When the semantic segmentation model receives the regional image, the semantic segmentation network in the model will perform semantic segmentation on the regional image to generate a mask of the drivable area, and the generative adversarial network in the model will use the regional image to perform hole repair on the mask generated by the semantic segmentation network to generate a repaired mask. The semantic segmentation model outputs the mask generated by the generative adversarial network as the mask of the drivable area corresponding to the regional image. The processor can obtain the mask of the drivable area output for the regional image from the output end of the semantic segmentation model and use the mask of the drivable area as the target mask of the drivable area.
[0058] S103: Use the drivable area marked by the target drivable area mask as the drivable area of the vehicle.
[0059] In the method provided by the embodiment of the present invention, the drivable area marked in the target drivable area mask is used as the drivable area of the current vehicle in the target area.
[0060] Based on the method provided by the embodiment of the present invention, when it is necessary to detect the drivable area of a vehicle in a target area, determine the area image corresponding to the target area; input the area image into a pre-constructed semantic segmentation model to obtain the drivable area mask output by the semantic segmentation model for the area image, and use the drivable area mask output by the semantic segmentation model as the target drivable area mask; the semantic segmentation model is composed of a trained semantic segmentation network and a trained generative adversarial network; the semantic segmentation network is used to perform semantic segmentation on the image input into the semantic segmentation model to generate the drivable area mask corresponding to the image, and the generative adversarial network is used to utilize the image input into the semantic segmentation model to perform hole repair on the drivable area mask generated by the semantic segmentation network, generate the repaired drivable area mask, and use the repaired drivable area mask as the output data of the semantic segmentation model; use the drivable area marked by the target drivable area mask as the drivable area of the vehicle. Applying the method provided by the embodiment of the present invention, the area image of the target area is processed by a pre-constructed semantic segmentation model, and the drivable area mask output by the semantic segmentation model is used as the identification basis for the drivable area. The semantic segmentation model performs operations by integrating a generative adversarial network on the basis of the semantic segmentation network, and the operations of the generative adversarial network are relatively simple. The operation time of the semantic segmentation model and the occupation of computing resources are not much different from the operations of the semantic segmentation network. When applying the semantic segmentation model to generate the drivable area mask, the generative adversarial network can be used to perform hole repair on the drivable area mask generated by the semantic segmentation network, so that the drivable area mask output by the semantic segmentation model is a mask after hole repair, and there is no need to apply image post-processing technology to repair the mask during the detection process, which can effectively shorten the time-consuming of the drivable area detection process, reduce the consumption of computing resources during the detection process, is beneficial to reducing the computing burden on the vehicle side, and avoids having an adverse impact on vehicle operation.
[0061] In Figure 1 On the basis of the method shown, as Figure 2 shown, in the method provided by the embodiment of the present invention, the construction process of the semantic segmentation model mentioned in step S102 includes:
[0062] S201: Construct an original semantic segmentation network and an original generative adversarial network;
[0063] In the method provided by the embodiments of the present invention, during the construction of the semantic segmentation model, first, a network for implementing semantic segmentation is constructed based on the semantic segmentation algorithm, and this network is used as the original semantic segmentation network. A generative adversarial network is constructed based on the network structure of the generative adversarial network, and this network is used as the original generative adversarial network.
[0064] S202: Construct an original model; the original model is composed of the original semantic segmentation network and the original generative adversarial network. The input end of the original model is the input end of the original semantic segmentation network. The output end of the original semantic segmentation network is connected to the input end of the original generative adversarial network, and the output end of the original generative adversarial network is the output end of the original model;
[0065] In the method provided by the embodiments of the present invention, the input end of the original semantic segmentation network is used as the input end of the model. The output end of the original semantic segmentation network is connected to the input end of the original generative adversarial network, and the output end of the original generative adversarial network is used as the output end of the model. Thus, the original semantic segmentation network and the original generative adversarial network are integrated to obtain the original model.
[0066] S203: Determine a first training dataset; the first training dataset includes a plurality of first training samples. The sample feature of each first training sample is the regional sample image corresponding to the sample, and the sample label is the ground truth mask of the drivable area corresponding to the sample;
[0067] In the method provided by the embodiments of the present invention, images of different regions can be collected in advance, and these images are used as regional sample images. The accurate drivable area in each regional sample image is determined, and a corresponding mask is generated based on the accurate drivable area in each regional sample image to mark the accurate drivable area in the image. This mask is used as the ground truth mask of the drivable area of the image. For each regional sample image, this image is used as the sample feature, and the ground truth mask of the drivable area of this image is used as the sample label. Thus, the training sample corresponding to this image is constructed, and the training samples corresponding to each regional sample image are used as the first training samples. The first training dataset is composed of each training sample.
[0068] S204: Independently train the original semantic segmentation network based on the first training dataset, and use the independently trained original semantic segmentation network as the first network;
[0069] In the method provided by the embodiments of the present invention, during the overall training process, for the training of the original semantic segmentation network, it is first regarded as an independent training object, and the original semantic segmentation network is independently trained through the first training dataset, that is, the first training dataset only acts on the original semantic segmentation network and only adjusts the parameters of the original semantic segmentation network. After the training process is completed, the original semantic segmentation network that has been independently trained is called the first network.
[0070] S205: Determine the second training dataset; the second training dataset includes multiple second training samples, and the sample feature of each second training sample is the regional sample image corresponding to the sample and the to-be-repaired mask corresponding to the sample, and the sample label is the ground truth mask of the drivable area corresponding to the sample, and the to-be-repaired mask corresponding to the sample is a drivable area mask with a hole area.
[0071] In the method provided by the embodiments of the present invention, similar to the construction of the first training sample, multiple second training samples are constructed based on images of different regions. Specifically, in addition to determining the ground truth mask of the drivable area corresponding to the regional sample image, it is also necessary to determine the to-be-repaired mask corresponding to the image. The to-be-repaired mask is also a mask used to mark the drivable area, but there are hole areas in the mask and it does not accurately mark the image. The regional sample image and its corresponding to-be-repaired mask are used as sample features, and the ground truth mask of the drivable area corresponding to the image is used as the sample label, thereby constructing the training sample corresponding to the image. These training samples are the second training samples, and the second training dataset is composed of each second training sample.
[0072] S206: Independently train the original generative adversarial network based on the second training dataset, and regard the original generative adversarial network that has been independently trained as the second network.
[0073] In the method provided by the embodiments of the present invention, during the overall training process, for the training of the original generative adversarial network, it is first regarded as an independent training object, and the original generative adversarial network is independently trained through the second training dataset, that is, the second training dataset only acts on the original generative adversarial network and only adjusts the parameters of the original generative adversarial network. After the training process is completed, the original generative adversarial network that has been independently trained is called the second network.
[0074] S207: Regard the original model composed of the first network and the second network as the target model.
[0075] In the method provided by the embodiment of the present invention, after the original semantic segmentation network and the original generative adversarial network in the original model are independently trained, the two networks in the original model are respectively evolved into the first network and the second network. The original model composed of the first network and the second network is used as the target model, that is, the original model after the original semantic segmentation network and the original generative adversarial network are both independently trained.
[0076] S208: Based on the first training dataset, train the target model, and use the trained target model as the semantic segmentation model.
[0077] In the method provided by the embodiment of the present invention, training the target model with the first training dataset specifically means jointly training the first network and the second network in the target model. That is, when applying a certain training sample for training, after inputting the sample features into the target model, the first network can perform semantic segmentation based on the sample features to generate a corresponding drivable area mask, and the second network repairs the holes in the drivable area mask generated by the first network based on the sample features to generate a repaired drivable area mask. During this process, the first network is adjusted by combining the output of the first network and the sample labels, and the second network is adjusted by combining the output of the second network and the sample labels to achieve the training of the target model. The trained target model is used as the semantic segmentation model. It can be understood that the first network that completes the overall training is the semantic segmentation network in the semantic segmentation model, and the second network that completes the overall training is the generative adversarial network in the semantic segmentation model.
[0078] Based on the method provided by the embodiment of the present invention, during the construction of the semantic segmentation model, the original semantic segmentation network and the original generative adversarial network are first independently trained, so that the two networks can first focus on the learning of their own tasks. For the networks that have been trained separately, joint training is then carried out, enabling the two networks to cooperate better, which is beneficial to improving the accuracy of the semantic segmentation model, and then improving the accuracy of the finally generated drivable area mask.
[0079] In Figure 2 Based on the method shown, in the method provided by the embodiment of the present invention, the process of determining the second training dataset mentioned in step S205 includes:
[0080] For each of the first training samples, input the sample features of the first training sample into the first network to obtain the drivable area mask output by the first network for the first training sample, and use the drivable area mask as the semantic segmentation mask corresponding to the first training sample;
[0081] In the method provided by the embodiments of the present invention, each second training sample is constructed by using a separately trained semantic segmentation network (i.e., the first network) and a first training dataset. Specifically, the first network is used to perform semantic segmentation on each first training sample, and the drivable area mask generated by the first network for each first training sample is used as the semantic segmentation mask corresponding to the sample.
[0082] For the semantic segmentation mask corresponding to each of the first training samples, it is determined whether there is a hole area in the semantic segmentation mask. If there is a hole area in the semantic segmentation mask, the first training sample is used as a sample to be repaired.
[0083] In the method provided by the embodiments of the present invention, hole mask detection is respectively performed on each semantic segmentation mask to identify whether there is a hole area in each semantic segmentation mask, and the first training sample corresponding to the semantic segmentation mask with a hole area is used as a sample to be repaired.
[0084] For each of the samples to be repaired, a hole sample corresponding to the sample to be repaired is constructed. The sample features of the hole sample are the regional sample image corresponding to the hole sample and the mask to be repaired corresponding to the hole sample, and the sample label is the true drivable area mask corresponding to the hole sample; the regional sample image corresponding to the hole sample is the regional sample image corresponding to the sample to be repaired, the mask to be repaired corresponding to the hole sample is the semantic segmentation mask corresponding to the sample to be repaired, and the true drivable area mask corresponding to the hole sample is the true drivable area mask corresponding to the sample to be repaired.
[0085] Each of the hole samples is used as the second training sample, and the second training dataset is composed of the respective second training samples.
[0086] In the method provided by the embodiments of the present invention, for each sample to be repaired, its corresponding regional sample image and semantic segmentation mask are used as a set of sample features to construct a corresponding hole sample, and the true drivable area mask of the sample to be repaired is used to label the hole sample as the sample label of the hole sample. Each hole sample constructed based on each sample to be repaired is used as each second training sample to form a second training dataset.
[0087] Based on the method provided by the embodiments of the present invention, masks with void regions can be screened from the drivable area masks generated by an independently trained semantic segmentation network, so as to construct a second training sample. The second training dataset can be processed based on the first training dataset without additional data collection, which is beneficial to improving the efficiency of sample construction. Secondly, the masks to be repaired in the second training sample are the masks generated by the semantic segmentation network, which enables the generative adversarial network to perform void repair training on the output features of the semantic segmentation network, and can better cooperate with the semantic segmentation network to improve the accuracy of the finally generated drivable area masks.
[0088] Based on the method provided by the above embodiments, in the method provided by the embodiments of the present invention, the process of determining whether there are void regions in the semantic segmentation mask includes:
[0089] Perform connected region detection on the semantic segmentation mask to obtain each connected region corresponding to the semantic segmentation mask;
[0090] In the method provided by the embodiments of the present invention, in the process of determining whether there are void regions in the semantic segmentation mask, first perform connected domain analysis on the semantic segmentation mask to detect each connected region in the semantic segmentation mask. The detection process of the connected region can be implemented based on existing image connected domain analysis methods. For example, various connected domain detection functions can be used for connected domain analysis, such as the bwlabel function, the connectedComponents function, etc. Connected domain analysis can also be performed using marker-based algorithms, such as the Two-Pass method, the seed filling method, etc. Connected domain analysis can also be implemented using segmentation-based algorithms, such as graph theory methods, region growing methods, etc.
[0091] It should be noted that in actual application scenarios, the specific implementation method of connected region detection can be selected according to actual needs, which does not affect the function implementation of the method provided by the embodiments of the present invention.
[0092] Determine the area of each connected region;
[0093] In the method provided by the embodiments of the present invention, the area of each connected region can be calculated based on the number of pixels included in each connected region.
[0094] For each connected region, compare the area of the connected region with a preset area threshold. If the area of the connected region is less than the area threshold, mark the connected region as a void;
[0095] In the method provided by the embodiments of the present invention, an area threshold can be set according to actual needs. This area threshold can be set based on the area size of the hole mask. Generally, the areas in the mask with an area smaller than this threshold are holes.
[0096] In the process of determining whether there are hole regions in the current semantic segmentation mask, the area of each connected region of the current semantic segmentation mask is respectively compared with a preset area threshold, and the connected regions with an area smaller than the preset area threshold are marked as holes.
[0097] If there is at least one connected region marked as a hole among all the connected regions, it is determined that there is a hole region in this semantic segmentation mask.
[0098] In the method provided by the embodiments of the present invention, if at least one connected region is marked as a hole, then it is determined that there is a hole region in the current semantic segmentation mask. If each connected region is not marked as a hole, then it is determined that there is no hole region in the current semantic segmentation mask.
[0099] Based on the method provided by the embodiments of the present invention, by detecting connected regions and comparing the sizes of the areas of connected regions, it is possible to identify whether there are hole regions in the semantic segmentation mask. The processing process is relatively simple, which is beneficial to improving the processing efficiency.
[0100] In Figure 1 Based on the method shown, in the method provided by the embodiments of the present invention, the semantic segmentation network is a network constructed based on the U-net network.
[0101] In the method provided by the embodiments of the present invention, a semantic segmentation network is constructed based on the network structure of the U-net network. The U-net network is a network model based on a fully convolutional neural network, which is an existing semantic segmentation algorithm. The U-net network consists of a symmetric encoder (downsampling path) and decoder (upsampling path), and there are also skip connections in the middle. The encoder consists of convolutional layers and pooling layers, which are used to extract image features and reduce the resolution. The decoder consists of convolutional layers and upsampling layers, which are used to restore the low-resolution feature maps to the original resolution. The skip connections connect the feature maps of the corresponding layers in the encoder and decoder, which helps to retain detailed information.
[0102] Based on the method provided by the embodiments of the present invention, a semantic segmentation network is constructed based on the U-net network. The structure of the U-net network is simple, and its requirements for training data are less. It supports training models with a small amount of data, which is beneficial to reducing the cost and difficulty of data annotation. A semantic segmentation network with good performance can be obtained in scenarios with limited data. Secondly, the U-net network realizes semantic segmentation by classifying each pixel point, and can obtain better segmentation accuracy, improving the accuracy of semantic segmentation.
[0103] Based on Figure 1 the method shown, in the method provided by the embodiments of the present invention, the loss function in the training process of the semantic segmentation network includes: a cross-entropy loss function and a Dice loss function.
[0104] In the method provided by the embodiments of the present invention, the loss function applied in the training process of the semantic segmentation network adopts a cross-entropy loss function and a Dice loss function. The cross-entropy loss function is used to measure the difference between the probability distribution output by the semantic segmentation network and the true label. The cross-entropy loss function can be shown as follows:
[0105] (Equation 1)
[0106] where N is the number of samples, C is the number of categories of semantic segmentation (the categories in this solution are the drivable area and the non-drivable area), y i,c is the probability of the true label, is the probability predicted by the model.
[0107] The Dice loss function is an index for evaluating the degree of overlap and is used to measure the similarity between the area predicted by the model and the true label. The Dice loss function can be shown as follows:
[0108] (Equation 2)
[0109] where N is the number of samples, y i is the binary pixel value of the true label, is the probability value output by the model.
[0110] Based on the method provided by the embodiments of the present invention, a cross-entropy loss function and a Dice loss function are adopted as the loss functions in the training process of the semantic segmentation network. The calculation method of the cross-entropy loss function is simple and has good mathematical properties, which is beneficial to improving the training speed and stability of training. The Dice loss function focuses on the overlapping part between the prediction result and the true label and has good performance in the scenario of unbalanced sample categories, which is beneficial to improving the training effect of the semantic segmentation network and further improving the performance of the semantic segmentation model.
[0111] Based on Figure 1 the method shown, in the method provided by the embodiments of the present invention, the loss function of the generator network in the training process of the generative adversarial network includes: an adversarial loss function and a reconstruction loss function.
[0112] In the method provided by the embodiments of the present invention, during the training process of the generative adversarial network, the loss functions applied to the generator network are the adversarial loss function and the reconstruction loss function. The evaluation of the adversarial loss is achieved by deceiving a simplified discriminator, and the adversarial loss aims to make the generated drivable area mask indistinguishable from the real drivable area mask in appearance.
[0113] In the embodiments of the present invention, the adversarial loss function is constructed using binary cross-entropy loss, and the adversarial loss function can be as follows:
[0114] (Equation 3)
[0115] The reconstruction loss is used to ensure that the generated drivable area mask is similar to the real drivable area mask at the pixel level. In the embodiments of the present invention, the reconstruction loss function is constructed using mean squared error loss, and the reconstruction loss function can be as follows:
[0116] (Equation 4)
[0117] In Equations 3 and 4, N represents the number of samples, x i represents the input sample, m i represents the conditional information, G(x i , m i ) represents the sample generated by the generator network, and D(...) represents the output of the discriminator network.
[0118] In the method provided by the embodiments of the present invention, the overall loss of the generator network is composed of a linear combination of the adversarial loss and the reconstruction loss, and the overall loss can be as follows:
[0119] (Equation 5)
[0120] Among them, λ adv represents the weight of the adversarial loss, and λ rec represents the weight of the reconstruction loss.
[0121] Based on the method provided by the embodiments of the present invention, training the generator network in the generative adversarial network by combining the adversarial loss and the reconstruction loss helps to prompt the generated drivable area mask by the generator network to be able to deceive the discriminator network and be similar to the real drivable area mask, which is beneficial to improving the training effect of the generative adversarial network and further improving the performance of the semantic segmentation model.
[0122] In Figure 1Based on the method shown above, in the method provided by the embodiments of the present invention, the process of using the drivable area marked by the target drivable area mask as the drivable area of the vehicle in step S103 includes:
[0123] Determine a drivable area image in the area image according to the pixel mapping relationship between the area image and the target drivable area mask;
[0124] Use the area represented by the drivable area image as the drivable area marked by the target drivable area mask to obtain the drivable area of the vehicle.
[0125] In the method provided by the embodiments of the present invention, the target drivable area mask is a binary image or a Boolean image with the same size as the original area image. Each pixel point in the target drivable area mask corresponds one by one to each pixel point in the original area image. Each pixel point in the target drivable area mask represents a drivable area or a non-drivable area. According to the pixel mapping relationship between the target drivable area mask and the area image, the pixel points in the area image mapped by the pixel points representing the drivable area in the target drivable area mask can be identified, and the image formed by these area image pixel points is regarded as the drivable area image. Using the area represented by the drivable area image as the drivable area marked by the target drivable area mask, that is, the drivable area of the vehicle. For example, for the mask of the drivable area, the pixel value of the drivable area is configured to be 255, and the pixel value of the non-drivable area is 0. The target drivable area mask is a binary image with pixel values of 0 or 255. When the pixel value of a pixel point is 255, it means that the pixel point represents a drivable area. When the pixel value of a pixel point is 0, it means that the pixel point represents a non-drivable area. According to the pixel mapping relationship between the target drivable area mask and the area image, among the pixel points of the area image, find the pixel points mapped by the drivable area mask pixel points with a pixel value of 255, and use the image area formed by these pixel points as the drivable area image.
[0126] To better illustrate the method provided by the embodiments of the present invention, based on the methods provided in the previous embodiments, combined with the actual application scenario, the embodiments of the present invention provide another method for detecting the drivable area of a vehicle. The method provided by the embodiments of the present invention integrates a generative adversarial network on a semantic segmentation network to construct a semantic segmentation model. The generator network in the generative adversarial network is responsible for using the original image to perform hole repair on the mask generated by the semantic segmentation network, and generating a mask after hole repair. When detecting the drivable area of the vehicle, use this semantic segmentation model to perform semantic segmentation and hole repair on the area image of the area to be detected, obtain a drivable area mask after hole repair, and thus realize the detection of the drivable area of the vehicle.
[0127] In the method provided by the embodiments of the present invention, a semantic segmentation network is constructed based on the U-net network, which consists of an encoder and a decoder. For the descriptions of the decoder and the encoder, reference can be made to the previous embodiments. During the application of the generative adversarial network, the structure for processing the input data and outputting the result is the generator network. Next, in combination with Figure 3 , the training process of the semantic segmentation model will be described. As Figure 3 shown, the semantic segmentation network includes an encoder and a decoder, and the generator refers to the generator network in the generative adversarial network.
[0128] The training process of the semantic segmentation model mainly includes:
[0129] Preparing training data;
[0130] In the method provided by the embodiments of the present invention, first, a plurality of training samples are prepared as training data. The sample feature of each training sample is the sample image, and the sample label is the true drivable area mask of the sample image. 2 / 3 of the training data is used for model training, and 1 / 3 of the training data is used for model testing.
[0131] Training the semantic segmentation network;
[0132] In the method provided by the embodiments of the present invention, first, the semantic segmentation network is trained with the training data. During the training process, the cross-entropy loss function and the Dice loss function are applied to evaluate the segmentation loss of the semantic segmentation network under normal circumstances. For the descriptions of the cross-entropy loss function and the Dice loss function, reference can be made to the previous embodiments.
[0133] Screening the masks with holes;
[0134] In the method provided by the embodiments of the present invention, the trained semantic segmentation network is used to perform semantic segmentation on the sample images in the training data to generate the corresponding masks for marking the drivable areas. From the masks generated by the semantic segmentation network, the masks with holes are screened out. Specifically, for each mask generated by the semantic segmentation network, connected component analysis can be performed to find all the connected components of each mask. For each connected component, its area (the number of pixels) is calculated. If the area of a certain connected component is less than a pre-set threshold, then this connected component is regarded as a hole. The masks with holes are retained, and the other masks can be discarded or further processed to construct masks with holes.
[0135] Training the generative adversarial network;
[0136] In the method provided by the embodiments of the present invention, based on the selected masks with holes, as well as the sample images and the real drivable area masks corresponding to these masks, the generative adversarial network is trained so that the generative adversarial network can master the ability to repair holes. In this training stage, the parameters of the semantic segmentation network are frozen by manually setting parameters, so that the generative adversarial network focuses on learning the task of hole repair. The generator network in the generative adversarial network consists of an encoder and a decoder. The encoder is responsible for encoding the input image and the mask with holes into a representation in the latent space. The decoder decodes the latent representation into a mask with holes repaired. During the training process, the generator network and the discriminator network are alternately optimized. The reconstruction loss function and the adversarial loss function are used as the loss functions of the generator network. The generator network generates a mask with holes repaired by minimizing the reconstruction loss and the adversarial loss. The discriminator network improves its discrimination ability by maximizing the loss to discriminate the real mask as real and the generated mask as fake. For the description of the reconstruction loss function and the adversarial loss function, refer to the previous embodiments.
[0137] Perform overall training on the semantic segmentation model.
[0138] In the method provided by the embodiments of the present invention, after separately training the semantic segmentation network and the generative adversarial network, all the parameters of the semantic segmentation network and the generator network in the generative adversarial network are unfrozen, and overall training is performed on the semantic segmentation model. As Figure 3 shown, during the overall training process, the image data in the sample respectively enter the encoder in the semantic segmentation network and the generator network (i.e., the generator) in the generative adversarial network. After the encoder and decoder in the semantic segmentation network sequentially process the data, a mask marking the drivable area is generated. The generated mask enters the generator network, and the generator network repairs the holes in the received mask based on the image data and outputs the repaired mask. Based on the ground truth mask (i.e., the real mask of the drivable area of the currently input image data) and the mask output by the semantic segmentation network, the semantic segmentation network is adjusted. Based on the ground truth mask and the mask generated by the generator network, the generative adversarial network is adjusted to achieve overall training.
[0139] Based on the method provided by the embodiments of the present invention, a generative adversarial network is integrated on the basis of the semantic segmentation network. Through model training, the generator network in the generative adversarial network masters the ability to repair holes in the drivable area mask. Therefore, the semantic segmentation model can perform secondary repair on the mask generated by the semantic segmentation network. During the detection process of the vehicle drivable area, through model inference, a drivable area mask with holes repaired can be obtained, and there is no need to apply an image post-processing algorithm for processing, which can shorten the processing time of the drivable area detection process and reduce the occupancy of vehicle-side computing resources.
[0140] WithFigure 1 Corresponding to the detection method of a vehicle drivable area shown above, an embodiment of the present invention further provides a detection device for a vehicle drivable area, which is used to Figure 1 implement the specific method shown above, and its structural schematic diagram is as shown in Figure 4 follows:
[0141] An image determination unit 301, configured to determine a regional image corresponding to the target area when it is necessary to detect the drivable area of a vehicle in the target area;
[0142] A semantic segmentation unit 302, configured to input the regional image into a pre-constructed semantic segmentation model, obtain a drivable area mask output by the semantic segmentation model for the regional image, and use the drivable area mask output by the semantic segmentation model as a target drivable area mask; the semantic segmentation model is composed of a trained semantic segmentation network and a trained generative adversarial network; the semantic segmentation network is configured to perform semantic segmentation on the image input into the semantic segmentation model to generate a drivable area mask corresponding to the image, and the generative adversarial network is configured to use the image input into the semantic segmentation model to perform hole repair on the drivable area mask generated by the semantic segmentation network, generate a repaired drivable area mask, and use the repaired drivable area mask as the output data of the semantic segmentation model;
[0143] A region extraction unit 303, configured to use the drivable area marked by the target drivable area mask as the drivable area of the vehicle.
[0144] By applying the device provided in the embodiment of the present invention, the regional image of the target area is processed through a pre-constructed semantic segmentation model, and the drivable area mask output by the semantic segmentation model is used as the identification basis for the drivable area. The semantic segmentation model performs operations by integrating a generative adversarial network on the basis of the semantic segmentation network, and the operations of the generative adversarial network are relatively simple. The operation time of the semantic segmentation model and the occupancy of computing resources are not much different from the operations of the semantic segmentation network. When applying the semantic segmentation model to generate a drivable area mask, the generative adversarial network can be used to perform hole repair on the drivable area mask generated by the semantic segmentation network, so that the drivable area mask output by the semantic segmentation model is a mask after hole repair. During the detection process, there is no need to apply image post-processing technology to repair the mask, which can effectively shorten the time-consuming of the drivable area detection process, reduce the consumption of computing resources during the detection process, is beneficial to reducing the computing burden on the vehicle side, and avoids causing adverse effects on vehicle operation.
[0145] In Figure 4Based on the device shown, the device provided by the embodiments of the present invention can further expand multiple units. For the functions of each unit, reference can be made to the descriptions in the respective embodiments provided for the method for detecting the drivable area of a vehicle above, and no further examples will be given here.
[0146] The embodiments of the present invention also provide a storage medium, which includes stored instructions. Among them, when the instructions run, they control the device where the storage medium is located to execute the method for detecting the drivable area of a vehicle as described above.
[0147] The embodiments of the present invention also provide an electronic device, and its structural schematic diagram is as Figure 5 shown, specifically including a memory 401 and one or more instructions 402. Among them, one or more instructions 402 are stored in the memory 401 and are configured to be executed by one or more processors 403 to perform the following operations on the one or more instructions 402:
[0148] When it is necessary to detect the drivable area of a vehicle in a target area, determine the area image corresponding to the target area;
[0149] Input the area image into a pre-constructed semantic segmentation model to obtain a drivable area mask output by the semantic segmentation model for the area image, and use the drivable area mask output by the semantic segmentation model as the target drivable area mask; the semantic segmentation model is composed of a trained semantic segmentation network and a trained generative adversarial network; the semantic segmentation network is used to perform semantic segmentation on the image input into the semantic segmentation model to generate a drivable area mask corresponding to the image, and the generative adversarial network is used to use the image input into the semantic segmentation model to perform hole repair on the drivable area mask generated by the semantic segmentation network to generate a repaired drivable area mask, and use the repaired drivable area mask as the output data of the semantic segmentation model;
[0150] Use the drivable area marked by the target drivable area mask as the drivable area of the vehicle.
[0151] Each embodiment in this specification is described in a progressive manner. For the same or similar parts among the embodiments, reference can be made to each other, and the differences between each embodiment and other embodiments are emphasized. In particular, for a system or system embodiment, since it is basically similar to the method embodiment, the description is relatively simple, and the relevant parts can refer to the description of the method embodiment. The systems and system embodiments described above are only illustrative. The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed to multiple network units. Some or all of the modules can be selected according to actual needs to achieve the purpose of the solution of this embodiment. A person of ordinary skill in the art can understand and implement it without creative work.
[0152] Those skilled in the art can further realize that the units and algorithm steps of each example described in combination with the embodiments disclosed in this article can be implemented by electronic hardware, computer software, or a combination of the two. To clearly illustrate the interchangeability of hardware and software, the components and steps of each example have been generally described according to functions in the above description. Whether these functions are executed in a hardware or software manner depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered to exceed the scope of the present invention.
[0153] The above description of the disclosed embodiments enables those skilled in the art to implement or use the present invention. Various modifications to these embodiments will be obvious to those skilled in the art, and the general principles defined herein can be implemented in other embodiments without departing from the spirit or scope of the present invention. Therefore, the present invention will not be limited to the embodiments shown herein, but will conform to the widest scope consistent with the principles and novel features disclosed herein.
Claims
1. A method for detecting a drivable area of a vehicle, characterized in that, Including: When it is necessary to detect the drivable area of a vehicle in a target area, determining a regional image corresponding to the target area; Inputting the regional image into a pre-constructed semantic segmentation model to obtain a drivable area mask output by the semantic segmentation model for the regional image, and using the drivable area mask output by the semantic segmentation model as a target drivable area mask; The semantic segmentation model is composed of a trained semantic segmentation network and a trained generative adversarial network; the semantic segmentation network is used to perform semantic segmentation on the image input into the semantic segmentation model to generate a drivable area mask corresponding to the image, and the generative adversarial network is used to utilize the image input into the semantic segmentation model to perform hole repair on the drivable area mask generated by the semantic segmentation network to generate a repaired drivable area mask, and using the repaired drivable area mask as the output data of the semantic segmentation model; Regarding the drivable area marked by the target drivable area mask as the drivable area of the vehicle.
2. The detection method of the vehicle drivable area according to claim 1, wherein The construction process of the semantic segmentation model includes: Constructing an original semantic segmentation network and an original generative adversarial network; Constructing an original model; the original model is composed of the original semantic segmentation network and the original generative adversarial network, the input end of the original model is the input end of the original semantic segmentation network, the output end of the original semantic segmentation network is connected to the input end of the original generative adversarial network, and the output end of the original generative adversarial network is the output end of the original model; Determining a first training data set; the first training data set includes a plurality of first training samples, and the sample feature of each first training sample is the regional sample image corresponding to the sample, and the sample label is the true drivable area mask corresponding to the sample; Based on the first training data set, independently training the original semantic segmentation network, and using the independently trained original semantic segmentation network as a first network; Determining a second training data set; the second training data set includes a plurality of second training samples, and the sample feature of each second training sample is the regional sample image corresponding to the sample and the mask to be repaired corresponding to the sample, and the sample label is the true drivable area mask corresponding to the sample, and the mask to be repaired corresponding to the sample is a drivable area mask with a hole area; Based on the second training data set, independently training the original generative adversarial network, and using the independently trained original generative adversarial network as a second network; Regarding the original model composed of the first network and the second network as a target model; Based on the first training data set, training the target model, and using the trained target model as the semantic segmentation model.
3. The detection method of the vehicle drivable area according to claim 2, characterized in that, The determination of the second training data set includes: For each of the first training samples, inputting the sample feature of the first training sample into the first network to obtain a drivable area mask output by the first network for the first training sample, and using the drivable area mask as the semantic segmentation mask corresponding to the first training sample; For each semantic segmentation mask corresponding to the first training sample, determine whether there is a hole region in the semantic segmentation mask. If there is a hole region in the semantic segmentation mask, then use the first training sample as a sample to be repaired; For each sample to be repaired, construct a hole sample corresponding to the sample to be repaired. The sample features of the hole sample are the regional sample image corresponding to the hole sample and the mask to be repaired corresponding to the hole sample, and the sample label is the ground truth mask of the drivable region corresponding to the hole sample; the regional sample image corresponding to the hole sample is the regional sample image corresponding to the sample to be repaired, the mask to be repaired corresponding to the hole sample is the semantic segmentation mask corresponding to the sample to be repaired, and the ground truth mask of the drivable region corresponding to the hole sample is the ground truth mask of the drivable region corresponding to the sample to be repaired; Use each hole sample as the second training sample, and the second training dataset is composed of each second training sample.
4. The detection method of the vehicle drivable area according to claim 3, characterized in that The determination of whether there is a hole region in the semantic segmentation mask includes: Perform connected component detection on the semantic segmentation mask to obtain each connected component corresponding to the semantic segmentation mask; Determine the area of each connected component; For each connected component, compare the area of the connected component with a preset area threshold. If the area of the connected component is less than the area threshold, then mark the connected component as a hole; If at least one of the connected components is marked as a hole among all the connected components, then determine that there is a hole region in the semantic segmentation mask.
5. The detection method of the vehicle drivable area according to claim 1, wherein, The semantic segmentation network is a network constructed based on the U-net network.
6. The detection method of the vehicle drivable area according to claim 1, characterized in that, The loss function in the training process of the semantic segmentation network includes: a cross-entropy loss function and a Dice loss function; The loss function of the generator network in the training process of the generative adversarial network includes: an adversarial loss function and a reconstruction loss function.
7. The detection method for the drivable area of a vehicle according to claim 1, wherein The process of using the drivable region marked by the target drivable region mask as the drivable region of the vehicle includes: According to the pixel mapping relationship between the regional image and the target drivable region mask, determine the drivable region image in the regional image; Use the region represented by the drivable region image as the drivable region marked by the target drivable region mask to obtain the drivable region of the vehicle.
8. A detection device for a vehicle's drivable area, characterized in that, It includes: An image determination unit, configured to determine the regional image corresponding to the target region when it is necessary to detect the drivable region of the vehicle in the target region; A semantic segmentation unit, configured to input the regional image into a pre-constructed semantic segmentation model, obtain the drivable region mask output by the semantic segmentation model for the regional image, and use the drivable region mask output by the semantic segmentation model as the target drivable region mask; The semantic segmentation model is composed of a trained semantic segmentation network and a trained generative adversarial network; the semantic segmentation network is used to perform semantic segmentation on the image input into the semantic segmentation model to generate a drivable area mask corresponding to the image, and the generative adversarial network is used to utilize the image input into the semantic segmentation model to perform hole repair on the drivable area mask generated by the semantic segmentation network, generate a repaired drivable area mask, and use the repaired drivable area mask as the output data of the semantic segmentation model; An area extraction unit is configured to use the drivable area marked by the target drivable area mask as the drivable area of the vehicle.
9. A storage medium, characterized in that, The storage medium includes stored instructions, wherein when the instructions are running, the device where the storage medium is located is controlled to execute the method for detecting the drivable area of a vehicle according to any one of claims 1 to 7.
10. An electronic device, characterized in that, It includes a memory and one or more instructions, wherein the one or more instructions are stored in the memory and are configured to be executed by one or more processors to perform the method for detecting the drivable area of a vehicle according to any one of claims 1 to 7.