Road condition detection method, electronic device, and storage medium

Abstract

The application provides a road condition detection method, an electronic device and a storage medium. The method comprises: acquiring an image in front of a vehicle as a detection image; inputting the detection image into a trained semantic segmentation model, the semantic segmentation model comprising a backbone network and a head network; extracting features of the detection image by using the backbone network to obtain a plurality of feature maps; inputting the plurality of feature maps into the head network, processing the plurality of feature maps by a first segmentation network of the head network to obtain a first recognition result, and processing the plurality of feature maps by a second segmentation network of the head network to obtain a second recognition result; and determining whether the vehicle can continue driving based on the first recognition result and the second recognition result. The application can improve the safety of automatic driving.

Description

Technical Field

[0001] This application relates to the field of computer vision technology, and more particularly to a road condition detection method, electronic device, and storage medium. Background Technology

[0002] Environmental perception is a crucial technology in autonomous driving. Currently, most environmental perception functions are achieved using deep learning-based semantic segmentation methods. Semantic segmentation methods use deep learning segmentation models to identify objects in images. However, these methods can only identify predefined object categories, such as roads, pedestrians, and vehicles. In real-world applications, road scenes are extremely complex. If unknown object categories appear in the road scene, the trained model often misidentifies or fails to identify them, potentially leading to vehicles colliding directly with these objects and causing traffic accidents. Summary of the Invention

[0003] In view of the above, it is necessary to provide road condition detection methods, electronic devices and storage media to solve the problem that the application model cannot identify unknown types of objects, which leads to the vehicle's inability to make corresponding judgments, thereby avoiding traffic accidents.

[0004] This application provides a road condition detection method, which includes: acquiring an image of the front of a vehicle as a detection image; inputting the detection image into a trained semantic segmentation model, the semantic segmentation model including a backbone network and a head network; using the backbone network to extract features from the detection image to obtain multiple feature maps; inputting the multiple feature maps into the head network, the first segmentation network of the head network processing the multiple feature maps to obtain a first recognition result, and the second segmentation network of the head network processing the multiple feature maps to obtain a second recognition result; and determining whether the vehicle can continue to drive based on the first recognition result and the second recognition result.

[0005] In an optional implementation, the method further includes: constructing a semantic segmentation model and training the semantic segmentation model, including: acquiring a training image; inputting the training image into the backbone network for feature extraction to obtain multiple training feature maps; inputting the multiple training feature maps into the head network, whereby the first segmentation network processes the multiple training feature maps to obtain a first training result of the training image; calculating a first loss value of the first segmentation network using a preset loss function based on the first training result and a preset first expected result; processing the multiple training feature maps using the second segmentation network to obtain a second training result of the training image; calculating a second loss value of the second segmentation network using the preset loss function based on the second training result and a preset second expected result; and adjusting the parameters of the semantic segmentation model based on the first loss value and the second loss value to obtain the trained semantic segmentation model.

[0006] In one optional implementation, the acquisition of the first training result and the second training result includes: using the first segmentation network to upsample and deconvolve the plurality of training feature maps to obtain a plurality of first training feature maps of the same size as the training image; using a first softmax layer to classify each first training feature map according to a first preset pixel category to obtain the probability of each pixel classification in the first training feature map; selecting the category corresponding to the maximum probability value as the category corresponding to the pixel; and outputting the first training result, wherein the first preset pixel category includes a plurality of predefined object categories; using the second segmentation network to upsample and deconvolve the plurality of training feature maps to obtain a plurality of second training feature maps of the same size as the detection image; using a second softmax layer to classify each training feature map according to a second preset pixel category to obtain the probability of each pixel classification in the second training feature map; selecting the category corresponding to the maximum probability value as the category corresponding to the pixel; and outputting the second training result, wherein the second preset pixel category includes two predefined road categories: lane or non-lane.

[0007] In one optional implementation, adjusting the parameters of the semantic segmentation model based on the first loss value and the second loss value to obtain the trained semantic segmentation model includes: adding the first loss value and the second loss value to obtain the loss value of the semantic segmentation model; and adjusting the parameters of the semantic segmentation model using gradient descent to minimize the loss value of the semantic segmentation model, thereby obtaining the trained semantic segmentation model.

[0008] In an optional implementation, the method further includes: using the encoding network of a SegNet network as the backbone network; using the decoding network of a SegNet network as the first segmentation network in the head network; and adding a decoding network of a new SegNet network as the second segmentation network in the head network.

[0009] In one optional implementation, the first segmentation network of the head network processes the plurality of feature maps to obtain a first recognition result, including: inputting the detection image into the backbone network for convolution and max pooling operations to obtain a plurality of feature maps of the detection image; the first segmentation network upsampling and deconvolution processing the plurality of feature maps to obtain a plurality of first feature maps of the same size as the detection image; classifying each first feature map according to a first preset pixel category using the first softmax layer, and outputting the category information of each pixel in the detection image; determining the category of all objects in the detection image based on the category information of each pixel, and using the category of all objects in the detection image as the first recognition result.

[0010] In one optional implementation, the second segmentation network of the head network processes the plurality of feature images to obtain a second recognition result, including: the second segmentation network upsampling and deconvolution processing the plurality of feature images to obtain a plurality of second feature images of the same size as the detection image; and using the second softmax layer to classify each second feature image according to a second preset pixel category to determine the road category corresponding to the detection image as the second recognition result.

[0011] In one optional implementation, determining whether the vehicle can continue driving based on the first identification result and the second identification result includes: if the first identification result indicates that all object categories in the detection image have been identified, determining whether the vehicle can continue driving based on the categories of all objects in the first identification result; or if the first identification result indicates that there are unidentifiable objects in the detection image and the second identification result indicates that the road category is a lane, determining that the vehicle can continue driving; or if the first identification result indicates that there are unidentifiable objects in the detection image and the second identification result indicates that the road category is a non-lane, determining that the vehicle cannot continue driving.

[0012] This application also provides an electronic device, which includes a processor and a memory. The processor is used to execute a computer program stored in the memory to implement the road condition detection method.

[0013] Embodiments of this application also provide a computer-readable storage medium storing at least one instruction that, when executed by a processor, implements the road condition detection method.

[0014] The technical solution provided in this application can redefine and retrain the object category when the preset model cannot identify the object category in front of the vehicle, thereby ensuring that the autonomous vehicle can identify it again. By combining the two identifications, the identification results are more accurate, thereby improving the safety of autonomous driving. Attached Figure Description

[0015] Figure 1 This is a flowchart of a road condition detection method provided in an embodiment of this application.

[0016] Figure 2 This is a structural diagram of the semantic segmentation model provided in the embodiments of this application.

[0017] Figure 3 A schematic diagram of the first identification result provided in an embodiment of this application.

[0018] Figure 4 This is a schematic diagram of the second identification result provided in an embodiment of this application.

[0019] Figure 5 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application. Detailed Implementation

[0020] To better understand the above-mentioned objectives, features, and advantages of this application, the application will be described in detail below with reference to the accompanying drawings and specific embodiments. It should be noted that the specific embodiments described herein are merely illustrative and are not intended to limit the scope of this application.

[0021] Numerous specific details are set forth in the following description to provide a thorough understanding of this application. The described embodiments are merely some, not all, of the embodiments described herein. All other embodiments obtained by those skilled in the art based on the embodiments of this application without inventive effort are within the scope of protection of this application.

[0022] Hereinafter, the terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of indicated technical features. Thus, a feature defined as "first" or "second" may explicitly or implicitly include one or more of that feature. In the description of some embodiments of this application, words such as "exemplary" or "for example" are used to indicate examples, illustrations, or descriptions. Any embodiment or design described as "exemplary" or "for example" in some embodiments of this application should not be construed as being more preferred or advantageous than other embodiments or designs. Specifically, the use of words such as "exemplary" or "for example" is intended to present the relevant concepts in a specific manner.

[0023] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this application belongs. The terminology used herein is for the purpose of describing particular embodiments only and is not intended to be limiting of the application.

[0024] See Figure 1 As shown, Figure 1 A flowchart illustrating a road condition detection method provided in this application embodiment. The method is applied to electronic devices (e.g., Figure 5 In the electronic device 5 shown, the electronic device can be any electronic product that can interact with the user, such as a personal computer, tablet computer, smartphone, personal digital assistant (PDA), game console, interactive network television (Internet Protocol Television, IPTV), smart wearable device, etc.

[0025] The electronic device is a device capable of automatically performing numerical calculations and / or information processing according to pre-set or stored instructions. Its hardware includes, but is not limited to: microprocessors, application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), digital signal processors (DSPs), embedded devices, etc.

[0026] The electronic device may also include network devices and / or user devices. The network devices include, but are not limited to, a single network server, a server group consisting of multiple network servers, or a cloud based on cloud computing consisting of a large number of hosts or network servers.

[0027] The network in which the electronic device is located includes, but is not limited to: the Internet, wide area network, metropolitan area network, local area network, virtual private network (VPN), etc.

[0028] The method specifically includes the following steps.

[0029] 101. Acquire the image of the front of the vehicle as the detection image.

[0030] In at least one embodiment of this application, a camera installed inside or outside the vehicle is used to capture images of the area in front of the vehicle (e.g., the field of view), and the captured images are used as detection images.

[0031] In other embodiments, images obtained from the video footage of a dashcam can also be used as detection images using OpenCV technology. This application does not specifically limit the method of obtaining detection images.

[0032] 102. Construct a semantic segmentation model and complete the training of the semantic segmentation model.

[0033] In at least one embodiment of this application, the semantic segmentation model structure references Figure 2 As shown, the semantic segmentation model includes a backbone network and a head network, wherein the head network includes a first segmentation network and a second segmentation network.

[0034] In at least one embodiment of this application, completing the training of the semantic segmentation model includes:

[0035] Acquire training images;

[0036] The training images are input into the backbone network for feature extraction to obtain multiple training feature maps;

[0037] The multiple training feature maps are input into the head network, and the first segmentation network processes the multiple training feature maps to obtain the first training result of the training image;

[0038] Based on the first training result and the preset first expected result, the first loss value of the first segmentation network is calculated using a preset loss function;

[0039] The second segmentation network processes the plurality of training feature maps to obtain a second training result of the training image;

[0040] Based on the second training result and the preset second expected result, the second loss value of the second segmentation network is calculated using the preset loss function;

[0041] Based on the first loss value and the second loss value, the parameters of the semantic segmentation model are adjusted to obtain the trained semantic segmentation model.

[0042] In at least one embodiment of this application, acquiring the training image includes:

[0043] The semantic segmentation model is trained using images from the PASCAL Visual Object Classes (VOC) dataset, images from the Cityscapes dataset, or self-taken road condition images. This application does not impose specific limitations on this; for example, images of various road scenes can be used as training images, and these training images can include different objects as detection targets, such as vehicles, pedestrians, trees, and roadblocks.

[0044] In at least one embodiment of this application, if self-taken road condition images are used as training images, the method includes: performing data augmentation processing on the self-taken road condition images to increase the number of training images, wherein the data augmentation processing includes flipping, rotating, scaling, and shifting the training sample images. In this embodiment, performing data augmentation operations on self-taken road condition images can increase the number of training images, thereby improving the robustness of the semantic segmentation model.

[0045] In at least one embodiment of this application, the training image is input into the backbone network for feature extraction to obtain multiple training feature maps, including:

[0046] The encoding network of the SegNet network is used as the backbone network of the semantic segmentation model.

[0047] The encoding network of the SegNet network includes convolutional layers, batch normalization (BN) layers, ReLU activation layers, and max-pooling layers.

[0048] The training image is input into the convolutional layer to perform convolution operations to extract the feature values ​​of the training image. The feature values ​​are then standardized by the BN layer to calculate the current learning rate. After processing by the ReLU activation layer and the max-pooling layer, multiple training feature maps are output.

[0049] In at least one embodiment of this application, the step of inputting the plurality of training feature maps into the head network, and the first segmentation network processing the plurality of training feature maps to obtain a first training result of the training image includes:

[0050] The decoding network of the SegNet network is used as the first segmentation network in the head network of the semantic segmentation model. The decoding layer of the SegNet network includes an upsampling layer, a convolutional layer, and a first softmax layer. In this embodiment, the multiple training feature maps are input to the upsampling layer for upsampling, enlarging the training feature maps to the same size as the training image. Then, the upsampled training feature maps are input to the convolutional layer for convolution, obtaining the processed first training feature map. This first training feature map is then input to the first softmax layer for classification according to a first preset pixel category, obtaining the probability A of each pixel's classification in the training image. ik , wherein, A ik This represents the probability that the i-th pixel in the training image belongs to the k-th category. The category corresponding to the maximum probability value is selected as the category corresponding to that pixel. The category information to which each pixel belongs in the training image is output as the first training result. The category of all objects in the training image is determined based on the category information to which each pixel belongs.

[0051] In this embodiment, the semantic segmentation model is trained based on training images and corresponding pixel category labels, and multiple pixel categories can be predetermined. For example, the first preset pixel categories predicted by the first softmax layer include 19 predefined object categories, including vehicles, pedestrians, trees, roadblocks, streetlights, buildings, etc. For example, the pixel classification categories include vehicles (k=0), pedestrians (k=1), trees (k=2), roadblocks (k=3), streetlights (k=4), and buildings (k=5). After classification by the first softmax layer according to the first preset pixel categories, the probability values ​​of the i-th pixel are: A i0 =0.94, A i1 =0.23, A i2 =0.13, A i3 =0.03, A i4 =0.02, A i5 =0.01, with a maximum probability of 0.94. Since k=0, the object category can be confirmed as a vehicle. Therefore, in this example, by calculating and comparing the probability of classifying the i-th pixel, the i-th pixel can be determined to be a vehicle.

[0052] In at least one embodiment of this application, the step of calculating the first loss value of the first segmentation network using a preset loss function based on the first training result and a preset first expected result includes:

[0053] The loss function is:

[0054]

[0055] Where LOSS represents the first loss, and y represents the preset first expected result. This represents the first training result.

[0056] In at least one embodiment of this application, the second segmentation network processes the plurality of training feature maps to obtain a second training result of the training image, including:

[0057] A new SegNet network decoding network is added as the second segmentation network in the head network of the semantic segmentation model. The newly added SegNet network decoding layer includes an upsampling layer, a convolutional layer, and a second softmax layer.

[0058] The multiple training feature maps are input into the upsampling layer for upsampling, enlarging the training feature maps to the same size as the training image. Then, the upsampled training feature maps are input into the convolutional layer for convolution, obtaining a second training feature map. Finally, the second training feature map is input into the second softmax layer for classification according to a second preset pixel category, obtaining the probability A of each pixel in the training image being classified. bq , wherein, A bq The probability that the b-th pixel in the training image belongs to the b-th category is represented by the value of the pixel with the highest probability. The category corresponding to the highest probability value is selected as the category corresponding to the pixel, and the road category corresponding to the training image is determined as the second training result.

[0059] In this embodiment, the second preset pixel category includes two predefined road categories: lane or non-lane. For example, the second softmax layer predicts two object categories, namely lane or non-lane. For example, the pixel classification categories include lane (q=10) and non-lane (q=15). After classification by the second softmax layer according to the second preset pixel category, the probability values ​​of the b-th pixel are obtained as follows: A b10 =0.86, A b15 =0.33, with a maximum probability of 0.86. Since q=10, the category can be determined as lane. Therefore, in this example, by calculating and comparing the probability of classifying the b-th pixel, the road category of the b-th pixel can be determined as lane. In this embodiment, if an object in the training image is identified as a lane, it indicates that the object is not an obstacle; if an object in the training image is identified as not a lane, it indicates that the object is an obstacle.

[0060] In at least one embodiment of this application, the method of calculating the second loss value of the second segmentation network using the preset loss function based on the second training result and the preset second expected result is similar to the method of calculating the first loss value of the first segmentation network using the preset loss function, and will not be described again here.

[0061] In at least one embodiment of this application, adjusting the parameters of the semantic segmentation model based on the first loss value and the second loss value to obtain the trained semantic segmentation model includes:

[0062] The loss value of the semantic segmentation model is obtained by adding the first loss value and the second loss value.

[0063] The parameters of the semantic segmentation model are adjusted using gradient descent to minimize the loss value, resulting in a trained semantic segmentation model. In this embodiment, the gradient descent algorithm used includes stochastic gradient descent or mini-batch gradient descent. This application does not specifically limit the algorithm. In this embodiment, adjusting the parameters of the semantic segmentation model includes adjusting the learning rate of the semantic segmentation model or the number of iterations for training the training images.

[0064] 103. Input the detected image into the backbone network of the trained semantic segmentation model for feature extraction to obtain multiple feature maps.

[0065] In at least one embodiment of this application, the detected image is input into the convolutional layer of the backbone network to perform convolution operations to extract the feature values ​​of the detected image. The feature values ​​are then standardized by the BN layer to calculate the current learning rate. After processing by the ReLU activation layer and the max-pooling layer, multiple feature maps are output.

[0066] 104. The plurality of feature maps are input into the head network, and the first segmentation network of the head network processes the plurality of feature maps and outputs a first recognition result.

[0067] In at least one embodiment of this application, the plurality of feature maps are input into the head network, and the first segmentation network of the head network processes the plurality of feature maps to output a first recognition result, including:

[0068] The detected image is input into the backbone network for convolution and max pooling operations to obtain multiple feature maps of the detected image;

[0069] The first segmentation network performs upsampling and deconvolution processing on the multiple feature maps to obtain multiple feature maps of the same size as the detection image;

[0070] The first softmax layer is used to classify the multiple feature maps according to a first preset pixel category, outputting the category information of each pixel in the detection image; based on the category information of each pixel, the category of all objects in the detection image is determined, and the category of all objects in the detection image is used as the first recognition result. For example, as... Figure 3 As shown, Figure 3 This is a schematic diagram of the first recognition result provided in an embodiment of this application. The diagram shows that the recognition result obtained by the first segmentation network from the detected image classifies the object categories in the detected image by pixels, thereby obtaining the object categories on the detected image.

[0071] 105. The second segmentation network of the head network processes the multiple feature maps and outputs a second recognition result.

[0072] In at least one embodiment of this application, the second segmentation network of the head network processes the plurality of feature maps and outputs a second recognition result, including:

[0073] The second segmentation network upsamples and deconvolves the multiple feature maps to obtain multiple feature maps of the same size as the detection image;

[0074] The second softmax layer is used to classify the multiple feature maps according to a second preset pixel category, and the road category corresponding to the detected image is determined as the second recognition result, wherein the road category is lane or non-lane. For example, Figure 4 As shown, Figure 4 This is a schematic diagram of the second recognition result provided in an embodiment of this application. The diagram shows the recognition result obtained by the second segmentation network from the detected image, which classifies the object categories in the detected image by pixels to determine the road category corresponding to the detected image. In this embodiment, the lane is considered a non-obstacle, and the non-lane is considered an obstacle.

[0075] The process of obtaining the first recognition result described above can refer to the process of obtaining the first training result described above. Similarly, the process of obtaining the second recognition result can refer to the process of obtaining the second training result described above. It will not be repeated here.

[0076] It should be noted that the first segmentation network and the second segmentation network process the received feature maps simultaneously. When the first segmentation network obtains the first recognition result, it judges the recognized category to determine the next operation of the vehicle. When the first recognition result shows that there is an unrecognizable category, the second recognition result is invoked, and the next operation of the vehicle is determined based on the second recognition result.

[0077] 106. Based on the first identification result and the second identification result, determine whether the vehicle can continue to drive.

[0078] In at least one embodiment of this application, determining whether the vehicle can continue to drive based on the first identification result and the second identification result includes:

[0079] If the first recognition result indicates that all object categories in the detected image have been identified, then it is determined whether the vehicle can continue driving based on the object categories in the first recognition result; or

[0080] If the first identification result indicates the presence of an unidentifiable object in the detected image and the second identification result indicates the road category is a lane, it is considered that there is no obstacle in front of the vehicle, and the vehicle can continue driving; or

[0081] If the first recognition result indicates the presence of an unidentifiable object in the detected image and the second recognition result indicates the road category is non-lane, it is considered that there is an obstacle in front of the vehicle, and the vehicle is determined not to continue driving.

[0082] In at least one embodiment of this application, when a first recognition result is obtained through the first segmentation network, if the first recognition result cannot identify the object category, a second recognition result is invoked to determine whether the vehicle can continue driving. For example, if there is a stroller in front of the vehicle, and the stroller category was not included in the training of the first segmentation network, the first segmentation network cannot identify the stroller in front of the vehicle. In this case, the second recognition result is invoked, and if the second recognition result indicates that the road category is not a lane, it is considered that there is an obstacle in front of the vehicle, and it is determined that the vehicle cannot continue driving.

[0083] The above description is merely a specific embodiment of this application, but the scope of protection of this application is not limited thereto. For those skilled in the art, improvements can be made without departing from the inventive concept of this application, but these all fall within the scope of protection of this application.

[0084] like Figure 5 As shown, Figure 5 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application. The electronic device 5 includes a memory 501, at least one processor 502, a computer program 503 stored in the memory 501 and executable on the at least one processor 502, and at least one communication bus 504.

[0085] Those skilled in the art will understand that Figure 5The schematic diagram shown is merely an example of the electronic device 5 and does not constitute a limitation on the electronic device 5. It may include more or fewer components than shown, or combine certain components, or different components. For example, the electronic device 5 may also include input / output devices, network access devices, etc.

[0086] The at least one processor 502 may be a Central Processing Unit (CPU), or other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The at least one processor 502 may be a microprocessor or any conventional processor, etc. The at least one processor 502 is the control center of the electronic device 5, connecting various parts of the electronic device 5 via various interfaces and lines.

[0087] The memory 501 can be used to store the computer program 503. The at least one processor 502 implements various functions of the electronic device 5 by running or executing the computer program 503 stored in the memory 501 and calling data stored in the memory 501. The memory 501 may mainly include a program storage area and a data storage area. The program storage area may store the operating system, application programs required for at least one function (such as sound playback function, image playback function, etc.), etc.; the data storage area may store data created according to the use of the electronic device 5 (such as audio data), etc. In addition, the memory 501 may include non-volatile memory, such as hard disk, memory, plug-in hard disk, smart media card (SMC), secure digital (SD) card, flash card, at least one disk storage device, flash memory device, or other non-volatile solid-state storage device.

[0088] If the modules / units integrated in the electronic device 5 are implemented as software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, all or part of the processes in the methods of the above embodiments can also be implemented by a computer program instructing related hardware. The computer program can be stored in a computer-readable storage medium, and when executed by a processor, it can implement the steps of the various method embodiments described above. The computer program includes computer program code, which can be in the form of source code, object code, executable files, or certain intermediate forms. The computer-readable medium can include: any entity or device capable of carrying the computer program code, a recording medium, a USB flash drive, a portable hard drive, a magnetic disk, an optical disk, a computer memory, and a read-only memory (ROM).

[0089] It will be apparent to those skilled in the art that this application is not limited to the details of the exemplary embodiments described above, and that this application can be implemented in other specific forms without departing from the spirit or essential characteristics of this application. Therefore, the embodiments should be considered illustrative and non-limiting in all respects, and the scope of this application is defined by the appended claims rather than the foregoing description. Thus, all variations falling within the meaning and scope of equivalents of the claims are intended to be embraced within this application. No appended diagram markings in the claims should be construed as limiting the scope of the claims.

[0090] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of this application and are not intended to limit it. Although this application has been described in detail with reference to preferred embodiments, those skilled in the art should understand that modifications or equivalent substitutions can be made to the technical solutions of this application without departing from the spirit and scope of the technical solutions of this application.

Claims

1. A road condition detection method, characterized in that, The road condition detection method includes: Acquire an image of the front of the vehicle as the detection image; The detected image is input into the trained semantic segmentation model, which includes a backbone network and a head network. The backbone network is used to extract features from the detected image to obtain multiple feature maps; The multiple feature maps are input into the head network. The first segmentation network of the head network processes the multiple feature maps to determine the category of all objects in the detection image. A first recognition result is obtained based on the category of all objects in the detection image. The second segmentation network of the head network processes the multiple feature maps to determine the road category corresponding to the detection image and obtain a second recognition result. Determining whether the vehicle can continue to drive based on the first identification result and the second identification result includes: when the first segmentation network obtains the first identification result, judging the identified category to determine whether the vehicle can continue to drive; when the first identification result shows an unidentifiable category, calling the second identification result and judging whether the vehicle can continue to drive based on the second identification result.

2. The road condition detection method according to claim 1, characterized in that, The method further includes: Constructing a semantic segmentation model and training the semantic segmentation model includes: Acquire training images; The training images are input into the backbone network for feature extraction to obtain multiple training feature maps; The multiple training feature maps are input into the head network, and the first segmentation network processes the multiple training feature maps to obtain the first training result of the training image; Based on the first training result and the preset first expected result, the first loss value of the first segmentation network is calculated using a preset loss function; The second segmentation network processes the plurality of training feature maps to obtain a second training result of the training image; Based on the second training result and the preset second expected result, the second loss value of the second segmentation network is calculated using the preset loss function; Based on the first loss value and the second loss value, the parameters of the semantic segmentation model are adjusted to obtain the trained semantic segmentation model.

3. The road condition detection method according to claim 2, characterized in that, The methods for obtaining the first training result and the second training result include: The first segmentation network is used to upsample and deconvolve the multiple training feature maps to obtain multiple first training feature maps of the same size as the training image. The first softmax layer is used to classify each first training feature map according to the first preset pixel category to obtain the probability of each pixel in the training image. The category corresponding to the maximum probability value is selected as the category corresponding to the pixel, and the first training result is output. The first preset pixel category includes multiple predefined object categories. The second segmentation network is used to upsample and deconvolve the multiple training feature maps to obtain multiple second training feature maps of the same size as the detection image. The second softmax layer is used to classify each training feature map according to the second preset pixel category to obtain the probability of each pixel in the training image. The category corresponding to the maximum probability value is selected as the category corresponding to the pixel, and the second training result is output. The second preset pixel category includes two predefined road categories: lane or non-lane.

4. The road condition detection method according to claim 2, characterized in that, The step of adjusting the parameters of the semantic segmentation model based on the first loss value and the second loss value to obtain the trained semantic segmentation model includes: The loss value of the semantic segmentation model is obtained by adding the first loss value and the second loss value. The parameters of the semantic segmentation model are adjusted using gradient descent to minimize the loss value of the semantic segmentation model, thus obtaining the trained semantic segmentation model.

5. The road condition detection method according to any one of claims 1 to 4, characterized in that, The method further includes: The encoding network of the SegNet network is used as the backbone network; The decoding network of the SegNet network is used as the first segmentation network in the head network; and A new SegNet decoding network is added as the second segmentation network in the head network.

6. The road condition detection method according to claim 5, characterized in that, The first segmentation network of the head network processes the multiple feature maps to obtain a first recognition result, including: The detected image is input into the backbone network for convolution and max pooling operations to obtain multiple feature maps of the detected image; The first segmentation network performs upsampling and deconvolution processing on the multiple feature maps to obtain multiple first feature maps of the same size as the detection image; The first softmax layer is used to classify each first feature map according to a first preset pixel category, and the category information of each pixel in the detected image is output. The category of all objects in the detected image is determined based on the category information of each pixel, and the category of all objects in the detected image is used as the first recognition result.

7. The road condition detection method according to claim 5, characterized in that, The second segmentation network of the head network processes the multiple feature images to obtain a second recognition result, including: The second segmentation network upsamples and deconvolves the multiple feature maps to obtain multiple second feature maps of the same size as the detection image; The second softmax layer is used to classify each second feature map according to the second preset pixel category, and the road category corresponding to the detected image is determined as the second recognition result.

8. The road condition detection method according to claim 7, characterized in that, Determining whether the vehicle can continue to drive based on the first identification result and the second identification result includes: If the first recognition result indicates that all object categories in the detected image have been identified, then it is determined whether the vehicle can continue driving based on the object categories in the first recognition result; or If the first identification result indicates the presence of an unidentifiable object in the detected image and the second identification result indicates the road category is a lane, then the vehicle is determined to continue driving; or If the first identification result indicates the presence of an unidentifiable object in the detected image and the second identification result indicates the road category is non-lane, then the vehicle is determined not to continue driving.

9. An electronic device, characterized in that, The electronic device includes a processor and a memory, the processor being configured to execute a computer program stored in the memory to implement the road condition detection method as described in any one of claims 1 to 8.

10. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores at least one instruction, which, when executed by a processor, implements the road condition detection method as described in any one of claims 1 to 8.

Images