A method and apparatus for generating a semantic segmentation network

Abstract

This application discloses a method and apparatus for generating semantic segmentation networks, relating to the field of artificial intelligence. By pre-setting the downsampling rate range, the total number of layers, and the hierarchical search space, the number of poorly optimized networks can be minimized, ensuring that candidate networks meet the structural requirements of practical applications. By determining the downsampling probability of each node and using a genetic algorithm and hierarchical search space, the operations at each node in the network can be iteratively updated, resulting in multiple candidate networks. This process of obtaining multiple candidate networks requires no human intervention, thus reducing labor costs. By utilizing the information entropy of the candidate networks to obtain their evaluation scores, and selecting the candidate network with the highest evaluation score as the semantic segmentation network, the performance comparison of multiple candidate networks is relatively fast, resulting in high efficiency in generating the semantic segmentation network.

Description

Technical Field

[0001] This application relates to the field of artificial intelligence, and in particular to a method and apparatus for generating semantic segmentation networks. Background Technology

[0002] Semantic segmentation is a typical computer vision problem that involves taking some raw data (such as planar images) as input and transforming them into masks with highlighted regions of interest.

[0003] In the field of autonomous driving, the detection of drivable areas falls under the category of semantic segmentation. Currently, semantic segmentation functions typically involve AI algorithm engineers designing deep networks, which require data collection and annotation for autonomous driving scenarios. The collected data is then used to train the deep network to obtain a semantic segmentation network (hereinafter referred to as a semantic segmentation network). However, this method of generating semantic segmentation networks requires the involvement of algorithm engineers with extensive practical experience in the design process, as well as a large number of annotators for labeling, resulting in significant manpower costs and low generation efficiency. Summary of the Invention

[0004] In view of this, this application provides a method and apparatus for generating semantic segmentation networks, thereby generating semantic segmentation networks that can be used to detect drivable areas of autonomous vehicles, and achieving the goal of reducing labor costs and improving generation efficiency.

[0005] The method for generating semantic segmentation networks provided in this application is implemented as follows:

[0006] Within a preset downsampling rate range, the downsampling probability of each node is determined based on the preset total number of layers in the network, the layer number of each node in the network, and the downsampling rate corresponding to each node.

[0007] Based on the genetic algorithm, according to the hierarchical search space and the downsampling probability of each node, the convolution operation, pooling operation, skip connection operation and downsampling operation of each node in the initial network are mutated to obtain multiple candidate networks. The hierarchical search space includes: the set of convolution operation, pooling operation and skip connection operation corresponding to the node.

[0008] Evaluation scores are obtained based on the information entropy of candidate networks, and the candidate network with the highest evaluation score is selected as the semantic segmentation network. The semantic segmentation network is used to predict the drivable area of ​​autonomous vehicles.

[0009] Optionally, the downsampling probability of each node is determined based on the preset total number of layers in the network, the layer number of each node in the network, and the downsampling factor corresponding to each node, including:

[0010] The downsampling probability of each node is determined using the following formula:

[0011]

[0012] Where P(downsample) is the downsampling probability of a node, r is the downsampling factor corresponding to the node, L is the preset total number of layers in the network, and l is the layer number of the node in the network.

[0013] Optionally, before obtaining an evaluation score based on the information entropy of the candidate networks and selecting the candidate network with the highest evaluation score from multiple candidate networks as the semantic segmentation network, the process also includes:

[0014] The parameters of the candidate network are randomly initialized using a Gaussian distribution;

[0015] Input the test image with Gaussian noise into the candidate network and calculate the upper bound information entropy of Gaussian output of each node in the candidate network;

[0016] Multiply the upper bound of the Gaussian information entropy output by each node in the candidate network by the size of the feature map output by each node in the candidate network to obtain the information entropy corresponding to each node in the candidate network.

[0017] The information entropy of each node in the candidate network is summed to obtain the evaluation score of the candidate network.

[0018] Optionally, the upper bound information entropy of the Gaussian output of each node in the candidate network is calculated, including:

[0019] The upper bound information entropy of the Gaussian output of each node in the candidate network is calculated using the following formula:

[0020]

[0021] Where H(F) is the upper bound of the Gaussian information entropy of the node output, h D The size of the feature map output by the node.

[0022] Optionally, based on a genetic algorithm, before mutating the convolution, pooling, skip connection, and downsampling operations of each node in the initial network according to the hierarchical search space and the downsampling probability of each node to obtain multiple candidate networks, the following steps are also included:

[0023] Select the first convolution operation and the second convolution operation from the set of convolution operations, and then concatenate the first convolution operation and the second convolution operation.

[0024] The first pooling operation is selected from the pooling operation set, and the first jump connection operation is selected from the jump connection operation set. The first pooling operation and the first jump connection operation are connected after the first convolution operation and the second convolution operation to obtain the initial network.

[0025] Optionally, the set of convolution operations includes: 3×3 depthwise separable convolution, 5×5 depthwise separable convolution, 3×3 ordinary convolution, and 5×5 ordinary convolution;

[0026] The pooling operations set includes: 3×3 average pooling, 3×3 max pooling, and no pooling.

[0027] The set of jump connection operations includes: jump connections with and jump connections without.

[0028] Optionally, the downsampling rate range includes:

[0029] The range from 8x downsampling to 32x downsampling.

[0030] Optionally, after obtaining the evaluation score based on the information entropy of the candidate networks and selecting the candidate network with the highest evaluation score from multiple candidate networks as the semantic segmentation network, the process further includes:

[0031] Acquire real-time radar images of autonomous vehicles;

[0032] A semantic segmentation network is used to classify pixels in real-time radar images to obtain a set of drivable pixels and a set of non-drivable pixels;

[0033] The drivable area for autonomous vehicles is determined based on the set of drivable pixels.

[0034] This application also provides an apparatus for generating a semantic segmentation network, comprising:

[0035] The determination module is used to determine the downsampling probability of each node within a preset downsampling rate range, based on the preset total number of layers in the network, the layer in which each node is located, and the downsampling rate corresponding to each node.

[0036] The mutation module is used to mutate the convolution, pooling, skip connection, and downsampling operations of each node in the initial network based on the genetic algorithm, according to the hierarchical search space and the downsampling probability of each node, to obtain multiple candidate networks. The hierarchical search space includes: the set of convolution operations, the set of pooling operations, and the set of skip connection operations corresponding to the node.

[0037] The selection module is used to obtain an evaluation score based on the information entropy of the candidate networks and select the candidate network with the highest evaluation score from multiple candidate networks as the semantic segmentation network. The semantic segmentation network is used to predict the drivable area of ​​autonomous vehicles.

[0038] This application also provides a computer device, including: a processor coupled to a memory, the memory storing at least one computer program instruction, the at least one computer program instruction being loaded and executed by the processor to enable the computer device to implement a method for generating a semantic segmentation network.

[0039] Therefore, the beneficial effects of this application are as follows: By pre-setting the downsampling rate range, the total number of layers, and the hierarchical search space of the network, the number of networks with insufficient structural optimization can be reduced to the greatest extent, so that the candidate network can meet the structural needs of practical applications; by determining the downsampling probability of each node and based on the genetic algorithm and the hierarchical search space, the operations in each node of the network can be continuously iterated and updated to obtain multiple candidate networks. At this time, the process of obtaining multiple candidate networks does not require human intervention, thus reducing labor costs; at the same time, by using the information entropy of the candidate networks to obtain the evaluation score of the candidate networks, and selecting the candidate network with the highest evaluation score from multiple candidate networks as the semantic segmentation network, since the information entropy reflects the amount of information that the candidate network can contain, it can reflect the performance of the candidate network. At this time, the performance of the candidate network can be determined without long-term training of the candidate network, and the speed of comparing the performance of multiple candidate networks is fast, so the efficiency of generating the semantic segmentation network is also high. Attached Figure Description

[0040] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only embodiments of this application. For those skilled in the art, other drawings can be obtained based on the provided drawings.

[0041] Figure 1 This is a flowchart of the first embodiment of this application;

[0042] Figure 2 A schematic diagram of an overall architecture provided for an embodiment of this application;

[0043] Figure 3 A schematic diagram of a network-level search space provided for an embodiment of this application;

[0044] Figure 4 A schematic diagram of a hierarchical search space provided for an embodiment of this application;

[0045] Figure 5 This is a flowchart of steps S501 to S506 in the second embodiment of this application;

[0046] Figure 6 This is a flowchart of steps S507 to S513 in the second embodiment of this application;

[0047] Figure 7 This is a schematic diagram of an apparatus according to this application;

[0048] Figure 8 This is a schematic diagram of a computer device according to this application. Detailed Implementation

[0049] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0050] The inventors discovered that Neural Architecture Search (NAS) is an optimization-based algorithm that can automatically explore various potential network architectures, providing greater flexibility and more efficient solutions for network structure design. Furthermore, experiments showed that NAS can achieve performance comparable to or even better than manually designed network architectures in tasks such as image classification, object detection, and semantic segmentation. Based on this analysis and the inventors' experience in image segmentation tasks, this application proposes a method for generating semantic segmentation networks based on NAS.

[0051] Please see Figure 1 The specific steps of the first embodiment of this application are as follows:

[0052] S101: Within the preset downsampling rate range, determine the downsampling probability of each node based on the preset total number of layers in the network, the layer number of each node in the network, and the downsampling rate corresponding to each node.

[0053] Please see Figure 2 , Figure 2 This is a schematic diagram of an overall architecture provided for an embodiment of this application. In this application, the improvements to NAS mainly include optimizations in three parts: search space, network search strategy, and performance evaluation. The search space part will be described first below.

[0054] The search space is used to define all possible structures and operations of the network. In this application, the selection of the search space can be divided into network-level and layer-level. The network-level search space is used to select appropriate paths for the network, i.e., the timing of downsampling in the network, while the layer-level search space is used to define the possible operations of each layer in the network.

[0055] Please see Figure 3 , Figure 3This is a schematic diagram of a network-level search space provided in an embodiment of this application. The network-level search space imposes restrictions on the path search of the network regarding the downsampling rate range of the network and a preset total number of layers.

[0056] The smallest unit of the network-level search space is a node (cell). Figure 3 Each circle in the diagram represents a node, and a node corresponds to a layer in the network structure.

[0057] Since the network obtained in this application is used for semantic segmentation, a suitable downsampling rate range for the semantic segmentation task can be set, enabling the obtained network to perform the semantic segmentation task well. In this case, network path search is only performed within the preset downsampling rate range, and after passing through a preset total number of layers (L), an output head is connected. It should be noted that within the preset downsampling rate range, for each additional downsampling operation, the number of layers in the network decreases by one.

[0058] Specifically, a maximum downsampling factor of 32 and a minimum downsampling factor of 8 can be set, resulting in a downsampling factor range from 8x to 32x. It should be noted that in dense image prediction, the maximum downsampling factor typically does not exceed 32x. Therefore, the maximum downsampling factor can be set to other values ​​according to actual needs, but the general principle is that it should be less than 32x. Furthermore, the minimum downsampling factor can also be set to other values ​​according to actual needs. However, since the feature map size before 8x downsampling is relatively large, requiring significant network resources, it is advisable to choose not to search the network before 8x downsampling to speed up the network search.

[0059] In some implementations, before determining the downsampling probability of each node within a preset downsampling rate range, based on the preset total number of layers in the network, the layer number of each node in the network, and the downsampling rate corresponding to each node, it is necessary to obtain the network-level search space and the layer-level search space in advance.

[0060] After determining the search space of the deep network, a certain network search strategy needs to be used to generate the corresponding deep network. Specifically, considering that most current deep networks perform downsampling operations in the early stages of the network, this application needs to randomly select paths in the network, concatenate the nodes in the network, and calculate the probability of downsampling at each node (i.e., network-level model search) when reaching each node.

[0061] Because a downsampling factor range is defined in the network-level search space, the networks obtained from the search may correspond to different downsampling factors. Therefore, the downsampling factors corresponding to nodes in different networks are also different. When calculating the downsampling probability of a node, the total number of layers in the network and the downsampling factor corresponding to each node in the network are first determined based on the downsampling factor of the network, and then the layer number of each node in the network is determined. Taking a network with a downsampling factor of 8 as an example, the total number of layers is 10. In this case, the larger the downsampling factor, the fewer the number of layers in the network. For example, a network with a downsampling factor of 16 has a total of 9 layers, and a network with a downsampling factor of 32 has a total of 8 layers.

[0062] After determining the above data, the downsampling probability of each node can be determined according to the following formula:

[0063]

[0064] Where P(downsample) is the downsampling probability of a node, r is the downsampling factor corresponding to the node, L is the preset total number of layers in the network, and l is the layer number of the node in the network.

[0065] As can be seen from the above formula, as the network downsampling ratio increases, the downsampling probability of each node decreases exponentially. Similarly, as the total number of network layers increases, the downsampling probability of each node also decreases accordingly.

[0066] It should be noted that the downsampling factor of the network determines the number of nodes in the network that need to be downsampled. For example, if the downsampling factor of the network is 8, it means that three nodes in the network need to be downsampled by 2x.

[0067] S102: Based on the genetic algorithm, according to the hierarchical search space and the downsampling probability of each node, the convolution operation, pooling operation, skip connection operation and downsampling operation of each node in the initial network are mutated to obtain multiple candidate networks.

[0068] Please see Figure 4 , Figure 4 This diagram illustrates a hierarchical search space provided in an embodiment of this application. Since each node in the network can perform convolution, pooling, and skip operations, the hierarchical search space can include: the set of convolution operations, the set of pooling operations, and the set of skip operations corresponding to each node.

[0069] Specifically, please see Figure 4In part (a), the set of convolutional operations can include: 3×3 depthwise separable convolution, 5×5 depthwise separable convolution, 3×3 ordinary convolution, and 5×5 ordinary convolution; the set of pooling operations can include: 3×3 average pooling, 3×3 max pooling, and no pooling; the set of skip connections can include: using skip connections and not using skip connections. It should be noted that the specific operations included in the above sets of convolutional operations, pooling operations, and skip connections can also be set according to actual needs.

[0070] After completing the network-level model search, that is, after piecing together the various nodes of the network, the network path can be determined, and the hierarchical network search can begin.

[0071] Since the hierarchical search space contains the set of convolution operations, pooling operations, and skip operations that each node can choose from, and the downsampling probability of each node can be used to determine whether each node needs to be downsampled, the internal operations of all nodes in the network can be randomly initialized to determine an initial network, and the internal operations of each node in the initial network can be iterated using a genetic algorithm.

[0072] In some implementations, the initial network can be determined as follows: select a first convolution operation and a second convolution operation from the set of convolution operations, and connect the first convolution operation and the second convolution operation; select a first pooling operation from the set of pooling operations, select a first skip connection operation from the set of skip connection operations, and connect the first pooling operation and the first skip connection operation after the first convolution operation and the second convolution operation to obtain the initial network.

[0073] It should be noted that the mutation probability of convolution, pooling, and skip operations can be set to 0.5, while the mutation probability of downsampling operations is the downsampling probability. Specifically, the mutation probabilities of convolution, pooling, and skip operations can be set according to actual needs and are not limited to 0.5 as mentioned above.

[0074] For other implementations, please refer to Figure 4 In part (b) of the diagram, for nodes in the network that require downsampling, the convolution stride of these nodes can be set to 2, thereby reducing the size of the feature map. For other implementation methods, please refer to [link to relevant documentation]. Figure 4 In part (c), for nodes located at the output head position in the network, concat needs to be added after the convolution part, pooling part and skip connection part of the node, and the feature map size is processed by 1×1 convolution, and finally upsampling is used.

[0075] S103: Obtain the evaluation score based on the information entropy of the candidate network, and select the candidate network with the highest evaluation score from multiple candidate networks as the semantic segmentation network.

[0076] The evaluation score is obtained based on the information entropy of the candidate networks. In conventional NAS methods, the performance evaluation part usually involves training the searched deep networks on a specified dataset, and then evaluating the performance of the deep networks using evaluation metrics. However, training deep networks is an extremely time-consuming task, usually requiring several days or even weeks to complete. Training and evaluating each of the large number of deep networks obtained through NAS is almost impossible.

[0077] The inventors recognized that the amount of information a deep network can contain determines the upper limit of the network, and a network with higher information entropy can theoretically achieve better performance. Therefore, in this application, the performance evaluation part adopts an evaluation method based on network information entropy. The network's information entropy can be calculated through only one forward propagation, and this is used as the performance index of the network to compare with other networks.

[0078] Therefore, in some implementations, the evaluation score of the candidate network can be achieved as follows: the parameters of the candidate network are randomly initialized with a Gaussian distribution; a test image with Gaussian noise is input into the candidate network, and the upper bound information entropy of the Gaussian output of each node in the candidate network is calculated; the upper bound information entropy of the Gaussian output of each node in the candidate network is multiplied by the size of the feature map output of each node in the candidate network to obtain the information entropy corresponding to each node in the candidate network; the information entropy corresponding to each node in the candidate network is summed to obtain the evaluation score of the candidate network.

[0079] It should be noted that the semantic segmentation network is used to predict the drivable area of ​​autonomous vehicles. In other implementations, the semantic segmentation network generated by the method provided in this application can also be applied to scenarios such as medical image analysis and video surveillance.

[0080] Specifically, an evaluation score is obtained based on the information entropy of the candidate network. After selecting the candidate network with the highest evaluation score from multiple candidate networks as the semantic segmentation network, real-time radar images of autonomous vehicles can be obtained. The semantic segmentation network is used to classify the pixels in the real-time radar images to obtain a set of drivable pixels and a set of non-drivable pixels. Based on the set of drivable pixels, the drivable area of ​​the autonomous vehicle is determined.

[0081] In the first embodiment of this application, by pre-setting the downsampling rate range, the preset total number of layers, and the hierarchical search space of the network, the number of networks with insufficiently optimized structures can be minimized, enabling candidate networks to meet the structural needs of practical applications. By determining the downsampling probability of each node and based on the genetic algorithm and the hierarchical search space, the operations in each node of the network can be continuously iterated and updated, thereby obtaining multiple candidate networks. At this time, the process of obtaining multiple candidate networks does not require human intervention, thus reducing labor costs. At the same time, by using the information entropy of the candidate networks to obtain the evaluation score of the candidate networks, and selecting the candidate network with the highest evaluation score from multiple candidate networks as the semantic segmentation network, since the information entropy reflects the amount of information that the candidate network can contain, it can reflect the performance of the candidate network. At this time, the performance of the candidate networks can be determined without long-term training, and the speed of comparing the performance of multiple candidate networks is fast, thus the efficiency of generating semantic segmentation networks is also high.

[0082] In the second embodiment, a specific implementation of this application is described in conjunction with a specific application scenario.

[0083] Please see Figure 5 The specific steps of the second embodiment of this application are as follows:

[0084] S501: Obtain the network-level search space and the hierarchical search space.

[0085] S502: Within the range of 8x downsampling to 32x downsampling, determine the downsampling probability of each node based on the preset total number of layers in the network, the layer number of each node in the network, and the downsampling factor corresponding to each node.

[0086] It should be noted that in this step, the various nodes of the network will be spliced ​​together.

[0087] S503: Select the first convolution operation and the second convolution operation from the set of convolution operations, and concatenate the first convolution operation and the second convolution operation.

[0088] Specifically, the order in which the first convolution operation and the second convolution operation are connected can be either the first convolution operation first and the second convolution operation last, or the second convolution operation first and the first convolution operation last.

[0089] By concatenating the first and second convolution operations, the convolutional part of a node can be obtained. The first and second convolution operations are randomly selected. By selecting two convolution operations for the convolutional part of a node, there are potentially 12 possibilities for the convolutional part of each node, thus giving the network more options. It should be noted that other numbers of convolutional operations can also be selected for each node according to actual needs.

[0090] It should be noted that step S503 needs to be performed on each node in the network. At this time, the convolutional parts of different nodes in the network can be different.

[0091] S504: Select the first pooling operation from the pooling operation set, select the first jump connection operation from the jump connection operation set, and connect the first pooling operation and the first jump connection operation after the first convolution operation and the second convolution operation to obtain the initial network.

[0092] The first pooling operation can be any pooling operation from the set of pooling operations, and the first skip-connection operation can be any skip-connection operation from the set of skip-connection operations. By concatenating the first pooling operation with the first convolution operation and the second convolution operation, the pooled part of the node can be obtained; by concatenating the first skip-connection operation with the first convolution operation and the second convolution operation, the skip-connection part of the node can be obtained.

[0093] It should be noted that step S504 needs to be performed on each node in the network. At this time, the pooling part of different nodes in the network can be different, and the jump connection part of different nodes in the network can also be different.

[0094] In some implementations, after obtaining the initial network, the network performance is evaluated and the evaluation score of the initial network is recorded.

[0095] S505: Based on a genetic algorithm, according to the hierarchical search space and the downsampling probability of each node, the convolution operation, pooling operation, skip connection operation and downsampling operation of each node in the initial network are mutated to obtain multiple candidate networks.

[0096] S506: Initialize the parameters of the candidate network using a Gaussian distribution.

[0097] It should be noted that steps S506 to S509 need to be performed on multiple candidate networks separately to evaluate the performance of each candidate network.

[0098] S507: Input the Gaussian noise test image into the candidate network and calculate the upper bound information entropy of the Gaussian output of each node in the candidate network.

[0099] Since the semantic segmentation network generated in this application is used to detect the drivable area of ​​autonomous vehicles, the test image can be a historical radar image of the autonomous vehicle.

[0100] In some implementations, the upper bound of the Gaussian information entropy of the output of each node in the candidate network can be calculated using the following formula:

[0101]

[0102] Where H(F) is the upper bound of the Gaussian information entropy of the node output, h D The size of the feature map output by the node.

[0103] S508: Multiply the upper bound of the Gaussian information entropy output by each node in the candidate network by the size of the feature map output by each node in the candidate network to obtain the information entropy corresponding to each node in the candidate network.

[0104] S509: Add up the information entropy of each node in the candidate network to obtain the evaluation score of the candidate network.

[0105] The information entropy of each node in the candidate network indicates the amount of information that each node can hold. By adding up the information entropy of each node in the candidate network, we can obtain the total amount of information that the candidate network can hold. The more information that the candidate network can hold, the better its network performance.

[0106] In some implementations, the network performance of a candidate network can be evaluated whenever a candidate network is obtained.

[0107] In other implementations, when the number of candidate networks reaches a preset threshold, in order to control the population size and select the best to continue propagating, candidate networks with poor performance will be discarded based on the evaluation scores of multiple candidate networks.

[0108] S510: Obtain the evaluation score based on the information entropy of the candidate network, and select the candidate network with the highest evaluation score from multiple candidate networks as the semantic segmentation network.

[0109] It should be noted that when the number of iterations reaches the preset threshold, step S510 is executed.

[0110] S511: Acquire real-time radar images of autonomous vehicles.

[0111] S512: Use a semantic segmentation network to classify pixels in real-time radar images to obtain a set of drivable pixels and a set of non-drivable pixels.

[0112] S513: Determine the drivable area for autonomous vehicles based on the set of drivable pixels.

[0113] In the second embodiment of this application, in the search space part, the search space is optimized by adding some prior conditions in advance, thereby increasing the information density of the search space and minimizing invalid searches. In the network search strategy part, by defining network-level search strategies and hierarchical search strategies respectively, the specific paths of the deep network and the corresponding operations in each layer can be initially determined. In the performance evaluation part, the performance of multiple candidate networks is compared using an information entropy-based evaluation method, which can greatly improve the speed of comparing the performance of deep networks. The above method greatly reduces the interference of subjective factors of algorithm engineers, can automatically search for a model that performs well in semantic segmentation tasks, reduce labor costs, and improve the generation efficiency of semantic segmentation networks.

[0114] Please see Figure 7 This application provides an apparatus 700 for generating a semantic segmentation network, including: a determination module 701, a mutation module 702, and a selection module 703.

[0115] Determining module 701: Used to determine the downsampling probability of each node within a preset downsampling rate range, based on the preset total number of layers in the network, the layer number of each node in the network, and the downsampling rate corresponding to each node.

[0116] Mutation module 702: Based on a genetic algorithm, it mutates the convolution operation, pooling operation, skip connection operation and downsampling operation of each node in the initial network according to the hierarchical search space and the downsampling probability of each node, to obtain multiple candidate networks. The hierarchical search space includes: the set of convolution operation, pooling operation and skip connection operation corresponding to the node.

[0117] Selection module 703: used to obtain an evaluation score based on the information entropy of the candidate network, and select the candidate network with the highest evaluation score from multiple candidate networks as the semantic segmentation network. The evaluation score is obtained based on the information entropy of the candidate network. The semantic segmentation network is used to predict the drivable area of ​​autonomous vehicles.

[0118] Optionally, module 701 is specifically used to determine the downsampling probability of each node according to the following formula:

[0119]

[0120] Where P(downsample) is the downsampling probability of a node, r is the downsampling factor corresponding to the node, L is the preset total number of layers in the network, and l is the layer number of the node in the network.

[0121] Optionally, an apparatus 700 for generating a semantic segmentation network further includes an initialization module 704 and a computation module 705.

[0122] Initialization module 704: Used to perform Gaussian distribution random initialization of the parameters of the candidate network.

[0123] Calculation module 705: used to input the test image with Gaussian noise into the candidate network and calculate the upper bound information entropy of Gaussian output of each node in the candidate network.

[0124] Calculation module 705: It is also used to multiply the upper bound information entropy of the Gaussian output of each node in the candidate network with the size of the feature map output of each node in the candidate network to obtain the information entropy corresponding to each node in the candidate network.

[0125] Calculation module 705: It is also used to sum the information entropy corresponding to each node in the candidate network to obtain the evaluation score of the candidate network.

[0126] Optionally, the calculation module 705 is specifically used to calculate the Gaussian upper bound information entropy of the output of each node in the candidate network according to the following formula:

[0127]

[0128] Where H(F) is the upper bound of the Gaussian information entropy of the node output, h D The size of the feature map output by the node.

[0129] Optionally, the selection module 703 is further configured to: select a first convolution operation and a second convolution operation from the set of convolution operations, and connect the first convolution operation and the second convolution operation; select a first pooling operation from the set of pooling operations, select a first skip connection operation from the set of skip connection operations, and connect the first pooling operation and the first skip connection operation after the first convolution operation and the second convolution operation to obtain the initial network.

[0130] Optionally, an apparatus 700 for generating a semantic segmentation network further includes an acquisition module 706 and a classification module 707.

[0131] Acquisition module 706: Used to acquire real-time radar images of autonomous vehicles.

[0132] Classification module 707: Used to classify pixels in real-time radar images using a semantic segmentation network to obtain a set of drivable pixels and a set of non-drivable pixels.

[0133] Determining module 701: is also used to determine the drivable area of ​​the autonomous vehicle based on the set of drivable pixels.

[0134] Regarding the apparatus in the above embodiments, the specific manner in which each module performs its operation has been described in detail in the embodiments related to the method, and will not be elaborated upon here.

[0135] It should be noted that the apparatus for generating semantic segmentation networks provided in the above embodiments is only illustrated by the division of the above functional modules when implementing the function of generating semantic segmentation networks. In practical applications, the above functions can be assigned to different functional modules as needed, that is, the internal structure of the apparatus for generating semantic segmentation networks can be divided into different functional modules to complete all or part of the functions described above. In addition, the apparatus for generating semantic segmentation networks and the method embodiments for generating semantic segmentation networks provided in the above embodiments belong to the same concept, and the specific implementation process can be found in the method embodiments, which will not be repeated here.

[0136] Please see Figure 8 This application also provides a computer device 800, including: a processor 801 and a memory 802.

[0137] The processor 801 is coupled to the memory 802, which stores at least one computer program instruction. The at least one computer program instruction is loaded and executed by the processor 801 to enable the computer device to implement a method for generating a semantic segmentation network.

[0138] Finally, it should be noted that in this document, relational terms such as "first" and "second" are used only to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.

[0139] The above description of the disclosed embodiments enables those skilled in the art to make or use this application. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of this application. Therefore, this application is not to be limited to the embodiments shown herein, but is to be accorded the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. A method for generating semantic segmentation networks, characterized in that, The method includes: Within a preset downsampling rate range, the downsampling probability of each node is determined based on the preset total number of layers in the network, the layer number of each node in the network, and the downsampling rate corresponding to each node. This determination includes: determining the downsampling probability of each node according to the following formula: in, Let be the downsampling probability of the node. The downsampling factor corresponding to the node. This is the preset total number of layers in the network. This represents the layer number of a node in the network. Based on the genetic algorithm, according to the hierarchical search space and the downsampling probability of each node, the convolution operation, pooling operation, skip connection operation and downsampling operation of each node in the initial network are mutated to obtain multiple candidate networks. The hierarchical search space includes: the set of convolution operation, pooling operation and skip connection operation corresponding to the node. An evaluation score is obtained based on the information entropy of the candidate networks, and the candidate network with the highest evaluation score is selected from multiple candidate networks as the semantic segmentation network, which is used to predict the drivable area of ​​autonomous vehicles. Acquire real-time radar images of the autonomous vehicle; The semantic segmentation network is used to classify the pixels in the real-time radar image to obtain a set of drivable pixels and a set of non-drivable pixels; The drivable area of ​​the autonomous vehicle is determined based on the set of drivable pixels.

2. The method of claim 1, wherein, Before obtaining an evaluation score based on the information entropy of the candidate networks and selecting the candidate network with the highest evaluation score from among the candidate networks as the semantic segmentation network, the process further includes: The parameters of the candidate network are randomly initialized using a Gaussian distribution; The test image with Gaussian noise is input into the candidate network, and the upper bound information entropy of Gaussian output of each node in the candidate network is calculated. The calculation of the Gaussian upper bound information entropy output by each node in the candidate network includes: calculating the Gaussian upper bound information entropy output by each node in the candidate network according to the following formula: wherein, is the upper bound information entropy of the Gaussian output by the node, is the feature map size output by the node; Multiply the Gaussian upper bound information entropy output by each node in the candidate network by the feature map size output by each node in the candidate network to obtain the information entropy corresponding to each node in the candidate network. The information entropy corresponding to each node in the candidate network is summed to obtain the evaluation score of the candidate network.

3. The method of claim 1, wherein, Before obtaining multiple candidate networks by mutating the convolution, pooling, skip connection, and downsampling operations of each node in the initial network based on the genetic algorithm, according to the hierarchical search space and the downsampling probability of each node, the process further includes: Select a first convolution operation and a second convolution operation from the set of convolution operations, and connect the first convolution operation and the second convolution operation. A first pooling operation is selected from the pooling operation set, and a first jump-connection operation is selected from the jump-connection operation set. The first pooling operation and the first jump-connection operation are then connected after the first convolution operation and the second convolution operation to obtain the initial network.

4. The method according to any one of claims 1 to 2, characterized in that, The set of convolution operations includes: 3×3 depthwise separable convolution, 5×5 depthwise separable convolution, 3×3 ordinary convolution, and 5×5 ordinary convolution; The set of pooling operations includes: 3×3 average pooling, 3×3 max pooling, and no pooling. The set of jump connection operations includes: using jump connections and not using jump connections.

5. The method according to any one of claims 1 to 3, characterized in that, The downsampling rate range includes: The range from 8x downsampling to 32x downsampling.

6. An apparatus for generating a semantic segmentation network, the apparatus comprising: The device includes: The determination module is used to determine the downsampling probability of each node within a preset downsampling rate range, based on the preset total number of layers in the network, the layer number of each node in the network, and the downsampling rate corresponding to each node. Determining the downsampling probability of each node based on the preset total number of layers in the network, the layer number of each node in the network, and the downsampling rate corresponding to each node includes: determining the downsampling probability of each node according to the following formula: wherein, is a down-sampling probability of a node, is a down-sampling rate corresponding to a node, is a preset total number of layers of a network, is a layer number in which a node is located in a network; The mutation module is used to mutate the convolution operation, pooling operation, skip connection operation and downsampling operation of each node in the initial network based on the genetic algorithm, according to the hierarchical search space and the downsampling probability of each node, to obtain multiple candidate networks. The hierarchical search space includes: the set of convolution operation, pooling operation and skip connection operation corresponding to the node. The selection module is used to obtain an evaluation score based on the information entropy of the candidate networks, and select the candidate network with the highest evaluation score from among the candidate networks as the semantic segmentation network, which is used to predict the drivable area of ​​the autonomous vehicle. An acquisition module is used to acquire real-time radar images of the autonomous vehicle; The classification module is used to classify the pixels in the real-time radar image using the semantic segmentation network to obtain a set of drivable pixels and a set of non-drivable pixels; The determining module is further configured to determine the drivable area of ​​the autonomous vehicle based on the drivable pixel set.

7. A computer device, comprising: The computer device includes: a processor coupled to a memory, the memory storing at least one computer program instruction, the at least one computer program instruction being loaded and executed by the processor to enable the computer device to implement the method of any one of claims 1-5.

Images