Streetscape image semantic segmentation system for complex city scene

Abstract

The invention relates to the technical field of image segmentation, and discloses a street view image semantic segmentation system for a complex city scene, and the system comprises a double-edge energy ratio calculation module which obtains a double-edge energy ratio through calculation; the priori mask generation module is used for generating a priori mask; the segmentation network construction module is used for outputting logarithmic probabilities and probabilities of various categories through a trunk segmentation network; the total loss construction module is used for constructing total loss; the mutual suppression optimization module is used for calculating to obtain a preliminary semantic segmentation result; and the semantic segmentation output module is used for calculating to obtain a final semantic segmentation result. According to the method, a unified process of double-edge energy ratio driving direction convolution, separation loss and reasoning period mutual suppression is adopted, repeated boundaries and false objects caused by reflection and transmission mixing in a complex city scene containing glass and a water surface are reduced, pixel-level boundary positions are stabilized, and road space, building facades and traffic element division are consistent.

Description

Technical Field

[0001] This invention relates to the field of image segmentation technology, and more specifically, to a semantic segmentation system for street view images in complex urban scenes. Background Technology

[0002] In modern cities, double-layered and thick glass curtain walls, shop windows, bus shelters, and other glass enclosures are widely used. After rain, water accumulation and thin water films on the road surface are also quite common. When a camera photographs these areas, the transmission channel of the real scene and the reflection channel of the environment enter the imaging system simultaneously. Both the front and back surfaces of the glass will produce reflections, and the reflection intensity decreases with the angle of incidence and the material. The specular reflection component of the water surface will increase with the increase of the viewing angle. These conditions allow distant high-contrast structures and nearby objects to be recorded together in the same field of view.

[0003] Under the influence of multiple glass interfaces, the reflection channel will have a slight spatial displacement and energy reduction relative to the transmission channel. This manifests as two almost collinear edge traces with a spacing of only a few pixels along the boundary normal, one of which is stronger and the other weaker. In the water surface and thin water film area, distant structures will be reflected onto the road surface or facade through the mirror, forming traces that are adjacent to or overlap with the real boundary. Due to the influence of sensor resolution and sampling discreteness, these traces will be compressed into a very small spacing range, resulting in the simultaneous inclusion of transmission and reflection components in the neighborhood of a single pixel, forming a pixel-level mixture that is difficult to distinguish directly by color or texture.

[0004] Mainstream street scene semantic segmentation processes are based on pixel cross-entropy. The training data usually does not independently label the reflection and transmission layers, and the evaluation metrics are not sensitive to slight shifts in boundary positions. This makes it easy for the network to simultaneously reinforce two lines or arbitrarily choose one when encountering repeated edge regions, resulting in two types of errors: first, misclassifying the mirror image as a real object, leading to boundary duplication; second, the real boundary shifting towards the mirror image. These errors originate from pixel neighborhood mixing caused by multipath imaging, rather than from single noise. Existing segmentation methods that do not have specific parameters and processing procedures designed for this phenomenon are difficult to achieve stable pixel-level classification in densely populated urban glass and water areas. Summary of the Invention

[0005] This invention provides a semantic segmentation system for street view images in complex urban scenarios, solving the technical problems mentioned in the background.

[0006] This invention provides a semantic segmentation system for street view images in complex urban scenarios, comprising: The dual-edge energy ratio calculation module calculates the gradient vector and normal unit vector based on the street view image, constructs a profile along the normal and detects double maxima with the same sign, obtains the energy of the main and secondary edges and the distance between the two edges, and calculates the dual-edge energy ratio accordingly. The prior mask generation module concatenates the double-edge energy ratio, double-edge spacing, and normal unit vector into a feature tensor, and generates a prior mask based on a preset threshold. The segmentation network construction module establishes an isotropic branch and a one-dimensional directional convolutional branch along the normal direction. It generates gating coefficients based on the double edge energy ratio and the prior mask and integrates them into the two branches to obtain a fused feature map. The street scene image, feature tensor and fused feature map are concatenated and input into the backbone segmentation network, and the log odds and probabilities of each category are output. The total loss construction module calculates the boundary strength based on the maximum probability map of each category, determines the corresponding pixels according to the double edge spacing and normal unit vector, constructs the separation loss only at pixels with a prior mask of 1, and fuses it with the cross-entropy loss to obtain the total loss, which is used to update the backbone segmentation network. The mutual inhibition optimization module optimizes the class log probability of the current pixel and the corresponding pixel based on the boundary strength and double edge energy ratio of the corresponding pixel, and obtains the preliminary semantic segmentation result. The semantic segmentation output module solidifies the preliminary semantic segmentation results into the final semantic segmentation results and outputs them, simultaneously outputting a continuous risk map based on the dual edge energy ratio and a binary high-risk map based on the prior mask.

[0007] Furthermore, the pixel values ​​of the red, green, and blue channels of the street view image are obtained. The pixel values ​​of the three channels are weighted and summed using three non-negative constants that sum to 1 to obtain the brightness image. The first-order partial derivatives of the brightness image in the x and y directions are calculated respectively. The first-order partial derivatives in the two directions are combined to form the gradient vector. The gradient vector is divided by the sum of the magnitude of the gradient vector and a small constant greater than 0 to obtain the normal unit vector. Define a displacement range along the normal unit vector, centered at 0, with the maximum displacement in both positive and negative directions being a sampling radius greater than 0; within this displacement range, calculate the dot product of the brightness image gradient corresponding to each displacement position and the normal unit vector to form a normal profile.

[0008] Furthermore, the normal profile is smoothed by convolution using a one-dimensional Gaussian kernel with a standard deviation greater than 0, resulting in a smoothed normal profile. In the smoothed normal profile, locate two local maxima points with the same sign; calculate the absolute value of the values ​​corresponding to the two local maxima points, take the value with the larger absolute value as the principal edge energy, and the value with the smaller absolute value as the secondary edge energy; calculate the absolute value of the difference in displacement between the two local maxima points, and take it as the double edge spacing. The ratio of secondary edge energy to primary edge energy is multiplied by the attenuation factor to obtain the dual-edge energy ratio. This attenuation factor is constructed based on the dual-edge spacing and a spacing attenuation scale constant greater than 0, and the attenuation factor decreases as the dual-edge spacing increases.

[0009] Furthermore, the two components of the double-edge energy ratio, double-edge spacing, and normal unit vector are concatenated along the channel dimension to obtain a four-channel feature tensor. For each pixel, the double edge energy ratio is judged. If the double edge energy ratio is greater than or equal to the preset threshold, the mask value corresponding to the pixel is 1; otherwise, the mask value corresponding to the pixel is 0, thus obtaining the binarized prior mask.

[0010] Furthermore, a convolution operation is performed on the street view image using a neighborhood of size 3×3 to obtain an isotropic branch output; a one-dimensional convolution kernel of positive odd length is set, and the sampling position is determined along the normal unit vector to perform a one-dimensional convolution operation on the brightness image to obtain an directional convolution branch output. The initial gating coefficients are obtained by multiplying the dual edge energy ratio by the gating slope constant and then processing the product result through a logic function. Subtract the initial gating coefficient from 1, multiply by the corresponding prior mask, and then add the initial gating coefficient to obtain the final gating coefficient.

[0011] Further, subtract the final gating coefficient from 1 and multiply it with the output of the isotropic branch, then multiply the final gating coefficient with the output of the directional convolution branch, and finally add the two product results to obtain the fused feature map. The pixel values, feature tensors, and fused feature maps of the red, green, and blue channels of the street view image are concatenated along the channel dimension to obtain the input tensor of the backbone segmentation network. The input tensor is fed into the backbone segmentation network, which performs feature extraction and mapping on the input tensor and outputs the log odds of each preset category. Normalize the logarithmic odds of all preset categories to obtain the probability of each category.

[0012] Furthermore, for each pixel, the maximum value among all category probabilities is selected to form a probability maximum map; the first-order partial derivatives of the probability maximum map in the x and y directions are calculated, and the square root of the sum of the squares of the partial derivatives in the two directions is taken to obtain the boundary strength of each pixel; Using the current pixel as the reference point, move the pixel distance corresponding to the double edge spacing along the direction indicated by the normal unit vector to determine the corresponding pixel for each pixel.

[0013] Furthermore, for each pixel, determine whether the value of the prior mask is 1; if it is 1, calculate the product of the double edge energy ratio, the self-boundary strength, and the boundary strength of the corresponding pixel; if it is 0, set the product result of the pixel to 0; sum the product results of all pixels and divide by the total number of pixels to obtain the separation loss. Obtain the labeled category for each pixel, calculate the negative logarithm of the probability of each pixel corresponding to the labeled category, sum the negative logarithm results of all pixels and divide by the total number of pixels to obtain the cross-entropy loss; Multiply the weighting coefficients by the separation loss, and then add the product to the cross-entropy loss to obtain the total loss.

[0014] Furthermore, the suppression coefficient, the double edge energy ratio of the current pixel, and the boundary strength of the corresponding pixel are multiplied together to obtain the first mutual suppression weight; The second mutual inhibition weight is obtained by multiplying the inhibition coefficient, the double edge energy ratio of the current pixel, and the boundary strength of the current pixel. Subtract the first mutual inhibition weight from the original class log odds of the current pixel to obtain the updated class log odds of the current pixel; subtract the second mutual inhibition weight from the original class log odds of the corresponding pixel to obtain the updated class log odds of the corresponding pixel. The updated log probabilities of each class for all pixels are normalized exponentially to obtain the updated probabilities of each class for each pixel. For each pixel, the category with the highest probability value after the update is selected as the initial segmentation category for that pixel. The initial segmentation categories of all pixels constitute the initial semantic segmentation result.

[0015] Furthermore, the preliminary semantic segmentation result is determined as the final semantic segmentation result; the pixel-level numerical field corresponding to the double edge energy ratio is used as the continuous risk map; and the prior mask is directly used as the binary high-risk map.

[0016] The beneficial effects of this invention are as follows: This invention uses a unified process of dual-edge energy ratio-driven directional convolution, separation loss, and inference period mutual inhibition to reduce the repetitive boundaries and false objects caused by the mixing of reflection and transmission in complex urban scenes containing glass and water surfaces, stabilize pixel-level boundary positions, and ensure consistent division of road space, building facades, and traffic elements; especially in dense areas such as glass curtain walls, shop windows, and water accumulation, it maintains stable segmentation performance, ensures the accuracy of traversable area extraction and object contour description, provides high-quality image segmentation support for subsequent related applications, and thus meets the stringent requirements of street scene semantic segmentation in real-world scenarios; at the same time, it outputs continuous risk maps and binary high-risk maps, which facilitates the location of potential error areas. Attached Figure Description

[0017] Figure 1 This is a schematic diagram of a street scene image semantic segmentation system for complex urban scenarios according to the present invention; Figure 2 This is a flowchart of a street scene image semantic segmentation method for complex urban scenarios according to the present invention.

[0018] In the figure: Dual edge energy ratio calculation module 101, prior mask generation module 102, segmentation network construction module 103, total loss construction module 104, mutual inhibition optimization module 105, semantic segmentation output module 106. Detailed Implementation

[0019] The subject matter described herein will now be discussed with reference to exemplary embodiments. It should be understood that these embodiments are discussed only to enable those skilled in the art to better understand and implement the subject matter described herein, and changes may be made to the function and arrangement of the elements discussed without departing from the scope of this specification. Various processes or components may be omitted, substituted, or added as needed in the examples. Furthermore, features described in some examples may be combined in other examples.

[0020] It should be noted that, unless otherwise defined, the technical or scientific terms used in one or more embodiments of the present invention should have the ordinary meaning understood by one of ordinary skill in the art to which this invention pertains. The terms "first," "second," and similar terms used in one or more embodiments of the present invention do not indicate any order, quantity, or importance, but are merely used to distinguish different components. Terms such as "comprising" or "including" indicate that the element or object preceding the term encompasses the elements or objects listed following the term and their equivalents, without excluding other elements or objects. Terms such as "connected" or "linked" are not limited to physical or mechanical connections, but can include electrical connections, whether direct or indirect. Terms such as "upper," "lower," "left," and "right" are used only to indicate relative positional relationships; when the absolute position of the described object changes, the relative positional relationship may also change accordingly.

[0021] like Figures 1-2 As shown, a semantic segmentation system for street view images in complex urban scenes includes: The dual-edge energy ratio calculation module 101 is used to calculate the gradient vector and normal unit vector based on the street view image, construct a profile along the normal and detect double maxima with the same sign, obtain the energy of the main and secondary edges and the distance between the two edges, and calculate the dual-edge energy ratio accordingly. The prior mask generation module 102 is used to concatenate the double edge energy ratio, double edge spacing and normal unit vector into a feature tensor, and generate a prior mask according to a preset threshold. The segmentation network construction module 103 is used to establish isotropic branches and one-dimensional directional convolutional branches along the normal direction. It generates gating coefficients based on the double edge energy ratio and prior mask and integrates them into the two branches to obtain a fused feature map. It concatenates the street scene image, feature tensor and fused feature map into the backbone segmentation network and outputs the log odds and probabilities of each category. The total loss construction module 104 is used to calculate the boundary strength based on the maximum probability map of each class, determine the corresponding pixels according to the double edge spacing and normal unit vector, construct the separation loss only at pixels with a prior mask of 1, and fuse it with the cross-entropy loss to obtain the total loss, which is used to update the backbone segmentation network. The mutual inhibition optimization module 105 is used to perform mutual inhibition optimization on the class log probability of the current pixel and the corresponding pixel based on the boundary strength and double edge energy ratio of the corresponding pixel, so as to obtain the preliminary semantic segmentation result. The semantic segmentation output module 106 is used to solidify the preliminary semantic segmentation result into the final semantic segmentation result and output it, and simultaneously outputs a continuous risk map based on the dual edge energy ratio and a binary high-risk map based on the prior mask.

[0022] In one embodiment of the present invention, the pixel values ​​of the red, green and blue channels of a street view image are obtained. The pixel values ​​of the three channels are weighted and summed using three non-negative constants that sum to 1 to obtain a brightness image. The first-order partial derivatives of the brightness image in the x and y directions are calculated respectively. The first-order partial derivatives in the two directions are combined to form a gradient vector. The gradient vector is divided by the sum of the magnitude of the gradient vector and a small constant greater than 0 to obtain a normal unit vector. Define a displacement range along the normal unit vector, centered at 0, with the maximum displacement in both positive and negative directions being a sampling radius greater than 0; within this displacement range, calculate the dot product of the brightness image gradient corresponding to each displacement position and the normal unit vector to form a normal profile.

[0023] It should be noted that the pixel values ​​of the red, green, and blue channels represent the values ​​of the red, green, and blue light components corresponding to each pixel in the street view color image, respectively, with values ​​ranging from 0 to 255. The non-negative weights of the pixel values ​​of the red, green, and blue channels are 0.2126, 0.7152, and 0.0722, respectively. In the single-channel image derived from the color street view image, the value of each pixel reflects the light intensity (brightness) at that location, which is used for subsequent gradient and edge analysis to eliminate color interference. The first-order partial derivatives in the x (horizontal axis) and y (vertical axis) directions of the brightness image are calculated to reflect the brightness gradient intensity in the horizontal and vertical directions, which can be calculated using the Sobel operator and the Prewitt operator. The values ​​are calculated as follows: Specifically, taking the Sobel operator as an example, the first-order partial derivative of the brightness image in the x direction is calculated as follows: (brightness value of the pixel to the right of the pixel × 1 + brightness value of the pixel to the lower right of the pixel × 2 + brightness value of the pixel to the upper right of the pixel × 1) - (brightness value of the pixel to the left of the pixel × 1 + brightness value of the pixel to the lower left of the pixel × 2 + brightness value of the pixel to the upper left of the pixel × 1); the first-order partial derivative of the brightness image in the y direction is calculated as follows: (brightness value of the pixel below the pixel × 1 + brightness value of the pixel to the lower right of the pixel × 2 + brightness value of the pixel to the lower left of the pixel × 1) - (brightness value of the pixel above the pixel × 1 + brightness value of the pixel to the upper right of the pixel × 2 + brightness value of the pixel to the upper left of the pixel × 1).

[0024] It should be noted that the gradient vector is composed of the partial derivatives in the horizontal and vertical directions, and its direction points to the direction of the fastest increase in brightness; the small constant is set to 0.0001 to prevent the denominator from being 0; the sampling radius represents the sampling range when constructing the profile along the normal unit vector, that is, the number of pixels extending from the current pixel in both the positive and negative directions of the normal, usually taken as 5 to 10 pixels. If the street scene image has a high resolution (such as 4K), it can be finely adjusted according to the resolution ratio. For example, when the resolution is doubled, the sampling radius is taken as 10 to 20 pixels; the normal profile represents a one-dimensional data sequence along the direction of the normal unit vector. Each data point is the dot product of the gradient vector at the corresponding displacement position and the normal unit vector, reflecting the brightness change trend along the edge normal.

[0025] Specifically, normal section The calculation formula is as follows: ,in Let represent the normal unit vector, and s represent the displacement along the normal unit vector. This represents the sampling radius that is greater than 0. This represents the pixel position of pixel x after being shifted by a unit vector s along the normal direction. It represents the first-order partial derivative.

[0026] In one embodiment of the present invention, a smoothed normal profile is obtained by convolution smoothing the normal profile with a one-dimensional Gaussian kernel with a standard deviation greater than 0. In the smoothed normal profile, locate two local maxima points with the same sign; calculate the absolute value of the values ​​corresponding to the two local maxima points, take the value with the larger absolute value as the principal edge energy, and the value with the smaller absolute value as the secondary edge energy; calculate the absolute value of the difference in displacement between the two local maxima points, and take it as the double edge spacing. The ratio of secondary edge energy to primary edge energy is multiplied by the attenuation factor to obtain the dual-edge energy ratio. This attenuation factor is constructed based on the dual-edge spacing and a spacing attenuation scale constant greater than 0, and the attenuation factor decreases as the dual-edge spacing increases.

[0027] It should be noted that the one-dimensional Gaussian kernel is used to suppress normal profile noise. The values ​​within the kernel conform to a Gaussian distribution, with the central value being large and gradually decreasing towards both ends. The standard deviation determines the smoothness. The length of the one-dimensional Gaussian kernel is an odd number that is twice an integer plus 1 to ensure central symmetry, usually 5 or 7. The kernel length must match the standard deviation of the Gaussian kernel. For example, when the standard deviation is 0.5, the kernel length is 5, and when the standard deviation is 1.0, the kernel length is 7. This avoids the kernel being too long, which would lead to over-smoothing, or too short, which would lead to insufficient noise suppression.

[0028] It should be noted that the value at a certain position in the smoothed normal profile is equal to the original values ​​of the normal profile at that position and its adjacent positions multiplied by the values ​​at the corresponding positions of the one-dimensional Gaussian kernel, and then all products are added together, i.e., convolution operation; local maxima represent points in the smoothed normal profile where the value is greater than the values ​​at their left and right adjacent positions, and the two maxima have the same sign, i.e., both are positive or both are negative; the principal edge energy reflects the gradient strength of the principal edge, and the secondary edge energy reflects the gradient strength of the secondary edge; the displacement corresponding to the local maximum point represents the displacement along the normal unit vector corresponding to the two local maxima points with the same sign in the smoothed normal profile; the double edge spacing table The pixel distance between the main edge and the secondary edge along the normal unit vector direction reflects the proximity of the two edges. In multipath glass scenes, this is usually sub-pixel to several pixels. The spacing attenuation scale constant controls the rate attenuation factor changes with the spacing between the two edges. The larger the value, the slower the rate attenuation factor decreases as the spacing increases. It is usually set to 2 to 5 pixels. If the glass thickness in the street scene is large (the spacing between the two edges may reach 8 pixels), it can be adjusted to 5 to 8 pixels to ensure that when the spacing between the two edges exceeds twice the constant, the attenuation factor is less than or equal to 0.1 (i.e., the spacing penalty is significant). The larger the value of the energy ratio between the two edges, the higher the intensity of the secondary edge and the higher the risk of it being close to the main edge.

[0029] Specifically, the energy ratio of the two edges The calculation formula is as follows: ,in Indicates the energy of the main edge. Indicates the secondary side energy. Indicates the spacing between the two edges. represents a spacing decay scale constant greater than 0, and exp represents an exponential function with the natural constant as the base.

[0030] In one embodiment of the present invention, the two components of the double edge energy ratio, double edge spacing, and normal unit vector are concatenated in the channel dimension to obtain a four-channel feature tensor. For each pixel, the double edge energy ratio is judged. If the double edge energy ratio is greater than or equal to the preset threshold, the mask value corresponding to the pixel is 1; otherwise, the mask value corresponding to the pixel is 0, thus obtaining the binarized prior mask.

[0031] It should be noted that the adjustment of the preset threshold should be based on the scene characteristics of the street view image. If there are many multi-path interference targets such as glass / water surfaces in the scene (e.g., commercial street view), the threshold can be appropriately reduced, for example, from 0.3 to 0.35, to expand the coverage of high-risk areas and avoid missed detections. If the scene is mainly composed of non-interfering targets such as building walls and road surfaces (e.g., suburban street view), the threshold can be appropriately increased, for example, from 0.45 to 0.5, to shrink the high-risk area and reduce the impact of misjudgments on regular segmentation. After constructing the feature tensor, numerical normalization processing is required. After normalization, all channel values ​​are in the range of 0 to 1 to avoid the network over-focusing on a certain type of feature due to differences in numerical range. The prior mask represents a binary image that may contain main / secondary shadow edge interference in the labeled street view image. The spatial resolution is consistent with the street view image, and the pixel value is only 0 or 1, where 1 represents a high-risk area and 0 represents a low-risk area.

[0032] In one embodiment of the present invention, a street view image is convolved with a neighborhood of size 3×3 to obtain an isotropic branch output; a one-dimensional convolution kernel of positive odd length is set, and the sampling position is determined along the normal unit vector to perform a one-dimensional convolution operation on the brightness image to obtain an oriented convolution branch output. The initial gating coefficients are obtained by multiplying the dual edge energy ratio by the gating slope constant and then processing the product result through a logic function. Subtract the initial gating coefficient from 1, multiply by the corresponding prior mask, and then add the initial gating coefficient to obtain the final gating coefficient.

[0033] It should be noted that when performing isotropic convolution on the street view image, a local region consisting of 3 rows and 3 columns of pixels around each pixel is selected to uniformly extract features in all directions of the pixel, avoiding directional bias. The output of the isotropic branch represents the feature map obtained after processing the street view image using 3×3 two-dimensional convolution. The spatial resolution is consistent with the street view image. The value of each pixel in the isotropic branch output is equal to the street view image pixel values ​​of all pixels in the 3×3 neighborhood of that pixel, multiplied by the weights of the corresponding positions of the 3×3 learnable convolution kernel, and then all the product results are summed. A one-dimensional convolution with a positive odd length... The kernel is used to extract directional features along the normal unit vector. Its length is a positive odd number to ensure central symmetry. The weights within the kernel are learnable parameters. The output of the directional convolution branch represents the feature map obtained after performing a one-dimensional convolution on the brightness image along the normal unit vector. Its spatial resolution is consistent with that of the brightness image. It is used to enhance the geometric distinction between the main and secondary edges of high-risk areas. The value of each pixel in the output of the directional convolution branch is equal to the brightness image pixel value corresponding to the sampling position (the number of which is equal to the length of the one-dimensional convolution kernel) determined along the normal unit vector. These values ​​are then multiplied by the weights at the corresponding positions of the one-dimensional convolution kernel, and all the product results are summed.

[0034] It should be noted that the sampling position is determined along the normal unit vector with the current pixel coordinates as the center. Based on the one-dimensional convolution kernel of positive odd length (let the length be K), the K positions to be sampled are calculated. The coordinates of the sampling position = the current pixel coordinates + integer k × the x / y components of the normal unit vector. The value of k ranges from -(K-1) / 2 to +(K-1) / 2. For example, when K=3, k=-1, 0, 1. If the calculated coordinates are not integers (sub-pixel positions), the brightness image pixel value of that position is obtained by bilinear interpolation, that is, by calculating the brightness values ​​of the four integer coordinate pixels around the sub-pixel according to the distance weight.

[0035] It should be noted that the gating slope constant represents a scalar constant that adjusts the sensitivity of the double-edge energy ratio to the initial gating coefficient. A value greater than 0 controls the rate of change of the logic function output. The gating slope constant needs to be adjusted according to the distribution of the double-edge energy ratio. If the double-edge energy ratio in the street view image is mostly concentrated between 0.2 and 0.8 (medium risk), the gating slope constant should be set to 3 to 4 to enhance the sensitivity of the gating coefficient to changes in risk. If the double-edge energy ratio is mostly concentrated between 0 and 0.3 (low risk) or 0.7 and 1 (high risk), the gating slope constant should be... The gating coefficient is set to 5 to increase its discriminative power (closer to 0 for low risk, closer to 1 for high risk); the logistic function is the Sigmoid activation function, which maps the product of the double edge energy ratio and the gating slope constant to the interval between 0 and 1; the initial gating coefficient represents the continuous weight reflecting the degree of preference of the double edge energy ratio (risk intensity) for the directional convolution branch, with a value between 0 and 1, and the larger the value, the more inclined to select the directional branch; the final gating coefficient represents the gating weight after fusing the prior mask (high-risk area localization), with a value between 0 and 1.

[0036] Specifically, the final gating coefficient The calculation formula is as follows: ,in Represents the initial gating coefficient. This indicates the prior mask.

[0037] In one embodiment of the present invention, the final gating coefficient is subtracted from 1 and multiplied by the output of the isotropic branch, and then the final gating coefficient is multiplied by the output of the directional convolution branch. Finally, the two product results are added together to obtain the fused feature map. The pixel values, feature tensors, and fused feature maps of the red, green, and blue channels of the street view image are concatenated along the channel dimension to obtain the input tensor of the backbone segmentation network. The input tensor is fed into the backbone segmentation network, which performs feature extraction and mapping on the input tensor and outputs the log odds of each preset category. Normalize the logarithmic odds of all preset categories to obtain the probability of each category.

[0038] It should be noted that the fused feature map represents the feature map output by the dual-branch output through the final weighted fusion of gating coefficients, and the spatial resolution is consistent with the dual-branch output; the input tensor of the backbone segmentation network integrates the original channels of the street view image, the feature tensor, and the fused feature map, and the number of channels after concatenation is 8; the backbone segmentation network is a neural network structure that realizes the core function of semantic segmentation, including feature extraction (downsampling) and class mapping (decoding) modules, which map the input tensor to the log odds of each class; the normalization of the log odds means converting the log odds of each class into a probability distribution between 0 and 1 with a sum of 1, such as the Softmax activation function.

[0039] Specifically, feature map fusion The calculation formula is as follows: in Represents the initial gating coefficient. This indicates isotropic branch output. This indicates the output of the directional convolution branch.

[0040] It should be noted that the backbone segmentation network adopts an encoder-decoder architecture, such as U-Net or ResUNet, which includes convolutional layers, pooling layers, upsampling layers, and skip connections. In the feature extraction stage, 3×3 convolutional layers and max-pooling layers (stride 2) are used alternately, doubling the number of channels with each downsampling, for a total of 3 to 4 downsampling operations to obtain multi-scale features. In the class mapping stage, transposed convolutional layers (stride 2) are used for upsampling. After each upsampling, the feature map is concatenated with the feature map from the corresponding downsampling stage (skip connection). Finally, a 1×1 convolutional layer maps the number of channels to the preset number of classes, outputting the log-probability of each class. The skip connections in the backbone segmentation network use a channel concatenation method, i.e., the features from the upsampling stage... The image (number of channels C) and the corresponding feature map (number of channels 2C) from the downsampling stage are concatenated along the channel dimension, resulting in 3C channels after concatenation. Before concatenation, a 1×1 convolution is performed on the downsampling feature map (reducing the number of channels from 2C to C) to ensure that the number of channels is consistent. Then, spatial alignment is performed. If the resolutions are different, the downsampling feature map is cropped. In addition, during the training of the backbone segmentation network, the optimizer Adam is used, with an initial learning rate of 0.001, which decays to 0.5 every 10 epochs. The batch size is set to 8 to 16, and the number of training epochs is set to 50 to 100. An early stopping strategy is adopted, i.e., training is stopped if the validation set loss does not decrease for 5 consecutive epochs to avoid overfitting.

[0041] In one embodiment of the present invention, for each pixel, the maximum value among all category probabilities is selected to form a probability maximum map; the first-order partial derivatives of the probability maximum map in the x and y directions are calculated, and the square root of the sum of the squares of the partial derivatives in the two directions is taken to obtain the boundary strength of each pixel; Using the current pixel as the reference point, move the pixel distance corresponding to the double edge spacing along the direction indicated by the normal unit vector to determine the corresponding pixel for each pixel.

[0042] It should be noted that the probability maximum image is a single-channel image, where each pixel value is the maximum probability among all preset categories, reflecting the confidence level of the pixel's most likely category. The spatial resolution is consistent with the street view image. The calculation of the first-order partial derivatives of the probability maximum image in the x and y directions is the same as the calculation of the first-order partial derivatives of the brightness image in the x and y directions, and will not be elaborated here. Boundary strength represents a non-negative value indicating the clarity of the category boundary at a quantized pixel; a larger value indicates that the pixel is closer to the category boundary. The spatial resolution is related to the probability maximum. Figure 1 The corresponding pixel represents the pixel that is paired with the current pixel along the normal unit vector direction. It corresponds to the main edge / secondary edge pairing relationship in the double edge. That is, the coordinate of the corresponding pixel = the x coordinate of the current pixel + the double edge spacing × the x component of the normal unit vector, and the y coordinate of the current pixel + the double edge spacing × the y component of the normal unit vector.

[0043] Specifically, boundary strength The calculation formula is as follows: ,in Represents the spatial gradient operator. Describing the L2 norm, and Let represent the first-order partial derivatives of the probability maximum graph in the x and y directions, respectively.

[0044] In one embodiment of the present invention, for each pixel, it is determined whether the value of the prior mask is 1; if it is 1, the product of the double edge energy ratio, the self-boundary strength and the boundary strength of the corresponding pixel is calculated; if it is 0, the product result corresponding to the pixel is set to 0; the product results of all pixels are summed and then divided by the total number of pixels to obtain the separation loss. Obtain the labeled category for each pixel, calculate the negative logarithm of the probability of each pixel corresponding to the labeled category, sum the negative logarithm results of all pixels and divide by the total number of pixels to obtain the cross-entropy loss; Multiply the weighting coefficients by the separation loss, and then add the product to the cross-entropy loss to obtain the total loss.

[0045] It should be noted that the total number of pixels refers to the total number of pixels contained in the street view image; the separation loss represents the loss term for targeted suppression of high-risk areas, and its value is non-negative, with a larger value indicating a more severe error pattern; the labeled category is the discrete category value, which is a system preset parameter; the cross-entropy loss represents the basic loss term that measures the difference between the pixel's predicted category and the labeled category, and its value is non-negative, with a larger value indicating a more severe classification error; the weight coefficient represents a positive scalar constant that adjusts the contribution of the separation loss to the total loss, used to balance the priority of the basic classification loss and the double-edge suppression loss; the total loss represents the optimization objective of the backbone segmentation network, comprehensively reflecting the classification error and the degree of double-edge interference, with a smaller value indicating better network performance.

[0046] Specifically, total loss The calculation formula is as follows: ,in Indicates the total number of pixels. This represents the probability that pixel x belongs to the labeled category y. The weighting coefficients represent the separation loss. Indicates the prior mask, Indicates the energy ratio at both edges. Indicates boundary strength. This indicates the boundary strength of the corresponding pixel.

[0047] It should be noted that when the calculated corresponding pixel coordinates exceed the image boundary, the corresponding pixel is set as the pixel closest to the image edge. Simultaneously, the product of the double-edge energy ratio, its own boundary strength, and the boundary strength of the corresponding pixel is set to 0 to avoid invalid pixels outside the boundary affecting the loss calculation. The labeling category must use a pixel-level discrete labeling format, and the labeling category of each pixel must be completely consistent with the system's preset category set, such as glass, building, road surface, pedestrian, etc. The initial value of the weight coefficient is 0.2, and it is subsequently adjusted based on the validation set performance. If the segmentation error of the double-edge interference region on the validation set is still large, for example, the IoU (Intersection over Union) of the glass region is less than 0.6, the weight coefficient can be increased to 0.3. If the segmentation accuracy of the basic category (e.g., road surface) decreases, for example, the IoU decreases by more than 5%, the weight coefficient can be reduced to 0.1. After adjustment, the network needs to be retrained until convergence.

[0048] In one embodiment of the present invention, the suppression coefficient, the double edge energy ratio of the current pixel, and the boundary strength of the corresponding pixel are multiplied together to obtain the first mutual suppression weight. The second mutual inhibition weight is obtained by multiplying the inhibition coefficient, the double edge energy ratio of the current pixel, and the boundary strength of the current pixel. Subtract the first mutual inhibition weight from the original class log odds of the current pixel to obtain the updated class log odds of the current pixel; subtract the second mutual inhibition weight from the original class log odds of the corresponding pixel to obtain the updated class log odds of the corresponding pixel. The updated log probabilities of each class for all pixels are normalized exponentially to obtain the updated probabilities of each class for each pixel. For each pixel, the category with the highest probability value after the update is selected as the initial segmentation category for that pixel. The initial segmentation categories of all pixels constitute the initial semantic segmentation result.

[0049] It should be noted that the suppression coefficient represents a positive scalar constant that controls the strength of log-odds mutual suppression. This prevents excessive suppression from causing class confusion or insufficient suppression of secondary edges. The initial value of the suppression coefficient is 0.7, and it is subsequently adjusted based on the segmentation performance in the double-edge region of the validation set. If the segmentation IoU of the glass / water surface region (double-edge interference area) in the validation set is below 0.65, the suppression coefficient can be increased to 0.9; if the segmentation IoU of the regular region (such as the road surface) decreases by more than 3% due to suppression, the suppression coefficient can be reduced to 0.5. After adjustment, inference verification needs to be performed again to ensure optimal overall segmentation accuracy. The first mutual suppression weight represents the weight value used to subtract the log-odds of the current pixel class. The larger the value, the stronger the suppression of the current pixel's log-odds, and it is positively correlated with double-edge risk and alignment boundary strength. The second mutual suppression weight represents the weight value used to subtract the log-odds of the corresponding pixel class. The larger the value, the stronger the suppression of the corresponding pixel's log-odds, and it is positively correlated with double-edge risk and current pixel boundary strength.

[0050] It should be noted that the original class log odds represent the real numbers corresponding to each class of the pixel output by the backbone segmentation network before the mutual suppression processing, reflecting the network's original confidence that the pixel belongs to each class; the updated class log odds represent the real numbers corresponding to each class of the pixel after the mutual suppression weights are subtracted, weakening the confidence that the pixel is judged as a strong boundary; the exponential normalization process means converting the updated class log odds (any real number) into a probability distribution in the interval of 0 to 1, where the sum of the probabilities of all classes is 1; the preliminary semantic segmentation result represents a pixel-level temporary class label map with a spatial resolution consistent with the street view image, reflecting the preliminary segmentation effect after mutual suppression optimization.

[0051] Specifically, the updated probability of category c at pixel x. The calculation formula is as follows: ,in This represents a predefined set of categories for semantic segmentation, such as glass, buildings, roads, and pedestrians. and Representing the category c and at pixel x respectively The log-probability after mutual inhibition optimization, exp represents the exponential function with the natural constant as the base.

[0052] In one embodiment of the present invention, the preliminary semantic segmentation result is determined as the final semantic segmentation result; the pixel-level numerical field corresponding to the double edge energy ratio is used as a continuous risk map; and the prior mask is directly used as a binary high-risk map.

[0053] In one embodiment of the present invention, at the pixel where the prior mask is 1, the corresponding pixel can be determined along the normal unit vector according to the double edge spacing. If the preliminary semantic segmentation result is inconsistent with the preliminary semantic segmentation result of the corresponding pixel, the two locations are simultaneously assigned the category with the larger log probability according to the optimized log probabilities of each category, thus obtaining the alignment consistency result. At the boundary of this result, the boundary strength of adjacent positions is compared with the log probabilities of each category along the normal unit vector, and the current category is replaced by the position with the larger value, thus obtaining the boundary fitting result. In the region where the prior mask is 0, eight-neighbor majority voting is performed on isolated pixels and hole filling is performed to obtain the corrected semantic segmentation result. The corrected semantic segmentation result is solidified into the final semantic segmentation result and output. The pixel-level numerical field corresponding to the double edge energy ratio is used as a continuous risk map. The prior mask is directly used as a binary high-risk map.

[0054] It should be noted that the alignment consistency result represents a temporary segmentation result in high-risk areas (prior mask is 1) that unifies the category of the current pixel with that of the aligned pixel; the boundary fitting result represents a temporary segmentation result after optimization of the boundary region (pixels with different neighboring categories in the alignment consistency result), making the boundary fitting of the region category clearer and improving the boundary segmentation accuracy; the majority vote represents the operation of correcting the category of isolated pixels by counting the occurrence of each category in the eight neighbors of the isolated pixel in non-high-risk areas (prior mask is 0), and taking the category with the most occurrences. If there are multiple categories with the same number of occurrences, the category with the higher logarithmic probability after optimization is taken. A hole represents a small, outlier region in a non-high-risk area that is completely surrounded by a certain category and has an area smaller than a preset threshold. For example, a small patch on the road surface that is misidentified as a building is considered segmentation noise. To determine whether a region is a hole, a hole area threshold needs to be set. Areas smaller than this threshold are considered holes, while areas larger are considered valid regions. For example, the hole area threshold = input image width × input image height × 0.005% (e.g., for a 512×512 image: 512×512×0.005%≈13 pixels). If there are small valid targets in the scene (e.g., traffic signs), the threshold needs to be reduced to 0.001% (approximately 2 pixels) to avoid misidentifying valid targets as holes. Hole filling involves correcting the hole regions in non-high-risk areas to the dominant category surrounding the hole, eliminating small noise regions. The final semantic segmentation result is a pixel-level category label map after alignment, boundary fitting, majority voting, and hole filling, with spatial resolution consistent with the street view image.

[0055] It should be noted that during boundary fitting, the fitting score is used to measure whether adjacent positions along the normal direction are suitable as a boundary category. The larger the value, the more suitable it is as the correction category for the current boundary pixel. Specifically, 1. First, determine the two adjacent positions of the boundary pixel along the normal unit vector (step size is 1 pixel, positive direction is the direction of the normal vector, negative direction is the opposite); for each adjacent position, calculate the tradeoff coefficient (usually 0.5) × the boundary strength of that position + (1 - tradeoff coefficient) × the maximum log probability of each category after optimization at that position to obtain the fitting score for that position; 2. Majority voting counting rule: for isolated pixels in non-high-risk areas, count the number of pixels in each preset category within its eight neighborhoods. If the neighboring pixels exceed the image boundary, they are not counted; if a certain category has the most and is unique, that category is selected; if there are two or more categories with the same and most, count them. Calculate the average log-odds of these categories after optimization, and take the category with the largest average; 3. Connected component analysis of holes: For the segmentation results of non-high-risk areas, generate binary maps for each category by grouping them; For the complement of each binary map (i.e., the region that is not in that category), use a connected component labeling algorithm (e.g., 8-connected labeling) to label all connected regions; For each connected region, determine whether it is completely surrounded by the binary map. All pixels outside the region belong to the category of the binary map, and the region area (number of pixels) is less than the hole area threshold. If these conditions are met, it is determined to be a hole; The boundary fitting operation iterates 2 times by default. Each iteration performs a fitting update on all boundary pixels; If after a certain iteration, the category of all boundary pixels is completely consistent with that before the iteration (no change), the iteration is stopped early, and no further iterations are needed; If there is still a change after 2 iterations, the iteration is also stopped.

[0056] It should be noted that the tradeoff coefficient is set to 0.5 by default, balancing boundary strength (reflecting geometric boundary clarity) and log odds (reflecting class confidence). If the street view image boundary is blurry (e.g., in foggy weather), the tradeoff coefficient can be increased to 0.7, relying more on log odds. If the image boundary is clear but the class is confused (e.g., glass and buildings), it can be reduced to 0.3, relying more on boundary strength. Boundary pixels are identified by the difference in class among their eight neighbors. That is, for any pixel, if at least one pixel in its eight neighbors has a different class than its own (based on the alignment consistency result), it is determined to be a boundary pixel. If all neighboring pixels have the same class as itself, it is a non-boundary pixel and boundary fitting is not performed.

[0057] It should be noted that the continuous risk map reflects the strength of double-edge interference at each location with pixel-level values, helping to judge the reliability of the segmentation results and identify areas that may be misclassified due to multipath interference, providing guidance for subsequent annotation correction and result verification; the binary high-risk map quickly locates high-interference areas through clear region division, making it easy to quickly avoid risk areas or take targeted processing strategies in engineering applications; the combination of the two presents a fine distribution of risks and provides simple region identification, meeting the needs of different use cases, which will not be elaborated here.

[0058] In one embodiment of the present invention, such as Figure 2 As shown, a semantic segmentation method for street view images in complex urban scenes includes the following steps: Step S201: Calculate the gradient vector and normal unit vector based on the street view image, construct a profile along the normal and detect double maxima with the same sign, obtain the main and secondary edge energy and the double edge spacing, and calculate the double edge energy ratio accordingly. Step S202: Concatenate the double edge energy ratio, double edge spacing and normal unit vector into a feature tensor, and generate a priori mask according to a preset threshold. Step S203: Establish isotropic branches and one-dimensional directional convolutional branches along the normal direction. Generate gating coefficients based on the double edge energy ratio and prior mask and integrate them into the two branches to obtain a fused feature map. Concatenate the street scene image, feature tensor and fused feature map and input them into the backbone segmentation network to output the log odds and probabilities of each category. Step S204: Calculate the boundary strength based on the maximum probability map of each category, determine the corresponding pixels according to the double edge spacing and normal unit vector, construct the separation loss only at pixels with a prior mask of 1, and fuse it with the cross-entropy loss to obtain the total loss, which is used to update the backbone segmentation network. Step S205: Based on the boundary strength and double edge energy ratio of the corresponding pixel, perform cross-inhibition optimization on the class log probability of the current pixel and the corresponding pixel to obtain the preliminary semantic segmentation result. Step S206: Solidify the preliminary semantic segmentation result into the final semantic segmentation result and output it, and simultaneously output the continuous risk map based on the dual edge energy ratio and the binary high-risk map based on the prior mask.

[0059] It should be noted that the interval and threshold sizes are set for ease of comparison. The size of the threshold depends on the amount of sample data and the base number set by those skilled in the art for each set of sample data, as long as it does not affect the proportional relationship between the parameter and the quantized value. Furthermore, the above formulas are all dimensionless calculations, and the formulas are derived from software simulations using a large amount of collected data to obtain the most recent real-world results. The preset parameters in the formulas are set by those skilled in the art according to the actual situation.

[0060] The embodiments of this example have been described above. However, this example is not limited to the specific implementation methods described above. The specific implementation methods described above are merely illustrative and not restrictive. Those skilled in the art can make many other forms based on the guidance of this example, and all of them are within the protection scope of this example.

Claims

1. A semantic segmentation system for street view images in complex urban scenarios, characterized in that, include: The dual-edge energy ratio calculation module calculates the gradient vector and normal unit vector based on the street view image, constructs a profile along the normal and detects double maxima with the same sign, obtains the energy of the main and secondary edges and the distance between the two edges, and calculates the dual-edge energy ratio accordingly. The prior mask generation module concatenates the double-edge energy ratio, double-edge spacing, and normal unit vector into a feature tensor, and generates a prior mask based on a preset threshold. The segmentation network construction module establishes an isotropic branch and a one-dimensional directional convolutional branch along the normal direction. It generates gating coefficients based on the double edge energy ratio and the prior mask and integrates them into the two branches to obtain a fused feature map. The street scene image, feature tensor and fused feature map are concatenated and input into the backbone segmentation network, and the log odds and probabilities of each category are output. The total loss construction module calculates the boundary strength based on the maximum probability map of each category, determines the corresponding pixels according to the double edge spacing and normal unit vector, constructs the separation loss only at pixels with a prior mask of 1, and fuses it with the cross-entropy loss to obtain the total loss, which is used to update the backbone segmentation network. The mutual inhibition optimization module optimizes the class log probability of the current pixel and the corresponding pixel based on the boundary strength and double edge energy ratio of the corresponding pixel, and obtains the preliminary semantic segmentation result. The semantic segmentation output module solidifies the preliminary semantic segmentation results into the final semantic segmentation results and outputs them, simultaneously outputting a continuous risk map based on the dual edge energy ratio and a binary high-risk map based on the prior mask.

2. The street view image semantic segmentation system for complex urban scenes according to claim 1, characterized in that, Obtain the pixel values ​​of the red, green, and blue channels of the street view image. Use three non-negative constants that sum to 1 to perform a weighted sum of the pixel values ​​of the three channels to obtain the brightness image. Calculate the first-order partial derivatives of the brightness image in the x and y directions respectively. Combine the first-order partial derivatives in the two directions to form a gradient vector. Divide the gradient vector by the sum of the magnitude of the gradient vector and a small constant greater than 0 to obtain the normal unit vector. Define a displacement range along the normal unit vector, centered at 0, with the maximum displacement in both positive and negative directions being a sampling radius greater than 0; within this displacement range, calculate the dot product of the brightness image gradient corresponding to each displacement position and the normal unit vector to form a normal profile.

3. The street view image semantic segmentation system for complex urban scenes according to claim 2, characterized in that, The normal profile is smoothed by convolution with a one-dimensional Gaussian kernel with a standard deviation greater than 0 to obtain a smoothed normal profile. In the smoothed normal profile, locate two local maxima points with the same sign; calculate the absolute value of the values ​​corresponding to the two local maxima points, take the value with the larger absolute value as the principal edge energy, and the value with the smaller absolute value as the secondary edge energy; calculate the absolute value of the difference in displacement between the two local maxima points, and take it as the double edge spacing. The ratio of secondary edge energy to primary edge energy is multiplied by the attenuation factor to obtain the dual-edge energy ratio. This attenuation factor is constructed based on the dual-edge spacing and a spacing attenuation scale constant greater than 0, and the attenuation factor decreases as the dual-edge spacing increases.

4. The street view image semantic segmentation system for complex urban scenes according to claim 1, characterized in that, By concatenating the two components of the double-edge energy ratio, double-edge spacing, and normal unit vector along the channel dimension, a four-channel feature tensor is obtained. For each pixel, the double edge energy ratio is judged. If the double edge energy ratio is greater than or equal to the preset threshold, the mask value corresponding to the pixel is 1; otherwise, the mask value corresponding to the pixel is 0, thus obtaining the binarized prior mask.

5. A street view image semantic segmentation system for complex urban scenes according to claim 1, characterized in that, By performing convolution operations on the street view image within a 3×3 neighborhood, isotropic branch outputs are obtained; by setting a one-dimensional convolution kernel of positive odd length and determining the sampling position along the normal unit vector, one-dimensional convolution operations are performed on the brightness image to obtain directional convolution branch outputs. The initial gating coefficients are obtained by multiplying the dual edge energy ratio by the gating slope constant and then processing the product result through a logic function. Subtract the initial gating coefficient from 1, multiply by the corresponding prior mask, and then add the initial gating coefficient to obtain the final gating coefficient.

6. A street view image semantic segmentation system for complex urban scenes according to claim 5, characterized in that, Subtract the final gating coefficient from 1 and multiply it by the output of the isotropic branch. Then multiply the final gating coefficient by the output of the directional convolution branch. Finally, add the two products together to obtain the fused feature map. The pixel values, feature tensors, and fused feature maps of the red, green, and blue channels of the street view image are concatenated along the channel dimension to obtain the input tensor of the backbone segmentation network. The input tensor is fed into the backbone segmentation network, which performs feature extraction and mapping on the input tensor and outputs the log odds of each preset category. Normalize the logarithmic odds of all preset categories to obtain the probability of each category.

7. A street view image semantic segmentation system for complex urban scenes according to claim 1, characterized in that, For each pixel, select the maximum value among all category probabilities to form a probability maximum map; Calculate the first-order partial derivatives of the probability maximum image in the x and y directions, and take the square root of the sum of the squares of the partial derivatives in the two directions to obtain the boundary intensity of each pixel; Using the current pixel as the reference point, move the pixel distance corresponding to the double edge spacing along the direction indicated by the normal unit vector to determine the corresponding pixel for each pixel.

8. A street view image semantic segmentation system for complex urban scenes according to claim 7, characterized in that, For each pixel, determine whether the value of the prior mask is 1; if it is 1, calculate the product of the double edge energy ratio, the boundary strength of the pixel itself, and the boundary strength of the corresponding pixel; if it is 0, set the product result of the pixel to 0; sum the product results of all pixels and divide by the total number of pixels to obtain the separation loss. Obtain the labeled category for each pixel, calculate the negative logarithm of the probability of each pixel corresponding to the labeled category, sum the negative logarithm results of all pixels and divide by the total number of pixels to obtain the cross-entropy loss; Multiply the weighting coefficients by the separation loss, and then add the product to the cross-entropy loss to obtain the total loss.

9. A street view image semantic segmentation system for complex urban scenes according to claim 1, characterized in that, The first mutual inhibition weight is obtained by multiplying the inhibition coefficient, the double edge energy ratio of the current pixel, and the boundary strength of the corresponding pixel. The second mutual inhibition weight is obtained by multiplying the inhibition coefficient, the double edge energy ratio of the current pixel, and the boundary strength of the current pixel. Subtract the first mutual inhibition weight from the original class log odds of the current pixel to obtain the updated class log odds of the current pixel; subtract the second mutual inhibition weight from the original class log odds of the corresponding pixel to obtain the updated class log odds of the corresponding pixel. The updated log probabilities of each class for all pixels are normalized exponentially to obtain the updated probabilities of each class for each pixel. For each pixel, the category with the highest probability value after the update is selected as the initial segmentation category for that pixel. The initial segmentation categories of all pixels constitute the initial semantic segmentation result.

10. A semantic segmentation system for street view images in complex urban scenes according to claim 1, characterized in that, The preliminary semantic segmentation result is determined as the final semantic segmentation result; the pixel-level numerical field corresponding to the double edge energy ratio is used as the continuous risk map; and the prior mask is directly used as the binary high-risk map.

Images