Semantic segmentation model training method and semantic segmentation method
By introducing instance-level feature loss during semantic segmentation model training, and combining U-shaped neural networks and FPN structures, the problem of inaccurate segmentation of adjacent similar categories is solved, achieving higher segmentation accuracy and faster segmentation speed.
Patent Information
- Application Number
- CN202411155861.7
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-08-22
- Publication Date
- 2026-03-03
AI Technical Summary
Existing semantic segmentation models struggle to accurately segment adjacent and similar pixels that do not belong to the same category. In particular, categories such as grassland, trees, and farmland are easily confused in map semantic segmentation. Furthermore, instance segmentation schemes rely on clustering radii, resulting in slow speeds and inapplicability to instances of different sizes.
In the training process of the semantic segmentation model, instance-level feature loss is introduced. Feature extraction and fusion are performed through U-shaped neural network and FPN structure. Intra-class and inter-class loss are combined to optimize feature expression and improve the model’s discriminativeness at the instance level.
It significantly improves the accuracy of semantic segmentation, accurately segmenting similar and adjacent instances of different categories, avoiding the limitation of cluster radius, and has faster segmentation speed and better segmentation effect.
Smart Images
Figure CN121600248A_ABST
Abstract
Description
Technical Field
[0001] This application relates to the field of artificial intelligence technology, and in particular to semantic segmentation model training methods and semantic segmentation methods. Background Technology
[0002] Semantic segmentation is a task in computer vision that aims to assign each pixel in an image to a specific category, thereby distinguishing different objects and scenes within the image. This segmentation method not only identifies objects in an image but also recognizes the environment and background surrounding those objects, providing a semantic understanding of the entire scene.
[0003] Semantic segmentation in related technologies generally suffers from low segmentation accuracy, especially the inability to accurately segment adjacent and similar pixels that are not of the same category. Summary of the Invention
[0004] This application provides a semantic segmentation model training method and a semantic segmentation method, which can significantly improve the segmentation accuracy of semantic segmentation, especially for adjacent and similar pixels that are not of the same category.
[0005] According to one aspect of the embodiments of this application, a semantic segmentation model training method is provided, the method comprising:
[0006] Obtain sample images, where each sample instance in the sample image has a corresponding category label;
[0007] The sample image is input into a preset model, which triggers the preset model to perform the following operations: downsampling the sample image to obtain at least one first feature; upsampling the at least one first feature to obtain at least one second feature; and performing semantic segmentation on any second feature to obtain a segmentation result.
[0008] Based on any of the second features and each of the category labels, the corresponding instance feature loss is determined, wherein the instance feature loss indicates the feature loss at the sample instance granularity;
[0009] Based on the segmentation results and the category labels, determine the semantic segmentation loss;
[0010] Based on the instance feature loss and the semantic segmentation loss, the parameters of the preset model are adjusted to obtain the semantic segmentation model.
[0011] According to another aspect of the embodiments of this application, a semantic segmentation method is provided, the method comprising:
[0012] Acquire the target image;
[0013] The target image is input into the semantic segmentation model to obtain the semantic segmentation result;
[0014] The semantic segmentation model is trained using the aforementioned semantic segmentation model training method.
[0015] According to one aspect of the embodiments of this application, a semantic segmentation model training apparatus is provided, the apparatus comprising:
[0016] The sample acquisition module is used to acquire sample images, wherein each sample instance in the sample image has a corresponding category label;
[0017] The training module is used to perform the following operations:
[0018] The sample image is input into a preset model, which triggers the preset model to perform the following operations: downsampling the sample image to obtain at least one first feature; upsampling the at least one first feature to obtain at least one second feature; and performing semantic segmentation on any second feature to obtain a segmentation result.
[0019] Based on any of the second features and each of the category labels, the corresponding instance feature loss is determined, wherein the instance feature loss indicates the feature loss at the sample instance granularity;
[0020] Based on the segmentation results and the category labels, determine the semantic segmentation loss;
[0021] Based on the instance feature loss and the semantic segmentation loss, the parameters of the preset model are adjusted to obtain the semantic segmentation model.
[0022] According to one aspect of the embodiments of this application, a semantic segmentation apparatus is provided, the apparatus comprising:
[0023] The image acquisition module is used to acquire the target image;
[0024] The segmentation module is used to input the target image into the semantic segmentation model to obtain the semantic segmentation result;
[0025] The semantic segmentation model is trained using the aforementioned semantic segmentation model training method.
[0026] According to one aspect of the embodiments of this application, a computer device is provided, the computer device including a processor and a memory, the memory storing at least one instruction, at least one program, code set or instruction set, the at least one instruction, the at least one program, the code set or instruction set being loaded and executed by the processor to implement the above-described semantic segmentation model training method, or semantic segmentation method.
[0027] According to one aspect of the embodiments of this application, a computer-readable storage medium is provided, wherein at least one instruction, at least one program, code set, or instruction set is stored in the storage medium, wherein the at least one instruction, the at least one program, the code set, or the instruction set is loaded and executed by a processor to implement the above-described semantic segmentation model training method, or semantic segmentation method.
[0028] According to one aspect of the embodiments of this application, a computer program product is provided, the computer program product including computer instructions stored in a computer-readable storage medium. A processor of a computer device reads the computer instructions from the computer-readable storage medium, and the processor executes the computer instructions, causing the computer device to perform an action to implement the above-described semantic segmentation model training method, or semantic segmentation method.
[0029] The technical solution provided in this application can bring the following beneficial effects:
[0030] This application proposes a semantic segmentation model training method and a semantic segmentation method. Based on traditional semantic segmentation loss, an additional instance-level feature loss, i.e., instance feature loss, is added during the training process of the semantic segmentation model. This guides the model to improve its feature extraction and feature fusion capabilities, resulting in more discriminative features at the instance level. Highly discriminative features can significantly improve semantic segmentation performance, enabling the trained semantic segmentation model to accurately segment even similar, adjacent instances that do not belong to the same category, thereby significantly improving the accuracy of semantic segmentation.
[0031] Compared with semantic segmentation model training methods in related technologies, the semantic segmentation model training method proposed in this application can accurately segment similar adjacent instances that do not belong to the same category, thereby achieving better segmentation results. Compared with instance segmentation methods in related technologies, the semantic segmentation model training method proposed in this application does not require feature clustering, does not rely on cluster radius, does not have the problem of difficulty in simultaneously segmenting instances of different sizes, and has a faster speed.
[0032] In summary, the semantic segmentation model trained based on the semantic segmentation model training method proposed in this application has superior semantic segmentation accuracy compared to related technologies, and even superior segmentation performance compared to instance segmentation schemes in related technologies. It also avoids the drawbacks of instance segmentation schemes. It is a semantic segmentation model with outstanding advantages that is fast, has good segmentation performance, and is not constrained by feature clustering and cluster radius. Attached Figure Description
[0033] To more clearly illustrate the technical solutions in the embodiments of this application, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.
[0034] Figure 1 This is a schematic diagram of an application runtime environment provided in one embodiment of this application;
[0035] Figure 2 This is a flowchart of a semantic segmentation model training method provided in one embodiment of this application;
[0036] Figure 3 This is a schematic flowchart of an intra-class loss determination method provided in one embodiment of this application;
[0037] Figure 4 This is a schematic flowchart of an embodiment of the inter-class loss determination method provided in this application;
[0038] Figure 5 This is a schematic diagram of a preset model provided in one embodiment of this application;
[0039] Figure 6 This is a schematic flowchart of a semantic segmentation method provided in one embodiment of this application;
[0040] Figure 7 This is a schematic diagram illustrating the semantic segmentation effect provided in one embodiment of this application;
[0041] Figure 8 This is a block diagram of a semantic segmentation model training apparatus provided in one embodiment of this application;
[0042] Figure 9 This is a block diagram of a semantic segmentation apparatus provided in one embodiment of this application;
[0043] Figure 10 This is a structural block diagram of a computer device provided in one embodiment of this application. Detailed Implementation
[0044] Before describing the specific embodiments of this application, the relevant technical background related to the embodiments of this application will be introduced to facilitate understanding by those skilled in the art.
[0045] Semantic segmentation is a task in computer vision that aims to assign each pixel in an image to a specific category, thereby distinguishing different objects and scenes within the image. This segmentation method not only identifies objects in an image but also recognizes the environment and background surrounding those objects, providing a semantic understanding of the entire scene.
[0046] In related technologies, semantic segmentation models are typically trained using deep learning to achieve semantic segmentation. However, during the prediction process, the relationships and interactions between pixel features can only be determined within the receptive field through feature fusion, making it difficult to accurately segment adjacent and similar pixels that do not belong to the same category. Taking map semantic segmentation as an example, satellite imagery contains objects such as grassland, trees, and farmland. These objects have a certain degree of similarity, and if they are close together, they are easily confused and difficult to segment accurately. Furthermore, semantic segmentation models are more easily influenced by a large number of objects; for example, if four patches of grassland surround a piece of farmland, the farmland is likely to be predicted as grassland.
[0047] Related technologies also propose improving semantic segmentation performance through instance segmentation. Instance segmentation is a computer vision task that combines the characteristics of object detection and semantic segmentation. It not only identifies the categories of all objects in an image but also distinguishes different objects (instances) within the same category. For example, each patch of grass and each field mentioned above is an instance, and a scene with four patches of grass surrounding a field has five instances.
[0048] Compared to semantic segmentation, instance segmentation yields significantly more refined results. Instance segmentation focuses more on the segmentation of individual objects (instances), and its performance on adjacent and similar instances that do not belong to the same category is slightly better than that of semantic segmentation. However, instance segmentation relies on feature clustering, which, as a post-processing step, makes the inference computation too slow. Furthermore, this approach is highly dependent on conditions such as cluster radius, is sensitive to outliers, and lacks consideration for overall features, making it difficult to segment instances of different sizes simultaneously under the same cluster radius.
[0049] This application proposes a semantic segmentation model training method and a semantic segmentation method. Based on the traditional semantic segmentation loss, this method adds an instance-level feature loss, i.e., instance feature loss, to the semantic segmentation model training process. This guides the model to improve its feature extraction and feature fusion capabilities, resulting in more discriminative features at the instance granularity. Highly discriminative features can significantly improve the semantic segmentation effect, enabling the trained semantic segmentation model to accurately segment even similar adjacent instances that do not belong to the same category, thereby significantly improving the accuracy of semantic segmentation.
[0050] Compared with semantic segmentation model training methods in related technologies, the semantic segmentation model training method proposed in this application can accurately segment similar adjacent instances that do not belong to the same category, thereby achieving better segmentation results. Compared with instance segmentation model training methods in related technologies, the semantic segmentation model training method proposed in this application does not require feature clustering, does not rely on cluster radius, does not have the difficulty of simultaneously segmenting instances of different sizes, and has a faster speed.
[0051] In summary, the semantic segmentation model trained based on the semantic segmentation model training method proposed in this application has superior semantic segmentation accuracy compared to related technologies, and even superior segmentation performance compared to instance segmentation schemes in related technologies. It also avoids the drawbacks of instance segmentation schemes. It is a semantic segmentation model with outstanding advantages that is fast, has good segmentation performance, and is not constrained by feature clustering and cluster radius.
[0052] To make the objectives, technical solutions, and advantages of this application clearer, the embodiments of this application will be described in further detail below with reference to the accompanying drawings. It should be noted that all data used in the embodiments of this application has been fully authorized by the relevant parties before use.
[0053] Please refer to Figure 1 This diagram illustrates an application runtime environment provided in one embodiment of this application under an exemplary implementation. The application runtime environment may include: terminal 10 and server 20.
[0054] Terminal 10 includes, but is not limited to, electronic devices such as mobile phones, computers, smart voice interaction devices, smart home appliances, in-vehicle terminals, game consoles, e-book readers, multimedia playback devices, and wearable devices. Application clients can be installed on terminal 10.
[0055] In this embodiment, the application described above can be any application that uses the semantic segmentation model training method and semantic segmentation method proposed in this embodiment. Typically, the application is a map application. Of course, besides map applications, other types of applications can also use the semantic segmentation model training method and semantic segmentation method proposed in this embodiment. For example, news applications, social applications, search applications, interactive entertainment applications, browser applications, shopping applications, content sharing applications, virtual reality (VR) applications, augmented reality (AR) applications, etc., are not limited in this embodiment. Optionally, the terminal 10 runs a client of the above application.
[0056] Server 20 provides background services to clients of applications in terminal 10. For example, server 20 can be a background server for the aforementioned applications. Server 20 can be a standalone physical server, a server cluster or distributed system composed of multiple physical servers, or a cloud server providing basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communication, middleware services, domain name services, security services, CDN (Content Delivery Network), and big data and artificial intelligence platforms. Optionally, server 20 can simultaneously provide background services to applications in multiple terminals 10.
[0057] Optionally, terminal 10 and server 20 can communicate with each other via network 30. Terminal 10 and server 20 can be directly or indirectly connected via wired or wireless communication, which is not limited herein.
[0058] Please refer to Figure 2 The document illustrates a flowchart of a semantic segmentation model training method according to an exemplary embodiment. This method can be applied to a computer device, which refers to an electronic device capable of data computation and processing. For example, the entity executing each step may be... Figure 1 The relevant devices in the application's runtime environment are shown. This method may include the following steps:
[0059] Step S201. Obtain sample images. Each sample instance in the sample images has a corresponding category label.
[0060] This application does not limit the number of sample images. In order to achieve better training results, a large number of sample images can be collected to construct a sample set.
[0061] This application does not limit the sample set, which can cover multiple categories, each with multiple entities. For example, multiple entities belonging to the "grassland" category, multiple entities belonging to the "trees" category, multiple entities belonging to the "farmland" category, and multiple entities belonging to the "building" category. The foreground to be identified can include N categories, where "grassland," "trees," "farmland," and "building" all belong to the aforementioned N categories, and N is a positive integer greater than 1. In addition, the background is a separate category. Thus, the sample set can cover N+1 categories, each with multiple entities.
[0062] This application does not limit any sample image in the sample set, but can cover at least one of the aforementioned N+1 categories. The covered category has at least one entity, and the entity has a corresponding category label.
[0063] This application does not limit the source of the sample images and sample sets. For example, they can come from digital topographic maps. By training a semantic segmentation model, high-accuracy semantic segmentation of digital topographic maps can be achieved, which can significantly improve the segmentation accuracy of digital topographic maps and accurately segment easily mis-segmented categories such as "grassland", "trees" and "farmland" in digital topographic maps, thereby significantly improving the visualization effect of digital topographic maps.
[0064] Step S202. Input the above sample image into the preset model and trigger the preset model to perform the following operations: downsample the above sample image to obtain at least one first feature; upsample the above at least one first feature to obtain at least one second feature; perform semantic segmentation on any of the above second features to obtain a segmentation result.
[0065] This application does not limit the network structure used in the downsampling and upsampling processes. For example, the downsampling and upsampling processes can be implemented using a U-shaped neural network. The U-shaped neural network (U-Net) is named for its unique structure, which includes a shrinking (encoding) path and a symmetrical expanding (decoding) path, connected by skip connections. The shrinking (encoding) path, which performs downsampling in this application embodiment, consists of multiple convolutional and pooling layers, progressively reducing spatial resolution while increasing the number of feature channels to extract high-level features of the image. The expanding (decoding) path is the inverse of the shrinking (encoding) path, used to perform upsampling in this application embodiment. This path progressively restores spatial resolution through upsampling (such as transposed convolution). Skip connections concatenate features from the shrinking (encoding) path with corresponding features from the expanding (decoding) path, which helps retain more positional information in the expanding (decoding) process. These two paths are structurally symmetrical, hence the name U-shaped neural network. The U-shaped neural network (U-Net) can be used as the skeleton of the preset model in this application embodiment.
[0066] At the end of the extension (decoding) path of the U-Net, a functional layer for semantic segmentation can be superimposed to achieve semantic segmentation. This functional layer can be a normalization layer or other structures, which does not pose an obstacle to implementation and will not be discussed further.
[0067] In some implementations, a U-Net (U-Net) combined with an FPN structure can be used to build a pre-defined model. This model can be downsampled by multiple convolutional layers combined with normalization layers and feature fusion layers, and then upsampled using the inverse process of the FPN structure to restore the spatial resolution consistent with the original input image, thereby performing semantic segmentation. The FPN structure, or Feature Pyramid Network, is a deep learning architecture widely used in computer vision. By constructing a top-down pyramid structure, the FPN structure effectively combines low-level fine spatial information with high-level strong semantic information.
[0068] The spatial resolution of the feature map formed by each of the aforementioned at least one second feature can be gradually increased until it is restored to the same spatial resolution as the sample image input to the preset model. The size of the feature map obtained at this point is (w*h*(N+1)), where N+1 represents the number of categories covered by the prediction result, N is the number of foreground categories, and w and h are the width and height dimensions of the sample image.
[0069] Step S203. Based on any of the above-mentioned second features and each of the above-mentioned category labels, determine the corresponding instance feature loss, wherein the instance feature loss indicates the feature loss at the sample instance granularity.
[0070] For any second feature, the corresponding instance feature loss can be calculated. This application does not limit which second features are used to calculate the corresponding instance feature loss. The limitation can be made according to the actual situation and does not constitute an obstacle to implementation.
[0071] The instance feature loss indicates the feature loss at the sample instance granularity, which is the loss calculated at the feature level based on the category label corresponding to each sample instance in the aforementioned sample image. This loss reflects the discriminative power loss of the pixel features obtained by the aforementioned preset model in abstracting the features of the sample instances. This application proposes that insufficient pixel feature discriminative power of instances is the core reason for poor semantic segmentation performance in related technologies. This application proposes to guide the model to increase the discriminative power of instance pixel features at the feature abstraction level by using the category label corresponding to the sample instance, thereby fundamentally solving the problem of poor semantic segmentation performance. This is a specific solution proposed by this application for the current situation of insufficient semantic segmentation capability in the specific task of semantic segmentation.
[0072] The instance feature loss is responsible for guiding the model to learn the representation of pixel features of sample instances in the sample image, making the pixel features of sample instances of the same category more similar, and the pixel features of sample instances of different categories more different, thereby increasing the discriminative power of the pixel features obtained by the model from the feature abstraction of sample instances.
[0073] In one implementation, the instance feature loss includes at least one of the following:
[0074] Intra-class loss is used to indicate the difference in pixel features corresponding to sample instances of the same class determined based on the second feature mentioned above;
[0075] Inter-class loss is used to indicate the similarity of different pixel features of sample instances of different categories determined based on the second feature mentioned above.
[0076] For any second loss output from a downsampling layer, the corresponding intra-class loss and / or inter-class loss can be calculated. The intra-class loss guides the pixel features of sample instances of the same class to be as similar as possible in the model output, while the inter-class loss guides the pixel features of sample instances of different classes to be significantly different, thereby improving the model's ability to distinguish between adjacent similar entities.
[0077] In summary, instance feature loss can include intra-class loss to guide the model to narrow the distance between pixel features generated by sample instances of the same class, or inter-class loss to guide the model to widen the distance between pixel features generated by sample instances of different classes. These two types of losses guide the model to improve the feature discrimination of sample instances from different perspectives, intra-class and inter-class, and can improve the accuracy of semantic segmentation in a cooperative or selective manner.
[0078] Step S204. Based on the above segmentation results and the above category labels, determine the semantic segmentation loss.
[0079] Semantic segmentation loss is a common loss for semantic segmentation tasks, and this application will not elaborate on it or limit its calculation method. In an exemplary implementation, the standard cross-entropy loss function can be used to calculate the semantic segmentation loss. For example, it can be based on the formula... Calculate the semantic segmentation loss, where L represents the semantic segmentation loss, L i The semantic segmentation loss is for category i, where N is the number of foreground categories, and p is the number of categories. i Let y be the predicted probability of the segmented category i. i For the corresponding category label.
[0080] Step S205. Based on the above instance feature loss and the above semantic segmentation loss, adjust the parameters of the above preset model to obtain the semantic segmentation model.
[0081] In this embodiment, the obtained loss of all instance features and the semantic segmentation loss described above are weighted and fused to determine the total loss. Of course, this embodiment does not limit the weights and can set them according to the actual situation. Then, based on the total loss, all or some important parameters in the above preset model are adjusted until training is completed, and the semantic segmentation model is obtained.
[0082] This application does not limit the parameter tuning method; for example, gradient descent can be used to adjust the parameters. This application also does not limit the stopping condition for training; for example, training can stop when the total loss is less than a preset loss threshold or when the number of parameter tuning attempts exceeds a preset number. The preset loss threshold and the preset number of attempts can be set according to actual conditions, and will not be elaborated here.
[0083] This application's embodiments, based on traditional semantic segmentation loss, add an additional instance-level feature loss, i.e., instance feature loss, to the semantic segmentation model training process. This guides the model to improve its feature extraction and feature fusion capabilities, resulting in more discriminative features at the instance granularity. Highly discriminative features can significantly improve semantic segmentation performance, enabling the trained semantic segmentation model to accurately segment even similar adjacent instances that do not belong to the same category, thereby significantly improving the accuracy of semantic segmentation.
[0084] Please refer to Figure 3 The diagram illustrates a flowchart of an intra-class loss determination method according to an embodiment of this application. The method for determining the corresponding instance feature loss based on any of the aforementioned second features and each of the aforementioned category labels includes:
[0085] Step S301. For any category label, determine the corresponding feature mean, which is the average value of the pixel features corresponding to the sample instance with the above category label in the above second feature.
[0086] For any category label, we can determine which sample instances it corresponds to in the sample image and which pixels it covers. This allows us to determine the pixel features corresponding to these pixels in the second feature. The mean of these pixel features is the feature mean, denoted as μ. c In the following text, for each category j, there is a corresponding μ. c .
[0087] Step S302. For the pixel features of each sample instance with the above category labels, determine the corresponding pixel feature loss based on the difference between the above pixel features and the above feature mean; determine the single-class loss corresponding to the above category label based on the pixel feature loss corresponding to the pixel features of each sample instance with the above category labels.
[0088] For any sample entity's pixel feature x i The corresponding pixel feature loss can be calculated. This application does not limit the calculation formulas for pixel feature loss and single-class loss; any formula that can quantify the differences in pixel features between different entities of a single class is acceptable.
[0089] Step S303. Based on the single-class loss corresponding to each of the above category labels, obtain the above intra-class loss.
[0090] The embodiments of this application can determine the intra-class loss based on the sum or mean of the single-class losses corresponding to each of the above-mentioned category labels. The embodiments of this application do not limit the calculation formula of the intra-class loss, as long as it can achieve the purpose of quantifying the pixel feature differences between different entities in each single class.
[0091] By calculating the differences in pixel features among sample instances in each category, a single-class loss is obtained. The intra-class loss is then calculated by combining the single-class losses for each category. This guides the model to penalize instances of the same category with large differences in pixel features, thereby narrowing the distance between pixel features generated by sample instances of the same category and improving the discriminative and recognizable properties of pixel features for semantic segmentation tasks.
[0092] In one embodiment, determining the corresponding pixel feature loss based on the difference between the pixel feature and the mean feature includes: when the difference between the pixel feature and the mean feature is less than a first difference threshold, determining the pixel feature loss as a target preset value, wherein the target preset value indicates that the corresponding pixel feature loss is not included in the corresponding single-class loss.
[0093] For example, the target preset value can be 0, based on the formula. This allows the pixel feature loss corresponding to the aforementioned target preset value to be excluded from the corresponding single-class loss, thereby obtaining the intra-class loss L. var "+" indicates that only ||μ c -x i ||-δ v The calculation is only performed if the result is positive, or in other words, only if μ... c and x i Calculations are only performed when the difference is sufficiently large, δ v Represents the first difference threshold, ||μ c -x i || represents the difference between the pixel features of a sample instance of category j and the mean of its corresponding features. ||μ c -x i ||-δ vThis represents the difference between the difference and the first difference threshold. If the difference is negative, the corresponding pixel feature loss can be set to the target preset value and not included in the calculation result. N c δ represents the number of sample entities in category j. v The settings can be adjusted according to the actual situation, and will not be elaborated upon here. Other parameters have been described above and will not be repeated here.
[0094] This application's embodiments limit the single-class loss to only consider pixel feature loss when the difference is large, such as greater than a first difference threshold. This guides the model to penalize cases where the pixel feature differences between sample instances of the same class are large, but not to penalize cases where the pixel feature differences between sample instances of the same class are small (these differences are not included in the calculation results). In this way, the pixel features extracted by the model for instances of the same class will be similar, thereby enhancing the model's segmentation accuracy for instances of the same class.
[0095] Please refer to Figure 4 The diagram illustrates a flowchart of an embodiment of the inter-class loss determination method. The method, based on any of the aforementioned second features and each of the aforementioned category labels, determines the corresponding instance feature loss, including:
[0096] Step S401. Determine the target set. Any combination of labels in the target set includes two different category labels. The two category labels in different combination of labels are not exactly the same.
[0097] For the aforementioned N+1 categories, let CC = N+1, where CC represents the total number of categories. Any pairwise combination of the CC categories yields the corresponding label combinations, and the unique label combinations form the target set.
[0098] Step S402. For any combination of labels, determine the mean value of the features corresponding to each category label. The mean value of the features is the average value of the pixel features of the sample instances with the corresponding category labels in the second feature.
[0099] The meaning and calculation method of the characteristic mean have been described above and will not be repeated here.
[0100] Step S403. Determine the label combination loss based on the difference in the mean features corresponding to each category label; determine the inter-class loss based on the label combination loss corresponding to each of the above label combinations in the above target set.
[0101] The embodiments of this application can determine the above-mentioned inter-class loss based on the sum or mean of the label combination loss corresponding to each label combination. The embodiments of this application do not limit the calculation formula of the label combination loss and the inter-class loss, as long as it can achieve the purpose of quantifying the distance of the pixel features of each label combination.
[0102] By calculating the differences in pixel features of sample instances under different categories, the inter-class loss is obtained. This guides the model to penalize cases where the differences in pixel features between sample instances of different categories are small, thereby widening the distance between pixel features generated by sample instances of different categories and improving the discriminativeness and recognizability of pixel features for semantic segmentation tasks.
[0103] In one embodiment, determining the label combination loss based on the difference between the mean features corresponding to each category label includes: when the difference between the mean features corresponding to each category label is greater than a second difference threshold, determining the label combination loss as a target preset value, wherein the target preset value indicates that the corresponding label combination loss is not included in the inter-class loss.
[0104] For example, the target preset value can be 0, based on the formula. This allows the label combination loss corresponding to the aforementioned target preset value to be excluded from the inter-class loss, thus obtaining the inter-class loss L. dist "+" indicates that only δ is present. d -||μ ca -μ cb δ is calculated only if the result of || is positive. d This represents the second difference threshold, μ. ca and μ cb δ represents the mean features of categories ca and cb in the label combination. d -||μ ca -μ cb || represents the difference between the mean of this feature and the second difference threshold. If the difference is negative, the corresponding label combination loss can be set to the target preset value and not included in the calculation result. δ d The settings can be adjusted according to the actual situation, and will not be elaborated upon here. Other parameters have been described above and will not be repeated here.
[0105] This application embodiment limits the label combination loss to only consider label combination losses with small differences, such as less than a second difference threshold, thereby guiding the model to penalize cases where the pixel feature differences between sample instances of different categories are small, but not cases where the pixel feature differences between sample instances of different categories are large. In this way, the pixel features extracted by the model for instances of different categories will be dissimilar, thereby enhancing the model's ability to distinguish instances of different categories and improving segmentation accuracy.
[0106] In summary, intra-class loss requires features within a class to be close to or similar to the class center, while inter-class loss requires the central features of each class to be far from the class centers of other classes. By supervising the feature representation of the model through intra-class and inter-class losses, the feature distribution at each stage is optimized, thereby improving feature quality.
[0107] In one implementation, please refer to Figure 5 The diagram illustrates a preset model according to an embodiment of this application. The preset model includes a feature extraction layer 501, a feature fusion layer 502, and a semantic segmentation layer 503. The feature extraction layer 501 includes at least two downsampling layers 5011 connected in sequence, and the feature fusion layer 502 includes at least two upsampling layers 5021 connected in sequence. Downsampling the sample image to obtain at least one first feature includes: downsampling the sample image based on the at least two downsampling layers 5011 connected in sequence to obtain a first feature output by each downsampling layer 5011. Upsampling the at least one first feature to obtain at least one second feature includes: upsampling the at least one first feature based on the at least two upsampling layers 5021 connected in sequence to obtain a second feature output by each upsampling layer 5021. Semantic segmentation of any of the second features includes: semantically segmenting the second feature output by the upsampling layer 5021 connected to the semantic segmentation layer 503 to obtain the segmentation result.
[0108] This application embodiment does not limit the structure of the feature extraction layer 501, which can achieve multi-scale downsampling through sequentially connected downsampling layers. For example, a 34-layer residual network ResNet34 can be used as the main model of the feature extraction layer 501. Each downsampling layer can consist of a convolutional layer, a normalization layer, and an activation layer. Downsampling layers closer to the input focus on extracting basic features such as edge textures, while downsampling layers farther from the input focus on abstracting and combining basic textures. The normalization layer normalizes the input features to a normal distribution. The activation layer performs non-linear mapping on the input features to enhance generalization ability. This application embodiment can add several dilated convolutional layers 504 between the feature extraction layer 501 and the feature fusion layer 502 to increase the receptive field and improve feature representation ability.
[0109] The feature fusion layer 502 can adopt a Feature Pyramid Network (FPN) structure, which achieves feature fusion and upsampling through multiple upsampling layers. Each upsampling layer performs a deconvolution operation to achieve upsampling, which has the effect of scaling and provides the necessary feature information for higher-level upsampling fusion. The input of each upsampling layer comes not only from the output of the previous upsampling layer, but also from the output of the same-level downsampling layer. Fusing these two types of features can improve the quality of the second feature.
[0110] This application does not limit the structure of the preset model, and other structures can also be used. The semantic segmentation model training method of this application not only achieves semantic segmentation but also has instance segmentation capabilities, without requiring feature clustering. It addresses the problem of inaccurate differentiation of similar categories in the semantic segmentation model by introducing instance feature loss during the upsampling stage. For example, if it is necessary to distinguish between trees, grassland, and farmland, the instance feature loss requires intra-class constraints to ensure that pixel features within the same category are as similar as possible, and that pixel features between different categories are significantly different. It also overcomes the problem of inaccurate loss measurement caused by uneven instance sizes and imbalanced difficulty. Through joint learning of multi-task loss functions using semantic segmentation loss and instance feature loss, mutual supervision and assistance are provided to solve the semantic segmentation problem of similar categories end-to-end, significantly improving the semantic segmentation effect and speed.
[0111] Please refer to Figure 6 This diagram illustrates a semantic segmentation method provided in an embodiment of this application. The method includes:
[0112] Step S601. Obtain the target image.
[0113] This application does not limit the content and source of the target image; for example, it can be a digital terrain map that has not undergone semantic segmentation.
[0114] The quality and semantic segmentation effect of digital topographic maps have a significant impact on the application effectiveness of digital topographic map products. Background areas in digital topographic maps are rich and varied, encompassing many similar types of land features. Manual segmentation is time-consuming, labor-intensive, and inefficient. The semantic segmentation model trained using the embodiments of this application can be used for automatic segmentation, resolving the problem of misclassification of adjacent similar land feature instances. This allows for fully automated data production, which can then be uploaded as map background data for user use.
[0115] Step S602. Input the target image into the semantic segmentation model to obtain the semantic segmentation result; wherein the semantic segmentation model is trained by the aforementioned semantic segmentation model training method.
[0116] The embodiments of this application can be applied to the field of map processing, such as performing semantic segmentation on digital terrain maps obtained based on satellite imagery, remote sensing imagery, and other technical means. Please refer to... Figure 7 The diagram illustrates the semantic segmentation effect proposed in the embodiments of this application. Figure 7(a) shows the original digital terrain map, revealing that trees and grasslands are interspersed and similar in shape. Since trees and grasslands share some textural similarity, and the actual distribution in the training set shows a significantly larger number of grasslands than trees, the semantic segmentation model may be more inclined to learn the grassland category while relatively ignoring the tree category. This can lead to trees being misclassified as grassland when they are adjacent to each other. Please refer to [reference needed]. Figure 7 (b) is the semantic segmentation result of the semantic segmentation model in the related technology. Only a few trees (dark area) are identified, and a large number of trees are misidentified as grass (light area). Figure 7 (c) shows the semantic segmentation result of the semantic segmentation model in this embodiment. It can be seen that trees and grassland are accurately segmented, with dark and light areas interspersed, significantly improving the segmentation effect. For map-specific scenarios, the semantic segmentation model in this embodiment greatly improves the segmentation effect of green areas, grasslands, and farmland, allowing many bare areas on digital terrain maps to be given green vegetation, enhancing the rendering effect. This also improves automated map production efficiency and significantly increases the human productivity of map product production.
[0117] The following are embodiments of the apparatus of this application, which can be used to execute embodiments of the method of this application. For details not disclosed in the apparatus embodiments of this application, please refer to the embodiments of the method of this application.
[0118] Please refer to Figure 8 The diagram illustrates a block diagram of a semantic segmentation model training apparatus according to an exemplary embodiment. This apparatus has the function of implementing the aforementioned semantic segmentation model training method; this function can be implemented in hardware or by hardware executing corresponding software. The apparatus can be a computer device or can be located within a computer device. The apparatus may include:
[0119] The sample acquisition module 801 is used to acquire sample images, where each sample instance in the sample image has a corresponding category label.
[0120] Training module 802 is used to perform the following operations:
[0121] The sample image is input into a preset model, which triggers the preset model to perform the following operations: downsampling the sample image to obtain at least one first feature; upsampling the at least one first feature to obtain at least one second feature; and performing semantic segmentation on any of the second features to obtain a segmentation result.
[0122] Based on any of the above-mentioned second features and each of the above-mentioned category labels, the corresponding instance feature loss is determined, and the above-mentioned instance feature loss indicates the feature loss at the sample instance granularity;
[0123] Based on the above segmentation results and the above category labels, the semantic segmentation loss is determined;
[0124] Based on the instance feature loss and semantic segmentation loss described above, the parameters of the preset model are adjusted to obtain the semantic segmentation model.
[0125] In one implementation, the instance feature loss described above includes at least one of the following:
[0126] Intra-class loss is used to indicate the difference in pixel features corresponding to sample instances of the same class determined based on the second feature mentioned above;
[0127] Inter-class loss is used to indicate the similarity of different pixel features of sample instances of different categories determined based on the second feature mentioned above.
[0128] In one embodiment, the training module 802 is configured to perform the following operations:
[0129] For any category label, determine the corresponding feature mean, which is the average pixel feature of the sample instance with the above category label in the above second feature;
[0130] For each sample instance with the above category labels, the pixel features are determined based on the difference between the above pixel features and the above feature mean.
[0131] Based on the pixel feature loss corresponding to the pixel features of each sample instance with the above category labels, determine the single-class loss corresponding to the above category labels.
[0132] Based on the single-class loss corresponding to each of the above category labels, the above intra-class losses are obtained.
[0133] In one embodiment, the training module 802 is configured to perform the following operations:
[0134] Determine the target set, where any combination of labels in the target set includes two different category labels, and the two category labels in different combinations are not exactly the same;
[0135] For any combination of labels, determine the mean value of the features corresponding to each category label. The mean value of the features is the average value of the pixel features of the sample instances with the corresponding category labels in the second feature.
[0136] The label combination loss is determined based on the difference between the mean features corresponding to each category label.
[0137] Based on the label combination loss corresponding to each of the above label combinations in the target set, the above inter-class loss is determined.
[0138] In one embodiment, the training module 802 is configured to perform the following operations:
[0139] If the difference between the aforementioned pixel feature and the aforementioned feature mean is less than a first difference threshold, the aforementioned pixel feature loss is determined as a target preset value. The aforementioned target preset value indicates that the corresponding pixel feature loss is not included in the corresponding single-class loss.
[0140] In one embodiment, the training module 802 is configured to perform the following operations:
[0141] If the difference between the mean features corresponding to each of the above categories of labels is greater than the second difference threshold, the label combination loss is determined as the target preset value. The target preset value indicates that the corresponding label combination loss is not included in the inter-class loss.
[0142] In one implementation, the aforementioned preset model includes at least two downsampling layers, at least two upsampling layers, and a semantic segmentation layer connected in sequence.
[0143] The above-mentioned downsampling of the sample image to obtain at least one first feature includes: downsampling the sample image based on at least two sequentially connected downsampling layers to obtain the first feature output by each of the downsampling layers;
[0144] The above-mentioned upsampling of at least one first feature to obtain at least one second feature includes: upsampling the at least one first feature based on at least two sequentially connected upsampling layers to obtain the second feature output by each of the above-mentioned upsampling layers;
[0145] The above-mentioned semantic segmentation of any of the second features includes: performing semantic segmentation on the second feature output by the upsampling layer connected to the above-mentioned semantic segmentation layer to obtain the above-mentioned segmentation result.
[0146] It should be noted that the apparatus provided in the above embodiments is only illustrated by the division of the above functional modules when implementing its functions. In actual applications, the above functions can be assigned to different functional modules as needed, that is, the internal structure of the device can be divided into different functional modules to complete all or part of the functions described above. In addition, the apparatus and method embodiments provided in the above embodiments belong to the same concept, and the specific implementation process can be found in the method embodiments, which will not be repeated here.
[0147] Please refer to Figure 9 The diagram illustrates a block diagram of a semantic segmentation apparatus according to an exemplary embodiment. This apparatus has the functionality to implement the semantic segmentation method described above; this functionality can be implemented in hardware or by hardware executing corresponding software. The apparatus can be a computer device or can be located within a computer device. The apparatus may include:
[0148] Image acquisition module 901 is used to acquire target images;
[0149] The segmentation module 902 is used to input the target image mentioned above into the semantic segmentation model to obtain the semantic segmentation result;
[0150] The semantic segmentation model mentioned above was trained using the aforementioned semantic segmentation model training method.
[0151] It should be noted that the apparatus provided in the above embodiments is only illustrated by the division of the above functional modules when implementing its functions. In actual applications, the above functions can be assigned to different functional modules as needed, that is, the internal structure of the device can be divided into different functional modules to complete all or part of the functions described above. In addition, the apparatus and method embodiments provided in the above embodiments belong to the same concept, and the specific implementation process can be found in the method embodiments, which will not be repeated here.
[0152] Please refer to Figure 10 It illustrates a structural block diagram of a computer device according to an exemplary embodiment, for performing the above-described semantic segmentation model training method or semantic segmentation method. Specifically:
[0153] Computer device 1000 includes a central processing unit (CPU) 1001, a system memory 1004 including random access memory (RAM) 1002 and read-only memory (ROM) 1003, and a system bus 1005 connecting the system memory 1004 and the CPU 1001. Computer device 1000 also includes a basic input / output system (I / O system) 1006 to facilitate information transfer between various devices within the computer, and a mass storage device 1007 for storing the operating system 1013, application programs 1014, and other program modules 1015.
[0154] The basic input / output system 1006 includes a display 1008 for displaying information and an input device 1009 for user input, such as a mouse or keyboard. Both the display 1008 and the input device 1009 are connected to the central processing unit 1001 via an input / output controller 1010 connected to the system bus 1005. The basic input / output system 1006 may also include the input / output controller 1010 for receiving and processing input from multiple other devices such as a keyboard, mouse, or electronic stylus. Similarly, the input / output controller 1010 also provides output to a display screen, printer, or other types of output devices.
[0155] Mass storage device 1007 is connected to central processing unit 1001 via a mass storage controller (not shown) connected to system bus 1005. Mass storage device 1007 and its associated computer-readable media provide non-volatile storage for computer device 1000. That is, mass storage device 1007 may include computer-readable media (not shown) such as hard disk or CD-ROM (Compact Disc Read-Only Memory) drive.
[0156] Without loss of generality, computer-readable media can include computer storage media and communication media. Computer storage media includes volatile and non-volatile, removable and non-removable media implemented using any method or technology for storing information such as computer-readable instructions, data structures, program modules, or other data. Computer storage media includes RAM, ROM, EPROM (Erasable Programmable Read Only Memory), EEPROM (Electrically Erasable Programmable Read Only Memory), flash memory or other solid-state storage technologies, CD-ROM, DVD (Digital Video Disc) or other optical storage, magnetic tape cassettes, magnetic tape, disk storage, or other magnetic storage devices. Of course, those skilled in the art will recognize that computer storage media are not limited to the above-mentioned types. The system memory 1004 and mass storage device 1007 described above can be collectively referred to as memory.
[0157] According to various embodiments of this application, the computer device 1000 can also be connected to a remote computer on a network, such as the Internet. That is, the computer device 1000 can be connected to the network 1012 via the network interface unit 1011 connected to the system bus 1005, or the network interface unit 1011 can be used to connect to other types of networks or remote computer systems (not shown).
[0158] The aforementioned memory also includes a computer program stored in the memory and configured to be executed by one or more processors to implement the aforementioned semantic segmentation model training method or semantic segmentation method.
[0159] In an exemplary embodiment, a computer-readable storage medium is also provided, wherein at least one instruction, at least one program, code set, or instruction set is stored therein, wherein the at least one instruction, at least one program, code set, or instruction set is executed by a processor to implement the semantic segmentation model training method or semantic segmentation method.
[0160] Specifically, the training method for this semantic segmentation model includes:
[0161] Obtain sample images, where each sample instance in the sample images has a corresponding category label;
[0162] The sample image is input into a preset model, which triggers the preset model to perform the following operations: downsampling the sample image to obtain at least one first feature; upsampling the at least one first feature to obtain at least one second feature; and performing semantic segmentation on any of the second features to obtain a segmentation result.
[0163] Based on any of the above-mentioned second features and each of the above-mentioned category labels, the corresponding instance feature loss is determined, and the above-mentioned instance feature loss indicates the feature loss at the sample instance granularity;
[0164] Based on the above segmentation results and the above category labels, the semantic segmentation loss is determined;
[0165] Based on the instance feature loss and semantic segmentation loss described above, the parameters of the preset model are adjusted to obtain the semantic segmentation model.
[0166] In one exemplary implementation, the instance feature loss described above includes at least one of the following:
[0167] Intra-class loss is used to indicate the difference in pixel features corresponding to sample instances of the same class determined based on the second feature mentioned above;
[0168] Inter-class loss is used to indicate the similarity of different pixel features of sample instances of different categories determined based on the second feature mentioned above.
[0169] In an exemplary implementation, determining the corresponding instance feature loss based on any of the aforementioned second features and each of the aforementioned category labels includes:
[0170] For any category label, determine the corresponding feature mean, which is the average pixel feature of the sample instance with the above category label in the above second feature;
[0171] For each sample instance with the above category labels, the pixel features are determined based on the difference between the above pixel features and the above feature mean.
[0172] Based on the pixel feature loss corresponding to the pixel features of each sample instance with the above category labels, determine the single-class loss corresponding to the above category labels.
[0173] Based on the single-class loss corresponding to each of the above category labels, the above intra-class losses are obtained.
[0174] In an exemplary implementation, determining the corresponding instance feature loss based on any of the aforementioned second features and each of the aforementioned category labels includes:
[0175] Determine the target set, where any combination of labels in the target set includes two different category labels, and the two category labels in different combinations are not exactly the same;
[0176] For any combination of labels, determine the mean value of the features corresponding to each category label. The mean value of the features is the average value of the pixel features of the sample instances with the corresponding category labels in the second feature.
[0177] The label combination loss is determined based on the difference between the mean features corresponding to each category label.
[0178] Based on the label combination loss corresponding to each of the above label combinations in the target set, the above inter-class loss is determined.
[0179] In an exemplary implementation, determining the corresponding pixel feature loss based on the difference between the pixel features and the mean of the features includes:
[0180] If the difference between the aforementioned pixel feature and the aforementioned feature mean is less than a first difference threshold, the aforementioned pixel feature loss is determined as a target preset value. The aforementioned target preset value indicates that the corresponding pixel feature loss is not included in the corresponding single-class loss.
[0181] In one exemplary implementation, determining the label combination loss based on the difference in the mean features corresponding to each category label includes:
[0182] If the difference between the mean features corresponding to each of the above categories of labels is greater than the second difference threshold, the label combination loss is determined as the target preset value. The target preset value indicates that the corresponding label combination loss is not included in the inter-class loss.
[0183] In one exemplary embodiment, the aforementioned preset model includes at least two downsampling layers, at least two upsampling layers, and a semantic segmentation layer connected in sequence.
[0184] The above-mentioned downsampling of the sample image to obtain at least one first feature includes: downsampling the sample image based on at least two sequentially connected downsampling layers to obtain the first feature output by each of the downsampling layers;
[0185] The above-mentioned upsampling of at least one first feature to obtain at least one second feature includes: upsampling the at least one first feature based on at least two sequentially connected upsampling layers to obtain the second feature output by each of the above-mentioned upsampling layers;
[0186] The above-mentioned semantic segmentation of any of the second features includes: performing semantic segmentation on the second feature output by the upsampling layer connected to the above-mentioned semantic segmentation layer to obtain the above-mentioned segmentation result.
[0187] Specifically, the semantic segmentation method includes:
[0188] The image acquisition module is used to acquire the target image;
[0189] The segmentation module is used to input the target image into the semantic segmentation model to obtain the semantic segmentation result;
[0190] The semantic segmentation model mentioned above was trained using the aforementioned semantic segmentation model training method.
[0191] Optionally, the computer-readable storage medium may include: ROM (Read Only Memory), RAM (Random Access Memory), SSD (Solid State Drives), or optical disc, etc. The random access memory may include ReRAM (Resistance Random Access Memory) and DRAM (Dynamic Random Access Memory).
[0192] In an exemplary embodiment, a computer program product or computer program is also provided, which includes computer instructions stored in a computer-readable storage medium. A processor of a computer device reads the computer instructions from the computer-readable storage medium and executes the computer instructions, causing the computer device to perform the aforementioned semantic segmentation model training method or semantic segmentation method.
[0193] It should be understood that "multiple" as used herein refers to two or more. "And / or" describes the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A alone, A and B simultaneously, or B alone. The character " / " generally indicates that the preceding and following related objects are in an "or" relationship. Furthermore, the step numbers described herein are merely illustrative of one possible execution order. In some other embodiments, the steps may not be executed in numerical order, such as two steps with different numbers being executed simultaneously, or two steps with different numbers being executed in the reverse order of the illustration. This application does not limit this.
[0194] In addition, in the specific embodiments of this application, data such as user information are involved. When the above embodiments of this application are applied to specific products or technologies, user permission or consent is required, and the collection, use and processing of related data must comply with the relevant laws, regulations and standards of the relevant countries and regions.
[0195] The above are merely exemplary embodiments of this application and are not intended to limit this application. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this application shall be included within the protection scope of this application.
Claims
1. A semantic segmentation model training method, characterized in that, The method includes: Obtain sample images, where each sample instance in the sample image has a corresponding category label; The sample image is input into a preset model, which triggers the preset model to perform the following operations: downsampling the sample image to obtain at least one first feature; upsampling the at least one first feature to obtain at least one second feature; and performing semantic segmentation on any second feature to obtain a segmentation result. Based on any of the second features and each of the category labels, the corresponding instance feature loss is determined, wherein the instance feature loss indicates the feature loss at the sample instance granularity; Based on the segmentation results and the category labels, determine the semantic segmentation loss; Based on the instance feature loss and the semantic segmentation loss, the parameters of the preset model are adjusted to obtain the semantic segmentation model.
2. The method according to claim 1, characterized in that, The instance feature loss includes at least one of the following: Intra-class loss is used to indicate the difference in pixel features corresponding to sample instances of the same class determined based on the second feature; Inter-class loss is used to indicate the similarity of different pixel features of sample instances of different categories determined based on the second feature.
3. The method according to claim 2, characterized in that, The step of determining the corresponding instance feature loss based on any of the second features and each of the category labels includes: For any category label, determine the corresponding feature mean, which is the average pixel feature of the sample instance with the category label in the second feature; For each sample instance with the category label, the pixel features are determined based on the difference between the pixel features and the feature mean. Based on the pixel feature loss corresponding to the pixel features of each sample instance with the category label, determine the single-class loss corresponding to the category label; The intra-class loss is obtained based on the single-class loss corresponding to each of the aforementioned category labels.
4. The method according to claim 2, characterized in that, The step of determining the corresponding instance feature loss based on any of the second features and each of the category labels includes: Determine a target set, wherein any combination of labels in the target set includes two different category labels, and the two category labels in different combinations of labels are not exactly the same; For any combination of labels, determine the mean feature value corresponding to each category label, where the mean feature value is the average pixel feature value of the sample instance with the corresponding category label in the second feature. The label combination loss is determined based on the difference between the mean features corresponding to each category label. The inter-class loss is determined based on the label combination loss corresponding to each label combination in the target set.
5. The method according to claim 3, characterized in that, The step of determining the corresponding pixel feature loss based on the difference between the pixel features and the feature mean includes: If the difference between the pixel feature and the mean feature is less than a first difference threshold, the pixel feature loss is determined as a target preset value, and the target preset value indicates that the corresponding pixel feature loss is not included in the corresponding single-class loss.
6. The method according to claim 4, characterized in that, The determination of the label combination loss based on the difference in the mean features corresponding to each category label includes: If the difference between the mean features corresponding to each category label is greater than the second difference threshold, the label combination loss is determined as a target preset value, and the target preset value indicates that the corresponding label combination loss is not included in the inter-class loss.
7. The method according to claim 1, characterized in that, The preset model includes at least two downsampling layers, at least two upsampling layers, and a semantic segmentation layer connected in sequence. The step of downsampling the sample image to obtain at least one first feature includes: downsampling the sample image based on the at least two sequentially connected downsampling layers to obtain the first feature output by each downsampling layer; The step of upsampling the at least one first feature to obtain at least one second feature includes: upsampling the at least one first feature based on the at least two sequentially connected upsampling layers to obtain the second feature output by each of the upsampling layers; The step of semantic segmentation of any of the second features includes: performing semantic segmentation on the second feature output by the upsampling layer connected to the semantic segmentation layer to obtain the segmentation result.
8. A semantic segmentation method, characterized in that, The method includes: Acquire the target image; The target image is input into the semantic segmentation model to obtain the semantic segmentation result; The semantic segmentation model is trained using the semantic segmentation model training method described in any one of claims 1 to 7.
9. A semantic segmentation model training device, characterized in that, The device includes: The sample acquisition module is used to acquire sample images, wherein each sample instance in the sample image has a corresponding category label; The training module is used to perform the following operations: The sample image is input into a preset model, which triggers the preset model to perform the following operations: downsampling the sample image to obtain at least one first feature; upsampling the at least one first feature to obtain at least one second feature; and performing semantic segmentation on any second feature to obtain a segmentation result. Based on any of the second features and each of the category labels, the corresponding instance feature loss is determined, wherein the instance feature loss indicates the feature loss at the sample instance granularity; Based on the segmentation results and the category labels, determine the semantic segmentation loss; Based on the instance feature loss and the semantic segmentation loss, the parameters of the preset model are adjusted to obtain the semantic segmentation model.
10. A semantic segmentation device, characterized in that, The device includes: The image acquisition module is used to acquire the target image; The segmentation module is used to input the target image into the semantic segmentation model to obtain the semantic segmentation result; The semantic segmentation model is trained using the semantic segmentation model training method described in any one of claims 1 to 7.
11. A computer device, characterized in that, The computer device includes a processor and a memory, wherein the memory stores at least one instruction, at least one program, a code set, or an instruction set, wherein the at least one instruction, the at least one program, the code set, or the instruction set is loaded and executed by the processor to implement the semantic segmentation model training method as described in any one of claims 1 to 7, or the semantic segmentation method as described in claim 8.
12. A computer-readable storage medium, characterized in that, The storage medium stores at least one instruction, at least one program, code set, or instruction set, wherein the at least one instruction, the at least one program, the code set, or the instruction set is loaded and executed by a processor to implement the semantic segmentation model training method as described in any one of claims 1 to 7, or the semantic segmentation method as described in claim 8.
13. A computer program product, characterized in that, The computer program product includes computer instructions, the processor of the computer device reads the computer instructions, and the processor of the computer device executes the computer instructions to implement the semantic segmentation model training method as described in any one of claims 1 to 7, or the semantic segmentation method as described in claim 8.