Multi-scale image semantic segmentation method and system

Abstract

The invention relates to a multi-scale image semantic segmentation method and system, and the method comprises the steps: extracting an original image feature of a target image based on an encoder, and obtaining a continuous feature vector of the target image; discretizing the continuous feature vector to obtain a discrete feature vector; performing scale-by-scale prediction according to a scale autoregression model based on the discrete feature vector to obtain a predicted image feature consistent with the resolution of the target image; and based on the predicted image features, obtaining a predicted classification result of each pixel of the target image according to a decoder so as to construct a multi-scale image semantic segmentation model. According to the method, the image segmentation task can be regarded as an image generation task, objects of different scales can be segmented, the universality is higher, and the method has better performance in edge detail processing and global information integration.

Description

Technical Field

[0001] The present application relates to the field of computer vision, and in particular to a multi-scale image semantic segmentation method and system. Background Art

[0002] Semantic segmentation is a core task in computer vision, which aims to assign a category label to each pixel in an image. Unlike image classification, which only focuses on the label of the entire image, the semantic segmentation task focuses on the pixel-level understanding of the image. Each pixel is classified as an object category, which makes semantic segmentation widely used in autonomous driving, medical imaging, remote sensing images, industrial inspection and other fields. In the past few decades, semantic segmentation technology has gone through several stages of development, especially after the introduction of deep learning methods, semantic segmentation technology has made a qualitative leap.

[0003] Early semantic segmentation methods were based on traditional image processing and machine learning algorithms, such as graph cuts and image watershed algorithms. Traditional semantic segmentation research mainly focuses on image segmentation methods based on manual feature extraction. For example, segmentation methods based on low-level visual features such as edges, textures, and colors include the use of classic algorithms such as K-means clustering, graph cuts, and region growing. The graph cut method divides the image into different regions by optimizing the energy function, while the region growing method achieves segmentation by gradually merging similar pixels. However, traditional image segmentation methods usually lack an understanding of the global information of the image and cannot effectively capture the complex relationships between objects, especially in the presence of noise or complex backgrounds, where the performance is greatly reduced.

[0004] The current mainstream methods basically rely on deep neural networks, especially convolutional neural networks (CNN) and their variants. With the rapid development of deep learning, especially convolutional neural networks (CNN), semantic segmentation technology has made great progress. In 2014, Long et al. proposed the fully convolutional network (FCN), used CNN for semantic segmentation, and pioneered the study of semantic segmentation in the era of deep learning. The core idea of ​​FCN is to replace the fully connected layer with the convolution layer, so that the network can process input images of any size and output segmentation maps of the same size as the input image. U-Net is a structure proposed for the problem of medical image segmentation. It has a symmetrical "U"-shaped architecture and uses a large number of skip connections to combine low-level and high-level features, which improves the recovery of details and is particularly suitable for segmentation tasks with small images and complex targets. DeepLab is a series of semantic segmentation methods proposed by Google, which is characterized by the use of dilated convolution to improve the perception of multi-scale objects by expanding the receptive field of the convolution kernel. In addition, DeepLabv3 introduces spatial pyramid pooling (ASPP) to further enhance the ability to capture global context information. PSPNet introduces a pyramid pooling module and combines feature maps of different scales to improve the expression of global context information. Its advantage is that it has a good performance in multi-scale object segmentation in complex scenes.

[0005] Although existing deep learning-based methods have performed well in semantic segmentation tasks, they still have shortcomings in edge detail processing, global information integration, and multi-scale issues.

[0006] Therefore, there is an urgent need to provide a multi-scale image semantic segmentation method and system to solve the technical problems raised above. Summary of the invention

[0007] The present application provides a multi-scale image semantic segmentation method, system, device and medium, which regards the image segmentation task as an image generation task, can segment objects of different scales, has higher versatility, and has better performance in edge detail processing and global information integration.

[0008] In a first aspect, the present application provides a multi-scale image semantic segmentation method, comprising the following steps: extracting original image features of a target image based on an encoder and obtaining a continuous feature vector of the target image; discretizing the continuous feature vector to obtain a discrete feature vector; performing scale-by-scale prediction according to a scale autoregressive model based on the discrete feature vector to obtain predicted image features consistent with the resolution of the target image; and obtaining a predicted classification result for each pixel of the target image according to a decoder based on the predicted image features to construct a multi-scale image semantic segmentation model.

[0009] Optionally, the continuous feature vector is discretized to obtain a discrete feature vector, including: presetting an initialization codebook; retrieving a matching feature vector similar to each original feature vector of the target image in the codebook; replacing the corresponding original feature vector with each matching feature vector to achieve feature discretization of the target image and obtain a discretized space.

[0010] Optionally, retrieving a matching feature vector similar to each original feature vector of the target image in the codebook includes: calculating similarity between the original feature vectors and the codebook vectors in the codebook respectively; and determining the codebook vector with the smallest calculated similarity as the matching feature vector most similar to the original feature vector.

[0011] Optionally, performing similarity calculation on the original feature vector and the codebook vector in the codebook respectively includes: calculating the Euclidean distance, cosine distance or Manhattan distance between the original feature vector and the codebook vector.

[0012] Optionally, a scale-by-scale prediction is performed based on the discrete feature vector according to a scale autoregressive model to obtain predicted image features consistent with the resolution of the target image, including: based on the discrete feature vector, a scale autoregressive model is used to perform scale-by-scale prediction in a discretized space in an order from coarse to fine and from global contour to local refinement to obtain predicted image features consistent with the resolution of the target image.

[0013] Optionally, based on the predicted image features, a predicted classification result of each pixel of the target image is obtained according to the decoder to construct a multi-scale image semantic segmentation model, including: generating a probability distribution of category labels based on the obtained predicted image features consistent with the resolution of the target image; obtaining the predicted classification result of each pixel of the target image through the probability distribution of the category labels; wherein the predicted classification results can represent different categories by different colors.

[0014] Optionally, the multi-scale image semantic segmentation method further includes: calculating the gap between the predicted classification result and the actual result to further train the multi-scale image semantic segmentation model.

[0015] Optionally, the gap between the predicted result and the actual result is calculated to further train the multi-scale image semantic segmentation model, including: calculating the gap between the predicted classification result and the actual image label according to a cross entropy loss function to train the multi-scale image semantic segmentation model to improve image resolution.

[0016] In the second aspect, the present application provides a multi-scale image semantic segmentation system, which includes: an acquisition module, used to extract the original image features of the target image based on the encoder, and obtain a continuous feature vector of the target image; a discretization module, used to discretize the continuous feature vector to obtain a discrete feature vector; a processing module, used to perform scale-by-scale prediction based on the discrete feature vector according to a scale autoregressive model to obtain predicted image features consistent with the resolution of the target image; and a model construction module, used to obtain a predicted classification result for each pixel of the target image according to the decoder based on the predicted image features to form a multi-scale image semantic segmentation model.

[0017] In a third aspect, the present application also provides a computer device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the steps of the above-described method when executing the computer program.

[0018] In a fourth aspect, a computer-readable storage medium is provided, on which a computer program is stored, and when the computer program is executed by a processor, the steps of the above-mentioned method are implemented.

[0019] This application has at least the following advantages: The above steps mainly extract the original image features of the target image through the encoder and obtain the continuous feature vector of the target image, then discretize the continuous feature vector of the target image to obtain a discrete feature vector, and input the discrete feature vector into the scale autoregression model for scale-by-scale prediction until the predicted image features consistent with the resolution of the target image are obtained. Finally, based on the predicted image features, the predicted classification results of each pixel of the target image are obtained according to the decoder, and a multi-scale image semantic segmentation model is constructed according to the predicted classification results. The method of the present application regards the image segmentation task as an image generation task, which is more versatile and can be better integrated with other tasks, such as Wensheng images, Wensheng videos, etc., and is based on the scale autoregression method. It can segment objects of different scales, has higher versatility, and has better performance in edge detail processing and global information integration. BRIEF DESCRIPTION OF THE DRAWINGS

[0020] Figure 1 A schematic diagram showing the steps of a multi-scale image semantic segmentation method in one embodiment; Figure 2 A flowchart showing a multi-scale image semantic segmentation method in one embodiment; Figure 3 A comparison diagram showing an output image and an image semantic segmentation result in one embodiment; Figure 4 A structural block diagram showing a multi-scale image semantic segmentation system in one embodiment; Figure 5 FIG. 4 is a schematic structural diagram of a computer device in one embodiment. DETAILED DESCRIPTION

[0021] The present application is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application.

[0022] It should be noted that the terms used herein are only for describing specific embodiments and are not intended to limit the exemplary embodiments according to the present application. As used herein, unless the context clearly indicates otherwise, the singular form is also intended to include the plural form. In addition, it should be understood that when "including" and / or "comprising" are used in this specification, it indicates the presence of features, steps, operations, devices, components and / or combinations thereof.

[0023] To make the purpose, technical scheme and advantages of the embodiments of the present application clearer, the embodiments of the present application will be described in detail below in conjunction with the accompanying drawings. However, it will be appreciated by those skilled in the art that in the embodiments of the present application, many technical details are proposed in order to enable the reader to better understand the present application. However, even without these technical details and various changes and modifications based on the following embodiments, the technical scheme claimed in the present application can also be implemented. The markings of the following embodiments are for convenience of description and should not constitute any limitation to the specific implementation of the present application. The various embodiments can be combined with each other and quoted from each other without contradiction.

[0024] Figure 1 A flowchart of a multi-scale image semantic segmentation method provided in an embodiment of the present application is shown in FIG. Figure 1 , Figure 2 As shown, the method may include the following steps: S201, extracting original image features of a target image based on an encoder, and obtaining a continuous feature vector of the target image; S202, discretizing the continuous feature vector to obtain a discrete feature vector; S203, performing scale-by-scale prediction according to a scale autoregressive model based on the discrete feature vector to obtain predicted image features consistent with the resolution of the target image; S204, obtaining a predicted classification result for each pixel of the target image according to the decoder based on the predicted image features to construct a multi-scale image semantic segmentation model.

[0025] In the embodiment of the present application, it should be noted that image semantic segmentation is an important branch in the field of artificial intelligence computer vision. It combines technologies such as image classification, target detection and image segmentation, and mainly classifies images at the pixel level. The result of semantic segmentation is to turn the image into a color block with certain semantic information. Semantic segmentation technology can identify the semantic category of each color block and mark each pixel with its corresponding label, thereby realizing the semantic reasoning process from the bottom layer to the high layer, and finally obtaining a segmentation map with semantic annotation information of each pixel. Here, the original image features of the target image are extracted by the encoder, and the continuous feature vector of the target image is obtained, and then the continuous feature vector of the target image is discretized to obtain a discrete feature vector, and the discrete feature vector is input into the scale autoregressive model for scale-by-scale prediction until the predicted image features consistent with the resolution of the target image are obtained, and finally the predicted classification results of each pixel of the target image are obtained according to the decoder based on the predicted image features, and a multi-scale image semantic segmentation model is constructed according to the predicted classification results, so that the image segmentation task can be regarded as an image generation task, and objects of different scales can be segmented, which is more versatile and has better performance in edge detail processing and global information integration.

[0026] The following is a detailed description of each step: Please refer to Figure 1 , Figure 2 As shown, step S201, extracting original image features of a target image based on an encoder, and obtaining a continuous feature vector of the target image.

[0027] In this embodiment, it should be noted that the encoder includes a structure based on a convolutional neural network, such as a ResNet structure, and may also be a structure based on a Transformer, such as a structure based on a Swin Transformer.

[0028] In an example, a RGB three-channel image , where H and W are the height and width of the image. After passing through the encoder network, the continuous feature vector of the target image is obtained, i.e., the feature map .

[0029] Please continue to refer to Figure 1 , Figure 2 As shown, step S202, discretize the continuous feature vector to obtain a discrete feature vector.

[0030] In this embodiment, it should be noted that an initialization codebook must first be preset; then, matching feature vectors similar to each original feature vector of the target image are retrieved in the codebook; finally, each matching feature vector replaces the corresponding original feature vector to achieve feature discretization of the target image and obtain a discretization space; wherein the discretization space is a space for completing the discretization of the original feature vector, and by introducing a feature discretization scheme, the image segmentation task has generalization properties.

[0031] Specifically, in one example, the preset initialization codebook is to pre-generate a discrete codebook , where there are N c-dimensional discrete vectors. Find and Each matching feature vector with similar encoding replaces the corresponding original feature vector to generate a matching feature vector. Variables with the same dimensions , the discrete encoding process can be expressed as: In addition, the original feature vector is respectively similar to the codebook vector in the codebook, specifically, the Euclidean distance, cosine distance or Manhattan distance between the original feature vector and the codebook vector is calculated. Find and Each matching feature vector with similar encoding is in Find and The closest vector in each encoding position of is obtained to obtain the matching feature vector.

[0032] In an example, a preset codebook consists of 100 codebook vectors, and the Euclidean distances of the feature vectors of the target image are calculated with these 100 codebook vectors respectively. The corresponding codebook vector with the smallest Euclidean distance is the vector most similar to the feature vector of the target image. Then, the feature vector of the target image is replaced with the most similar codebook vector, which is the feature discretization process.

[0033] Reference Figure 1 , Figure 2 As shown, step S203, based on the discrete feature vector, scale-by-scale prediction is performed according to the scale autoregressive model to obtain predicted image features consistent with the resolution of the target image; In this embodiment, it should be noted that scale-by-scale prediction is achieved based on discrete feature vectors according to the scale autoregressive model in the discretization space in an order from coarse to fine, from global contour to local refinement, until the predicted image features consistent with the resolution of the target image are obtained.

[0034] In this embodiment, it should be noted that the obtained discrete feature vector is used as an input value in the discretized space for training according to the scale autoregressive model, and the image is adjusted from coarse to fine, from global contour to local fine-tuning, to achieve scale-by-scale prediction of the image. On feature maps of different scales, the autoregressive model aggregates the information of the context associated with the feature vector using operators such as convolution, and uses the context information to predict higher-resolution image feature details. Compared with the previous "pixel-by-pixel" method, that is, generating row by row from the upper left corner of the image, this application retains the spatial structure and scale information of the image as much as possible, and does not lose context information due to reasons such as blocking, so as to better capture and learn complex visual distributions and achieve scale-by-scale prediction. Among them, from coarse to fine means that the resolution is getting higher and higher, and scale by scale also means that the resolution is getting higher and higher, and one resolution corresponds to one scale.

[0035] In one example, after obtaining the discretized vector Then, the discretized vector As input data, the scale autoregressive model is used to predict the next scale of the feature vector from coarse to fine, and the result is the same as The same size and quantity , that is, the predicted image features are obtained that are consistent with the resolution of the target image.

[0036] Reference Figure 1 , Figure 2 As shown, step S204, based on the predicted image features, the predicted classification result of each pixel of the target image is obtained according to the decoder to construct a multi-scale image semantic segmentation model; In this embodiment, it should be noted that the predicted image features The predicted classification result of each pixel of the target image is obtained by inputting it into the decoder. Specifically, the predicted image features in the discretized space are input into the decoder to generate a probability distribution of the category label, and the predicted classification result of each pixel of the target image is obtained through the probability distribution of the category label; wherein, the predicted classification result can be represented by different colors for different categories; the category label can connect the feature map to a multi-layer perceptron to obtain a heat map with the same resolution as the original image, and the pixel value on the heat map is the category label of each pixel.

[0037] For example, suppose the target image has 45 labels, that is, the model predicts 45 numbers, the sum of which is 1, and each number represents the probability of the 45 labels, thus obtaining the probability distribution of each pixel. Figure 3 As shown in the figure, by displaying these 45 types of labels in different colors, the image segmentation can be more intuitive. By considering the image segmentation task as an image generation task, efficient integration of global information is achieved, which is more versatile.

[0038] In addition, a multi-scale image semantic segmentation model is constructed by predicting and classifying each pixel of the target image, where the multi-scale includes any number of downsampling, such as ½, ¼, 1 / 8, 1 / 1024, etc. Specifically, for image downsampling, for example, the original image resolution is w×h, from coarse to fine, first predict a low-resolution feature map, such as w / 8×h / 8, and compare it with the resolution of the target image. If it is different, predict it, and then predict a feature with a higher resolution, such as w / 4×h / 4, until the predicted feature map has the same resolution as the target image. By constructing a multi-scale image semantic segmentation model, the method can take into account the segmentation of objects of all scales, and can take into account both smaller and finer objects, such as telephone poles, and larger objects, such as buildings.

[0039] Reference Figure 1 , Figure 2 As shown, step S205, calculating the difference between the predicted result and the actual result to further train the multi-scale image semantic segmentation model; In this embodiment, it should be noted that the multi-scale image semantic segmentation model is trained to improve the image resolution by calculating the gap between the prediction result and the real image label according to the cross entropy loss function.

[0040] The cross-entropy loss function is a commonly used loss function in machine learning and deep learning, and plays an important role in classification tasks. It is based on the concept of cross in information theory, and is used to measure the difference between the model's predicted probability distribution and the true label distribution, and is used to optimize parameters in model training. By using the cross-entropy loss function to optimize the multi-scale image semantic segmentation model, it is easier to guide the model parameters to adjust in the right direction, especially when the error is large, the weight update speed is faster, and because its loss value is directly related to the predicted probability, it can help the model avoid overfitting to a certain extent. Input the preprocessed test image, and use the multi-scale image semantic segmentation model to output the predicted image of the test image, which is the image semantic segmentation result image corresponding to the test image.

[0041] The implementation principle of this embodiment: The above steps mainly extract the original image features of the target image through the encoder, and obtain the continuous feature vector of the target image, then discretize the continuous feature vector of the target image to obtain a discrete feature vector, and input the discrete feature vector into the scale autoregression model for scale-by-scale prediction until the predicted image features consistent with the resolution of the target image are obtained, and finally the predicted image features are input into the decoder to generate the category label of each pixel of the target image, and the classification result can be obtained according to the probability distribution of the category label. The method of the present application regards the image segmentation task as an image generation task, and uses the network end-to-end to generate the category label of the original input image. Through the scale autoregression model, the method can realize the segmentation of objects of different scales, with higher versatility and better performance in edge detail processing and global information integration.

[0042] Reference Figure 4 As shown, the present application also provides a multi-scale image semantic segmentation system, which may include: an acquisition module 301, a discretization module 302, a processing module 303, and a model building module 304. The main functions of each component module are as follows: An acquisition module 301 is used to extract original image features of a target image based on an encoder and obtain a continuous feature vector of the target image; A discretization module 302 is used to discretize the continuous feature vector to obtain a discrete feature vector; A processing module 303 is used to perform scale-by-scale prediction based on the discrete feature vector according to a scale autoregressive model to obtain predicted image features consistent with the resolution of the target image; The model building module 304 is used to obtain the predicted classification result of each pixel of the target image according to the decoder based on the predicted image features to build a multi-scale image semantic segmentation model.

[0043] like Figure 5 , is a block diagram of a computer device according to an embodiment of the present application. The computer device is intended to represent various forms of digital computers or mobile devices. The digital computer may include a desktop computer, a portable computer, a workbench, a personal digital assistant, a server, a mainframe computer, and other suitable computers. The mobile device may include a tablet computer, a smart phone, a wearable device, etc.

[0044] like Figure 5 As shown, the device 600 includes a computing unit 601, a ROM 602, a RAM 603, a bus 604, and an input / output (I / O) interface 605. The computing unit 601, the ROM 602, and the RAM 603 are connected to each other via the bus 604. The input / output (I / O) interface 605 is also connected to the bus 604.

[0045] The computing unit 601 may perform various processes in the method embodiment of the present application according to the computer instructions stored in the read-only memory (ROM) 602 or the computer instructions loaded from the storage unit 608 to the random access memory (RAM) 603. The computing unit 601 may be various general and / or special processing components with processing and computing capabilities. The computing unit 601 may include, but is not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various dedicated artificial intelligence (AI) computing chips, various computing units running machine learning model algorithms, digital signal processors (DSPs), and any appropriate processors, controllers, microcontrollers, etc. In some embodiments, the method provided in the embodiment of the present application may be implemented as a computer software program, which is tangibly contained in a computer-readable storage medium, such as the storage unit 608.

[0046] RAM 603 may also store various programs and data required for the operation of device 600. Part or all of a computer program may be loaded and / or installed on device 600 via ROM 602 and / or communication unit 609.

[0047] The input unit 606, output unit 607, storage unit 608 and communication unit 609 in the device 600 can be connected to the I / O interface 605. The input unit 606 can be, for example, a keyboard, a mouse, a touch screen, a microphone, etc. The output unit 607 can be, for example, a display, a speaker, an indicator light, etc. The device 600 can exchange information, data, etc. with other devices through the communication unit 609. It should be noted that the device may also include other components necessary for normal operation, or may only include components necessary for implementing the solution of the present application, rather than all the components shown in the figure.

[0048] Various implementations of the systems and techniques described here can be realized in digital electronic circuitry, integrated circuitry, field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), systems on chips (SOCs), load programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations of them.

[0049] The computer instructions for implementing the method of the present application can be written in any combination of one or more programming languages. These computer instructions can be provided to the computing unit 601, so that when the computer instructions are executed by the computing unit 601 such as a processor, the steps involved in the method embodiment of the present application are executed.

[0050] The computer-readable storage medium provided in the present application may be a tangible medium that may contain or store computer instructions for executing the steps involved in the method embodiments of the present application. The computer-readable storage medium may include, but is not limited to, electronic, magnetic, optical, electromagnetic, and other forms of storage media.

[0051] The above specific implementations do not constitute a limitation on the protection scope of this application. It should be understood by those skilled in the art that various modifications, combinations, sub-combinations and substitutions can be made according to design requirements and other factors. Any modifications, equivalent substitutions and improvements made within the spirit and principles of this application should be included in the protection scope of this application.

Claims

1. A multi-scale image semantic segmentation method, characterized in that: The following steps are involved: Extracting original image features of the target image based on the encoder and obtaining a continuous feature vector of the target image; discretizing the continuous feature vector to obtain a discrete feature vector; Based on the discrete feature vector, a scale-by-scale prediction is performed according to a scale autoregressive model to obtain predicted image features consistent with the resolution of the target image; Based on the predicted image features, a predicted classification result of each pixel of the target image is obtained according to the decoder to construct a multi-scale image semantic segmentation model.

2. The multi-scale image semantic segmentation method according to claim 1, characterized in that: The step of discretizing the continuous feature vector to obtain a discrete feature vector includes: Preset initialization codebook; Retrieving a matching feature vector similar to each original feature vector of the target image in the codebook; Each matching feature vector replaces the corresponding original feature vector to achieve feature discretization of the target image and obtain a discretization space.

3. The multi-scale image semantic segmentation method according to claim 2, characterized in that: The retrieving a matching feature vector similar to each original feature vector of the target image in the codebook comprises: Performing similarity calculations on the original feature vectors and the codebook vectors in the codebook respectively; The codebook vector with the smallest calculated similarity is determined as the matching feature vector that is most similar to the original feature vector.

4. The multi-scale image semantic segmentation method according to claim 3, characterized in that: The calculating similarity between the original feature vector and the codebook vector in the codebook respectively includes: The Euclidean distance, cosine distance or Manhattan distance between the original feature vector and the codebook vector is calculated.

5. The multi-scale image semantic segmentation method according to claim 2, characterized in that: The step of performing scale-by-scale prediction based on the discrete feature vector according to a scale autoregressive model to obtain predicted image features consistent with the resolution of the target image includes: Based on the discrete feature vector, according to the scale autoregressive model, in the discretization space, the scale-by-scale prediction is realized in the order from coarse to fine, from global contour to local fine-tuning, so as to obtain the predicted image features consistent with the resolution of the target image.

6. The multi-scale image semantic segmentation method according to claim 5, characterized in that: The method of obtaining a predicted classification result for each pixel of the target image according to the decoder based on the predicted image features to construct a multi-scale image semantic segmentation model includes: Generating a probability distribution of a category label based on the obtained predicted image features consistent with the resolution of the target image; The predicted classification result of each pixel of the target image is obtained through the probability distribution of the category label; wherein the predicted classification result can represent different categories by different colors.

7. The multi-scale image semantic segmentation method according to claim 1, characterized in that: The following steps are also included: The gap between the predicted classification result and the actual result is calculated to further train the multi-scale image semantic segmentation model.

8. The multi-scale image semantic segmentation method according to claim 7, characterized in that: The step of calculating the difference between the predicted classification result and the actual result and further training the multi-scale image semantic segmentation model includes: The multi-scale image semantic segmentation model is trained to improve image resolution by calculating the gap between the predicted classification result and the real image label according to the cross entropy loss function.

9. A multi-scale image semantic segmentation system, characterized in that: include: An acquisition module, used for extracting original image features of a target image based on an encoder, and obtaining a continuous feature vector of the target image; A discretization module, used for discretizing the continuous feature vector to obtain a discrete feature vector; A processing module, configured to perform scale-by-scale prediction based on the discrete feature vector according to a scale autoregressive model to obtain predicted image features consistent with the resolution of the target image; A model building module is used to obtain a predicted classification result of each pixel of the target image according to the decoder based on the predicted image features to build a multi-scale image semantic segmentation model.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 8 are implemented.

Images