Lightweight network-based scene segmentation method and system

Abstract

The application discloses a scene segmentation method and system based on a lightweight network, and belongs to the technical field of scene segmentation. The method comprises the following steps: acquiring a road scene image; inputting the road scene image into a trained lightweight network to perform scene segmentation and acquire a predicted output image; the training process of the lightweight network comprises the following steps: dividing a training set into a support set and a query set, inputting the support set into an encoder, extracting features by adopting a convolution operation through a backbone network with fixed parameters, acquiring middle-level features and high-level features, generating a prototype vector, and constructing a prior mask; splicing the prior mask, the prototype vector and the middle-level features, and performing a hollow convolution to acquire a feature fusion image; combining a transformer, a convolution and a bilinear difference in a decoding stage to restore the information of the input image and acquire the predicted output image; and optimizing the lightweight network through a DICE loss function. The method can balance real-time performance and accuracy, and solves the problems of low segmentation accuracy, tight computing resources and long training time in the prior art.

Description

Technical Field

[0001] This invention relates to the field of scene segmentation technology, and in particular to a scene segmentation method and system based on lightweight networks. Background Technology

[0002] The statements in this section merely refer to the background art related to this invention and do not necessarily constitute prior art.

[0003] Scene segmentation, also known as semantic segmentation, is one of the main tasks in computer vision, aiming to perform precise pixel-level segmentation of images. Semantic segmentation uses color to distinguish categories, with each category corresponding to a specific color, thereby obtaining useful semantic category information for image segmentation. Images are mainly composed of pixels, and different categories are associated with different colors, resulting in different RGB values. By referring to the pixels in the surrounding area, it is possible to determine which category the region belongs to. Then, prediction is made using the category of each pixel, ultimately yielding a segmented image containing semantic information.

[0004] Scene segmentation is currently widely used in various fields, including autonomous driving, medical image analysis, remote sensing image interpretation, drone and aerial imagery, video analysis, portrait segmentation, virtual reality and augmented reality, agriculture, environmental protection and resource management, and game development. Through semantic segmentation, accurate identification and localization of objects, structures, and scenes in images can be achieved, bringing more refined and intelligent visual solutions to various industries.

[0005] As society progresses and the number of cars continues to rise, autonomous driving technology is gaining increasing attention as vehicles become more versatile. In the field of autonomous driving, scene segmentation is a crucial cornerstone, and accurate scene segmentation forms the solid foundation for object detection. Onboard cameras or radar input real-time road condition images, which are then processed by lightweight scene segmentation algorithms to output images with semantic information. Each pixel in the image is accurately assigned to a specific category or object, achieving a high-precision understanding of the surrounding environment. This enables autonomous vehicles to accurately identify various elements such as roads, lane lines, traffic signs, and pedestrians, thus providing reliable data support for driving decisions.

[0006] However, in autonomous driving, in complex road conditions, vehicles or buildings may obscure parts of the road, causing some objects or regions in the image to be unable to be fully observed. This occlusion and incompleteness increases the difficulty of semantic segmentation tasks and affects the accuracy of segmentation.

[0007] Because there may be a significant imbalance in the number of different categories on the road (such as vehicles, pedestrians, road signs, etc.), the model may not learn enough for certain categories, thus affecting accuracy.

[0008] If we simply improve the scene segmentation algorithm to increase its accuracy, the algorithm will become more complex, requiring more memory and time for neural network training. This will also lead to real-time delays, which will not meet the real-time requirements of autonomous driving. Summary of the Invention

[0009] To address the shortcomings of existing technologies, this invention provides a scene segmentation method, system, electronic device, and computer-readable storage medium based on a lightweight network. In the encoding stage, convolution is used for feature extraction. In the decoding stage, information of the input image is recovered by combining transformer, convolution, and bilinear interpolation operations to generate an output image with semantic information. Finally, DICE Loss is used to calculate the loss difference between the predicted image and the real image to obtain the optimal result and output the predicted image.

[0010] In a first aspect, the present invention provides a scene segmentation method based on a lightweight network; A scene segmentation method based on lightweight networks includes: Acquire road scene images; The road scene image is input into the trained lightweight network for scene segmentation to obtain the predicted output image; The training process for lightweight networks includes: The training set is divided into a support set and a query set, which are input into the encoder. Through a backbone network with fixed parameters, convolutional operations are used to extract features, obtain intermediate and high-level features, generate prototype vectors, and construct a priori mask. The priori mask, prototype vectors, and intermediate features are concatenated and dilated convolution is performed to obtain a feature fusion map. In the decoding stage, information from the input image is restored by combining transformer, convolution, and bilinear interpolation to obtain the predicted output image; Optimize lightweight networks using the DICE loss function.

[0011] Furthermore, the step of using convolutional operations for feature extraction to obtain intermediate and high-level features and generate prototype vectors includes: The query image and support images are input into a parallel ImageNet pre-trained backbone network for feature extraction, obtaining query intermediate features, query high-level features, support intermediate features, and support high-level features; The support mid-level features and support mask are fused through masked average pooling to generate a prototype vector.

[0012] Preferably, the prototype vector is represented as: , in, This represents the average pooling operation, and L represents the interpolation and padding operation. This indicates support for masks. This indicates support for intermediate features.

[0013] Preferably, the construction of the prior mask specifically involves: reconstructing the structure to generate prior codes based on the pixel-by-pixel relationship between the query intermediate features and the supporting intermediate features, as well as between the query high-level features and the supporting high-level features, and then performing normalization processing.

[0014] Preferably, the step of combining transformer, convolution, and bilinear interpolation to reconstruct the information of the input image during the decoding stage specifically includes: The prototype vector and the query intermediate features are input into the transformer decoder, and the query features and prototype vector are multiplied by a dot product to obtain the embedding of the predicted object segmentation mask. The fused feature map is input into the pixel decoder, and the output is the predicted feature map; The embedding of the predicted object segmentation mask and the predicted feature map are fused, and upsampling is performed using bilinear interpolation to obtain the predicted output map.

[0015] Furthermore, the step of concatenating the prior mask, prototype vector, and intermediate features, and performing dilated convolution to obtain the feature fusion map includes: Cascade intermediate features to obtain intermediate query features, concatenate the prior mask, prototype vector and intermediate query features, and perform dilated convolution operation to obtain fused feature map; Feature enhancement is performed on the fused feature map.

[0016] Furthermore, the DICE loss function is expressed as: , in, The predicted output for the image. This represents the actual label image of this pixel. This represents the weighting coefficients applied to each category. Represents the predicted output. Represents the actual label.

[0017] Secondly, the present invention provides a scene segmentation system based on a lightweight network; A scene segmentation system based on a lightweight network includes: The acquisition module is configured to acquire road scene images; The scene segmentation module is configured to input road scene images into a trained lightweight network for scene segmentation and obtain a predicted output image. The training process for lightweight networks includes: The training set is divided into a support set and a query set, which are input into the encoder. Through a backbone network with fixed parameters, convolutional operations are used to extract features, obtain intermediate and high-level features, generate prototype vectors, and construct a priori mask. The priori mask, prototype vectors, and intermediate features are concatenated and dilated convolution is performed to obtain a feature fusion map. In the decoding stage, information from the input image is restored by combining transformer, convolution, and bilinear interpolation to obtain the predicted output image; Optimize lightweight networks using the DICE loss function.

[0018] Thirdly, the present invention provides an electronic device; An electronic device includes a memory and a processor, as well as computer instructions stored in the memory and running on the processor, wherein the computer instructions, when executed by the processor, complete the steps of the scene segmentation method based on lightweight networks described above.

[0019] Fourthly, the present invention provides a computer-readable storage medium; A computer-readable storage medium for storing computer instructions, which, when executed by a processor, complete the steps of the above-described scene segmentation method based on a lightweight network.

[0020] Compared with the prior art, the beneficial effects of the present invention are: 1. The technical solution provided by this invention requires normalization of the pixels in the image to eliminate the influence of other transformation functions on image transformation, adjusting the pixel values ​​to the range of 0 to 1.

[0021] 2. The technical solution provided by this invention extracts features from images through a fixed backbone network. In the encoding stage, convolution is used for feature extraction, and intermediate and high-level features are fused and converted into prior codes. This fully learns the spatial and semantic information contained in the image, making the features richer and thus improving the accuracy of scene segmentation. At the same time, the intermediate and high-level features are pre-trained in ImageNet, and the process of generating the prior mask does not add an additional training process, thus achieving the requirement of lightweight operation.

[0022] 3. The technical solution provided by this invention, in the decoding stage, restores the information of the input image by combining transformer, convolution and bilinear interpolation operations, and generates an output image with semantic information; finally, DICE Loss is used to calculate the loss difference between the predicted image and the real image to obtain the optimal result and output the predicted image.

[0023] 4. The technical solution provided by this invention can optimize autonomous driving technology, meet the requirements of real-time performance and accuracy of autonomous driving, and overcome the problems of object obstruction, complex road conditions, limited computing resources, and long training time. Attached Figure Description

[0024] The accompanying drawings, which form part of this invention, are used to provide a further understanding of the invention. The illustrative embodiments of the invention and their descriptions are used to explain the invention and do not constitute an improper limitation of the invention.

[0025] Figure 1 A flowchart illustrating the scene segmentation method based on lightweight networks provided in an embodiment of the present invention; Figure 2 This is a schematic diagram of the image preprocessing process provided in an embodiment of the present invention; Figure 3 A schematic diagram of the framework of a lightweight network provided in an embodiment of the present invention; Figure 4 This is a schematic diagram of the SASPP module provided in an embodiment of the present invention. Detailed Implementation

[0026] It should be noted that the following detailed descriptions are exemplary and intended to provide further illustration of the invention. Unless otherwise specified, all technical and scientific terms used in this invention have the same meaning as commonly understood by one of ordinary skill in the art to which this invention pertains.

[0027] It should be noted that the terminology used herein is for the purpose of describing particular embodiments only and is not intended to limit the exemplary embodiments of the present invention. As used herein, unless the context clearly indicates otherwise, the singular form is also intended to include the plural form. Furthermore, it should be understood that the terms “comprising” and “having”, and any variations thereof, are intended to cover non-exclusive inclusion, for example, a process, method, system, product, or apparatus that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus.

[0028] Where there is no conflict, the embodiments and features in the embodiments of the present invention can be combined with each other.

[0029] Example 1 During autonomous driving, issues such as overlapping and occlusion of objects, optical fibers, and complex road conditions can lead to complex and blurry features in the acquired road scene images, making them difficult to segment or resulting in low segmentation accuracy. However, simply improving the neural network to increase accuracy would require a relatively large amount of memory and time for training, affecting the real-time performance of scene segmentation. Therefore, this invention provides a scene segmentation method based on a lightweight network.

[0030] Next, combined Figures 1-4 This embodiment provides a detailed description of a scene segmentation method based on lightweight networks. The scene segmentation method based on lightweight networks includes the following steps: S1. Acquire road scene images and perform preprocessing. The road scene images are acquired through onboard cameras or radar of autonomous vehicles.

[0031] Specifically, the preprocessing of the road scene image involves pixel normalization.

[0032] S2. Input the road scene image into the trained lightweight network for scene segmentation and obtain the predicted output image.

[0033] The lightweight network uses ResNet50 as the backbone for its encoding part and Transformer as the decoder for its decoding part.

[0034] As one implementation method, the training process of a lightweight network includes: (1) Divide the training set into a support set and a query set, input them into the encoder, and use convolution operation to extract features through a backbone network with fixed parameters to obtain intermediate and high-level features, generate prototype vectors, and construct a priori mask; concatenate the priori mask, prototype vectors and intermediate features, and perform dilated convolution to obtain a feature fusion map.

[0035] (2) In the decoding stage, combine transformer, convolution and bilinear interpolation to restore the information of the input image and obtain the predicted output image.

[0036] (3) Optimize the lightweight network using the DICE loss function.

[0037] Specifically, the training process for a lightweight network is as follows: Step 1: Collect driving videos and extract road scene images from the video frames. Accurately label these road scene images and define the category label for each image to create training and testing sets.

[0038] For example, step 1 includes: Step 101: Since the video is formed by rapidly changing images, the driving video is segmented into a series of road scene images. A dataset is formed by extracting one road scene image every 10 frames.

[0039] Step 102: Assign a label to each pixel in each road scene image, with each category corresponding to a specific number, and obtain the corresponding binary image.

[0040] When labeling a large number of pixels one by one, it is inevitable to waste a lot of manpower and labeling errors will occur. By taking the above steps, the number of images required can be reduced, the network performance can be improved, and manpower can be effectively saved.

[0041] Step 103: Based on the training set, randomly crop and augment the input and labeled images in the training set to expand the training set; if the image size is larger than the original image, crop from random points; if the image size is smaller than the original image, fill the boundaries; finally, perform random horizontal or vertical flipping operations. Step 2: Calculate the mean and variance of each pixel in the image, build a background model, normalize the pixels, and extract scene features. Specifically, this includes: Step 201: Calculate the average value and variance of all image pixels to obtain the background model.

[0042] Step 202: Subtract the mean from the image and divide by the variance to obtain data that conforms to a normal distribution. This process not only eliminates the influence of the average image brightness but also enhances the accuracy of network calculations to some extent.

[0043] Step 3: Divide the preprocessed training set into a support set and a query set and input them into a lightweight convolutional network for training. During the training process, use the backbone network to extract features and use transformer and convolution to fuse feature maps, which ensures speed and effectively reduces the error rate.

[0044] Referring to Figure 3, the specific process of step 3 is as follows: Step 301: Input the query images in the query set and the support images in the support set into the parallel ImageNet pre-trained backbone network for feature extraction, and obtain the corresponding query intermediate features, query high-level features, support intermediate features and support high-level features. Here, the query intermediate features, query high-level features, support intermediate features and support high-level features are feature maps.

[0045] The ImageNet pre-trained backbone network is a ResNet50 network model pre-trained on the ImageNet dataset.

[0046] Specifically, (1) using a 7×7 convolution with a stride of 2, we obtain the supporting image intermediate features with a feature size of 119×119×128 and the query intermediate features with a feature size of 60×60×1024, as well as the supporting high-level features and query high-level features with a feature size of 60x60x2048. The intermediate features contain rich spatial information, and the high-level features contain rich semantic information.

[0047] To reduce computational parameters, 1×1 convolution is used to perform 1×1 convolution operations on intermediate and high-level features, reducing the number of channels. The combined number of channels for intermediate and high-level features is C=64.

[0048] (2) The supporting image is abstracted into a prototype vector, and the intermediate and high-level features are fused by the mask average pooling method to obtain a prototype vector with a feature size of 1×1×256.

[0049] The formula for the prototype vector is as follows:

[0050] In the formula, This indicates the average pooling operation. It uses interpolation and padding operations to... Remodeling into Same shape, Represented as a support mask, it is a foreground binary image obtained by performing image processing operations on the support image. This indicates support for intermediate features.

[0051] If the input is set to 5-shot when training the model, the average of the 5 prototype vectors obtained will be the new prototype vector.

[0052] (3) The query intermediate and query high-level features, support intermediate and support high-level features associated with the support mask are used to generate a priori mask, which indicates the pixel-wise relationship between the query features and the support features. A pixel with a higher value in the query features means that it is strongly correlated with at least one pixel of the target object in the support features. Therefore, the pixel in the query image is likely to be classified into the target class. The priori mask transformation generation formula is expressed as follows:

[0053] in, These represent the pixels in the query image and the supporting images, respectively. The query image indicates whether the query is for intermediate or advanced features, and the supporting images indicate whether they support intermediate or advanced features. This means that the maximum cosine similarity calculated between each query pixel and the supporting pixels is preserved. Reconstruct the structure to generate a priori mask Normalize it to a range between 0 and 1.

[0054] (4) such as Figure 4 The SASPP module shown uses dilated convolution sampling to achieve multi-scale receptive fields with different dilation rates. When the dilation rate is large, the receptive field expands, capturing more contextual information; when the dilation rate is small, more detailed information can be preserved.

[0055] Therefore, in this embodiment, the prior mask, prototype vector and intermediate query features are concatenated and then input into the SASPP (Dental Spatial Convolutional Pooling Pyramid) module to obtain the fused feature map and perform feature enhancement. The dilation rate is set to [6, 12, 18].

[0056]

[0057] in, Indicates the prior mask, This refers to the prototype vector. This indicates intermediate query features, representing a cascade of intermediate query features. This refers to SASPP operations.

[0058] Step 4: In the decoding stage, combine transformer, convolution and bilinear interpolation to restore the information of the input image and obtain the predicted output image.

[0059] For example, the specific process of step 4 is as follows: Step 401: First, the feature fusion map is processed by a convolutional layer with a stride of 1 and a kernel of 1×1. The 1×1 convolution is used to reduce the number of channels and remove redundant information. Then, a convolutional layer with a stride of 1 and a kernel of 1×1 is used to process the output feature map to enhance nonlinearity. The resulting feature map has a size of 60×60×64.

[0060] Step 402: Input the prototype vector and the query intermediate features into the transformer decoder. The transformer, acting as the decoder, performs a dot product between the query intermediate features and the prototype vector to obtain the embedding of the predicted object segmentation mask, as shown in the following formula:

[0061] in, This represents the activation function.

[0062] The fused feature map is input into the pixel decoder, and the output is the predicted feature map.

[0063] In this embodiment, the pixel decoder is a PPM decoder.

[0064] Step 403: Use FEM (Feature Enrichment Module) as the feature fusion module. The feature fusion module will embed and fuse the predicted feature map and the predicted object segmentation mask to obtain the predicted output map.

[0065] The FEM module constructs hierarchical relationships, using information from fine features to enrich the coarse feature map through a top-down path, and finally fuses the features from the two decoders together for output.

[0066] Step 404: Upsample the image using bilinear interpolation. The resulting predicted output image has the same size as the original image, with a feature size of 473×473×number of categories.

[0067] As one implementation method, when training a lightweight network, the predicted output image is compared with the original image, and the network is optimized using a loss function, which is expressed as:

[0068] in, The predicted output represents the image. This represents the actual label image of this pixel. This represents the weighting coefficients applied to each category. Represents the predicted output. Represents the actual label.

[0069] Finally, the same data augmentation and pruning operations as those performed on the training set were applied to each test set, and the average crossover ratio (CRO) and foreground-background crossover ratio (FCR) were used as metrics to evaluate the network's performance.

[0070] Example 2 This embodiment discloses a scene segmentation system based on a lightweight network, including: The acquisition module is configured to acquire road scene images; The scene segmentation module is configured to input road scene images into a trained lightweight network for scene segmentation and obtain a predicted output image. The training process for lightweight networks includes: The training set is divided into a support set and a query set, which are input into the encoder. Through a backbone network with fixed parameters, convolutional operations are used to extract features, obtain intermediate and high-level features, and fuse them to generate a prototype vector. A prior mask is constructed. The prior mask, prototype vector, and intermediate features are concatenated and dilated convolution is performed to obtain a feature fusion map. In the decoding stage, information from the input image is restored by combining transformer, convolution, and bilinear interpolation to obtain the predicted output image; Optimize lightweight networks using the DICE loss function.

[0071] It should be noted that the acquisition module and scene segmentation module mentioned above correspond to the steps in Embodiment 1. The examples and application scenarios implemented by the above modules and corresponding steps are the same, but are not limited to the content disclosed in Embodiment 1. It should be noted that the above modules, as part of the system, can be executed in a computer system such as a set of computer-executable instructions.

[0072] Example 3 Embodiment 3 of the present invention provides an electronic device, including a memory and a processor, as well as computer instructions stored in the memory and running on the processor. When the computer instructions are executed by the processor, they complete the above-mentioned steps.

[0073] Example 4 Embodiment 4 of the present invention provides a computer-readable storage medium for storing computer instructions, which, when executed by a processor, complete the above-described steps.

[0074] This invention is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the invention. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart illustrations and / or block diagrams. Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0075] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0076] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment, whereby a series of operational steps are performed to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0077] The descriptions of each embodiment in the above embodiments have different focuses. For parts not described in detail in a certain embodiment, please refer to the relevant descriptions in other embodiments.

[0078] The above description is merely a preferred embodiment of the present invention and is not intended to limit the invention. Various modifications and variations can be made to the present invention by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the scope of protection of the present invention.

Claims

1. A scene segmentation method based on lightweight networks, characterized in that, include: Acquire road scene images; The road scene image is input into the trained lightweight network for scene segmentation to obtain the predicted output image; The training process for lightweight networks includes: The training set is divided into a support set and a query set, which are input into the encoder. Through a backbone network with fixed parameters, convolutional operations are used to extract features, obtain intermediate and high-level features to generate prototype vectors, and construct a priori mask. The priori mask, prototype vectors and intermediate features are concatenated and dilated convolution is performed to obtain a feature fusion map. The step of using convolutional operations for feature extraction to obtain intermediate and high-level features and generate prototype vectors includes: The query image and supporting images are input into a parallel ImageNet pre-trained backbone network for feature extraction, obtaining query intermediate features, query high-level features, supporting intermediate features, and supporting high-level features; The support mid-level features and the support mask are fused through masked average pooling to generate a prototype vector. The decoding stage combines transformers, convolutions, and bilinear interpolation to reconstruct information from the input image, specifically including: The prototype vector and the query intermediate features are input into the transformer decoder, and the query intermediate features and the prototype vector are multiplied by a dot product to obtain the embedding of the predicted object segmentation mask. The fused feature map is input into the pixel decoder, and the output is the predicted feature map; The embedding of the predicted object segmentation mask and the predicted feature map are fused, and upsampling is performed using bilinear interpolation to obtain the predicted output map; Optimize lightweight networks using the DICE loss function.

2. The scene segmentation method based on lightweight networks as described in claim 1, characterized in that, The prototype vector is represented as: ， in, This represents the average pooling operation, and L represents the interpolation and padding operation. This indicates support for masks. This indicates support for intermediate features.

3. The scene segmentation method based on lightweight networks as described in claim 1, characterized in that, The construction of the prior mask specifically involves: reconstructing the structure to generate prior codes based on the pixel-by-pixel relationship between the query intermediate features and the supporting intermediate features, as well as between the query high-level features and the supporting high-level features, and then performing normalization processing.

4. The scene segmentation method based on lightweight networks as described in claim 1, characterized in that, The step of concatenating the prior mask, prototype vector, and intermediate features, and then performing dilated convolution to obtain the feature fusion map includes: Cascade queries intermediate features to obtain intermediate query features, concatenate the prior mask, prototype vector and intermediate query features, and perform dilated convolution operation to obtain fused feature map; Feature enhancement is performed on the fused feature map.

5. The scene segmentation method based on lightweight networks as described in claim 1, characterized in that, The DICE loss function is expressed as follows: ， in, The predicted output for the image. This represents the actual label image of this pixel. This represents the weighting coefficients applied to each category. Represents the predicted output. Represents the actual label.

6. A scene segmentation system based on lightweight networks, employing the scene segmentation method based on lightweight networks as described in any one of claims 1-5, characterized in that, include: The acquisition module is configured to acquire road scene images; The scene segmentation module is configured to input road scene images into a trained lightweight network for scene segmentation and obtain a predicted output image. The training process for lightweight networks includes: The training set is divided into a support set and a query set, which are input into the encoder. Through a backbone network with fixed parameters, convolutional operations are used for feature extraction to obtain intermediate and high-level features, which are then fused to generate a prototype vector and construct a priori mask. The priori mask, prototype vector, and intermediate features are concatenated and dilated convolutions are performed to obtain a feature fusion map. The feature extraction using convolutional operations to obtain intermediate and high-level features and generate the prototype vector includes: The query image and supporting images are input into a parallel ImageNet pre-trained backbone network for feature extraction, obtaining query intermediate features, query high-level features, supporting intermediate features, and supporting high-level features; The support mid-level features and the support mask are fused through masked average pooling to generate a prototype vector. The decoding stage combines transformers, convolutions, and bilinear interpolation to reconstruct information from the input image, specifically including: The prototype vector and the query intermediate features are input into the transformer decoder, and the query intermediate features and the prototype vector are multiplied by a dot product to obtain the embedding of the predicted object segmentation mask. The fused feature map is input into the pixel decoder, and the output is the predicted feature map; The embedding of the predicted object segmentation mask and the predicted feature map are fused, and upsampling is performed using bilinear interpolation to obtain the predicted output map; Optimize lightweight networks using the DICE loss function.

7. An electronic device, characterized in that, It includes a memory and a processor, as well as computer instructions stored in the memory and running on the processor, which, when executed by the processor, complete the steps of the scene segmentation method based on a lightweight network as described in any one of claims 1-5.

8. A computer-readable storage medium, characterized in that, Used to store computer instructions, which, when executed by a processor, complete the steps of the scene segmentation method based on a lightweight network as described in any one of claims 1-5.

Images