Automatic driving image segmentation method, storage medium and electronic equipment
By adopting a hybrid encoder network and feature compensation decoder network in autonomous driving image segmentation, combined with transformer, convolutional neural network and multi-scale channel attention mechanism, the problem of accuracy and speed imbalance in autonomous driving image segmentation is solved, and efficient and accurate image segmentation effect is achieved.
Patent Information
- Application Number
- CN202410359107.9
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-03-27
- Publication Date
- 2025-04-15
AI Technical Summary
In autonomous driving image segmentation, the existing technology is difficult to balance between accuracy and speed. The model based on pure convolutional neural networks is complex and cannot effectively extract global context information. However, the transformer-based model has the problem of semantic gap and excessive computational burden.
An autonomous driving image segmentation method based on transformer and attention mechanism is adopted. By constructing a hybrid encoder network and feature compensation decoder network, combining convolutional neural network, multi-scale channel attention mechanism and graph convolution, local semantic features and global context information are extracted, and the semantic gap between the encoder and the decoder is compensated through feature compensation blocks.
This method effectively reduces the complexity of the model and the calculation amount, improves the segmentation accuracy and efficiency, meets the needs of driving scenario segmentation, and is highly robust and practical.
Smart Images
Figure CN119832005A_ABST
Abstract
Description
Technical Field
[0001] The present application relates to the field of image processing, and in particular to an autonomous driving image segmentation method, storage medium and electronic device based on transformer and attention mechanism. Background Art
[0002] The environment in which autonomous vehicles are located is complex and changeable. Light intensity, obstacle occlusion, weather changes, etc. pose huge challenges to the semantic scene of autonomous driving visual perception. Autonomous driving systems usually require accurate road boundary and obstacle detection, while image semantic segmentation based on deep learning provides pixel-level classification of images for the semantic understanding of the entire scene and can make more reasonable and detailed judgments for the upper-level decisions of the system, so it is particularly suitable for autonomous vehicles and driver assistance.
[0003] At present, semantic segmentation models based on deep learning are mainly based on pure convolutional neural networks, such as the network structure of autoencoders. Since many models based on the above methods currently stack a large number of convolutional modules, although the segmentation accuracy is improved, the network complexity is high and the amount of calculation is large. At the same time, the global context information cannot be extracted well, resulting in poor segmentation accuracy. At present, a transformer-based segmentation model has been proposed, which effectively solves the shortcomings of pure convolutional neural network segmentation models that do not consider global context information. However, due to the large semantic gap in its encoding and decoding structure, the model performance is poor, and due to the large computational burden of operating on image sequences, it cannot meet the needs of driving scene segmentation. Summary of the invention
[0004] Based on this, in order to solve the problem of unbalanced accuracy and speed in autonomous driving image segmentation, the present invention provides an autonomous driving image segmentation method, storage medium and electronic device based on transformer and attention mechanism. This method can effectively make up for the semantic gap between the hybrid encoder and decoder, does not need to update too many weight parameters, has a small amount of calculation, and meets the needs of driving scene segmentation.
[0005] In a first aspect, the present application provides an automatic driving image segmentation method, wherein the method comprises:
[0006] S1: Acquire the surrounding environment image from the perspective of the car in the state of automatic driving and pre-process it to obtain image P;
[0007] S2: construct a hybrid encoder network, input the image P into the hybrid encoder network for feature extraction, wherein the hybrid encoder network includes M repeatedly stacked hybrid encoding blocks;
[0008] S3: performing channel composition and graph convolution on the feature map outputted by the first hybrid coding block of the hybrid encoder network after processing the image P, and then splicing the feature map finally outputted by the hybrid encoder network;
[0009] S4: Construct a feature compensation decoder network, input the outputs of S2 and S3 into the constructed feature compensation decoder network, and obtain the segmentation result.
[0010] Through the above method, image features are extracted through multiple hybrid coding blocks, and the semantic gap between the hybrid encoder network and the decoder is made up through the feature compensation decoder network. There is no need to update too many weight parameters, the amount of calculation is small, and the driving scene segmentation requirements are met.
[0011] In one embodiment, the preprocessing comprises:
[0012] The image size of the surrounding environment is set to H×W;
[0013] Normalize all H×W images to [0, 1].
[0014] In one embodiment, the hybrid encoder network includes four repeatedly stacked hybrid encoding blocks, and the hybrid encoding block includes a convolutional neural network block, a multi-scale channel attention block, and a transformer block. The hybrid encoding block integrates the advantages of convolutional neural networks, multi-scale channel attention mechanisms, and transformers.
[0015] In one embodiment, inputting the image P into the hybrid encoder network for feature extraction includes:
[0016] S21: In the convolutional neural network block of the first hybrid coding block, the image P is convolved to obtain a local semantic feature F local , F local =ConvBlock(P);
[0017] S22: In the multi-scale channel attention block of the first hybrid coding block, the features f1, f2, f3, and f4 of the four scales of 4×4, 8×8, 16×16, and 32×32 of the image P are obtained respectively, and the four scale features are concatenated to obtain the aggregated feature
[0018] Among them, Concat means concatenation;
[0019] The aggregated features of the four scales are obtained through the channel attention mechanism to obtain the multi-scale channel attention feature F i ,
[0020] Among them, GAP represents the global average pooling layer, Conv is the convolutional layer, σ is the sigmoid activation function, Represents matrix multiplication;
[0021] The local semantic features are concatenated with the multi-scale channel attention features to obtain
[0022] S23: Serialization Input into the transformer block of the hybrid coding block, wherein the transformer block includes a window multi-head self-attention layer (W-MHSA), a sliding window multi-head self-attention layer (SW-MHSA), a layer normalization layer (LN) and a multi-layer perceptron (MLP). Finally, the output of the transformer block of the first hybrid coding block is
[0023]
[0024] F t1 =MLP(LN(F t1 ′))+F t1 ′,
[0025] F′ t2 =SW-MHSA(LN(F t1 )),
[0026]
[0027] Among them, W-MHSA represents the window multi-head attention operation; LN represents the layer normalization operation; MLP represents the multi-layer perceptron; SW-MHSA represents the sliding window multi-head self-attention operation; + represents the addition of corresponding elements;
[0028] S24: Input the feature map output by the transformer block of the first hybrid coding block into the second hybrid coding block, and repeat steps S21-S23 to obtain the output of the second hybrid coding block
[0029] S25: Input the feature map output by the second hybrid coding block into the third hybrid coding block, and repeat step S24 to obtain the output of the third hybrid coding block.
[0030] S26: The feature map output by the third hybrid coding block Input the fourth mixed coding block, repeat step S25 to obtain the output of the fourth mixed coding block
[0031] The hybrid encoder network can be used to extract local semantic features, multi-scale channel attention features and global contextual semantic features. The constructed hybrid encoder network can reduce the size of the input feature map and learn the feature information contained in the input feature map.
[0032] In one embodiment, the channel composition and graph convolution include: converting each channel of the feature graph output after the image P is processed by the first hybrid coding block into a vertex, constructing the edges between the vertices using the Euclidean distance, constructing the graph G, and then performing the graph convolution, which is specifically expressed as follows:
[0033]
[0034] F graph =f(N,A)=η(T l W l N),
[0035] Among them, is the edge information of graph G, for The degree matrix, R is the feature representation of each vertex, and N is the output feature of the first mixed encoder A is the adjacency matrix of graph G, η is the RELU activation function, W l is the learnable parameter of each layer of graph convolution filter;
[0036] The feature map output by the hybrid encoder network is concatenated with the feature map output by the hybrid encoder network, comprising: The output F after convolution with the channel composition and graph graph To splice,
[0037]
[0038] In one of the embodiments, the feature compensation decoder network includes three feature compensation blocks that are repeatedly stacked;
[0039] Construct a feature compensation decoder network, input the outputs of S2 and S3 into the constructed feature compensation decoder network, and obtain the segmentation result: the output of S3 is combined with the output of step S25 Input the first feature compensation block, and compare the output of the first feature compensation block with the output of step S24. Input the second feature compensation block, and compare the output of the second feature compensation block with the output of step S23. The third feature compensation block is input, and the output of the third feature compensation block is convolved and up-sampled to obtain the segmentation result.
[0040] In one embodiment, the operation process of the first feature compensation block includes:
[0041] S41: The output F of S3 all The output of step S25 Average pooling is performed from the X and Y directions respectively, and reshaped to the input size, which are Its formula is:
[0042]
[0043]
[0044]
[0045]
[0046] Among them, reshape represents the reshaping operation, Avg_X represents the average pooling operation in the X direction, and Avg_Y represents the average pooling operation in the Y direction;
[0047] S42: Concatenate the reshaped features and perform convolution operations. The formula is:
[0048]
[0049]
[0050] S43: and The new feature vectors obtained by splitting are Aggregate in the X and Y directions respectively, and perform average pooling, convolution and sigmoid activation on the aggregated features. The formula is expressed as:
[0051]
[0052]
[0053] Where σ is the sigmoid activation function;
[0054] Then multiply the two vectors to get the compensation vector A, which is expressed as:
[0055]
[0056] in represents the matrix multiplication operation, and then F all Connected in series with A, we get the output of the first feature compensation block Its formula is:
[0057]
[0058] Among them, ⊙ represents the dot product operation;
[0059] Preferably, the operation process of the second and third characteristic compensation blocks is the same as the operation process of the first characteristic compensation block, including:
[0060] S44: The output of the first feature compensation block The output of step S24 Repeat steps S41-S43 to obtain the output of the second feature compensation block.
[0061] S45: The output of the second feature compensation block The output of step S23 Repeat steps S41-S43 to obtain the output of the third feature compensation block
[0062] In one embodiment, the output of the third feature compensation block is The formula for convolution and upsampling is expressed as:
[0063]
[0064] F′ up =Conv(F up ),
[0065] Among them, U pSample Represents an upsampling operation.
[0066] In a second aspect, the present application further provides a storage medium having a computer program stored thereon, wherein the program, when executed, implements the method described in the first aspect of the present invention.
[0067] In a third aspect, an embodiment of the present application provides an electronic device, including:
[0068] one or more processors;
[0069] a storage device having one or more programs stored thereon;
[0070] When the one or more programs are executed by the one or more processors, the one or more processors are enabled to implement the method according to the first aspect of the present invention.
[0071] Compared with the prior art, the present invention has the following beneficial effects:
[0072] The present invention combines the advantages of transformer, convolutional neural network and multi-scale channel attention mechanism by cleverly combining them, and can better extract local features, global contextual semantic information and multi-scale semantic information. At the same time, the present invention introduces graph convolutional neural network, takes into account the topological structure between channels, and makes feature map fusion easier. In order to make up for the semantic gap between the hybrid encoder and decoder, the present invention designs a feature compensation block in the decoder to effectively alleviate the semantic gap between the hybrid encoder and decoder. The network structure proposed in the present invention achieves good results in segmentation accuracy, efficiency and computational complexity, has strong robustness, is not only applicable to the field of autonomous driving environment, but also applicable to other semantic segmentation scenarios, and has good practicality. BRIEF DESCRIPTION OF THE DRAWINGS
[0073] Figure 1 is a flow chart of the method of the present invention;
[0074] Figure 2 It is the overall network structure diagram of the present invention;
[0075] Figure 3 It is the overall structure diagram of the hybrid coding block;
[0076] Figure 4 The structure diagram of the multi-scale channel attention layer in the hybrid encoding block;
[0077] Figure 5 It is the structural diagram of the transformer layer in the hybrid coding block;
[0078] Figure 6 This is the structural diagram of the convolutional network layer in the hybrid coding block;
[0079] Figure 7 is a structural diagram of a feature compensation block in a feature compensation decoder;
[0080] Figure 8 The surrounding environment image from the car's perspective;
[0081] Fig. 9 The image is processed by the method of the present invention. DETAILED DESCRIPTION
[0082] In order to make the purpose, technical solution and advantages of the present application more clearly understood, the present application is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application.
[0083] Reference to "embodiments" herein means that a particular feature, structure, or characteristic described in conjunction with the embodiments may be included in at least one embodiment of the present application. The appearance of the phrase in various locations in the specification does not necessarily refer to the same embodiment, nor is it an independent or alternative embodiment that is mutually exclusive with other embodiments. It is explicitly and implicitly understood by those skilled in the art that the embodiments described herein may be combined with other embodiments.
[0084] When implementing it, Figure 1-9 As shown, in this embodiment, an automatic driving image segmentation method, storage medium and electronic device based on transformer and attention mechanism uses the surrounding environment image from the perspective of the car as the target image data set, and specifically includes the following steps:
[0085] S1: Obtain the surrounding environment image from the perspective of the car in the state of automatic driving and preprocess it to obtain image P. The specific operations are as follows:
[0086] S11: Obtain the surrounding environment image for autonomous driving, such as Figure 8 As shown, the image size of the surrounding environment is set to H×W. In this embodiment, the size is uniformly set to 32×32;
[0087] S12: Normalize all H×W images to [0, 1].
[0088] S2: Construct a hybrid encoder network, wherein the hybrid encoder network includes M repeatedly stacked hybrid encoding blocks, in this embodiment, M is 4, each of the hybrid encoding blocks includes a convolutional neural network block, a multi-scale channel attention block and a transformer block, and input the image P into the hybrid encoder network for feature extraction. The specific operations are as follows:
[0089] S21: In the convolutional neural network block of the first hybrid coding block, the image P is convolved to obtain a local semantic feature F local , F local =ConvBlock(P);
[0090] S22: In the multi-scale channel attention block of the first hybrid coding block, the features f1, f2, f3, and f4 of the four scales of 4×4, 8×8, 16×16, and 32×32 of the image P are obtained respectively, and the four scale features are concatenated to obtain the aggregated feature
[0091] Among them, Concat means concatenation;
[0092] The aggregated features of the four scales are obtained through the channel attention mechanism to obtain the multi-scale channel attention feature F i ,
[0093] Among them, GAP represents the global average pooling layer, Conv is the convolutional layer, σ is the sigmoid activation function, Represents matrix multiplication;
[0094] The local semantic features are concatenated with the multi-scale channel attention features to obtain
[0095] S23: Serialization Input into the transformer block of the hybrid coding block, wherein the transformer block includes a window multi-head self-attention layer (W-MHSA), a sliding window multi-head self-attention layer (SW-MHSA), a layer normalization layer (LN) and a multi-layer perceptron (MLP). Finally, the output of the transformer block of the first hybrid coding block is
[0096]
[0097] F t1 =MLP(LN(F t1 ′))+F t1 ′,
[0098] F′ t2 =SW-MHSA(LN(F t1 )),
[0099]
[0100] Among them, W-MHSA represents the window multi-head attention operation; LN represents the layer normalization operation; MLP represents the multi-layer perceptron; SW-MHSA represents the sliding window multi-head self-attention operation; + represents the addition of corresponding elements;
[0101] S24: Input the feature map output by the transformer block of the first hybrid coding block into the second hybrid coding block and repeat steps S21-S23, that is, replace the image P with the feature map output by the transformer block of the first hybrid coding block and repeat steps S21-S23 to obtain the output of the second hybrid coding block
[0102] S25: Input the output feature map of the second hybrid coding block into the third hybrid coding block, repeat steps S21-S23, that is, replace the image P with the feature map output by the transformer block of the second hybrid coding block and repeat steps S21-S23 to obtain the output of the third hybrid coding block
[0103] S26: Input the output feature map of the third hybrid coding block into the fourth hybrid coding block, repeat the above steps, that is, replace the image P with the output feature map of the third hybrid coding block and repeat S21-S23 to obtain the output of the fourth hybrid coding block.
[0104] S3: After the image P is processed by the first hybrid coding block of the hybrid encoder network and the output feature map is subjected to channel composition and graph convolution, the specific operations are as follows:
[0105] Each channel of the feature map output by the first hybrid coding block of the image P is converted into a vertex, and the edges between the vertices are constructed using the Euclidean distance to construct the graph G, which is then convolved. The specific expression is as follows:
[0106]
[0107] F graph =f(N,A)=η(T l W l N),
[0108] Among them, is the edge information of graph G, for The degree matrix, R is the feature representation of each vertex, and N is the output feature of the first mixed encoder A is the adjacency matrix of graph G, η is the RELU activation function, W l is the learnable parameter of each layer of graph convolution filter, T l T is the corresponding T of each layer of graph convolution;
[0109] Then, it is concatenated with the feature map output by the hybrid encoder network. The specific operations are as follows:
[0110] The output of the hybrid encoder network The output F after convolution with the channel composition and graph graph To splice,
[0111] S4: construct a feature compensation decoder network. In this embodiment, the feature compensation decoder network includes three repeatedly stacked feature compensation blocks, and further includes a convolution layer and an upsampling layer. The outputs of S2 and S3 are input into the constructed feature compensation decoder network, and the output of the feature compensation decoder network is convolved and upsampled to obtain the segmentation result. The specific operations are as follows:
[0112] The output of S3 is compared with the output of step S25. A first feature compensation block is input, and the operation process of the first feature compensation block includes:
[0113] S41: The output F of S3 all The output of step S25 Average pooling is performed from the X and Y directions respectively, and reshaped to the input size, which are Its formula is:
[0114]
[0115]
[0116]
[0117]
[0118] Among them, reshape represents the reshaping operation, Avg_X represents the average pooling operation in the X direction, and Avg_Y represents the average pooling operation in the Y direction;
[0119] S42: Concatenate the reshaped features and perform convolution operations. The formula is:
[0120]
[0121]
[0122] S43: and The new feature vectors obtained by splitting are Aggregate in the X and Y directions respectively, and perform average pooling, convolution and sigmoid activation on the aggregated features. The formula is expressed as:
[0123]
[0124]
[0125] Where σ is the sigmoid activation function;
[0126] Then multiply the two vectors to get the compensation vector A, which is expressed as:
[0127]
[0128] in represents the matrix multiplication operation, and then F all Connected in series with A, we get the output of the first feature compensation block Its formula is:
[0129]
[0130] Among them, ⊙ represents the dot product operation;
[0131] The operation process of the second and third characteristic compensation blocks is the same as the operation process of the first characteristic compensation block, including:
[0132] S44: The output of the first feature compensation block The output of step S24 Replace F all and Repeat steps S41-S43 to obtain the output of the second feature compensation block.
[0133] S45: The output of the second feature compensation block The output of step S23 Replace F all and Repeat steps S41-S43 to obtain the output of the third feature compensation block
[0134] S46: The output of the third feature compensation block Convolution and upsampling are performed to obtain the final segmentation result. The formula is as follows:
[0135]
[0136] F′ up =Conv(F up ),
[0137] Among them, U pSample Represents an upsampling operation.
[0138] In this example, the final segmentation result is obtained through a convolution operation with a convolution kernel size of 1×1. Fig. 9 shown.
[0139] It should be understood that, for the aforementioned various method embodiments, although the various steps in the flow chart are displayed in sequence according to the indication of the arrows, these steps are not necessarily executed in sequence according to the order indicated by the arrows. Unless there is a clear description in this article, the execution of these steps does not have a strict order restriction, and these steps can be executed in other orders. Moreover, at least a portion of the steps in the flow chart of the method embodiment may include a plurality of sub-steps or a plurality of stages, and these sub-steps or stages are not necessarily executed at the same time, but can be executed at different times, and the execution order of these sub-steps or stages is not necessarily carried out in sequence, but can be executed in turn or alternately with other steps or at least a portion of the sub-steps or stages of other steps.
[0140] In one embodiment, a storage medium is provided, on which a computer program is stored. When the computer program is executed by a processor, the steps in the above method embodiments are implemented.
[0141] In one embodiment, an electronic device is provided, comprising:
[0142] one or more processors;
[0143] a storage device having one or more programs stored thereon;
[0144] When the one or more programs are executed by the one or more processors, the one or more processors implement the steps in the above method embodiments.
[0145] Those skilled in the art can understand that all or part of the processes in the above-mentioned embodiments can be completed by instructing the relevant hardware through a computer program, and the computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to the memory, database or other medium used in the embodiments provided by the present invention can include at least one of non-volatile and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetoresistive random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. As an illustration and not limitation, RAM can be in various forms, such as static random access memory (SRAM) or dynamic random access memory (DRAM). The database involved in each embodiment provided by the present invention may include at least one of a relational database and a non-relational database. Non-relational databases may include distributed databases based on blockchains, etc., but are not limited thereto. The processor involved in each embodiment provided by the present invention may be a general-purpose processor, a central processing unit, a graphics processor, a digital signal processor, a programmable logic device, a data processing logic device based on quantum computing, etc., but are not limited thereto.
[0146] The technical features of the above embodiments can be combined arbitrarily. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification. In the above embodiments, the description of each embodiment has its own emphasis. For parts that are not described in detail in a certain embodiment, please refer to the relevant description of other embodiments.
[0147] The above-mentioned embodiments only express several implementation methods of the present invention, and the descriptions thereof are relatively specific and detailed, but they cannot be understood as limiting the scope of the invention patent. It should be pointed out that, for ordinary technicians in this field, several variations and improvements can be made without departing from the concept of the present invention, and these all belong to the protection scope of the present invention. Therefore, the protection scope of the patent of the present invention shall be subject to the attached claims.
Claims
1. An automatic driving image segmentation method, characterized in that: The method comprises: S1: Acquire the surrounding environment image from the perspective of the car in the state of automatic driving and pre-process it to obtain image P; S2: construct a hybrid encoder network, input the image P into the hybrid encoder network for feature extraction, wherein the hybrid encoder network includes M repeatedly stacked hybrid encoding blocks; S3: performing channel composition and graph convolution on the feature map outputted by the first hybrid coding block of the hybrid encoder network after processing the image P, and then splicing the feature map finally outputted by the hybrid encoder network; S4: Construct a feature compensation decoder network, input the outputs of S2 and S3 into the constructed feature compensation decoder network, and obtain the segmentation result.
2. The method according to claim 1, wherein: The pre-processing comprises: The image size of the surrounding environment is set to H×W; Normalize all H×W images to [0, 1].
3. The method according to claim 1 or 2, wherein: The hybrid encoder network includes four repeatedly stacked hybrid encoding blocks, and the hybrid encoding block includes a convolutional neural network block, a multi-scale channel attention block and a transformer block.
4. The method according to claim 3, wherein: Inputting image P into the hybrid encoder network for feature extraction includes: S21: In the convolutional neural network block of the first hybrid coding block, the image P is convolved to obtain a local semantic feature F local , F local =ConvBlock(P); S22: In the multi-scale channel attention block of the first hybrid coding block, the features f1, f2, f3, and f4 of the four scales of 4×4, 8×8, 16×16, and 32×32 of the image P are obtained respectively, and the four scale features are concatenated to obtain the aggregated feature Among them, Concat means concatenation; The aggregated features of the four scales are obtained through the channel attention mechanism to obtain the multi-scale channel attention feature F i , Among them, GAP represents the global average pooling layer, Conv is the convolutional layer, σ is the sigmoid activation function, Represents matrix multiplication; The local semantic features are concatenated with the multi-scale channel attention features to obtain S23: Serialization Input into the transformer block of the hybrid coding block, wherein the transformer block includes a window multi-head self-attention layer (W-MHSA), a sliding window multi-head self-attention layer (SW-MHSA), a layer normalization layer (LN) and a multi-layer perceptron (MLP). Finally, the output of the transformer block of the first hybrid coding block is F t1 =MLP(LN(F t1 ′))+F t1 ′, F′ t2 =SW-MHSA(LN(F t1 )), Among them, W-MHSA represents the window multi-head attention operation; LN represents the layer normalization operation; MLP represents the multi-layer perceptron; SW-MHSA represents the sliding window multi-head self-attention operation; + represents the addition of corresponding elements; S24: Input the feature map output by the transformer block of the first hybrid coding block into the second hybrid coding block, and repeat steps S21-S23 to obtain the output of the second hybrid coding block S25: Input the feature map output by the second hybrid coding block into the third hybrid coding block, and repeat step S24 to obtain the output of the third hybrid coding block. S26: Input the feature map output by the third hybrid coding block into the fourth hybrid coding block, and repeat step S25 to obtain the output of the fourth hybrid coding block.
5. The method according to claim 4, wherein: The channel composition and graph convolution include: converting each channel of the feature graph output after the image P is processed by the first hybrid coding block into a vertex, constructing the edges between the vertices using the Euclidean distance, constructing the graph G, and then performing the graph convolution, which is specifically expressed as follows: F graph =f(N,A)=η(T l W l N),, Among them, is the edge information of graph G, for The degree matrix, R is the feature representation of each vertex, and N is the output feature of the first mixed encoder A is the adjacency matrix of graph G, η is the RELU activation function, W l is the learnable parameter of each layer of graph convolution filter; The feature map output by the hybrid encoder network is concatenated with the feature map output by the hybrid encoder network, comprising: The output F after convolution with the channel composition and graph graph To splice, 6. The method according to claim 5, wherein: The feature compensation decoder network includes three feature compensation blocks that are repeatedly stacked; Construct a feature compensation decoder network, input the outputs of S2 and S3 into the constructed feature compensation decoder network, and obtain the segmentation result including: combining the output of S3 with the output of step S25 Input the first feature compensation block, and compare the output of the first feature compensation block with the output of step S24. Input the second feature compensation block, and compare the output of the second feature compensation block with the output of step S23. The third feature compensation block is input, and the output of the third feature compensation block is convolved and up-sampled to obtain the segmentation result.
7. The method according to claim 6, wherein: The operation process of the first feature compensation block includes: S41: The output F of S3 all The output of step S25 Average pooling is performed from the X and Y directions respectively, and reshaped to the input size, which are Its formula is: Among them, reshape represents the reshaping operation, Avg_X represents the average pooling operation in the X direction, and Avg_Y represents the average pooling operation in the Y direction; S42: Concatenate the reshaped features and perform convolution operations. The formula is: S43: and The new feature vectors obtained by splitting are Aggregate in the X and Y directions respectively, and perform average pooling, convolution and sigmoid activation on the aggregated features. The formula is expressed as: Where σ is the sigmoid activation function; Then multiply the two vectors to get the compensation vector A, which is expressed as: in represents the matrix multiplication operation, and then F all Connected in series with A, we get the output of the first feature compensation block Its formula is: Among them, ⊙ represents the dot product operation; Preferably, the operation process of the second and third characteristic compensation blocks is the same as the operation process of the first characteristic compensation block, including: S44: The output of the first feature compensation block The output of step S24 Repeat steps S41-S43 to obtain the output of the second feature compensation block. S45: The output of the second feature compensation block The output of step S23 Repeat steps S41-S43 to obtain the output of the third feature compensation block 8. The method according to claim 7, wherein: The output of the third feature compensation block The formula for convolution and upsampling is expressed as: F′ up =Conv(F up ), Among them, UpSample represents an upsampling operation.
9. A storage medium having a computer program stored thereon, wherein: When the program is executed, the method according to any one of claims 1 to 8 is implemented.
10. An electronic device comprising: one or more processors; a storage device having one or more programs stored thereon; When the one or more programs are executed by the one or more processors, the one or more processors are enabled to implement the method according to any one of claims 1 to 8.