Lightweight image segmentation method and device, computer equipment and storage medium

Abstract

The invention belongs to the field of artificial intelligence and digital medical treatment, and relates to a lightweight image segmentation method, which comprises the steps of inputting a to-be-segmented image into a trained image segmentation model which comprises a Ghost encoder, a Ghost decoder and a connection layer; performing multi-level feature extraction on the to-be-segmented image through a Ghost encoder to obtain a level image feature map and a down-sampling feature map; performing Ghost convolution and up-sampling on the down-sampling feature map through a connection layer to obtain enhanced up-sampling features; and performing multi-level up-sampling operation on the enhanced up-sampling feature and the level image feature map through a Ghost decoder, and outputting an image segmentation result. The invention further provides a lightweight image segmentation device, computer equipment and a storage medium. In addition, the invention also relates to a block chain technology, and the to-be-segmented image can be stored in a block chain. The method is lighter in weight, can improve the segmentation precision, is higher in reasoning speed, is smaller in parameter quantity, and can achieve the real-time segmentation of an image.

Description

Technical Field

[0001] The present application relates to the fields of artificial intelligence and digital medical technology, and in particular to a lightweight image segmentation method, apparatus, computer equipment and storage medium. Background Art

[0002] Medical image analysis equipment, including magnetic resonance imaging (MRI), computed tomography (CT) and X-ray imaging, has become a basic device for clinical diagnosis. As an important method of medical image analysis, medical image segmentation plays a vital role in the process of computer-aided diagnosis, and provides a reliable basis for clinical diagnosis and early diagnosis of diseases by helping doctors make accurate judgments.

[0003] U-Net is undoubtedly one of the most successful methods in image segmentation tasks, especially in medical image segmentation. The U-Net network model is a fully convolutional neural network with an encoding and decoding structure. The encoding network part is used to extract target feature information, and the decoding network part is used to restore image detail information and use deconvolution to upsample the feature map to restore it to the input image size. A skip connection structure is used between the corresponding stages of the encoding and decoding parts to reuse low-level feature information and better restore image detail information.

[0004] However, with the rapid development of society, more and more tasks are applied to more and more complex scenes, and the requirements for segmentation algorithms in complex scenes are becoming higher and higher. Complex backgrounds and diverse category targets lead to inaccurate segmentation results of segmentation algorithms. At the same time, deep learning algorithms often need to calculate a large number of parameters, which are complex in calculation and slow in reasoning. It is difficult to meet real-time requirements in an environment with poor hardware facilities. Summary of the invention

[0005] The purpose of the embodiments of the present application is to propose a lightweight image segmentation method, apparatus, computer equipment and storage medium to solve the technical problems in the related art that the image segmentation performance of the segmentation method in complex scenes is poor, the network parameters and calculation amount are large, and the model training and reasoning efficiency are low.

[0006] In order to solve the above technical problems, the embodiment of the present application provides a lightweight image segmentation method, which adopts the following technical solution:

[0007] Acquire an image to be segmented, and input the image to be segmented into a trained image segmentation model, wherein the image segmentation model includes a Ghost encoder, a Ghost decoder, and a connection layer, and the connection layer connects the Ghost encoder and the Ghost decoder;

[0008] Performing multi-level feature extraction on the image to be segmented by the Ghost encoder to obtain a hierarchical image feature map and a down-sampled feature map, and transmitting the hierarchical image feature map to the Ghost decoder;

[0009] Performing Ghost convolution and upsampling on the downsampled feature map through the connection layer to obtain enhanced upsampled features, and transmitting the enhanced upsampled features to the Ghost decoder;

[0010] The Ghost decoder performs multi-level upsampling operations on the enhanced upsampling features and the hierarchical image feature map, and outputs an image segmentation result.

[0011] In order to solve the above technical problems, the embodiment of the present application further provides a lightweight image segmentation device, which adopts the following technical solution:

[0012] An acquisition module, used for acquiring an image to be segmented, and inputting the image to be segmented into a trained image segmentation model, wherein the image segmentation model comprises a Ghost encoder, a Ghost decoder and a connection layer, and the connection layer connects the Ghost encoder and the Ghost decoder;

[0013] A feature extraction module, used for performing multi-level feature extraction on the image to be segmented through the Ghost encoder to obtain a hierarchical image feature map and a down-sampled feature map, and transmitting the hierarchical image feature map to the Ghost decoder;

[0014] A connection module, configured to perform Ghost convolution and upsampling on the downsampled feature map through the connection layer to obtain enhanced upsampled features, and transmit the enhanced upsampled features to the Ghost decoder;

[0015] An upsampling module is used to perform multi-level upsampling operations on the enhanced upsampling features and the hierarchical image feature map through the Ghost decoder, and output an image segmentation result.

[0016] In order to solve the above technical problems, the embodiment of the present application further provides a computer device, which adopts the following technical solution:

[0017] The computer device includes a memory and a processor, wherein the memory stores computer-readable instructions, and the processor implements the steps of the lightweight image segmentation method as described above when executing the computer-readable instructions.

[0018] In order to solve the above technical problems, the embodiment of the present application further provides a computer-readable storage medium, which adopts the following technical solution:

[0019] The computer-readable storage medium stores computer-readable instructions, and when the computer-readable instructions are executed by a processor, the steps of the lightweight image segmentation method described above are implemented.

[0020] Compared with the prior art, this application has the following beneficial effects:

[0021] The present application provides a lightweight image segmentation method, which performs a segmentation task on an image to be segmented by using a trained image segmentation model including a Ghost encoder, a Ghost decoder and a connection layer, wherein the Ghost encoder performs multi-level convolution operations through a corresponding Ghost module to improve the feature extraction capability and make it more lightweight; the Ghost decoder also fuses multi-level features, i.e., multi-scale features, through a corresponding Ghost module, uses upsampling to restore the features, better combines context information, improves segmentation accuracy, and at the same time, has a faster inference speed, fewer parameters, and can achieve real-time image segmentation. BRIEF DESCRIPTION OF THE DRAWINGS

[0022] In order to more clearly illustrate the scheme in the present application, a brief introduction is given below to the drawings required for use in the description of the embodiments of the present application. Obviously, the drawings described below are some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0023] Figure 1 is an exemplary system architecture diagram to which the present application may be applied;

[0024] Figure 2 is a flow chart of an embodiment of a lightweight image segmentation method according to the present application;

[0025] Figure 3 It is a structural diagram of the image segmentation model of the present application;

[0026] Figure 4 yes Figure 2 A flowchart of a specific implementation of step S202;

[0027] Figure 5 is a structural schematic diagram of an embodiment of a lightweight image segmentation device according to the present application;

[0028] Figure 6 It is a structural diagram of an embodiment of a computer device according to the present application. DETAILED DESCRIPTION

[0029] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as those commonly understood by technicians in the technical field of the present application; the terms used in the specification of the application herein are only for the purpose of describing specific embodiments and are not intended to limit the present application; the terms "including" and "having" and any variations thereof in the specification and claims of the present application and the above-mentioned drawings are intended to cover non-exclusive inclusions. The terms "first", "second", etc. in the specification and claims of the present application or the above-mentioned drawings are used to distinguish different objects, not to describe a specific order.

[0030] Reference to "embodiments" herein means that a particular feature, structure, or characteristic described in conjunction with the embodiments may be included in at least one embodiment of the present application. The appearance of the phrase in various locations in the specification does not necessarily refer to the same embodiment, nor is it an independent or alternative embodiment that is mutually exclusive with other embodiments. It is explicitly and implicitly understood by those skilled in the art that the embodiments described herein may be combined with other embodiments.

[0031] In order to enable those skilled in the art to better understand the solution of the present application, the technical solution in the embodiments of the present application will be clearly and completely described below in conjunction with the accompanying drawings.

[0032] The embodiments of the present application can acquire and process relevant data based on artificial intelligence technology. Among them, artificial intelligence (AI) is the theory, method, technology and application system that uses digital computers or machines controlled by digital computers to simulate, extend and expand human intelligence, perceive the environment, acquire knowledge and use knowledge to obtain the best results.

[0033] AI basic technologies generally include sensors, dedicated AI chips, cloud computing, distributed storage, big data processing technology, operation / interaction systems, mechatronics, etc. AI software technologies mainly include computer vision technology, robotics technology, biometrics technology, speech processing technology, natural language processing technology, and machine learning / deep learning.

[0034] This application provides a lightweight image segmentation method that can be applied to Figure 1 In the system architecture 100 shown, the system architecture 100 may include terminal devices 101, 102, 103, a network 104 and a server 105. The network 104 is used to provide a medium for communication links between the terminal devices 101, 102, 103 and the server 105. The network 104 may include various connection types, such as wired, wireless communication links or optical fiber cables, etc.

[0035] Users can use terminal devices 101, 102, 103 to interact with server 105 through network 104 to receive or send messages, etc. Various communication client applications can be installed on terminal devices 101, 102, 103, such as web browser applications, shopping applications, search applications, instant messaging tools, email clients, social platform software, etc.

[0036] Terminal devices 101, 102, 103 can be various electronic devices with display screens and supporting web browsing, including but not limited to smart phones, tablet computers, e-book readers, MP3 players (Moving Picture Experts Group Audio Layer III), MP4 (Moving Picture Experts Group Audio Layer IV) players, laptop computers, desktop computers, etc.

[0037] The server 105 may be a server that provides various services, such as a background server that provides support for web pages displayed on the terminal devices 101 , 102 , and 103 .

[0038] It should be noted that the lightweight image segmentation method provided in the embodiment of the present application is generally executed by a server / terminal device, and accordingly, the lightweight image segmentation device is generally arranged in the server / terminal device.

[0039] It should be understood that Figure 1 The number of terminal devices, networks and servers in the embodiment is only for illustration. Any number of terminal devices, networks and servers may be provided according to the implementation requirements.

[0040] Continue to refer Figure 2 , shows a flowchart of an embodiment of a lightweight image segmentation method according to the present application, comprising the following steps:

[0041] Step S201, obtaining an image to be segmented, and inputting the image to be segmented into a trained image segmentation model, wherein the image segmentation model includes a Ghost encoder, a Ghost decoder and a connection layer.

[0042] Image segmentation plays a vital role in the medical field. It provides important support for clinical surgery, preoperative diagnosis, intraoperative navigation and postoperative evaluation. The purpose of medical image segmentation is to make the changes of anatomical or pathological structures in the image clearer; it plays a vital role in computer-aided diagnosis and intelligent medical care, greatly improving the efficiency and accuracy of diagnosis, thereby improving the success rate of treatment. Medical image segmentation is applied in tumor image segmentation, brain structure image segmentation, heart image segmentation, skin lesion image segmentation, blood vessel image segmentation, lung image segmentation, oral image segmentation, etc.

[0043] In this embodiment, the image to be segmented is a medical image to be segmented, and the image to be segmented can be obtained through a wired connection or a wireless connection. It should be noted that the wireless connection method may include but is not limited to a 3G / 4G connection, a WiFi connection, a Bluetooth connection, a WiMAX connection, a Zigbee connection, a UWB (ultra wideband) connection, and other wireless connection methods currently known or to be developed in the future.

[0044] Input the image to be segmented into the trained image segmentation model. The image segmentation model is an improved model based on the Unet network model. Figure 3 As shown, it includes a Ghost encoder, a Ghost decoder and a connection layer, wherein the Ghost encoder uses the Ghost module as the backbone network for feature extraction, and the Ghost decoder uses the Ghost module as the backbone network for multi-scale feature fusion, and combines upsampling to restore the detail information of the image to better combine the context information.

[0045] It should be emphasized that in order to further ensure the privacy and security of the image to be segmented, the image to be segmented can also be stored in a node of a blockchain.

[0046] The blockchain referred to in this application is a new application model of computer technologies such as distributed data storage, peer-to-peer transmission, consensus mechanism, encryption algorithm, etc. Blockchain is essentially a decentralized database, a string of data blocks generated by cryptographic methods. Each data block contains a batch of network transaction information, which is used to verify the validity of its information (anti-counterfeiting) and generate the next block. Blockchain can include the underlying blockchain platform, platform product service layer, and application service layer.

[0047] Step S202, performing multi-level feature extraction on the image to be segmented through the Ghost encoder to obtain a hierarchical image feature map and a down-sampled feature map, and transmitting the hierarchical image feature map to the Ghost decoder.

[0048] Among them, the Ghost encoder includes a first Ghost coding layer, a second Ghost coding layer, a third Ghost coding layer and a fourth Ghost coding layer. The feature map size of each Ghost coding layer is different, that is, the feature dimension and the number of channels are different. For example, the feature map size of the first Ghost coding layer is 512×512×64, the feature map size of the second Ghost coding layer is 256×256×128, the feature map size of the third Ghost coding layer is 128×128×256, and the feature map size of the fourth Ghost coding layer is 64×64×512.

[0049] In some optional implementations, see Figure 4 As shown, the above-mentioned steps of performing multi-level feature extraction on the segmented image through the Ghost encoder to obtain a hierarchical image feature map and a down-sampled feature map include:

[0050] Step S401, extracting features of the image to be segmented through the first Ghost coding layer to obtain a first image feature map, and transmitting the first image feature map to the corresponding level of the Ghost decoder.

[0051] In this embodiment, a Ghost block is used to replace the Conv-BN-ReLU block in the original U-Net to improve the performance of the network and reduce its parameters.

[0052] In some optional implementations, the first Ghost coding layer, the second Ghost coding layer, the third Ghost coding layer and the fourth Ghost coding layer independently include at least two layers of Ghost convolution modules and corresponding pooling modules, wherein the first Ghost coding layer is connected to the second Ghost coding layer through the first pooling module, the second Ghost coding layer is connected to the third Ghost coding layer through the second pooling module, the third Ghost coding layer is connected to the fourth Ghost coding layer through the third pooling module, and the fourth Ghost coding layer is connected to the connection layer through the fourth pooling module.

[0053] The image to be segmented is sequentially passed through each Ghost convolution module of the first Ghost encoding layer for feature extraction, and finally the first image feature map is obtained.

[0054] Taking each Ghost coding layer including a first Ghost convolution module and a second Ghost convolution module as an example, in this embodiment, the step of extracting features of the image to be segmented through the first Ghost coding layer to obtain a first image feature map includes:

[0055] Input the image to be segmented into the first Ghost convolution module for feature extraction to obtain the first Ghost convolution feature;

[0056] The first Ghost convolution feature is input into the second Ghost convolution module for feature extraction to obtain a first image feature map.

[0057] The Ghost convolution module is used for feature extraction, which reduces the number of parameters and calculations of the overall model, thereby reducing the number of parameters and calculations during reasoning and improving the reasoning rate and segmentation efficiency.

[0058] In some optional implementations of this embodiment, the step of inputting the image to be segmented into the first Ghost convolution module for feature extraction to obtain the first Ghost convolution feature includes:

[0059] Segment the image to be segmented into a first input feature and a second input feature according to the number of channels of the first Ghost convolution module;

[0060] Performing intrinsic convolution on the first input feature to obtain an intrinsic feature map;

[0061] Performing depth-wise separable convolution on the second input feature to obtain a depth-wise separable feature map;

[0062] The intrinsic feature map and the depth-separable feature map are concatenated and fused to obtain a concatenated fusion feature;

[0063] Perform residual connection on the spliced ​​fusion features and the image to be segmented, and output the first Ghost convolution feature.

[0064] Specifically, assume that the feature map of the image to be segmented input to the first Ghost convolution module is I, and its dimension is H×W×C, where H and W are the height and width of the feature map, respectively, and C is the number of channels. The input feature map I is split into two equal parts along the channel dimension (i.e., the number of channels), the first input feature I1 and the second input feature I2, and the number of channels in each part is C / 2. A 1x1 convolution operation (intrinsic convolution) is applied to I1. This operation can increase or decrease the number of channels, but usually remains unchanged. The output intrinsic feature map is F intrinsic The intrinsic characteristic diagram is F intrinsic BatchNormalization (BN) and ReLU activation functions are applied to enhance the expressiveness and nonlinearity of features. A depth-separable convolution (DWconv) is applied to I2, which usually includes spatial convolution and 1x1 channel-by-channel mixed convolution to obtain a depth-separable feature map F. generated This step can further extract features and reduce the number of parameters. BatchNormalization is performed on the depth-separable feature map to stabilize the training process. intrinsic and depthwise separable feature map F generatedThe new feature map F is formed by concatenation in the channel dimension, and the number of channels is restored to C. The input feature map I is residually connected with the new feature map F to output the final first Ghost convolution feature.

[0065] This process not only enhances the representation ability of the network, but also keeps the spatial size of the input and output unchanged through residual connections. The design of the Ghost convolution module allows the network to learn different features on different channels while reducing the computational complexity through depth-wise separable convolution.

[0066] Step S402, down-sample the first image feature map to obtain a first pooled feature, and input the first pooled feature into the second Ghost encoding layer.

[0067] In this embodiment, downsampling is pooling, which compresses the input feature map, reduces the features, leads to a reduction in parameters, and simplifies the complexity of convolution calculation; secondly, the receptive field is increased layer by layer, and important information in the feature map is retained, while improving the overfitting problem. Commonly used pooling methods include mean-pooling, max-pooling, and stochastic-pooling, etc. Furthermore, max-pooling can be used.

[0068] Step S403: extract features from the first pooled features through the second Ghost encoding layer to obtain a second image feature map, and transmit the second image feature map to the corresponding level of the Ghost decoder.

[0069] The second Ghost coding layer includes at least two layers of Ghost convolution modules. The second Ghost coding layer differs from the first Ghost coding layer in feature dimension and number of channels.

[0070] The first pooled features are sequentially passed through each Ghost convolution module of the second Ghost encoding layer for feature extraction, and finally the second image feature map is obtained.

[0071] The Ghost decoder level corresponding to the second Ghost encoding layer is the second Ghost decoding layer.

[0072] Step S404: downsample the second image feature map to obtain a second pooled feature, and input the second pooled feature into the third Ghost encoding layer.

[0073] Continue downsampling and pass the feature information to the deep network to improve the model's ability to extract features of different scales and better integrate contextual information.

[0074] Step S405: extract features from the second pooled features through the third Ghost encoding layer to obtain a third image feature map, and transmit the third image feature map to the corresponding level of the Ghost decoder.

[0075] The third Ghost coding layer includes at least two layers of Ghost convolution modules. The third Ghost coding layer is different from the second Ghost coding layer and the first Ghost coding layer in terms of feature dimension and number of channels.

[0076] The second pooled features are sequentially passed through the Ghost convolution modules of each layer of the third Ghost encoding layer for feature extraction, and finally the third image feature map is obtained.

[0077] The Ghost decoder level corresponding to the third Ghost encoding layer is the third Ghost decoding layer.

[0078] Step S406, down-sample the third image feature map to obtain a third pooled feature, and input the third pooled feature into the fourth Ghost coding layer.

[0079] The feature information is further passed to the deep network through downsampling.

[0080] Step S407, extracting features from the third pooled features through the fourth Ghost coding layer to obtain a fourth image feature map, and transmitting the fourth image feature map to the corresponding level of the Ghost decoder.

[0081] The fourth Ghost coding layer includes at least two layers of Ghost convolution modules. The fourth Ghost coding layer is different from the first Ghost coding layer, the second Ghost coding layer and the third Ghost coding layer in terms of feature dimension and number of channels.

[0082] The third pooled features are sequentially passed through the Ghost convolution modules of each layer of the fourth Ghost encoding layer for feature extraction, and finally a fourth image feature map is obtained.

[0083] The Ghost decoder level corresponding to the fourth Ghost encoding layer is the fourth Ghost decoding layer.

[0084] Step S408, down-sample the fourth image feature map to obtain a down-sampled feature map, and input the down-sampled feature map into the connection layer.

[0085] In this embodiment, feature extraction and pooling operations of different scales are performed in sequence through Ghost coding layers of different levels to obtain a feature map containing more feature information of different scales, and the feature information is gradually transmitted to the deep network, thereby improving the network's ability to extract target features of different scales, making full use of the global context information of the image, and improving the comprehensiveness and accuracy of feature extraction.

[0086] Step S203, performing Ghost convolution and upsampling on the downsampled feature map through the connection layer to obtain enhanced upsampled features, and transmitting the enhanced upsampled features to the Ghost decoder.

[0087] In this embodiment, the connection layer includes two layers of Ghost modules, through which the downsampled feature maps are sequentially subjected to Ghost convolution. The image size remains unchanged after the Ghost convolution of the connection layer, and the learning ability of the network can be enhanced by adding additional feature representations while keeping the spatial sizes of the input and output feature maps unchanged.

[0088] The features obtained after Ghost convolution are upsampled to increase the image size. During the upsampling process, the number of image channels remains unchanged, which can reduce the computational complexity.

[0089] Step S204, performing multi-level upsampling operations on the enhanced upsampling features and the hierarchical image feature map through a Ghost decoder, and outputting an image segmentation result.

[0090] The Ghost decoder includes a first Ghost decoding layer, a second Ghost decoding layer, a third Ghost decoding layer, a fourth Ghost decoding layer and a classification layer.

[0091] By upsampling through the Ghost decoder, the image size becomes larger and deep information is extracted. In the upsampling process, the shallow information of the Ghost coding layer on the left is fused, that is, the features of the corresponding Ghost coding layer on the left are spliced.

[0092] In some optional implementations, the step of performing multi-level upsampling operations on the enhanced upsampled features and the hierarchical image feature map through the Ghost decoder and outputting the image segmentation result includes:

[0093] The fourth level fusion feature is obtained by splicing the enhanced up-sampling feature and the fourth image feature map through the fourth Ghost decoding layer, and the fourth level fusion feature is subjected to Ghost convolution to obtain the first deconvolution feature;

[0094] Upsampling the first deconvolution feature to obtain a first upsampled feature map;

[0095] The first up-sampled feature map and the third image feature map are concatenated through the third Ghost decoding layer to obtain a third-level fusion feature, and the third-level fusion feature is subjected to Ghost convolution to obtain a second deconvolution feature;

[0096] Upsampling the second deconvolution feature to obtain a second upsampled feature map;

[0097] The second up-sampled feature map and the second image feature map are concatenated through the second Ghost decoding layer to obtain a second-level fusion feature, and the second-level fusion feature is subjected to Ghost convolution to obtain a third deconvolution feature;

[0098] Upsampling the third deconvolution feature to obtain a third upsampled feature map;

[0099] The third up-sampled feature map and the first image feature map are concatenated through the first Ghost decoding layer to obtain a first-level fusion feature, and the first-level fusion feature is subjected to Ghost convolution to obtain a third deconvolution feature;

[0100] The third deconvolution feature is classified through the classification layer and the image segmentation result is output.

[0101] In some optional implementations, the first Ghost decoding layer, the second Ghost decoding layer, the third Ghost decoding layer and the fourth Ghost decoding layer respectively and independently include at least two layers of Ghost deconvolution modules, wherein the first Ghost decoding layer is connected to the second Ghost decoding layer through a first upsampling module, the second Ghost decoding layer is connected to the third Ghost decoding layer through a second upsampling module, the third Ghost decoding layer is connected to the fourth Ghost decoding layer through a third upsampling module, and the fourth Ghost decoding layer is connected to the connection layer through a fourth upsampling module.

[0102] Taking each Ghost decoding layer including the first Ghost deconvolution module and the second Ghost deconvolution module as an example, the image size of the enhanced upsampled feature is 64×64×512, the image size of the fourth image feature map is 64×64×512, and the image size of the fourth-level fusion feature after splicing is 64×64×1024; the fourth-level fusion feature is subjected to the first Ghost convolution to obtain a feature with an image size of 64×64×512, and then the 64×64×512 feature is subjected to the second Ghost convolution to obtain the image size of the first deconvolution feature of 64×64×256, and the first deconvolution feature is upsampled to obtain the first upsampled feature map with an image size of 128×128×256; similarly, the third-level fusion feature The image size is 128×128×512. Two Ghost convolutions are performed through the third Ghost decoding layer, and the resulting image sizes are 128×128×256 and 128×128×128 respectively. The second upsampling feature map is 256×256×128, and the second-level fusion feature is 256×256×256; two Ghost convolutions are performed through the second Ghost decoding layer, and the resulting image sizes are 256×256×128 and 256×256×64 respectively. The third upsampling feature map is 512×512×64, and the first-level fusion feature is 512×512×128; two Ghost convolutions are performed through the first Ghost decoding layer, and the resulting image sizes are 512×512×64 and 512×512×64 respectively.

[0103] In this embodiment, the classification layer uses a 1×1 convolutional layer to perform segmentation classification prediction to obtain an image segmentation result.

[0104] Through the decoding operation of the Ghost decoder, the spatial resolution can be restored, and the high-resolution feature map from the Ghost encoder can be spliced ​​or added with the low-resolution feature map of the Ghost decoder. This feature fusion helps to retain more detail information in the final segmented image and improve the segmentation accuracy.

[0105] This application uses the Ghost encoder to perform multi-level convolution operations to improve the feature extraction capability and make it more lightweight; the Ghost decoder is used to fuse multi-level features, that is, multi-scale features, and upsampling is used to restore the features, better combine contextual information, and improve the segmentation effect. At the same time, it has a faster reasoning speed, fewer parameters, and can realize real-time image segmentation.

[0106] In some optional implementations, the first Ghost convolution module includes an intrinsic convolution layer, a first ghost convolution layer, a BN layer with a ReLU activation function, a depth-separable convolution layer, a second ghost convolution layer, a BN layer, a convolution attention layer, and a splicing layer.

[0107] The step of inputting the image to be segmented into the first Ghost convolution module for feature extraction to obtain the first Ghost convolution feature includes:

[0108] Input the image to be segmented into the intrinsic convolution layer for convolution to obtain the intrinsic feature map;

[0109] Input the intrinsic feature map into the first ghost convolution layer for processing to obtain the first convolution feature;

[0110] The first convolution feature is normalized through the BN layer and the ReLU activation function to obtain the normalized convolution feature;

[0111] The normalized convolutional features are convolved channel by channel through the depth-wise separable convolutional layer to obtain the deep convolutional features;

[0112] Input the deep convolutional features into the second ghost convolutional layer to obtain optimized deep convolutional features;

[0113] The optimized deep convolution features are normalized through the BN layer to obtain normalized deep convolution features;

[0114] The normalized deep convolutional features are paid attention to by the convolutional attention layer to obtain a deep separable feature map.

[0115] The intrinsic feature map and the depth-separable feature map are concatenated and fused through the concatenation layer to output the first Ghost convolutional feature.

[0116] In this embodiment, a depthwise separable convolution layer with a stride of 2 performs depthwise separable convolution, which helps to reduce the amount of calculation and the amount of parameters.

[0117] Through the intrinsic convolution layer, the image to be segmented is subjected to ordinary convolution to obtain the intrinsic feature map, and then the intrinsic feature map is processed one by one. Linear transformation and attention calculation are performed to obtain another part of the depth-separable feature map, which is then fused with the intrinsic feature map. In this way, the total number of parameters of the model is reduced to simplify the model.

[0118] In some optional implementations, before the step of obtaining the image to be segmented and inputting the image to be segmented into the trained image segmentation model, the following steps are further included:

[0119] Obtain an image dataset, where each image in the image dataset carries a known true segmentation label, and randomly divide the image dataset into a training set and a test set;

[0120] Input the training set into the pre-built deep neural network model to obtain the predicted segmentation results;

[0121] Calculate the loss value between the predicted segmentation result and the corresponding true segmentation label based on the preset loss function;

[0122] The network weight parameters of the deep neural network model are adjusted according to the loss value, and the iterative training is continued until the deep neural network model training is completed to obtain the model to be verified;

[0123] Input the test set into the model to be verified to obtain the verification segmentation result;

[0124] The segmentation accuracy is calculated based on the verification segmentation result. When the segmentation accuracy is greater than or equal to a preset threshold, the model to be verified is output as an image segmentation model.

[0125] In this embodiment, the image data set is acquired according to the classification task. For example, for brain tumor image segmentation, a brain tumor image data set is acquired, and the real segmentation labels corresponding to the brain tumor image data set include necrotic tumor core, peritumoral edema area, enhanced tumor, whole tumor, etc.; for tooth segmentation, a tooth image data set is acquired, and the real segmentation labels corresponding to the tooth image data set include tooth position, tooth contour, tooth structure, tooth key points and tooth lesion information, etc. The deep neural network model is trained based on the image data set to obtain the predicted segmentation result. Based on the loss value between the predicted segmentation result and the corresponding real segmentation label, the network weight parameter is adjusted, and the iterative training is continued until the model converges to obtain the model to be verified. Among them, the judgment conditions for model convergence include: the loss value cannot continue to decrease or the number of iterations reaches a preset number.

[0126] The test set is then used to verify the model to be verified. When the segmentation accuracy is greater than or equal to the preset threshold, the model to be verified is output as the image segmentation model. When the segmentation accuracy is less than the preset threshold, it is necessary to increase the number of samples, or modify the model parameters and retrain.

[0127] By training image segmentation models, we can enhance lesion detection and improve the accuracy of disease diagnosis. We can integrate data from different imaging modalities to provide more comprehensive diagnostic information, while improving data availability and standardizing the segmentation process.

[0128] The image segmentation method of the present application can be applied in hospitals and medical institutions to make the changes in anatomical or pathological structures in images clearer; it plays a vital role in computer-aided diagnosis and intelligent medicine, greatly improving the efficiency and accuracy of diagnosis.

[0129] In some optional implementations, before the step of inputting the image to be segmented into the trained image segmentation model, the following steps are further included:

[0130] The image to be segmented is preprocessed.

[0131] The preprocessing is as follows:

[0132] 1) Resize:

[0133] Resize the image to the dimensions required by the model input. This usually involves a scaling operation, which can be nearest neighbor interpolation, bilinear interpolation, or higher-order methods.

[0134] 2) Color space conversion:

[0135] If necessary, convert the image from the RGB color space to another color space, such as grayscale, HSV, or Lab.

[0136] 3) Grayscale:

[0137] Edge detection or texture analysis, converting color images to grayscale images can reduce the amount of calculation and remove color information.

[0138] 4) Histogram equalization:

[0139] The histogram of the image is equalized to enhance the contrast of the image and make the brightness distribution of the image more uniform.

[0140] 5) Noise Removal:

[0141] This is done through filtering methods such as spatial filtering and frequency domain filtering. In this embodiment, spatial filtering can be used. Spatial filtering can be divided into linear filtering methods represented by mean filtering and nonlinear filtering methods represented by median filtering.

[0142] 6) Edge Detection:

[0143] Edge detection refers to the abstract processing and feature discovery of contours in an image. Commonly used edge detection algorithms include Sobel, gradient, Canny, Roberts and other edge algorithms.

[0144] 7) Normalization:

[0145] Normalize the image pixel values, for example, scaling the pixel values ​​to between 0 and 1, or standardizing them according to the distribution of the training data.

[0146] 8) Data Augmentation:

[0147] Increase the diversity of image data through methods such as rotation, translation, scaling, shearing, and color jitter to improve the generalization ability of the model.

[0148] 9) Filling and cropping:

[0149] Pad or crop the image to ensure consistency with the input data.

[0150] Through preprocessing, the image quality can be improved, the useful features in the image can be enhanced, making it easier for the model to recognize and classify, improving the model reasoning efficiency and improving the segmentation accuracy.

[0151] The present application can be used in many general or special computer system environments or configurations. For example: personal computers, server computers, handheld or portable devices, tablet devices, multiprocessor systems, microprocessor-based systems, set-top boxes, programmable consumer electronics, network PCs, minicomputers, mainframe computers, distributed computing environments including any of the above systems or devices, etc. The present application can be described in the general context of computer-executable instructions executed by a computer, such as program modules. Generally, program modules include routines, programs, objects, components, data structures, etc. that perform specific tasks or implement specific abstract data types. The present application can also be practiced in distributed computing environments, in which tasks are performed by remote processing devices connected through a communication network. In a distributed computing environment, program modules can be located in local and remote computer storage media including storage devices.

[0152] Those skilled in the art can understand that all or part of the processes in the above-mentioned embodiments can be implemented by instructing the relevant hardware through computer-readable instructions, and the computer-readable instructions can be stored in a computer-readable storage medium. When the program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, the aforementioned storage medium can be a non-volatile storage medium such as a disk, an optical disk, a read-only memory (ROM), or a random access memory (RAM).

[0153] It should be understood that, although the steps in the flowchart of the accompanying drawings are displayed in sequence as indicated by the arrows, these steps are not necessarily executed in sequence in the order indicated by the arrows. Unless otherwise specified herein, there is no strict order restriction on the execution of these steps, and they can be executed in other orders. Moreover, at least a part of the steps in the flowchart of the accompanying drawings may include multiple sub-steps or multiple stages, and these sub-steps or stages are not necessarily executed at the same time, but can be executed at different times, and their execution order is not necessarily sequential, but can be executed in turn or alternately with other steps or at least a part of the sub-steps or stages of other steps.

[0154] Further references Figure 5 , as a response to the above Figure 2 The present application provides an embodiment of a lightweight image segmentation device. Figure 2Corresponding to the method embodiment shown, the device can be specifically applied to various electronic devices.

[0155] like Figure 5 As shown, the lightweight image segmentation device 500 described in this embodiment includes: an acquisition module 501, a feature extraction module 502, a connection module 503 and an up-sampling module 504. Among them:

[0156] The acquisition module 501 is used to acquire an image to be segmented, and input the image to be segmented into a trained image segmentation model, wherein the image segmentation model includes a Ghost encoder, a Ghost decoder and a connection layer, and the connection layer connects the Ghost encoder and the Ghost decoder;

[0157] The feature extraction module 502 is used to perform multi-level feature extraction on the image to be segmented through the Ghost encoder to obtain a hierarchical image feature map and a down-sampled feature map, and transmit the hierarchical image feature map to the Ghost decoder;

[0158] The connection module 503 is used to perform Ghost convolution and upsampling on the downsampled feature map through the connection layer to obtain enhanced upsampled features, and transmit the enhanced upsampled features to the Ghost decoder;

[0159] The upsampling module 504 is used to perform multi-level upsampling operations on the enhanced upsampling features and the hierarchical image feature map through the Ghost decoder, and output an image segmentation result.

[0160] It should be emphasized that in order to further ensure the privacy and security of the image to be segmented, the image to be segmented can also be stored in a node of a blockchain.

[0161] Based on the above-mentioned lightweight image segmentation device 500, multi-level convolution operations are performed through the Ghost encoder to improve the feature extraction capability and make it more lightweight; multi-level features, that is, multi-scale features are fused through the Ghost decoder, and upsampling is used to restore the features, better combine context information, and improve the segmentation effect. At the same time, it has a faster reasoning speed, less parameters, and can realize real-time image segmentation.

[0162] In some optional implementations, the Ghost encoder includes a first Ghost encoding layer, a second Ghost encoding layer, a third Ghost encoding layer, and a fourth Ghost encoding layer; the feature extraction module 502 includes a first feature submodule, a first pooling submodule, a second feature submodule, a second pooling submodule, a third feature submodule, a third pooling submodule, a fourth feature submodule, and a fourth pooling submodule, wherein:

[0163] A first feature submodule, configured to extract features of the image to be segmented through the first Ghost coding layer to obtain a first image feature map, and transmit the first image feature map to a corresponding layer of the Ghost decoder;

[0164] A first pooling submodule, configured to downsample the first image feature map to obtain a first pooling feature, and input the first pooling feature into the second Ghost encoding layer;

[0165] A second feature submodule, configured to extract features from the first pooled features through the second Ghost encoding layer to obtain a second image feature map, and transmit the second image feature map to a corresponding level of the Ghost decoder;

[0166] A second pooling submodule, used for downsampling the second image feature map to obtain a second pooling feature, and inputting the second pooling feature into the third Ghost encoding layer;

[0167] A third feature submodule, configured to extract features from the second pooling features through the third Ghost encoding layer to obtain a third image feature map, and transmit the third image feature map to a corresponding level of the Ghost decoder;

[0168] A third pooling submodule, used for downsampling the third image feature map to obtain a third pooling feature, and inputting the third pooling feature into the fourth Ghost coding layer;

[0169] A fourth feature submodule, used to extract the third pooling feature through the fourth Ghost encoding layer to obtain a fourth image feature map, and transmit the fourth image feature map to the corresponding layer of the Ghost decoder;

[0170] The fourth pooling submodule is used to downsample the fourth image feature map to obtain a downsampled feature map, and input the downsampled feature map into the connection layer.

[0171] Through different levels of Ghost coding layers, feature extraction and pooling operations of different scales are performed in sequence to obtain feature maps containing more feature information of different scales, and the feature information is gradually transmitted to the deep network, which improves the network's ability to extract target features of different scales, can make full use of the global context information of the image, and improve the comprehensiveness and accuracy of feature extraction.

[0172] In some optional implementations of this embodiment, the first Ghost coding layer, the second Ghost coding layer, the third Ghost coding layer, and the fourth Ghost coding layer independently include at least two layers of Ghost convolution modules, namely, a first Ghost convolution module and a second Ghost convolution module; the first feature submodule includes a first convolution unit and a second convolution unit, wherein:

[0173] The first convolution unit is used to input the image to be segmented into the first Ghost convolution module for feature extraction to obtain a first Ghost convolution feature;

[0174] The second convolution unit is used to input the first Ghost convolution feature into the second Ghost convolution module for feature extraction to obtain a first image feature map.

[0175] The Ghost convolution module is used for feature extraction, which reduces the number of parameters and calculations of the overall model, thereby reducing the number of parameters and calculations during reasoning and improving the reasoning rate and segmentation efficiency.

[0176] In some optional implementations, the first Ghost convolution module includes an intrinsic convolution layer, a first ghost convolution layer, a BN layer and a ReLU activation function, a depth-separable convolution layer, a second ghost convolution layer, a BN layer, a convolution attention layer, and a splicing layer; the first convolution unit is further used to:

[0177] Inputting the image to be segmented into the intrinsic convolution layer for convolution to obtain an intrinsic feature map;

[0178] Inputting the intrinsic feature map into the first ghost convolution layer for processing to obtain a first convolution feature;

[0179] Normalizing the first convolution feature through the BN layer and the ReLU activation function to obtain a normalized convolution feature;

[0180] Performing channel-by-channel convolution on the normalized convolutional features through the depth-wise separable convolutional layer to obtain depth-wise convolutional features;

[0181] Inputting the deep convolutional features into the second ghost convolutional layer to obtain optimized deep convolutional features;

[0182] Normalizing the optimized deep convolutional features through the BN layer to obtain normalized deep convolutional features;

[0183] Performing attention calculation on the normalized deep convolutional features through the convolutional attention layer to obtain a deep separable feature map;

[0184] The intrinsic feature map and the depth-separable feature map are spliced ​​and fused through the splicing layer to output the first Ghost convolution feature.

[0185] This embodiment can reduce the total number of parameters of the model to simplify the model.

[0186] In some optional implementations, the Ghost decoder includes a first Ghost decoding layer, a second Ghost decoding layer, a third Ghost decoding layer, a fourth Ghost decoding layer and a classification layer; the upsampling module 504 is further used for:

[0187] Splicing the enhanced up-sampling feature and the fourth image feature map through the fourth Ghost decoding layer to obtain a fourth-level fusion feature, and performing Ghost convolution on the fourth-level fusion feature to obtain a first deconvolution feature;

[0188] Upsampling the first deconvolution feature to obtain a first upsampled feature map;

[0189] splicing the first up-sampled feature map and the third image feature map through the third Ghost decoding layer to obtain a third-level fusion feature, and performing Ghost convolution on the third-level fusion feature to obtain a second deconvolution feature;

[0190] Upsampling the second deconvolution feature to obtain a second upsampled feature map;

[0191] Splicing the second up-sampled feature map and the second image feature map through the second Ghost decoding layer to obtain a second-level fusion feature, and performing Ghost convolution on the second-level fusion feature to obtain a third deconvolution feature;

[0192] Upsampling the third deconvolution feature to obtain a third upsampled feature map;

[0193] splicing the third up-sampled feature map and the first image feature map through the first Ghost decoding layer to obtain a first-level fusion feature, and performing Ghost convolution on the first-level fusion feature to obtain a third deconvolution feature;

[0194] The third deconvolution feature is classified by the classification layer, and an image segmentation result is output.

[0195] Through the decoding operation of the Ghost decoder, the spatial resolution can be restored, and the high-resolution feature map from the Ghost encoder can be spliced ​​or added with the low-resolution feature map of the Ghost decoder. This feature fusion helps to retain more detail information in the final segmented image and improve the segmentation accuracy.

[0196] In some optional implementations, the lightweight image segmentation apparatus 500 further includes a training module, which is used to:

[0197] Obtain an image dataset, wherein each image in the image dataset carries a known true segmentation label, and randomly divide the image dataset into a training set and a test set;

[0198] Inputting the training set into a pre-built deep neural network model to obtain a predicted segmentation result;

[0199] Calculating a loss value between the predicted segmentation result and the corresponding true segmentation label based on a preset loss function;

[0200] Adjusting the network weight parameters of the deep neural network model according to the loss value, and continuing the iterative training until the training of the deep neural network model is completed to obtain a model to be verified;

[0201] Inputting the test set into the model to be verified to obtain a verification segmentation result;

[0202] The segmentation accuracy is calculated based on the verification segmentation result, and when the segmentation accuracy is greater than or equal to a preset threshold, the model to be verified is output as an image segmentation model.

[0203] By training image segmentation models, we can enhance lesion detection and improve the accuracy of disease diagnosis. We can integrate data from different imaging modalities to provide more comprehensive diagnostic information, while improving data availability and standardizing the segmentation process.

[0204] In some optional implementations, the lightweight image segmentation apparatus 500 further includes a preprocessing module for preprocessing the image to be segmented.

[0205] Through preprocessing, the image quality can be improved, the useful features in the image can be enhanced, making it easier for the model to recognize and classify, improving the model reasoning efficiency and improving the segmentation accuracy.

[0206] To solve the above technical problems, the present application also provides a computer device. Figure 6 , Figure 6 This is a basic structural block diagram of the computer device in this embodiment.

[0207] The computer device 6 includes a memory 61, a processor 62, and a network interface 63 that are interconnected through a system bus. It should be noted that the figure only shows a computer device 6 with components 61-63, but it should be understood that it is not required to implement all the components shown, and more or fewer components can be implemented instead. Among them, those skilled in the art can understand that the computer device here is a device that can automatically perform numerical calculations and / or information processing according to pre-set or stored instructions, and its hardware includes but is not limited to microprocessors, application specific integrated circuits (Application Specific Integrated Circuit, ASIC), programmable gate arrays (Field-Programmable Gate Array, FPGA), digital processors (Digital Signal Processor, DSP), embedded devices, etc.

[0208] The computer device may be a computing device such as a desktop computer, a notebook, a PDA, a cloud server, etc. The computer device may interact with a user through a keyboard, a mouse, a remote controller, a touch pad, or a voice control device.

[0209] The memory 61 includes at least one type of readable storage medium, and the readable storage medium includes flash memory, hard disk, multimedia card, card-type memory (for example, SD or DX memory, etc.), random access memory (RAM), static random access memory (SRAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), programmable read-only memory (PROM), magnetic memory, disk, optical disk, etc. In some embodiments, the memory 61 can be an internal storage unit of the computer device 6, such as a hard disk or memory of the computer device 6. In other embodiments, the memory 61 can also be an external storage device of the computer device 6, such as a plug-in hard disk, a smart memory card (Smart Media Card, SMC), a secure digital (Secure Digital, SD) card, a flash card (FlashCard), etc. equipped on the computer device 6. Of course, the memory 61 can also include both the internal storage unit of the computer device 6 and its external storage device. In this embodiment, the memory 61 is generally used to store the operating system and various application software installed on the computer device 6, such as computer-readable instructions of a lightweight image segmentation method, etc. In addition, the memory 61 can also be used to temporarily store various types of data that have been output or are to be output.

[0210] The processor 62 may be a central processing unit (CPU), a controller, a microcontroller, a microprocessor, or other data processing chips in some embodiments. The processor 62 is generally used to control the overall operation of the computer device 6. In this embodiment, the processor 62 is used to run computer-readable instructions stored in the memory 61 or process data, such as computer-readable instructions for running the lightweight image segmentation method.

[0211] The network interface 63 may include a wireless network interface or a wired network interface. The network interface 63 is generally used to establish a communication connection between the computer device 6 and other electronic devices.

[0212] This embodiment implements the steps of the lightweight image segmentation method in the above embodiment when the processor executes computer-readable instructions stored in the memory, and performs multi-level convolution operations through the Ghost encoder to improve the feature extraction capability and make it more lightweight; the Ghost decoder fuses multi-level features, that is, multi-scale features, and uses upsampling to restore the features, better combine context information, and improve the segmentation effect. At the same time, it has a faster reasoning speed, less parameters, and can realize real-time image segmentation.

[0213] The present application also provides another implementation mode, namely, providing a computer-readable storage medium, wherein the computer-readable storage medium stores computer-readable instructions, and the computer-readable instructions can be executed by at least one processor so that the at least one processor executes the steps of the lightweight image segmentation method as described above, performs multi-level convolution operations through the Ghost encoder, improves the feature extraction capability and is more lightweight; fuses multi-level features, i.e., multi-scale features, through the Ghost decoder, uses upsampling to restore the features, better combines context information, improves the segmentation effect, and at the same time, has a faster reasoning speed, less parameters, and can realize real-time image segmentation.

[0214] Through the description of the above implementation methods, those skilled in the art can clearly understand that the above-mentioned embodiment methods can be implemented by means of software plus a necessary general hardware platform, and of course by hardware, but in many cases the former is a better implementation method. Based on such an understanding, the technical solution of the present application, or the part that contributes to the prior art, can be embodied in the form of a software product, which is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk), and includes a number of instructions for a terminal device (which can be a mobile phone, computer, server, air conditioner, or network device, etc.) to execute the methods described in each embodiment of the present application.

[0215] Obviously, the embodiments described above are only some embodiments of the present application, rather than all embodiments. The preferred embodiments of the present application are given in the accompanying drawings, but they do not limit the patent scope of the present application. The present application can be implemented in many different forms. On the contrary, the purpose of providing these embodiments is to make the understanding of the disclosure of the present application more thorough and comprehensive. Although the present application is described in detail with reference to the aforementioned embodiments, for those skilled in the art, it is still possible to modify the technical solutions recorded in the aforementioned specific implementation methods, or to perform equivalent replacement of some of the technical features therein. Any equivalent structure made using the contents of the specification and drawings of this application, directly or indirectly used in other related technical fields, is similarly within the scope of patent protection of this application.

Claims

1. A lightweight image segmentation method, characterized in that: The steps include: Acquire an image to be segmented, and input the image to be segmented into a trained image segmentation model, wherein the image segmentation model includes a Ghost encoder, a Ghost decoder, and a connection layer, and the connection layer connects the Ghost encoder and the Ghost decoder; Performing multi-level feature extraction on the image to be segmented by the Ghost encoder to obtain a hierarchical image feature map and a down-sampled feature map, and transmitting the hierarchical image feature map to the Ghost decoder; Performing Ghost convolution and upsampling on the downsampled feature map through the connection layer to obtain enhanced upsampled features, and transmitting the enhanced upsampled features to the Ghost decoder; The Ghost decoder performs multi-level upsampling operations on the enhanced upsampling features and the hierarchical image feature map, and outputs an image segmentation result.

2. The lightweight image segmentation method according to claim 1, characterized in that: The Ghost encoder includes a first Ghost encoding layer, a second Ghost encoding layer, a third Ghost encoding layer and a fourth Ghost encoding layer; the step of performing multi-level feature extraction on the image to be segmented by the Ghost encoder to obtain a hierarchical image feature map and a down-sampled feature map includes: Performing feature extraction on the image to be segmented through the first Ghost encoding layer to obtain a first image feature map, and transmitting the first image feature map to a corresponding layer of the Ghost decoder; Downsampling the first image feature map to obtain a first pooling feature, and inputting the first pooling feature into the second Ghost encoding layer; Performing feature extraction on the first pooled features through the second Ghost encoding layer to obtain a second image feature map, and transmitting the second image feature map to a corresponding layer of the Ghost decoder; Downsampling the second image feature map to obtain a second pooling feature, and inputting the second pooling feature into the third Ghost encoding layer; Extracting features from the second pooled features through the third Ghost encoding layer to obtain a third image feature map, and transmitting the third image feature map to a corresponding level of the Ghost decoder; Downsampling the third image feature map to obtain a third pooling feature, and inputting the third pooling feature into the fourth Ghost encoding layer; Extracting features from the third pooling feature through the fourth Ghost encoding layer to obtain a fourth image feature map, and transmitting the fourth image feature map to a corresponding level of the Ghost decoder; The fourth image feature map is downsampled to obtain a downsampled feature map, and the downsampled feature map is input into the connection layer.

3. The lightweight image segmentation method according to claim 2, characterized in that: The first Ghost coding layer, the second Ghost coding layer, the third Ghost coding layer and the fourth Ghost coding layer independently include at least two layers of Ghost convolution modules, namely a first Ghost convolution module and a second Ghost convolution module; the step of extracting features of the image to be segmented through the first Ghost coding layer to obtain a first image feature map includes: Inputting the image to be segmented into the first Ghost convolution module for feature extraction to obtain a first Ghost convolution feature; The first Ghost convolution feature is input into the second Ghost convolution module for feature extraction to obtain a first image feature map.

4. The lightweight image segmentation method according to claim 3, characterized in that: The first Ghost convolution module includes an intrinsic convolution layer, a first ghost convolution layer, a BN layer and a ReLU activation function, a depth-separable convolution layer, a second ghost convolution layer, a BN layer, a convolution attention layer and a splicing layer; the step of inputting the image to be segmented into the first Ghost convolution module for feature extraction to obtain the first Ghost convolution feature includes: Inputting the image to be segmented into the intrinsic convolution layer for convolution to obtain an intrinsic feature map; Inputting the intrinsic feature map into the first ghost convolution layer for processing to obtain a first convolution feature; Normalizing the first convolution feature by using the BN layer and the ReLU activation function to obtain a normalized convolution feature; Performing channel-by-channel convolution on the normalized convolutional features through the depth-wise separable convolutional layer to obtain depth-wise convolutional features; Inputting the deep convolutional features into the second ghost convolutional layer to obtain optimized deep convolutional features; Normalizing the optimized deep convolutional features through the BN layer to obtain normalized deep convolutional features; Performing attention calculation on the normalized deep convolutional features through the convolutional attention layer to obtain a deep separable feature map; The intrinsic feature map and the depth-separable feature map are spliced ​​and fused through the splicing layer to output the first Ghost convolution feature.

5. The lightweight image segmentation method according to claim 2, characterized in that: The Ghost decoder includes a first Ghost decoding layer, a second Ghost decoding layer, a third Ghost decoding layer, a fourth Ghost decoding layer and a classification layer. The step of performing a multi-level upsampling operation on the enhanced upsampling feature and the hierarchical image feature map by the Ghost decoder and outputting an image segmentation result includes: Splicing the enhanced up-sampling feature and the fourth image feature map through the fourth Ghost decoding layer to obtain a fourth-level fusion feature, and performing Ghost convolution on the fourth-level fusion feature to obtain a first deconvolution feature; Upsampling the first deconvolution feature to obtain a first upsampled feature map; splicing the first up-sampled feature map and the third image feature map through the third Ghost decoding layer to obtain a third-level fusion feature, and performing Ghost convolution on the third-level fusion feature to obtain a second deconvolution feature; Upsampling the second deconvolution feature to obtain a second upsampled feature map; Splicing the second up-sampled feature map and the second image feature map through the second Ghost decoding layer to obtain a second-level fusion feature, and performing Ghost convolution on the second-level fusion feature to obtain a third deconvolution feature; Upsampling the third deconvolution feature to obtain a third upsampled feature map; splicing the third up-sampled feature map and the first image feature map through the first Ghost decoding layer to obtain a first-level fusion feature, and performing Ghost convolution on the first-level fusion feature to obtain a third deconvolution feature; The third deconvolution feature is classified by the classification layer, and an image segmentation result is output.

6. The lightweight image segmentation method according to claim 1, characterized in that: Before the step of obtaining the image to be segmented and inputting the image to be segmented into the trained image segmentation model, the following step is also included: Obtain an image dataset, wherein each image in the image dataset carries a known true segmentation label, and randomly divide the image dataset into a training set and a test set; Inputting the training set into a pre-built deep neural network model to obtain a predicted segmentation result; Calculating a loss value between the predicted segmentation result and the corresponding true segmentation label based on a preset loss function; Adjusting the network weight parameters of the deep neural network model according to the loss value, and continuing the iterative training until the training of the deep neural network model is completed to obtain a model to be verified; Inputting the test set into the model to be verified to obtain a verification segmentation result; The segmentation accuracy is calculated based on the verification segmentation result, and when the segmentation accuracy is greater than or equal to a preset threshold, the model to be verified is output as an image segmentation model.

7. The lightweight image segmentation method according to claim 1, characterized in that: Before the step of inputting the image to be segmented into the trained image segmentation model, the method further includes: The image to be segmented is preprocessed.

8. A lightweight image segmentation device, characterized in that: include: An acquisition module, used for acquiring an image to be segmented, and inputting the image to be segmented into a trained image segmentation model, wherein the image segmentation model comprises a Ghost encoder, a Ghost decoder and a connection layer, and the connection layer connects the Ghost encoder and the Ghost decoder; A feature extraction module, used for performing multi-level feature extraction on the image to be segmented through the Ghost encoder to obtain a hierarchical image feature map and a down-sampled feature map, and transmitting the hierarchical image feature map to the Ghost decoder; A connection module, configured to perform Ghost convolution and upsampling on the downsampled feature map through the connection layer to obtain enhanced upsampled features, and transmit the enhanced upsampled features to the Ghost decoder; An upsampling module is used to perform multi-level upsampling operations on the enhanced upsampling features and the hierarchical image feature map through the Ghost decoder, and output an image segmentation result.

9. A computer device, characterized in that: The method comprises a memory and a processor, wherein the memory stores computer-readable instructions, and the processor implements the steps of the lightweight image segmentation method according to any one of claims 1 to 7 when executing the computer-readable instructions.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores computer-readable instructions, and when the computer-readable instructions are executed by a processor, the steps of the lightweight image segmentation method according to any one of claims 1 to 7 are implemented.

Images