Double-layer sampling medical image pre-training method and system based on mixed distribution

Abstract

The invention discloses a double-layer sampling medical image pre-training method and system based on mixed distribution, which combine a center cutting strategy based on Gaussian distribution and a mask selection strategy based on local gradient direction distribution and are used for improving the precision of medical image pre-training and segmentation. According to the Gaussian distribution center cutting strategy, cutting is carried out by using a reference center point conforming to Gaussian distribution, so that a cutting area is more concentrated on a main part of an image; and a mask selection strategy based on local gradient direction distribution effectively selects a foreground part with a large amount of information in the medical image by capturing shape features of the image, so that the understanding and analysis capability of the model on key features of the image in a mask reconstruction process is improved. Experimental results show that the method of the invention significantly improves the precision and reliability of medical image segmentation.

Description

Background Art

[0002] Center cropping and random cropping are common image preprocessing methods in the 3D medical image segmentation training process. Center cropping is to cut a fixed-size segment from the central area of ​​the image. Its main purpose is to capture the most important part of the image, usually the area containing key anatomical structures or lesions. Since medical images usually have high resolution and large size, directly using the entire image for processing may incur huge computational overhead and redundant information. Through center cropping, the image size can be effectively reduced, the computational efficiency can be improved, and the area with the most diagnostic value can be retained. This method is particularly suitable for scenarios where the target area is concentrated in the center of the image, such as certain types of tumor detection or organ segmentation.

[0003] Random cropping randomly selects any area in the image for cropping to increase the diversity of the data, thereby helping the model improve its generalization ability. Random cropping can generate image fragments at different locations, covering a wider feature space and reducing the model's dependence on specific areas or specific structures. This is especially important for dealing with situations where image content is unevenly distributed or the location of the target area is uncertain. By increasing the diversity of training data, random cropping can effectively improve the model's performance on unknown data.

[0004] In the field of mask reconstruction pre-training, random masking and progressive masking are common mask selection strategies. Random masking randomly selects areas in the image for masking. During training, random masking selects different parts of the image to mask, forcing the model to use other uncovered parts of the image for prediction and recovery during reconstruction. This method can effectively improve the model's ability to recover some missing information and enhance its robustness. However, a potential problem with random masking is that it may not fully utilize all the information in the image, especially when the masked area is small, the model may tend to rely on uncovered areas for recovery.

[0005] Progressive masking helps the model learn features from simple to complex by gradually increasing the size or number of masked areas. This method aims to enable the model to handle simpler tasks at the beginning of training, such as restoring small missing areas. As training progresses, the difficulty of masking is gradually increased, allowing the model to face more complex and large-scale missing information. Progressive masking can effectively guide the model to gradually adapt and learn from simple features to complex features, thereby improving learning effects and generalization capabilities.

[0006] Currently, a random cropping sampling method is used. Random cropping may remove key features or information in the image. Moreover, due to the uncertainty of the cropped area, the amount and complexity of information received by the model may be inconsistent between different training batches, which may affect the stability of training.

[0007] There is another sampling method that uses a center cropping method. Center cropping relies too much on the central area, which will cause the model to be less sensitive to non-central information and ignore the useful features contained in the edge area.

[0008] There is also a method that uses a random mask strategy to mask the data. The random mask may cover the background area and ignore the key feature area in the image, resulting in the model being unable to effectively learn the important foreground features in the image. The randomness of the mask may cause the model to receive inconsistent information between different training batches, thereby affecting the stability of the training.

[0009] Another existing method is to use a progressive masking strategy to mask the data. The progressive masking strategy gradually increases the size or number of masked areas as the training progresses, which may prolong the training time. When the masked area is small in the early stage of training, the model may over-rely on unmasked features, resulting in overfitting of these features. Summary of the invention

[0010] The technical problem to be solved by the present invention is to provide a medical image pre-training method and system with a double-layer sampling based on a mixed distribution in response to the deficiencies in the above-mentioned prior art, so as to solve the problem that the existing pre-training methods cannot effectively capture the key features in medical images, thus resulting in insufficient model performance.

[0011] The present invention adopts the following technical solutions: A double-layer sampling medical image pre-training method based on mixed distribution includes the following steps: Preprocess the original medical images; Crop the preprocessed medical image into multiple image blocks of the same size; Formulate a masking strategy, perform masking operations on the obtained image blocks, and generate masked data and retained data; Construct a mask reconstruction self-supervised learning network based on convolutional neural network, input the generated masked data into the mask reconstruction self-supervised pre-training network, and complete the double-layer sampling medical image pre-training; The weights of the obtained mask reconstruction self-supervised learning network are transferred to a fully supervised segmentation model with the same network structure, and the fully supervised segmentation model is trained using labeled image data.

[0012] Preferably, the preprocessing of the original medical image is specifically performed as follows: Add a channel dimension to the image data; rearrange the coordinate axis direction of the image and standardize the coordinate axis direction of the image to the right, front, and top format; change the voxel size of the image Adjust to ; Normalize the pixel intensity of the image to the specified range; fill the image space to reach the specified space size; calculate the effective target area within the image size in the training set, generate a center point that conforms to the Gaussian distribution with a mean of 0 and a variance of 1 in the target area, scale the corresponding center point into the valid area, crop with the corresponding geometric center as the reference point, and extract the image block of the specified size from each image.

[0013] Preferably, the original medical images include magnetic resonance imaging, computer tomography, positron emission tomography and ultrasound imaging images.

[0014] Preferably, the voxel size of the image is adjusted for:

[0015] in, is the interpolation weight function, and The original and resized images respectively.

[0016] Preferably, the pixel intensities of the image are normalized to a specified range as follows:

[0017] in, is the original intensity value, is the normalized intensity value, is the original intensity range, is the target intensity range.

[0018] Preferably, extracting an image block of a specific size from each image is specifically as follows:

[0019] in, for, for, are the parameters of the Gaussian distribution, is the effective target area, is the minimum size of the image block that can be extracted, is the maximum size of the image block that can be extracted.

[0020] Preferably, the masked data and the retained data are generated specifically as follows: The image block is split according to its channel dimension, with each channel as a two-dimensional slice; each slice is converted to an 8-bit unsigned integer and the pixel value range is standardized to 0 to 255; then the slice size is resized to Pixels; Calculate the value of each pixel in the image The gradient of Divide the image into pieces of size Small blocks of pixels, in each small block, according to the gradient direction of the pixel point, its corresponding gradient amplitude is projected into a predetermined angle interval to form a direction histogram; Combine several adjacent small blocks into a large block and perform L2 normalization on the direction histogram within the block; The directional histogram vectors of all normalized blocks are concatenated into a long vector to form the final feature vector; then, the feature value of the corresponding image block is obtained by calculating the average value of these feature vectors; Sort the image blocks according to their eigenvalues ​​and select several blocks with the largest eigenvalues ​​to generate masks; Use the mask matrix to generate masked data and retained data , ⊙ represents element-by-element multiplication.

[0021] Preferably, constructing a mask reconstruction self-supervised learning network based on a convolutional neural network is specifically as follows: The masked data Input to encoder In the feature representation ; Input the feature Z output by the encoder into the decoder, the decoder The goal is to reconstruct the masked data area; calculate the reconstructed data and real data The mean square error between them; construct a total loss function including reconstruction loss and regularization loss; use the optimization algorithm Adam to minimize the total loss function.

[0022] Preferably, the total loss function Specifically:

[0023] in, is the regularization weight, yes Regularized loss, Reconstruction data and real data The mean square error between .

[0024] In a second aspect, an embodiment of the present invention provides a medical image pre-training system based on a double-layer sampling of mixed distribution, comprising: A preprocessing module is used to preprocess the original medical images; The network module crops the preprocessed medical image into multiple image blocks of the same size, and sequentially inputs the image blocks as input data into the convolutional neural network; The strategy module formulates a mask strategy, performs mask operations on the obtained image blocks, and generates masked data and retained data; The construction module builds a mask reconstruction self-supervised learning network based on a convolutional neural network, inputs the generated masked data into the mask reconstruction self-supervised pre-training network, and completes the double-layer sampling medical image pre-training; The fine-tuning module transfers the weights of the obtained mask reconstruction self-supervised learning network to the fully supervised segmentation model with the same network structure, and trains the fully supervised segmentation model using labeled image data.

[0025] In a third aspect, a computer device comprises a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the computer program, the steps of the above-mentioned two-layer sampling medical image pre-training method based on mixed distribution are implemented.

[0026] In a fourth aspect, an embodiment of the present invention provides a computer-readable storage medium, including a computer program, which, when executed by a processor, implements the steps of the above-mentioned two-layer sampling medical image pre-training method based on mixed distribution.

[0027] Compared with the prior art, the present invention has at least the following beneficial effects: A two-layer sampling medical image pre-training method based on mixed distribution. The center cropping strategy based on Gaussian distribution uses the reference center point that conforms to the Gaussian distribution for cropping, so that the cropped area is more concentrated on the main part of the image. This method can not only reduce the important information that may be lost during the cropping process, but also ensure that the key areas of the image are retained, thereby improving the model's understanding and analysis of the image content. Combined with the mask selection strategy based on the local gradient direction distribution, the foreground part of the medical image with more information can be effectively selected. This mask selection strategy identifies areas with high information density in the image by analyzing the changes in the local gradient direction, and then generates a suitable mask to cover these areas. In the process of mask reconstruction, the model can learn the important information of the image more deeply and significantly improve the understanding and analysis of the key features of the image.

[0028] Furthermore, the present invention provides comprehensive optimization by adjusting voxel size, normalizing pixel intensity, and reasonably extracting image blocks; first, voxel size adjustment ensures that medical images of different resolutions can be processed at a unified scale, reducing errors caused by different scanning conditions, thereby improving the consistency of image data. Secondly, pixel intensity normalization eliminates the intensity differences produced by different imaging devices, so that medical image data from different modalities can be analyzed within the same intensity range, which helps to improve the generalization ability of the model and the stability of training. The extraction of image blocks adopts a method that combines Gaussian distribution with local gradient information to ensure that the extracted image blocks can cover the key areas with large amounts of information in the image without being too large or too small, effectively enhancing the model's ability to capture image details and key features. This series of preprocessing steps provides high-quality input data for subsequent modeling and reconstruction of medical images, thereby significantly improving the performance and accuracy of the model in image analysis.

[0029] Furthermore, in the mixed distribution-based two-layer sampling medical image pre-training method, the original medical images include magnetic resonance imaging (MRI), computed tomography (CT), positron emission tomography (PET) and ultrasound imaging (Ultrasound). The purpose of this setting is to enhance the versatility and adaptability of the model. By introducing multiple imaging modes, the characteristics of different imaging technologies can be fully utilized to improve the generalization ability of the pre-trained model. Magnetic resonance imaging is good at capturing the fine structure of soft tissue, while computed tomography helps the model better identify hard tissue and bone contours. Positron emission tomography mainly reflects metabolic activity and biochemical changes. Although the image is blurry, it can provide important information about functional lesions. Ultrasound imaging faces large noise and low contrast problems, but its dynamic capture ability is particularly critical for soft tissue analysis. By integrating these different image data during the pre-training process, the model can learn the characteristic expressions of various images, improve the recognition ability in low signal-to-noise ratio environments, and enhance the adaptability to various medical scenarios, so that it can perform more robustly and accurately in subsequent tasks such as medical image segmentation and lesion detection.

[0030] Furthermore, the design of the mask strategy aims to improve the data quality and model performance in the pre-training process of medical images through effective feature extraction and mask generation. First, the image blocks are divided into two-dimensional slices according to the channel dimension, and the pixel values ​​are normalized to 0 to 255. This process not only simplifies data processing, but also ensures the consistency of pixel values ​​between different channels, which is convenient for subsequent calculations and analysis. Next, by calculating the gradient of each pixel point, the edge information and texture features of the image can be captured, thus laying the foundation for the subsequent histogram calculation; in the process of dividing the image into small blocks and calculating the direction histogram, this step can effectively extract the local features of the image, while taking into account the gradient direction, enhancing the sensitivity to shape and contour. By combining small blocks and performing L2 normalization on the direction histogram, the feature differences between different blocks can be reduced, and the stability and consistency of the features can be improved. Finally, the directional histograms of all normalized blocks are concatenated into a long vector and their average value is calculated, which not only reduces the feature dimension but also provides an overall feature description, which is convenient for subsequent feature sorting and mask generation; sorting according to the eigenvalue and selecting the block with the largest eigenvalue for mask generation ensures that the mask can focus on the area with large information, thereby improving the effect of the mask reconstruction process. The mask data and retained data generated by element-by-element multiplication enable the model to fully learn effective features during the training process and better identify and reconstruct key areas in medical images in subsequent tasks. In general, the mask operation of the obtained image blocks improves the performance of the model and the ability to understand the key features of medical images through reasonable feature extraction and mask generation strategies.

[0031] Furthermore, the design of a mask reconstruction self-supervised learning network based on convolutional neural network aims to improve the performance and accuracy of the model in processing masked medical images through feature reconstruction and loss optimization. First, the masked data is input into the encoder E to obtain the feature representation Z. The main purpose of this process is to extract the latent features in the data and effectively capture the key information that affects the reconstruction. By encoding the masked data, the model can learn the contextual information in the missing area to provide support for subsequent reconstruction; next, the feature Z is input into the decoder D, and the goal of the decoder is to reconstruct the masked data area. Through this step, the model needs to convert the latent features extracted by the encoder back to the space of the original image. This process not only promotes the recovery of image details, but also enhances the model's ability in generation tasks. The effect of reconstruction will be evaluated in subsequent steps to ensure that the model can accurately understand and reconstruct the masked area in the feature space; the mean square error between the reconstructed data and the real data is calculated to quantify the accuracy of the model reconstruction. This loss indicator provides feedback for the optimization process, helping the model adjust parameters to improve the reconstruction effect. At the same time, by constructing a total loss function that includes reconstruction loss and regularization loss, the overfitting of the model can be effectively avoided, ensuring that the training process not only focuses on the accuracy of reconstruction, but also maintains the generalization ability of the model; finally, the optimization algorithm Adam is used to minimize the total loss function, making full use of the advantages of the Adam optimizer in processing sparse gradients, and improving the convergence speed and stability of the model. Therefore, through feature reconstruction, loss calculation and optimization, an efficient training process is constructed to improve the model's understanding and recovery capabilities of key features in medical images, thereby enhancing the effect and reliability in practical applications.

[0032] It can be understood that the beneficial effects of the second aspect mentioned above can be found in the relevant description of the first aspect mentioned above, and will not be repeated here.

[0033] In summary, the mixed distribution-based double-layer sampling medical image pre-training method of the present invention significantly improves the pre-processing effect and model performance of medical images by introducing the center clipping strategy of Gaussian distribution and the mask selection strategy based on local gradient direction. In general, the present invention constructs an efficient medical image pre-training process through a series of reasonable step designs, so that the model can deeply understand the key features in medical images, significantly improve its performance and accuracy in image analysis, and thus provide more reliable support for clinical applications.

[0034] The technical solution of the present invention is further described in detail below through the accompanying drawings and embodiments. BRIEF DESCRIPTION OF THE DRAWINGS

[0035] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the following briefly introduces the drawings to be used in the description of the embodiments. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0036] Figure 1 is a flow chart of the present invention; Figure 2 It is a structural diagram of the present invention; Figure 3 Framework diagram for mask selection strategy; Figure 4 A schematic diagram of a computer device provided by an embodiment of the present invention; Figure 5 The present invention is a block diagram of an electronic device provided according to an embodiment. DETAILED DESCRIPTION

[0037] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0038] In the description of the present invention, it should be understood that the terms “include” and “comprises” indicate the presence of described features, wholes, steps, operations, elements and / or components, but do not exclude the presence or addition of one or more other features, wholes, steps, operations, elements, components and / or collections thereof.

[0039] It should also be understood that the terms used in the present specification are only for the purpose of describing specific embodiments and are not intended to limit the present invention. As used in the present specification and the appended claims, the singular forms "a", "an" and "the" are intended to include plural forms unless the context clearly indicates otherwise.

[0040] It should be further understood that the term "and / or" used in the present specification and the appended claims refers to any combination of one or more of the associated listed items and all possible combinations, and includes these combinations. For example, A and / or B can represent: A exists alone, A and B exist at the same time, and B exists alone. In addition, the character " / " in the present invention generally indicates that the associated objects are in an "or" relationship.

[0041] It should be understood that, although the terms first, second, third, etc. may be used to describe preset ranges, etc. in the embodiments of the present invention, these preset ranges should not be limited to these terms. These terms are only used to distinguish preset ranges from each other. For example, without departing from the scope of the embodiments of the present invention, the first preset range may also be referred to as the second preset range, and similarly, the second preset range may also be referred to as the first preset range.

[0042] The word "if" as used herein may be interpreted as "at the time of" or "when" or "in response to determining" or "in response to detecting", depending on the context. Similarly, the phrases "if it is determined" or "if (stated condition or event) is detected" may be interpreted as "when it is determined" or "in response to determining" or "when detecting (stated condition or event)" or "in response to detecting (stated condition or event)", depending on the context.

[0043] Various structural schematic diagrams of the embodiments disclosed in the present invention are shown in the accompanying drawings. These figures are not drawn to scale, and some details are magnified and some details may be omitted for the purpose of clear expression. The shapes of various regions and layers shown in the figures and the relative sizes and positional relationships therebetween are only exemplary, and may deviate in practice due to manufacturing tolerances or technical limitations, and those skilled in the art may additionally design regions / layers with different shapes, sizes, and relative positions according to actual needs.

[0044] The present invention provides a double-layer sampling medical image pre-training method based on mixed distribution, which is combined with a center cropping strategy based on Gaussian distribution and a mask selection strategy based on local gradient direction distribution to improve the accuracy of medical image pre-training and segmentation. The center cropping strategy of Gaussian distribution uses a reference center point that conforms to the Gaussian distribution for cropping, so that the cropped area is more concentrated on the main part of the image; while the mask selection strategy based on local gradient direction distribution captures the shape features of the image and effectively selects the foreground part with a large amount of information in the medical image, thereby improving the model's understanding and analysis capabilities of the key features of the image during the mask reconstruction process. Experimental results show that the method of the present invention significantly improves the accuracy and reliability of medical image segmentation.

[0045] See also Figure 1 The present invention provides a double-layer sampling medical image pre-training method based on mixed distribution, which describes the characteristics of image shape by capturing the local gradient direction distribution, and then performs component masking according to patch information to improve the representation learning ability of the network. The specific steps are as follows: S1. Preprocessing of original medical images (including magnetic resonance imaging (MRI), computed tomography (CT), positron emission tomography (PET), and ultrasound imaging (US)); First, increase the dimension of the original image, rearrange the coordinate axis direction of the image, adjust the image resolution and pixel intensity, and fill it; then calculate the effective target area within the image size; generate a center point in the target area that conforms to a Gaussian distribution with a mean of 0 and a variance of 1; then scale the center point to the size of the effective target area, and crop based on the geometric center point.

[0046] S101, adding a channel dimension to the image data; S102, rearrange the coordinate axis direction of the image, and standardize the coordinate axis direction of the image into a right, front, and top format; S103: Adjust the voxel size of the image for :

[0047] in, is the interpolation weight function, and The original and resized images respectively.

[0048] S104, normalizing the pixel intensity of the image to a specified range:

[0049] in, is the original intensity value, is the normalized intensity value, is the original intensity range, is the target intensity range.

[0050] S105, filling the image with space to make it reach the specified space size; S106, calculating the effective target area within the image size in the training set, generating a center point that conforms to a Gaussian distribution with a mean of 0 and a variance of 1 within the target area, scaling the center point to the effective area, cropping with the geometric center as a reference point, and extracting image blocks of a specific size from each image.

[0051]

[0052] in, are the parameters of the Gaussian distribution, is a valid target area.

[0053] S2, calculating the Gaussian distribution reference center point of the image, cropping the medical image preprocessed in step S1 into multiple image blocks of the same size, and inputting the image blocks as input data into the self-supervised pre-training network in sequence; S3, formulating a mask strategy, performing a mask operation on the image block obtained in step S2, and generating masked data and retained data; See also Figure 3 , the mask selection strategy is as follows: S301, split the image block according to its channel dimension, and each channel is regarded as a two-dimensional slice. Each slice is converted into an 8-bit unsigned integer, and the pixel value range is standardized to 0 to 255. Then, these slices are resized to Pixels; S302: for each pixel of the image Calculate gradients to extract edge information and directional features; Flat gradient It is determined by calculating the difference in image pixel values ​​in the horizontal direction. The specific calculation method uses the Sobel operator formula as follows:

[0054] Vertical Gradient The difference in the image pixel values ​​is calculated in the vertical direction, and the calculation formula is:

[0055] Gradient Amplitude Represents the magnitude of the gradient at the pixel point, and its calculation formula is:

[0056] Gradient direction Represents the angle of the gradient vector relative to the horizontal axis, and its calculation formula is:

[0057] S303, constructing a direction histogram; First, divide the image into pieces of size In each small block of pixels, the corresponding gradient amplitude is projected into a predetermined angle interval according to the gradient direction of the pixel point to form a direction histogram.

[0058] Specifically, the direction is divided into 9 intervals, each covering an angle range of 20°. For each pixel, its gradient magnitude is assigned to the corresponding angle interval according to the gradient direction and accumulated into the histogram.

[0059] S304, block normalization; Combine several adjacent small blocks into a larger block and perform L2 normalization on the direction histogram within the block:

[0060] S305, concatenating all normalized direction histogram vectors of the blocks into a long vector to form a final feature vector; then, calculating the average value of these feature vectors to obtain the feature value of the image block; S306: Sort the image blocks according to their eigenvalues ​​and select several blocks with larger eigenvalues ​​for generating masks. These blocks with larger eigenvalues ​​usually contain more key information, and mask the elements in the data at the mask rate. By preferentially selecting blocks with larger information content, a more representative mask is generated.

[0061] S307 . Use the mask matrix to generate masked data and retained data.

[0062] Masked data and retained data They are:

[0063]

[0064] where ⊙ represents element-wise multiplication.

[0065] S4. Build a mask reconstruction self-supervised learning network based on convolutional neural network (CNN), input the generated masked data into the mask reconstruction self-supervised pre-training network, and complete the double-layer sampling medical image pre-training; S401, masked data Input to encoder In the feature representation ; Feature Representation Specifically:

[0066] S402: Input the feature Z output by the encoder into the decoder. The goal is to reconstruct the masked data region; Masked area reconstructed by decoder for:

[0067] in, It is a feature.

[0068] S403, calculating reconstruction data and real data Mean square error (MSE) between

[0069] in, and They are Samples, The true and reconstructed values ​​of the features.

[0070] S404, the total loss includes reconstruction loss and regularization loss;

[0071]

[0072] in, is the regularization weight, yes Regularized loss, are model parameters.

[0073] S405, using the optimization algorithm Adam to minimize the total loss function.

[0074] Update model parameters :

[0075] in, is the learning rate, is the gradient of the loss function with respect to the model parameters.

[0076] S5. Transfer the weights of the trained self-supervised mask reconstruction network to a fully supervised segmentation model with the same network structure, and train the fully supervised segmentation model using labeled image data.

[0077] In this phase, the segmentation model is trained using labeled image data to optimize the performance of the model in the actual segmentation task.

[0078] It will be appreciated by those skilled in the art that various aspects of the present invention may be implemented as systems, methods or program products. Therefore, various aspects of the present invention may be specifically implemented in the following forms, namely: complete hardware implementation, complete software implementation (including firmware, microcode, etc.), or a combination of hardware and software implementations, which may be collectively referred to herein as "circuits", "modules" or "platforms".

[0079] In another embodiment of the present invention, a double-layer sampling medical image pre-training system based on mixed distribution is provided, which can be used to implement the above-mentioned double-layer sampling medical image pre-training method based on mixed distribution. Specifically, the double-layer sampling medical image pre-training system based on mixed distribution includes a pre-processing module, a network module, a strategy module, a construction module and a fine-tuning module.

[0080] Among them, the preprocessing module preprocesses the original medical image; Add a channel dimension to the image data; rearrange the coordinate axis direction of the image and standardize the coordinate axis direction of the image to the right, front, and top format; change the voxel size of the image Adjust to ; Normalize the pixel intensity of the image to the specified range; fill the image space to reach the specified space size; calculate the effective target area within the image size in the training set, generate a center point that conforms to the Gaussian distribution with a mean of 0 and a variance of 1 in the target area, scale the corresponding center point into the valid area, crop with the corresponding geometric center as the reference point, and extract the image block of the specified size from each image.

[0081] Among them, the original medical images include magnetic resonance imaging, computed tomography, positron emission tomography and ultrasound imaging images.

[0082] Adjust the voxel size of the image for:

[0083] in, is the interpolation weight function, and The original and resized images respectively.

[0084] Normalize the pixel intensity of an image to a specified range as follows:

[0085] in, is the original intensity value, is the normalized intensity value, is the original intensity range, is the target intensity range.

[0086] Extracting image blocks of a specific size from each image is as follows:

[0087] in, for, for, are the parameters of the Gaussian distribution, is the effective target area, is the minimum size of the image block that can be extracted, is the maximum size of the image block that can be extracted.

[0088] A network module that crops the preprocessed medical image into multiple image blocks of the same size; The strategy module formulates a mask strategy, performs mask operations on the obtained image blocks, and generates masked data and retained data; The image block is split according to its channel dimension, with each channel as a two-dimensional slice; each slice is converted to an 8-bit unsigned integer and the pixel value range is standardized to 0 to 255; then the slice size is resized to Pixels; calculate each pixel in the image The gradient of Small blocks of pixels, in each small block, according to the gradient direction of the pixel point, its corresponding gradient amplitude is projected into a predetermined angle interval to form a direction histogram; several adjacent small blocks are combined into a large block, and the direction histogram in the block is L2 normalized; the direction histogram vectors of all normalized blocks are connected in series into a long vector to form the final feature vector; then, the eigenvalue of the corresponding image block is obtained by calculating the average value of these eigenvectors; the image blocks are sorted according to their eigenvalues, and several blocks with the largest eigenvalues ​​are selected to generate masks; the masked data is generated using the mask matrix and retained data , ⊙ represents element-by-element multiplication.

[0089] The construction module builds a mask reconstruction self-supervised learning network based on a convolutional neural network, inputs the generated masked data into the mask reconstruction self-supervised pre-training network, and completes the double-layer sampling medical image pre-training; The masked data Input to encoder In the feature representation ; Input the feature Z output by the encoder into the decoder, the decoder The goal is to reconstruct the masked data area; calculate the reconstructed data and real data The mean square error between them; construct a total loss function including reconstruction loss and regularization loss; use the optimization algorithm Adam to minimize the total loss function, the total loss function Specifically:

[0090] in, is the regularization weight, yes Regularized loss, Reconstruction data and real data The mean square error between .

[0091] The fine-tuning module transfers the weights of the obtained mask reconstruction self-supervised learning network to the fully supervised segmentation model with the same network structure, and trains the fully supervised segmentation model using labeled image data.

[0092] In another embodiment of the present invention, a terminal device is provided, which includes a processor and a memory, wherein the memory is used to store a computer program, the computer program includes program instructions, and the processor is used to execute the program instructions stored in the computer storage medium. The processor may be a central processing unit (CPU), or other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field-programmable gate arrays (FPGA) or other programmable logic devices, discrete gates or transistor logic devices, discrete hardware components, etc. It is the computing core and control core of the terminal, which is suitable for implementing one or more instructions, and is specifically suitable for loading and executing one or more instructions to implement the corresponding method flow or corresponding function; the processor described in the embodiment of the present invention can be used for the operation of the double-layer sampling medical image pre-training method based on mixed distribution, including: Preprocess the original medical image; crop the preprocessed medical image into multiple image blocks of the same size; formulate a masking strategy, perform masking operations on the obtained image blocks, and generate masked data and retained data; construct a mask reconstruction self-supervised learning network based on a convolutional neural network, input the generated masked data into the mask reconstruction self-supervised pre-training network, and complete the double-layer sampling medical image pre-training; transfer the weights of the mask reconstruction self-supervised learning network to a fully supervised segmentation model with the same network structure, and use labeled image data to train the fully supervised segmentation model.

[0093] In another embodiment of the present invention, the present invention also provides a storage medium, specifically a computer-readable storage medium (Memory), which is a memory device in a terminal device for storing programs and data. It is understandable that the computer-readable storage medium here can include both the built-in storage medium in the terminal device and the extended storage medium supported by the terminal device, and can be any tangible medium containing or storing a program, which can be used by an instruction execution system, device or device or used in combination with it. The computer-readable storage medium provides a storage space, which stores the operating system of the terminal. In addition, one or more instructions suitable for being loaded and executed by the processor are also stored in the storage space, and these instructions can be one or more computer programs (including program codes). It should be noted that more specific examples (non-exhaustive list) of the computer-readable storage medium here include: an electrical connection with one or more wires, a portable disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above.

[0094] Computer readable storage media also include data signals propagated in baseband or as part of a carrier wave, which carry readable program codes. Such propagated data signals can take a variety of forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination of the above. The readable storage medium can also be any readable medium other than a readable storage medium, which can send, propagate, or transmit a program for use by or in conjunction with an instruction execution system, device, or device. The program code contained on the readable storage medium can be transmitted using any appropriate medium, including but not limited to wireless, wired, optical cable, RF, etc., or any suitable combination of the above.

[0095] Program code for performing the operations of the present invention may be written in any combination of one or more programming languages, including object-oriented programming languages ​​such as Java, C++, etc., and conventional procedural programming languages ​​such as "C" or similar programming languages. The program code may be executed entirely on the user computing device, partially on the user device, as a separate software package, partially on the user computing device and partially on a remote computing device, or entirely on a remote computing device or server. In the case of a remote computing device, the remote computing device may be connected to the user computing device through any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computing device (e.g., through the Internet using an Internet service provider).

[0096] The processor may load and execute one or more instructions stored in a computer-readable storage medium to implement the corresponding steps of the two-layer sampling medical image pre-training method based on mixed distribution in the above embodiment; the processor may load and execute the following steps: Preprocess the original medical image; crop the preprocessed medical image into multiple image blocks of the same size; formulate a masking strategy, perform masking operations on the obtained image blocks, and generate masked data and retained data; construct a mask reconstruction self-supervised learning network based on a convolutional neural network, input the generated masked data into the mask reconstruction self-supervised pre-training network, and complete the double-layer sampling medical image pre-training; transfer the weights of the mask reconstruction self-supervised learning network to a fully supervised segmentation model with the same network structure, and use labeled image data to train the fully supervised segmentation model.

[0097] See also Figure 4 , the terminal device is a computer device, and the computer device 60 of this embodiment includes: a processor 61, a memory 62, and a computer program 63 stored in the memory 62 and executable on the processor 61. When the computer program 63 is executed by the processor 61, the two-layer sampling medical image pre-training method based on mixed distribution in the embodiment is implemented. To avoid repetition, it is not described one by one here. Alternatively, when the computer program 63 is executed by the processor 61, the functions of each model / unit in the two-layer sampling medical image pre-training system based on mixed distribution in the embodiment are implemented. To avoid repetition, it is not described one by one here.

[0098] The computer device 60 may be a computing device such as a desktop computer, a notebook, a PDA, or a cloud server. The computer device 60 may include, but is not limited to, a processor 61 and a memory 62. Those skilled in the art will appreciate that Figure 4 This is only an example of the computer device 60 and does not constitute a limitation of the computer device 60. It may include more or fewer components than shown in the figure, or a combination of certain components, or different components. For example, the computer device may also include input and output devices, network access devices, buses, etc.

[0099] The processor 61 may be a central processing unit (CPU), or other general-purpose processors, central processing units, graphics processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field-programmable gate arrays (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, data processing logic based on quantum computing, discrete hardware components, etc. The general-purpose processor may be a microprocessor or the processor may be any conventional processor, etc.

[0100] The memory 62 may be an internal storage unit of the computer device 60, such as a hard disk or memory of the computer device 60. The memory 62 may also be an external storage device of the computer device 60, such as a plug-in hard disk, a smart media card (SMC), a secure digital (SD) card, a flash card, etc., equipped on the computer device 60.

[0101] Furthermore, the memory 62 may include both an internal storage unit of the computer device 60 and an external storage device. The memory 62 is used to store computer programs and other programs and data required by the computer device. The memory 62 may also be used to temporarily store data that has been output or is to be output.

[0102] Any reference to a memory, database or other medium used in the embodiments provided in the present application may include at least one of a non-volatile and a volatile memory. Non-volatile memory may include a read-only memory (ROM), a magnetic tape, a floppy disk, a flash memory, an optical memory, a high-density embedded non-volatile memory, a resistive random access memory (ReRAM), a magnetic random access memory (MRAM), a ferroelectric random access memory (FRAM), a phase change memory (PCM), a graphene memory, etc. Volatile memory may include a random access memory (RAM) or an external cache memory, etc. As an illustration and not limitation, RAM may be in various forms, such as a static random access memory (SRAM) or a dynamic random access memory (DRAM), etc.

[0103] The database involved in each embodiment provided in this application may include at least one of a relational database and a non-relational database. The non-relational database may include a distributed database based on blockchain, etc., but is not limited thereto. The processor involved in each embodiment provided in this application may be a general-purpose processor, a central processing unit, a graphics processor, a digital signal processor, a programmable logic device, a data processing logic device based on quantum computing, etc., but is not limited thereto.

[0104] See also Figure 5 The terminal device is an electronic device 600, and the electronic device is in the form of a general computing device. The components of the electronic device may include, but are not limited to: at least one processing unit 610, at least one storage unit 620, a bus 630 connecting different platform components (including the storage unit 620 and the processing unit 610), a display unit 640, etc.

[0105] The storage unit stores program codes, which can be executed by the processing unit 610, so that the processing unit 610 performs the steps according to various exemplary embodiments of the present invention described in the above method section of this specification. For example, the processing unit 610 can perform the following steps: Figure 1 Follow the steps shown in .

[0106] The storage unit 620 may include a readable medium in the form of a volatile storage unit, such as a random access memory unit (RAM) 6201 and / or a cache memory unit 6202 , and may further include a read-only memory unit (ROM) 6203 .

[0107] The storage unit 620 may also include a program / utility 6204 having a set (at least one) of program modules 6205, such program modules 6205 including but not limited to: an operating system, one or more application programs, other program modules, and program data, each of which or some combination may include an implementation of a network environment.

[0108] Bus 630 may represent one or more of several types of bus structures, including a memory unit bus or memory unit controller, a peripheral bus, an accelerated graphics port, a processing unit, or a local bus using any of a variety of bus architectures.

[0109] The electronic device 600 may also communicate with one or more external devices 700 (e.g., keyboards, pointing devices, Bluetooth devices, etc.), may also communicate with one or more devices that enable a user to interact with the electronic device 600, and / or communicate with any device that enables the electronic device 600 to communicate with one or more other computing devices (e.g., routers, modems, etc.). Such communication may be performed via an input / output (I / O) interface 650. Furthermore, the electronic device 600 may also communicate with one or more networks (e.g., a local area network (LAN), a wide area network (WAN), and / or a public network, such as the Internet) via a network adapter 660. The network adapter 660 may communicate with other modules of the electronic device 600 via a bus 630. It should be understood that, although not shown in the figure, other hardware and / or software modules may be used in conjunction with the electronic device 600, including but not limited to: microcode, device drivers, redundant processing units, external disk drive arrays, RAID systems, tape drives, and data backup storage platforms, etc.

[0110] In order to make the purpose, technical solutions and advantages of the embodiments of the present invention clearer, the technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, rather than all of the embodiments. The components of the embodiments of the present invention described and shown in the drawings here can usually be arranged and designed in various different configurations. Therefore, the following detailed description of the embodiments of the present invention provided in the drawings is not intended to limit the scope of the claimed invention, but merely represents selected embodiments of the present invention. Based on the embodiments in the present invention, all other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of the present invention.

[0111] Step S1 uses a center cropping strategy based on Gaussian distribution to make the cropped area cover different positions, increase the diversity of data, and help the model learn more robust features. In this process, the effective size of the image is first calculated, and then a random number with a mean of 0 and a variance of 1 is generated and scaled to a valid patch size range. Since the probability density of the Gaussian distribution is highest in the central area, this method tends to select an area close to the geometric center of the image for cropping, thereby retaining the main content of the image. Gaussian distribution provides a more natural offset pattern, simulating the distribution of observation points or object positions in real scenes, which helps to improve the generalization ability of the model.

[0112] Step S3 extracts the local shape features of the image blocks so that the mask can more accurately reflect the key information in the image. In this process, the feature values ​​of the image blocks are first sorted, and then the important image blocks are adaptively selected to generate the mask based on the sorting results. This method improves the effectiveness of the mask, thereby improving the accuracy of the overall image segmentation.

[0113] Table 1

[0114] The experimental results are shown in Table 1. The method of the present invention performs well in the medical image segmentation task and significantly improves the accuracy and reliability of segmentation.

[0115] In summary, the present invention, a double-layer sampling medical image pre-training method and system based on mixed distribution, has shown good performance in practical applications. Through experimental verification of a large number of medical image data sets, the center cropping strategy based on Gaussian distribution and the mask selection strategy based on local gradient direction distribution not only effectively improve the representation learning ability of the model in the pre-training stage, but also perform well in downstream tasks such as tumor segmentation; these strategies can effectively reduce the consumption of training time and computing resources, while improving the accuracy and robustness of the model.

[0116] The technicians in the relevant field can clearly understand that for the convenience and simplicity of description, only the division of the above-mentioned functional units and modules is used as an example for illustration. In practical applications, the above-mentioned function allocation can be completed by different functional units and modules as needed, that is, the internal structure of the device can be divided into different functional units or modules to complete all or part of the functions described above. The functional units and modules in the embodiment can be integrated in a processing unit, or each unit can exist physically separately, or two or more units can be integrated in one unit. The above-mentioned integrated unit can be implemented in the form of hardware or in the form of software functional units. In addition, the specific names of the functional units and modules are only for the convenience of distinguishing each other, and are not used to limit the scope of protection of this application. The specific working process of the units and modules in the above-mentioned system can refer to the corresponding process in the aforementioned method embodiment, which will not be repeated here.

[0117] In the above embodiments, the description of each embodiment has its own emphasis. For parts that are not described or recorded in detail in a certain embodiment, reference can be made to the relevant descriptions of other embodiments.

[0118] Those skilled in the art will appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed in the present invention can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of the present invention.

[0119] In the embodiments provided by the present invention, it should be understood that the disclosed devices / terminals and methods can be implemented in other ways. For example, the device / terminal embodiments described above are only schematic. For example, the division of the modules or units is only a logical function division. There may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms.

[0120] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0121] In addition, each functional unit in each embodiment of the present invention may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of software functional units.

[0122] If the integrated module / unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the present invention implements all or part of the processes in the above-mentioned embodiment method, and can also be completed by instructing the relevant hardware through a computer program. The computer program can be stored in a computer-readable storage medium, and the computer program can implement the steps of the above-mentioned various method embodiments when executed by the processor. Among them, the computer program includes computer program code, and the computer program code can be in source code form, object code form, executable file or some intermediate form. The computer-readable medium may include: any entity or device capable of carrying the computer program code, recording medium, U disk, mobile hard disk, disk, optical disk, computer memory, read-only memory (ROM), random access memory (RAM), electric carrier signal, telecommunication signal and software distribution medium, etc. It should be noted that the content contained in the computer-readable medium can be appropriately increased or decreased according to the requirements of legislation and patent practice in the jurisdiction. For example, in some jurisdictions, according to legislation and patent practice, computer-readable media do not include electric carrier signals and telecommunication signals.

[0123] The present application is described with reference to the flowcharts and / or block diagrams of the methods, devices (systems), and computer program products according to the embodiments of the present application. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of the processes and / or boxes in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0124] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0125] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0126] The above contents are only for explaining the technical idea of ​​the present invention and cannot be used to limit the protection scope of the present invention. Any changes made on the basis of the technical solution in accordance with the technical idea proposed by the present invention shall fall within the protection scope of the claims of the present invention.

Claims

1. A two-layer sampling medical image pre-training method based on mixed distribution, characterized in that: The following steps are involved: Preprocessing the original medical images; Crop the preprocessed medical image into multiple image blocks of the same size; Formulate a masking strategy, perform masking operations on the obtained image blocks, and generate masked data and retained data; Construct a mask reconstruction self-supervised learning network based on convolutional neural network, input the generated masked data into the mask reconstruction self-supervised pre-training network, and complete the double-layer sampling medical image pre-training; The weights of the obtained mask reconstruction self-supervised learning network are transferred to a fully supervised segmentation model with the same network structure, and the fully supervised segmentation model is trained using labeled image data.

2. The medical image pre-training method based on double-layer sampling of mixed distribution according to claim 1, characterized in that: The preprocessing of the original medical image is as follows: Add a channel dimension to the image data; rearrange the coordinate axis direction of the image and standardize the coordinate axis direction of the image to the right, front, and top format; change the voxel size of the image Adjust to ; Normalize the pixel intensity of the image to the specified range; fill the image space to reach the specified space size; calculate the effective target area within the image size in the training set, generate a center point that conforms to the Gaussian distribution with a mean of 0 and a variance of 1 in the target area, scale the corresponding center point into the valid area, crop with the corresponding geometric center as the reference point, and extract the image block of the specified size from each image.

3. The medical image pre-training method based on double-layer sampling of mixed distribution according to claim 2, characterized in that: The raw medical images include magnetic resonance imaging, computed tomography, positron emission tomography, and ultrasound imaging images.

4. The medical image pre-training method based on double-layer sampling of mixed distribution according to claim 2, characterized in that: Adjust the voxel size of the image for: in, is the interpolation weight function, and The original and resized images respectively.

5. The medical image pre-training method based on double-layer sampling of mixed distribution according to claim 2, characterized in that: Normalize the pixel intensity of an image to a specified range as follows: in, is the original intensity value, is the normalized intensity value, is the original intensity range, is the target intensity range.

6. The medical image pre-training method based on double-layer sampling of mixed distribution according to claim 2, characterized in that: Extracting image blocks of a specific size from each image is as follows: in, for, for, are the parameters of the Gaussian distribution, is the effective target area, is the minimum size of the image block that can be extracted, is the maximum size of the image block that can be extracted.

7. The medical image pre-training method based on double-layer sampling of mixed distribution according to claim 1, characterized in that: The masked data and retained data are generated as follows: The image block is split according to its channel dimension, with each channel as a two-dimensional slice; each slice is converted to an 8-bit unsigned integer and the pixel value range is standardized to 0 to 255; then the slice size is resized to Pixels; Calculate the value of each pixel in the image The gradient of Divide the image into pieces of size Small blocks of pixels, in each small block, according to the gradient direction of the pixel point, its corresponding gradient amplitude is projected into a predetermined angle interval to form a direction histogram; Combine several adjacent small blocks into a large block and perform L2 normalization on the direction histogram within the block; The directional histogram vectors of all normalized blocks are concatenated into a long vector to form the final feature vector; then, the feature value of the corresponding image block is obtained by calculating the average value of these feature vectors; Sort the image blocks according to their eigenvalues ​​and select several blocks with the largest eigenvalues ​​to generate masks; Use the mask matrix to generate masked data and retained data , ⊙ represents element-by-element multiplication.

8. The medical image pre-training method based on double-layer sampling of mixed distribution according to claim 1, characterized in that: Constructing a mask reconstruction self-supervised learning network based on convolutional neural network is as follows: The masked data Input to encoder In the feature representation ; The feature Z output by the encoder is input into the decoder. The goal is to reconstruct the masked data area; calculate the reconstructed data and real data The mean square error between them; construct a total loss function including reconstruction loss and regularization loss; use the optimization algorithm Adam to minimize the total loss function.

9. The medical image pre-training method based on double-layer sampling of mixed distribution according to claim 8, characterized in that: Total loss function Specifically: in, is the regularization weight, yes Regularized loss, Reconstruction data and real data The mean square error between .

10. A two-layer sampling medical image pre-training system based on mixed distribution, characterized in that: include: A preprocessing module is used to preprocess the original medical images; The network module crops the preprocessed medical image into multiple image blocks of the same size, and sequentially inputs the image blocks as input data into the convolutional neural network; The strategy module formulates the mask strategy, performs mask operations on the obtained image blocks, and generates masked data and retained data; The construction module builds a mask reconstruction self-supervised learning network based on a convolutional neural network, inputs the generated masked data into the mask reconstruction self-supervised pre-training network, and completes the double-layer sampling medical image pre-training; The fine-tuning module transfers the weights of the obtained mask reconstruction self-supervised learning network to the fully supervised segmentation model with the same network structure, and trains the fully supervised segmentation model using labeled image data.

Images