Medical image feature processing method and device based on neural network and electronic equipment

Abstract

The application discloses a medical image feature processing method and device based on a neural network and electronic equipment, and relates to the technical field of medical treatment. The method comprises the following steps: extracting initial image features from a medical image through an imageomics feature extraction strategy and / or N feature extraction models, wherein the feature extraction model determines whether to freeze model parameters and the number of model parameters to be frozen according to the number of training samples; processing the initial image features into target image features through a neural network, wherein the dimension of the target image features is lower than that of the initial image features; and processing a prediction task corresponding to the medical image based on the target image features through the neural network. The application solves the technical problem that, in the prior art, when features are calculated based on an image ROI region and a prediction task is processed, the effectiveness of medical image feature acquisition is insufficient due to the limitation of a small number of training samples and the singularity of a feature calculation method, thereby affecting the execution accuracy of the prediction task.

Description

Technical Field

[0001] This application relates to the fields of medical technology and artificial intelligence, and more specifically, to a method, apparatus, and electronic device for medical image feature processing based on neural networks. Background Technology

[0002] Common medical images (such as CT images, MR images, PET images, etc.) are characterized by high dimensionality (e.g., 3D) and small sample size. Due to this characteristic, classification tasks (such as efficacy prediction) usually require more complex deep neural network models (with more parameters) to effectively process high-dimensional input images. However, models with more parameters also require more training samples to fit the model. The small sample size characteristic of medical images becomes a constraint at this point.

[0003] Under the aforementioned constraints, existing technologies, when calculating features based on image ROIs (Regions of Interest) and performing prediction tasks, suffer from insufficient effectiveness in acquiring medical image features due to limited training samples and the simplistic nature of feature calculation methods, thus affecting the accuracy of prediction task execution.

[0004] There is currently no effective solution to the above problems. Summary of the Invention

[0005] This application provides a method, apparatus, and electronic device for medical image feature processing based on neural networks, which at least solves the technical problem in the prior art that when calculating features based on image ROI regions and performing prediction tasks, the effectiveness of medical image feature acquisition is insufficient due to the limited number of training samples and the singularity of feature calculation methods, thereby affecting the accuracy of prediction task execution.

[0006] According to one aspect of this application, a neural network-based medical image feature processing method is provided, comprising: extracting initial image features from a medical image using a radiomics feature extraction strategy and / or N feature extraction models, wherein the radiomics feature extraction strategy includes a radiomics algorithm for extracting image features, N is an integer greater than or equal to 1, and the feature extraction models select whether to freeze model parameters and the number of model parameters to be frozen based on the number of training samples; processing the initial image features into target image features using a neural network, wherein the dimension of the target image features is lower than the dimension of the initial image features; and processing a prediction task corresponding to the medical image based on the target image features using a neural network.

[0007] Optionally, initial image features are extracted from medical images using N feature extraction models, including: when N is an integer greater than 1, at least N initial image features are extracted from medical images using N feature extraction models, wherein different feature extraction models extract different initial image features, and / or different feature extraction models use different feature extraction methods.

[0008] Optionally, initial image features are extracted from medical images using N feature extraction models, including: when the medical image is divided into a slice sequence consisting of M slices, the initial image features are obtained by performing multiple target operations on the slice sequence using N feature extraction models, wherein the multiple target operations are used to ensure that feature extraction is completed for each slice in the slice sequence.

[0009] Optionally, the i-th target operation in a series of target operations includes the following steps:

[0010] X adjacent slices are selected from the slice sequence as a slice combination, and N feature extraction models are used to extract the initial image features corresponding to the slice combination. Here, X is equal to the number of image processing channels set by the feature extraction model, and M is an integer greater than X.

[0011] The method of sequentially shifting slices in the slice sequence is used to select X adjacent slices used in the (i+1)th target operation from the slice sequence.

[0012] Optionally, the neural network-based medical image feature processing method further includes: when selecting X adjacent slices by translating slice order, the number of slices selected in each translation is greater than or equal to 1 and less than or equal to X.

[0013] Optionally, the i-th target operation in a series of target operations includes the following steps:

[0014] Determine the center slice of the region of interest from the slice sequence;

[0015] The first and last slices are selected from the slice sequence and together with the center slice to form a slice combination. The initial image features corresponding to the slice combination are extracted from the slice combination determined in this study using N feature extraction models. The number of slices in the slice combination is equal to the number of image processing channels set by the feature extraction model.

[0016] The method involves shifting the first and last slices toward the center slice to select the slice combination to be used in the next target operation from the slice sequence. Each target operation uses a slice combination that includes the center slice, but the non-center slices included in the slice combination used by any two target operations are different slices.

[0017] Optionally, initial image features are extracted from medical images using N feature extraction models, including: when the medical image is divided into a slice sequence consisting of M slices, selecting X slices from the slice sequence and performing feature extraction on the selected X slices to obtain initial image features; and / or selecting 1 slice from the slice sequence and copying the selected 1 slice X times, performing feature extraction on the copied slices to obtain initial image features, where X is equal to the number of image processing channels set by the feature extraction model.

[0018] According to another aspect of the embodiments of this application, a medical image feature processing device based on a neural network is also provided. The device includes: a feature extraction unit, configured to extract initial image features from a medical image using a radiomics feature extraction strategy and / or N feature extraction models, wherein the radiomics feature extraction strategy includes a radiomics algorithm for extracting image features, N is an integer greater than or equal to 1, and the feature extraction models select whether to freeze model parameters and the number of model parameters to be frozen based on the number of training samples; a feature dimensionality reduction unit, configured to process the initial image features into target image features using a neural network, wherein the dimension of the target image features is lower than the dimension of the initial image features; and a task processing unit, configured to process a prediction task corresponding to the medical image based on the target image features using a neural network.

[0019] According to another aspect of the embodiments of this application, a computer-readable storage medium is also provided, wherein a computer program is stored in the computer-readable storage medium, wherein when the computer program is executed, the device where the computer-readable storage medium is located performs the above-described neural network-based medical image feature processing method.

[0020] According to another aspect of the embodiments of this application, an electronic device is also provided, wherein the electronic device includes one or more processors and a memory, the memory being used to store one or more programs, wherein when the one or more programs are executed by one or more processors, the one or more processors cause the one or more processors to perform the above-described neural network-based medical image feature processing method.

[0021] This application first extracts initial image features from medical images using a radiomics feature extraction strategy and / or N feature extraction models. The radiomics feature extraction strategy includes a radiomics algorithm for extracting image features, where N is an integer greater than or equal to 1. The feature extraction models select whether to freeze model parameters and the number of parameters to freeze based on the number of training samples. Then, a neural network processes the initial image features into target image features, where the dimensionality of the target image features is lower than that of the initial image features. Finally, the neural network performs a prediction task based on the target image features for the corresponding medical image.

[0022] As described above, this application extracts features from medical images using a radiomics feature extraction strategy and / or N feature extraction models. This allows for the capture of feature information from multiple angles and levels, significantly enhancing the richness and diversity of features and addressing the problem of overly simplistic feature extraction methods in existing technologies. Furthermore, after obtaining the initial image features, this application can use a neural network to reduce the dimensionality of the initial image features into target image features, thereby eliminating redundant feature dimensions and making the remaining feature data more suitable for downstream prediction tasks, thus improving the processing efficiency of the neural network model for feature data.

[0023] Finally, in this application, the feature extraction model can also flexibly adjust the state of the model parameters (including whether to freeze model parameters and how many model parameters to freeze) based on the number of training samples. For example, when there are few training samples, most model parameters are frozen, and only the key layers are fine-tuned. This can reduce the requirement for a large number of samples while ensuring the model's feature extraction capability. Thus, even with a small number of training samples, a feature extraction model with a sufficiently good fit can be obtained. This solves the technical problem in the prior art where, when calculating features based on image ROI regions and performing prediction tasks, the effectiveness of medical image feature acquisition is insufficient due to the limited number of training samples and the single feature calculation method, thereby affecting the accuracy of the prediction task. Attached Figure Description

[0024] The accompanying drawings, which are included to provide a further understanding of this application and form part of this application, illustrate exemplary embodiments of this application and are used to explain this application, but do not constitute an undue limitation of this application. In the drawings:

[0025] Figure 1 This is a flowchart of an optional neural network-based medical image feature processing method according to an embodiment of this application;

[0026] Figure 2 This is a schematic diagram of an optional medical image feature processing method according to an embodiment of this application;

[0027] Figure 3 This is a schematic diagram of a target operation according to an embodiment of this application;

[0028] Figure 4 This is a schematic diagram of another target operation according to an embodiment of this application;

[0029] Figure 5 This is a schematic diagram of an optional neural network-based medical image feature processing device according to an embodiment of this application. Detailed Implementation

[0030] To enable those skilled in the art to better understand the present application, the technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present application, and not all embodiments. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative effort should fall within the scope of protection of the present application.

[0031] It should be noted that the terms "first," "second," etc., in the specification, claims, and accompanying drawings of this application are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments of this application described herein can be implemented in orders other than those illustrated or described herein. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover non-exclusive inclusion; for example, a process, method, system, product, or apparatus that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus.

[0032] It should also be noted that the information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, stored data, and displayed data) collected in this application are information and data authorized by the user or fully authorized by all parties. Furthermore, the collection, storage, use, processing, transmission, provision, disclosure, and application of related data all comply with the relevant laws, regulations, and standards of the relevant regions, necessary confidentiality measures have been taken, and they do not violate public order and good morals. Corresponding operation entry points are provided for users to choose to authorize or refuse. For example, this system has interfaces with relevant users or organizations. Before obtaining relevant information, a request to obtain the information needs to be sent to the aforementioned user or organization through the interface, and the relevant information is obtained only after receiving consent from the aforementioned user or organization.

[0033] According to an embodiment of this application, an embodiment of a medical image feature processing method based on neural networks is provided. It should be noted that the steps shown in the flowchart in the accompanying drawings can be executed in a computer system such as a set of computer-executable instructions. Furthermore, although a logical order is shown in the flowchart, in some cases, the steps shown or described may be executed in a different order than that shown here.

[0034] Figure 1 This is a flowchart of an optional neural network-based medical image feature processing method according to an embodiment of this application, such as... Figure 1 As shown, the method includes the following steps:

[0035] Step S101: Initial image features are extracted from medical images using radiomics feature extraction strategies and / or N feature extraction models.

[0036] In step S101, the radiomics feature extraction strategy includes a radiomics algorithm for extracting image features, where N is an integer greater than or equal to 1, and the feature extraction model selects whether to freeze model parameters and the number of model parameters to be frozen based on the number of training samples.

[0037] Optionally, a medical information processing system can serve as the execution subject of the neural network-based medical image feature processing method in this application embodiment. The medical information processing system can be a software system or an embedded system combining hardware and software. Furthermore, those skilled in the art should understand that, besides using a medical information processing system as the execution subject of the neural network-based medical image feature processing method in this application embodiment, other entities such as devices or apparatuses can also serve as the method execution subject in this application embodiment; this application embodiment does not particularly limit this type of entity.

[0038] In the embodiments of this application, the medical image can be a multimodal medical image or a medical image of various dimensions, such as a 2D image or a 3D image. The feature extraction model can be a model for processing multimodal medical images or a model for processing medical images of various dimensions. For example, the feature extraction model includes, but is not limited to, a model for processing 2D images and a model for processing 3D images.

[0039] Optionally, in step S101, the initial image features can be extracted from the medical image using the following method:

[0040] Method 1: Initial image features are extracted from medical images using a radiomics feature extraction strategy alone. The radiomics feature extraction strategy includes radiomics algorithms for extracting image features, such as feature statistics algorithms.

[0041] Method 2: Use a single feature extraction model (i.e., when N=1) to extract initial image features from medical images. For example, use a feature extraction model to extract initial image features directly from 3D images, or use a feature extraction model to extract initial image features directly from 2D images. Alternatively, use a feature extraction model to first divide the 3D image into a slice sequence composed of multiple 2D image slices, and then extract initial image features based on the slice sequence.

[0042] Method 3 involves using multiple feature extraction models (N > 1) to extract initial image features from medical images. Different feature extraction models extract different initial image features, and / or different feature extraction models employ different feature extraction methods. This method can extract multiple complementary features, improving feature richness.

[0043] Method 4 uses both radiomics feature extraction strategy and feature extraction model (i.e., when N=1) to extract initial image features from medical images. This combines Method 1 and Method 2, thereby improving the diversity and richness of extracted features by using a combination of radiomics feature extraction strategy and feature extraction model.

[0044] Method 5 uses radiomics feature extraction strategies and multiple feature extraction models (when N is greater than 1) to extract initial image features from medical images. This combines Method 1 and Method 3, thereby improving the diversity and richness of extracted features by using radiomics feature extraction strategies and multiple feature extraction models in combination.

[0045] It should also be noted that the feature extraction model can choose whether to freeze model parameters and the number of model parameters to be frozen based on the number of training samples. For example, when the number of training samples is greater than the target threshold, the model parameters are not frozen; when the number of training samples is less than or equal to the target threshold, the model parameters are frozen. Moreover, there is a negative correlation between the number of training samples and the number of model parameters to be frozen. That is, the fewer the number of training samples, the more model parameters need to be frozen, until all model parameters are frozen when the number of training samples is less than a preset number.

[0046] As described above, in order to train a neural network on a small medical dataset, this application selects to freeze some or all of the model parameters based on the number of training samples. This allows the model to be trained with only a very small number of parameters (e.g., the model weights of the last few layers of the neural network) and achieves a good fitting effect. This approach significantly reduces the number of network parameters that need to be fitted, thereby reducing the number of training samples required.

[0047] Step S102: The initial image features are processed into target image features through a neural network, wherein the dimension of the target image features is lower than the dimension of the initial image features.

[0048] Optionally, the medical information processing system can use neural networks to perform dimensionality reduction on the initial image features. Since a large number of features can be extracted from medical images through the aforementioned radiomics feature extraction strategies and / or N feature extraction models, it is necessary to further effectively filter redundant feature dimensions and combine these features from different sources to complete model training or prediction tasks. For example, after feature processing of the input medical image and the delineated ROI region during the feature extraction stage, features corresponding to multiple feature extraction models are obtained. The dimension of each feature is typically above 1000. The medical information processing system uses neural networks to perform dimensionality reduction on each feature.

[0049] Step S103: The prediction task corresponding to the medical image is processed by the neural network based on the target image features.

[0050] Optionally, the medical information processing system utilizes neural networks to process prediction tasks corresponding to medical images based on low-dimensional target image features, such as predicting patient treatment efficacy, predicting patient symptoms, predicting patient disease risk, predicting medical image quality, predicting medical image category labels, etc.

[0051] Figure 2 This is a schematic diagram of an optional medical image feature processing method according to an embodiment of this application, such as... Figure 2 As shown, after the medical information processing system receives medical images, it utilizes radiomics methods (radiomics algorithms) and / or N pre-trained models (corresponding to the N feature extraction models mentioned above, for example...) Figure 2 The pre-trained models 1, 2, ..., N extract high-dimensional features (corresponding to the initial image features mentioned above) from medical images. Then, the high-dimensional features are reduced in dimensionality using a neural network to obtain low-dimensional features (corresponding to the target image features mentioned above). Finally, the medical information processing system uses the classification model of the neural network to perform prediction tasks corresponding to the medical images based on the obtained low-dimensional features.

[0052] In one optional embodiment, when N is an integer greater than 1, at least N initial image features are extracted from the medical image through N feature extraction models, wherein different feature extraction models extract different initial image features, and / or different feature extraction models use different feature extraction methods.

[0053] Alternatively, assuming the ROI is a GTV region, the N feature extraction models can include the following models:

[0054] Model 1: The first type of model can be used, which employs a 3D CNN network architecture and is pre-trained on a large number of 3D CT images using a self-supervised learning method. This model can extract 3D features containing the GTV region, and supports input image sizes of various sizes. The output feature size is 512.

[0055] Model 2: A second type of model can be used, which employs a 2D CNN network architecture and is pre-trained on 2D natural images using supervised learning methods. This model can extract features from three slices within the GTV region, with an input image size of... The output feature size is 2048.

[0056] As can be seen from the above, by using multiple feature extraction models to extract features from medical images, feature information of medical images can be captured from multiple angles and levels, thereby greatly improving the richness and diversity of features and solving the problem of overly simplistic feature extraction methods in existing technologies.

[0057] In one optional embodiment, initial image features are extracted from medical images using N feature extraction models, including: when the medical image is divided into a slice sequence consisting of M slices, the initial image features are obtained by performing multiple target operations on the slice sequence using the N feature extraction models, wherein the multiple target operations are used to ensure that feature extraction is completed for each slice in the slice sequence.

[0058] Optionally, M is an integer greater than 1. For example, when the medical image is a 3D image, the medical information processing system can divide the medical image into a sequence of M 2D slices. Then, the medical information processing system can use one or more feature extraction models to perform multiple target operations on the slice sequence. The purpose of these multiple target operations is to ensure that feature extraction is completed for each slice in the slice sequence, thereby obtaining sufficiently rich and diverse initial image features. It should be noted that the feature extraction model can be a model specifically designed for processing 3D images, a model specifically designed for processing 2D images, or even a model that can handle both 3D and 2D images.

[0059] It should be noted that the feature extraction method provided in this application can generate a large number of features, while existing feature selection schemes are difficult to effectively process relevant image features. Therefore, embodiments of this application employ neural networks for feature selection and dimensionality reduction, including but not limited to pooling methods, 1D and 2D convolution methods, and combinations thereof. An optional technical solution includes: co-training the neural network with a downstream neural network classification model to achieve adaptive feature dimensionality reduction, improve model accuracy, and reduce the cost of manual trial and error and adjustment. Specifically, the feature dimensionality reduction method in the neural network can adaptively fuse and process multiple features (such as radiomics features, ResNet features, VGG features, etc.) through gradient descent during the model fitting process, resulting in better performance and higher model accuracy compared to manual selection and dimensionality reduction.

[0060] In addition, neural network technology is developing rapidly, and various innovative feature dimensionality reduction methods can be seamlessly integrated into the neural network provided in the embodiments of this application to achieve the goal of continuously optimizing the neural network design.

[0061] In one optional embodiment, the i-th target operation in a series of target operations includes the following steps:

[0062] X adjacent slices are selected from the slice sequence as a slice combination, and N feature extraction models are used to extract the initial image features corresponding to the slice combination. Here, X is equal to the number of image processing channels set by the feature extraction model, and M is an integer greater than X.

[0063] The method of sequentially shifting slices in the slice sequence is used to select X adjacent slices used in the (i+1)th target operation from the slice sequence.

[0064] Optionally, medical images are typically single-channel grayscale 3D images, while neural network models usually accept RGB three-channel 2D images. Based on this, X can be set to 3, such as... Figure 3 As shown, the first target operation provided in this application embodiment includes the following steps:

[0065] Step 1: Begin the first target operation by selecting three adjacent slices, starting from either the beginning or end of the slice sequence. For example... Figure 3 The process begins by selecting three adjacent slices from the beginning as the first slice combination, including slice 1, slice 2, and slice 3. Then, feature extraction is performed on these three slices to obtain the features corresponding to slice 1, slice 2, slice 3, and the relationship features between the three slices.

[0066] Step 2 involves selecting three adjacent slices from the slice sequence for the next target operation by performing a sequential shift within the slice sequence. Figure 3As shown, during the second target operation, a slice is shifted sequentially in the slice sequence, and slices 2, 3, and 4 are selected as the second slice combination. Then, feature extraction is performed on these three slices to obtain the features corresponding to slice 2, slice 3, slice 4, and the correlation features between the three slices (or only the features corresponding to slice 4 can be extracted, without repeating the extraction of the features of slices 2 and 3, that is, for the repeated slices in different slice combinations, only one feature operation is performed).

[0067] Step 3, as follows Figure 3 As shown, during the third target operation, slices 3, 4, and 5 are selected as the third slice combination.

[0068] This process continues until all slices in the detected slice sequence have completed feature extraction. Then, feature fusion and dimensionality reduction are performed to obtain the fused and dimensionality-reduced features.

[0069] It should be noted that X=3 is only an example. In reality, X refers to the number of model channels. If the number of model channels changes, for example, to 4, then X here will be 4. This application embodiment supports adaptive adjustment of the value of X based on the number of model channels.

[0070] Through the above multiple target operations, a large number of image features can be extracted based on the slice sequence corresponding to the medical image, thereby achieving the technical effect of improving feature diversity.

[0071] In one alternative embodiment, when selecting X adjacent slices by translating slice order, the number of slices selected in each translation is greater than or equal to 1 and less than or equal to X.

[0072] It should also be noted that, although in Figure 3 In the example, when performing the target operation, one slice is shifted in the slice sequence each time. However, in practical applications, the number of slices shifted each time only needs to not exceed X. For example, when X=3, the number of slices shifted each time can be 1, 2 or 3. The premise is that after performing multiple target operations, it is necessary to ensure that each slice in the slice sequence has completed feature extraction.

[0073] In one optional embodiment, the i-th target operation in a series of target operations includes the following steps:

[0074] Determine the center slice of the region of interest from the slice sequence;

[0075] The first and last slices are selected from the slice sequence and together with the center slice to form a slice combination. The initial image features corresponding to the slice combination are extracted from the slice combination determined in this study using N feature extraction models. The number of slices in the slice combination is equal to the number of image processing channels set by the feature extraction model.

[0076] The method involves shifting the first and last slices toward the center slice to select the slice combination to be used in the next target operation from the slice sequence. Each target operation uses a slice combination that includes the center slice, but the non-center slices included in the slice combination used by any two target operations are different slices.

[0077] Optionally, Figure 4 This is a schematic diagram illustrating another target operation according to an embodiment of this application, such as... Figure 4 As shown, this target operation is to find the center slice of the ROI region, combine the center slice with the first and last slices, and each time fix the center slice while shifting the first and last slices towards the center. Specifically, it includes the following steps:

[0078] Step one: First, select the first and last slices from the slice sequence, and combine them with the center slice to form a slice combination, such as... Figure 4 As shown, the slice combination includes slice 1, slice M and center slice. Then, the features corresponding to slice 1, slice M, center slice and the correlation features between the three slices are extracted.

[0079] Step two, shift the first slice backward by one slice, shift the last slice forward by one slice, and combine it with the center slice to form a slice combination, as shown below. Figure 4 As shown, the slice combination includes slice 2, slice M-1 and the central slice. Then, the features of these three slices and the correlation features between these three slices are extracted.

[0080] Step 3: Building upon Step 2, continue to sequentially shift the two non-center slices to obtain slice 3, slice M-2, and the center slice, forming a slice combination. Then, extract the features of these three slices and the correlation features between them.

[0081] This process continues until all slices in the detected slice sequence have completed feature extraction. Then, feature fusion and dimensionality reduction are performed to obtain the fused and dimensionality-reduced features.

[0082] In one optional embodiment, when a medical image is divided into a slice sequence consisting of M slices, the medical information processing system can select X slices from the slice sequence and perform feature extraction on the selected X slices to obtain initial image features; and / or, select 1 slice from the slice sequence and copy the selected 1 slice X times, and perform feature extraction on the copied slices to obtain initial image features, wherein X is equal to the number of image processing channels set by the feature extraction model.

[0083] Optionally, during the initial image feature extraction process, the following two feature extraction methods can also be selected:

[0084] Feature extraction method 1: Randomly select X slices from the slice sequence, and use the selected X slices as representative slices for feature extraction to obtain the initial image features.

[0085] Feature extraction method 2: Randomly select one slice from the slice sequence, then copy the slice X times, and use the copied slice as a representative slice for feature extraction to obtain the initial image features.

[0086] As can be seen from the above, this application provides multiple feature extraction methods for medical image feature extraction, and multiple feature extraction methods can be used in combination, thereby improving the flexibility of feature extraction method selection. Users can choose the feature extraction method according to the actual scenario.

[0087] According to another aspect of the embodiments of this application, a medical image feature processing device based on a neural network is also provided, wherein... Figure 5 This is a schematic diagram of an optional neural network-based medical image feature processing device according to an embodiment of this application, as shown below. Figure 5 As shown, the device includes: a feature extraction unit 501, a feature dimensionality reduction unit 502, and a task processing unit 503.

[0088] Optionally, the feature extraction unit 501 is used to extract initial image features from medical images through a radiomics feature extraction strategy and / or N feature extraction models, wherein the radiomics feature extraction strategy includes a radiomics algorithm for extracting image features, N is an integer greater than or equal to 1, and the feature extraction model selects whether to freeze model parameters and the number of model parameters to be frozen based on the number of training samples; the feature dimensionality reduction unit 502 is used to process the initial image features into target image features through a neural network, wherein the dimension of the target image features is lower than the dimension of the initial image features; the task processing unit 503 is used to process the prediction task corresponding to the medical image based on the target image features through a neural network.

[0089] Optionally, the feature extraction unit 501 includes: a feature extraction subunit, used to extract at least N initial image features from the medical image through N feature extraction models when N is an integer greater than 1, wherein different feature extraction models extract different initial image features, and / or different feature extraction models use different feature extraction methods.

[0090] Optionally, the feature extraction unit 501 includes a processing subunit, which, when a medical image is divided into a slice sequence consisting of M slices, performs multiple target operations on the slice sequence using N feature extraction models to obtain initial image features, wherein the multiple target operations are used to ensure that feature extraction is completed for each slice in the slice sequence.

[0091] Optionally, the processing subunit includes: a first execution module, configured to select X adjacent slices from the slice sequence as a slice combination, and extract initial image features corresponding to the slice combination from the slice combination through N feature extraction models, wherein X is equal to the number of image processing channels set by the feature extraction models, and M is an integer greater than X; and a second execution module, configured to select X adjacent slices used in the (i+1)th target operation from the slice sequence by performing slice sequential translation in the slice sequence.

[0092] Optionally, the neural network-based medical image feature processing device further includes: a slice selection unit, used to select X adjacent slices by sequential slice translation, wherein the number of slices selected in each translation is greater than or equal to 1 and less than or equal to X.

[0093] Optionally, the processing subunit includes: a third execution unit, used to determine the center slice of the region of interest from the slice sequence; a fourth execution unit, used to select the first slice and the last slice from the slice sequence, and form a slice combination together with the center slice, and extract the initial image features corresponding to the slice combination from the slice combination determined in this operation through N feature extraction models, wherein the number of slices included in the slice combination is equal to the number of image processing channels set by the feature extraction models; and a fifth execution unit, used to select the slice combination to be used in the next target operation from the slice sequence by shifting the first slice and the last slice towards the center slice, wherein the slice combination used in each target operation includes the center slice, but the non-center slices included in the slice combination used in any two target operations are different slices.

[0094] Optionally, the feature extraction unit 501 includes: a first processing subunit, configured to select X slices from the slice sequence when the medical image is divided into a slice sequence consisting of M slices, and perform feature extraction on the selected X slices to obtain initial image features; and / or to select 1 slice from the slice sequence, and copy the selected 1 slice X times, and perform feature extraction on the copied slices to obtain initial image features, wherein X is equal to the number of image processing channels set by the feature extraction model.

[0095] According to another aspect of the embodiments of this application, a computer-readable storage medium is also provided, wherein a computer program is stored in the computer-readable storage medium, wherein when the computer program is executed, the device where the computer-readable storage medium is located performs the above-described neural network-based medical image feature processing method.

[0096] According to another aspect of the embodiments of this application, an electronic device is also provided, wherein the electronic device includes one or more processors and a memory, the memory being used to store one or more programs, wherein when the one or more programs are executed by one or more processors, the one or more processors cause the one or more processors to perform the above-described neural network-based medical image feature processing method.

[0097] The sequence numbers of the embodiments in this application are for descriptive purposes only and do not represent the superiority or inferiority of the embodiments.

[0098] In the above embodiments of this application, the descriptions of each embodiment have different focuses. For parts not described in detail in a certain embodiment, please refer to the relevant descriptions of other embodiments.

[0099] In the several embodiments provided in this application, it should be understood that the disclosed technical content can be implemented in other ways. The device embodiments described above are merely illustrative; for example, the division of units can be a logical functional division, and in actual implementation, there may be other division methods. For instance, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the displayed or discussed mutual coupling, direct coupling, or communication connection may be through some interfaces; the indirect coupling or communication connection between units or modules may be electrical or other forms.

[0100] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0101] Furthermore, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit.

[0102] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as a USB flash drive, read-only memory (ROM), random access memory (RAM), portable hard drive, magnetic disk, or optical disk.

[0103] The above description is only a preferred embodiment of this application. It should be noted that for those skilled in the art, several improvements and modifications can be made without departing from the principle of this application, and these improvements and modifications should also be considered within the scope of protection of this application.

Claims

1. A neural network-based medical image feature processing method, characterized by, The method comprises the following steps: extracting initial image features from the medical image by an imageomics feature extraction strategy and N feature extraction models, including: in the case that the medical image is divided into a slice sequence composed of M slices, performing multiple target operations on the slice sequence by the N feature extraction models to obtain initial image features, wherein the multiple target operations are used to ensure that feature extraction is completed for each slice in the slice sequence; the imageomics feature extraction strategy comprises an imageomics algorithm for extracting image features, and N is an integer greater than or equal to 1; processing the initial image features into target image features by a neural network, wherein the dimension of the target image features is lower than that of the initial image features; processing a prediction task corresponding to the medical image based on the target image features by the neural network; the i-th target operation in the multiple target operations comprises execution mode one or execution mode two: execution mode one, comprising: selecting X adjacent slices as a slice combination from the slice sequence, and extracting initial image features corresponding to the slice combination from the slice combination by the N feature extraction models, wherein X is equal to the number of image processing channels set by the feature extraction model, M is an integer greater than X, and then X adjacent slices used in the i+1-th target operation are selected from the slice sequence by performing slice sequence translation in the slice sequence; execution mode two, comprising: determining a center slice of a region of interest from the slice sequence; selecting a head slice and a tail slice from the slice sequence to form a slice combination with the center slice, and extracting initial image features corresponding to the slice combination from the slice combination by the N feature extraction models, wherein the number of slices included in the slice combination is equal to the number of image processing channels set by the feature extraction model; and selecting a slice combination used in the next target operation from the slice sequence by translating the head slice and the tail slice to the center slice, wherein the slice combination used in each target operation includes the center slice, but the non-center slices included in the slice combinations used in any two target operations are different. 2.The neural network-based medical image feature processing method of claim 1, wherein, extracting initial image features from the medical image by N feature extraction models, comprising: when N is an integer greater than 1, extracting at least N initial image features from the medical image by the N feature extraction models, wherein the initial image features extracted by different feature extraction models are different, and / or the feature extraction modes adopted by different feature extraction models are different. 3.The neural network-based medical image feature processing method of claim 1, wherein, The neural network-based medical image feature processing method further comprises: when X adjacent slices are selected by slice sequence translation, the number of slices selected by each translation is greater than or equal to 1 and less than or equal to X. 4.The neural network-based medical image feature processing method of claim 1, wherein, The neural network-based medical image feature processing method further comprises: determining whether to freeze the model parameters of the feature extraction model and the number of model parameters to be frozen according to the number of training samples.

5. A neural network-based medical image feature processing apparatus, characterized by comprising: ​ The feature extraction unit is configured to extract initial image features from the medical image by an imageomics feature extraction strategy and N feature extraction models, including: in the case that the medical image is divided into a slice sequence composed of M slices, performing multiple target operations on the slice sequence by the N feature extraction models to obtain initial image features, wherein the multiple target operations are configured to ensure that feature extraction is completed for each slice in the slice sequence; wherein the imageomics feature extraction strategy includes an imageomics algorithm for extracting image features, and N is an integer greater than or equal to 1. The feature dimension reduction unit is configured to process the initial image features into target image features by a neural network, wherein the dimension of the target image features is lower than that of the initial image features. The task processing unit is configured to process a prediction task corresponding to the medical image based on the target image features by the neural network. The i-th target operation in the multiple target operations includes execution mode one or execution mode two: Execution mode one includes: selecting X adjacent slices as a slice combination from the slice sequence, and extracting initial image features corresponding to the slice combination from the slice combination by the N feature extraction models, wherein X is equal to the number of image processing channels set by the feature extraction model, and M is an integer greater than X; and selecting X adjacent slices used in the i+1-th target operation from the slice sequence in a slice sequence translation manner. Execution mode two includes: determining a center slice of a region of interest from the slice sequence; selecting a head slice and a tail slice from the slice sequence to form a slice combination with the center slice, and extracting initial image features corresponding to the slice combination from the determined slice combination by the N feature extraction models, wherein the number of slices included in the slice combination is equal to the number of image processing channels set by the feature extraction model; and selecting a slice combination used in the next target operation from the slice sequence in a manner of translating the head slice and the tail slice to the center slice, wherein the slice combination used in each target operation includes the center slice, but the non-center slices included in the slice combinations used in any two target operations are different slices.

6. A computer readable storage medium characterized by, The computer readable storage medium stores a computer program, wherein when the computer program runs, the device in which the computer readable storage medium is located performs the neural network-based medical image feature processing method in any one of claims 1 to 4.

7. An electronic device, comprising: The device includes one or more processors and a memory configured to store one or more programs, wherein when the one or more programs are executed by the one or more processors, the one or more processors perform the neural network-based medical image feature processing method in any one of claims 1 to 4.

Images