Medical information retrieval method and related equipment

Abstract

The embodiment of the invention provides a medical information retrieval method and related equipment. The medical information retrieval method comprises the steps that a medical image is input into a segmentation model, a target two-dimensional image is obtained, and the target two-dimensional image is a two-dimensional image containing predetermined features; inputting the target two-dimensional image into an encoder, and obtaining a high-order feature image through multiple times of down-sampling; obtaining a low-dimensional vector according to the high-order feature image; and performing comparison in a database according to the low-dimensional vectors to obtain a preset number of similar two-dimensional images. According to the technical scheme provided by the embodiment of the invention, the accuracy of similar two-dimensional image retrieval can be improved, so that the problem that the retrieved medical image is not accurate enough due to the lack of key clues by a depth model at the present stage is solved.

Description

Technical Field

[0001] The present application relates to the field of computer and communication technology, and in particular, to a medical information retrieval method and related equipment. Background Art

[0002] With the explosive growth of medical imaging, a large number of doctors are needed to complete medical research, medical analysis and medical diagnosis based on medical imaging.

[0003] However, doctors currently have limited energy, and reading medical images takes a certain amount of time and is highly dependent on the doctor's personal experience and vision, which makes the reading results easily interfered by subjective factors. Although there are solutions to use deep models for medical image retrieval, the current deep models lack the use of key clues, resulting in inaccurate retrieved medical images, which affects medical research, medical analysis, and medical diagnosis. Summary of the invention

[0004] The embodiments of the present application provide a medical information retrieval method and related equipment, which can at least to some extent overcome the problem that the current deep model lacks the use of key clues, resulting in inaccurate retrieved medical images.

[0005] Other features and advantages of the present application will become apparent from the following detailed description, or may be learned in part by the practice of the present application.

[0006] According to one aspect of an embodiment of the present application, a medical information retrieval method is provided, comprising: inputting a medical image into a segmentation model to obtain a target two-dimensional image, wherein the target two-dimensional image is a two-dimensional image containing predetermined features; inputting the target two-dimensional image into an encoder to obtain a high-order feature image after multiple downsampling; obtaining a low-dimensional vector based on the high-order feature image; and comparing the low-dimensional vector in a database to obtain a predetermined number of similar two-dimensional images.

[0007] The step of inputting the medical image into the segmentation model to obtain the target two-dimensional image specifically includes: inputting the medical image into the segmentation model to obtain the target three-dimensional image, wherein the target three-dimensional image is a three-dimensional image containing predetermined features; and slicing the target three-dimensional image to obtain the target two-dimensional image.

[0008] In an embodiment of the present application, slicing the target three-dimensional image to obtain a target two-dimensional image specifically includes: slicing the target three-dimensional image to obtain a two-dimensional slice image; and screening images containing predetermined features in the two-dimensional slice image to obtain a target two-dimensional image.

[0009] In an embodiment of the present application, each downsampling passes through the convolution layer, batch normalization layer and activation function twice.

[0010] In an embodiment of the present application, obtaining a low-dimensional vector based on the high-order feature image specifically includes: passing the high-order feature image through a fully connected layer to obtain a low-dimensional retrieval code; and performing L2 normalization on the low-dimensional retrieval code to obtain a low-dimensional vector.

[0011] In an embodiment of the present application, the method also includes: obtaining a target two-dimensional image sample set, the target two-dimensional image sample set including multiple target two-dimensional image samples, each target two-dimensional image sample having a classification label corresponding to the predetermined feature contained therein; inputting the target two-dimensional image samples into an encoder, and obtaining a high-order feature image after multiple downsampling; obtaining a low-dimensional vector based on the high-order feature image; sampling the target two-dimensional image samples according to the classification label to obtain a similar sample pair and a heterogeneous sample pair, the similar sample pair including two target two-dimensional image samples with the same classification label, and the heterogeneous sample pair including two target two-dimensional image samples with different classification labels; comparing the low-dimensional vectors based on the similar sample pair and the heterogeneous sample pair, and updating the parameters of the encoder until the first predetermined end condition is reached, and the training is terminated to obtain a trained encoder.

[0012] In an embodiment of the present application, a comparison of low-dimensional vectors is performed based on the similar sample pairs and the heterogeneous sample pairs, and parameters of the encoder are updated until a first predetermined end condition is reached, and the training is terminated to obtain a trained encoder, specifically including: determining intra-class similarity based on the similar sample pairs; determining inter-class similarity based on the heterogeneous sample pairs; and updating parameters of the encoder based on the intra-class similarity and inter-class similarity until a first predetermined end condition is reached, and the training is terminated to obtain a trained encoder.

[0013] In an embodiment of the present application, the encoder is updated according to the intra-class similarity and the inter-class similarity until a first predetermined end condition is reached, the training is terminated, and a trained encoder is obtained, specifically including: determining a similarity loss function according to the intra-class similarity and the inter-class similarity; and updating the encoder according to the similarity loss function until a first predetermined end condition is reached, the training is terminated, and a trained encoder is obtained.

[0014] In an embodiment of the present application, the encoder parameters are updated according to the intra-class similarity and the inter-class similarity until a first predetermined end condition is reached, the training is terminated, and a trained encoder is obtained, specifically including: if in the target two-dimensional image sample set, there are only no more than a first predetermined number of target two-dimensional image samples, and the intra-class similarity obtained by the same class sample pairs thereof is greater than the inter-class similarity obtained by the different class sample pairs thereof, the encoder parameters are updated; if in the target two-dimensional image sample set, there are more than a first predetermined number of target two-dimensional image samples, and the intra-class similarity obtained by the same class sample pairs thereof is greater than the inter-class similarity obtained by the different class sample pairs thereof, the first predetermined end condition is reached, the training is terminated, and a trained encoder is obtained.

[0015] In an embodiment of the present application, each target two-dimensional image sample also carries a contour label corresponding to the predetermined feature contained therein. After the target two-dimensional image sample is input into the encoder and a high-order feature image is obtained through multiple downsampling, the method further includes: the high-order feature image is upsampled multiple times to obtain a feature contour image; and the encoder parameters are updated according to the feature contour image and the contour label until a second predetermined end condition is reached, thereby terminating the training and obtaining a trained encoder.

[0016] In an embodiment of the present application, the encoder parameters are updated according to the feature contour image and the contour label until a second predetermined end condition is reached, the training is terminated, and a trained encoder is obtained, which specifically includes: determining a contour loss function according to the feature contour image and the contour label; and updating the encoder parameters according to the contour loss function until a second predetermined end condition is reached, the training is terminated, and a trained encoder is obtained.

[0017] In an embodiment of the present application, the encoder is updated with parameters according to the characteristic contour image and the contour label until a second predetermined end condition is reached, the training is terminated, and a trained encoder is obtained, specifically including: determining contour similarity according to the characteristic contour image and the contour label; if in the target two-dimensional image sample set, only no more than a second predetermined number of target two-dimensional image samples have contour similarities greater than a predetermined similarity threshold, updating the encoder parameters; if in the target two-dimensional image sample set, more than a second predetermined number of target two-dimensional image samples have contour similarities greater than a predetermined similarity threshold, the first predetermined end condition is reached, the training is terminated, and a trained encoder is obtained.

[0018] In an embodiment of the present application, each target two-dimensional image sample also carries a size label and a density label corresponding to the predetermined features contained therein. After the target two-dimensional image sample is input into the encoder and a high-order feature image is obtained through multiple downsampling, the method further includes: bilinear upsampling the high-order feature image to obtain an enlarged feature map; the enlarged feature map is passed through a fully connected layer to obtain a feature size and a feature density; and the encoder parameters are updated according to the feature size and size label, the feature density and density label, until a third predetermined end condition is reached, the training is terminated, and a trained encoder is obtained.

[0019] In an embodiment of the present application, the parameters of the encoder are updated according to the feature size and size label, the feature density and density label, until a third predetermined end condition is reached, the training is terminated, and a trained encoder is obtained, specifically including: determining a prediction loss function according to the feature size and size label, the feature density and density label; according to the prediction loss function, the parameters of the encoder are updated until a third predetermined end condition is reached, the training is terminated, and a trained encoder is obtained.

[0020] In an embodiment of the present application, the parameters of the encoder are updated according to the feature size and size label, the feature density and density label, until a third predetermined end condition is reached, the training is terminated, and a trained encoder is obtained, specifically including: if in the target two-dimensional image sample set, only no more than a third predetermined number of target two-dimensional image samples correspond to the same feature size and size label, or only no more than a fourth predetermined number of target two-dimensional image samples correspond to the same feature density and density label, then the parameters of the encoder are updated; if in the target two-dimensional image sample set, more than a third predetermined number of target two-dimensional image samples correspond to the same feature size and size label, and more than a fourth predetermined number of target two-dimensional image samples correspond to the same feature density and density label, then the first predetermined end condition is reached, the training is terminated, and a trained encoder is obtained.

[0021] According to one aspect of an embodiment of the present application, a medical information retrieval device is provided, comprising: an image cutting module, used to input a medical image into a segmentation model to obtain a target two-dimensional image, wherein the target two-dimensional image is a two-dimensional image containing predetermined features; a high-order image module, used to input the target two-dimensional image into an encoder to obtain a high-order feature image after multiple downsampling; a low-dimensional vector module, used to obtain a low-dimensional vector based on the high-order feature image; and an image output module, used to compare in a database based on the low-dimensional vector to obtain a predetermined number of similar two-dimensional images.

[0022] In an embodiment of the present application, the image cutting module specifically includes: a segmentation submodule, which is used to input the medical image into a segmentation model to obtain a target three-dimensional image, wherein the target three-dimensional image is a three-dimensional image containing predetermined features; and a slicing submodule, which is used to slice the target three-dimensional image to obtain a target two-dimensional image.

[0023] In an embodiment of the present application, the slicing submodule specifically includes: a slicing unit, used to slice the target three-dimensional image to obtain a two-dimensional slice image; and a screening unit, used to screen images containing predetermined features in the two-dimensional slice image to obtain a target two-dimensional image.

[0024] In an embodiment of the present application, the low-dimensional vector specifically includes: a low-dimensional retrieval submodule, which is used to pass the high-order feature image through a fully connected layer to obtain a low-dimensional retrieval code; a low-dimensional vector submodule, which is used to perform L2 normalization on the low-dimensional retrieval code to obtain a low-dimensional vector.

[0025] In an embodiment of the present application, the medical information retrieval device also includes: a sample acquisition module, which is used to acquire a target two-dimensional image sample set, wherein the target two-dimensional image sample set includes multiple target two-dimensional image samples, and each target two-dimensional image sample has a classification label corresponding to the predetermined feature it contains; a sample input module, which is used to input the target two-dimensional image sample into an encoder, and obtain a high-order feature image after multiple downsampling; a sample processing module, which is used to obtain a low-dimensional vector based on the high-order feature image; a sample sampling module, which is used to sample the target two-dimensional image samples according to the classification label to obtain a similar sample pair and a heterogeneous sample pair, wherein the similar sample pair includes two target two-dimensional image samples with the same classification label, and the heterogeneous sample pair includes two target two-dimensional image samples with different classification labels; a first update module, which is used to compare the low-dimensional vectors based on the similar sample pair and the heterogeneous sample pair, and update the parameters of the encoder until the first predetermined end condition is reached, the training is terminated, and a trained encoder is obtained.

[0026] In an embodiment of the present application, the first update module specifically includes: an intra-class similarity submodule, used to determine the intra-class similarity based on the same class sample pairs; an inter-class similarity submodule, used to determine the inter-class similarity based on the heterogeneous sample pairs; a first update submodule, used to update the parameters of the encoder according to the intra-class similarity and the inter-class similarity until the first predetermined end condition is reached, thereby ending the training and obtaining a trained encoder.

[0027] In an embodiment of the present application, the first updating submodule is specifically used to execute: determining a similarity loss function based on the intra-class similarity and the inter-class similarity; updating parameters of the encoder based on the similarity loss function until a first predetermined end condition is reached, thereby ending the training and obtaining a trained encoder.

[0028] In an embodiment of the present application, the first updating submodule is specifically used to execute: if in the target two-dimensional image sample set, there are only no more than a first predetermined number of target two-dimensional image samples, and the intra-class similarity obtained by the sample pairs of the same class is greater than the inter-class similarity obtained by the sample pairs of different classes, then the parameters of the encoder are updated; if in the target two-dimensional image sample set, there are more than a first predetermined number of target two-dimensional image samples, and the intra-class similarity obtained by the sample pairs of the same class is greater than the inter-class similarity obtained by the sample pairs of different classes, then the first predetermined ending condition is met, the training is ended, and a trained encoder is obtained.

[0029] In an embodiment of the present application, the medical information retrieval device also includes: a contour tracing module, which is used to obtain a feature contour image by multiple upsampling of the high-order feature image; a second updating module, which is used to update the parameters of the encoder according to the feature contour image and the contour label until a second predetermined end condition is reached, thereby terminating the training and obtaining a trained encoder.

[0030] In an embodiment of the present application, the second update module is specifically used to execute: determining a contour loss function based on the feature contour image and the contour label; and updating parameters of the encoder based on the contour loss function until a second predetermined end condition is reached, thereby ending the training and obtaining a trained encoder.

[0031] In an embodiment of the present application, the second update module is specifically used to execute: determining the contour similarity based on the feature contour image and the contour label; if in the target two-dimensional image sample set, only no more than a second predetermined number of target two-dimensional image samples have contour similarities greater than a predetermined similarity threshold, then updating the parameters of the encoder; if in the target two-dimensional image sample set, more than a second predetermined number of target two-dimensional image samples have contour similarities greater than a predetermined similarity threshold, then reaching the first predetermined end condition, ending the training, and obtaining a trained encoder.

[0032] In an embodiment of the present application, each target two-dimensional image sample also carries a size label and a density label corresponding to the predetermined features contained therein, and the medical information retrieval device also includes: an enlarged feature module, which is used to perform bilinear upsampling on the high-order feature image to obtain an enlarged feature map; a size density module, which is used to obtain the feature size and feature density through a fully connected layer on the enlarged feature map; and a third update module, which is used to update the parameters of the encoder according to the feature size and size label, the feature density and density label, until a third predetermined end condition is reached, thereby terminating the training and obtaining a trained encoder.

[0033] In an embodiment of the present application, the third update module is specifically used to execute: determining a prediction loss function based on the feature size and size label, the feature density and density label; updating parameters of the encoder based on the prediction loss function until a third predetermined end condition is reached, thereby ending training and obtaining a trained encoder.

[0034] In an embodiment of the present application, the third update module is specifically used to execute: if in the target two-dimensional image sample set, only no more than a third predetermined number of target two-dimensional image samples correspond to feature sizes and size labels that are consistent, or only no more than a fourth predetermined number of target two-dimensional image samples correspond to feature densities and density labels that are consistent, then update the parameters of the encoder; if in the target two-dimensional image sample set, more than a third predetermined number of target two-dimensional image samples correspond to feature sizes and size labels that are consistent, and more than a fourth predetermined number of target two-dimensional image samples correspond to feature densities and density labels that are consistent, then the first predetermined end condition is met, the training is ended, and a trained encoder is obtained.

[0035] According to one aspect of an embodiment of the present application, a computer-readable medium is provided, on which a computer program is stored. When the computer program is executed by a processor, the medical information retrieval method as described in the above embodiment is implemented.

[0036] According to one aspect of an embodiment of the present application, an electronic device is provided, comprising: one or more processors; a storage device for storing one or more programs, which, when the one or more programs are executed by the one or more processors, enables the one or more processors to implement the medical information retrieval method as described in the above embodiments.

[0037] In the technical solutions provided in some embodiments of the present application, the medical image is first input into the segmentation model for cutting to obtain multiple two-dimensional images, and the two-dimensional image containing the predetermined features in the multiple three-dimensional images is screened out, that is, the target two-dimensional image, and then the target three-dimensional image is input into the encoder to obtain a high-order feature image, and the high-order feature image is converted into a low-dimensional vector. The high-order feature image filters out the noise in the original medical image and contains rich semantic information related to the predetermined features, which are all key clues that are strongly associated with the predetermined features. Finally, according to the clue information contained in the low-dimensional vector, a comparison is made in the database to retrieve several similar two-dimensional images similar to the target two-dimensional image, which can improve the accuracy of similar two-dimensional image retrieval, thereby solving the problem that the current deep model lacks the use of key clues, resulting in inaccurate retrieved medical images.

[0038] These similar two-dimensional images can be used as references to judge the content of the target three-dimensional image, providing doctors with reliable reference materials for medical research and medical diagnosis and treatment, and reducing the burden on relevant personnel.

[0039] It should be understood that the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the present application. BRIEF DESCRIPTION OF THE DRAWINGS

[0040] The drawings herein are incorporated into the specification and constitute a part of the specification, showing embodiments consistent with the present application, and together with the specification, are used to explain the principles of the present application. Obviously, the drawings described below are only some embodiments of the present application, and for ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work. In the drawings:

[0041] Figure 1 A schematic diagram of an exemplary system architecture to which the technical solution of the embodiments of the present application can be applied is shown.

[0042] Figure 2 A flow chart of a medical information retrieval method provided in an embodiment of the present application is shown.

[0043] Figure 3 A schematic diagram of a medical image processing process provided in an embodiment of the present application is shown.

[0044] Figure 4 Shown according to Figure 2 A specific implementation flow chart of step S100 in the medical information retrieval method shown in the corresponding embodiment.

[0045] Figure 5 A flow chart of another medical information retrieval method provided in an embodiment of the present application is shown.

[0046] Figure 6 A schematic diagram of another medical image processing process provided in an embodiment of the present application is shown.

[0047] Figure 7 A schematic diagram of the structure of a medical information retrieval device provided in an embodiment of the present application.

[0048] Figure 8 A schematic diagram of the structure of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0049] Example embodiments will now be described more fully with reference to the accompanying drawings. However, example embodiments can be implemented in a variety of forms and should not be construed as limited to the examples set forth herein; rather, these embodiments are provided so that this application will be more comprehensive and complete and fully convey the concept of the example embodiments to those skilled in the art.

[0050] In addition, described feature, structure or characteristic can be combined in one or more embodiments in any suitable manner. In the following description, many specific details are provided to provide a full understanding of the embodiments of the present application. However, those skilled in the art will appreciate that the technical scheme of the present application can be put into practice without one or more of the specific details, or other methods, components, devices, steps, etc. can be adopted. In other cases, known methods, devices, realizations or operations are not shown or described in detail to avoid blurring the various aspects of the application.

[0051] The block diagrams shown in the accompanying drawings are merely functional entities and do not necessarily correspond to physically independent entities. That is, these functional entities may be implemented in software form, or in one or more hardware modules or integrated circuits, or in different networks and / or processor devices and / or microcontroller devices.

[0052] The flowcharts shown in the accompanying drawings are only exemplary and do not necessarily include all the contents and operations / steps, nor must they be executed in the order described. For example, some operations / steps can be decomposed, and some operations / steps can be combined or partially combined, so the actual execution order may change according to actual conditions.

[0053] Figure 1 A schematic diagram of an exemplary system architecture to which the technical solution of the embodiments of the present application can be applied is shown.

[0054] like Figure 1 As shown, the system architecture may include terminal devices (such as Figure 1The embodiment of the present invention is a schematic diagram of a mobile phone 101, a tablet computer 102, a portable computer 103, a desktop computer, etc., a network 104, and a server 105. The network 104 is used to provide a medium for a communication link between the terminal device and the server 105. The network 104 may include various connection types, such as a wired communication link, a wireless communication link, etc.

[0055] It should be understood that Figure 1 The number of terminal devices, networks and servers in the embodiment is only for illustration. According to the implementation requirements, there may be any number of terminal devices, networks and servers. For example, the server 105 may be a server cluster composed of multiple servers.

[0056] The user can use the terminal device to interact with the server 105 through the network 104 to receive or send messages, etc. The server 105 can be a server that provides various services. For example, the user uses the terminal device 103 (or the terminal device 101 or 102) to upload a medical image to the server 105, and the server 105 can input the medical image into the segmentation model to obtain a target two-dimensional image, which is a two-dimensional image containing predetermined features; input the target two-dimensional image into the encoder, and obtain a high-order feature image after multiple downsampling; obtain a low-dimensional vector based on the high-order feature image; and compare the low-dimensional vector in the database to obtain a predetermined number of similar two-dimensional images.

[0057] It should be noted that the medical information retrieval method provided in the embodiment of the present application is generally executed by the server 105, and accordingly, the medical information retrieval device is generally set in the server 105. However, in other embodiments of the present application, the terminal device may also have similar functions as the server, so as to execute the medical information retrieval scheme provided in the embodiment of the present application.

[0058] The implementation details of the technical solution of the embodiment of the present application are described in detail below:

[0059] Figure 2 A flowchart of a medical information retrieval method according to an embodiment of the present application is shown. The medical information retrieval method can be executed by a server, which can be Figure 1 Refer to the server shown in Figure 2 As shown, the medical information retrieval method at least includes:

[0060] Step S100, inputting the medical image into the segmentation model to obtain a target two-dimensional image, wherein the target two-dimensional image is a two-dimensional image containing predetermined features;

[0061] Step S200, inputting the target two-dimensional image into an encoder, and obtaining a high-order feature image through multiple downsampling;

[0062] Step S300, obtaining a low-dimensional vector according to the high-order feature image;

[0063] Step S400: performing comparison in a database based on the low-dimensional vector to obtain a predetermined number of similar two-dimensional images.

[0064] In the embodiments of the present application, reference can be made to Figure 3 First, the medical image is input into the segmentation model for cutting to obtain multiple two-dimensional images, and the two-dimensional image containing the predetermined features in the multiple three-dimensional images is screened out, which is the target two-dimensional image, and then the target three-dimensional image is input into the encoder to obtain a high-order feature image, which is converted into a low-dimensional vector. The high-order feature image filters out the noise in the original medical image and contains rich semantic information related to the predetermined features. These are all key clues that are strongly associated with the predetermined features. Finally, according to the clue information contained in the low-dimensional vector, a comparison is made in the database to retrieve several similar two-dimensional images similar to the target two-dimensional image, which can improve the accuracy of similar two-dimensional image retrieval, thereby solving the problem that the current deep model lacks the use of key clues, resulting in inaccurate retrieved medical images.

[0065] These similar two-dimensional images can be used as references to judge the content of the target three-dimensional image, providing doctors with reliable reference materials for medical research and medical diagnosis and treatment, and reducing the burden on relevant personnel.

[0066] In step S100, the segmentation model first cuts the medical image to obtain multiple two-dimensional images, and then selects the target two-dimensional image from the multiple two-dimensional images. The target two-dimensional image is a two-dimensional image containing predetermined features, which can be part features, such as cardia, posterior basal segment of right lower lobe, ulna and other part features, or abnormal features, such as image problems such as light coloring and blurring, and organ problems such as nodules, burrs, and lesions.

[0067] Specifically, in some embodiments, the specific implementation of step S100 can be found in Figure 4 . Figure 4 is based on Figure 2 The detailed description of step S100 in the medical information retrieval method shown in the corresponding embodiment, in the medical information retrieval method, step S100 may include the following steps:

[0068] Step S110, inputting the medical image into the segmentation model to obtain a target three-dimensional image, wherein the target three-dimensional image is a three-dimensional image containing predetermined features;

[0069] Step S120, slicing the target three-dimensional image to obtain a target two-dimensional image.

[0070] In the embodiment of the present application, the segmentation model first cuts the medical image to obtain multiple three-dimensional images, and selects the target three-dimensional image from the multiple three-dimensional images. The target three-dimensional image is a three-dimensional image containing predetermined features.

[0071] The target three-dimensional image is then sliced ​​to obtain a plurality of two-dimensional slices, and the target two-dimensional image is screened out from the plurality of two-dimensional images. The target two-dimensional image is a two-dimensional image containing predetermined features.

[0072] Specifically, in some embodiments, the specific implementation of step S120 can refer to the following embodiments. Figure 4 The detailed description of step S120 in the medical information retrieval method shown in the corresponding embodiment, in the medical information retrieval method, step S120 may include the following steps:

[0073] Slicing the target three-dimensional image to obtain a two-dimensional slice image;

[0074] Images containing predetermined features are screened in the two-dimensional slice images to obtain a target two-dimensional image.

[0075] In an embodiment of the present application, a target three-dimensional image is sliced ​​to obtain multiple two-dimensional slices. Among these two-dimensional slices, not every slice can retain the predetermined features, so it is necessary to screen out the two-dimensional images containing the predetermined features from the multiple two-dimensional images as the target two-dimensional images.

[0076] In step S200, after the target two-dimensional image is input into the encoder, it will first pass through a network block twice, which includes a 3×3 convolution layer, a batch normalization layer, and an activation function layer. Then it will pass through a 2×2 maximum pooling layer to obtain an initial feature image, which will be gradually reduced through multiple downsampling operations to obtain a high-order feature image.

[0077] Specifically, each downsampling operation needs to pass through a downsampling block twice. A downsampling block includes a 3×3 convolution layer, a batch normalization layer, and an activation function layer.

[0078] In one embodiment of the present application, the initial feature image is downsampled 4 times to obtain a high-order feature image. If the initial feature image is feature image d1, feature image d1 undergoes one downsampling operation to obtain feature image d2, feature image d2 undergoes one downsampling operation to obtain feature image d3, feature image d3 undergoes one downsampling operation to obtain feature image d4, and feature image d5 undergoes one downsampling operation. Feature image d5 is the final high-order feature image.

[0079] In step S300, the high-order feature image is mapped to a low-dimensional vector through a fully connected layer, and the low-dimensional vector may be a 512-dimensional vector.

[0080] Specifically, in some embodiments, the specific implementation of step S300 can refer to this embodiment. Figure 2 The detailed description of step S300 in the medical information retrieval method shown in the corresponding embodiment, in the medical information retrieval method, step S300 may include the following steps:

[0081] The high-order feature image is passed through a fully connected layer to obtain a low-dimensional retrieval code;

[0082] The low-dimensional retrieval code is L2 normalized to obtain a low-dimensional vector.

[0083] In an embodiment of the present application, the high-order feature image is mapped to a low-dimensional retrieval code through a fully connected layer, which can be a 512-dimensional vector. After the retrieval code is row-L2 normalized, a low-dimensional vector is obtained. The low-dimensional vector, like the retrieval code, can be a 512-dimensional vector.

[0084] In step S400, the low-dimensional vector is compared and screened in the database to determine the similarity between the low-dimensional vector of the feature image in the database and the low-dimensional vector of the target two-dimensional image, and the feature images in the database are sorted according to the similarity, and a predetermined number of feature images with the highest similarity are output as similar three-dimensional images. The predetermined number can be set in advance according to the needs, and can be 2, 3, 5, etc., which is not limited in the present disclosure.

[0085] In some embodiments, Figure 5 As shown, the training method of the above encoder includes:

[0086] Step S500 , obtaining a target two-dimensional image sample set, wherein the target two-dimensional image sample set includes a plurality of target two-dimensional image samples, and each target two-dimensional image sample has a classification label corresponding to a predetermined feature included therein.

[0087] Step S600: Input the target two-dimensional image sample into an encoder, and obtain a high-order feature image through multiple downsampling.

[0088] Step S700: obtaining a low-dimensional vector according to the high-order feature image.

[0089] Step S800, sampling the target two-dimensional image samples according to the classification labels to obtain similar sample pairs and heterogeneous sample pairs, wherein the similar sample pairs include two target two-dimensional image samples with the same classification labels, and the heterogeneous sample pairs include two target two-dimensional image samples with different classification labels.

[0090] Step S900, comparing low-dimensional vectors according to the homogeneous sample pairs and the heterogeneous sample pairs, updating parameters of the encoder until a first predetermined end condition is reached, and the training is terminated to obtain a trained encoder.

[0091] In this embodiment, it can be combined with reference to Figure 6 , sample the target two-dimensional image samples by classification labels to obtain similar sample pairs and heterogeneous sample pairs, wherein the similar sample pairs include two target two-dimensional image samples with the same classification labels, and the heterogeneous sample pairs include two target two-dimensional image samples with different classification labels, and by analyzing the similarities of the target two-dimensional images with the same classification labels and the differences of the target two-dimensional images with different classification labels, the encoder parameters are updated and optimized to obtain a trained encoder.

[0092] In this embodiment, the target two-dimensional image samples in the target two-dimensional image sample set can be obtained from sample medical images, and the corresponding target two-dimensional image samples can be obtained by segmenting the sample medical images.

[0093] Specifically, in some embodiments, the specific implementation of step S900 can refer to this embodiment. Figure 5 The detailed description of step S900 in the medical information retrieval method shown in the corresponding embodiment, in the medical information retrieval method, step S900 may include the following steps:

[0094] Determine the intra-class similarity based on the sample pairs of the same class;

[0095] Determining inter-class similarity based on the heterogeneous sample pairs;

[0096] According to the intra-class similarity and inter-class similarity, parameters of the encoder are updated until a first predetermined end condition is reached, and the training is terminated to obtain a trained encoder.

[0097] There are multiple first predetermined termination conditions, which may be that the intra-class similarity is greater than the inter-class similarity, or that the loss function converges or is less than a predetermined loss.

[0098] In one embodiment, the updating of parameters of the encoder according to the intra-class similarity and the inter-class similarity until a first predetermined end condition is reached, and the training is terminated to obtain a trained encoder, specifically includes:

[0099] If, in the target two-dimensional image sample set, there are only no more than a first predetermined number of target two-dimensional image samples, and the intra-class similarity obtained by forming the same class sample pairs thereof is greater than the inter-class similarity obtained by forming the different class sample pairs thereof, then updating the parameters of the encoder;

[0100] If there are more than a first predetermined number of target two-dimensional image samples in the target two-dimensional image sample set, and the intra-class similarity obtained by forming the same class sample pairs is greater than the inter-class similarity obtained by forming the different class sample pairs, then the first predetermined ending condition is met, the training is ended, and a trained encoder is obtained.

[0101] In another embodiment, the updating of parameters of the encoder according to the intra-class similarity and the inter-class similarity until a first predetermined end condition is reached, the training is terminated, and a trained encoder is obtained, specifically comprising:

[0102] Determine a similarity loss function according to the intra-class similarity and the inter-class similarity;

[0103] According to the similarity loss function, the parameters of the encoder are updated until a first predetermined end condition is reached, and the training is terminated to obtain a trained encoder.

[0104] In some embodiments, each target two-dimensional image sample also carries a contour label corresponding to the predetermined feature contained therein. After step S600, the method further includes:

[0105] The high-order feature image is up-sampled multiple times to obtain a feature contour image.

[0106] The encoder is parameter updated according to the feature contour image and the contour label until a second predetermined end condition is reached, and the training is terminated to obtain a trained encoder.

[0107] In this embodiment, it can be combined with reference to Figure 6 In addition to updating and optimizing parameters by analyzing the similarities of target two-dimensional images with the same classification labels and the differences of target two-dimensional images with different classification labels, the encoder can also update and optimize parameters by drawing the contour of a predetermined feature and comparing the difference between the drawn contour and the actual contour, thereby further improving the accuracy of the model.

[0108] In this embodiment, the number of times the high-order feature image is upsampled is consistent with the number of times the initial feature image is downsampled. After each upsampling, the upsampling result is spliced ​​and fused with the corresponding downsampled feature image to form a new feature image for the next upsampling until a feature contour image is obtained.

[0109] In one embodiment of the present application, the initial feature image is downsampled 4 times to obtain a high-order feature image. If the initial feature image is feature image d1, feature image d1 undergoes one downsampling operation to obtain feature image d2, feature image d2 undergoes one downsampling operation to obtain feature image d3, feature image d3 undergoes one downsampling operation to obtain feature image d4, and feature image d5 undergoes one downsampling operation. Feature image d5 is the final high-order feature image.

[0110] Then, correspondingly, the high-order feature image needs to be upsampled 4 times to obtain the feature contour image.

[0111] Specifically, the high-order feature image is upsampled and enlarged once to obtain an upsampled image, the upsampled image is spliced ​​and fused with the feature image d4, and then passes through a network block twice, and then a 2×2 maximum pooling layer is connected to obtain the feature image u1. The feature image u1 is upsampled and enlarged once to obtain an upsampled image, the upsampled image is spliced ​​and fused with the feature image d3, and then passes through a network block twice, and then a 2×2 maximum pooling layer is connected to obtain the feature image u2. The feature image u2 is upsampled and enlarged once to obtain an upsampled image, the upsampled image is spliced ​​and fused with the feature image d2, and then passes through a network block twice, and then a 2×2 maximum pooling layer is connected to obtain the feature image u3. The feature image u3 is upsampled and enlarged once to obtain an upsampled image, the upsampled image is spliced ​​and fused with the feature image d1, and then passes through a network block twice, and then a 2×2 maximum pooling layer is connected to obtain the feature image u4.

[0112] After the feature image u4 passes through a network block twice, it is connected to a 2×2 maximum pooling layer to obtain the feature contour image.

[0113] It should be noted that the above network block includes a 3×3 convolution layer, a batch normalization layer and an activation function layer.

[0114] This embodiment captures pixel-level contextual dependencies by accurately segmenting predetermined features. The pixel-level semantic segmentation model adopts the U-Net model. U-Net compensates for the loss of downsampling and improves segmentation accuracy. The U-Net encoder (downsampling) serves as the weight-sharing backbone of the three learning mechanisms, and is also used for qualitative learning and quantitative learning branches.

[0115] There are multiple second predetermined termination conditions, which may be that the contours depicted by most samples are highly similar to the actual contours or are slightly different and consistent, or that the loss function converges or is less than a predetermined loss.

[0116] In one embodiment, the updating of parameters of the encoder according to the characteristic contour image and the contour label until a second predetermined end condition is reached, and the training is terminated to obtain a trained encoder, specifically includes:

[0117] Determining contour similarity according to the feature contour image and the contour label;

[0118] If, in the target two-dimensional image sample set, only no more than a second predetermined number of target two-dimensional image samples have contour similarities greater than a predetermined similarity threshold, updating parameters of the encoder;

[0119] If more than a second predetermined number of target two-dimensional image samples in the target two-dimensional image sample set have contour similarities greater than a predetermined similarity threshold, the first predetermined end condition is met, the training is ended, and a trained encoder is obtained.

[0120] In another embodiment, the updating of parameters of the encoder according to the characteristic contour image and the contour label until a second predetermined end condition is reached, and the training is terminated to obtain a trained encoder, specifically includes:

[0121] Determining a contour loss function according to the feature contour image and the contour label;

[0122] According to the contour loss function, the parameters of the encoder are updated until a second predetermined end condition is reached, and the training is terminated to obtain a trained encoder.

[0123] Specifically, the contour loss function may be a DICE loss function, which may be calculated using the following formula:

[0124] DICE=2(predict∩true) / (predict∪true)

[0125] Among them, DICE is the contour loss function, predict is the predicted value, that is, the feature contour image, and true is the true value, that is, the contour label. The DICE loss is twice the ratio of the overlapping area between the feature contour image and the contour label to the total area of ​​the feature contour image plus the contour label. It objectively reflects the degree of overlap between the predicted value and the true value, which is conducive to the model to outline accurate feature contours, capture the details of specific features, and prevent them from being submerged by noise.

[0126] In some embodiments, each target two-dimensional image sample also carries a size label and a density label corresponding to the predetermined features contained therein. After step S600, the method further includes:

[0127] The high-order feature image is bilinearly upsampled to obtain an enlarged feature map;

[0128] The enlarged feature map is passed through a fully connected layer to obtain feature size and feature density;

[0129] The encoder is parameter updated according to the feature size and size label, the feature density and density label, until a third predetermined end condition is reached, the training is terminated, and a trained encoder is obtained.

[0130] In this embodiment, it can be combined with reference to Figure 6 In addition to updating and optimizing parameters by analyzing the similarities of target two-dimensional images with the same classification labels and the differences of target two-dimensional images with different classification labels, the encoder can also update and optimize parameters by determining the feature size and feature density of predetermined features, and then comparing the determined feature size with the actual feature size, and the determined feature density with the actual feature density, thereby further improving the accuracy of the model.

[0131] In this embodiment, the number of bilinear upsampling of the high-order feature image is consistent with the number of downsampling of the initial feature image. After each bilinear upsampling, the upsampling result is fused with the corresponding downsampled feature image by element-wise summation to form a new feature image for the next bilinear upsampling until the final enlarged feature map is obtained, and then the enlarged feature map is mapped through the fully connected layer to obtain the feature size and feature density.

[0132] In one embodiment of the present application, the initial feature image is downsampled 4 times to obtain a high-order feature image. If the initial feature image is feature image d1, feature image d1 undergoes one downsampling operation to obtain feature image d2, feature image d2 undergoes one downsampling operation to obtain feature image d3, feature image d3 undergoes one downsampling operation to obtain feature image d4, and feature image d5 undergoes one downsampling operation. Feature image d5 is the final high-order feature image.

[0133] Then, correspondingly, the high-order feature image needs to be upsampled 4 times to obtain the enlarged feature map.

[0134] Specifically, the high-order feature image is upsampled and enlarged once to obtain an upsampled image, the upsampled image is spliced ​​and fused with the feature image d4, and then passes through a network block twice, and then a 2×2 maximum pooling layer is connected to obtain the feature image f1. The feature image f1 is upsampled and enlarged once to obtain an upsampled image, the upsampled image is spliced ​​and fused with the feature image d3, and then passes through a network block twice, and then a 2×2 maximum pooling layer is connected to obtain the feature image f2. The feature image f2 is upsampled and enlarged once to obtain an upsampled image, the upsampled image is spliced ​​and fused with the feature image d2, and then passes through a network block twice, and then a 2×2 maximum pooling layer is connected to obtain the feature image f3. The feature image f3 is upsampled and enlarged once to obtain an upsampled image, the upsampled image is spliced ​​and fused with the feature image d1, and then passes through a network block twice, and then a 2×2 maximum pooling layer is connected to obtain the feature image f4.

[0135] The feature image f4 is the enlarged feature image.

[0136] The feature image f4 is then mapped through the fully connected layer to obtain the feature size and feature density.

[0137] It should be noted that the above network block includes a 3×3 convolution layer, a batch normalization layer and an activation function layer.

[0138] The feature size mentioned above may be the maximum circumscribed diameter of a predetermined feature, and the feature density may be the average grayscale value of the feature.

[0139] This embodiment encodes two key clues (size and density) as feature descriptors through regression prediction. Specifically, the feature size and feature density are described by the maximum circumscribed diameter and the average grayscale value (converted by HU) of the predetermined feature. The design of this regression learning branch is based on a consensus: low-level features contain rich detailed information, such as the texture of nodules; on the contrary, high-level features contain semantic information that describes the overall structure of nodules. Therefore, this embodiment uses a feature pyramid structure to fuse multi-scale features, aiming to adapt to the variable size and density of predetermined features and achieve regression prediction.

[0140] There are multiple third predetermined termination conditions, which may be that the feature size and size label depicted by most samples are consistent and the feature density and density label are consistent, or that the loss function converges or is less than a predetermined loss.

[0141] In one embodiment, updating the parameters of the encoder according to the feature size and the size label, the feature density and the density label, until a third predetermined end condition is reached, and the training is terminated to obtain a trained encoder, specifically including:

[0142] If, in the target two-dimensional image sample set, only no more than a third predetermined number of target two-dimensional image samples correspond to a consistent feature size and a consistent size label, or only no more than a fourth predetermined number of target two-dimensional image samples correspond to a consistent feature density and a consistent density label, then updating the parameters of the encoder;

[0143] If in the target two-dimensional image sample set, more than a third predetermined number of target two-dimensional image samples correspond to consistent feature sizes and size labels, and more than a fourth predetermined number of target two-dimensional image samples correspond to consistent feature densities and density labels, then the first predetermined end condition is met, the training is terminated, and a trained encoder is obtained.

[0144] In another embodiment, updating the parameters of the encoder according to the feature size and size label, the feature density and density label, until a third predetermined end condition is reached, and the training is terminated to obtain a trained encoder, specifically including:

[0145] Determining a prediction loss function according to the feature size and the size label, the feature density and the density label;

[0146] According to the prediction loss function, the parameters of the encoder are updated until a third predetermined end condition is reached, and the training is terminated to obtain a trained encoder.

[0147] Specifically, the prediction loss function can be a smooth L1 function, and its specific calculation method can refer to the following formula:

[0148]

[0149] z=|f(x)-|

[0150] Among them, SmoothL1 is the prediction loss function, z is the error loss, f(x) is the predicted value, that is, the feature size and feature density, and Y is the true value, that is, the size label and density label.

[0151] The following describes an apparatus embodiment of the present application, which can be used to execute the medical information retrieval method in the above-mentioned embodiment of the present application. For details not disclosed in the apparatus embodiment of the present application, please refer to the above-mentioned embodiment of the medical information retrieval method of the present application.

[0152] Figure 7 A block diagram of a medical information retrieval device according to an embodiment of the present application is shown.

[0153] Reference Figure 7 As shown, a medical information retrieval device 700 according to an embodiment of the present application includes:

[0154] An image cutting module 710 is used to input a medical image into a segmentation model to obtain a target two-dimensional image, wherein the target two-dimensional image is a two-dimensional image containing predetermined features;

[0155] A high-order image module 720 is used to input the target two-dimensional image into an encoder and obtain a high-order feature image through multiple downsampling;

[0156] A low-dimensional vector module 730, configured to obtain a low-dimensional vector according to the high-order feature image;

[0157] The image output module 740 is used to perform comparison in a database according to the low-dimensional vector to obtain a predetermined number of similar two-dimensional images.

[0158] In an embodiment of the present application, the image cutting module specifically includes: a segmentation submodule, which is used to input the medical image into a segmentation model to obtain a target three-dimensional image, wherein the target three-dimensional image is a three-dimensional image containing predetermined features; and a slicing submodule, which is used to slice the target three-dimensional image to obtain a target two-dimensional image.

[0159] In an embodiment of the present application, the slicing submodule specifically includes: a slicing unit, used to slice the target three-dimensional image to obtain a two-dimensional slice image; and a screening unit, used to screen images containing predetermined features in the two-dimensional slice image to obtain a target two-dimensional image.

[0160] In an embodiment of the present application, the low-dimensional vector specifically includes: a low-dimensional retrieval submodule, which is used to pass the high-order feature image through a fully connected layer to obtain a low-dimensional retrieval code; a low-dimensional vector submodule, which is used to perform L2 normalization on the low-dimensional retrieval code to obtain a low-dimensional vector.

[0161] In an embodiment of the present application, the medical information retrieval device also includes: a sample acquisition module, which is used to acquire a target two-dimensional image sample set, wherein the target two-dimensional image sample set includes multiple target two-dimensional image samples, and each target two-dimensional image sample has a classification label corresponding to the predetermined feature it contains; a sample input module, which is used to input the target two-dimensional image sample into an encoder, and obtain a high-order feature image after multiple downsampling; a sample processing module, which is used to obtain a low-dimensional vector based on the high-order feature image; a sample sampling module, which is used to sample the target two-dimensional image samples according to the classification label to obtain a similar sample pair and a heterogeneous sample pair, wherein the similar sample pair includes two target two-dimensional image samples with the same classification label, and the heterogeneous sample pair includes two target two-dimensional image samples with different classification labels; a first update module, which is used to compare the low-dimensional vectors based on the similar sample pair and the heterogeneous sample pair, and update the parameters of the encoder until the first predetermined end condition is reached, the training is terminated, and a trained encoder is obtained.

[0162] In an embodiment of the present application, the first update module specifically includes: an intra-class similarity submodule, used to determine the intra-class similarity based on the same class sample pairs; an inter-class similarity submodule, used to determine the inter-class similarity based on the heterogeneous sample pairs; a first update submodule, used to update the parameters of the encoder according to the intra-class similarity and the inter-class similarity until the first predetermined end condition is reached, thereby ending the training and obtaining a trained encoder.

[0163] In an embodiment of the present application, the first updating submodule is specifically used to execute: determining a similarity loss function based on the intra-class similarity and the inter-class similarity; updating parameters of the encoder based on the similarity loss function until a first predetermined end condition is reached, thereby ending the training and obtaining a trained encoder.

[0164] In an embodiment of the present application, the first updating submodule is specifically used to execute: if in the target two-dimensional image sample set, there are only no more than a first predetermined number of target two-dimensional image samples, and the intra-class similarity obtained by the sample pairs of the same class is greater than the inter-class similarity obtained by the sample pairs of different classes, then the parameters of the encoder are updated; if in the target two-dimensional image sample set, there are more than a first predetermined number of target two-dimensional image samples, and the intra-class similarity obtained by the sample pairs of the same class is greater than the inter-class similarity obtained by the sample pairs of different classes, then the first predetermined ending condition is met, the training is ended, and a trained encoder is obtained.

[0165] In an embodiment of the present application, the medical information retrieval device also includes: a contour tracing module, which is used to obtain a feature contour image by multiple upsampling of the high-order feature image; a second updating module, which is used to update the parameters of the encoder according to the feature contour image and the contour label until a second predetermined end condition is reached, thereby terminating the training and obtaining a trained encoder.

[0166] In an embodiment of the present application, the second update module is specifically used to execute: determining a contour loss function based on the feature contour image and the contour label; and updating parameters of the encoder based on the contour loss function until a second predetermined end condition is reached, thereby ending the training and obtaining a trained encoder.

[0167] In an embodiment of the present application, the second update module is specifically used to execute: determining the contour similarity based on the feature contour image and the contour label; if in the target two-dimensional image sample set, only no more than a second predetermined number of target two-dimensional image samples have contour similarities greater than a predetermined similarity threshold, then updating the parameters of the encoder; if in the target two-dimensional image sample set, more than a second predetermined number of target two-dimensional image samples have contour similarities greater than a predetermined similarity threshold, then reaching the first predetermined end condition, ending the training, and obtaining a trained encoder.

[0168] In an embodiment of the present application, each target two-dimensional image sample also carries a size label and a density label corresponding to the predetermined features contained therein, and the medical information retrieval device also includes: an enlarged feature module, which is used to perform bilinear upsampling on the high-order feature image to obtain an enlarged feature map; a size density module, which is used to obtain the feature size and feature density through a fully connected layer on the enlarged feature map; and a third update module, which is used to update the parameters of the encoder according to the feature size and size label, the feature density and density label, until a third predetermined end condition is reached, thereby terminating the training and obtaining a trained encoder.

[0169] In an embodiment of the present application, the third update module is specifically used to execute: determining a prediction loss function based on the feature size and size label, the feature density and density label; updating parameters of the encoder based on the prediction loss function until a third predetermined end condition is reached, thereby ending training and obtaining a trained encoder.

[0170] In an embodiment of the present application, the third update module is specifically used to execute: if in the target two-dimensional image sample set, only no more than a third predetermined number of target two-dimensional image samples correspond to feature sizes and size labels that are consistent, or only no more than a fourth predetermined number of target two-dimensional image samples correspond to feature densities and density labels that are consistent, then update the parameters of the encoder; if in the target two-dimensional image sample set, more than a third predetermined number of target two-dimensional image samples correspond to feature sizes and size labels that are consistent, and more than a fourth predetermined number of target two-dimensional image samples correspond to feature densities and density labels that are consistent, then the first predetermined end condition is met, the training is ended, and a trained encoder is obtained.

[0171] In the technical solutions provided in some embodiments of the present application, the medical image is first input into the segmentation model for cutting to obtain multiple two-dimensional images, and the two-dimensional image containing the predetermined features in the multiple three-dimensional images is screened out, that is, the target two-dimensional image, and then the target three-dimensional image is input into the encoder to obtain a high-order feature image, and the high-order feature image is converted into a low-dimensional vector. The high-order feature image filters out the noise in the original medical image and contains rich semantic information related to the predetermined features, which are all key clues that are strongly associated with the predetermined features. Finally, according to the clue information contained in the low-dimensional vector, a comparison is made in the database to retrieve several similar two-dimensional images similar to the target two-dimensional image, which can improve the accuracy of similar two-dimensional image retrieval, thereby solving the problem that the current deep model lacks the use of key clues, resulting in inaccurate retrieved medical images.

[0172] These similar two-dimensional images can be used as references to judge the content of the target three-dimensional image, providing doctors with reliable reference materials for medical research and medical diagnosis and treatment, and reducing the burden on relevant personnel.

[0173] Figure 8 A schematic diagram of the structure of a computer system suitable for implementing an electronic device of an embodiment of the present application is shown.

[0174] It should be noted that Figure 8 The computer system of the electronic device shown is only an example and should not bring any limitation to the functions and scope of use of the embodiments of the present application.

[0175] like Figure 8 As shown, the computer system includes a central processing unit (CPU) 1801, which can perform various appropriate actions and processes according to the program stored in the read-only memory (ROM) 1802 or the program loaded from the storage part 1808 to the random access memory (RAM) 1803, such as executing the method described in the above embodiment. In RAM 1803, various programs and data required for system operation are also stored. CPU 1801, ROM 1802 and RAM 1803 are connected to each other through bus 1804. Input / output (I / O) interface 1805 is also connected to bus 1804.

[0176] The following components are connected to the I / O interface 1805: an input section 1806 including a keyboard, a mouse, etc.; an output section 1807 including a cathode ray tube (CRT), a liquid crystal display (LCD), etc., and a speaker, etc.; a storage section 1808 including a hard disk, etc.; and a communication section 1809 including a network interface card such as a LAN (Local Area Network) card, a modem, etc. The communication section 1809 performs communication processing via a network such as the Internet. A drive 1810 is also connected to the I / O interface 1805 as needed. A removable medium 1811, such as a magnetic disk, an optical disk, a magneto-optical disk, a semiconductor memory, etc., is installed on the drive 1810 as needed so that a computer program read therefrom is installed into the storage section 1808 as needed.

[0177] In particular, according to an embodiment of the present application, the process described above with reference to the flowchart can be implemented as a computer software program. For example, an embodiment of the present application includes a computer program product, which includes a computer program carried on a computer-readable medium, and the computer program includes a computer program for executing the method shown in the flowchart. In such an embodiment, the computer program can be downloaded and installed from a network through a communication section 1809, and / or installed from a removable medium 1811. When the computer program is executed by a central processing unit (CPU) 1801, various functions defined in the system of the present application are executed.

[0178] It should be noted that the computer-readable medium shown in the embodiment of the present application may be a computer-readable signal medium or a computer-readable storage medium or any combination of the above two. The computer-readable storage medium may be, for example, - but not limited to - an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or device, or any combination of the above. More specific examples of computer-readable storage media may include, but are not limited to: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM), a flash memory, an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In the present application, a computer-readable storage medium may be any tangible medium containing or storing a program, which may be used by an instruction execution system, device or device or used in combination with it. In the present application, a computer-readable signal medium may include a data signal propagated in a baseband or as part of a carrier wave, wherein a computer-readable computer program is carried. Such propagated data signals may take a variety of forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination of the above. Computer-readable signal media may also be any computer-readable medium other than computer-readable storage media, which may send, propagate, or transmit programs for use by or in conjunction with an instruction execution system, apparatus, or device. The computer program contained on the computer-readable medium may be transmitted using any appropriate medium, including but not limited to: wireless, wired, etc., or any suitable combination of the above.

[0179] The flowchart and block diagram in the accompanying drawings illustrate the possible architecture, functions and operations of the system, method and computer program product according to various embodiments of the present application. Wherein, each box in the flowchart or block diagram can represent a module, a program segment, or a part of the code, and the above-mentioned module, program segment, or a part of the code contains one or more executable instructions for realizing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the box can also occur in a different order from the order marked in the accompanying drawings. For example, two boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the block diagram or flowchart, and the combination of boxes in the block diagram or flowchart can be implemented with a dedicated hardware-based system that performs a specified function or operation, or can be implemented with a combination of dedicated hardware and computer instructions.

[0180] The units involved in the embodiments described in this application may be implemented by software or hardware, and the units described may also be set in a processor. The names of these units do not, in some cases, constitute limitations on the units themselves.

[0181] As another aspect, the present application also provides a computer-readable medium, which may be included in the electronic device described in the above embodiment; or may exist independently without being assembled into the electronic device. The above computer-readable medium carries one or more programs, and when the above one or more programs are executed by an electronic device, the electronic device implements the method described in the above embodiment.

[0182] It should be noted that, although several modules or units of the equipment for action execution are mentioned in the above detailed description, this division is not mandatory. In fact, according to the embodiments of the present application, the features and functions of two or more modules or units described above can be embodied in one module or unit. On the contrary, the features and functions of one module or unit described above can be further divided into being embodied by multiple modules or units.

[0183] Through the description of the above implementation methods, it is easy for those skilled in the art to understand that the example implementation methods described here can be implemented by software or by combining software with necessary hardware. Therefore, the technical solution according to the implementation methods of the present application can be embodied in the form of a software product, which can be stored in a non-volatile storage medium (which can be a CD-ROM, a USB flash drive, a mobile hard disk, etc.) or on a network, and includes several instructions to enable a computing device (which can be a personal computer, a server, a touch terminal, or a network device, etc.) to execute the method according to the implementation methods of the present application.

[0184] Those skilled in the art will readily appreciate other embodiments of the present application after considering the specification and practicing the embodiments disclosed herein. The present application is intended to cover any variations, uses or adaptations of the present application, which follow the general principles of the present application and include common knowledge or customary technical means in the art that are not disclosed in the present application.

[0185] It should be understood that the present application is not limited to the precise structures that have been described above and shown in the drawings, and that various modifications and changes may be made without departing from the scope thereof. The scope of the present application is limited only by the appended claims.

Claims

1. A medical information retrieval method, characterized in that: include: Inputting the medical image into the segmentation model to obtain a target two-dimensional image, wherein the target two-dimensional image is a two-dimensional image containing predetermined features; Input the target two-dimensional image into an encoder, and obtain a high-order feature image after multiple downsampling; Obtaining a low-dimensional vector according to the high-order feature image; According to the low-dimensional vector, comparison is performed in a database to obtain a predetermined number of similar two-dimensional images.

2. The medical information retrieval method according to claim 1, characterized in that: The step of inputting the medical image into the segmentation model to obtain the target two-dimensional image specifically includes: Inputting the medical image into the segmentation model to obtain a target three-dimensional image, wherein the target three-dimensional image is a three-dimensional image containing predetermined features; The target three-dimensional image is sliced ​​to obtain a target two-dimensional image.

3. The medical information retrieval method according to claim 1, characterized in that: The step of obtaining a low-dimensional vector according to the high-order feature image specifically includes: The high-order feature image is passed through a fully connected layer to obtain a low-dimensional retrieval code; The low-dimensional retrieval code is L2 normalized to obtain a low-dimensional vector.

4. The medical information retrieval method according to claim 1, characterized in that: The method further comprises: Acquire a target two-dimensional image sample set, wherein the target two-dimensional image sample set includes a plurality of target two-dimensional image samples, and each target two-dimensional image sample has a classification label corresponding to a predetermined feature included therein; Input the target two-dimensional image sample into the encoder, and obtain a high-order feature image after multiple downsampling; Obtaining a low-dimensional vector according to the high-order feature image; According to the classification label, the target two-dimensional image samples are sampled to obtain a homogeneous sample pair and a heterogeneous sample pair, wherein the homogeneous sample pair includes two target two-dimensional image samples with the same classification label, and the heterogeneous sample pair includes two target two-dimensional image samples with different classification labels; According to the similar sample pairs and the heterogeneous sample pairs, low-dimensional vectors are compared, and parameters of the encoder are updated until a first predetermined end condition is reached, and the training is terminated to obtain a trained encoder.

5. The medical information retrieval method according to claim 4, characterized in that: The comparing low-dimensional vectors according to the similar sample pairs and the heterogeneous sample pairs, updating the parameters of the encoder until a first predetermined end condition is reached, and the training is terminated to obtain a trained encoder, specifically comprising: Determine the intra-class similarity based on the sample pairs of the same class; Determining inter-class similarity based on the heterogeneous sample pairs; According to the intra-class similarity and inter-class similarity, parameters of the encoder are updated until a first predetermined end condition is reached, and the training is terminated to obtain a trained encoder.

6. The medical information retrieval method according to claim 4, characterized in that: Each target two-dimensional image sample also has a contour label corresponding to the predetermined feature contained therein. After the target two-dimensional image sample is input into the encoder and a high-order feature image is obtained through multiple downsampling, the method further includes: The high-order feature image is upsampled multiple times to obtain a feature contour image; The encoder is parameter updated according to the feature contour image and the contour label until a second predetermined end condition is reached, and the training is terminated to obtain a trained encoder.

7. The medical information retrieval method according to claim 4, characterized in that: Each target two-dimensional image sample also has a size label and a density label corresponding to the predetermined feature contained therein. After the target two-dimensional image sample is input into the encoder and a high-order feature image is obtained through multiple downsampling, the method further includes: The high-order feature image is bilinearly upsampled to obtain an enlarged feature map; The enlarged feature map is passed through a fully connected layer to obtain feature size and feature density; The encoder is parameter updated according to the feature size and size label, the feature density and density label, until a third predetermined end condition is reached, the training is terminated, and a trained encoder is obtained.

8. A medical information retrieval device, characterized in that: The medical information retrieval device comprises: An image cutting module, used for inputting a medical image into a segmentation model to obtain a target two-dimensional image, wherein the target two-dimensional image is a two-dimensional image containing predetermined features; A high-order image module, used for inputting the target two-dimensional image into an encoder, and obtaining a high-order feature image through multiple downsampling; A low-dimensional vector module, used to obtain a low-dimensional vector according to the high-order feature image; The image output module is used to compare the low-dimensional vectors in a database to obtain a predetermined number of similar two-dimensional images.

9. A computer readable medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the medical information retrieval method according to any one of claims 1 to 7 is implemented.

10. An electronic device, characterized in that: include: one or more processors; A storage device for storing one or more programs, which, when executed by the one or more processors, enables the one or more processors to implement the medical information retrieval method as described in any one of claims 1 to 7.

Images