Methods of training a recognition model to recognize medical site image integrity and related products

Abstract

This application discloses a method and related products for training a recognition model to identify the integrity of medical site images. The method includes: segmenting and extracting various tissues and organs from a medical site image to obtain three-dimensional images corresponding to each tissue and organ; projecting the medical site image and the three-dimensional images corresponding to each tissue and organ along multiple projection directions to form corresponding two-dimensional projection images, and labeling them, wherein the labels include integrity labels and incompleteness labels; training a generative model for separating incomplete image information based on the labeled two-dimensional projection images to generate a target incomplete image; and training a recognition model for identifying the integrity of medical site images based on the labeled two-dimensional projection images and the target incomplete image. Using the solution of this application, the accuracy and robustness of the model in judging the integrity of medical site images can be improved.

Description

Technical Field

[0001] This application generally relates to the field of medical image processing technology. More specifically, this application relates to a method for training a recognition model for identifying the integrity of images of medical sites, a method for identifying the integrity of images of medical sites, an electronic device, and a computer-readable storage medium. Background Technology

[0002] In clinical medical imaging diagnosis, CT and MR scans are widely used for the chest, abdomen, head and neck, and pelvis. The completeness of the imaging data—whether the scan area fully covers the target area—is a fundamental prerequisite for ensuring the accuracy of subsequent diagnosis, analysis, and treatment decisions. Although most clinical scans are complete, in practice, incomplete images may occur due to factors such as incomplete scan coverage or slice loss during data transmission. This can severely impact a physician's interpretation of the disease.

[0003] Current medical image integrity assessments often employ thresholding methods, which segment organ masks, calculate the length of the region of interest (ROI) within the mask, and compare it to a preset minimum threshold to determine integrity. However, this method has significant limitations. First, for incomplete cases where only a few slices are missing, the reduction in ROI length is not significant, making effective identification difficult using simple thresholding. Second, existing methods primarily assess integrity for single organs, while integrity assessment for complex sites composed of multiple organs (such as the chest containing the heart and lungs) is far more complex, requiring comprehensive consideration of the condition of each internal organ. Furthermore, the number of incomplete samples in clinical practice is relatively small, while deep learning models require a large number of samples for training, posing a challenge to building high-precision integrity assessment models.

[0004] In view of this, there is an urgent need to provide a scheme for training a recognition model for identifying the integrity of medical site images, so as to determine the integrity of the site through multi-organ multi-directional projection constraints, and to effectively expand the training dataset by combining incomplete sample generation technology, thereby improving the accuracy and robustness of the model in judging the integrity of medical site images. Summary of the Invention

[0005] In order to at least address one or more of the technical problems mentioned above, this application proposes a scheme for training a recognition model for identifying the integrity of medical site images in several aspects.

[0006] In a first aspect, this application provides a method for training a recognition model to identify the integrity of medical site images, comprising: segmenting and extracting various tissues and organs in the medical site image to obtain three-dimensional images corresponding to each tissue and organ; projecting the medical site image and the three-dimensional images corresponding to each tissue and organ along multiple projection directions to form corresponding two-dimensional projection images, and labeling them, wherein the labels include integrity labels and incompleteness labels; training a generation model for separating incomplete image information based on the labeled two-dimensional projection images to generate a target incomplete image; and training a recognition model to identify the integrity of medical site images based on the labeled two-dimensional projection images and the target incomplete image.

[0007] In some embodiments, segmenting and extracting each tissue and organ in a medical site image to obtain a three-dimensional image corresponding to each tissue and organ includes: segmenting and extracting each tissue and organ in the medical site image to obtain a segmentation mask for each tissue and organ; and multiplying the segmentation mask of each tissue and organ with the medical site image to obtain a three-dimensional image corresponding to each tissue and organ.

[0008] In some embodiments, the plurality of projection directions include at least two of the coronal, sagittal, and transverse projection directions.

[0009] In some embodiments, the generative model includes at least a first encoder, a second encoder, a first decoder, and a second decoder. Training the generative model for separating incomplete image information based on labeled 2D projected images to generate a target incomplete image includes: inputting 2D projected images labeled with integrity tags into the first encoder and the second encoder for feature encoding, extracting first image content features and first incomplete information features, and calculating a first loss based on the first incomplete information features; inputting the first image content features and the first incomplete information features into the first decoder for feature decoding to obtain a first reconstructed projection, and calculating a second loss based on the 2D projected images labeled with integrity tags and the first reconstructed projection; inputting 2D projected images labeled with incompleteness tags into the first encoder and the second encoder for feature encoding, extracting second image content features and second incomplete information features; inputting the second image content features and the second incomplete information features into the second decoder for feature decoding to obtain a second reconstructed projection, and calculating a third loss based on the 2D projected images labeled with incompleteness tags and the second reconstructed projection; and training the generative model for separating incomplete image information based on the total loss of the first loss, the second loss, and the third loss to generate the target incomplete image.

[0010] In some embodiments, the generative model further includes an integrity discriminator, and the method further includes: performing integrity discrimination on the first image content features and the second image content features based on the integrity discriminator; performing integrity adversarial discrimination by combining gradient inversion to decouple the image content features from the integrity state, and calculating a fourth loss; training a generative model for separating incomplete image information based on the total loss of the first loss, the second loss, the third loss, and the fourth loss to generate a target incomplete image.

[0011] In some embodiments, the generative model further includes a classifier, and the method further includes: determining, based on the classifier, whether the first incomplete information feature and the second incomplete information feature contain information for distinguishing completeness states, and calculating a fifth loss; training a generative model for separating incomplete image information based on the total loss of the first loss, the second loss, the third loss, the fourth loss, and the fifth loss, to generate a target incomplete image.

[0012] In some embodiments, training a recognition model for identifying the integrity of medical site images based on labeled two-dimensional projected images and the incomplete target image includes: inputting the labeled two-dimensional projected images and the incomplete target image into the recognition model for integrity recognition and calculating a loss function; and training the recognition model for identifying the integrity of medical site images based on the loss function.

[0013] In a second aspect, this application provides a method for identifying the integrity of medical site images, comprising: acquiring an image of the medical site to be identified; inputting the image of the medical site to be identified into an identification model trained according to the multiple embodiments of the first aspect above for integrity identification, and obtaining an identification result.

[0014] In a third aspect, this application provides an electronic device, including: a processor; and a memory, wherein program instructions are stored for training a recognition model for identifying the integrity of images of medical sites, wherein when the program instructions are executed by the processor, the electronic device implements one or more embodiments of the first aspect described above; or the device stores program instructions for identifying the integrity of images of medical sites, wherein when the program instructions are executed by the processor, the electronic device implements embodiments of the second aspect described above.

[0015] In a fourth aspect, this application provides a computer-readable storage medium having stored thereon computer program instructions for training a recognition model for recognizing the integrity of images of medical sites, wherein when executed by one or more processors, the computer program instructions cause to implement one or more embodiments of the first aspect described above; or having stored computer program instructions for recognizing the integrity of images of medical sites, wherein when executed by one or more processors, the computer program instructions cause to implement embodiments of the second aspect described above.

[0016] Using the above-described scheme for training a recognition model to identify the integrity of medical site images, this embodiment of the application extracts various tissues and organs from the medical site images through segmentation and performs multi-directional projection to reduce data processing complexity while preserving key structural information. By utilizing a generative model to learn from limited real samples and generate target incomplete images with distributions similar to real incomplete images, the model training challenge caused by the scarcity of incomplete samples is effectively solved. The ultimately trained recognition model can integrate information from multiple organs and directions, more accurately determining the integrity of medical site images. Attached Figure Description

[0017] The above and other objects, features, and advantages of exemplary embodiments of this application will become readily understood by reading the following detailed description with reference to the accompanying drawings. In the drawings, several embodiments of this application are illustrated by way of example and not limitation, and the same or corresponding reference numerals denote the same or corresponding parts, wherein:

[0018] Figure 1 This is an exemplary flowchart illustrating a method 100 for training a recognition model for identifying the integrity of images of medical sites according to an embodiment of this application; Figure 2 This is an exemplary flowchart illustrating the overall training of a recognition model for identifying the integrity of images of medical sites according to embodiments of this application; Figure 3 This is an exemplary schematic diagram illustrating a generated model according to an embodiment of this application; Figure 4 This is an exemplary flowchart illustrating a method 400 for identifying the integrity of medical site images according to an embodiment of this application; Figure 5 An exemplary structural block diagram of an electronic device 500 according to an embodiment of this application is shown. Detailed Implementation

[0019] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of this application. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0020] It should be understood that the terms "comprising" and "including" used in the specification and claims of this application indicate the presence of the described features, integrals, steps, operations, elements and / or components, but do not exclude the presence or addition of one or more other features, integrals, steps, operations, elements, components and / or collections thereof.

[0021] It should also be understood that the terminology used herein is for the purpose of describing particular embodiments only and is not intended to limit the application. As used in this specification and claims, the singular forms “a,” “an,” and “the” are intended to include the plural forms unless the context clearly indicates otherwise. It should also be understood that the term “and / or” as used in this specification and claims refers to any combination and all possible combinations of one or more of the associated listed items, and includes such combinations.

[0022] As used in this specification and claims, the term "if" may be interpreted, depending on the context, as "when," "once," "in response to determination," or "in response to detection." Similarly, the phrase "if determined" or "if [described condition or event] is detected" may be interpreted, depending on the context, as "once determined," "in response to determination," "once [described condition or event] is detected," or "in response to detection of [described condition or event]."

[0023] The specific embodiments of this application will now be described in detail with reference to the accompanying drawings.

[0024] Figure 1 This is an exemplary flowchart illustrating a method 100 for training a recognition model for identifying the integrity of images of medical sites according to an embodiment of this application. Figure 1As shown, the method 100 may include: Step S101: Segmenting and extracting various tissues and organs in the medical site image to obtain three-dimensional images corresponding to each tissue and organ; Step S102: Projecting the medical site image and the three-dimensional images corresponding to each tissue and organ along multiple projection directions into corresponding two-dimensional projection images, and labeling them, wherein the labels include integrity labels and incompleteness labels; Step S103: Training a generation model for separating incomplete image information based on the labeled two-dimensional projection images to generate a target incomplete image; Step S104: Training a recognition model for recognizing the integrity of medical site images based on the labeled two-dimensional projection images and the target incomplete image.

[0025] First, in step S101, the various tissues and organs in the medical site image are segmented and extracted to obtain the three-dimensional images corresponding to each tissue and organ.

[0026] It is understood that medical site imaging refers to three-dimensional CT or three-dimensional MR images of routinely scanned clinical sites such as the chest, abdomen, head and neck, spine, and pelvis. Three-dimensional images possess volumetric data in three dimensions: length, width, and slice thickness, and can completely present the spatial structure of internal tissues and organs. Tissues and organs refer to the functional anatomical structures within the target site; for example, the chest includes organs such as the heart, lungs, and mediastinum, while the abdomen includes organs such as the liver, kidneys, and spleen. Site integrity refers to whether the analyzed site (such as the chest, abdomen, head and neck, spine, and pelvis) is complete during scanning, i.e., whether the scan range completely covers all necessary anatomical structures of the site. Since a site is composed of multiple organs, such as the chest including the heart and lungs, if any key organ is incomplete, the image of that site is usually considered incomplete. Therefore, this application embodiment uses segmentation and extraction of organs within a site to comprehensively determine whether the site is complete.

[0027] In some embodiments, individual tissues and organs in the medical site image can be segmented and extracted to obtain segmentation masks for each tissue and organ. These segmentation masks are then multiplied by the medical site image to obtain a corresponding 3D image. In some implementation scenarios, a pre-trained TotalSegmentator model can be used. This model is an automatic multi-organ segmentation model adapted to CT / MR images, capable of accurately identifying and outputting regional information of all target organs within the medical site. Specifically, the acquired 3D CT / MR medical site image is input into the TotalSegmentator model. The model automatically identifies all target tissues and organs in the image using a pre-trained convolutional neural network and outputs an independent segmentation mask for each organ. The segmentation mask is a binary image with the same dimensions as the original 3D image. Pixels in the mask are marked as 1 for target organ regions and 0 for non-organ background regions.

[0028] Next, the segmentation mask of each tissue / organ is multiplied with the medical site image to obtain the corresponding 3D image. The multiplication operation involves multiplying pixel-by-pixel values. In the original medical site image, pixel values ​​represent image intensity values, while the segmentation mask only retains the organ region. After multiplication, pixel values ​​in non-organ regions are reset to zero, retaining only the density information of the target organ region, ultimately resulting in an independent 3D image for each tissue / organ. Assuming a medical site contains X tissues / organs, segmentation and extraction yield X organ 3D images, which, combined with the original medical site 3D image, result in X+1 3D images. This allows for the removal of background interference, accurate extraction of independent 3D images for each organ, and the separate storage and subsequent independent analysis of multi-organ images, improving the effectiveness of multi-organ constrained analysis.

[0029] Based on the obtained three-dimensional images of each tissue and organ, in step S102, the medical site images and the three-dimensional images of each tissue and organ are projected into corresponding two-dimensional projection images along multiple projection directions, and labels are added, including integrity labels and incompleteness labels.

[0030] Because 3D image data is massive, directly using it for model training would result in enormous computational resource consumption and extremely low training efficiency. Furthermore, site integrity assessment does not require 3D spatial information; it only needs to determine whether there are breaks or defects in the image in various directions. Therefore, this application's embodiments project 3D images onto a 2D plane. For example, maximum density projection is used to achieve 3D-to-2D dimensionality reduction. Maximum density projection refers to taking the maximum density of each pixel in the 3D image along a specified viewing direction, mapping the 3D spatial structure to a 2D plane. This preserves the key contours and density features of organs, adapting to the needs of integrity assessment.

[0031] In some embodiments, the multiple projection directions may include at least two of the coronal, sagittal, and transverse planes. The coronal plane refers to a plane formed by perpendicularly cutting along the left-right direction of the human body, which can show the left-right distribution and vertical extension characteristics of organs. The sagittal plane refers to a plane formed by perpendicularly cutting along the front-back direction of the human body, which can show the front-back distribution and vertical extension characteristics of organs. The transverse plane refers to a plane formed by horizontally cutting along the vertical direction of the human body, which can show the cross-sectional structure of organs and their left-right and front-back distribution characteristics.

[0032] Preferably, a three-directional full projection is employed. Each 3D image (including the original site image and images of each organ) undergoes maximum density projection along three directions, generating three 2D projection images from a single 3D image, ultimately resulting in 3×(X+1) 2D projection images. Based on this, image integrity information is comprehensively captured from three orthogonal dimensions, forming a pseudo-3D analysis effect, avoiding the omission of missing details in a single projection direction, and ensuring the comprehensiveness and reliability of integrity assessment. After projection, labels can be added manually or automatically. Integrity labels indicate images without defects and fully covering the scan area, while incomplete labels indicate images with missing slices, incomplete scan areas, organ truncation, etc.

[0033] Further, in step S103, a generative model for separating incomplete image information is trained based on the labeled two-dimensional projected image to generate the target incomplete image.

[0034] It is understandable that in real-world clinical scenarios, incomplete image samples of medical sites are scarce, while deep learning models require a large number of samples for training. Insufficient samples can lead to overfitting and poor generalization. Therefore, this application's embodiments achieve incomplete sample amplification by training a generative model. By separating the content features and incomplete information features of the image, incomplete images that conform to the real distribution are generated through feature recombination, thus expanding the training sample library and providing sufficient data support for subsequent recognition model training.

[0035] In some embodiments, the generative model may include at least a first encoder, a second encoder, a first decoder, and a second decoder, and training the generative model for separating incomplete image information based on labeled two-dimensional projected images to generate a target incomplete image includes: inputting two-dimensional projected images labeled with integrity tags into the first encoder and the second encoder respectively for feature encoding, extracting first image content features and first incomplete information features accordingly, and calculating a first loss based on the first incomplete information features; inputting the first image content features and the first incomplete information features into the first decoder for feature decoding to obtain a first reconstructed projection, and calculating a second loss based on the two-dimensional projected images labeled with integrity tags and the first reconstructed projection.

[0036] The two-dimensional projected images labeled with incompleteness are input into the first encoder and the second encoder for feature encoding, respectively, to extract the second image content features and the second incomplete information features. The second image content features and the second incomplete information features are input into the second decoder for feature decoding to obtain the second reconstructed projection. A third loss is calculated based on the two-dimensional projected images labeled with incompleteness and the second reconstructed projection. A generative model for separating incomplete image information is trained based on the total loss of the first loss, the second loss and the third loss to generate the target incomplete image.

[0037] Specifically, firstly, the two-dimensional projected image labeled with integrity tags... The first image content features are extracted by inputting them into the first encoder Ec and the second encoder Em, respectively. and the first incomplete information feature Among them, image content features represent anatomical content unrelated to the integrity state, while incomplete information features represent defect patterns relative to the integrity state. Considering the input is a two-dimensional projection image labeled with integrity tags, the incomplete information features... The value approaches 0, meaning the complete projection does not contain any incomplete information. In this scenario, based on the first incomplete information feature... Calculate the first loss , The L2 norm is represented. This first loss constrains the incomplete information features of the complete image to approach zero, ensuring that the second encoder extracts effective missing features only when processing incomplete images, thus achieving a preliminary separation between content features and incomplete information features.

[0038] Next, the content features of the first image and the first incomplete information feature The input is fed into the first decoder Dc for feature decoding to obtain the first reconstructed projection. Among them, the first reconstructed projection The two-dimensional projected image tends to be consistent with the input complete projection, thus achieving completeness labeling. and the first reconstructed projection Calculate the second loss , This represents the L1 norm. This second loss constrains the reconstruction accuracy of the complete image, ensuring that the content features extracted by the first encoder can completely restore the anatomical structure of the original image, thus improving the effectiveness of the content features.

[0039] Additionally, two-dimensional projected images labeled with incompleteness tags The second image content features are also extracted by inputting them into the first encoder Ec and the second encoder Em, respectively. Second incomplete information features Furthermore, the content features of the second image... Second incomplete information features The input is fed into the second decoder Dm for decoding to obtain the second reconstructed projection. Due to the second reconstructed projection The incomplete projection tends to align with the input, thus forming a two-dimensional projected image based on the labeling of incompleteness. Second Reconstructed Projection Calculate the third loss The third loss constrains the reconstruction accuracy of the incomplete image, ensuring that the incomplete information features extracted by the second encoder can accurately restore the original image's defect pattern, thus improving the effectiveness of the incomplete information features. The training and generation model is constrained by the total loss of the first, second, and third losses to generate the target incomplete image. In the implementation scenario, the total loss can be calculated by the weighted sum of the first, second, and third losses.

[0040] In some implementation scenarios, the encoders described above can consist of convolutional layers, activation layers, and pooling layers, such as two convolutional layers (3×3), ReLU activation, and max pooling. The decoders can consist of deconvolutional layers and activation layers, such as two deconvolutional layers (3×3), ReLU activation, and a 1×1 convolution. The number of layers can be set according to the actual needs of the scenario, and this application does not impose any restrictions in this regard.

[0041] In some embodiments, the generative model may further include an integrity discriminator, which then performs integrity discrimination on the first image content features and the second image content features based on the integrity discriminator; performs integrity adversarial discrimination by combining gradient inversion to decouple the image content features from the integrity state, and calculates a fourth loss; and trains a generative model for separating incomplete image information based on the total loss of the first loss, the second loss, the third loss and the fourth loss to generate the target incomplete image.

[0042] Specifically, to prevent the extracted content from carrying incomplete information, an integrity discriminator is added. To determine the content features of the first input image. Second image content characteristics Whether it originates from a complete or incomplete projection. Next, a gradient inversion layer is used to constrain the content features of the first image. Second image content characteristics Extract, let The inability to distinguish whether the source is a complete or incomplete projection of the content representation ensures that the extracted content representation is free from incomplete states, thus further separating the content representation from the incomplete whole. A fourth loss can be calculated within this framework. y represents the integrity label. This represents the predicted output value.

[0043] Furthermore, the generative model is trained based on the total loss constraint of the first, second, third, and fourth losses to generate incomplete images of the target. Similarly, the aforementioned total loss can be calculated based on the weighted sum of the four losses. This further enhances the separation effect between content features and incomplete information features, improves the feature purification capability of the generative model, and ensures that the generated incomplete images have realistic anatomical structures and accurate defect patterns.

[0044] In some embodiments, the generative model may further include a classifier, which then determines whether the first incomplete information feature and the second incomplete information feature contain information for distinguishing the integrity state, and calculates a fifth loss; the generative model for separating incomplete image information is trained based on the total loss of the first loss, the second loss, the third loss, the fourth loss and the fifth loss to generate the target incomplete image.

[0045] Specifically, to ensure that incomplete information is extracted from the incomplete information representation, a classifier can be added to determine whether the input first and second incomplete information features can distinguish between complete and incomplete projections, thereby further ensuring that the incomplete information representation carries missing information. This includes calculating the fifth loss. Where y represents the integrity label, This represents the predicted output value. Further, the generative model is trained based on the total loss constraint of the first, second, third, fourth, and fifth losses. Specifically, the total loss can be calculated using the following formula:

[0046] in, These are the weights for different loss functions.

[0047] Training the aforementioned model and updating parameters through weights allows for the stripping of incomplete information representations, and the incomplete target image can be reconstructed using content representation. Simultaneously, during training, after inputting different incomplete projections, all incomplete information representations are saved; the incomplete representations differ for different types of incomplete projections. Therefore, this embodiment can also obtain different defect pattern prototypes by performing K-means clustering analysis on the acquired incomplete representations. In the generation stage, any complete projection is input into the first encoder Ec to obtain a complete content representation. Incomplete information representations are extracted from different defect pattern prototypes, and the second decoder Dm generates a target incomplete projection that conforms to the incomplete projection distribution. Using this generated incomplete projection, more incomplete samples can be realistically obtained, supporting the establishment of an integrity model.

[0048] In some implementations, the integrity discriminator described above may include, for example, two convolutional (3×3) layers, a ReLU layer, a pooling layer, one fully connected layer, and a Sigmoid layer. The classifier layer described above may include, for example, a fully connected network.

[0049] Finally, in step S104, a recognition model for identifying the integrity of medical site images is trained based on the labeled 2D projected image and the incomplete target image. In some embodiments, the labeled 2D projected image and the incomplete target image can be input into the recognition model for integrity identification, and a loss function can be calculated; the recognition model for identifying the integrity of medical site images is then trained based on the loss function. In some implementation scenarios, the aforementioned recognition model can be, for example, a Transformer classification model.

[0050] Specifically, the original labeled 3×(X+1) two-dimensional projected images are merged with the generated incomplete target images to construct a training dataset for the recognition model, which is divided into training, validation, and test sets proportionally. The training set images are input into the Transformer classification model. The model captures global features (organ distribution, projection integrity) and local features (defective regions, organ edges) of the two-dimensional images through a multi-head self-attention mechanism, outputting image integrity prediction results. The loss function is cross-entropy loss. The loss function is minimized through backpropagation, iteratively updating the parameters of the self-attention layer and fully connected layer of the Transformer model. During training, the model's accuracy, precision, and recall are evaluated using the validation set after each iteration, retaining the optimal model parameters from the validation set to avoid overfitting. After training, the model's generalization ability is verified using the test set, ultimately obtaining a well-trained integrity recognition model. The technical effect is a highly accurate and strongly generalized recognition model that can accurately identify the integrity of medical images, adapting to the detection needs of real-world clinical scenarios.

[0051] As can be seen from the above description, the embodiments of this application segment medical site images into multiple organ images and perform maximum density projection along multiple projection directions, thereby efficiently compressing three-dimensional volume data into two-dimensional projected images. While retaining the multi-organ, multi-view structural information required for determining integrity, this greatly reduces the complexity of data processing and the consumption of computing resources.

[0052] Furthermore, by employing a generative model based on dual encoders, dual decoders, an integrity discriminator, and a classifier, and by decoupling image content representation from "incomplete information representation," combined with adversarial training using gradient inversion layers and defect pattern prototypes obtained through cluster analysis, a large number of realistic and diverse incomplete target images can be generated from a limited set of real samples. This effectively solves the model training difficulty problem caused by imbalanced positive and negative samples. The final trained recognition model can comprehensively analyze the projection features of various organs in various directions to accurately determine the integrity of the affected area, significantly improving the accuracy and robustness of the assessment, and providing reliable technical support for automated quality control and diagnostic assistance in clinical images.

[0053] Figure 2This is an exemplary flowchart illustrating the overall training of a recognition model for identifying the integrity of images of medical sites according to embodiments of this application. Figure 2 As shown, in step S201, an image of the medical site is acquired, and in step S202, each tissue and organ in the medical site image is segmented and extracted to obtain a segmentation mask for each tissue and organ. In step S203, the segmentation mask for each tissue and organ is multiplied by the medical site image to obtain a three-dimensional image corresponding to each tissue and organ.

[0054] Next, in step S204, the medical site image and the corresponding 3D images of each tissue and organ are projected into corresponding 2D projected images along multiple projection directions using, for example, maximum density projection, and labeled. Based on the labeled 2D projected images, in step S205, a generative model that separates incomplete image information is trained to obtain the target incomplete image. Further, in step S206, a recognition model is jointly trained using the labeled 2D projected images and the target incomplete image output by the generative model. For more details regarding the aforementioned steps, please refer to the above... Figure 1 The description of the subject matter will not be repeated here.

[0055] Figure 3 This is an exemplary schematic diagram illustrating a generated model according to an embodiment of this application. For example... Figure 3 As shown, the generative model may include a first encoder Ec and a second encoder Em, a first decoder Dc and a second decoder Dm, and an integrity discriminator. Classifier During training, two-dimensional projected images labeled with integrity will be used. 301 is input into the first encoder Ec and the second encoder Em respectively to extract the first image content features. 303 and the first incomplete information feature 304, and the first image content features 303 and the first incomplete information feature 304 is input to the first decoder Dc for feature decoding to obtain the first reconstructed projection. 305. Further, the two-dimensional projected image labeled with incompleteness tags... 302 is also input into the first encoder Ec and the second encoder Em respectively to extract the second image content features. 306 and second incomplete information features 307, and the second image content features 306 and second incomplete information features Input 307 is fed into the second decoder Dm for decoding to obtain the second reconstructed projection. 308.

[0056] To prevent the extracted content representation from carrying incomplete information, the first image content features are also used. 303. Second Image Content Characteristics 306 Input Integrity Discriminator So that It cannot distinguish whether the source is a complete or incomplete projection of the content representation, and outputs the predicted value. To ensure that incomplete information is extracted from the incomplete information representation, the first incomplete information feature is used. 304 and second incomplete information characteristics 307 Input Classifier This will ensure that incomplete information is represented with missing information and output predicted values. The process involves calculating the corresponding loss and then using the total loss to constrain the training, resulting in a trained generative model.

[0057] Based on the trained generative model used in the generation stage, the two-dimensional projected image G is input into the first encoder Ec to obtain the first image content features. 303 and the first incomplete information feature 304, Selecting different defect pattern prototypes as incomplete features This image is then further input into the second decoder, Dm, to obtain the incomplete target image, Gen. More details can be found above. Figure 1 The description of the subject matter will not be repeated here.

[0058] Figure 4 This is an exemplary flowchart illustrating a method 400 for identifying the integrity of medical site images according to an embodiment of this application. Figure 4 As shown, in step S401, an image of the medical part to be identified is acquired, and in step S402, the image of the medical part to be identified is input into the trained recognition model for integrity recognition to obtain the recognition result. The recognition model directly outputs whether the medical part image is complete.

[0059] Figure 5 An exemplary structural block diagram of an electronic device 500 according to an embodiment of this application is shown. It will be understood that the device implementing the solution of this application may be a single device (e.g., a computing device) or a multifunctional device including various peripheral devices.

[0060] like Figure 5As shown, the electronic device of this application may include a central processing unit (“CPU”) 511, which may be a general-purpose CPU, a dedicated CPU, or other information processing and program execution unit. Furthermore, the electronic device 500 may also include a mass storage device 512 and a read-only memory (“ROM”) 513. The mass storage device 512 may be configured to store various types of data, including images of medical sites, two-dimensional projection images, incomplete target images, recognition results, algorithm data, intermediate results, and various programs required to run the electronic device 500. The ROM 513 may be configured to store data and instructions required for the power-on self-test of the electronic device 500, the initialization of various functional modules in the system, the drivers for the system's basic input / output, and the booting of the operating system.

[0061] Optionally, the electronic device 500 may also include other hardware platforms or components, such as the tensor processing unit (“TPU”) 514, graphics processing unit (“GPU”) 515, field-programmable gate array (“FPGA”) 516, and machine learning unit (“MLU”) 517 shown. It is understood that although various hardware platforms or components are shown in the electronic device 500, they are merely exemplary and not limiting, and those skilled in the art can add or remove corresponding hardware as needed. For example, the electronic device 500 may include only a CPU, associated storage devices, and interface devices to implement the method of training a recognition model for identifying the integrity of medical site images or the method for identifying the integrity of medical site images as described in this application.

[0062] In some embodiments, to facilitate data transmission and interaction with external networks, the electronic device 500 of this application further includes a communication interface 518, through which it can connect to a local area network / wireless local area network (“LAN / WLAN”) 505, and further through the LAN / WLAN to connect to a local server 506 or to the Internet (“Internet”) 507. Alternatively or additionally, the electronic device 500 of this application can also directly connect to the Internet or cellular network via the communication interface 518 based on wireless communication technology, such as wireless communication technology based on 3G (“3G”), 4G (“4G”), or 5G (“5G”). In some application scenarios, the electronic device 500 of this application can also access the server 508 and database 509 of an external network as needed to obtain various known algorithms, data, and modules, and can remotely store various data, such as various data or instructions for presenting, for example, medical site images, two-dimensional projection images, incomplete target images, recognition results, etc.

[0063] Peripherals of the electronic device 500 may include a display device 502, an input device 503, and a data transmission interface 504. In one embodiment, the display device 502 may include, for example, one or more speakers and / or one or more visual displays, configured to train the recognition model for identifying the integrity of medical site images of this application or to provide voice prompts and / or display images and videos for identifying the integrity of medical site images. The input device 503 may include, for example, a keyboard, mouse, microphone, posture capture camera, or other input buttons or controls, configured to receive audio data input and / or user commands. The data transmission interface 504 may include, for example, a serial interface, parallel interface, or Universal Serial Bus interface (“USB”), Small Computer System Interface (“SCSI”), Serial ATA, FireWire (“FireWire”), PCI Express, and High Definition Multimedia Interface (“HDMI”), configured for data transmission and interaction with other devices or systems. According to the solution of this application, the data transmission interface 504 can receive acquired medical site images from the image acquisition device and transmit medical site images or various other types of data or results to the electronic device 500.

[0064] The aforementioned CPU 511, mass storage 512, ROM 513, TPU 514, GPU 515, FPGA 516, MLU 517, and communication interface 518 of the electronic device 500 of this application can be interconnected via bus 519, and can interact with peripheral devices through this bus. In one embodiment, the CPU 511 can control other hardware components and peripheral devices in the electronic device 500 through bus 519.

[0065] The above combination Figure 5 An electronic device that can be used to execute this application is described. It should be understood that the device structure or architecture described herein is merely exemplary, and the implementation method and implementation entity of this application are not limited thereto, but can be changed without departing from the spirit of this application.

[0066] Based on the foregoing description in conjunction with the accompanying drawings, those skilled in the art will understand that the embodiments of this application can also be implemented by software programs. Therefore, this application also provides a computer-readable storage medium storing computer-readable instructions for training a recognition model for identifying the integrity of images of medical sites. When executed by one or more processors, these computer-readable instructions can be used to implement the embodiments of this application in conjunction with the accompanying drawings. Figure 1 The method described herein for training a recognition model for identifying the integrity of images of medical sites; or storing computer-readable instructions for identifying the integrity of images of medical sites, which, when executed by one or more processors, can be used to implement the present application in conjunction with the appendix. Figure 4 The method described is for identifying the integrity of images of medical sites.

[0067] It should be noted that although the operations of the method of this application are described in a specific order in the accompanying drawings, this does not require or imply that these operations must be performed in that specific order, or that all the operations shown must be performed to achieve the desired result. On the contrary, the steps depicted in the flowchart can be performed in a different order. Additionally or alternatively, certain steps may be omitted, multiple steps may be combined into one step, and / or one step may be broken down into multiple steps.

[0068] It should be understood that when the terms "first," "second," "third," and "fourth," etc., are used in the claims, specification, and drawings of this application, they are used only to distinguish different objects and not to describe a specific order. The terms "comprising" and "including" as used in the specification and claims of this application indicate the presence of the described features, integrals, steps, operations, elements, and / or components, but do not exclude the presence or addition of one or more other features, integrals, steps, operations, elements, components, and / or collections thereof.

[0069] It should also be understood that the terminology used herein is for the purpose of describing particular embodiments only and is not intended to limit the application. As used in this specification and claims, the singular forms “a,” “an,” and “the” are intended to include the plural forms unless the context clearly indicates otherwise. It should also be understood that the term “and / or” as used in this specification and claims refers to any combination and all possible combinations of one or more of the associated listed items, and includes such combinations.

[0070] While numerous embodiments of this application have been shown and described herein, it will be apparent to those skilled in the art that such embodiments are provided by way of example only. Many modifications, alterations, and alternatives will arise for those skilled in the art without departing from the spirit and intent of this application. It should be understood that various alternatives to the embodiments of this application described herein may be employed in the practice of this application. The appended claims are intended to define the scope of protection of this application and therefore cover equivalents or alternatives within the scope of these claims.

Claims

1. A method for training a recognition model to identify the integrity of medical site images, comprising: Segment and extract various tissues and organs from medical images to obtain three-dimensional images of each tissue and organ; The images of the medical sites and the three-dimensional images of each tissue and organ are projected into corresponding two-dimensional projection images along multiple projection directions, and labeled, wherein the labels include integrity labels and incomplete labels. A generative model based on labeled 2D projected images is used to separate incomplete image information in order to generate incomplete target images. A recognition model for identifying the integrity of medical body parts is trained based on labeled two-dimensional projected images and incomplete images of the target.

2. The method according to claim 1, wherein segmenting and extracting each tissue and organ in the medical site image to obtain a three-dimensional image corresponding to each tissue and organ includes: The various tissues and organs in the image of the medical site are segmented and extracted to obtain the segmentation mask of each tissue and organ; The segmentation mask of each tissue or organ is multiplied with the image of the medical site to obtain the three-dimensional image corresponding to each tissue or organ.

3. The method of claim 1, wherein the plurality of projection directions includes at least two projection directions selected from the coronal plane, the sagittal plane, and the transverse plane.

4. The method according to claim 1, wherein the generative model includes at least a first encoder, a second encoder, a first decoder, and a second decoder, and training a generative model for separating incomplete image information based on labeled two-dimensional projected images to generate a target incomplete image includes: The two-dimensional projected image labeled with integrity is input into the first encoder and the second encoder respectively for feature encoding, and the first image content feature and the first incomplete information feature are extracted accordingly. The first loss is calculated based on the first incomplete information feature. The first image content features and the first incomplete information features are input into the first decoder for feature decoding to obtain the first reconstructed projection, and the second loss is calculated based on the two-dimensional projected image with the marked integrity label and the first reconstructed projection. The two-dimensional projected images labeled with incompleteness are input into the first encoder and the second encoder respectively for feature encoding, and the second image content features and the second incomplete information features are extracted accordingly. The second image content features and the second incomplete information features are input into the second decoder for feature decoding to obtain the second reconstructed projection, and the third loss is calculated based on the two-dimensional projected image labeled with the incomplete information and the second reconstructed projection. A generative model for separating incomplete image information is trained based on the total loss of the first loss, the second loss, and the third loss to generate an incomplete image of the target.

5. The method according to claim 4, wherein the generated model further includes an integrity discriminator, and the method further includes: The integrity discriminator is used to determine the integrity of the first image content features and the second image content features. Joint gradient inversion is used for integrity adversarial discrimination to decouple image content features from integrity state, and a fourth loss is calculated. A generative model for separating incomplete image information is trained based on the total loss of the first loss, the second loss, the third loss, and the fourth loss to generate an incomplete image of the target.

6. The method of claim 5, wherein the generative model further includes a classifier, and the method further includes: Based on the classifier, determine whether the first incomplete information feature and the second incomplete information feature contain information for distinguishing the integrity status, and calculate the fifth loss; A generative model for separating incomplete image information is trained based on the total loss of the first loss, the second loss, the third loss, the fourth loss, and the fifth loss to generate an incomplete image of the target.

7. The method according to claim 1, wherein training a recognition model for identifying the integrity of medical site images based on labeled two-dimensional projected images and the incomplete target image comprises: The two-dimensional projected image of the labeled tag and the incomplete image of the target are input into the recognition model for integrity recognition, and the loss function is calculated. A recognition model for identifying the integrity of medical site images is trained based on the aforementioned loss function.

8. A method for identifying the integrity of an image of a medical site, comprising: Acquire images of the medical site to be identified; The image of the medical part to be identified is input into the recognition model trained by the method according to any one of claims 1-7 for completeness, and the recognition result is obtained.

9. An electronic device, comprising: processor; as well as A memory storing program instructions for training a recognition model for identifying the integrity of images of medical sites, wherein when the program instructions are executed by the processor, the electronic device performs the method according to any one of claims 1-7; or storing program instructions for identifying the integrity of images of medical sites, wherein when the program instructions are executed by the processor, the electronic device performs the method according to claim 8.

10. A computer-readable storage medium storing computer program instructions thereon for training a recognition model for recognizing the integrity of images of medical sites, the computer program instructions, when executed by one or more processors, causing the implementation of the method according to any one of claims 1-7; or storing computer program instructions for recognizing the integrity of images of medical sites, the computer program instructions, when executed by one or more processors, causing the implementation of the method according to claim 8.

Images