Image processing method and device, storage medium and electronic equipment

Through deep learning models, especially ViT and TAB/SSTB modules, PET/CT image features are extracted, the automation problem of image quality evaluation is solved, and the rapid and accurate image quality evaluation is achieved, manual intervention and cost are reduced, and diagnostics are improved. Efficiency and consistency.

CN118967631APending Publication Date: 2024-11-15SHANGHAI PANORAMIC MEDICAL IMAGING DIAGNOSIS CENT CO LTD +1
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202411066152.1
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2024-08-05
Publication Date
2024-11-15

AI Technical Summary

Technical Problem

The prior art is difficult to automatically evaluate PET/CT image quality, resulting in repeated scanning and diagnostic errors caused by image quality problems. The evaluation process relies on manual intervention, which has problems such as subjective factors and difficulty in obtaining data.

Method used

Using deep learning model, including ViT module and TAB/SSTB module, image features are extracted through multi-layer self-attention mechanism and attention mechanism, a key-layer image quality evaluation model is constructed, the image is divided into tiles and the self-attention mechanism is used to calculate attention. The torque matrix generates weighted feature representations to realize automatic quality evaluation of PET/CT images.

Benefits of technology

Achieve rapid and accurate PET/CT image quality assessment, reduce manual intervention and repeated scanning, improve diagnostic accuracy and efficiency, reduce human resource costs, and reduce subjective factors, improve standardization and consistency of evaluation results. sex.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN118967631A_ABST
    Figure CN118967631A_ABST
Patent Text Reader

Abstract

The invention provides an image processing method and device, a storage medium and electronic equipment. The image processing method comprises the following steps: acquiring a PET / CT key layer image and a quality score of the PET / CT key layer image; constructing a key layer image quality evaluation model; training the key layer image quality evaluation model by using the key layer image and the quality score thereof; verifying the key layer image quality evaluation model on different source data sets; and processing a to-be-evaluated key layer image by using the key layer image quality evaluation model to obtain a quality score of the to-be-evaluated key layer image. According to the image processing method, the PET / CT image quality can be automatically evaluated.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present application belongs to the field of medical image processing technology, and relates to an image processing method, and in particular to an image processing method, device, storage medium and electronic equipment. Background Art

[0002] PET / CT (Positron Emission Tomography / Computed Tomography) is a medical imaging technology that combines positron emission tomography (PET) and computed tomography (CT) and is used to detect and diagnose a variety of diseases, especially cancer. In PET / CT diagnosis, qualified image quality is a prerequisite for a correct diagnosis. However, in clinical work, the image quality of PET / CT may be affected by various factors. How to automatically evaluate the quality of PET / CT images has become one of the technical problems that relevant technicians need to solve urgently. Summary of the invention

[0003] The purpose of this application is to provide an image processing method, device, storage medium and electronic device for realizing automatic evaluation of PET / CT image quality.

[0004] In a first aspect, the present application provides an image processing method, which includes: obtaining PET / CT key layer images and their quality scores; constructing a key layer image quality assessment model; training the key layer image quality assessment model using the key layer images and their quality scores; verifying the key layer image quality assessment model on different source data sets; and processing the key layer images to be evaluated using the key layer image quality assessment model to obtain the quality scores of the key layer images to be evaluated.

[0005] In an implementation of the first aspect, obtaining PET / CT key layer images and their quality scores includes: obtaining a reflux PET / CT image; selecting the key layer image from the PET / CT image; marking the boundaries of key organs on the key layer image; and determining the quality score of the key layer image.

[0006] In an implementation of the first aspect, the image processing method also includes: dividing the key layer image into a training set, a validation set, an internal test set and an external test set, wherein the training set is used to train the key layer image quality assessment model, the validation set is used to optimize the key layer image quality assessment model, the internal test set is used to evaluate the performance of the key layer image quality assessment model, and the external test set is used to evaluate the generalization performance of the key layer image quality assessment model.

[0007] In an implementation manner of the first aspect, the key layer image quality assessment model is a deep learning model.

[0008] In an implementation of the first aspect, the key layer image quality assessment model includes a ViT module, and the ViT module implements feature extraction of the key layer image through a multi-layer self-attention mechanism.

[0009] In an implementation of the first aspect, the key layer image quality assessment model includes a TAB module and an SSTB module, and the TAB module and the SSTB module are used to implement the application of attention mechanism in channel and spatial dimensions, and synergistically enhance the interaction between global and local regions.

[0010] In an implementation of the first aspect, the image processing method also includes: segmenting the key layer image into multiple tiles and constructing a tile sequence; applying a self-attention mechanism to the tile sequence to calculate an attention matrix; and using the attention matrix to weight the original feature vector to generate a weighted feature representation.

[0011] In a second aspect, an embodiment of the present application provides an image processing device, comprising: a data acquisition module for acquiring PET / CT key layer images and their quality scores; a model construction module for constructing a key layer image quality assessment model; a model training module for training the key layer image quality assessment model using the key layer images and their quality scores; a model verification module for verifying the key layer image quality assessment model on different source data sets; and an image quality scoring module for processing the key layer images to be evaluated using the key layer image quality assessment model to obtain the quality scores of the key layer images to be evaluated.

[0012] In a third aspect, an embodiment of the present application provides a computer-readable storage medium having a computer program stored thereon, and when the computer program is executed by a processor, the method described in any one of the first aspects of the embodiments of the present application is implemented.

[0013] In a fourth aspect, an embodiment of the present application provides an electronic device, comprising: a memory storing a computer program; and a processor communicatively connected to the memory, for executing any of the methods described in the first aspect of the embodiment of the present application when the computer program is called.

[0014] The image processing method provided in the embodiment of the present application can quickly evaluate the quality of a large number of PET / CT images, and the evaluation process basically does not require human participation, thereby saving a lot of time and human resources. In addition, the method can reduce the cost of repeated scanning and diagnostic errors caused by image quality problems, which is conducive to improving the accuracy and efficiency of diagnosis and treatment.

[0015] In the embodiment of the present application, the quality of key layer images is scored based on predefined algorithms and criteria, which can improve the standardization and reliability of the evaluation process, reduce the influence of human subjective factors, and improve the accuracy and consistency of the evaluation results.

[0016] The image processing method of the embodiment of the present application can quickly and accurately determine the quality score of key layer images, which is helpful for medical personnel to promptly discover image quality problems and take necessary measures in a timely manner. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] Figure 1 Shown is a schematic diagram of the structure of an electronic device in an embodiment of the present application.

[0018] Figure 2 Shown is a flow chart of an image processing method provided by an embodiment of the present application.

[0019] Figure 3 Shown is a flow chart of obtaining PET / CT key layer images and their quality scores in an embodiment of the present application.

[0020] Figure 4 Shown is a schematic diagram of a key layer image quality assessment model in an embodiment of the present application.

[0021] FIG. 5A to FIG. 5D Shown is an example diagram of scoring rules for different parts in the embodiments of the present application.

[0022] Fig. 6A and Figure 6B Shown is an example diagram of the score distribution of the PET head dataset and the torso dataset in the embodiments of the present application.

[0023] 7A to 7C Shown is an example diagram of the score distribution of the CT head dataset and the torso dataset in the embodiment of the present application.

[0024] Figure 8 Shown is a structural schematic diagram of an image processing device provided in an embodiment of the present application.

[0025] Component number description

[0026] 1 Electronic equipment

[0027] 11 Memory

[0028] 12 Display

[0029] 13 General Purpose Processor

[0030] 131 CPU

[0031] 132 Neural Network Processor

[0032] 8 Image processing device

[0033] 81 Data acquisition module

[0034] 82 Model building modules

[0035] 83 Model training module

[0036] 84 Model Validation Module

[0037] 85 Image Quality Scoring Module

[0038] Steps S21 to S25

[0039] Steps S31 to S34 DETAILED DESCRIPTION

[0040] The following describes the embodiments of the present application through specific examples, and those skilled in the art can easily understand other advantages and effects of the present application from the contents disclosed in this specification. The present application can also be implemented or applied through other different specific embodiments, and the details in this specification can also be modified or changed in various ways based on different viewpoints and applications without departing from the spirit of the present application. It should be noted that the following embodiments and features in the embodiments can be combined with each other without conflict.

[0041] It should be noted that the illustrations provided in the following embodiments are only schematic illustrations of the basic concept of the present application, and thus the drawings only show components related to the present application rather than being drawn according to the number, shape and size of components in actual implementation. In actual implementation, the type, quantity and proportion of each component may be changed at will, and the component layout may also be more complicated.

[0042] In the description of this application, the terms "first" and "second" are used for descriptive purposes only and should not be understood as indicating or implying relative importance or implicitly indicating the number of the indicated technical features. Thus, a feature defined as "first" or "second" may explicitly or implicitly include one or more of the features. In the description of this application, the meaning of "plurality" is two or more, unless otherwise clearly and specifically defined.

[0043] In this application, unless otherwise clearly specified and limited, the terms "connected" and "connection" should be understood in a broad sense, for example, it can be a mechanical connection or an electrical connection; it can be a direct connection or an indirect connection through an intermediate medium, it can be the internal connection of two elements or the interaction relationship between two elements. For ordinary technicians in this field, the specific meanings of the above terms in this application can be understood according to specific circumstances.

[0044] Some technical solutions use methods based on image features to evaluate the quality of PET / CT images. These technical solutions involve extracting various features from PET / CT images and evaluating the quality of the images based on these features. The above features include local contrast, grayscale histogram, texture features, etc. Based on the statistical properties of these features, a model can be established to automatically evaluate the quality of PET / CT images.

[0045] Some technical solutions use machine learning-based methods to evaluate the quality of PET / CT images. In these technical solutions, machine learning algorithms, such as support vector machines (SVM) and random forests, are used to automatically identify and evaluate the quality of PET / CT images through training models. These algorithms can learn from a large amount of labeled data to achieve automatic evaluation of image quality.

[0046] Some technical solutions use statistical methods to evaluate the quality of PET / CT images. In these technical solutions, the quality of images is evaluated by analyzing the statistical properties of the images. For example, the quality of PET / CT images can be quantitatively evaluated using indicators such as signal-to-noise ratio (SNR), contrast, and resolution.

[0047] Some technical solutions use deep learning-based methods to evaluate the quality of PET / CT images. In these technical solutions, deep learning models, such as convolutional neural networks (CNNs), are used to directly learn features from image data and predict image quality scores.

[0048] Some technical solutions combine multiple methods to evaluate the quality of PET / CT images. For example, feature extraction and machine learning methods can be combined to consider both local and global features of the image to improve the accuracy and stability of the evaluation results.

[0049] However, there are some problems with the quality assessment methods of PET / CT images in the above technical solutions. For methods based on image features, their implementation relies on manually designed feature extraction methods, and these features may not fully capture the complex information of the image, so they cannot accurately reflect the quality of the image. For methods based on machine learning, a large amount of labeled data is usually required to train the model, but obtaining and labeling this data may be very expensive and time-consuming. In addition, due to the medical nature of PET / CT images, privacy and ethical issues will also limit the acquisition and use of data. Furthermore, due to the diversity and complexity of PET / CT images, the generalization ability of the trained model in different data sets or application scenarios may be limited, which means that the model may not be accurately generalized to new data. For methods based on deep learning models, since the internal working mechanism of deep learning models is usually lacking in interpretability, that is, it is difficult to explain how the model makes predictions, this may make it difficult to understand the factors that affect the model on image quality. In addition, the assessment of image quality is often affected by individual subjective factors, and different doctors or researchers may give different scores to the same image. The lack of standardized evaluation methods also makes it difficult to compare and reproduce the results in different schemes.

[0050] In view of the problems existing in the above technical solutions, the present application provides an image processing method. Next, the image processing method provided by the present application will be described in detail by way of specific embodiments combined with the accompanying drawings.

[0051] Figure 1 The diagram shows a schematic diagram of the structure of an electronic device in an embodiment of the present application. Figure 1 As shown, the electronic device 1 includes a memory 11 , a display 12 and at least one general purpose processor 13 .

[0052] The memory 11 may include a volatile memory, such as a random access memory (RAM) and a cache. The memory 11 may also include a non-volatile memory, such as a read-only memory (ROM), a flash memory, a hard disk drive (HDD) or a solid-state drive (SSD). The memory 11 may be used to store program instructions for the processor to call and execute at least a portion of the image processing method provided in the embodiment of the present application.

[0053] The display 12 may specifically include a display screen (display panel). In some implementations, the display panel may be configured in the form of a liquid crystal display (LCD), an organic light-emitting diode (OLED), or the like. In addition, the display 12 may also be a touch panel (touch screen, touch screen), which may include a display screen and a touch-sensitive surface. When the touch-sensitive surface detects a touch operation on or near it, it is transmitted to the general processor 13 to determine the type of touch event, and then the general processor 13 provides a corresponding visual output on the display device according to the type of touch event.

[0054] The general processor 13 can be any type of device capable of processing electronic instructions. In the embodiment of the present application, the electronic device 1 may include one or more general processors 13, such as a central processing unit (CPU) 131 and a neural network processor (NPU) 132. One or two. In addition, it may also include a graphics processor (GPU), a microprocessor, a microcontroller, a main processor, a controller, and an ASIC (Application Specific Integrated Circuit). The general processor 13 is configured to execute various types of digital storage instructions, such as software or firmware programs stored in the memory 11, which enables the electronic device 1 to provide a variety of services. For example, the processor 11 can execute a program or process data to execute at least a portion of the image processing method provided in the embodiment of the present application.

[0055] The main function of the central processing unit 131 is to parse computer instructions and process data in computer software, realize overall control of the electronic device 1, and control and allocate all hardware resources of the electronic device 1 (such as storage resources, communication resources, I / O interfaces, etc.).

[0056] Neural network processor is a general term for new processors based on neural network algorithms and acceleration. It is specially designed for artificial intelligence and is used to accelerate neural network calculations and solve the problem of low efficiency of traditional chips in neural network calculations.

[0057] It should be noted that the name of the neural network processor does not constitute a limitation of the present application. For example, in other application scenarios, the neural network processor can also be transformed and replaced by other processors with similar functions, such as a tensor processor (Tensor Processing Unit, TPU), a deep learning processor (Deep learning Processing Unit, DPU), and so on.

[0058] Figure 2 The flowchart of the image processing method provided by the embodiment of the present application is shown. Figure 2 As shown, the image processing method provided in the embodiment of the present application includes the following steps S21 to S25.

[0059] S21, obtain PET / CT key layer images and their quality scores.

[0060] S22, constructing a key layer image quality assessment model. For example, the key layer image can be used as a model input to construct a suitable clustering model or regression model as the key layer image quality assessment model.

[0061] S23, using the key layer images and their quality scores to train the key layer image quality assessment model. The trained key layer image quality assessment model can automatically score the image quality of the PET / CT key layer images according to the features learned from the images.

[0062] S24, validate the key layer image quality assessment model on different source datasets.

[0063] S25, using the key layer image quality assessment model to process the key layer image to be evaluated to obtain a quality score of the key layer image to be evaluated.

[0064] See also Figure 3 In some implementations, obtaining PET / CT key layer images and their quality scores includes the following steps S31 to S34.

[0065] S31, obtaining the returned PET / CT images. Specifically, the hospital may screen the included data according to the pre-agreed data inclusion and exclusion criteria, and return the included PET / CT images, which may come from multiple data centers (multiple hospitals).

[0066] S32, selecting key layer images from the PET / CT images. The selection of key layer images can be done manually by an radiologist or automatically by an image processing device, and this application does not limit this.

[0067] For example, for whole-body PET images, images at 7 levels can be selected as key layer images, and for CT images, images at 9 levels can be selected as key layer images.

[0068] S33, marking the boundaries of key organs on the key layer image. Specifically, after selecting the key layer image, the boundaries of several key organs can be outlined and marked on the cross-sectional images of all or part of the key layers to assist in the positioning of the key layers. The key organs can be determined according to actual needs, for example, including the aortic arch, right lung, tracheal carina, mediastinum, liver, pancreas, spleen, esophagus, iliac artery bifurcation, ilium, etc.

[0069] S34, determining the quality score of the key layer images. Specifically, the imaging doctor can evaluate and score all key layer images (including PET key layer images and CT key layer images) according to a unified standard based on clinical experience.

[0070] In some implementations, the selection results of key layer images, the boundary marking results of key organs, and the quality scores are independently marked by two experienced radiologists and then reviewed by a senior radiologist (two marks and one review) to reduce subjective errors.

[0071] In some implementations, the image processing method may further include: dividing the key layer images into a training set, a validation set, an internal test set, and an external test set. The training set is used to preliminarily establish and train the key layer image quality assessment model, the validation set is used to adjust and optimize the key layer image quality assessment model, the internal test set is used to evaluate the performance of the key layer image quality assessment model, and the external test set is used to evaluate the generalization performance of the key layer image quality assessment model.

[0072] For example, for key layer images and their annotation results from multiple data centers, the key layer images and their annotation results from one or two data centers with the most data are selected and divided into training sets, validation sets, and internal test sets according to a certain ratio (e.g., 7:2:1). The key layer images and their annotation results from other data centers are used as external test sets to evaluate the generalization performance of the model.

[0073] In some implementations, the image processing method may further include: evaluating the performance of the key layer image quality assessment model.

[0074] For example, the doctor's score can be used as a reference standard to calculate the score accuracy (Accuracy) of the key layer image quality assessment model.

[0075] Exemplarily, the key layer image quality assessment model processes the key layer images and obtains the quality score as a multi-classification problem. The classification can be merged and converted into a binary classification problem to evaluate the sensitivity and specificity of the key layer image quality assessment model, draw the ROC curve and calculate the AUC indicator.

[0076] For example, a human-computer controlled trial can be designed to calculate the inter-reader consistency between the key layer image quality assessment model and the physician's score and calculate the inter-group consistency (ICC) index.

[0077] For example, the generalization ability of the model can be evaluated by comparing the performance of the key layer image quality assessment model on the internal test set and the external test set.

[0078] In some implementations, the key layer image quality assessment model is a deep learning model.

[0079] Figure 4 The diagram shows a key layer image quality assessment model in an embodiment of the present application. Figure 4 As shown in FIG, the key layer image quality assessment model includes a ViT (Vision Transformer) module. The ViT module extracts the features of the key layer images through a multi-layer self-attention mechanism.

[0080] The key layer image quality assessment model can also include a TAB (Transposed Attention Block) module and a SSTB (Scale Swin Transformer Block) module. The TAB module and the SSTB module are used to implement the application of the attention mechanism in the channel and spatial dimensions, and synergistically enhance the interaction between the global and local regions.

[0081] like Figure 4 As shown, the input image is divided into several tiles of the same size, which are arranged in rows or columns to form a tile sequence, denoted by X = [x1, x2, ..., x N ], where N is the number of tiles after image segmentation, x iis a feature vector of size D, representing the i-th tile. Position encoding is applied to each position in the input sequence to obtain position embedding, which is used to represent the position information in the image. The tile sequence and position embedding are input to the ViT module for feature extraction. The ViT module implements feature extraction through a multi-layer self-attention mechanism. Each self-attention layer calculates the attention weights of each position in the input sequence and applies these weights to the original feature vector to generate the final feature representation.

[0082] For example, the self-attention mechanism is applied to the input sequence X to calculate the attention matrix A as shown in Formula 1:

[0083]

[0084] Where W Q and W K is the weight matrix of query and key, d k is the dimension of the attention head.

[0085] The original feature vector X is weighted using the attention matrix to generate the weighted feature representation Y as shown in Equation 2:

[0086] Y=AX, formula 2.

[0087] To increase the expressiveness of the model, multiple attention heads can be used to repeat the above process. A feedforward network can be added after each attention layer to perform nonlinear transformations on the weighted feature representations.

[0088] The features extracted by the ViT module are concatenated in the Feature Map Concat module and then input into the TAB module. After passing through the SSTB module, the final weighted score is output through two branches (scoring branch and weight branch).

[0089] According to the above, the ViT module is used for image feature extraction in the embodiment of the present application. Compared with traditional convolutional neural networks (such as CNN), ViT can better capture the global features and long-distance dependencies of images, and has better characterization capabilities. In addition, the TAB module is introduced in the embodiment of the present application, and the adaptive reorganization and adjustment of image features are achieved by performing transposition operations on channels and spatial dimensions, thereby enhancing the interaction between global and local areas. In combination with the STTB module, information of different scales in the image can be more effectively captured, and the model's ability to represent image features is improved. By combining the TAB module and the STTB module, synergistic enhancement between global and local areas can be achieved. Therefore, the image processing method provided in the embodiment of the present application can make more full use of the global and local information in the image, and improve the characterization ability of image features and the accuracy of the model.

[0090] It should be noted that the above is only a feasible way of the embodiment of the present application, but the present application is not limited thereto. In addition, in some embodiments, classic CNN architectures such as ResNet, VGG, etc. can be used, and new CNN architectures can also be designed to extract image features. In some other embodiments, variants of other attention mechanisms other than TAB and STTB can also be used, such as self-attention mechanisms (Self-Attention), multi-head attention (Multi-Head Attention), etc. These attention mechanisms can introduce attention at different levels and dimensions to achieve interaction between global and local features. In some other embodiments, the image can be segmented into multiple regions, and each region is modeled and analyzed to extract local features. These local features are then summarized and combined with global features for comprehensive evaluation. This method requires the use of image segmentation algorithms and regional modeling techniques, such as Mask R-CNN, etc. In some further embodiments, image enhancement and data enhancement techniques can be used to improve image quality, increase the robustness and generalization ability of the model, and operations such as rotation, scaling, and flipping can also be used to increase the diversity of training data, thereby improving the performance of the model. In some other embodiments, an integrated learning method can be used to combine the prediction results of multiple different models to obtain a more accurate final prediction result. This approach can reduce the overfitting of the model and improve the robustness and generalization ability of the model.

[0091] In some implementations, the quality score of the key layer image ranges from 0 to 5. According to the score, the quality of the key layer image can be divided into: extremely poor (less than 1.5 points), poor (greater than or equal to 1.5 and less than 2.5 points), medium (greater than or equal to 2.5 and less than 3.5 points), good (greater than or equal to 3.5 and less than 4.5 points), and excellent (greater than or equal to 4.5 points).

[0092] During the training process, the key layer image quality assessment model can be used to predict the quality score of the key layer image. When verifying the accuracy of the results, the classification standard can be used to classify the predicted results according to the above classification rules, and the consistency between the classification results and the classification results corresponding to the real scores can be used as the prediction quality evaluation standard.

[0093] For example, Figure 5A Shown is an example diagram of the scoring rules for the basal ganglia and cerebellar vermis. Figure 5B Shown is an example of the scoring rules for the tracheal carina and aortic arch. Figure 5C An example diagram showing the scoring rules for the liver and pancreas. Figure 5D An example of the scoring rule for the patellar artery bifurcation is shown. It can be seen from the figure that there is no image of the pancreas and the patellar artery bifurcation that is classified as an excellent score.

[0094] For example, Fig. 6A Shown is an example of the score distribution of the PET torso dataset. Figure 6B The following is an example of the score distribution of the PET head dataset. Fig. 6A and 6B As shown in the figure, the scores of the PET torso data set are mostly between poor and medium, and the scores of the pelvis and pancreas, liver and tracheal carina are similar. The scores of the head data are mostly concentrated in the medium level.

[0095] For example, Fig. 7A Shown is an example of the score distribution of the CT head dataset. Figure 7B Shown is an example of the score distribution of the CT torso dataset. Figure 7C The following is an example of the score distribution of the CT torso dataset. Figures 7A to 7C As shown in the figure, the score distribution of the CT head dataset and the PET head dataset are almost concentrated at a medium level, and the scores of the CT torso dataset are mostly at a good or excellent level.

[0096] For example, Table 1 and Table 2 show examples of calculation results and confusion matrix, respectively. Combining the scoring results of the key layer image quality assessment model and the scoring results of the doctor, the confusion matrix can be obtained, and the recall rate, precision rate and f1 are calculated, so that the PET / CT image quality can be automatically scored with high accuracy.

[0097] Table 1. Calculation results example

[0098] category Recall Accuracy f1 Excellent 1.00 1.00 1.00 good 1.00 1.00 1.00 medium 0.97 0.88 0.93 Difference 0.71 0.80 0.75 Very bad 0.33 1.00 0.50

[0099] Table 2. Confusion matrix example

[0100] Excellent 0 0 0 0 1 good 0 0 0 2 0 medium 0 5 37 0 0 Difference 2 12 1 0 0 Very bad 1 0 0 0 0 category Very bad Difference medium good Excellent

[0101] In some implementations, the image processing method may further include: reconstructing the PET / CT image. During the reconstruction process, appropriate reconstruction algorithms and parameters may be selected to minimize noise and artifacts while retaining information useful for diagnosis.

[0102] In some implementations, the image processing method may further include: removing noise and artifacts. Since PET / CT images may be interfered by noise from various sources, reducing noise and artifacts is crucial to image quality assessment. For example, filters, statistical reconstruction methods, or post-processing techniques may be used to reduce noise and artifacts in images.

[0103] In some implementations, the image processing method may further include: aligning and correcting the PET and CT images. Since the PET and CT images are obtained by different imaging technologies, it is necessary to ensure accurate alignment and correction between the PET and CT images during the image quality assessment process, including calibrating the space and intensity of the images, and reducing or eliminating artifacts or deviations between the PET and CT images.

[0104] The protection scope of the image processing method provided by the embodiment of the present application is not limited to the execution order of the steps listed in the present embodiment. All solutions implemented by adding, reducing or replacing steps in the prior art based on the principles of the present application are included in the protection scope of the present application.

[0105] The embodiments of the present application also provide an image processing device, which can implement the image processing method provided in the embodiments of the present application. However, the implementation device of the image processing method provided in the embodiments of the present application includes but is not limited to the structure of the image processing device listed in the embodiments of the present application. All structural deformations and replacements of the prior art made according to the principles of the embodiments of the present application are included in the protection scope of the present application.

[0106] Figure 8 A schematic diagram of an image processing device provided by an embodiment of the present application is shown. Figure 8 As shown, the image processing device 8 includes a data acquisition module 81, a model construction module 82, a model training module 83, a model verification module 84 and an image quality scoring module 85. The data acquisition module 81 is used to acquire PET / CT key layer images and their quality scores. The model construction module 82 is used to build a key layer image quality assessment model. The model training module 83 is used to train the key layer image quality assessment model using the key layer images and their quality scores. The model verification module 84 is used to verify the key layer image quality assessment model on different source data sets. The image quality scoring module 85 is used to process the key layer images to be evaluated using the key layer image quality assessment model to obtain the quality score of the key layer images to be evaluated.

[0107] It should be noted that the image processing device 8 includes a data acquisition module 81, a model building module 82, a model training module 83, a model verification module 84 and an image quality scoring module 85. Figure 2 Steps S21 to S25 in the image processing method shown correspond to each other and are not described in detail here.

[0108] In the several embodiments provided in the present application, it should be understood that the disclosed system, device or method can be implemented in other ways. For example, the device embodiments described above are only schematic. For example, the division of modules / units is only a logical function division. There may be other division methods in actual implementation, such as multiple modules or units can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be an indirect coupling or communication connection through some interfaces, devices or modules or units, which can be electrical, mechanical or other forms.

[0109] The modules / units described as separate components may or may not be physically separated, and the components displayed as modules / units may or may not be physical modules, that is, they may be located in one place, or they may be distributed on multiple network units. Some or all of the modules / units may be selected according to actual needs to achieve the purpose of the embodiments of the present application. For example, the functional modules / units in the various embodiments of the present application may be integrated into one processing module, or each module / unit may exist physically separately, or two or more modules / units may be integrated into one module / unit.

[0110] Those of ordinary skill in the art should further appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, computer software, or a combination of the two. In order to clearly illustrate the interchangeability of hardware and software, the composition and steps of each example have been generally described in the above description according to function. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of this application.

[0111] The present application embodiment also provides a computer-readable storage medium on which a computer program is stored. A person of ordinary skill in the art can understand that all or part of the steps in the method for implementing the above embodiment can be completed by instructing the processor through a program, and the program can be stored in a computer-readable storage medium, and the storage medium is a non-transitory medium, such as a random access memory, a read-only memory, a flash memory, a hard disk, a solid-state hard disk, a magnetic tape, a floppy disk, an optical disc, and any combination thereof. The above storage medium can be any available medium that a computer can access or a data storage device such as a server or a data center that includes one or more available media integrated. The available medium can be a magnetic medium (e.g., a floppy disk, a hard disk, a tape), an optical medium (e.g., a digital video disc (DVD)), or a semiconductor medium (e.g., a solid-state hard disk (SSD)), etc.

[0112] The present application embodiment may also provide a computer program product, the computer program product including one or more computer instructions. When the computer instructions are loaded and executed on a computing device, the process or function described in the embodiment of the present application is generated in whole or in part. The computer instructions may be stored in a computer-readable storage medium, or transmitted from one computer-readable storage medium to another computer-readable storage medium, for example, the computer instructions may be transmitted from a website site, computer or data center by wired (e.g., coaxial cable, optical fiber, digital subscriber line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) mode to another website site, computer or data center.

[0113] When the computer program product is executed by a computer, the computer executes the method described in the above method embodiment. The computer program product may be a software installation package, and when the above method is required, the computer program product may be downloaded and executed on a computer.

[0114] The embodiment of the present application also provides an electronic device. The electronic device includes a memory and a processor. The memory stores a computer program. The processor is connected to the memory for communication, and executes the image processing method provided in the embodiment of the present application when the computer program is called.

[0115] In some implementations, the electronic device may further include a display for displaying a GUI (Graphical User Interface) related to the image processing method.

[0116] The descriptions of the processes or structures corresponding to the above-mentioned figures have different emphases. For parts that are not described in detail in a certain process or structure, please refer to the relevant descriptions of other processes or structures.

[0117] In summary, the present application embodiment provides an image processing method, device, storage medium and electronic device, which can automatically score the quality of PET / CT key layer images, and this method can improve the evaluation efficiency, realize rapid evaluation of a large number of images, and reduce manual intervention and time-consuming. In addition, the automatic scoring of the present application embodiment can reduce the human error of the scoring process, reduce the influence of individual subjective factors through standardized algorithms and criteria, and improve the accuracy and consistency of the evaluation results. In addition, the automatic scoring of the present application embodiment can save human resource costs, and can promptly discover image quality problems, reducing the cost of repeated scanning and diagnostic errors caused by poor quality. Furthermore, the automatic scoring of the present application embodiment helps to optimize clinical decision-making, improves the accuracy and efficiency of diagnosis and treatment, and brings important benefits to medical imaging workflow and clinical practice. Therefore, the present application effectively overcomes the various defects of the prior art and has a higher industrial value.

[0118] The above embodiments are merely illustrative of the principles and effects of the present application and are not intended to limit the present application. Anyone familiar with the technology may modify or change the above embodiments without violating the spirit and scope of the present application. Therefore, all equivalent modifications or changes made by a person of ordinary skill in the art without departing from the spirit and technical ideas disclosed in the present application shall still be covered by the claims of the present application.

Claims

1. An image processing method, characterized in that: The image processing method comprises: Obtain PET / CT key layer images and their quality scores; Construct key layer image quality assessment model; Using the key layer images and their quality scores to train the key layer image quality assessment model; Validating the key layer image quality assessment model on different source data sets; The key layer image to be evaluated is processed using the key layer image quality assessment model to obtain a quality score of the key layer image to be evaluated.

2. The image processing method according to claim 1, characterized in that: Acquisition of key PET / CT image layers and their quality scores include: Obtain PET / CT images of reflux; Selecting the key layer image from the PET / CT image; Marking the boundaries of key organs on the key layer image; A quality score of the key layer image is determined.

3. The image processing method according to claim 1, characterized in that: The image processing method further comprises: The key layer images are divided into a training set, a validation set, an internal test set and an external test set, wherein the training set is used to train the key layer image quality assessment model, the validation set is used to optimize the key layer image quality assessment model, the internal test set is used to evaluate the performance of the key layer image quality assessment model, and the external test set is used to evaluate the generalization performance of the key layer image quality assessment model.

4. The image processing method according to claim 1, characterized in that: The key layer image quality assessment model is a deep learning model.

5. The image processing method according to claim 4, characterized in that: The key layer image quality assessment model includes a ViT module, and the ViT module realizes feature extraction of the key layer image through a multi-layer self-attention mechanism.

6. The image processing method according to claim 4, characterized in that: The key layer image quality assessment model includes a TAB module and an SSTB module, and the TAB module and the SSTB module are used to implement the application of the attention mechanism in the channel and spatial dimensions, and synergistically enhance the interaction between the global and local regions.

7. The image processing method according to claim 4, characterized in that: The image processing method further comprises: Segmenting the key layer image into a plurality of tiles and constructing a tile sequence; Applying a self-attention mechanism to the tile sequence to calculate an attention matrix; The original feature vector is weighted using the attention matrix to generate a weighted feature representation.

8. An image processing device, characterized in that: The image processing device comprises: Data acquisition module, used to obtain PET / CT key layer images and their quality scores; Model building module, used to build key layer image quality assessment model; A model training module, used for training the key layer image quality assessment model using the key layer image and its quality score; A model verification module, used for verifying the key layer image quality assessment model on different source data sets; The image quality scoring module is used to process the key layer image to be evaluated using the key layer image quality assessment model to obtain the quality score of the key layer image to be evaluated.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the method according to any one of claims 1 to 7 is implemented.

10. An electronic device, characterized in that: The electronic device comprises: A memory storing a computer program; A processor is communicatively connected to the memory, and executes the method according to any one of claims 1 to 7 when calling the computer program.