Artificial intelligence auxiliary multi-target imaging tracking system based on focus form

Abstract

The invention relates to the technical field of imaging tracking, and particularly discloses an artificial intelligence-assisted multi-target imaging tracking system based on a lesion form, which utilizes an artificial intelligence image recognition and segmentation technology to perform multi-scale feature extraction on a medical image, and combines shallow and deep feature coding to realize multi-target imaging tracking. And semantic information-oriented joint perception representation is carried out, so that semantic segmentation of the image is intelligently generated. And identifying and analyzing a focus area form in the first medical image. And then, repeating the above process for the medical image of the same patient in another time period, and comparing the morphological characteristics of the lesion area in the two images so as to realize accurate tracking of the lesion area. According to the method, a plurality of lesions can be identified and analyzed in the same image at the same time, so that all lesion areas are rapidly and accurately marked, more detail information morphological characteristics are captured, personal errors are reduced, the limitation of a traditional image analysis means is overcome, and a powerful auxiliary tool is provided for clinicians.

Description

Technical Field

[0001] The present application relates to the field of imaging tracking technology, and more specifically, to an artificial intelligence-assisted multi-target imaging tracking system based on lesion morphology. Background Art

[0002] As a key source of information in the diagnosis and treatment of diseases, medical imaging is of great importance. In clinical practice, many diseases present the characteristics of multiple lesions, which greatly increases the difficulty of diagnosis and treatment. Taking cancer as an example, multiple tumor lesions in the liver of patients with multiple liver cancers have completely different growth rates, invasive abilities, and responses to treatment due to differences in the liver microenvironment, such as blood supply and tissue metabolism.

[0003] However, traditional imaging analysis methods mainly rely on the doctor's experience and naked eye observation, which makes it difficult to conduct comprehensive, detailed and dynamic monitoring and analysis of multiple lesions at the same time, so it is impossible to provide complete and accurate disease information, which seriously affects the accuracy of diagnosis and the formulation of treatment plans.

[0004] Therefore, an artificial intelligence-assisted multi-target imaging and tracking solution based on lesion morphology is desired. Summary of the invention

[0005] The present application provides an artificial intelligence-assisted multi-target imaging tracking system based on lesion morphology, which can simultaneously identify and analyze multiple lesions in the same image to quickly and accurately mark all lesion areas and capture more detailed information and morphological features, reducing human errors, thereby overcoming the limitations of traditional image analysis methods and providing clinicians with a powerful auxiliary tool.

[0006] In the first aspect, an artificial intelligence-assisted multi-target imaging tracking system based on lesion morphology is provided, comprising: A first medical image acquisition module, used to acquire a first medical image of a target patient; A medical image multi-scale extraction module, used for performing a medical image multi-scale feature scanning on the first medical image to obtain a medical image shallow feature coding feature map and a medical image deep feature coding feature map; A medical image multi-scale feature perception module, used for performing semantic information-guided deep-shallow feature joint perception on the medical image shallow feature encoding feature map and the medical image deep feature encoding feature map to obtain a medical image shallow-deep joint perception encoding feature map; A medical image semantic segmentation module, used for performing semantic segmentation on the shallow-deep joint perceptual coding feature map of the medical image to obtain a semantic segmentation result of the medical image; A lesion region identification module, used for identifying the lesion region in the first medical image based on the semantic segmentation result of the medical image to obtain a first lesion region identification result; A lesion area morphological feature determination module is used to determine the morphological features of each lesion area in the first lesion area identification result based on the semantic segmentation result of the medical image.

[0007] In the above-mentioned artificial intelligence-assisted multi-target imaging and tracking system based on lesion morphology, the morphological characteristics of each lesion area are the size, shape, boundary and density of each lesion area.

[0008] In the above-mentioned artificial intelligence-assisted multi-target imaging and tracking system based on lesion morphology, the medical image multi-scale extraction module is used to: input the first medical image into a medical image multi-scale feature scanner to obtain the medical image shallow feature encoding feature map and the medical image deep feature encoding feature map.

[0009] In the above-mentioned artificial intelligence-assisted multi-target imaging and tracking system based on lesion morphology, the medical image multi-scale feature scanner is a medical image multi-scale feature scanner based on a hollow pyramid network model.

[0010] In the above-mentioned artificial intelligence-assisted multi-target imaging tracking system based on lesion morphology, the medical image multi-scale feature perception module includes: A medical image upsampling unit, used for upsampling the medical image deep feature encoding feature map to obtain an upsampled medical image deep feature encoding feature map; A medical image deep feature field modulation unit, configured to perform field modulation on the upsampled medical image deep feature encoding feature map based on the medical image shallow feature encoding feature map to obtain a medical image field modulated deep feature map; The medical image feature joint perception unit is used to perform feature joint perception of the medical image shallow feature encoding feature map and the medical image field modulation deep feature map with multiple attention structures to obtain the medical image shallow-deep joint perception encoding feature map.

[0011] In the above-mentioned artificial intelligence-assisted multi-target imaging and tracking system based on lesion morphology, the upsampled medical image deep feature encoding feature map and the medical image shallow feature encoding feature map have the same size.

[0012] In the above-mentioned artificial intelligence-assisted multi-target imaging and tracking system based on lesion morphology, the medical image deep feature field modulation unit is used to: After feature connection of the upsampled medical image deep feature encoding feature map and the medical image shallow feature encoding feature map, a convolution kernel-based gated convolution semantic information field prediction is performed on the obtained medical image shallow-deep feature connection encoding feature map to obtain a medical image feature semantic information field; The upsampled medical image deep feature encoding feature map is mapped to the medical image feature semantic information field to obtain the medical image field modulation deep feature map.

[0013] In the above-mentioned artificial intelligence-assisted multi-target imaging tracking system based on lesion morphology, the medical image semantic segmentation module is used to: input the shallow-deep joint perceptual coding feature map of the medical image into a semantic segmenter based on the Softmax function to obtain the semantic segmentation result of the medical image.

[0014] The above-mentioned artificial intelligence-assisted multi-target imaging tracking system based on lesion morphology also includes: A second medical image acquisition module, used to acquire a second medical image of the target patient; A second medical image processing module, used for processing the second medical image to obtain a second lesion region identification result and morphological features of each lesion region in the second lesion region identification result; The lesion area target tracking module is used to track the lesion area target based on the morphological features of each lesion area in the first lesion area identification result and the morphological features of each lesion area in the second lesion area identification result.

[0015] The present application provides an artificial intelligence-assisted multi-target imaging tracking system based on lesion morphology, which uses artificial intelligence image recognition and segmentation technology to implement multi-scale feature extraction on medical images, combines shallow and deep feature encoding, and performs semantic information-guided joint perception representation, thereby intelligently generating semantic segmentation of images. Based on this, the morphology of the lesion area in the first medical image is identified and analyzed. Subsequently, the above process is repeated for another period of medical images of the same patient, and the morphological features of the lesion area in the two images are compared to achieve accurate tracking of the lesion area. The present application can simultaneously identify and analyze multiple lesions in the same image, so as to quickly and accurately mark all lesion areas, and capture more detailed information morphological features, reduce human errors, thereby overcoming the limitations of traditional image analysis methods, and providing a powerful auxiliary tool for clinicians. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the drawings of the embodiments of the present application are briefly introduced below. Obviously, the drawings described below only relate to some embodiments of the present application, and are not intended to limit the present application.

[0017] Figure 1 This is a schematic block diagram of an artificial intelligence-assisted multi-target imaging and tracking system based on lesion morphology according to an embodiment of the present application.

[0018] Figure 2 This is a data flow diagram of an artificial intelligence-assisted multi-target imaging and tracking system based on lesion morphology according to an embodiment of the present application.

[0019] Figure 3 This is a schematic block diagram of a medical image multi-scale feature perception module in an artificial intelligence-assisted multi-target imaging tracking system based on lesion morphology in an embodiment of the present application.

[0020] Figure 4 This is a schematic block diagram of the second medical image processing in the artificial intelligence-assisted multi-target imaging tracking system based on lesion morphology in an embodiment of the present application.

[0021] Figure 5 This is a schematic flowchart of the artificial intelligence-assisted multi-target imaging tracking method based on lesion morphology according to an embodiment of the present application. DETAILED DESCRIPTION

[0022] The technical solutions in the embodiments of the present application will be described clearly and completely below in conjunction with the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present application, not all embodiments. Based on the embodiments of the present application, all other embodiments obtained by ordinary technicians in this field without creative work also fall within the scope of protection of the present application.

[0023] In view of the above technical problems, the technical concept of the present application is to obtain a first medical image of a target patient object, and use an artificial intelligence-based image recognition and segmentation technology to perform multi-scale feature extraction on the first medical image, so as to intelligently obtain the semantic segmentation result of the medical image based on the joint perception representation guided by the semantic information between the shallow feature coding features of the medical image and the deep feature coding features of the medical image. Then, the lesion area in the first lesion area is identified based on the segmentation result, and its morphological characteristics are analyzed. At the same time, the second medical image of the target patient object is obtained, and the above processing steps are also performed to achieve accurate tracking of the lesion area based on the morphological characteristics of each lesion area in the first lesion area identification result and the second lesion area identification result. The present application can simultaneously identify and analyze multiple lesions in the same image, so as to quickly and accurately mark all lesion areas, and capture more detailed information morphological characteristics, reduce human errors, thereby overcoming the limitations of traditional image analysis methods and providing a powerful auxiliary tool for clinicians.

[0024] Specifically, in the technical solution of the present application, if Figure 1 and Figure 2As shown, the artificial intelligence-assisted multi-target imaging tracking system based on lesion morphology includes: a first medical image acquisition module 10, used to acquire a first medical image of a target patient object; a medical image multi-scale extraction module 20, used to perform medical image multi-scale feature scanning on the first medical image to obtain a medical image shallow feature coding feature map and a medical image deep feature coding feature map; a medical image multi-scale feature perception module 30, used to perform semantic information-guided deep and shallow feature joint perception on the medical image shallow feature coding feature map and the medical image deep feature coding feature map to obtain a medical image shallow-deep joint perception coding feature map; a medical image semantic segmentation module 40, used to perform semantic segmentation on the medical image shallow-deep joint perception coding feature map to obtain a semantic segmentation result of the medical image; a lesion area identification module 50, used to identify the lesion area in the first medical image based on the semantic segmentation result of the medical image to obtain a first lesion area identification result; a lesion area morphological feature determination module 60, used to determine the morphological features of each lesion area in the first lesion area identification result based on the semantic segmentation result of the medical image.

[0025] Exemplarily, in the first medical image acquisition module 10, a first medical image of the target patient object is acquired. It should be understood that in actual clinical diagnosis and treatment, many patients do not have only a single lesion, and it is more common to suffer from multiple tumors or multiple lesions. These multiple lesions each have different development characteristics and change trends, which may be affected by the progression of the disease itself and the differential effects of different treatment methods. The medical image of the target patient object provides a snapshot of the state of the disease at a specific point in time. This image not only records important morphological features such as the location, size, shape, boundary and density of each lesion, but also lays the foundation for subsequent analysis and evaluation of treatment effects. Especially when faced with multiple tumors or other complex diseases, it is crucial to accurately capture this initial information because they are an important basis for formulating personalized treatment plans.

[0026] In one embodiment, in order to ensure that the first medical image obtained can fully reflect the lesion situation, high-resolution imaging technology is used, such as computed tomography (CT), magnetic resonance imaging (MRI) or positron emission tomography (PET). Doctors or technicians will choose the most suitable imaging method according to specific clinical needs and adjust the parameters to obtain the clearest image. At the same time, in actual operation, considering that many patients do not have only a single lesion, it is more common to have multiple tumors or multiple lesions, so there are multiple lesion targets in the first medical image obtained. These lesions each have different development characteristics and change trends, and they may be affected by the progression of the disease itself and the differences in different treatment methods. Through high-precision imaging technology and professional operation, it can be ensured that the first medical image obtained contains all important lesion information, so as to prepare for the subsequent in-depth analysis.

[0027] Exemplarily, in the medical image multi-scale extraction module 20, the first medical image is subjected to a medical image multi-scale feature scanning to obtain a medical image shallow feature coding feature map and a medical image deep feature coding feature map. It should be understood that medical images contain multi-scale information from macroscopic organ structures to microscopic tissue textures. For example, in a lung image, macroscopic scales such as the overall outline of the lungs and the division of lobes can be presented, and microscopic texture information such as alveolar morphology and tumor cell characteristics can also be observed. And these different scales of information are very important for accurately identifying lesions and judging the condition. Therefore, in the technical solution of the present application, in one embodiment, the medical image multi-scale extraction module 20 is used to: input the first medical image into a medical image multi-scale feature scanner to obtain the medical image shallow feature coding feature map and the medical image deep feature coding feature map. Among them, the medical image multi-scale feature scanner is a medical image multi-scale feature scanner based on a hollow pyramid network model.

[0028] Specifically, the first medical image is input into a medical image multi-scale feature scanner based on a hollow pyramid network model to expand the receptive field without reducing the image resolution, thereby capturing features at different scales, and obtaining a shallow feature encoding feature map of the medical image and a deep feature encoding feature map of the medical image. In particular, the hollow convolution is the core operation of the hollow pyramid network, which can increase the receptive field of the convolution kernel without increasing the number of parameters and the amount of calculation. This enables the network to obtain information in a wider area, thereby better understanding the relationship between different areas in the medical image. For example, when analyzing nodules in lung CT images, the hollow convolution can simultaneously consider the tissue information around the nodules, which helps to more accurately judge the nature of the nodules. By constructing a pyramid structure, the model can extract features at different levels, each level corresponding to a different scale. This hierarchical feature extraction method can adapt to the multi-scale characteristics of medical images, encode features at different scales separately, and provide rich materials for subsequent comprehensive analysis.

[0029] Exemplarily, in the medical image multi-scale feature perception module 30, the shallow feature coding feature map of the medical image and the deep feature coding feature map of the medical image are subjected to semantic information-guided deep and shallow feature joint perception to obtain the shallow-deep joint perception coding feature map of the medical image. It should be understood that considering that the shallow feature coding feature map of the medical image mainly reflects the more intuitive and local information in the image, such as detail features such as edges and textures. The deep feature coding feature map of the medical image focuses on extracting more abstract and more semantically connotative features. However, it is often difficult to grasp the semantic information of each region in the image as a whole based on these shallow features alone. Similarly, if only deep features are considered, it is easy to lose the detailed position information contained in some shallow features. Therefore, in order to make full use of the detail information advantages of shallow features and the semantic understanding advantages of deep features, so as to more comprehensively understand the features of medical images, the present application performs semantic information-guided deep and shallow feature joint perception on the shallow feature coding feature map of the medical image and the deep feature coding feature map of the medical image to obtain the shallow-deep joint perception coding feature map of the medical image. In this way, a semantic association bridge can be built between the shallow feature encoding feature map and the deep feature encoding feature map of the medical image, so as to accurately guide the fusion process of the two types of features based on the semantic information contained in the image, so that the fused features can more accurately and comprehensively reflect the actual situation of the image.

[0030] In one embodiment, Figure 3As shown, the medical image multi-scale feature perception module 30 includes: a medical image upsampling unit 31, used to upsample the medical image deep feature coding feature map to obtain an upsampled medical image deep feature coding feature map; a medical image deep feature field modulation unit 32, used to perform field modulation on the upsampled medical image deep feature coding feature map based on the medical image shallow feature coding feature map to obtain a medical image field modulated deep feature map; a medical image feature joint perception unit 33, used to perform feature joint perception of multiple attention structures on the medical image shallow feature coding feature map and the medical image field modulated deep feature map to obtain the medical image shallow-deep joint perception coding feature map.

[0031] Exemplarily, in the medical image upsampling unit 31, the medical image deep feature encoding feature map is upsampled to obtain an upsampled medical image deep feature encoding feature map. The upsampled medical image deep feature encoding feature map has the same size as the medical image shallow feature encoding feature map. Specifically, the process can be expressed as follows:

[0032] in, is the medical image deep feature encoding feature map, is the upsampling operation, It is the feature map of upsampled medical image deep feature encoding.

[0033] Specifically, when upsampling the feature map of the deep feature encoding of medical images, we are actually restoring some spatial details that may have been compressed or lost in the early network layers. This process is not just a simple increase in the resolution of the image, but more importantly, it attempts to reconstruct those fine-grained structural features, such as important visual clues such as the edges of lesions. By using methods such as transposed convolution (deconvolution) or interpolation, fine-grained structural features can be reconstructed, which is crucial for medical image processing.

[0034] In one embodiment, the medical image deep feature field modulation unit 32 is used to: after feature connection of the upsampled medical image deep feature encoding feature map and the medical image shallow feature encoding feature map, perform convolution kernel-based gated convolution semantic information field prediction on the obtained medical image shallow-deep feature connection encoding feature map to obtain the medical image feature semantic information field. Specifically, the process can be expressed as follows:

[0035] in, is the shallow feature encoding feature map of the medical image, is the feature connection operation, is a convolutional code with a convolution kernel of 3×3. is a convolutional code with a convolution kernel of 1×1, It is the semantic information field of medical image features.

[0036] The upsampled medical image deep feature encoding feature map is mapped to the medical image feature semantic information field to obtain the medical image field modulation deep feature map. Specifically, the process can be expressed by the formula:

[0037] in, It is the point multiplication by position. It is a deep feature map of field modulation in medical images.

[0038] Specifically, after the upsampled deep feature encoding feature map of medical images is feature-connected with the shallow feature encoding feature map, the resulting shallow-deep feature connection encoding feature map of medical images is fed into a convolution kernel-based gated convolution semantic information field predictor. This process is to construct a representation that can reflect the global and local semantic structure of the image. Through this semantic information field predictor, the model can adaptively adjust the importance of different feature channels to better capture the complex patterns in the image and provide rich contextual information for subsequent feature modulation. When this feature-connected and rich information-containing shallow-deep feature connection encoding feature map is predicted by the convolution kernel-based gated convolution semantic information field, it can not only understand the relationship between the various parts of the image, but also identify which areas are more critical to the current task (such as lesion detection). The final output is a medical image feature semantic information field. Next, the upsampled deep feature encoding feature map is mapped to this newly generated medical image feature semantic information field to form a medical image field modulation deep feature map. In this way, the truly important elements in the deep feature map can be guided and strengthened, while suppressing irrelevant information. In other words, the process of field modulation is to use the semantic information field to re-weight and upsample each element in the deep feature encoding feature map of medical images to make it more in line with the needs of actual scenarios. The result of this is to enhance the model's ability to understand specific semantic areas, allowing it to distinguish different tissue types or lesions at a finer scale.

[0039] Exemplarily, in the medical image feature joint perception unit 33, the medical image shallow feature encoding feature map and the medical image field modulation deep feature map are subjected to feature joint perception of multiple attention structures to obtain the medical image shallow-deep joint perception encoding feature map. Specifically, the process can be expressed by the formula:

[0040]

[0041]

[0042] in, is a deep point-by-point convolution operation, is the batch normalization operation, It is a medical image field modulation deep enhancement feature map. is the activation function, is the global average pooling operation, It is a shallow point-by-point convolution operation. It is a shallow enhanced feature map of medical images. Click on the location to add. It is the shallow-deep joint perceptual coding feature map of the medical image.

[0043] Specifically, the joint perception of features of the shallow feature encoding feature map of medical images and the field-modulated deep feature map of medical images with multiple attention structures aims to make full use of the rich detail information carried by the shallow features and the high-level semantic understanding provided by the deep features, and combine the advantages of the two in an intelligent way to form a more comprehensive and accurate feature representation, namely, the shallow-deep joint perception encoding feature map of medical images. When the shallow feature encoding feature map of medical images and the field-modulated deep feature map of medical images are sent to the multiple attention structure, it is actually simulating the way the human visual system processes information. In this process, the model will flexibly allocate attention resources according to the task requirements and automatically learn the feature combination that is most beneficial to the current task. For example, when identifying tiny nodules in lung CT images, the model may pay more attention to the local detail features that can distinguish the edges of the nodules; while when evaluating the overall size and shape of liver tumors, it may pay more attention to the deep features that reflect the global structure. The multiple attention mechanism allows the model to dynamically adjust its sensitivity according to the importance of different regions. Specifically, it can enhance or suppress the feature response at certain specific locations through a series of weight calculations. This mechanism not only takes into account changes at the level of individual pixels, but also involves contextual relationships in a larger range. For example, if an area is judged to be a possible lesion, the information of the surrounding adjacent areas will also be given a higher weight because they help to confirm the exact nature of the area. In this way, the shallow feature encoding feature map of medical images and the field-modulated deep feature map of medical images complement each other to form a multi-scale, multi-level feature representation system. The final generated shallow-deep joint perceptual encoding feature map of medical images not only contains rich low-level visual information (such as texture, boundaries, etc.), but also incorporates high-level semantic interpretations (such as tissue type, possibility of lesions, etc.). This enables the model to perform well in subsequent tasks, whether it is accurate target detection, segmentation or other complex computer vision analysis.

[0044] Exemplarily, in the medical image semantic segmentation module 40, the shallow-deep joint perceptual coding feature map of the medical image is semantically segmented to obtain the semantic segmentation result of the medical image. In one embodiment, the medical image semantic segmentation module is used to: input the shallow-deep joint perceptual coding feature map of the medical image into a semantic segmenter based on the Softmax function to obtain the semantic segmentation result of the medical image. It should be understood that, considering that in the medical image analysis scenario, each pixel in the image needs to be divided into different semantic categories, such as normal tissue, different types of lesions (such as nodules and inflammatory areas in lung images, cysts and tumors in liver images, etc.) and background categories. Based on this, the shallow-deep joint perceptual coding feature map of the medical image is semantically segmented by the semantic segmenter of the Softmax function to obtain the semantic segmentation result of the medical image. It should be understood that the Softmax function can calculate the probability of each pixel belonging to each category based on the input joint perceptual coding feature map, thereby realizing the classification of the medical image pixel level. For example, for a certain pixel point, the probability of belonging to normal liver tissue obtained by the Softmax function is 0.8, the probability of belonging to liver tumor is 0.15, and the probability of belonging to other categories is even lower. It can be more intuitively considered that the pixel is likely to belong to the normal liver tissue category.

[0045] Here, in the case where the medical image shallow feature encoding feature map and the medical image deep feature encoding feature map respectively represent image semantic features of different scales and depth dimensions of the first medical image, when performing feature joint perception based on semantic information field guided migration, the complexity of the semantic information field guided migration mechanism under the differences in image semantic feature scales and depths will lead to insufficient long-distance joint perception representation of the medical image shallow-deep joint perception encoding feature map, thereby reducing the expression effect of the medical image shallow-deep joint perception encoding feature map and affecting the accuracy of the semantic segmentation result of the medical image obtained by inputting the semantic segmentor based on the Softmax function.

[0046] Preferably, when the shallow-deep joint perceptual coding feature map of the medical image is input into a semantic segmenter based on a Softmax function to obtain a semantic segmentation result of the medical image, the shallow-deep joint perceptual coding feature map of the medical image is optimized, specifically comprising the steps of: Arrange the characteristic values ​​in the shallow-deep joint perceptual coding feature map of the medical image in ascending order to obtain a shallow-deep joint perceptual coding feature set of the medical image; In response to the first Eigenvalue With Eigenvalue The absolute value of the difference between them is less than or equal to the distance difference hyperparameter ,Right now , calculate the The weighted sum between the eigenvalue and the absolute value of the difference is the optimized Eigenvalue ,in, and represents different weighting hyperparameters; In response to the first The eigenvalue and The absolute value of the difference between the eigenvalues ​​is greater than the distance difference hyperparameter ,Right now : For each eigenvalue of the shallow-deep joint perceptual coding feature set of the medical image, the eigenvalue and the sum of its square are squared, and then all eigenvalues ​​of the shallow-deep joint perceptual coding feature set of the medical image are summed to obtain the multimodal inner product mapping value of the shallow-deep joint perceptual coding of the medical image :

[0047] The scale of the shallow-deep joint perceptual coding feature map of the medical image is It is equal to the width of the feature matrix of the shallow-deep joint perceptual coding feature map of the medical image multiplied by the height and then multiplied by the number of channels of the shallow-deep joint perceptual coding feature map of the medical image; The medical image shallow-deep joint perceptual coding multimodal inner product mapping value is further divided by the square of the scale of the medical image shallow-deep joint perceptual coding feature map to obtain the medical image shallow-deep joint perceptual coding high-dimensional basis value

[0048] The high-dimensional basis value of the shallow-deep joint perceptual coding of the medical image is multiplied by the After the eigenvalue, calculate the product with the first The weighted sum between the eigenvalues ​​is used to obtain the optimized first Eigenvalue ,in, and represents different weighting hyperparameters; The optimization of the shallow-deep joint perceptual coding feature set of the medical image is combined The eigenvalues ​​are used to obtain an optimized shallow-deep joint perceptual coding feature map of medical images, wherein the minimum eigenvalue of the shallow-deep joint perceptual coding feature set of the medical images remains unchanged.

[0049] That is, for the shallow-deep joint perception coding feature map of the medical image, under the condition of predetermined eigenvalue sequential distribution, due to the problem of low overall collaborative representation efficiency caused by the cross-region distance exceeding the local spacing limit parameter, a high-dimensional basis representation method of multimodal inner product mapping is adopted to parse the network interaction topology structure of the entire domain of eigenvalues.

[0050] In this way, by simulating the dynamic evolution process of multi-scale high-dimensional latent primitives and reconstructing the correlation representation pattern between feature dimensions, the encoding reconstruction of the shallow-deep joint perceptual coding feature map of medical images under the spatiotemporal continuous distribution characteristics is realized, thereby enhancing its multi-level feature fusion performance and improving the accuracy of the semantic segmentation results of medical images obtained by inputting the semantic segmentator based on the Softmax function.

[0051] Exemplarily, in the lesion area identification module 50, based on the semantic segmentation result of the medical image, the lesion area in the first medical image is identified to obtain the first lesion area identification result. It should be understood that, considering that the semantic segmentation result of the medical image divides different semantic categories from the data level, the medical image contains a variety of different tissues and structures, it is easy to confuse when actually viewing and analyzing it only by relying on the semantic segmentation category distinction, and it is difficult to quickly focus on the key lesion area. Therefore, in the technical solution of the present application, based on the semantic segmentation result of the medical image, the lesion area in the first medical image is identified to obtain the first lesion area identification result. That is, the lesion area is specially identified, which can clearly distinguish it from the normal tissue area and other non-lesion abnormal areas, improve the recognition, and help the user to more accurately focus on the lesion related information that needs to be analyzed. For example, in the lung CT image, the lesion area of ​​the lung nodule is identified, which can avoid confusion with the surrounding normal bronchial, blood vessel and other structures, and is more conducive to subsequent analysis and judgment.

[0052] In one embodiment, in the process of marking the lesion area in the first medical image based on the semantic segmentation result of the medical image, the entire segmentation result map will be traversed, and which areas should be marked as lesions will be determined according to a predefined threshold or rule set. For example, if most of the pixels in a continuous area are classified as "suspected malignant tumors", then this area will be regarded as an independent lesion and will be specially marked. At the same time, for those pixels with blurred boundaries but still with a high probability of belonging to the lesion category, the system may apply additional smoothing or other technical means to optimize the final lesion contour. In order to make these identifications more intuitive and easy to understand, in a specific embodiment, color coding can also be used to distinguish different lesion types. For example, red is used to indicate a highly suspected malignant tumor, green represents a nodule that may be benign, and yellow is used to indicate an area of ​​uncertainty. In addition, digital numbers or other forms of labels can be added to help doctors quickly locate and discuss specific lesions. All these visual enhancement measures are designed to enable clinical experts to more easily obtain key information from complex medical imaging data, thereby supporting their diagnostic decision-making process.

[0053] Exemplarily, in the lesion area morphological feature determination module 60, based on the semantic segmentation result of the medical image, the morphological features of each lesion area in the first lesion area identification result are determined. In one embodiment, the morphological features of each lesion area are the size, shape, boundary and density of each lesion area. It should be understood that, considering that many diseases have obvious characteristic differences in lesion morphology, these differences can serve as important diagnostic clues. For example, the lesion morphology of benign tumors is often more regular and has clear boundaries, while the lesions of malignant tumors are usually irregular in shape, with blurred boundaries and may have burr-like changes. Therefore, in order to more clearly and accurately determine the morphological features of each lesion area, in the technical solution of the present application, based on the semantic segmentation result of the medical image, the morphological features of each lesion area in the first lesion area identification result are determined. In this way, doctors can more accurately determine the specific type of disease suffered by patients. Taking breast diseases as an example, in breast mammography images, different diseases such as breast hyperplasia, breast fibroids or breast cancer can be distinguished based on the size of the mass or lesion, whether it is round or irregular in shape, whether the boundary is clear, and whether the density is uniform, thereby avoiding misdiagnosis and mistreatment and providing a reliable basis for subsequent targeted treatment.

[0054] In a specific embodiment, after obtaining the first lesion region identification result after semantic segmentation, a series of calculation and measurement operations are performed for each lesion region to obtain its morphological characteristics such as size, shape, boundary and density. Specifically: First, for the determination of size, the number of all pixels in the lesion region can be counted and converted into actual physical size (for example, the length unit is millimeter) according to the spatial resolution of the original CT image. In addition, the volume of the lesion can also be calculated, which is particularly useful for judging the growth rate of the tumor. For example, if the volume of a lesion increases significantly between two examinations, this may be a signal of worsening of the disease. Secondly, in terms of shape, mathematical morphology methods are used to describe the geometric characteristics of the lesion. For example, the circularity of the lesion can be calculated, which is defined as 4π×area / perimeter², and the closer the value is to 1, the more round the lesion is; the ellipse fitting error can also be calculated to measure the difference between the lesion and the ideal ellipse. Benign lesions usually have a more regular shape, while malignant tumors often show irregular and blurred edges. Next is the analysis of the boundary. This part focuses on the clarity and smoothness of the lesion edge. The system can detect and quantify the protrusions or depressions on the boundary, the so-called spiculation, which is an important sign to distinguish benign from malignant lesions. In addition, the gradient directional histogram (HOG) feature can be used to capture the texture changes at the boundary, so as to further refine the description of the boundary. Finally, the determination of density involves the analysis of the distribution of CT values ​​inside the lesion. Since different types of tissues have different absorption capabilities under X-rays, they will show different gray levels on CT images. The density characteristics inside the lesion can be described by histogram analysis, mean and standard deviation calculation. For example, solid tumors often show uniformly high density, while cystic lesions may show a lower and more dispersed CT value distribution. In addition, in some special cases, such as necrosis or calcification in the tumor, it will also cause abnormal increase or decrease in local density, which are important diagnostic clues. Combining the above four aspects of characteristics, a detailed report on the morphological characteristics of the lesion can be constructed. This report not only helps the analysis of the current case, but also provides an important reference for subsequent follow-up analysis.

[0055] In one embodiment, Figure 4As shown, the artificial intelligence-assisted multi-target imaging and tracking system based on lesion morphology also includes: a second medical image acquisition module 70, used to acquire a second medical image of the target patient object; a second medical image processing module 80, used to process the second medical image to obtain a second lesion area identification result and the morphological characteristics of each lesion area in the second lesion area identification result; a lesion area target tracking module 90, used to perform lesion area target tracking based on the morphological characteristics of each lesion area in the first lesion area identification result and the morphological characteristics of each lesion area in the second lesion area identification result.

[0056] Exemplarily, the second medical image acquisition module 70 is used to acquire the second medical image of the target patient object. It should be understood that when facing patients with multiple tumors or multiple lesions, if only relying on the first medical image initially acquired can only reflect the conditions of each lesion at a specific time point, and these lesions are in a dynamic change process. Based on this, in the technical solution of the present application, the second medical image of the target patient object is acquired, so that the changes that occur in different lesions over time can be captured, so as to fully grasp the dynamic changes of multiple lesions. In particular, the second medical image is acquired after a period of time from the first medical image.

[0057] Exemplarily, in the second medical image processing module 80, the second medical image is processed to obtain the second lesion area identification result and the morphological features of each lesion area in the second lesion area identification result. It should be understood that in order to more quickly and accurately determine the lesion area and detailed morphological features therein, and subsequently combine the various morphological features in the first medical image to perform comprehensive target tracking, the present application processes the second medical image to obtain the second lesion area identification result and the morphological features of each lesion area in the second lesion area identification result. In particular, the processing process is consistent with the processing process of the first medical image.

[0058] Exemplarily, the lesion area target tracking module 90 is used to track the lesion area target based on the morphological features of each lesion area in the first lesion area identification result and the morphological features of each lesion area in the second lesion area identification result. In other words, the impact of each lesion on the patient's body function and the response to subsequent treatment are independent and interrelated. For example, some lesions may show signs of improvement such as obvious shrinkage and morphological improvement at the beginning of treatment, while some lesions may continue to progress and deteriorate. These different changes jointly determine the patient's overall condition and prognosis. Therefore, target tracking based on the morphological features of each lesion area in the first and second lesion area identification results can accurately grasp the trend of the patient's overall condition to fully understand the dynamic changes of all lesions. In this way, it is possible to identify which lesions are the same (i.e., the same lesion) in two examinations and track the changes of these lesions. If the location, size or morphology of the lesion has changed significantly, the system can mark these changes to help doctors evaluate the progression of the disease or the effect of treatment.

[0059] In a specific embodiment, a chest CT scan image of a patient with multiple lung nodules is processed. The patient received a CT scan during the initial examination and was reexamined after a period of time (such as 6 months). The progression of the disease or the effect of treatment is evaluated by comparing the changes in lesions between the two scans. In this process, first, the initial CT scan image is processed using an artificial intelligence-assisted multi-target imaging tracking system based on lesion morphology to generate detailed semantic segmentation results. The system successfully identified the morphological features such as the location, size, shape, boundary and density of multiple lung nodules, and assigned a unique identifier to each nodule. This information is recorded as the first lesion area identification result. At the same time, for each identified nodule, the system calculates and saves its specific parameters, including but not limited to diameter, volume, circularity, edge smoothness, internal density distribution, etc., and establishes baseline data for subsequent comparative analysis. When the patient is reexamined, the same artificial intelligence system is applied again to the new CT scan image to obtain an updated semantic segmentation result and a new lesion area identification. The new lesion area is also assigned a unique identifier, and its morphological features are measured in detail. Next, the system compares the lesion area identification results in the initial examination with the results of the reexamination to find the corresponding relationship with similar position and morphological characteristics. For example, if a nodule in the initial examination finds a very close position in the reexamination, and its morphological characteristics are basically the same, it can be considered that this is a change in the same nodule. For newly appearing or disappearing nodules, the system will also specially mark them out to remind doctors to pay attention to the possible clinical significance of these changes. For those cases that are considered to belong to the same lesion, the system will further calculate the differences between them, such as how much the volume has increased, whether the shape has become more irregular, whether new protrusions have formed on the edge, etc. Especially for malignant tumors, rapid growth, irregular shape or burrs on the edge may be signs of worsening of the disease; on the contrary, if the nodule shrinks or remains stable, it may be a manifestation of effective treatment. Finally, the system will generate a detailed lesion area target tracking report, which contains all the change information of each lesion from the initial examination to the reexamination. This report not only helps doctors quickly grasp the development trend of the overall disease, but also provides specific analysis for each lesion, supporting more accurate diagnosis and the formulation of personalized treatment plans. For example, during the initial examination, the system found a benign nodule with a diameter of about 8 mm and a high degree of roundness. During the follow-up examination, the diameter of the nodule grew to 10 mm, and the roundness decreased slightly, but still remained within a reasonable range. Based on this change, the doctor may decide to continue observation without immediate surgical intervention. On the other hand, if another nodule that was originally small and regular suddenly increases in size and becomes extremely irregular in shape during the follow-up examination, this may indicate the need for further diagnostic testing or even consider the possibility of surgical resection.

[0060] In summary, the artificial intelligence-assisted multi-target imaging tracking system based on lesion morphology according to the embodiment of the present application is explained, which uses the image recognition and segmentation technology of artificial intelligence to implement multi-scale feature extraction on medical images, combines shallow and deep feature encoding, and performs a joint perception representation guided by semantic information, thereby intelligently generating semantic segmentation of images. Identify and analyze the morphology of the lesion area in the first medical image accordingly. Subsequently, the above process is repeated for another period of medical images of the same patient, and the morphological features of the lesion area in the two images are compared to achieve accurate tracking of the lesion area. The present application can simultaneously identify and analyze multiple lesions in the same image, so as to quickly and accurately mark all lesion areas, and capture more detailed information morphological features, reduce human errors, thereby overcoming the limitations of traditional image analysis methods, and providing a powerful auxiliary tool for clinicians.

[0061] Figure 5 FIG. 1 is a schematic flow chart of an artificial intelligence-assisted multi-target imaging tracking method based on lesion morphology according to an embodiment of the present application. Figure 5 As shown, the artificial intelligence-assisted multi-target imaging tracking method based on lesion morphology includes: step S1, obtaining a first medical image of a target patient object; step S2, performing a medical image multi-scale feature scanning on the first medical image to obtain a medical image shallow feature coding feature map and a medical image deep feature coding feature map; step S3, performing semantic information-guided deep and shallow feature joint perception on the medical image shallow feature coding feature map and the medical image deep feature coding feature map to obtain a medical image shallow-deep joint perception coding feature map; step S4, performing semantic segmentation on the medical image shallow-deep joint perception coding feature map to obtain a semantic segmentation result of the medical image; step S5, based on the semantic segmentation result of the medical image, marking the lesion area in the first medical image to obtain a first lesion area marking result; step S6, based on the semantic segmentation result of the medical image, determining the morphological characteristics of each lesion area in the first lesion area marking result.

[0062] Here, those skilled in the art will appreciate that the specific operations of each step in the above-mentioned artificial intelligence-assisted multi-target imaging tracking method based on lesion morphology have been described in the above reference. Figures 1 to 4 It has been introduced in detail in the description of the lesion morphology-based artificial intelligence-assisted multi-target imaging tracking system, and therefore, its repeated description will be omitted.

[0063] An embodiment of the present application further provides a computer program product, which includes a computer program code. When the computer program code runs on a computer, the computer implements the methods in the above embodiments of the present application.

[0064] An embodiment of the present application further provides a computer-readable storage medium, which stores computer instructions. When the computer instructions are executed on a computer, the computer implements the methods in the above embodiments of the present application.

[0065] An embodiment of the present application also provides a chip, including a circuit, for executing the methods in the above embodiments of the present application.

[0066] Those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working processes of the systems, devices and units described above can refer to the corresponding processes in the aforementioned method embodiments and will not be repeated here.

[0067] The prefixes such as "first" and "second" used in the embodiments of the present application are only used to distinguish different description objects, and have no limiting effect on the position, order, priority, quantity or content of the described objects. The use of prefixes such as ordinal numbers used to distinguish description objects in the embodiments of the present application does not constitute a limitation on the described objects. For the statement of the described objects, please refer to the description in the context of the claims or embodiments, and the use of such prefixes should not constitute an unnecessary limitation.

[0068] In the several embodiments provided in the present application, it should be understood that the disclosed systems, devices and methods can be implemented in other ways. For example, the device embodiments described above are only schematic. For example, the division of the units is only a logical function division. There may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms.

[0069] In the various embodiments of the present application, unless otherwise specified or logically conflicting, the terms and / or descriptions between the various embodiments are consistent and can be referenced to each other, and the technical features in different embodiments can be combined to form new embodiments according to their inherent logical relationships.

[0070] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0071] In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit.

[0072] The above is only a specific implementation of the present application, but the protection scope of the present application is not limited thereto. Any person skilled in the art who is familiar with the present technical field can easily think of changes or substitutions within the technical scope disclosed in the present application, which should be included in the protection scope of the present application. Therefore, the protection scope of the present application should be based on the protection scope of the claims.

Claims

1. An artificial intelligence-assisted multi-target imaging tracking system based on lesion morphology, characterized in that: include: A first medical image acquisition module, used to acquire a first medical image of a target patient; A medical image multi-scale extraction module, used for performing a medical image multi-scale feature scanning on the first medical image to obtain a medical image shallow feature coding feature map and a medical image deep feature coding feature map; A medical image multi-scale feature perception module, used for performing semantic information-guided deep-shallow feature joint perception on the medical image shallow feature encoding feature map and the medical image deep feature encoding feature map to obtain a medical image shallow-deep joint perception encoding feature map; A medical image semantic segmentation module, used for performing semantic segmentation on the shallow-deep joint perceptual coding feature map of the medical image to obtain a semantic segmentation result of the medical image; A lesion region identification module, used for identifying the lesion region in the first medical image based on the semantic segmentation result of the medical image to obtain a first lesion region identification result; A lesion area morphological feature determination module is used to determine the morphological features of each lesion area in the first lesion area identification result based on the semantic segmentation result of the medical image.

2. The artificial intelligence-assisted multi-target imaging tracking system based on lesion morphology according to claim 1 is characterized in that: The morphological characteristics of each lesion area are the size, shape, boundary and density of each lesion area.

3. The artificial intelligence-assisted multi-target imaging tracking system based on lesion morphology according to claim 2 is characterized in that: The medical image multi-scale extraction module is used to: input the first medical image into a medical image multi-scale feature scanner to obtain the medical image shallow feature encoding feature map and the medical image deep feature encoding feature map.

4. The artificial intelligence-assisted multi-target imaging tracking system based on lesion morphology according to claim 3 is characterized in that: The medical image multi-scale feature scanner is a medical image multi-scale feature scanner based on a hollow pyramid network model.

5. The artificial intelligence-assisted multi-target imaging tracking system based on lesion morphology according to claim 4 is characterized in that: The medical image multi-scale feature perception module includes: A medical image upsampling unit, used for upsampling the medical image deep feature encoding feature map to obtain an upsampled medical image deep feature encoding feature map; A medical image deep feature field modulation unit, configured to perform field modulation on the upsampled medical image deep feature encoding feature map based on the medical image shallow feature encoding feature map to obtain a medical image field modulated deep feature map; The medical image feature joint perception unit is used to perform feature joint perception of the medical image shallow feature encoding feature map and the medical image field modulation deep feature map with multiple attention structures to obtain the medical image shallow-deep joint perception encoding feature map.

6. The artificial intelligence-assisted multi-target imaging tracking system based on lesion morphology according to claim 5, characterized in that: The upsampled medical image deep feature encoding feature map and the medical image shallow feature encoding feature map have the same size.

7. The artificial intelligence-assisted multi-target imaging tracking system based on lesion morphology according to claim 6, characterized in that: The medical image deep feature field modulation unit is used to: After feature connection of the upsampled medical image deep feature encoding feature map and the medical image shallow feature encoding feature map, a convolution kernel-based gated convolution semantic information field prediction is performed on the obtained medical image shallow-deep feature connection encoding feature map to obtain a medical image feature semantic information field; The upsampled medical image deep feature encoding feature map is mapped to the medical image feature semantic information field to obtain the medical image field modulation deep feature map.

8. The artificial intelligence-assisted multi-target imaging tracking system based on lesion morphology according to claim 7, characterized in that: The medical image semantic segmentation module is used to: input the shallow-deep joint perceptual coding feature map of the medical image into a semantic segmenter based on the Softmax function to obtain the semantic segmentation result of the medical image.

9. The artificial intelligence-assisted multi-target imaging tracking system based on lesion morphology according to claim 1, characterized in that: Also includes: A second medical image acquisition module, used to acquire a second medical image of the target patient; A second medical image processing module, used for processing the second medical image to obtain a second lesion region identification result and morphological features of each lesion region in the second lesion region identification result; The lesion area target tracking module is used to track the lesion area target based on the morphological features of each lesion area in the first lesion area identification result and the morphological features of each lesion area in the second lesion area identification result.

Images