Image evaluation method and apparatus, readable storage medium and electronic device

Abstract

The application provides an image evaluation method and device, a readable storage medium and an electronic device, and relates to the technical field of image detection. The image evaluation method comprises the following steps: determining a to-be-evaluated medical image comprising a lesion area; determining a plurality of viewing angle positions corresponding to the lesion area; determining a plurality of point cloud features and a plurality of imageomic features corresponding to the lesion area based on the plurality of viewing angle positions corresponding to the lesion area, wherein the point cloud features are used to represent lesion shape information, and the imageomic features are used to represent texture information of the lesion and the anatomical tissue around the lesion; and determining an evaluation result corresponding to the to-be-evaluated medical image based on the plurality of point cloud features and the plurality of imageomic features. Further combining the point cloud features with the imageomic features is beneficial to evaluating the image from different angles and levels, integrating multi-angle information, avoiding one-sidedness and reducing errors, thereby helping doctors to more accurately diagnose diseases.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of image detection, in particular to an image evaluation method and device, a readable storage medium and an electronic device. BACKGROUND

[0002] At present, the screening of lung diseases and the like mainly relies on imaging methods, and the main imaging method is conventional computed tomography (CT) with a layer thickness of 5 mm. However, due to the large layer thickness and large pixel, the spatial resolution is low, which is not conducive to early detection and early diagnosis of diseases with strong concealment. The traditional technology performs target reconstruction on the CT image and performs imaging analysis to find reliable signs for judging diseases.

[0003] However, relying only on clinicians for diagnosis, on the one hand, the diagnosis efficiency is low, and on the other hand, only the diagnosis results of the doctors are taken as the final diagnosis results of the patients, which cannot guarantee the accuracy of the diagnosis. If misdiagnosis occurs, it will affect the timely treatment of patients. SUMMARY

[0004] In order to solve the above technical problems, the present application is proposed. The embodiments of the present application provide an image evaluation method and device, a readable storage medium and an electronic device.

[0005] In a first aspect, an embodiment of the present application provides an image evaluation method, which comprises: determining a to-be-evaluated medical image comprising a lesion area; determining a plurality of view positions corresponding to the lesion area; determining a plurality of point cloud features and a plurality of imageomic features corresponding to the lesion area based on the plurality of view positions corresponding to the lesion area, wherein the point cloud features are used to represent lesion shape information, and the imageomic features are used to represent texture information of the lesion and the surrounding anatomical tissues; and determining an evaluation result corresponding to the to-be-evaluated medical image based on the plurality of point cloud features and the plurality of imageomic features.

[0006] In combination with the first aspect, in some implementations of the first aspect, the determining of the plurality of point cloud features and the plurality of imageomic features corresponding to the lesion area based on the plurality of view positions corresponding to the lesion area comprises: determining a tissue region to which each of the plurality of view positions belongs; and determining the plurality of point cloud features and the plurality of imageomic features corresponding to the lesion area based on the tissue region to which each of the plurality of view positions belongs.

[0007] With reference to the first aspect, in some implementations of the first aspect, the lesion region is a lung lesion region, and the determining the plurality of point cloud features and the plurality of radiomics features corresponding to the lesion region based on the tissue region to which each of the plurality of view positions belongs comprises: for each of the plurality of view positions, if it is determined that the view position belongs to the lesion tissue, determining a radiomics feature corresponding to the lesion tissue; if it is determined that the view position belongs to the anatomical tissue, determining a radiomics feature corresponding to the view position in the anatomical tissue; and if it is determined that the view position belongs to the lung field region, determining a point cloud feature corresponding to the view position in the lung field region.

[0008] With reference to the first aspect, in some implementations of the first aspect, the determining the point cloud feature corresponding to the view position in the lung field region based on the target reconstruction result of the lung lesion region comprises: determining a target reconstruction result of the lung lesion region; and determining the point cloud feature corresponding to the view position in the lung field region based on the target reconstruction result of the lung lesion region.

[0009] With reference to the first aspect, in some implementations of the first aspect, the determining the point cloud feature corresponding to the view position in the lung field region based on the target reconstruction result of the lung lesion region comprises: determining a plurality of rays that can reach a surface of the target reconstruction result within a preset angle value range from the view position in the lung field region as a starting point; and determining the point cloud feature corresponding to the view position in the lung field region based on a length value of each of the plurality of rays.

[0010] With reference to the first aspect, in some implementations of the first aspect, the determining the radiomics feature corresponding to the view position in the anatomical tissue comprises: performing radiomics feature extraction on the to-be-evaluated medical image to determine the radiomics feature corresponding to the view position in the anatomical tissue; and / or, the determining the radiomics feature corresponding to the lesion tissue comprises: performing radiomics feature extraction on the lesion tissue to determine the radiomics feature corresponding to the lesion tissue.

[0011] With reference to the first aspect, in some implementations of the first aspect, the determining the evaluation result corresponding to the to-be-evaluated medical image based on the plurality of point cloud features and the plurality of radiomics features comprises: performing feature conversion on the plurality of point cloud features and the plurality of radiomics features to determine a classification item feature corresponding to the to-be-evaluated medical image, the classification item feature being used to represent a category feature of the lesion region in the to-be-evaluated medical image; and determining the evaluation result based on the classification item feature corresponding to the to-be-evaluated medical image.

[0012] With reference to the first aspect, in some implementations of the first aspect, the determining the plurality of view positions corresponding to the lesion region comprises: determining position and size information of the lesion region; and determining the plurality of view positions corresponding to the lesion region based on the position and size information of the lesion region.

[0013] In a second aspect, an embodiment of the present application provides an image evaluation device, which comprises: a first determination module configured to determine a medical image to be evaluated including a lesion region; a second determination module configured to determine a plurality of viewing angle positions corresponding to the lesion region; a third determination module configured to determine a plurality of point cloud features and a plurality of radiomics features corresponding to the lesion region based on the plurality of viewing angle positions corresponding to the lesion region, wherein the point cloud features are used to represent lesion shape information, and the radiomics features are used to represent texture information of the lesion and an anatomical tissue surrounding the lesion; and a fourth determination module configured to determine an evaluation result corresponding to the medical image to be evaluated based on the plurality of point cloud features and the plurality of radiomics features.

[0014] In a third aspect, an embodiment of the present application provides a computer readable storage medium, which stores a computer program, and the computer program is used to execute the method mentioned in the first aspect.

[0015] In a fourth aspect, an embodiment of the present application provides an electronic device, which comprises: a processor and a memory for storing processor-executable instructions; and the processor is configured to execute the method mentioned in the first aspect.

[0016] The image evaluation method provided by the embodiments of the present application determines a medical image to be evaluated including a lesion region, determines a plurality of viewing angle positions corresponding to the lesion region, determines a plurality of point cloud features and a plurality of radiomics features corresponding to the lesion region based on the plurality of viewing angle positions corresponding to the lesion region, and determines an evaluation result corresponding to the medical image to be evaluated based on the plurality of point cloud features and the plurality of radiomics features. The point cloud features can be used to focus on the change of the lesion shape, the radiomics features can reflect the change caused by the change of the lesion microenvironment on the image, and the point cloud features combined with the radiomics features can help evaluate the image from different angles and levels, integrate multi-angle information, avoid one-sidedness, reduce errors, and thus help doctors make more accurate disease diagnosis. BRIEF DESCRIPTION OF DRAWINGS

[0017] Figure 1 Fig. 1 shows a scene schematic diagram to which an embodiment of the present application is applicable.

[0018] Figure 2 Fig. 2 shows another scene schematic diagram to which an embodiment of the present application is applicable.

[0019] Figure 3 Fig. 3 shows a flowchart of an image evaluation method provided by an example embodiment of the present application.

[0020] Figure 4 Fig. 4 shows a flowchart of an image evaluation method provided by another example embodiment of the present application.

[0021] Figure 5aFig. 2 shows a flowchart of an image evaluation method according to another example embodiment of the present application.

[0022] Figure 5b Fig. 3 shows a schematic diagram of a viewing angle position in a lesion area according to an example embodiment of the present application.

[0023] Figure 6 Fig. 4 shows a flowchart of an image evaluation method according to another example embodiment of the present application.

[0024] Figure 7 Fig. 5 shows a flowchart of an image evaluation method according to another example embodiment of the present application.

[0025] Figure 8 Fig. 6 shows a flowchart of an image evaluation method according to another example embodiment of the present application.

[0026] Figure 9 Fig. 7 shows a flowchart of an image evaluation method according to another example embodiment of the present application.

[0027] Figure 10 Fig. 8 shows a structural diagram of an image evaluation device according to an example embodiment of the present application.

[0028] Figure 11 Fig. 9 shows a structural diagram of an electronic device according to an example embodiment of the present application. DETAILED DESCRIPTION

[0029] The technical solutions in the embodiments of the present application will be described clearly and completely below with reference to the drawings in the embodiments of the present application. Obviously, the described embodiments are only some of the embodiments of the present application, not all the embodiments.

[0030] Since the embodiments of the present application relate to the application of medical image feature extraction and deep learning, in order to facilitate understanding, the related terms and concepts related to deep learning that may be involved in the embodiments of the present application will be introduced simply below.

[0031] (1) CT: It is to use a precisely collimated X-ray beam, gamma ray, ultrasound, etc., together with a highly sensitive detector to make one after another cross-sectional scanning around a certain part of the human body, which has the characteristics of fast scanning time and clear image, and can be used for the examination of various diseases; according to the different rays used, it can be divided into X-ray CT (X-CT) and gamma ray CT (γ-CT), etc.

[0032] (2) Radiomics: A medical image analysis method that converts images into high-throughput features through pre-defined calculation methods for quantitative analysis. The concept of radiomics was first proposed by Lambin in 2012, which refers to extracting a large amount of image information from images (CT, MRI, PET, etc.) to realize tumor segmentation, feature extraction and model establishment, and to assist physicians in making the most accurate diagnosis by conducting deeper mining, prediction and analysis on massive data.

[0033] (3) Deep Learning (DL): Deep learning is a technique and research field of machine learning that realizes artificial intelligence in a computing system by establishing artificial neural networks (ANNs) with hierarchical structure. Since hierarchical ANNs can extract and filter input information layer by layer, deep learning has representation learning capabilities and can realize end-to-end supervised and unsupervised learning. The hierarchical ANNs used by deep learning have various forms, and the complexity of their hierarchy is commonly referred to as "depth". By construction type, the forms of deep learning include multi-layer perceptron, convolutional neural network, recurrent neural network, deep belief network, and other hybrid constructions. Deep learning uses data to update the parameters in its construction to achieve training goals, and this process is commonly referred to as "learning". The common method of learning is the gradient descent algorithm and its variants, and some statistical learning theory is used for optimization of the learning process. In terms of application, deep learning is used to learn high-dimensional data with complex structures and large samples, including computer vision, natural language processing, bioinformatics, automatic control, etc., and has achieved success in real-world problems such as face recognition, machine translation, and autonomous driving. Deep learning proposes a method for computers to automatically learn pattern features, and integrates feature learning into the model building process, thereby reducing the imperfections caused by human-designed features.

[0034] (4) Convolutional Neural Networks (CNN): Convolutional Neural Networks is a kind of feed forward neural networks containing convolution calculation and having deep structure, which is one of the representative algorithms of deep learning. Convolutional Neural Networks imitates the mechanism of biological visual perception to construct, which can be supervised learning and unsupervised learning. The convolution kernel parameter sharing in the hidden layer and the sparsity of interlayer connection make the convolutional neural network be able to learn the grid-like topology features such as pixels and audio with small amount of calculation, stable effect and no additional feature engineering requirements for data.

[0035] (5) Target reconstruction is a reconstruction technique used to improve the spatial resolution of one lung or a suspected area to achieve image magnification of the area. It is based on conventional spiral CT scanning, setting the conditions for target scanning, using original data for retrospective reconstruction, and according to the isotropic principle of 64-slice spiral CT, simple target reconstruction can achieve the same image effect as target scanning.

[0036] In the CT scan of lung occupying lesions, target reconstruction technology has the advantages of avoiding re-exposure, reducing tube wear, convenient and safe operation, obtaining more image information, etc. Therefore, target reconstruction technology can replace high-resolution CT in the application of lung nodules.

[0037] Visceral Pleural Invasion (VPI) is one of the important factors affecting the prognosis of lung cancer. For patients with peripheral non-small cell lung cancer without VPI, lung segment resection can be used instead of lung lobe resection, so as to protect lung function as much as possible without affecting survival rate. In addition, for surgically resected lung cancer, the presence of VPI significantly increases the probability of recurrence. The gold standard for judging VPI is pathological examination, but pathological examination is an invasive examination and time-consuming and laborious. CT is a routine examination method for lung cancer patients. Traditional technology proposes to perform target reconstruction on CT images and perform imaging analysis to find reliable signs to judge VPI. However, this method is easily affected by subjective judgment of doctors.

[0038] To solve the above technical problems, the embodiment of the present application provides an image evaluation method and device, a computer readable storage medium and an electronic device. The method uses point cloud features to achieve the purpose of focusing on the shape change of the lesion, uses imageomics features to reflect the changes caused by the change of the microenvironment of the lesion on the image, and further combines the point cloud features with the imageomics features, which is beneficial to evaluating the image from different angles and levels, comprehensively considering multi-angle information, avoiding one-sidedness, reducing errors, and thus helping doctors to more accurately diagnose diseases.

[0039] Exemplary system

[0040] Figure 1 As shown in the figure, the embodiment of the present application is applicable to a scene. As shown in the figure, Figure 1 As shown in the figure, the embodiment of the present application is applicable to a scene. As shown in the figure,

[0041] Specifically, the image acquisition device 2 is used to acquire the to-be-evaluated medical image corresponding to the subject and including the lesion area. The image acquisition device 2 can be a CT scanner, and the to-be-evaluated medical image can be a CT image sequence. The CT scanner is used to perform X-ray scanning on a human body part to obtain a CT image sequence corresponding to a human body lesion tissue organ. The image acquisition device 2 can also be an X-ray machine, a magnetic resonance imaging (MRI) device, or other devices with image acquisition functions, as long as it can acquire a to-be-evaluated medical image including a lesion area. The structure of the image acquisition device 2 is not limited in the present application.

[0042] The server 1 can be a server, a server group composed of multiple servers, a virtualization platform or a cloud computing service center. The type of the server 1 is not limited in the present application. The server 1 is used to acquire the to-be-evaluated medical image acquired by the image acquisition device 2, determine a plurality of perspective positions corresponding to the lesion area, determine a plurality of point cloud features and a plurality of imageomics features corresponding to the lesion area based on the plurality of perspective positions corresponding to the lesion area, wherein the point cloud features are used to represent the lesion shape information, and the imageomics features are used to represent the texture information of the lesion and the surrounding anatomical tissue, and determine the evaluation result corresponding to the to-be-evaluated medical image based on the plurality of point cloud features and the plurality of imageomics features. That is, the scene realizes an image evaluation method. Because Figure 1 As shown in the figure, the above scene realizes an image evaluation method by using the server 1, so that the scene can not only improve the adaptability of the scene, but also effectively reduce the computational complexity of the image acquisition device 2.

[0043] It should be noted that the present disclosure is also applicable to another scene.Figure 2 Another scene diagram to which the embodiments of the present application are applicable is shown. Specifically, the scene includes an image processing device 3, wherein the image processing device 3 includes an image acquisition module 31 and a calculation module 32, and there is a communication connection relationship between the image acquisition module 31 and the calculation module 32.

[0044] Specifically, the image acquisition module 31 in the image processing device 3 is configured to acquire a to-be-evaluated medical image corresponding to a subject, and the calculation module 32 in the image processing device 3 is configured to acquire the to-be-evaluated medical image including a lesion region; determine a plurality of viewing angle positions corresponding to the lesion region; determine a plurality of point cloud features and a plurality of imageomic features corresponding to the lesion region based on the plurality of viewing angle positions corresponding to the lesion region, wherein the point cloud features are used to represent lesion shape information, and the imageomic features are used to represent texture information of the lesion and the anatomical tissue around the lesion; and determine an evaluation result corresponding to the to-be-evaluated medical image based on the plurality of point cloud features and the plurality of imageomic features. That is, the scene implements an image evaluation method. Because Figure 2 The above scene uses the image processing device 3 to implement the image evaluation method, and does not need to perform data transmission operations with a server or other related devices, so that the above scene can ensure the real-time performance of the image evaluation method.

[0045] It should be understood that the embodiments of the present application can be applied to other scenes, such as a medical image reading system (MIRS) and the like. The application does not specifically limit the application scene of the image evaluation method.

[0046] Exemplary method

[0047] Figure 3 A flowchart of the image evaluation method provided by an example embodiment of the present application is shown. Specifically, as shown in Figure 3 The image evaluation method mentioned in the embodiments of the present application includes the following steps.

[0048] Step S310, a to-be-evaluated medical image including a lesion region is determined.

[0049] The to-be-evaluated medical image can be a computed tomography (CT) image, a magnetic resonance imaging (MRI) image, a computed radiography (CR) image, or a digital radiography (DR) image, and the present application does not specifically limit this. The image evaluation method provided by the embodiments of the present application can be applied to all medical images and has universality.

[0050] The embodiments of the present application do not limit the specific form of the medical image to be evaluated, which can be an original medical image, a preprocessed medical image, or a part of the original medical image, i.e., a part of the original medical image. In addition, the acquisition object corresponding to the medical image to be evaluated can be a human body or an animal body.

[0051] Exemplarily, the lesion region can include the following regions: a lung region, a mouth region, an esophagus region, a stomach region, an intestine region, a liver region, a gallbladder region, a pancreas region, a brain region, etc. The lesion region can be obtained by an expert delineating the region of interest in the medical image to be evaluated, which is not specifically limited in the present application.

[0052] Step S320, determining a plurality of view positions corresponding to the lesion region.

[0053] In some embodiments, the position and size information of the lesion region can be determined, and based on the position and size information of the lesion region, a plurality of view positions corresponding to the lesion region are determined. Specifically, after determining the lesion position in the lesion region, a cube containing the lesion can be determined according to the lesion center and the maximum diameter of the lesion. A position is taken on each corner, edge, and face of the cube as a view position, thereby obtaining 26 view positions, and adding the position of the lesion tissue itself, a total of 27 view positions. The number and position of the view positions are not specifically limited in the embodiments of the present application, and the view positions and number can be arbitrarily determined according to actual conditions, as long as they are reasonable.

[0054] Step S330, determining a plurality of point cloud features and a plurality of image features corresponding to the lesion region based on the plurality of view positions corresponding to the lesion region.

[0055] The point cloud feature is used to represent the lesion shape information. The point cloud feature can mimic the observation mode of a human expert and focus on the shape change of the lesion. The image feature is used to represent the texture information of the lesion and the surrounding anatomical tissue, and can reflect the changes caused by the changes in the tumor microenvironment on the image.

[0056] Step S340, determining an evaluation result corresponding to the medical image to be evaluated based on the plurality of point cloud features and the plurality of image features.

[0057] In some embodiments, a disease differential diagnosis model is constructed by training a transformer network in combination with the plurality of point cloud features and the plurality of image features, and the disease type is determined by using the disease differential diagnosis model, thereby determining the evaluation result corresponding to the medical image to be evaluated.

[0058] In some embodiments, multiple point cloud features and multiple radiomics features determined based on the lung cancer region are input into the transformer structure and combined with a self-attention mechanism to achieve non-invasive differential diagnosis of pleural invasion and determine the lung cancer assessment result corresponding to the medical image to be evaluated.

[0059] The image evaluation method provided in this application can use point cloud features to focus on changes in the shape of lesions, and use radiomics features to reflect the changes in the microenvironment of lesions on the images. Further combining point cloud features with radiomics features is beneficial for evaluating images from different angles and levels, integrating multi-angle information, avoiding one-sidedness, reducing errors, and thus helping doctors to make more accurate disease diagnoses.

[0060] Figure 4 The diagram shown is a flowchart illustrating an image evaluation method provided in another exemplary embodiment of this application. Specifically, in Figure 3 Extending from the illustrated embodiment Figure 4 The illustrated embodiment will be described in detail below. Figure 4 The differences between the embodiment shown in Figure 3 and the embodiment shown in Figure 4 are not repeated here.

[0061] like Figure 4 As shown, based on multiple viewpoints corresponding to the lesion area, multiple point cloud features and multiple radiomics features corresponding to the lesion area are determined (step S330), including the following steps.

[0062] Step S410: Determine the tissue region to which each of the multiple viewpoints belongs.

[0063] In some embodiments, for the lung cancer lesion area, the tissue regions to which the multiple viewpoints belong include at least one of the lesion tissue region, the anatomical tissue region, and the lung field region.

[0064] Step S420: Based on the tissue regions to which each of the multiple viewpoints belongs, determine multiple point cloud features and multiple radiomics features corresponding to the lesion region.

[0065] Specifically, the viewing positions can be divided into diseased and non-diseased areas based on the tissue regions to which they belong. Different features can be extracted and quantitatively analyzed based on the different types of tissue regions at multiple viewing positions.

[0066] The image evaluation method provided in this application adopts a multi-view analysis method based on the tissue regions to which multiple viewpoints belong. According to the different types of tissue regions to which multiple viewpoints belong, different features are extracted and quantitatively analyzed to achieve the purpose of determining multiple point cloud features and multiple radiomics features corresponding to the lesion region, thereby enabling a more comprehensive and accurate analysis of the lesion region.

[0067] Figure 5a The diagram shown is a flowchart illustrating an image evaluation method provided in another exemplary embodiment of this application. Specifically, in Figure 4 Extending from the illustrated embodiment Figure 5a The illustrated embodiment will be described in detail below. Figure 5a The illustrated embodiments and Figure 4 The differences between the embodiments shown are not repeated here, and the similarities are not repeated here.

[0068] like Figure 5a As shown, in the image evaluation method provided in this embodiment, the lesion area is the lung lesion area. Based on the tissue area to which each of the multiple view positions belongs, multiple point cloud features and multiple radiomics features corresponding to the lesion area are determined (step S420). For each of the multiple view positions, the following steps are performed.

[0069] Step S510: Determine the tissue region to which the viewing angle belongs. If the viewing angle is determined to belong to diseased tissue, proceed to step S520. If the viewing angle is determined to belong to anatomical tissue, proceed to step S530. If the viewing angle is determined to belong to the lung field region, proceed to step S540.

[0070] Step S520: Determine the radiomics characteristics corresponding to the lesion tissue.

[0071] Step S530: Determine the radiomics features corresponding to the viewpoint position within the anatomical tissue.

[0072] Step S540: Determine the point cloud features corresponding to the viewpoint position within the lung field region.

[0073] Figure 5b The diagram shown is a schematic representation of the viewpoint position within a lesion region provided in an exemplary embodiment of this application. Figure 5b As shown, for the lung cancer lesion region, a cube containing the lesion is determined based on the lesion center and its maximum diameter. A position is selected at each corner, edge, and face of this cube as a viewing position, plus the position of the lesion itself, resulting in a total of 27 viewing positions. Based on the actual anatomical location at each viewing position, these 27 viewing positions are divided into three categories, and features are extracted for each category. Viewing position 14 represents the lesion tissue, and features can be extracted from the entire lesion tissue using radiomics. Viewing positions 1-3, 4-6, 10-12, 15, and 19-21 represent anatomical tissues, such as the pleura, and radiomics features can be extracted from fixed-size cube regions (i.e., viewing positions within anatomical tissues) using radiomics. Viewing positions 7-9, 13, 16-18, and 22-27 represent regions within the lung field; for viewing positions within the lung field, point cloud features are extracted based on the target reconstruction results.

[0074] The image evaluation method provided by the embodiment of the present application can determine the radiomics features corresponding to the visual angle position in the anatomical tissue if it is determined that the visual angle position belongs to the anatomical tissue, determine the point cloud features corresponding to the visual angle position in the lung field region if it is determined that the visual angle position belongs to the lung field region, and has obvious advantages in evaluating the lesion nature, invasion range and adjacent relationship in the image.

[0075] Figure 6 The flowchart of the image evaluation method provided by another example embodiment of the present application is shown. Specifically, in the embodiment shown in the figure Figure 5a The embodiment shown in the figure is extended on the basis of Figure 6 The embodiment shown in the figure is extended on the basis of Figure 6 The embodiment shown in the figure is extended on the basis of Figure 5a The embodiment shown in the figure is extended on the basis of

[0076] As shown in the figure Figure 6 The image evaluation method mentioned in the embodiment of the present application determines the point cloud features corresponding to the visual angle position in the lung field region (step S540), which includes the following steps.

[0077] Step S610, determine the target reconstruction result of the lung lesion region.

[0078] Perform target reconstruction on the lung lesion region to obtain the target reconstruction result of the lung lesion region, and the target reconstruction result can represent the surface appearance of the lung lesion.

[0079] Step S620, determine the point cloud features corresponding to the visual angle position in the lung field region based on the target reconstruction result of the lung lesion region.

[0080] Specifically, for the visual angle position in the lung field region, the point cloud information based features can be constructed according to the target reconstruction result of the lung lesion region, that is, the point cloud features corresponding to the visual angle position in the lung field region.

[0081] The image evaluation method provided by the embodiment of the present application determines the target reconstruction result of the lung lesion region,

[0082] Based on the target reconstruction result of the lung lesion region, the point cloud features corresponding to the visual angle position in the lung field region can be determined, which can simulate the observation effect of human beings on the lesion appearance under different angles, visualize the analysis process of the lesion region, provide more diagnostic information that can be referred to for disease treatment, and also provide more reference data for future research on disease lesions.

[0083] Figure 7The diagram shown is a flowchart illustrating an image evaluation method provided in another exemplary embodiment of this application. Specifically, in Figure 6 Extending from the illustrated embodiment Figure 7 The illustrated embodiment will be described in detail below. Figure 7 The illustrated embodiments and Figure 6 The differences between the embodiments shown are not repeated here, and the similarities are not repeated here.

[0084] like Figure 7 As shown, the image evaluation method provided in this application embodiment determines the point cloud features corresponding to the viewpoint position within the lung field region based on the target reconstruction results of the lung lesion region (step S620), including the following steps.

[0085] Step S710: Determine multiple rays that can reach the surface of the target reconstruction result, starting from the viewpoint position within the lung field region and within the preset angle range.

[0086] Step S720: Based on the length values ​​of each of the multiple rays, determine the point cloud features corresponding to the viewpoint position within the lung field region.

[0087] Continue as Figure 5b As shown, taking viewpoint position 9 as an example, it represents a viewpoint position at the corner of the lesion area. Using viewpoint position 9 as the origin of the polar coordinate system, two angles can be used to represent the rays constructed from this point. Since viewpoint position 9 is located at the corner of the lesion area, the values ​​of its two angles range from 0 to 90°. The value space is evenly divided into N parts, and a ray is constructed for each angle. If the ray reaches the surface of the lesion target reconstruction result, the length of this ray is recorded as the feature value in this direction; if the ray does not reach the surface of the lesion target reconstruction result, the feature value in this direction is recorded as -1. Thus, N feature values ​​can be obtained. Based on these N feature values, N point cloud features corresponding to the viewpoint positions within the lung field region can be determined.

[0088] Taking viewpoint position 18 as an example, it represents a viewpoint position on the edge of the lesion area. Using viewpoint position 18 as the origin of the polar coordinate system, the values ​​of its two angles range from 0 to 90° and from 0 to 180°, respectively. Using the same method as above, N feature values ​​can be obtained. Taking viewpoint position 17 as an example, the values ​​of its two angles both range from 0 to 180°, and using the same method as above, N feature values ​​can also be obtained.

[0089] According to the above rules, each viewpoint within the lung field region can yield an N-dimensional point cloud feature, which mimics the human observation of the appearance of lesions from different angles.

[0090] The image evaluation method provided in this application determines multiple rays that can reach the surface of the target reconstruction result within a preset angle range, starting from the viewpoint position within the lung field region. Based on the length values ​​of each ray, the point cloud features corresponding to the viewpoint position within the lung field region are determined. By mimicking the human observation effect of the appearance of the lesion at different angles, the accuracy of lesion analysis is greatly improved by utilizing the features corresponding to each viewpoint position.

[0091] In some embodiments, determining the radiomics features corresponding to the viewpoint location within anatomical tissue includes: extracting radiomics features from the medical image to be evaluated to determine the radiomics features corresponding to the viewpoint location within anatomical tissue; and / or, determining the radiomics features corresponding to lesion tissue includes: extracting radiomics features from the lesion tissue to determine the radiomics features corresponding to the lesion tissue. By extracting texture information of lesion tissue and / or anatomical tissue through radiomics feature extraction, and focusing on the changes in the tumor microenvironment caused by changes in imaging, the accuracy of lesion tissue analysis in medical images can be improved, which has important guiding significance for the precise clinical treatment of lung cancer patients.

[0092] Figure 8 The diagram shown is a flowchart illustrating an image evaluation method provided in another exemplary embodiment of this application. Specifically, in Figure 3 Extending from the illustrated embodiment Figure 8 The illustrated embodiment will be described in detail below. Figure 8 The illustrated embodiments and Figure 3 The differences between the embodiments shown are not repeated here, and the similarities are not repeated here.

[0093] like Figure 8 As shown, the image evaluation method provided in this application embodiment determines the evaluation result corresponding to the medical image to be evaluated based on multiple point cloud features and multiple radiomics features (step S340), including the following steps.

[0094] Step S810: Perform feature transformation on multiple point cloud features and multiple radiomics features to determine the classification features corresponding to the medical image to be evaluated.

[0095] Among them, the classification feature is used to characterize the category characteristics of the lesion area in the medical image to be evaluated. For example, in the evaluation of lung cancer lesion area, if the output classification feature is 1, it means that the pleura is not invaded, and if the output classification feature is 0, it means that the pleura is invaded.

[0096] Step S820: Determine the evaluation result based on the classification features corresponding to the medical image to be evaluated.

[0097] Figure 9 The diagram shown is a flowchart illustrating an image evaluation method provided in another exemplary embodiment of this application. Figure 9As shown, a plurality of point cloud features and a plurality of image features corresponding to the lung cancer lesion area are input into the transformer structure, which includes N-dimensional point cloud features corresponding to each view position in the lung field area, M-dimensional image features corresponding to each view position in the anatomical tissue, and P-dimensional image features corresponding to the lesion tissue. For each type of feature, a different encoder (encoder A, encoder B, and encoder C) is used to convert the feature, and the unified feature length is T. Then, the position information is fused into each feature using position encoding. In addition, in order to realize classification, an additional classification item can be added, which can be a T-dimensional blank feature vector. The blank feature vector and other features are input into the transformer network, and the classification item feature is output through self-attention mechanism, which can predict whether there is pleural invasion. Through the transformer network combined with the self-attention mechanism, non-invasive differential diagnosis of pleural invasion is realized.

[0098] In some embodiments, the plurality of point cloud features and the plurality of image features are used as input features of a neural network model to determine disease classification features. Optionally, the neural network can be a convolutional neural network (CNN), a deep neural network (DNN), or a recurrent neural network (RNN), etc. The present application does not make specific limitations on this.

[0099] The image evaluation method provided by the embodiments of the present application converts a plurality of point cloud features and a plurality of image features to determine classification item features corresponding to the medical image to be evaluated, and the classification item features are used to represent the class features of the lesion area in the medical image to be evaluated. Based on the classification item features corresponding to the medical image to be evaluated, the evaluation result is determined, which provides practical value for clinical diagnosis and is beneficial to realize more efficient and standardized medical diagnosis.

[0100] The method embodiments of the present application are described in detail above in combination with Figures 1 to 9 , and the device embodiments of the present application are described in detail below in combination with Figure 10 and Figure 11 . It should be understood that the description of the method embodiments corresponds to the description of the device embodiments, and therefore, the parts not described in detail can be referred to the foregoing method embodiments.

[0101] Figure 10 As shown, the structure of the image evaluation device provided by an embodiment of the present application is shown. As shown in Figure 10As shown, the image evaluation device 1000 provided by the embodiments of the present application includes a first determination module 1001, a second determination module 1002, a third determination module 1003, and a fourth determination module 1004. The first determination module 1001 is configured to determine a to-be-evaluated medical image including a lesion region. The second determination module 1002 is configured to determine a plurality of view positions corresponding to the lesion region. The third determination module 1003 is configured to determine, based on the plurality of view positions corresponding to the lesion region, a plurality of point cloud features and a plurality of radiomics features corresponding to the lesion region, wherein the point cloud features are used to represent lesion shape information, and the radiomics features are used to represent texture information of the lesion and the anatomical tissue around the lesion. The fourth determination module 1004 is configured to determine, based on the plurality of point cloud features and the plurality of radiomics features, an evaluation result corresponding to the to-be-evaluated medical image.

[0102] In some embodiments, the third determination module 1003 is further configured to determine a tissue region to which each of the plurality of view positions belongs; and determine, based on the tissue region to which each of the plurality of view positions belongs, the plurality of point cloud features and the plurality of radiomics features corresponding to the lesion region.

[0103] In some embodiments, the third determination module 1003 is further configured to, for each of the plurality of view positions, if it is determined that the view position belongs to a lesion tissue, determine a radiomics feature corresponding to the lesion tissue; if it is determined that the view position belongs to an anatomical tissue, determine a radiomics feature corresponding to the view position in the anatomical tissue; and if it is determined that the view position belongs to a lung field region, determine a point cloud feature corresponding to the view position in the lung field region.

[0104] In some embodiments, the third determination module 1003 is further configured to determine a target reconstruction result of a lung lesion region; and determine, based on the target reconstruction result of the lung lesion region, the point cloud feature corresponding to the view position in the lung field region.

[0105] In some embodiments, the third determination module 1003 is further configured to determine a plurality of rays that can reach a surface of the target reconstruction result, starting from the view position in the lung field region and within a preset angle value range; and determine, based on a length value of each of the plurality of rays, the point cloud feature corresponding to the view position in the lung field region.

[0106] In some embodiments, the third determination module 1003 is further configured to perform radiomics feature extraction on the to-be-evaluated medical image to determine the radiomics feature corresponding to the view position in the anatomical tissue; and / or perform radiomics feature extraction on the lesion tissue to determine the radiomics feature corresponding to the lesion tissue.

[0107] In some embodiments, the fourth determining module 1004 is further configured to perform feature transformation on multiple point cloud features and multiple radiomics features to determine the classification feature corresponding to the medical image to be evaluated, wherein the classification feature is used to characterize the category feature of the lesion region in the medical image to be evaluated; and determine the evaluation result based on the classification feature corresponding to the medical image to be evaluated.

[0108] In some embodiments, the second determining module 1002 is further configured to determine the location and size information of the lesion area; and determine multiple viewpoint positions corresponding to the lesion area based on the location and size information of the lesion area.

[0109] Below, for reference Figure 11 This describes an electronic device according to embodiments of the present application. Figure 11 The diagram shown is a structural schematic of an electronic device provided in an embodiment of this application.

[0110] like Figure 11 As shown, the electronic device 1100 includes one or more processors 1101 and memory 1102.

[0111] The processor 1102 may be a central processing unit (CPU) or other form of processing unit with data processing capabilities and / or instruction execution capabilities, and may control other components in the electronic device 1100 to perform desired functions.

[0112] The memory 1102 may include one or more computer program products, which may include various forms of computer-readable storage media, such as volatile memory and / or non-volatile memory. Volatile memory may include, for example, random access memory (RAM) and / or cache memory. Non-volatile memory may include, for example, read-only memory (ROM), hard disk, flash memory, etc. One or more computer program instructions may be stored on the computer-readable storage medium, and the processor 1102 may execute the program instructions to implement the image evaluation methods of the various embodiments of this application mentioned above and / or other desired functions. Various content, such as medical images to be evaluated, may also be stored in the computer-readable storage medium.

[0113] In one example, the electronic device 1100 may also include an input device 1103 and an output device 1104, which are interconnected via a bus system and / or other forms of connection mechanism (not shown).

[0114] The input device 1103 may include, for example, a keyboard, a mouse, etc.

[0115] The output device 1104 can output various information including the evaluation result and the like to the outside. The output device 1104 can include, for example, a display, a speaker, a printer, a communication network and a remote output device connected thereto, and the like.

[0116] Of course, in order to simplify, Figure 11 Only some of the components in the electronic device 1100 related to the present application are shown in the figure, and components such as buses, input / output interfaces, and the like are omitted. In addition, the electronic device 1100 can include any other appropriate components according to the specific application.

[0117] In addition to the above method and device, the embodiments of the present application can also be a computer program product, which includes computer program instructions, which, when executed by a processor, cause the processor to perform the steps in the image evaluation method according to various embodiments of the present application described above in the specification.

[0118] The computer program product can be written in any combination of one or more programming languages, including an object-oriented programming language, such as Java, C++, and the like, and a conventional procedural programming language, such as the "C" language or the like. The program code can be executed entirely on the user computing device, partially on the user device, as a separate software package, partially on the user computing device and partially on a remote computing device, or entirely on a remote computing device or server.

[0119] In addition, the embodiments of the present application can also be a computer readable storage medium, which stores computer program instructions, which, when executed by a processor, cause the processor to perform the steps in the image evaluation method according to various embodiments of the present application described above in the specification.

[0120] The computer readable storage medium can take any combination of one or more readable media. The readable medium can be a readable signal medium or a readable storage medium. The readable storage medium may, for example, include but is not limited to an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or apparatus, or any combination of the above. More specific examples (non-exhaustive list) of readable storage media include: electrical connections having one or more wires, portable disks, hard disks, random access memories (RAM), read-only memories (ROM), erasable programmable read-only memories (EPROM or flash memory), optical fibers, portable compact disks read-only memories (CD-ROM), optical storage devices, magnetic storage devices, or any appropriate combination of the above.

[0121] The above describes the basic principles of the present application in combination with specific embodiments, but it needs to be pointed out that the advantages, benefits, effects and the like mentioned in the present application are only examples and are not limiting, and these advantages, benefits, effects and the like cannot be considered as necessary for each embodiment of the present application. In addition, the above specific details disclosed are only for the purpose of example and understanding, and are not limiting, and the above details do not limit the present application to be necessarily implemented with the above specific details.

[0122] The block diagrams of the devices, apparatuses, equipment, systems involved in the present application are only illustrative examples and are not intended to require or imply the connection, arrangement, configuration shown in the block diagrams. As those skilled in the art will recognize, these devices, apparatuses, equipment, systems can be connected, arranged, configured in any manner. Words such as "include", "contain", "have" and the like are open-ended words, mean "including but not limited to", and can be used interchangeably. The words "or" and "and" used herein mean the word "and / or", and can be used interchangeably unless the context clearly indicates otherwise. The word "such as" used herein means the phrase "such as but not limited to", and can be used interchangeably.

[0123] It also needs to be pointed out that in the devices, equipment and methods of the present application, each component or each step can be decomposed and / or recombined. These decompositions and / or recombinations should be considered as equivalent solutions of the present application.

[0124] The above description of the disclosed aspects is provided so that any person skilled in the art can make or use the present application. Various modifications to these aspects will be apparent to those skilled in the art, and the general principles defined herein can be applied to other aspects without departing from the scope of the present application. Therefore, the present application is not intended to be limited to the aspects shown herein, but is intended to be accorded the widest scope consistent with the principles and novel features disclosed herein.

[0125] The above description has been given for the purpose of illustration and description. Furthermore, this description is not intended to limit the embodiments of the present application to the forms disclosed herein. Although a number of example aspects and embodiments have been discussed above, those skilled in the art will recognize certain variations, modifications, changes, additions and sub-combinations thereof.

Claims

1. An image evaluation method, characterized in that, The method comprises: determining a medical image to be evaluated, the medical image comprising a lesion region, the lesion region being a lung lesion region; determining a plurality of view positions corresponding to the lesion region; based on the plurality of view positions corresponding to the lesion region, determining a plurality of point cloud features and a plurality of radiomics features corresponding to the lesion region, wherein the point cloud features are used to represent lesion shape information, and the radiomics features are used to represent texture information of the lesion and the anatomical tissue around the lesion; based on the plurality of point cloud features and the plurality of radiomics features, determining an evaluation result corresponding to the medical image to be evaluated; the method further comprises: determining a tissue region to which each of the plurality of view positions belongs; for each of the plurality of view positions, if it is determined that the view position belongs to a lesion tissue, determining a radiomics feature corresponding to the lesion tissue; if it is determined that the view position belongs to an anatomical tissue, determining a radiomics feature corresponding to the view position in the anatomical tissue; if it is determined that the view position belongs to a lung field region, determining a point cloud feature corresponding to the view position in the lung field region.

2. The image evaluation method according to claim 1, characterized in that the method further comprises: determining a target reconstruction result of the lung lesion region; based on the target reconstruction result of the lung lesion region, determining a point cloud feature corresponding to the view position in the lung field region.

3. The image evaluation method according to claim 2, characterized in that the method further comprises: starting from the view position in the lung field region, constructing a plurality of rays within a preset angle range; if the ray can reach the surface of the target reconstruction result, the length between the view position and the surface of the target reconstruction result is recorded as a feature value in the direction corresponding to the ray; if the ray cannot reach the surface of the target reconstruction result, the feature value in the direction corresponding to the ray is recorded as -1; based on the plurality of feature values, the point cloud feature corresponding to the view position in the lung field region is determined.

4. The image evaluation method of claim 1, wherein the method further comprises: performing radiomics feature extraction on the medical image to be evaluated to determine the radiomics feature corresponding to the view position in the anatomical tissue; and / or, the method further comprises: performing radiomics feature extraction on the lesion tissue to determine the radiomics feature corresponding to the lesion tissue.

5. The image evaluation method according to any one of claims 1 to 4, characterized in that, the method further comprises: performing feature conversion on the plurality of point cloud features and the plurality of radiomics features to determine a classification item feature corresponding to the medical image to be evaluated, the classification item feature being used to represent the category feature of the lesion region in the medical image to be evaluated. Determine the evaluation result based on the classification item features corresponding to the medical image to be evaluated.

6. The image evaluation method according to any one of claims 1 to 4, characterized in that, The determination of the multiple view positions corresponding to the lesion region comprises: Determine the position and size information of the lesion region. Determine the multiple view positions corresponding to the lesion region based on the position and size information of the lesion region.

7. An image evaluation apparatus, characterized by Comprise: The first determination module is used to determine a medical image to be evaluated comprising a lesion region, and the lesion region is a lung lesion region. The second determination module is used to determine the multiple view positions corresponding to the lesion region. The third determination module is used to determine the multiple point cloud features and multiple radiomics features corresponding to the lesion region based on the multiple view positions corresponding to the lesion region, wherein the point cloud features are used to represent the lesion shape information, and the radiomics features are used to represent the texture information of the lesion and the surrounding anatomical tissues. The fourth determination module is used to determine the evaluation result corresponding to the medical image to be evaluated based on the multiple point cloud features and the multiple radiomics features. The determination of the multiple point cloud features and the multiple radiomics features corresponding to the lesion region based on the multiple view positions corresponding to the lesion region comprises: Determine the tissue region to which each of the multiple view positions belongs. For each of the multiple view positions, If it is determined that the view position belongs to the lesion tissue, determine the radiomics features corresponding to the lesion tissue; If it is determined that the view position belongs to the anatomical tissue, determine the radiomics features corresponding to the view position in the anatomical tissue; If it is determined that the view position belongs to the lung field region, determine the point cloud features corresponding to the view position in the lung field region.

8. A computer-readable storage medium, characterized in that, The storage medium stores a computer program, and the computer program is used to execute the method in any one of claims 1 to 6.

9. An electronic device, comprising: The electronic device comprises: A processor; A memory for storing the processor-executable instructions; The processor is used to execute the method in any one of claims 1 to 6.

Images