Intelligent early screening system for lung cancer based on multi-modal data fusion

Abstract

The present application relates to the technical field of image recognition, and particularly relates to a lung cancer early intelligent screening system based on multi-modal data fusion. The present application firstly analyzes the change characteristics of lung parenchyma slices along a preset direction of accumulation and the position of a lung nodule region, and obtains a pathological density increment of the lung nodule region; then determines all blood vessel breakpoints in a preset peripheral region of the lung nodule region; analyzes a blood vessel physiological reference index of a lung parenchyma region, and further analyzes the connection cost between different blood vessel breakpoints to determine a pathological blood vessel truncation index of the lung nodule region; and then comprehensively outputs a risk reference index of the lung nodule region. The present application performs physical correction by means of the lung parenchyma change along the direction of accumulation, accurately quantifies the pathological density increment, further introduces normal blood vessels of the lung parenchyma as a reference, reduces the influence of the blood vessel truncation analysis caused by the lung nodule, accurately quantifies the pathological blood vessel truncation index, and then multi-modal pathological feature fusion is performed, thereby improving the accuracy of lung cancer early screening.

Description

Technical Field

[0001] This invention relates to the field of image recognition technology, and more specifically to an intelligent early screening system for lung cancer based on multimodal data fusion. Background Technology

[0002] Early lung cancer screening relies on thin-section chest CT imaging, but differentiating between benign and malignant sub-centimeter ground-glass nodules remains a challenge. In clinical practice, lung tissue is a soft tissue organ composed of a mixture of gas and fluid, and its internal density distribution is significantly affected by gravity. Under supine scanning protocols, due to gravitational deposition, the density of lung parenchyma naturally increases from the ventral to the dorsal side. Current screening methods lack a correction model for this gravitational gradient, leading to the misdiagnosis of high-density areas in the dorsal region caused by physiological deposition as pathological high-density consolidation.

[0003] Meanwhile, the pathological mechanism of invasive tumors involves fibrosis, which can stretch or sever surrounding blood vessels; while the vascular texture disorder caused by benign background lung diseases (such as COPD) is often naturally tortuous; existing technologies are mostly limited to multimodal morphological indicators such as the number of blood vessels or local curvature for early lung cancer screening, but lack feature analysis of the overall topological integrity of the microcirculation network, making it difficult to accurately distinguish whether blood vessels are pathologically damaged or physiologically tortuous in the case of complex background texture; the above factors together lead to poor accuracy in early lung cancer screening. Summary of the Invention

[0004] To address the technical problem of poor accuracy in early lung cancer screening, the present invention aims to provide an intelligent early lung cancer screening system based on multimodal data fusion. The specific technical solution adopted is as follows: A multimodal data fusion-based intelligent early lung cancer screening system, the system comprising: Lung image preprocessing module: used to extract the lung parenchyma region and lung nodule region from the lung CT 3D model, and obtain lung parenchyma slices of each anatomical layer in the preset deposition direction; Lung tissue density analysis module: used to obtain the pathological density increment of the lung nodule region based on the variation characteristics of the CT value of the lung parenchyma slice along the preset accumulation direction and the location distribution of the lung nodule region; Lung tissue vascular analysis module: used to determine the macroscopic blood supply vector based on the location distribution of the lung nodule region; to determine all vascular breakpoints within the preset outer region based on the vascular structure information of the lung nodule region; to obtain the vascular physiological reference index based on the vascular structure information of the lung parenchyma region; and to obtain the connection cost parameter between different vascular breakpoints by combining the connection vectors of different vascular breakpoints and their deviations relative to the macroscopic blood supply vector; and to obtain the pathological vascular truncation index of the lung nodule region based on the connection cost parameter between all different vascular breakpoints. Multimodal feature early warning module: used to output risk reference indicators for the lung nodule region based on the pathological density increment and the pathological vascular truncation index.

[0005] Furthermore, the method for obtaining the pathological density increment includes: Based on the variation characteristics of the CT values ​​of the lung parenchyma slices along the preset deposition direction, a density baseline function is fitted; The position of the geometric center of the lung nodule region in the preset deposition direction is projected into the density baseline function to obtain the predicted deposition density of the lung nodule region; the measured density is obtained based on the CT value of the lung nodule region. The pathological density increment of the pulmonary nodule region is obtained based on the deviation between the measured density and the predicted density of the effusion.

[0006] Furthermore, the method for obtaining the density baseline function includes: For each lung parenchyma slice, density parameters are obtained based on the distribution characteristics of CT values ​​of all voxels in the lung parenchyma slice; the linear fitting result of the density parameters of the lung parenchyma slice with the anatomical level is used as the density baseline function.

[0007] Furthermore, the method for obtaining the macroscopic blood supply vector includes: Obtain the hilar center in the three-dimensional lung CT model, and use the vector pointing from the hilar center to the geometric center of the lung nodule region as the macroscopic blood supply vector of the lung nodule region.

[0008] Furthermore, the method for obtaining the blood vessel breakpoint includes: The vascular breakpoints include upstream breakpoints and downstream breakpoints; all vascular segments within a preset outer region of the lung nodule region are extracted based on the Hessian matrix; for each vascular segment within the preset outer region, the vascular endpoint closest to the hilum center is taken as the upstream breakpoint, and the other vascular endpoint is taken as the downstream breakpoint.

[0009] Furthermore, the method for obtaining the vascular physiological reference index includes: The lung parenchyma region is divided to obtain all local lung parenchyma regions; based on the Hessian matrix, all vascular segments in each local lung parenchyma region are extracted, and each vascular segment is discretized into vascular micro-segments and its tangential vector is determined. The vascular physiological reference index is obtained based on the deviation of the magnitude of the sum of the tangential vectors of all vascular microsegments within each local lung parenchyma region from the sum of the magnitudes of the tangential vectors of all vascular microsegments.

[0010] Furthermore, obtaining vascular physiological reference indices includes: For each local lung parenchyma region, the magnitude of the sum of the tangential vectors of all vascular microsegments is used as the numerator, the sum of the magnitudes of the tangential vectors of all vascular microsegments is used as the denominator, and the ratio is used as the vascular straightness sub-parameter. The vascular physiological reference index is obtained by combining the vascular straightness sub-parameters of all the aforementioned local lung parenchyma regions.

[0011] Furthermore, the method for obtaining the connection cost parameter includes: Between each upstream breakpoint and each downstream breakpoint, a connection vector from the upstream breakpoint to the downstream breakpoint is obtained; a first connection cost parameter is determined based on the magnitude of the connection vector; a second connection cost parameter is obtained based on the minimum angle between the connection vector and the macroscopic blood supply vector, and the vascular physiological reference index. By combining the first connection cost parameter and the second connection cost parameter, the connection cost parameter between each upstream breakpoint and each downstream breakpoint is obtained.

[0012] Furthermore, the method for obtaining the pathological vessel truncation index includes: Based on the bipartite graph matching algorithm and the connection cost parameters between different vessel breakpoints, all matching pairs between the upstream breakpoint and the downstream breakpoint are obtained; by combining the connection cost parameters between the corresponding vessel breakpoints of all matching pairs, the pathological vessel truncation index is obtained.

[0013] Furthermore, the method for obtaining the risk reference indicators includes: The normalized value of the weighted fusion result of the pathological density increment and the pathological vascular truncation index is used as a risk reference indicator for the pulmonary nodule region.

[0014] The present invention has the following beneficial effects: This invention first extracts the lung parenchyma and pulmonary nodule regions from a three-dimensional lung CT model, providing a foundation for subsequent analysis. Then, it acquires lung parenchyma slices from each anatomical layer along a predetermined deposition direction, further analyzing the variation characteristics of CT values ​​along this direction to eliminate the influence of physiological deposition. Utilizing the location and distribution of pulmonary nodule regions, it accurately obtains the pathological density increment reflecting the abnormal increase in solid components due to cell proliferation within the nodule regions. It further determines the macroscopic blood supply vector and identifies all vascular breakpoints within a predetermined peripheral region of the pulmonary nodule region, preparing for subsequent analysis of pathological vascular truncation. Next, it analyzes the vascular structure information within the lung parenchyma region, obtaining a vascular physiological reference index characterizing the baseline curvature of healthy vessels to reduce sensitivity to the cost of reconnecting vascular breakpoints. Furthermore, it combines the connection vectors of different vascular breakpoints and their deviations relative to the macroscopic blood supply vector to analyze the connection cost parameters, thereby comprehensively evaluating the pathological vascular truncation index of the pulmonary nodule region. Finally, it performs multimodal fusion analysis of the pathological density increment and the pathological vascular truncation index to output a risk reference index for the pulmonary nodule region. This invention utilizes physical correction based on changes in lung parenchyma along the deposition direction to accurately quantify the pathological density increment in the lung nodule region. Furthermore, it introduces a background reference of normal blood vessels in the lung parenchyma to reduce the impact on vascular truncation analysis caused by lung nodules, accurately quantifying the pathological vascular truncation index in the lung nodule region. Subsequently, it performs multimodal pathological feature fusion, improving the accuracy of early lung cancer screening. Attached Figure Description

[0015] To more clearly illustrate the technical solutions and advantages in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0016] Figure 1 This is a block diagram of an early intelligent screening system for lung cancer based on multimodal data fusion, provided as an embodiment of the present invention. Detailed Implementation

[0017] To further illustrate the technical means and effects adopted by the present invention to achieve its intended purpose, the following, in conjunction with the accompanying drawings and preferred embodiments, details the specific implementation, structure, features, and effects of an intelligent early lung cancer screening system based on multimodal data fusion proposed according to the present invention. In the following description, different "one embodiment" or "another embodiment" do not necessarily refer to the same embodiment. Furthermore, specific features, structures, or characteristics in one or more embodiments can be combined in any suitable form.

[0018] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this invention pertains.

[0019] The following description, in conjunction with the accompanying drawings, details a specific scheme for an early intelligent screening system for lung cancer based on multimodal data fusion provided by this invention.

[0020] Please see Figure 1 The diagram shows a block diagram of an early intelligent screening system for lung cancer based on multimodal data fusion according to an embodiment of the present invention. The system includes a lung image preprocessing module 101, a lung tissue density analysis module 102, a lung tissue vascular analysis module 103, and a multimodal feature early warning module 104.

[0021] Lung image preprocessing module 101: used to extract the lung parenchyma region and lung nodule region from the lung CT three-dimensional model, and obtain lung parenchyma slices of each anatomical layer in the preset deposition direction.

[0022] In one embodiment of the present invention, the system reads chest CT image data from a single scan through the DICOM interface and constructs a three-dimensional voxel matrix. Each voxel in the three-dimensional voxel matrix stores the corresponding CT value (unit: HU). At the same time, the system reads in the whole lung region mask (i.e., the three-dimensional lung CT model) and the nodule region mask (lung nodule region) that have been generated in advance by segmentation networks such as U-Net.

[0023] To further identify the (healthy) lung parenchyma region in the 3D lung CT model, excluding the pulmonary nodule region: In order to eliminate interference from the density baseline caused by possible inflammatory exudates, ground-glass opacities, or focal atelectasis around the lesion (pulmonary nodule region), a suspected lesion mask covering the lesion and surrounding inflammatory region needs to be constructed. That is, a physical safety distance such as 10mm is set, and a 3D morphological expansion operation is performed on the pulmonary nodule region (expanding outward by 10mm) to generate an expanded mask covering the lesion and surrounding inflammatory region (i.e., the suspected lesion mask). Then, a difference operation is performed to subtract the suspected lesion mask from the whole lung region mask to obtain the (healthy) lung parenchyma region.

[0024] It should be noted that the lung CT 3D model, the lung parenchyma region, and the lung nodule region are all 3D models.

[0025] It should be noted that there may be multiple lung nodule regions in a three-dimensional lung CT model. The analysis method for the risk reference index of each lung nodule region is the same. This embodiment of the invention takes the presence of one lung nodule region as an example for analysis and description.

[0026] To further determine the physiological deposition direction caused by gravity under the supine scanning protocol, i.e. the preset deposition direction, a three-dimensional Cartesian coordinate system was constructed with coordinate parameters in mm. The y-axis was set as the anatomical gravity direction axis (corresponding to the anterior-posterior axis of the human body), with the positive direction of the y-axis pointing from the sternal side to the spinal side; the x-axis was the horizontal axis on the anatomical cross-section, usually pointing from the right rib side to the left rib side of the patient; and the z-axis was the anatomical cephalopelvic axis.

[0027] Then, along the preset accumulation direction (y-axis direction), the lung parenchyma region is uniformly divided into sections with a fixed step size to obtain lung parenchyma sections of several anatomical layers. The lung parenchyma sections refer to the slice plane perpendicular to the preset accumulation direction, arranged sequentially along the preset accumulation direction. The fixed step size ranges from 1 to 5 mm. In this example, 5 mm is used, which means that the thickness of each anatomical layer of lung parenchyma section is 5 mm.

[0028] Lung tissue density analysis module 102: used to obtain the pathological density increment of the lung nodule region based on the variation characteristics of the CT value of the lung parenchyma section along the preset accumulation direction and the location distribution of the lung nodule region.

[0029] Because of the natural gradient increase in lung parenchyma density from the ventral to the dorsal side under the supine scanning protocol due to gravity deposition, benign nodules (low density) on the dorsal side may be misjudged as high-density solid malignant nodules when superimposed on the high density of the background, while malignant nodules on the ventral side may be underestimated due to the low background density. To decouple the effects of gravity deposition from pathological solid changes and eliminate misjudgments caused by physiological gravity interference, this embodiment of the invention will use CT values ​​to assess lung tissue density. Then, based on the density variation characteristics of lung parenchyma sections along a preset deposition direction, the physiological changes of the natural gradient increase in lung parenchyma density will be analyzed. Furthermore, the location distribution of lung nodule regions along the preset deposition direction will be combined to assess the density baseline of lung nodule regions under the influence of natural deposition. Thus, the pathological density increment will be analyzed by combining the measured density of lung nodule regions.

[0030] The pathological density increment helps quantify density anomalies that cannot be explained by gravity, reflecting the increase in solid components caused by cell proliferation within the lesion, and providing a data basis for subsequent output of risk reference indicators.

[0031] Based on this, in a preferred embodiment of the present invention, the method for obtaining the pathological density increment includes: Based on the variation characteristics of CT values ​​of lung parenchyma sections along the preset deposition direction, a density baseline function is fitted; The position of the geometric center of the lung nodule region in the preset deposition direction is projected into the density baseline function to obtain the predicted deposition density of the lung nodule region; the measured density is obtained based on the CT value of the lung nodule region. The pathological density increment of the pulmonary nodule region is obtained based on the deviation between the measured density and the predicted density.

[0032] In a preferred embodiment of the present invention, the method for obtaining the density baseline function includes: For each lung parenchyma slice, density parameters are obtained based on the distribution characteristics of CT values ​​of all voxels in the lung parenchyma slice; the linear fitting result of the density parameters of the lung parenchyma slice with the anatomical level is used as the density baseline function.

[0033] As an example, taking any lung parenchyma slice k as an example, the distribution characteristics are represented by the mean value. The mean CT value of all voxels in lung parenchyma slice k is used as the density parameter Pk (unit: HU) of lung parenchyma slice k. The y-axis coordinate value of the set center, such as the centroid, of lung parenchyma slice k is used as the position number Yk. In the preset deposition direction, the lung parenchyma slices are sorted sequentially along the positive direction (y-axis), and the position number and density parameter are used as the feature parameters (Yk, Pk) of the lung parenchyma slice. Then, a series of feature parameters are input into the RANSAC algorithm solver for regression analysis. The number of iterations is 500-2000, which needs to be set based on the complexity of the lung anatomy and the number of feature parameters; in this example, 500 is used. Within the preset number of iterations, the solver randomly selects a subset of feature parameters to fit a linear model, calculates the distance from the remaining feature parameters to the fitted model, and removes outliers. After the iteration converges, the best-fit parameters are output. and And obtain the density baseline function; where the expression of the density baseline function is: ; is the gravity deposition gradient coefficient, in units of HU / mm, which characterizes the rate of increase in density; This represents the baseline physiological density of the ventral side, expressed in HU.

[0034] It should be noted that in other examples, implementers may also use linear regression analysis to fit the density baseline function.

[0035] The geometric center of the pulmonary nodule region, such as the centroid, is projected onto the preset deposition direction (y-axis coordinate value) and substituted into the density baseline function to calculate the predicted deposition density (unit: HU) of the pulmonary nodule region. The mean CT value of all gradients in the pulmonary nodule region is then used as the measured density (unit: HU). The difference between the measured density and the predicted deposition density is then normalized, for example, by first truncating negative values ​​(setting the part less than 0 to 0), and then dividing by the preset CT value of 1000HU as the pathological density increment of the pulmonary nodule region.

[0036] Lung tissue vascular analysis module 103: used to determine the macroscopic blood supply vector based on the location distribution of the lung nodule region; to determine all vascular breakpoints in the preset outer region based on the vascular structure information in the preset outer region of the lung nodule region; to obtain the vascular physiological reference index based on the vascular structure information in the lung parenchyma region; and to obtain the connection cost parameters between different vascular breakpoints by combining the connection vectors of different vascular breakpoints and their deviations from the macroscopic blood supply vector; and to obtain the pathological vascular truncation index of the lung nodule region based on the connection cost parameters between all different vascular breakpoints.

[0037] Because malignant tumors (invasive adenocarcinoma) are invasive, their growth process physically severs, engulfs, or pulls on the surrounding microvascular network, unlike benign lesions (such as inflammation or benign nodules) which only push and displace blood vessels. If a (malignant) nodule severs blood vessels, it will create a large break in space and angle. Forced repair and reconnection may require a very high physical cost. Therefore, from the perspective of connection cost, the abstract pathological damage can be quantified into a specific pathological vascular truncation index to help assess the risk reference index of the lung nodule area.

[0038] Furthermore, since CT images only provide morphological information of blood vessels and lack hemodynamic direction, it is impossible to determine whether the blood vessels flow towards or away from the nodules, and thus it is impossible to accurately construct repair and reconnection pathways to assess the connection cost; while the macroscopic anatomical pattern of pulmonary arteries and pulmonary veins in the lungs is mainly manifested as a centrifugal radial distribution, that is, the blood vessels enter the lung parenchyma from the center of the hilum, the blood vessels gradually branch and become thinner, and the overall direction is radial.

[0039] Based on this, the embodiments of the present invention first determine the macroscopic blood supply vector according to the location distribution of the pulmonary nodule region, providing a directional reference for subsequent vascular repair and reconnection to assess the connection cost; wherein, the macroscopic blood supply vector points from the center of the hilum to the pulmonary nodule region, characterizing the blood flow pattern.

[0040] Preferably, in one embodiment of the present invention, the method for obtaining the macroscopic blood supply vector includes: Obtain the hilar center in the 3D lung CT model, and use the vector pointing from the hilar center to the geometric center of the lung nodule region as the macroscopic blood supply vector of the lung nodule region.

[0041] As an example, the geometric centroid of the three-dimensional lung CT model is first taken as the hilum center, which can also be determined by the implementer based on a deep learning annotation model; then the pointing vector of the geometric center of the hilum center pointing to the lung nodule region, such as the centroid, is taken as the macroscopic blood supply vector of the lung nodule region.

[0042] The invasive growth of malignant tumors usually leads to the truncation of blood vessels at the edge of the lesion. Truncation of blood vessels located within the boundary of the nodule is most likely to directly reflect the invasiveness of the tumor. In order to eliminate the interference of normal background blood vessels far away from the lesion, the inventors of this invention first determine a preset peripheral region outside the lung nodule region, namely the region adjacent to the boundary of the nodule. Then, based on the vascular structure information in the preset peripheral region of the lung nodule region, all vascular breakpoints in the preset peripheral region are determined. This prepares for subsequent analysis of the reconnection cost of vascular breakpoints to quantify the pathological vascular truncation index that characterizes tumor invasiveness.

[0043] In one embodiment of the present invention, an expansion thickness of 2 mm is set, and a three-dimensional morphological expansion operation (expanding outward by 2 mm) is performed on the lung nodule region. The expanded region (excluding the lung nodule region) is used as the preset outer region of the lung nodule region. In other embodiments, the lung nodule region can be subtracted from the expansion mask processed by the lung image preprocessing module 101, and then used as the preset outer region of the lung nodule region. That is, the expansion thickness can be adjusted automatically, but it should not exceed the set physical safety distance.

[0044] Preferably, in one embodiment of the present invention, considering that by analyzing the correspondence between the eigenvalues ​​of the Hessian matrix and the geometric characteristics of the local structure, and combining multi-scale filtering and parameter optimization strategies, tubular or linear blood vessel features in a three-dimensional medical CT model can be effectively identified, thereby helping to determine blood vessel breakpoints; therefore, the method for obtaining blood vessel breakpoints includes: The vascular breakpoints include upstream breakpoints and downstream breakpoints; all vascular segments within a preset outer region of the lung nodule region are extracted based on the Hessian matrix; for each vascular segment within the preset outer region, the vascular endpoint closest to the hilum center is taken as the upstream breakpoint, and the other vascular endpoint is taken as the downstream breakpoint.

[0045] It should be noted that when extracting all vascular segments within the preset outer region of the lung nodule region, the implementer can also obtain vascular segments through other methods such as deep learning models (3D U-Net or VesselNet, etc.).

[0046] In another embodiment of the present invention, small blood vessel segments with a diameter of less than 1 mm can be removed, and the focus can be placed on thicker blood vessels to analyze truncation features and avoid introducing noise. Furthermore, the tangential vector of the blood vessel segment can be determined based on the Hessian matrix, and the upstream and downstream breakpoints can be determined by using the macroscopic blood supply vector. For each blood vessel segment, when the dot product of the tangential vector and the macroscopic blood supply vector is positive, the starting point of the tangential vector is taken as the upstream breakpoint; otherwise, the starting point of the tangential vector is taken as the downstream breakpoint.

[0047] Furthermore, age-related changes or chronic lung disease can cause the vascular texture throughout the lungs to exhibit natural tortuosity. This background noise can interfere with the judgment of the local vascular morphology of nodules. Therefore, this embodiment of the invention further obtains the vascular physiological reference index based on the vascular structure information in the lung parenchyma region. By analyzing the vascular structure morphology in the (healthy) lung parenchyma region, a background baseline of tortuous and disordered blood vessels is provided, thereby reducing the sensitivity to the tortuosity of blood vessels inside nodules, avoiding misjudgment and accurately quantifying the risk.

[0048] Preferably, in one embodiment of the present invention, considering the uneven distribution of pulmonary blood vessels, uniform sampling and averaging of the lung parenchyma region can eliminate statistical noise caused by local anatomical structures (such as bifurcation of large blood vessels), and more accurately obtain a general baseline reflecting the overall physiological state of the patient; further, the vascular segment of each local lung parenchyma region is determined, and the vascular segment is discretized into several micro-segments and the tangent vector is determined. The tangent vector can represent the instantaneous extension direction, thus preparing for subsequent analysis of vascular structure morphology and tortuosity; considering that the sum of the magnitudes of the tangent vectors of the vascular micro-segments can represent the total length of the blood vessel, and the magnitude of the sum of the tangent vectors of the vascular micro-segments can represent the net displacement of the blood vessel, when the tangent vectors of the vascular micro-segments in a straight blood vessel are accumulated, the directions do not cancel each other out, and the net displacement is approximately equal to the total length, while when the tangent vectors of the vascular micro-segments in a tortuous blood vessel are accumulated, the directions cut each other out, and the net displacement will be less than the total length; thus, this feature can be used to assess the tortuosity of the blood vessel; therefore, the method for obtaining the vascular physiological reference index includes: The lung parenchyma region is divided to obtain all local lung parenchyma regions; based on the Hessian matrix, all vascular segments in each local lung parenchyma region are extracted, and each vascular segment is discretized into vascular micro-segments and its tangential vector is determined. The vascular physiological reference index is obtained by calculating the deviation of the magnitude of the sum of the tangential vectors of all vascular microsegments within each local lung parenchyma region from the sum of the magnitudes of the tangential vectors of all vascular microsegments.

[0049] As an example, firstly, several sampling bodies with the same size as the lung nodule area are uniformly generated in the lung parenchyma region at preset intervals such as 20mm, i.e., local lung parenchyma regions, so as to measure the disorder of background vessels in the lung parenchyma region at the same physical scale as the lesion (lung nodule area); Further, all vascular segments within each local lung parenchyma region are obtained, and each vascular segment is then discretized into several tiny sub-segments, namely vascular microsegments. The length of each vascular microsegment is 1-2 mm, and 1 mm is used in this example. Further, the tangent vector of each voxel within each vascular microsegment is determined (when extracting vascular segments using the Hessian matrix, the eigenvector corresponding to the smallest eigenvalue can characterize the axial direction of the tubular structure in which the voxel is located). The tangent vector of each vascular microsegment is obtained by averaging the tangent vectors of all voxels within all vascular microsegments. In a preferred embodiment of the present invention, for each local lung parenchyma region, the magnitude of the sum of the tangential vectors of all vascular microsegments is used as the numerator, the sum of the magnitudes of the tangential vectors of all vascular microsegments is used as the denominator, and the ratio is used as the vascular straightness sub-parameter; by combining the vascular straightness sub-parameters of all local lung parenchyma regions, the vascular physiological reference index is obtained.

[0050] In each local lung parenchyma region, the closer the magnitude of the sum of the tangential vectors of all vascular microsegments is to the sum of the magnitudes of the tangential vectors of all vascular microsegments, the lower the tortuosity of the blood vessels in the local lung parenchyma region, the closer the ratio is to 1, and the larger the vascular straightness sub-parameter; among them, the maximum value of the vascular straightness sub-parameter is 1. Then, the straightness sub-parameters of blood vessels in all local lung parenchyma regions are averaged to obtain the vascular physiological reference index of the lung parenchyma region. The larger the vascular physiological reference index, the smaller the interference from the tortuous background of blood vessels, and the greater the reference value. Conversely, the tortuous background has a greater interference with the analysis of vascular structure in the lung nodule region.

[0051] It should be noted that the sum of the magnitudes of the tangential vectors of all vascular microsegments (denominator) is unlikely to be 0. When the denominator is 0, it means that there are no vascular segments in the local lung parenchyma region, and therefore they are not included in the analysis of the vascular physiological reference index. The vascular physiological reference index is a dimensionless parameter with a value range of 0-1.

[0052] After obtaining the vascular physiological reference index, the connection cost parameters between different vascular breakpoints can be obtained by combining the line vectors of different vascular breakpoints and their deviations relative to the macroscopic blood supply vector. The connection cost parameters comprehensively evaluate the cost of truncation and reconnection between breakpoints from the perspectives of the distance between the breakpoint lines and the deviation of the blood supply direction, thus preparing for the subsequent evaluation of the pathological vascular truncation index.

[0053] Preferably, in one embodiment of the present invention, considering that the reconnection of blood vessel breakpoints should be from upstream to downstream, all connection vectors can be obtained first; since the greater the distance between breakpoints, the greater the path reconnection cost, and the magnitude of the connection vector can characterize the reconnection distance between the upstream and downstream breakpoints, a first connection cost parameter can be obtained; considering that the larger the minimum angle between the connection vector and the macroscopic blood supply vector, the more the reconnection path deviates from the original blood vessel direction, and the greater the path reconnection cost, a second connection cost parameter can be obtained; and then, through comprehensive evaluation, the connection cost parameter between each upstream breakpoint and each downstream breakpoint can be obtained; the method for obtaining the connection cost parameter includes: Between each upstream breakpoint and each downstream breakpoint, obtain the connection vector from the upstream breakpoint to the downstream breakpoint; determine the first connection cost parameter based on the magnitude of the connection vector; obtain the second connection cost parameter based on the minimum angle between the connection vector and the macroscopic blood supply vector, and the vascular physiological reference index; fuse the first connection cost parameter and the second connection cost parameter to obtain the connection cost parameter between each upstream breakpoint and each downstream breakpoint.

[0054] As an example, taking the connection cost parameter between any upstream breakpoint and any downstream breakpoint as an example, the vector pointing from the upstream breakpoint to the downstream breakpoint is taken as the connection vector; the magnitude of the connection vector is further calculated, and the magnitude is normalized by dividing it by the maximum diameter of the preset outer region to obtain the first connection cost parameter. Then determine the minimum angle between the connecting vector and the macroscopic blood supply vector. Take the cosine value of the smallest included angle. This process transforms angle values ​​into easily quantifiable numerical values; the smaller the minimum included angle, the lower the reconnection cost. The cosine values ​​are then negatively correlated and normalized, specifically through... Negative correlation normalization is performed in this manner; Further integrate the vascular physiological reference index with Multiply to obtain the second connection cost parameter; apply background tortuosity penalty through vascular physiological reference index. The smaller the vascular physiological reference index, the greater the impact of background interference on the analysis of reconnection paths between upstream and downstream breakpoints. Therefore, it can be used as a weight to appropriately reduce the reconnection cost, thereby reducing the sensitivity to the curvature of reconnection paths between upstream and downstream breakpoints or deviation from the macroscopic blood supply direction. Finally, the first connection cost parameter and the second connection cost parameter are weighted and fused with weights of 0.6 and 0.4 respectively, and the weighted sum is used as the connection cost parameter between the upstream breakpoint and the downstream breakpoint. In other examples, the implementer can also adjust the weight values ​​according to the actual situation.

[0055] Furthermore, based on the connection cost parameters between all different vascular breakpoints, the pathological vascular truncation index of the pulmonary nodule region is obtained. The pathological vascular truncation index quantifies the vascular network truncation caused by fibrosis at the edge of the pulmonary nodule lesion, providing a data basis for the subsequent output of risk reference indicators.

[0056] Preferably, in one embodiment of the present invention, considering that bipartite graph matching forces the establishment of a one-to-one mechanical transmission chain, and virtual nodes can be used to handle breakage, the connection competition problem caused by the mismatch in the number of upstream and downstream vascular breakpoints can be effectively solved; the connection cost parameter between upstream and downstream vascular breakpoints can help evaluate the path reconnection cost; furthermore, upstream and downstream breakpoints can be matched to find the breakpoint reconnection scheme with the minimum total cost, thereby helping to obtain the pathological vascular truncation index, which helps quantify the pathological damage of pulmonary nodules to blood vessels; the method for obtaining the pathological vascular truncation index includes: Based on the bipartite graph matching algorithm and the connection cost parameters between different vessel breakpoints, all matching pairs between upstream and downstream breakpoints are obtained; the pathological vessel truncation index is obtained by combining the connection cost parameters between the corresponding vessel breakpoints of all matching pairs.

[0057] As an example, take the maximum value L between the number of upstream breakpoints and the number of downstream breakpoints, construct an L×L square matrix, and set the matrix elements in the square matrix to P (which must be greater than the theoretically calculated maximum connection cost parameter, with a value range of 2-10, and 2 in this example to represent extremely high impedance). Then, the connection cost parameters between each upstream breakpoint and each downstream breakpoint are mapped to the corresponding matrix positions for replacement. The first matrix element in the upper left corner represents the connection cost parameters between the upstream breakpoint with index 1 and the downstream breakpoint with index 1; the unreplaced matrix elements represent connection paths with extremely high connection costs. The Hungarian algorithm is further used to solve the square matrix to obtain all matching pairs between upstream and downstream breakpoints; the connection cost parameters between the corresponding vessel breakpoints of all matching pairs are averaged to obtain the pathological vessel truncation index; in other examples, the KM algorithm (Kuhn-Munkres Algorithm) can also be used to solve it.

[0058] Multimodal feature early warning module 104: used to output risk reference indicators for lung nodule regions based on pathological density increment and pathological vascular truncation index.

[0059] After obtaining the pathological density increment reflecting the space-occupying effect during tumor invasion and the pathological vascular truncation index reflecting the destructive effect during tumor invasion, the multimodal pathological feature data (pathological density increment and pathological vascular truncation index) can be further fused to assess and output risk reference indicators for pulmonary nodule regions.

[0060] It should be noted that the risk reference indicators are only used as objective data to characterize the possibility that the lung nodule area is an invasive tumor, and are used to provide relevant risk warnings for clinicians. The final clinical decision is made by the doctor in combination with other signs.

[0061] Preferably, in one embodiment of the present invention, the method for obtaining the risk reference indicator includes: The normalized value of the weighted fusion result of pathological density increment and pathological vascular truncation index is used as a risk reference indicator for pulmonary nodule regions.

[0062] As an example, the pathological density increment and the pathological vascular truncation index are weighted and summed, and the weighted sum is further normalized, for example, by mapping it to the sigmoid function, to obtain the risk reference index of the pulmonary nodule region. The weighting weights need to be obtained in advance through maximum likelihood estimation training on a feature dataset of historical confirmed cases (including the pathological gold standard); in this example, the weighting weights are 0.4 and 0.6, respectively. After determining the risk reference index, the risk level can also be determined based on preset clinical reference thresholds, such as setting a low-risk threshold of 0.3 and a high-risk threshold of 0.7; when the risk reference index is less than 0.3, a low-risk warning is issued; when the risk reference index is less than or equal to 0.3 and less than 0.7, a medium-risk warning is issued; when the risk reference index is greater than or equal to 0.7, a high-risk warning is issued.

[0063] It should be noted that the system provided in the above embodiments is only an example of the division of the above functional modules; in actual applications, the above functions can be assigned to different functional modules as needed, that is, the internal structure of the computer device can be divided into different functional modules to complete all or part of the functions described above.

[0064] In summary, this invention first extracts the lung parenchyma and pulmonary nodule regions from a three-dimensional lung CT model. It then analyzes the variation characteristics of lung parenchyma slices along a predetermined deposition direction and the location of pulmonary nodule regions to obtain the pathological density increment of the pulmonary nodule regions. Next, it identifies all vascular breakpoints within a predetermined peripheral region of the pulmonary nodule regions. The invention analyzes the vascular physiological reference index of the lung parenchyma and further analyzes the connection cost between different vascular breakpoints to determine the pathological vascular truncation index of the pulmonary nodule regions. Finally, it comprehensively outputs a risk reference index for the pulmonary nodule regions. This invention utilizes physical correction based on changes in lung parenchyma along the deposition direction to accurately quantify the pathological density increment of the pulmonary nodule regions. Furthermore, it introduces a background reference of normal lung parenchyma vessels to reduce the impact on vascular truncation analysis caused by pulmonary nodules, accurately quantifying the pathological vascular truncation index of the pulmonary nodule regions. Finally, it performs multimodal pathological feature fusion, improving the accuracy of early lung cancer screening.

[0065] It should be noted that the order of the above embodiments of the present invention is merely for descriptive purposes and does not represent the superiority or inferiority of the embodiments. The processes depicted in the accompanying drawings do not necessarily require a specific or sequential order to achieve the desired result. In some embodiments, multitasking and parallel processing are also possible or may be advantageous.

[0066] The various embodiments in this specification are described in a progressive manner. The same or similar parts between the various embodiments can be referred to each other. Each embodiment focuses on describing the differences from other embodiments.

Claims

1. A multimodal data fusion-based intelligent early lung cancer screening system, characterized in that, The system includes: Lung image preprocessing module: used to extract the lung parenchyma region and lung nodule region from the lung CT 3D model, and obtain lung parenchyma slices of each anatomical layer in the preset deposition direction; Lung tissue density analysis module: used to obtain the pathological density increment of the lung nodule region based on the variation characteristics of the CT value of the lung parenchyma slice along the preset accumulation direction and the location distribution of the lung nodule region; Lung tissue vascular analysis module: used to determine the macroscopic blood supply vector based on the location distribution of the lung nodule region; to determine all vascular breakpoints within the preset outer region based on the vascular structure information of the lung nodule region; to obtain the vascular physiological reference index based on the vascular structure information of the lung parenchyma region; and to obtain the connection cost parameter between different vascular breakpoints by combining the connection vectors of different vascular breakpoints and their deviations relative to the macroscopic blood supply vector; and to obtain the pathological vascular truncation index of the lung nodule region based on the connection cost parameter between all different vascular breakpoints. Multimodal feature early warning module: used to output risk reference indicators for the lung nodule region based on the pathological density increment and the pathological vascular truncation index.

2. The intelligent early lung cancer screening system based on multimodal data fusion according to claim 1, characterized in that, The method for obtaining the pathological density increment includes: Based on the variation characteristics of the CT values ​​of the lung parenchyma slices along the preset deposition direction, a density baseline function is fitted; The position of the geometric center of the lung nodule region in the preset accumulation direction is projected into the density baseline function to obtain the predicted accumulation density of the lung nodule region; the measured density is obtained based on the CT value of the lung nodule region. The pathological density increment of the pulmonary nodule region is obtained based on the deviation between the measured density and the predicted density of the effusion.

3. The intelligent early lung cancer screening system based on multimodal data fusion according to claim 2, characterized in that, The method for obtaining the density baseline function includes: For each lung parenchyma slice, density parameters are obtained based on the distribution characteristics of CT values ​​of all voxels in the lung parenchyma slice; the linear fitting result of the density parameters of the lung parenchyma slice with the anatomical level is used as the density baseline function.

4. The intelligent early lung cancer screening system based on multimodal data fusion according to claim 1, characterized in that, The method for obtaining the macroscopic blood supply vector includes: Obtain the hilar center in the three-dimensional lung CT model, and use the vector pointing from the hilar center to the geometric center of the lung nodule region as the macroscopic blood supply vector of the lung nodule region.

5. The intelligent early lung cancer screening system based on multimodal data fusion according to claim 1, characterized in that, The method for obtaining the blood vessel breakpoint includes: The vascular breakpoints include upstream breakpoints and downstream breakpoints; all vascular segments within a preset outer region of the lung nodule region are extracted based on the Hessian matrix; for each vascular segment within the preset outer region, the vascular endpoint closest to the hilum center is taken as the upstream breakpoint, and the other vascular endpoint is taken as the downstream breakpoint.

6. The intelligent early lung cancer screening system based on multimodal data fusion according to claim 1, characterized in that, The methods for obtaining the vascular physiological reference index include: The lung parenchyma region is divided to obtain all local lung parenchyma regions; based on the Hessian matrix, all vascular segments in each local lung parenchyma region are extracted, and each vascular segment is discretized into vascular micro-segments and its tangential vector is determined. The vascular physiological reference index is obtained based on the deviation of the magnitude of the sum of the tangential vectors of all vascular microsegments within each local lung parenchyma region from the sum of the magnitudes of the tangential vectors of all vascular microsegments.

7. The intelligent early lung cancer screening system based on multimodal data fusion according to claim 6, characterized in that, Obtaining vascular physiological reference indices includes: For each local lung parenchyma region, the magnitude of the sum of the tangential vectors of all vascular microsegments is used as the numerator, the sum of the magnitudes of the tangential vectors of all vascular microsegments is used as the denominator, and the ratio is used as the vascular straightness sub-parameter. The vascular physiological reference index is obtained by combining the vascular straightness sub-parameters of all the aforementioned local lung parenchyma regions.

8. The intelligent early lung cancer screening system based on multimodal data fusion according to claim 5, characterized in that, The method for obtaining the connection cost parameter includes: Between each upstream breakpoint and each downstream breakpoint, a connection vector from the upstream breakpoint to the downstream breakpoint is obtained; a first connection cost parameter is determined based on the magnitude of the connection vector; a second connection cost parameter is obtained based on the minimum angle between the connection vector and the macroscopic blood supply vector, and the vascular physiological reference index. By combining the first connection cost parameter and the second connection cost parameter, the connection cost parameter between each upstream breakpoint and each downstream breakpoint is obtained.

9. The intelligent early lung cancer screening system based on multimodal data fusion according to claim 8, characterized in that, The method for obtaining the pathological vessel truncation index includes: Based on the bipartite graph matching algorithm and the connection cost parameters between different vessel breakpoints, all matching pairs between the upstream breakpoint and the downstream breakpoint are obtained; by combining the connection cost parameters between the corresponding vessel breakpoints of all matching pairs, the pathological vessel truncation index is obtained.

10. The intelligent early lung cancer screening system based on multimodal data fusion according to claim 1, characterized in that, The methods for obtaining the risk reference indicators include: The normalized value of the weighted fusion result of the pathological density increment and the pathological vascular truncation index is used as a risk reference indicator for the pulmonary nodule region.

Images