Method and system for automatic analysis of medical image reports based on deep learning

Abstract

This invention provides a method and system for automatic analysis of medical image reports based on deep learning, belonging to the field of data processing technology. The method includes: performing cross-modal global cumulative superposition and calculation on a three-dimensional boundary change sequence and a text semantic distribution vector sequence to obtain an image-text association distribution matrix; performing an orthogonal decomposition family space transformation on the image-text association distribution matrix to extract the main components and obtain a core data vector set; based on the core data vector set, solving for the target coordinates of local gray-level abnormal regions in three-dimensional space and assigning text character indices to obtain a cross-modal space topological association set; and recombining the cross-modal space topological association set and the target coordinates to generate a medical image data analysis report. This invention effectively improves the efficiency of report generation and the systematic nature of data integration.

Description

Technical Field

[0001] This invention relates to the field of data processing technology, and in particular to a method and system for automatic analysis of medical image reports based on deep learning. Background Technology

[0002] In the process of medical image data processing, three core entities are mainly involved: medical image slice sequences, accompanying text character sets, and local gray-scale anomaly regions. Among them, the medical image slice sequences are continuously arranged along the slice stacking direction to form a three-dimensional scanning space, local gray-scale anomaly regions are distributed within this three-dimensional scanning space, and the accompanying text character sets are used to record descriptive information related to the local gray-scale anomaly regions. The three are bound together by spatial location and logical relationship. Existing technologies often have the following technical defects when processing the above data: when defining local gray-scale anomaly regions, most existing methods only delineate the boundaries of discrete two-dimensional slice layers independently, and often lack the ability to measure the continuous change of the spatial position of the boundary of the local gray-scale anomaly region along the slice stacking direction.

[0003] Taking ground-glass nodules in lung CT images as an example, these local gray-scale abnormal areas often exhibit a three-dimensional structure in which the size and shape gradually change along the stacking direction of the slices in a continuous CT slice sequence. Most existing processing methods often only independently identify one two-dimensional contour on a certain slice and another two-dimensional contour on an adjacent slice, and the boundary delineation between layers is often fragmented. Some processing methods may only focus on the pixel distribution within a single slice when extracting the boundary layer by layer, lacking point-by-point calculation of the continuous change of the boundary spatial position along the slice stacking direction. Since most methods fail to calculate the instantaneous rate of change of the spatial position of the ground-glass nodule boundary point by point along the slice stacking direction, when constructing the three-dimensional structure, it may often lead to morphological discontinuity in the boundary transition area between adjacent slices, which may not accurately reflect the continuous morphological changes of the ground-glass nodule in three-dimensional space, and thus may affect the accuracy of subsequent cross-modal data association. Summary of the Invention

[0004] The technical problem to be solved by the present invention is to provide a method and system for automatic analysis of medical image reports based on deep learning, which effectively improves the efficiency of report generation and the systematic nature of data integration.

[0005] To solve the above-mentioned technical problems, the technical solution of the present invention is as follows:

[0006] Firstly, a method for automatic analysis of medical image reports based on deep learning, the method comprising:

[0007] Step 1: Receive the medical image slice sequence and the accompanying text character set bound to the medical image slice sequence; perform grayscale normalization and denoising processing on the medical image slice sequence to generate a standardized pixel matrix sequence;

[0008] Step 2: Perform word segmentation and stop word filtering on the attached text character set to obtain a standard word sequence; perform density segmentation-based local gray-level anomaly region delimitation on the normalized pixel matrix sequence to obtain a continuous pixel cluster mask;

[0009] Step 3: Based on the continuous pixel cluster mask, calculate the instantaneous change rate of the spatial position of the boundary of the local gray-level anomaly region point by point along the slice stacking direction to obtain the three-dimensional boundary change sequence; perform semantic space mapping processing based on orthogonal decomposition family on the standard word sequence to obtain the text semantic distribution vector sequence containing semantic distribution information;

[0010] Step 4: Perform cross-modal global cumulative superposition and verification of the three-dimensional boundary change sequence and the text semantic distribution vector sequence to obtain the image-text association distribution matrix;

[0011] Step 5: Perform an orthogonal decomposition family space transformation on the image-text association distribution matrix to extract the main components and obtain the core data vector set;

[0012] Step 6: Based on the core data vector set, solve for the target coordinates of the local gray-scale anomaly region in three-dimensional space and assign text character indices to obtain the cross-modal space topological association set;

[0013] Step 7: Reorganize the cross-modal space topological association set and target coordinates to generate a medical image data analysis report.

[0014] Secondly, a deep learning-based automated medical image report analysis system includes:

[0015] The preprocessing module receives medical image slice sequences and associated text character sets bound to the medical image slice sequences; performs grayscale normalization and denoising on the medical image slice sequences to generate a standardized pixel matrix sequence.

[0016] The segmentation module is used to perform word segmentation and stop word filtering on the attached text character set to obtain a standard word sequence; and to perform density segmentation-based local gray-level anomaly region delimitation on the normalized pixel matrix sequence to obtain a continuous pixel cluster mask.

[0017] The calculation module is used to calculate the instantaneous rate of change of the spatial position of the boundary of the local gray-level anomaly region point by point along the slice stacking direction based on the continuous pixel cluster mask, and obtain the three-dimensional boundary change sequence; and to perform semantic space mapping processing based on orthogonal decomposition family on the standard word sequence to obtain the text semantic distribution vector sequence containing semantic distribution information.

[0018] The fusion module is used to perform cross-modal global cumulative superposition and calculation of the three-dimensional boundary change sequence and the text semantic distribution vector sequence to obtain the image-text association distribution matrix.

[0019] The extraction module is used to perform orthogonal decomposition family space transformation on the image-text association distribution matrix to extract the main components and obtain the core data vector set;

[0020] The allocation module is used to solve the target coordinates of local gray-level anomaly regions in three-dimensional space and allocate text character indices based on the core data vector set, so as to obtain a cross-modal space topological association set.

[0021] The output module is used to reorganize cross-modal space topological association sets and target coordinates to generate medical image data analysis reports.

[0022] Thirdly, a computing device includes:

[0023] One or more processors;

[0024] A storage device for storing one or more programs that, when executed by one or more processors, cause the one or more processors to implement the method.

[0025] Fourthly, a computer-readable storage medium storing a program that, when executed by a processor, implements the method.

[0026] The above-described solution of the present invention has at least the following beneficial effects:

[0027] By segmenting and filtering out redundant and meaningless characters through stop words, a normalized standard word sequence is obtained. Further, a co-occurrence matrix is ​​constructed based on word co-occurrence relationships, and text semantic space mapping is completed using orthogonal decomposition families, outputting a text semantic distribution vector sequence that can participate in numerical calculations. This process transforms unstructured natural language text, which cannot be directly computed, into structured feature vectors of a unified dimension, realizing the digital and quantitative representation of text semantic information and eliminating the data dimensionality difference between image pixel modalities and text character modalities. Through the outer product of feature vectors, a global cumulative superposition calculation is performed to construct an image-text association distribution matrix covering all boundary contour points and all text words, achieving the association matching of image dynamic spatial features and text semantic features. Simultaneously, covariance solving and singular value decomposition principal component extraction are performed on the association matrix to remove redundant interference components and invalid noise data within the matrix, effectively reducing matching errors caused by image artifacts and invalid text characters, and improving the anti-interference capability of the dual-modal data association results. Attached Figure Description

[0028] Figure 1This is a schematic diagram of the automatic analysis method for medical image reports based on deep learning provided in an embodiment of the present invention.

[0029] Figure 2 This is a schematic diagram of an automated medical image report analysis system based on deep learning, provided in an embodiment of the present invention. Detailed Implementation

[0030] Exemplary embodiments of the present disclosure will now be described in more detail with reference to the accompanying drawings. While exemplary embodiments of the present disclosure are shown in the drawings, it should be understood that the present disclosure may be implemented in various forms and should not be limited to the embodiments set forth herein. Rather, embodiments are provided so that this disclosure will be thorough and complete, and will fully convey the scope of the disclosure to those skilled in the art.

[0031] like Figure 1 As shown, embodiments of the present invention propose an automatic analysis method for medical image reports based on deep learning, the method comprising the following steps:

[0032] Step 1: Receive the medical image slice sequence and the accompanying text character set bound to the medical image slice sequence; perform grayscale normalization and denoising processing on the medical image slice sequence to generate a standardized pixel matrix sequence;

[0033] Step 2: Perform word segmentation and stop word filtering on the attached text character set to obtain a standard word sequence; perform density segmentation-based local gray-level anomaly region delimitation on the normalized pixel matrix sequence to obtain a continuous pixel cluster mask;

[0034] Step 3: Based on the continuous pixel cluster mask, calculate the instantaneous change rate of the spatial position of the boundary of the local gray-level anomaly region point by point along the slice stacking direction to obtain the three-dimensional boundary change sequence; perform semantic space mapping processing based on orthogonal decomposition family on the standard word sequence to obtain the text semantic distribution vector sequence containing semantic distribution information;

[0035] Step 4: Perform cross-modal global cumulative superposition and verification of the three-dimensional boundary change sequence and the text semantic distribution vector sequence to obtain the image-text association distribution matrix;

[0036] Step 5: Perform an orthogonal decomposition family space transformation on the image-text association distribution matrix to extract the main components and obtain the core data vector set;

[0037] Step 6: Based on the core data vector set, solve for the target coordinates of the local gray-scale anomaly region in three-dimensional space and assign text character indices to obtain the cross-modal space topological association set;

[0038] Step 7: Reorganize the cross-modal space topological association set and target coordinates to generate a medical image data analysis report.

[0039] In this embodiment of the invention, redundant and meaningless characters are removed through word segmentation and stop word filtering to obtain a normalized standard word sequence. Further, a co-occurrence matrix is ​​constructed based on word co-occurrence relationships, and text semantic space mapping is completed using orthogonal decomposition families, outputting a text semantic distribution vector sequence that can participate in numerical calculations. This process transforms unstructured natural language text that cannot be directly computed into structured feature vectors of a unified dimension, realizing the digital and quantitative representation of text semantic information and eliminating the data dimension difference between image pixel modalities and text character modalities. Global cumulative superposition calculation is completed through the feature vector outer product, constructing a graph-text association distribution matrix covering all boundary contour points and all text words, realizing the association matching of image dynamic spatial features and text semantic features. Simultaneously, covariance solving and singular value decomposition principal component extraction are performed on the association matrix to remove redundant interference components and invalid noise data within the matrix, effectively reducing matching errors caused by image artifacts and invalid text characters, and improving the anti-interference capability of the dual-modal data association results.

[0040] In a preferred embodiment of the present invention, step 1 involves receiving a medical image slice sequence and an accompanying text character set bound to the medical image slice sequence; performing grayscale normalization and denoising processing on the medical image slice sequence to generate a standardized pixel matrix sequence, specifically including: receiving two types of input data; the first type of input data is the medical image slice sequence, which is stored in a medical digital imaging and communication standard format, with each slice containing a 512-row × 512-column pixel matrix, and the grayscale value of each pixel ranging from 0 to 4095, representing the Heinz unit mapping of computed tomography scan values; the second type of input data is the accompanying text character set, which is stored in the form of a text string encoded in an eight-bit universal character set conversion format and bound to the medical image slice sequence through an examination number and a sequence number; reading all slices in the medical image slice sequence, arranging them sequentially from layer 1 to layer 300 according to the slice stacking direction, extracting the pixel matrix of each slice to obtain the original pixel matrix sequence; the original pixel matrix sequence in the first layer... Layer slice Line 1 The pixel value at the column position is recorded as the original grayscale value. ,in This is the slice number, with a value of [value]. ; The row number has a value of ; The column index has a value of .

[0041] Perform grayscale normalization on the original pixel matrix sequence by iterating through all pixel positions of all slices in the original pixel matrix sequence and counting all pixels. The minimum value among the values ​​is denoted as ; All statistics The maximum value among the values ​​is denoted as For each pixel position Use the original grayscale value of that pixel location. minus Get the difference, then divide the difference by and The difference between the two values ​​is the quotient, which is the normalized grayscale value of that pixel location, denoted as . Its value ranges from 0 to 1; all Arrange the pixels according to their original positions to obtain a normalized pixel matrix sequence; perform denoising processing on the normalized pixel matrix sequence using a Gaussian filtering algorithm, employing a weighted average of pixel neighborhoods to suppress image noise; and then, using the current pixel position to be processed... Centered on a pixel, a 5x5 square neighborhood window is taken around the pixel location. The position offset of each neighboring pixel within the window is denoted as . ,in Indicates the row direction offset, with a value of ; Indicates column offset, with a value of If the current pixel position is close to the slice boundary, causing some neighboring positions to exceed the valid range, then the neighboring positions exceeding the range are skipped, and only the neighboring positions within the range are used in the calculation. The determination of the Gaussian weight coefficients follows the Gaussian distribution law, with neighboring pixels closer to the window center having a larger weight, and neighboring pixels farther from the window center having a smaller weight. Specifically, the operation involves calculating the weight of each offset position within the window. The Euclidean distance from the center of the window is equal to the distance in the plane. The square of and The arithmetic square root of the sum of squares; the square of the distance divided by twice the preset standard deviation parameter. The square of, where The value is set to 2.0; the negative value of the natural exponential function is taken for the obtained quotient to obtain the unnormalized weight of the neighboring pixel position; the unnormalized weights of all neighboring pixel positions are summed to obtain the total weight, and then each unnormalized weight is divided by the total weight to obtain the normalized Gaussian weight coefficient. The sum of all weight coefficients is equal to 1.

[0042] Normalized grayscale value of each effective neighboring pixel position within the window Multiply by the corresponding Gaussian weight coefficient, then sum all the products; the sum is the denoised gray value, denoted as . Repeat the above denoising operation for each pixel position of each slice in the normalized pixel matrix sequence, and then... Arrange the pixels according to their original positions to obtain a normalized pixel matrix sequence.

[0043] This embodiment completes grayscale normalization and Gaussian filtering denoising preprocessing of image data, which can, to a certain extent, weaken interference problems such as uneven grayscale distribution, equipment imaging noise, and slice boundary artifacts that exist in the original CT images.

[0044] In a preferred embodiment of the present invention, step 2, performing word segmentation and stop word filtering on the accompanying text character set to obtain a standard word sequence; and performing density-based local gray-level anomaly region delimitation on the standardized pixel matrix sequence to obtain a continuous pixel cluster mask, may include:

[0045] Step 201: Based on the normalized pixel matrix sequence, extract the grayscale values ​​of each pixel within each slice to obtain the grayscale value distribution set of a single slice. Specifically, this includes: for the first pixel in the normalized pixel matrix sequence... Layer slices, in which Starting from 1 and going up to 300, iterate through all 512 rows × 512 columns of the slice, totaling 262,144 pixels, and record the grayscale value of each pixel. Extract the values ​​sequentially in row-major order. Row-major order means first extracting the first row and first column, then the first row and second column, and so on up to the first row and 512th column, then the second row and first column, and so on up to the 512th row and 512th column. Concatenate all 262,144 extracted grayscale values ​​end-to-end to form a one-dimensional array of length 262,144. This one-dimensional array is the [array name missing]. The grayscale value distribution of individual slices in the layer slices is obtained by performing the above traversal extraction operation on all 300 layer slices, resulting in 300 individual slice grayscale value distributions, which are then summarized to form a set of individual slice grayscale value distributions.

[0046] Step 202: Based on the single-slice grayscale value distribution set, calculate the difference between the grayscale value of each pixel and the average grayscale value of its eight neighboring pixels to obtain the local grayscale deviation value. Specifically, this includes: for the first pixel in the single-slice grayscale value distribution set... The first layer slice Line 1 For each pixel, determine its eight-neighbor location; the eight-neighbor location specifically includes the first pixel... Line 1 Column, No. Line 1 Column, No. Line 1 Column, No. Line 1 Column, No. Line 1 Column, No. Line 1 Column, No. Line 1 Column, No. Line 1 If a pixel is located at a slice boundary, causing a neighboring location to exceed the valid range, then that neighboring location is skipped, and only the neighboring locations within the valid range are retained for subsequent calculations; let the actual number of neighboring locations participating in the calculation be... The gray values ​​corresponding to each neighboring location within the effective range are... Extract the values, add them together to get the sum, and then divide the sum by 1 / 2. The resulting quotient is the average gray level of the eight neighboring pixels corresponding to that pixel. Subtracting the average gray level of the eight neighboring pixels from the pixel's own gray level and taking the absolute value of the difference is the local gray level deviation of that pixel, denoted as . Perform the above operation on each pixel within all slices of the single-slice grayscale value distribution set to obtain the corresponding pixel value. .

[0047] Step 203: Based on the local grayscale deviation value, pixels with local grayscale deviation values ​​greater than a preset density segmentation threshold are identified as grayscale anomalies, resulting in a set of grayscale anomalies. Based on this set, spatially adjacent grayscale anomalies are connected and merged into the same cluster to obtain a continuous pixel cluster mask. Specifically, this includes: performing grayscale anomaly detection, setting a preset density segmentation threshold, denoted as... In this embodiment The value was set to 0.15. This value was obtained by statistically analyzing the grayscale distribution of image samples from multiple cases of ground-glass nodules, based on the imaging characteristic of ground-glass nodules showing a slow grayscale transition between the nodules and the surrounding normal lung parenchyma in computed tomography (CT) images, while still exhibiting a discernible difference. Its local grayscale deviation value and Comparison: If Greater than If the pixel is an outlier, it is marked as a grayscale anomaly and its value is 1; if Less than or equal to If the pixel is not a grayscale anomaly, it is marked as 0; then, all pixel positions within the entire slice are traversed, and all pixel positions marked as 1 are identified. Collect them into a set to obtain a set of grayscale outliers. .

[0048] Construct a three-dimensional binary label space with the same three-dimensional dimensions as the original medical image slice sequence, where the values ​​of each dimension range from 0 to 1. From 1 to 300, From 1 to 512, The value of each voxel position in this marker space is determined as follows: if the three-dimensional coordinates of the voxel position belong to the set of gray-level anomalies... If it does not belong to, then the value is 1; if it does not belong to If the value is 0, then the value is 0; create a cluster identifier matrix of the same size as the label space, and set the initial value of each voxel position to 0, indicating that it has not yet been assigned to any cluster; set a cluster identifier counter variable. The initial value is 1; the label space is traversed voxel by voxel in the order of slice number from 1 to 300, row number from 1 to 512, and column number from 1 to 512; when a voxel position is found to have a value of 1 in the label space and still has a value of 0 in the cluster identifier matrix, the following seed filling operation is performed to create an empty queue to be processed. Add the current voxel position coordinates The rear of the line; from The head of the queue retrieves the coordinates of a voxel and assigns the corresponding position in the cluster identifier matrix a value. Search for the direct neighboring voxels of the voxel location along six spatial directions: slice number minus 1, slice number plus 1, row number minus 1, row number plus 1, column number minus 1, and column number plus 1. For each found neighboring voxel location, if the location is within the valid index range, has a value of 1 in the label space, and is still 0 in the cluster identifier matrix, then add the coordinates of the neighboring voxel location to the index. Tail of the queue. Repeat the operations of retrieving the coordinates of the head of the queue and searching for adjacent voxels until... Empty; at this point, all gray-level outliers that can be connected from the initial voxel through spatial adjacency in six directions have been grouped into the same cluster, and the cluster identifier is [cluster identifier]. ;Will The value is increased by 1, and a new identifier is assigned to the next cluster.

[0049] Continue traversing subsequent voxel positions in the label space, repeating the seed filling operation for each voxel position where the label space value is 1 and the cluster identifier matrix is ​​still 0; after all traversals are completed, the value of each voxel position in the cluster identifier matrix is ​​the identifier of the cluster to which that voxel belongs; in this embodiment, the ground glass nodule is one of the clusters with a cluster identifier equal to 1; the cluster identifier matrix is ​​the continuous pixel cluster mask, denoted as . .

[0050] Step 204: Segment the attached text character by character according to the attached text character set to obtain an initial character sequence; according to the initial character sequence, remove meaningless functional characters to obtain a standard token sequence, which specifically includes: The content of the attached text character set is the text description recorded during the film reading process, such as "A ground-glass opacity with a diameter of about 8 mm is visible in the apical segment of the upper lobe of the right lung, and the boundary is clear without burr sign"; Character-by-character segmentation means splitting the entire text string in the attached text character set into single characters. For Chinese text, split it into single Chinese characters; for English letters and numbers, split them into single letter or single digit characters. After splitting, arrange all the split characters in the order of their appearance in the original text to obtain an initial character sequence; each position in this initial character sequence stores a single split character, and its total length is the total number of all characters, denoted as ; The preset stop word list is a pre-compiled character list that收录了虚词字符 such as "的", "了", "在", "与", "及", "或", "所", "以", "而", "且", "之", "也", "乃", etc. that do not carry substantial semantic information in the medical image text description; Traverse each character in the initial character sequence and check one by one whether the character appears in ; If the current character belongs to , then delete the character from the sequence; If the current character does not belong to , then retain the character; Re-arrange all the retained characters tightly in their original order of appearance to obtain a standard token sequence, denoted as .

[0051] In this embodiment, by judging the abnormal area according to the local gray level difference, it can adapt to the imaging characteristics of slow gray level transition of ground-glass nodules, reduce the situation of mis-segmentation of normal lung parenchyma and missed segmentation of small lesion areas, and at the same time output a three-dimensional continuous pixel clustering cluster mask to completely retain the three-dimensional spatial connectivity relationship of the lesion. [[ID=id=17]]

[0052] In a preferred embodiment of the present invention, in step 3, according to the continuous pixel clustering cluster mask, measure the instantaneous change rate of the spatial position of the boundary of the local gray level abnormal area point by point along the slice stacking direction to obtain a three-dimensional boundary change sequence; Perform semantic space mapping processing based on the orthogonal decomposition family on the standard token sequence to obtain a text semantic distribution vector sequence containing semantic distribution information, which may include:

[0053] Step 301: According to the continuous pixel clustering cluster mask, extract the boundary contour points of the cluster along the slice stacking direction slice by slice to obtain the boundary contour point set of each slice, which specifically includes: For the cluster with cluster identification number 1 in It should be noted that the description of "收录了虚词字符" in the original text seems to be incomplete. It might be better to have a more accurate expression like "includes function words such as '的', '了', '在', '与', '及', '或', '所', '以', '而', '且', '之', '也', '乃', etc." for a more complete translation.Within a slice, the voxel with cluster identifier 1 constitutes a two-dimensional connected region on that slice layer, denoted as . ;right Each voxel position in Check whether its adjacent voxels in the four directly adjacent directions also belong to The four directly adjacent directions are, above, the first... Line 1 Column, directly below the Line 1 Column, leftmost Line 1 Column, Rightmost Line 1 If all adjacent voxels in all four directions are within the valid range and all belong to... Then the position of the voxel For internal points; if at least one of the four directions has a neighboring voxel outside the valid range or does not belong to the internal voxel. Then the position of the voxel For the boundary contour points; the first 3D spatial coordinates of all voxel positions identified as boundary contour points within a slice Collect them into a set, and get the first... The set of slice boundary contour points of a layer slice is denoted as Repeat the above judgment and collection operations layer by layer for all 300 slices to obtain a set of 300 slice boundary contour points. .

[0054] Step 302: Based on the boundary contour point set of each slice, calculate the three-dimensional spatial displacement vector between the boundary contour points corresponding to the spatial positions in two adjacent slices to obtain the boundary displacement vector set between slices. Specifically, this includes: for adjacent slices... Layer and first Layer slices, in which Values ​​range from 1 to 299, performing boundary contour point matching and displacement vector calculation; for The first in For each boundary contour point, its row number and column number are recorded as follows: and .exist The search row number falls within Within the interval and the column index falls within All candidate boundary contour points within the interval; among all candidate points satisfying this spatial proximity condition, calculate the squared planar distance between each candidate point and the boundary contour point in the two-dimensional plane formed by the row and column indices, i.e., the candidate point's row index minus... The square of the difference, plus the candidate point column index minus The square of the difference; select the candidate point that minimizes the square of the distance in the plane as the corresponding matching point; if If no candidate point satisfies the above spatial proximity condition, the boundary contour point will not participate in subsequent calculations in this layer. For a boundary contour point that successfully obtains a corresponding matching point, let the row number of the corresponding matching point be . Column number is Calculate its three-dimensional spatial displacement vector The three components are denoted as slice direction components. , line direction component Column direction components The calculation formula is: , , ;

[0055] in The unit is millimeters, and it is always equal to the slice spacing of 1.0 millimeter; and The units are all pixels; the three components are combined into a displacement vector. ;right The displacement vector is calculated for each of the successfully matched boundary contour points to obtain the first... The set of boundary displacement vectors between slices corresponding to the layer Repeat the above operation for all 299 pairs of adjacent slices to obtain 299 sets of boundary displacement vectors between slices. .

[0056] Step 303: Based on the inter-slice boundary displacement vector set, calculate the ratio of the magnitude of each displacement vector to the distance between adjacent slices to obtain the instantaneous rate of change of the spatial position of each boundary contour point, and summarize them to form a three-dimensional boundary change sequence; based on the standard lexical sequence, count the co-occurrence frequency of each lexical in the accompanying text character set and construct a lexical co-occurrence matrix, specifically including: receiving the inter-slice boundary displacement vector set. and standard lexical sequences ;for The Middle Layer displacement vectors Take its direction component Column direction components The magnitude of the displacement vector is calculated according to the standard definition of Euclidean distance in three-dimensional space, and the resulting instantaneous rate of change of spatial position is... The expression is: ;

[0057] in and The units are all pixels. The unit is per millimeter; the physical meaning of this rate is the change in position of the boundary contour point in three-dimensional space for every 1.0 millimeter advance along the slice stacking direction; if This indicates that the nodule boundary expands outward at that location; if This indicates that the nodule boundary narrows inward at that location; if This indicates that only translation occurs along the slice stacking direction; for Calculate each displacement vector one by one and according to slice number Numbered in ascending order from 1 to 299, and within the same slice, numbered according to the boundary contour points. Sort in ascending order, and select all The values ​​are sequentially connected end to end, and summed to form a three-dimensional boundary change sequence, denoted as ; The total length is denoted as , This equals the total number of successfully matched boundary profile points across all 299 layers; taking ground-glass nodules as an example, if each layer has an average of 40 successfully matched boundary profile points, then... .

[0058] Take the standard word sequence Its length is Each word element, traversed For each word element, add the first occurrence of that word element to a deduplication word element set. After the traversal is complete, Includes The number of all non-repeating words appearing in the text is denoted as . ;for Each lexical element in the sequence is assigned a number from 1 to 1. A unique sequence number is used to establish a bidirectional mapping table from lexical text to sequence number; a unique sequence number is constructed. Multiply The square matrix of columns is used as the word co-occurrence matrix, denoted as The initial value of all elements in the matrix is ​​set to 0; a sliding semantic window is set with a width of 5 consecutive words; the starting position of this window is set to... The 1 word element, currently covered by the window The The first to the second +4 lexical units, among which Values ​​range from 1 to From the five tokens in the window, extract any two tokens in different positions to form a co-occurrence token pair. Then, use a bidirectional mapping table to query the occurrences of these two tokens in the window. The corresponding serial numbers, let the two serial numbers be respectively and ;exist Execution in China: ;

[0059] The arrow in this formula It is an assignment operator, which in algorithm descriptions indicates that the result of the expression on the right is assigned to the variable on the left. This indicates the extraction of the word co-occurrence matrix. The Middle Line 1 The current value of a column element is incremented by 1 and then stored back into the column. Line 1 Column position; Similarly, the first Line 1 The value of each column element increases by 1; the combined meaning of the two equations is because of the word... and word elements They both appeared once in the current semantic window, so in the word co-occurrence matrix, the two corresponding positions ( OK List, OK The co-occurrence counts of each column are incremented once; among them for The Middle Line 1 The element values ​​of the column will slide the window from =1 slides backwards one by one until Each time, the window's starting number increases by 1; after the entire sliding process is complete, The value is The Middle The word element and the first The cumulative co-occurrence count of each word element across all semantic windows; matrix For a real symmetric square matrix, satisfying .

[0060] Step 304: Perform orthogonal decomposition on the word co-occurrence matrix and extract the feature vectors corresponding to a predetermined number of feature values ​​in descending order of feature values ​​to obtain the text semantic distribution vector sequence. Specifically, this includes: Orthogonal decomposition is the standard numerical calculation process for diagonalizing a real symmetric matrix in linear algebra. First, calculate... The eigenvalues, i.e., the solutions that make Subtract the eigenvalue multiplied by The value whose determinant of the first-order identity matrix is ​​zero; because Given a real symmetric matrix, the obtained... All eigenvalues ​​are real numbers; [The rest of the text is incomplete and cannot be translated.] The eigenvalues ​​are denoted as For each eigenvalue ( From 1 to Solve the homogeneous linear system of equations, with the coefficient matrix as follows: ,in for An identity matrix of order 1, where the unknown is a matrix of length 1. The column vectors are given, and the right-hand side of the equation is a vector of all zeros; solving this system of equations yields the corresponding... The eigenvectors are normalized to a unit length by calculating the square root of the sum of the squares of the eigenvector components, and then dividing each component by that square root. Since... For a real symmetric matrix, the eigenvectors corresponding to different eigenvalues ​​satisfy a pairwise orthogonal relationship, that is, the inner product of any two different eigenvectors is equal to zero; [The rest of the text appears to be incomplete and requires further context.] Arrange the eigenvalues ​​in descending order, such that the eigenvalue in the first position is the largest, denoted as . The second one is ranked second, and is recorded as Decrease sequentially to The corresponding eigenvectors are also rearranged in the same order. .

[0061] The preset number is 50, and the first 50 feature values ​​are extracted from the sorted feature value sequence. to And extract the feature vectors corresponding to each of these 50 feature values. to The dimension of each feature vector is Each of its components corresponds to The semantic load value of the corresponding word element in the semantic principal component direction is calculated. The larger the absolute value of the component, the more significant the contribution of the word element in that semantic direction. These 50 feature vectors are arranged in descending order of their corresponding feature values, and the resulting sequence is called the text semantic distribution vector sequence. .

[0062] In this embodiment, at the level of image feature extraction, the two-dimensional boundary contour points of the lesion are extracted layer by layer. The boundary displacement vector between slices is constructed through the adjacent slice neighbor point matching mechanism. The spatial position change rate of the lesion boundary along the stacking direction is further calculated. This can objectively quantify the expansion, contraction and translation change trends of the three-dimensional boundary of the ground-glass nodule, capture the subtle spatial deformation features of the lesion that are difficult to identify with the naked eye, and make up for the shortcoming that manual image reading can only qualitatively judge the lesion boundary and cannot quantify the dynamic changes of the boundary.

[0063] In a preferred embodiment of the present invention, step 4, which involves performing cross-modal global cumulative superposition and calculation on the three-dimensional boundary change sequence and the text semantic distribution vector sequence to obtain the image-text association distribution matrix, may include:

[0064] Step 401: Based on the 3D boundary change sequence, arrange the instantaneous change rates of each spatial location sequentially according to the slice order and the boundary contour point order to construct the boundary change feature vector, specifically including: The elements in the image have already been arranged in ascending order according to slice number and inner boundary contour point number in step 303, and step 401 directly uses this arrangement order; It can be considered as a one-dimensional vector, where each component corresponds to an instantaneous rate of change of spatial location. This one-dimensional vector is the boundary change characteristic vector, denoted as . .

[0065] Step 402: Directly use the text semantic distribution vector sequence as the text semantic feature vector; calculate the outer product of the boundary change feature vector and the text semantic feature vector based on the boundary change feature vector and the text semantic feature vector to obtain the image-text association distribution matrix. Specifically, this includes: based on the text semantic distribution vector sequence... and boundary change eigenvectors ; Contains 50 50-dimensional feature vectors First, concatenate the vectors end to end, then arrange them. of Each component, arranged in sequence of Each component, and so on up to of Each component has several parts. After concatenation, a one-dimensional vector is obtained, which is the text semantic feature vector, denoted as [vector]. ; Total dimension ;calculate and The outer product, which is a binary operation between two vectors, results in a matrix; let... Include Each component to , Include Each component to ;Build a Multiply A column matrix, denoted as ;matrix The Middle Line 1 The element value to be filled in the column position The calculation formula is: ;

[0066] in for The One portion, Values ; for The One portion, Values ;right and After filling in all possible combinations of values ​​element by element, the resulting matrix is... This is the image-text association distribution matrix; The physical meaning of an element is the instantaneous rate of change of the spatial position of a boundary contour point in the row where the element is located, and the column where the element is located corresponds to a certain text semantic feature component. The value of the element is the measure of the correlation strength between the two cross-modal features. A positive value indicates a positive association, while a negative value indicates a negative association.

[0067] This embodiment differs from simple feature splicing and fusion methods. The outer product operation can fully characterize the pairwise relationship between the microscopic deformation features of each lesion boundary and each type of text semantic feature. It can not only distinguish the positive promotion and negative inhibition relationship between two features, but also quantify the strength of the relationship between different feature combinations.

[0068] In a preferred embodiment of the present invention, step 5, performing an orthogonal decomposition family space transformation on the image-text association distribution matrix to extract the main components and obtain the core data vector set, may include:

[0069] Step 501: Calculate the covariance matrix of the image-text association distribution matrix to obtain the image-text covariance matrix; perform singular value decomposition on the image-text covariance matrix to obtain the left singular vector matrix, the singular value diagonal matrix, and the right singular vector matrix, specifically including: The dimension is Line × Column, calculation The mean of each column, for the Column, in which From 1 to All of the columns The sum of the values ​​of each element divided by The resulting quotient is the column mean of that column, denoted as . ;Will Subtract the column mean of each element from the matrix to obtain the mean-reduced matrix, denoted as . , of which Line 1 The element value of the column is ;calculate transpose and Multiplying itself, we get a Multiply The temporary square array of the column is denoted as ; The Middle Line 1 Column elements The calculation formula is: ;

[0070] in for The Middle Line 1 The element value of the column, i.e., the first... The instantaneous change rate characteristics of the boundary contour points in the first... Cross-modal association strength on each text semantic feature component; for The Middle Line 1 The element values ​​of the column; for The Middle Column mean of the column, for The Middle The column mean of the column; Indicates a temporary square formation The Middle Line 1 The element value at the column position, and All values ​​range from 1 to ,Will Divide each element in by The result Multiply The column matrix is ​​the text-image covariance matrix, denoted as... ; The Line 1 Column element representation The Middle Column data and the first Covariance between column data; the text-image covariance matrix As the matrix to be decomposed, singular value decomposition is performed; singular value decomposition is a standard method in linear algebra for decomposing a matrix into the product of three specific matrices. Multiply The real square matrix of columns Singular value decomposition represents it as: ;

[0071] Where the matrix It is a left singular vector matrix with dimension . Multiply List; Each column is a left singular vector, and each left singular vector is a column vector of unit length. Any two left singular vectors with different column numbers are orthogonal to each other. ;matrix It is a singular value diagonal matrix with dimension . Multiply List; Only the diagonal ones The nth element takes the value of a non-negative real number, and all elements outside the diagonal are zero; the nth element on the diagonal... Line 1 The elements of a column are denoted as , is called the first There are singular values, among which Values ​​range from 1 to .all The singular values ​​are arranged in descending order along the diagonal from top to bottom: ;

[0072] Singularity Size reflects In the Information intensity in each main direction, The larger the value, the more matrix variation information it carries in that direction; the matrix It is a right singular vector matrix with dimension . Line × List; Each column is a right singular vector, and all right singular vectors are pairwise orthogonal and of unit length. ; express The transpose matrix; the specific numerical calculation process of singular value decomposition is as follows: calculate , get one Given a real symmetric matrix; perform eigenvalue decomposition on this real symmetric matrix, the resulting eigenvalues ​​are the squares of the singular values, and the resulting eigenvectors are the right singular vectors; take the arithmetic square root of each eigenvalue to obtain the singular values. Then from the relational formula Find the left singular vectors one by one (when (time), among which For the first A right singular vector.

[0073] Step 502: Based on the singular value diagonal matrix, select a predetermined number of singular values ​​in descending order of singular values, and extract the left singular vectors corresponding to the selected singular values ​​in the left singular vector matrix to obtain the principal component vector set, specifically including: Arranged on the diagonal A singular value to The items have already been arranged from largest to smallest from top to bottom in step 501; the preset number is 30, and this value must meet the following conditions. conditions; from Take the first 30 elements from the diagonal in top-to-bottom order, that is... to These 30 singular values ​​are the largest selected singular values; each selected singular value exist The corresponding number in the middle The left singular vector of the column; extract sequentially From columns 1 to 30, we obtain 30 left singular vectors, denoted as... to The length of each left singular vector is . Arrange these 30 left singular vectors in descending order of their corresponding singular values, and summarize them to obtain the principal component vector group, denoted as . .

[0074] Step 503: Based on the principal component vector set, project the image-text association distribution matrix onto the subspace spanned by the principal component vector set to obtain the core data vector set. Specifically, the principal component vector set P consists of 30 K-dimensional unit principal component vectors. These 30 principal component vectors are pairwise orthogonal and linearly independent, together spanning a 30-dimensional linear subspace. The projection operation refers to the projection of the image-text association distribution matrix onto the subspace spanned by the principal component vector set P. Each line (this line is a) (3D row vector), and then connect the row vectors with the principal component vector groups respectively. The 30 principal component vectors in the image are subjected to inner product operations one by one. The result of each inner product operation is the projected coordinate value of that row in the direction of the corresponding principal component. After 30 inner product operations, the row is transformed from the original... One coordinate value is compressed into 30 projected coordinate values, that is, from the original The 3D space is compressed and mapped to a 30-dimensional subspace spanned by 30 principal component vectors. The first-ranked projected coordinate values ​​carry the main associated information components of that row vector, while the later-ranked noise components are automatically filtered out during this projection process. The process from the original... Compressing a 30-dimensional space to a 30-dimensional subspace means that the row originally used Number of values ​​(corresponding to) The description was initially based on the cross-modal correlation strength of each text semantic feature component, but after projection, it was re-described using 30 projected coordinate values, increasing the description dimension from... The dimension was reduced to 30, and the main correlation information was retained while secondary noise components were filtered out during the description process; the specific calculation method is as follows: for The Okay, among them Values ​​range from 1 to This behavior is a 3D row vector; this 3D row vectors and The inner product operation is performed on each of the 30 principal component vectors in the array. The specific process of the inner product operation is to perform the inner product operation on each of the row vectors. Each component and the principal component vector The sum of the product of the corresponding components at their respective positions is the projected coordinate value of that row along the direction of the principal component; line in Projected coordinate values ​​along the directions of each principal component The calculation formula is: ; in for The Line 1 Column elements, Principal Component Vector The One portion, The values ​​range from 1 to 30; the 30 projected coordinate values ​​obtained from the same row along the 30 principal component directions are arranged sequentially to form a 30-dimensional vector, denoted as . This 30-dimensional vector is the first... One core data vector; for All Perform the above projection operation on each row to obtain A 30-dimensional core data vector to These are aggregated to form the core data vector set.

[0075] This embodiment, based on covariance matrix construction and singular value decomposition, completes dimensionality reduction and noise reduction of the image-text association distribution matrix and extracts core association components, effectively improving the problems of high dimensionality, large data redundancy, and noise interference in the analysis results of cross-modal association matrices. By constructing the covariance matrix through mean removal processing, it accurately characterizes the data distribution correlation between different cross-modal association dimensions. Then, by using singular value decomposition to decompose the matrix information, it selects the principal components carrying the main association information based on the size of the singular values, and discards the minor components with extremely low proportions and mostly noise interference.

[0076] In a preferred embodiment of the present invention, step 6, which involves solving for the target coordinates of the local gray-level anomaly region in three-dimensional space and assigning text character indices based on the core data vector set to obtain a cross-modal space topological association set, may include: Step 601: Map each core data vector in the core data vector set back to the three-dimensional image space to obtain the coordinates of the local gray-level anomaly region in the three-dimensional space, denoted as the target coordinates; based on the core data vector set, retrieve the text semantic distribution vector sequence components corresponding to each core data vector in the image-text association distribution matrix, trace the original character positions of the standard words pointed to by the text semantic distribution vector sequence components in the accompanying text character set, and obtain the text character index, specifically including: based on the data correspondence established in steps 503 and 303, the first... Core data vectors Corresponding image-text correlation distribution matrix The OK, The Row corresponding boundary change feature vector The One portion, The Each component corresponds to a three-dimensional boundary change sequence. The The value that further corresponds to the first value. The first layer of slice One boundary contour point; slice number and boundary contour point numbering The restoration method involves accumulating the number of boundary contour points at each layer, and finding a sum that is not less than the sum of its initial values. slice number ,but equal Subtract before The cumulative number of boundary contour points of the layer; from step 301 slice boundary contour point set of layer slice Take out the first one Row number of each boundary contour point and column number The pixel coordinates are transformed to the patient's three-dimensional spatial coordinate system. During the transformation, spatial positioning parameters from medical digital imaging and communication standard formats are used to perform coordinate mapping. Specifically, this includes... Image position vector of layer slice , row direction cosine vector Column direction cosine vector pixel pitch The value is taken as 0.5 mm, and the slice spacing is taken as 1.0 mm; the calculation expression for the three coordinate axis components in the patient coordinate system is: ; ; ; in , , These are the three coordinate axis components in the patient's coordinate system, all in millimeters; These are the three components of the row direction cosine vector; These are the three components of the column direction cosine vector; The pixel spacing; the resulting coordinates That is, the first The target coordinates corresponding to each core data vector; for all Perform the above mapping operation on each of the core data vectors one by one to obtain... The coordinates of the nth target are summarized into a target coordinate set; for the nth target coordinate set... Core data vectors ,exist The All lines Among the elements, compare the absolute values ​​of each element and find the column index of the element with the largest absolute value, denoted as . ; Column number In text semantic feature vectors During the construction process, corresponding to A component within a certain feature vector; the method for determining the correspondence is as follows: Let... greater than or equal to The smallest integer of the given information is the corresponding text semantic distribution vector. The first in One, that is ;set up Then the corresponding The Each component corresponds to The Middle Each word element; through bidirectional mapping table query The text content of each word element is then used to construct a standard word element sequence from step 204. In the original character position mapping relationship preserved at the time, trace the start and end character indices of the word in the accompanying text character set; combine these two indices as the first... The text character index corresponding to each core data vector is denoted as . ; for all The above retrieval and tracing operations are performed on each of the core data vectors one by one to obtain... Text character index to This is summarized into a text character index set. .

[0077] Step 602 involves pairing each target coordinate with each text character index according to their correspondence to obtain a cross-modal space topological association set. Specifically, this includes receiving the target coordinate set and the text character index set, and for the same index... ,in From 1 to , No. Target coordinates With the Text character index They all originate from the same core data vector There is a natural sequence number correspondence between the two; the first... Target coordinates With the Text character index Pairing, and simultaneously calculating the first Core data vectors modulus As an indicator of correlation strength, the calculation formula is: ;

[0078] in for The One portion, Reflects The overall energy intensity in the core data vector space. The larger the value, the more significant the cross-modal correlation information carried by the vector. Constructing the... A cross-modal association record containing three fields: a 3D coordinate field for storage. Text index field storage Association strength field storage ; for all From 1 to Perform the pairing and building operations described above to generate These associated records are aggregated to form a cross-modal space topological association set. .

[0079] In this embodiment, the original text word positions corresponding to each core feature are traced in reverse to lock the image reading text fragments that are strongly associated with the local spatial location of the lesion; at the same time, the core vector magnitude is introduced as an association strength index to quantify the reliability of the matching between each set of image spatial locations and text descriptions.

[0080] In a preferred embodiment of the present invention, step 7, reorganizing the cross-modal spatial topological association set and target coordinates to generate a medical image data analysis report, may include:

[0081] Step 701: Based on the cross-modal spatial topological association set, sort the target coordinates and the text character indices paired with the target coordinates according to their spatial adjacency relationships to obtain a spatially ordered association sequence. Specifically, this includes: sorting the target coordinates and their paired text character indices according to the cross-modal spatial topological association set. As input, Include There are 10 associated records, each containing a target coordinate. The goal of spatial adjacency sorting is to make the target coordinates of two adjacent associated records as close as possible in three-dimensional space after sorting. Two auxiliary sets are created: the selected set is initially empty; the set to be selected initially contains... All 1 record; create an empty ordered sequence of results. Used to store sorted associated records; used to select target coordinates from the set of candidates. The record with the smallest Z-axis component is used as the starting record. ;Will Remove from the selection set and append to it. End; with target coordinates Using the search center point, calculate the target coordinates of each candidate record in the remaining records of the candidate set, one by one. The three-dimensional Euclidean distance between the search center point and the target point; the three-dimensional Euclidean distance is calculated according to the standard definition of the straight-line distance between two points, and is equal to the square root of the sum of the squares of the differences between the coordinate components of the two points; among all candidate records, the record that minimizes this distance value is selected as the next record. .Will Remove from the selection set and append to it. End; will Updated to ,by The target coordinates are used as the new search center point. The above distance calculation and nearest neighbor selection process is repeated until the set to be selected becomes empty. At this point... In The records have been arranged according to their spatial adjacency. That is, a spatially ordered associative sequence.

[0082] Step 702: Based on the spatially ordered association sequence, sequentially fill the coordinate values ​​of each target and the text character index values ​​paired with each target coordinate into the coordinate field and text index field of the preset report template to obtain a medical image data analysis report. Specifically, this includes: based on the spatially ordered association sequence... , Include The report is a list of related records sorted by spatial adjacency; the default report template is a structured text template, including a report header area, a main data area, and a report footer area; the report header area includes patient basic information fields and examination information fields; the main data area is a... The table has rows, each containing a serial number field, an X-coordinate field, a Y-coordinate field, a Z-coordinate field, and a text index field; the report footer area contains a generation time field; the specific operation for generating the report is to fill in the preset information corresponding to this inspection in each field of the report header area; in the main data area, process each row starting from row 1; for the... Okay, among them From 1 to , serial number Enter the serial number field for this row; retrieve The Middle Target coordinates of the associated records Fill the X, Y, and Z coordinate fields of the row with the three coordinate component values ​​respectively; retrieve the first... The text character index of each associated record is used to extract the corresponding original text fragment from the accompanying text character set based on its start and end sequence numbers and fill it into the text index field of that row; all After filling in all the fields, enter the current system timestamp in the generation time field at the end of the report. After completing all the fields, generate the medical image data analysis report.

[0083] This embodiment automatically completes the standardization and filling of information such as three-dimensional lesion coordinates, corresponding image reading text fragments, and data generation time, replacing the repetitive work of manually organizing image data, matching image reading text, and writing analysis reports, thereby reducing the workload of manual report writing and the risk of human input errors to a certain extent.

[0084] like Figure 2 As shown, embodiments of the present invention also provide an automated medical image report analysis system based on deep learning, comprising:

[0085] The preprocessing module receives medical image slice sequences and associated text character sets bound to the medical image slice sequences; performs grayscale normalization and denoising on the medical image slice sequences to generate a standardized pixel matrix sequence.

[0086] The segmentation module is used to perform word segmentation and stop word filtering on the attached text character set to obtain a standard word sequence; and to perform density segmentation-based local gray-level anomaly region delimitation on the normalized pixel matrix sequence to obtain a continuous pixel cluster mask.

[0087] The calculation module is used to calculate the instantaneous rate of change of the spatial position of the boundary of the local gray-level anomaly region point by point along the slice stacking direction based on the continuous pixel cluster mask, and obtain the three-dimensional boundary change sequence; and to perform semantic space mapping processing based on orthogonal decomposition family on the standard word sequence to obtain the text semantic distribution vector sequence containing semantic distribution information.

[0088] The fusion module is used to perform cross-modal global cumulative superposition and calculation of the three-dimensional boundary change sequence and the text semantic distribution vector sequence to obtain the image-text association distribution matrix.

[0089] The extraction module is used to perform orthogonal decomposition family space transformation on the image-text association distribution matrix to extract the main components and obtain the core data vector set;

[0090] The allocation module is used to solve the target coordinates of local gray-level anomaly regions in three-dimensional space and allocate text character indices based on the core data vector set, so as to obtain a cross-modal space topological association set.

[0091] The output module is used to reorganize cross-modal space topological association sets and target coordinates to generate medical image data analysis reports.

[0092] It should be noted that this system is a system corresponding to the above method. All implementation methods in the above method embodiments are applicable to this embodiment and can achieve the same technical effect.

[0093] Embodiments of the present invention also provide a computing device, including: a processor and a memory storing a computer program, wherein the computer program, when executed by the processor, performs the method described above. All implementations in the above method embodiments are applicable to this embodiment and can achieve the same technical effects.

[0094] Embodiments of the present invention also provide a computer-readable storage medium storing instructions that, when executed on a computer, cause the computer to perform the method described above. All implementations in the above method embodiments are applicable to this embodiment and can achieve the same technical effects.

[0095] The above description represents the preferred embodiments of the present invention. It should be noted that those skilled in the art can make various improvements and modifications without departing from the principles of the present invention, and these improvements and modifications should also be considered within the scope of protection of the present invention.

Claims

1. A method for automatic analysis of medical image reports based on deep learning, characterized in that, The method includes: Step 1: Receive the medical image slice sequence and the accompanying text character set bound to the medical image slice sequence; perform grayscale normalization and denoising processing on the medical image slice sequence to generate a standardized pixel matrix sequence; Step 2: Perform word segmentation and stop word filtering on the attached text character set to obtain a standard word sequence; perform density segmentation-based local gray-level anomaly region delimitation on the normalized pixel matrix sequence to obtain a continuous pixel cluster mask; Step 3: Based on the continuous pixel cluster mask, calculate the instantaneous change rate of the spatial position of the boundary of the local gray-level anomaly region point by point along the slice stacking direction to obtain the three-dimensional boundary change sequence; perform semantic space mapping processing based on orthogonal decomposition family on the standard word sequence to obtain the text semantic distribution vector sequence containing semantic distribution information; Step 4: Perform cross-modal global cumulative superposition and verification of the three-dimensional boundary change sequence and the text semantic distribution vector sequence to obtain the image-text association distribution matrix; Step 5: Perform an orthogonal decomposition family space transformation on the image-text association distribution matrix to extract the main components and obtain the core data vector set; Step 6: Based on the core data vector set, solve for the target coordinates of the local gray-scale anomaly region in three-dimensional space and assign text character indices to obtain the cross-modal space topological association set; Step 7: Reorganize the cross-modal space topological association set and target coordinates to generate a medical image data analysis report.

2. The method for automatic analysis of medical image reports based on deep learning according to claim 1, characterized in that, Perform word segmentation and stop word filtering on the attached text character set to obtain a standard word sequence; A density-segmented local gray-level anomaly region delineation is performed on the normalized pixel matrix sequence to obtain a continuous pixel cluster mask, including: Based on the standardized pixel matrix sequence, the gray values ​​of each pixel in each slice are extracted to obtain the gray value distribution set of a single slice. Based on the single-slice gray value distribution set, calculate the difference between the gray value of each pixel and the average gray value of the eight neighboring pixels corresponding to each pixel to obtain the local gray value deviation. Based on the local grayscale deviation value, pixels with local grayscale deviation values ​​greater than the preset density segmentation threshold are identified as grayscale anomalies, resulting in a set of grayscale anomalies. Based on the set of grayscale anomalies, spatially adjacent grayscale anomalies are connected and merged into the same cluster to obtain a continuous pixel cluster mask. Based on the accompanying text character set, the initial character sequence is obtained by segmenting each character; based on the initial character sequence, characters without meaningful functions are removed to obtain the standard word sequence.

3. The method for automatic analysis of medical image reports based on deep learning according to claim 2, characterized in that, Based on the continuous pixel cluster mask, the instantaneous rate of change of the spatial location of the boundary of the local gray-level anomaly region is calculated point by point along the slice stacking direction to obtain the three-dimensional boundary change sequence; semantic space mapping processing based on orthogonal decomposition family is performed on the standard word sequence to obtain the text semantic distribution vector sequence containing semantic distribution information, including: Based on the continuous pixel cluster mask, the boundary contour points of the clusters are extracted slice by slice along the slice stacking direction to obtain the boundary contour point set of each slice; Based on the set of boundary contour points of each slice, calculate the three-dimensional spatial displacement vector between the boundary contour points corresponding to the spatial positions in two adjacent slices to obtain the set of boundary displacement vectors between slices. Based on the set of boundary displacement vectors between slices, the ratio of the magnitude of each displacement vector to the distance between adjacent slices is calculated to obtain the instantaneous rate of change of the spatial position of each boundary contour point, and the results are summarized to form a three-dimensional boundary change sequence; based on the standard word sequence, the co-occurrence frequency of each word in the accompanying text character set is counted and a word co-occurrence matrix is ​​constructed. Perform orthogonal decomposition on the word co-occurrence matrix and extract the feature vectors corresponding to a preset number of feature values ​​in descending order of feature value to obtain the text semantic distribution vector sequence.

4. The method for automatic analysis of medical image reports based on deep learning according to claim 3, characterized in that, The three-dimensional boundary change sequence and the text semantic distribution vector sequence are cumulatively superimposed across modalities to obtain the image-text association distribution matrix, including: Based on the three-dimensional boundary change sequence, the instantaneous change rates of each spatial location are arranged sequentially according to the slice order and the boundary contour point order to construct the boundary change feature vector; The text semantic distribution vector sequence is directly used as the text semantic feature vector; based on the boundary change feature vector and the text semantic feature vector, the outer product of the boundary change feature vector and the text semantic feature vector is calculated to obtain the image-text association distribution matrix.

5. The method for automatic analysis of medical image reports based on deep learning according to claim 4, characterized in that, Perform an orthogonal decomposition-based family space transformation on the image-text association distribution matrix to extract the main components and obtain the core data vector set, including: Calculate the covariance matrix of the image-text association distribution matrix to obtain the image-text covariance matrix; perform singular value decomposition on the image-text covariance matrix to obtain the left singular vector matrix, the singular value diagonal matrix, and the right singular vector matrix; Based on the singular value diagonal matrix, select a predetermined number of singular values ​​in descending order of singular values, and extract the left singular vectors corresponding to the selected singular values ​​in the left singular vector matrix to obtain the principal component vector group; Based on the principal component vector set, the image-text association distribution matrix is ​​projected onto the subspace spanned by the principal component vector set to obtain the core data vector set.

6. The method for automatic analysis of medical image reports based on deep learning according to claim 5, characterized in that, Based on the core data vector set, the target coordinates of local gray-level anomaly regions in three-dimensional space are calculated and text character indices are assigned, resulting in a cross-modal space topological association set, including: Map each core data vector in the core data vector set back to the three-dimensional image space to obtain the coordinates of the local gray-level anomaly region in the three-dimensional space, which are denoted as the target coordinates. Based on the core data vector set, retrieve the text semantic distribution vector sequence component corresponding to each core data vector in the image-text association distribution matrix, trace the original character position of the standard word pointed to by the text semantic distribution vector sequence component in the accompanying text character set, and obtain the text character index. By pairing each target coordinate with each text character index according to the correspondence, a cross-modal space topological association set is obtained.

7. The method for automatic analysis of medical image reports based on deep learning according to claim 6, characterized in that, The cross-modal spatial topological association set and target coordinates are reorganized to generate a medical image data analysis report, including: Based on the cross-modal spatial topological association set, the target coordinates and the text character indices paired with the target coordinates are sorted according to the spatial adjacency relationship of the target coordinates to obtain a spatially ordered association sequence; Based on the spatially ordered association sequence, the numerical values ​​of each target coordinate and the numerical values ​​of the text character index paired with each target coordinate are sequentially filled into the coordinate field and text index field of the preset report template to obtain a medical image data analysis report.

8. A deep learning-based automated medical image report analysis system, wherein the system implements the method as described in any one of claims 1 to 7, characterized in that, include: The preprocessing module is used to receive medical image slice sequences and the accompanying text character set bound to the medical image slice sequences; Perform grayscale normalization and denoising on medical image slice sequences to generate standardized pixel matrix sequences; The segmentation module is used to perform word segmentation and stop word filtering on the attached text character set to obtain a standard word sequence; and to perform density segmentation-based local gray-level anomaly region delimitation on the normalized pixel matrix sequence to obtain a continuous pixel cluster mask. The calculation module is used to calculate the instantaneous rate of change of the spatial position of the boundary of the local gray-level anomaly region point by point along the slice stacking direction based on the continuous pixel cluster mask, and obtain the three-dimensional boundary change sequence; and to perform semantic space mapping processing based on orthogonal decomposition family on the standard word sequence to obtain the text semantic distribution vector sequence containing semantic distribution information. The fusion module is used to perform cross-modal global cumulative superposition and calculation of the three-dimensional boundary change sequence and the text semantic distribution vector sequence to obtain the image-text association distribution matrix. The extraction module is used to perform orthogonal decomposition family space transformation on the image-text association distribution matrix to extract the main components and obtain the core data vector set; The allocation module is used to solve the target coordinates of local gray-level anomaly regions in three-dimensional space and allocate text character indices based on the core data vector set, so as to obtain a cross-modal space topological association set. The output module is used to reorganize cross-modal space topological association sets and target coordinates to generate medical image data analysis reports.

9. A computing device, characterized in that, include: One or more processors; A storage device for storing one or more programs, which, when executed by one or more processors, cause the one or more processors to implement the method as described in any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a program that, when executed by a processor, implements the method as described in any one of claims 1 to 7.

Images