Multi-modal nasopharyngeal carcinoma report generation method under modal deficiency

Abstract

The invention provides a multi-modal nasopharyngeal carcinoma report generation method under modal deficiency, which comprises the following steps of: processing noise and a fuzzy region in an original image sequence from a preliminary multi-modal data set by adopting an image enhancement mode, and accurately positioning and identifying an incomplete part in combination with deficiency mode detection and a mask region to obtain an enhanced image sequence; extracting edge features and volume features of the lesion region from the enhanced image sequence by using a convolutional neural network, performing differentiation processing for age layering, and determining a lesion candidate region; for verification lesion information, transfer learning cross-age group analysis intermodal dependence is adopted, image fragment missing is processed in a data sparse scene, and a complete lesion description is generated; and generating a structured diagnosis report template according to the complete lesion description, fusing an age attribute automatic classification result, and generating a multi-modal diagnosis report for the child nasopharyngeal carcinoma tiny lesion.

Description

Technical Field

[0001] This invention relates to the field of information technology, and in particular to a method for generating multimodal nasopharyngeal carcinoma reports under modality loss. Background Technology

[0002] In the clinical diagnosis of nasopharyngeal carcinoma in children, the accurate generation of multimodal diagnostic reports is crucial to assisting doctors in assessing the condition. However, the core technical challenge that current technologies consistently face is the combination of the unique physiological development of children and the lack of multimodal data modalities. This results in the inability to effectively capture the tumor characteristics specific to children, thus hindering the formation of accurate lesion descriptions and reports. Specifically, the small lesions of nasopharyngeal carcinoma in children are characterized by blurred edges and unstable volume parameters, which differ significantly from the characteristics of adult tumors. This necessitates specialized technical means to focus on the unique characteristics of children. For example, in early childhood imaging, the tumor boundary appears irregularly diffused due to interference from the rapidly growing nasopharyngeal mucosa and immature bone structures, contrasting sharply with the clear boundaries and large, stable characteristics of adult tumors. In clinical practice, imaging sequences often exhibit noise and blurred areas due to equipment limitations and low cooperation from children during examinations. For instance, motion artifacts in CT scans cover the edges of potential lesions, further obscuring these developmental clues. At the same time, textual clinical descriptions often suffer from modal deficiencies, such as the lack of detailed age stratification records and incomplete descriptions of auxiliary symptoms. The combination of unique lesion characteristics and missing data modalities makes it difficult for existing masking mechanisms to simultaneously and accurately identify both age attributes and missing modalities in children. Age attribute determination requires complete image feature support, but noise and blurred areas caused by missing modalities can obscure age-related clues. Conversely, uncertainty in age information can lead to inaccurate localization of missing modalities, preventing the generation of accurate masks. Accurate masks are fundamental to the fusion of child-specific learnable cue vectors and remaining modal features. Masking bias directly interferes with the cue vectors' adaptation to child-specific features such as details of small lesions and age-related variations, preventing the cue vectors from effectively incorporating adaptive features of pediatric tumors. This results in blurred lesion localization, incomplete verification information, and ultimately, the inability to generate a multimodal diagnostic report that accurately reflects the condition of small lesions in children. In actual clinical scenarios, this dilemma manifests as doctors struggling to quickly confirm whether a report with blurred images and incomplete text represents an early, small lesion, delaying intervention. In summary, the differences in tumor pathological characteristics between children and adults are objectively real, and modality loss due to various practical factors during clinical data collection is also unavoidable. Currently, there is no adaptation solution for the combination of children's characteristics and modality loss, which hinders the report generation process and fails to meet the clinical need for accurate description of small lesions in pediatric nasopharyngeal carcinoma. Summary of the Invention

[0003] This invention provides a method for generating multimodal nasopharyngeal carcinoma reports under modality loss, mainly including: The original image sequences and auxiliary clinical descriptions of pediatric nasopharyngeal carcinoma patients were obtained from imaging equipment and text records through a multimodal data acquisition module. Convolutional neural networks were used to model age attributes based on physiological differences in children and automatically classify and label data to obtain a preliminary multimodal dataset. Image enhancement techniques are used to process noise and blurred regions in the original image sequence from the initial multimodal dataset. Incomplete parts are then accurately located and identified by missing pattern detection and masked region detection to obtain the enhanced image sequence. Convolutional neural networks are used to extract edge and volume features of lesion regions from enhanced image sequences, and differential processing is performed based on age stratification to determine candidate lesion regions. For candidate lesion regions, if there are parts with unclear boundaries, the mask is dynamically adjusted through an attention mechanism to focus on small-volume features and local details, and age-related features are combined for screening. If the boundaries are clear, the candidate lesion region is directly used to determine the location of potential lesions. The location of potential lesions is obtained by combining auxiliary clinical descriptions, edge features and volume features with attention mechanism multimodal fusion weight cross-validation. Incomplete parts are processed by cross-modal information compensation and missing data weight allocation to obtain the validated lesion information. To verify lesion information, transfer learning was used to analyze intermodal dependencies across age groups, and missing image segments were handled in data-sparse scenarios to generate complete lesion descriptions. A structured diagnostic report template is generated based on a complete lesion description, and the results of automatic classification based on age attributes are incorporated to generate a multimodal diagnostic report for small lesions of nasopharyngeal carcinoma in children.

[0004] Furthermore, the process involves acquiring raw image sequences and auxiliary clinical descriptions of pediatric nasopharyngeal carcinoma patients from imaging equipment and text records via a multimodal data acquisition module. Based on children's physiological differences, a convolutional neural network is used to model age attributes for automatic classification and data labeling, resulting in a preliminary multimodal dataset, including: The DICOM interface is used to read axial, coronal and sagittal image sequences of the nasopharynx of children from CT and MRI equipment. At the same time, the child's date of birth, first visit time, chief symptoms and past medical history are extracted from the electronic medical record system. The data is aligned according to the scanning parameters in the image header file and the timestamps in the text to obtain the original multimodal data with time matching. For the image sequences in the original multimodal data, a ResNet-50 convolutional neural network was used to extract three types of physiological features for each slice: the degree of ossification of the nasopharyngeal bones, the distribution of mucosal thickness, and the size of the adenoids. The correspondence between the feature values ​​and the age range was calculated based on the standard curve of children's growth and development to obtain age-stratified label data. The age-stratified label data is obtained, and the original multimodal data is differentiated according to the nasopharyngeal anatomy characteristics of different age groups. The feature matrix, text vector and age-stratified label data are then fused to construct a preliminary multimodal dataset.

[0005] Furthermore, the step of processing noise and blurred regions in the original image sequence from the preliminary multimodal dataset using image enhancement methods, and combining missing pattern detection and masked region precise localization to identify incomplete parts, yields an enhanced image sequence, including: Image sequences were extracted from the preliminary multimodal dataset. Gaussian noise in CT slices was processed using a nonlocal mean filtering algorithm. By calculating the weighted average of each pixel and similar structural blocks in its neighborhood, the texture details of the lesion were preserved, resulting in denoised image data. For the denoised image data, the blurred regions are identified by calculating the local gradient magnitude of each slice, and the edge blur is restored by performing deconvolution operation using Wiener filtering in the frequency domain. At the same time, the information entropy value of each slice is detected to mark the missing data positions, thus obtaining the edge restored image and the missing region marking. A binary mask image is generated based on the missing region markers. The missing boundary is expanded by morphological dilation. Gray values ​​of corresponding spatial locations are extracted from the front and back slices of the edge-recovered image. The pixel values ​​of the missing locations are calculated by trilinear interpolation to obtain an enhanced image sequence.

[0006] Furthermore, the step of extracting edge and volume features of lesion regions from the enhanced image sequence using a convolutional neural network, and performing differentiated processing based on age stratification to determine candidate lesion regions includes: Features are extracted from enhanced image sequences using a VGG-16 convolutional neural network, and a feature map is output. The gradient magnitude is calculated using the Sobel operator to mark potential edge points, and the edge points are connected to form a closed contour to obtain the lesion edge feature map. The lesion edge feature map is differentially optimized according to the age stratification label. For the early childhood label, the first gradient threshold is used to screen potential edge points and strengthen the connection weight of continuous edge points. For the preschool age label, the second gradient threshold is used to screen edge points. For the school age label, the third gradient threshold is used to screen edge points and weaken the edge point weight of the bone structure interference. The age-corrected edge feature map is obtained. The second gradient threshold is higher than the first gradient threshold, and the third gradient threshold is higher than the second gradient threshold. For the internal region of the lesion edge feature map after age correction, the single-layer volume value is calculated and accumulated. The volume calculation weight is adjusted according to the age stratification label. The first preset weight coefficient is used to compensate for the difference in tissue density in early childhood. The third preset weight coefficient is used to adapt to the anatomical characteristics of the mucosa-ossification transition stage in the preschool age. The second preset weight coefficient is used to correct the influence of ossification in school age, so as to obtain the age-corrected volume feature data. Based on the age-corrected volumetric feature data and the age-corrected lesion edge feature map, the roundness, eccentricity, and compactness are calculated, abnormal areas and potential lesion areas are marked, spatial coordinates are merged, and candidate lesion areas are determined.

[0007] Furthermore, regarding the candidate lesion region, if there are areas with unclear boundaries, an attention mechanism is used to dynamically adjust the mask to focus on small-volume features and local details, combined with age-related features for screening. If the boundaries are clear, the candidate lesion region is directly used to determine the location of potential lesions, including: For candidate lesion regions, the gray-level gradient difference and gradient direction change angle are calculated to determine the boundary clarity, and the boundary clarity identifier and blurry region distribution map are obtained. If the boundary clarity is marked as blurry, a self-attention mechanism is constructed to calculate the attention weights, and the attention head parameters are set according to the age layer label to obtain an age-adaptive feature map; Based on the age-adaptive feature map, a dynamic binary mask is generated, local feature vectors are extracted, cosine similarity is calculated, and the effective lesion part is fused to obtain a refined boundary. If the boundary clarity indicator is clear, the original lesion candidate region boundary is directly used to calculate the centroid coordinates and spatial bounding box parameters to determine the location of the potential lesion.

[0008] Furthermore, the acquisition of potential lesion locations is combined with auxiliary clinical descriptions, edge features, and volumetric features to perform attention-based multimodal fusion weighted cross-validation. Incomplete portions are processed through cross-modal information compensation and missing data weight allocation to obtain validated lesion information, including: The three-dimensional coordinate data of potential lesion locations are obtained. Auxiliary clinical descriptions, including the degree of nasal congestion, runny nose, hearing loss records, and neck palpation results in the chief complaint, are extracted from the electronic medical record. At the same time, the gradient distribution data in the edge feature map and the spatial occupancy information in the volume feature data are read. The text description is converted into a semantic vector through word embedding. The semantic vector is aligned with the image feature vector in dimension to obtain a multimodal feature set. For the aforementioned multimodal feature set, a cross-modal attention matrix is ​​constructed using a Transformer encoder. The correlation score between the text modality and the image modality is calculated using a multi-head attention mechanism. Feature values ​​at corresponding positions are extracted from other modalities as compensation. The correlation score is normalized using a softmax function to obtain fusion weights. The image features and text features are weighted and summed according to the fusion weights to obtain the fused feature representation. Cross-validation is performed based on the fused feature representation, and the correlation coefficient is calculated to adjust the contribution of each mode, resulting in a mode weight allocation matrix. Based on the modality weight allocation matrix, the multi-source features are recombined, and the spatial coordinates of the lesion, the malignancy score, and the clinical correlation are fused to obtain the verification lesion information.

[0009] Furthermore, the method employs transfer learning to analyze inter-modal dependencies across age groups to verify lesion information, handles missing image segments in data-sparse scenarios, and generates complete lesion descriptions, including: The feature set of nasopharyngeal carcinoma lesions in children of each age group is used as the source task pre-training feature library for transfer learning. For verifying lesion information, the cosine similarity between the current lesion feature vector and the feature vector of each age group is calculated. The age group with the highest similarity is selected to obtain the age feature mapping relationship. Based on the age feature mapping relationship, an intermodal dependency graph is constructed. The source task features of the transfer are used to optimize the calculation of modal dependency strength. Feature propagation is achieved by multiplying the adjacency matrix and the feature matrix to obtain the cross-modal feature completion matrix. The cross-modal feature completion matrix is ​​used to process image missing data, and the pixel values ​​of the missing slices are calculated by weighted averaging to obtain reconstructed complete image data; By integrating the reconstructed complete image data with the original verified lesion information, the data is organized according to six dimensions: lesion location, size, shape, edge, density, and enhancement, and combined to form a complete lesion description.

[0010] Furthermore, the process of generating a structured diagnostic report template based on a complete lesion description, incorporating automatic age attribute classification results, and generating a multimodal diagnostic report for small lesions in pediatric nasopharyngeal carcinoma includes: Based on the complete lesion description, extract six key diagnostic elements: lesion location, size, shape, margin, density, and enhancement. Organize these elements into three sections: examination findings, imaging manifestations, and diagnostic opinions. Fill in the child's name, age, examination date, and scan sequence parameters, and insert the content of the lesion description to obtain a preliminary report framework. The age attribute classification results are embedded into the diagnostic opinion section of the preliminary report framework. Differential diagnosis prompts for the corresponding age group are added, medical terminology is adjusted, and the correlation between imaging feature descriptions and clinical symptoms is integrated to output a multimodal diagnostic report for small lesions of nasopharyngeal carcinoma in children.

[0011] The technical solutions provided by the embodiments of the present invention may include the following beneficial effects: This invention discloses a multimodal nasopharyngeal carcinoma report generation method under modality loss. Nasopharyngeal carcinoma in children is difficult to detect accurately using traditional methods due to the wide age range of patients, significant differences in physiological structure, small and indistinct boundaries of early lesions, and often incomplete clinical data. This invention first acquires multimodal data and uses a convolutional neural network to automatically model age attributes for hierarchical data labeling. Then, it performs targeted enhancement and precise masking of missing regions on the original image sequences. Subsequently, it extracts candidate lesion regions by combining edge and volume features of age stratification. An attention mechanism dynamically focuses on the details of small lesions and achieves multimodal fusion weighted cross-validation of clinical description and images. Transfer learning is used to compensate for sparse and missing data across age groups. Finally, a complete structured diagnostic report incorporating age attributes is generated, significantly improving the detection accuracy and report consistency of early small lesions in pediatric nasopharyngeal carcinoma. Attached Figure Description

[0012] Figure 1 This is a flowchart of a method for generating multimodal nasopharyngeal carcinoma reports under modality loss according to the present invention.

[0013] Figure 2 This is a schematic diagram of a multimodal nasopharyngeal carcinoma report generation method under modality loss according to the present invention.

[0014] Figure 3 This is another schematic diagram of a multimodal nasopharyngeal carcinoma report generation method under modality loss according to the present invention. Detailed Implementation

[0015] To enable those skilled in the art to better understand the technical solutions in this specification, the technical solutions in the embodiments of this specification will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this specification, and not all embodiments. Based on the embodiments in this specification, all other embodiments obtained by those skilled in the art without creative effort should fall within the scope of protection of this specification.

[0016] like Figures 1-3 This embodiment of a method for generating multimodal nasopharyngeal carcinoma reports under modality loss may specifically include: Step S101: The original image sequence and auxiliary clinical description of pediatric nasopharyngeal carcinoma patients are obtained from imaging equipment and text records through the multimodal data acquisition module. Convolutional neural network is used to model age attributes based on physiological differences in children and automatically classify and label data to obtain a preliminary multimodal dataset.

[0017] The image sequences of the nasopharynx in the child are read from CT and MRI equipment in axial, coronal and sagittal planes via DICOM interface. At the same time, the child's date of birth, first visit time, chief complaint and past medical history are extracted from the electronic medical record system. The data are aligned according to the scanning parameters in the image header file and the timestamps in the text to obtain the original multimodal data with time matching. For the image sequences in the original multimodal data, a ResNet-50 convolutional neural network was used to extract three physiological features for each slice: ossification degree of the nasopharynx, mucosal thickness distribution, and adenoid size. Ossification degree was calculated as the mean gray value of the skeletal region divided by the standard gray value of the skull base. Mucosal thickness distribution was calculated using the mean and variance of the thickness of a 10×10 pixel region around the pharyngeal recess and the torus tubarius. Adenoid size was calculated as the ratio of the sagittal projection area of ​​the nasopharynx to the total area of ​​the nasopharynx. The correspondence between the feature values ​​and age intervals was calculated based on the standard curve of children's growth and development. If the ossification degree was below threshold T1 and the adenoid proportion in the nasopharynx exceeded threshold T2, it was labeled as early childhood. If the ossification degree was between threshold T1 and threshold T3, it was labeled as preschool age. If the ossification degree exceeded threshold T3, it was labeled as school age. Thresholds T1=0.3, T2=0.6, and T3=0.7 represent the fractional thresholds for ossification degree and adenoid proportion, respectively. T1, T2, and T3 were obtained through K-means clustering analysis of the feature values ​​of 500 cases, yielding age-stratified label data. Based on the nasopharyngeal anatomy characteristics of different age groups, the original multimodal data underwent differentiated processing. For images labeled as early childhood, a 3×3 convolutional kernel was used for edge-preserving filtering to remove motion artifacts; for images labeled as preschool and school-age, a 5×5 convolutional kernel was used for noise suppression. The processed image sequences were converted into feature matrices. Simultaneously, symptom duration, nasal congestion scores, and cervical lymph node descriptions were extracted from text records and converted into text vectors. The feature matrices, text vectors, and age-stratified label data were then fused to construct a preliminary multimodal dataset.

[0018] Specifically, in one implementation, when reading medical images of pediatric nasopharyngeal carcinoma patients from imaging equipment via the DICOM interface, the system automatically parses the metadata information in the DICOM file header, including scan time, slice thickness, pixel pitch, and window width and level parameters. Simultaneously, text records extracted from the electronic medical record system contain structured fields and unstructured descriptions. The structured fields cover the child's date of birth, first visit date, and ICD-10 diagnostic code, while the unstructured descriptions include a description of the degree of nasal congestion in the chief complaint, a record of rhinorrhea characteristics, and a history of adenoid hypertrophy. By comparing the StudyDate field of the image sequence with the visit timestamp in the medical record, temporal alignment of multi-source data is achieved.

[0019] Specifically, when processing nasopharyngeal images in children, the ResNet-50 convolutional neural network's residual connection structure effectively preserves subtle anatomical features extracted by the superficial network. The first three residual blocks of the network are specifically used to capture the degree of ossification, quantifying the ossification process by identifying the fusion status of the sphenoid ridge, pterygoid process, and skull base sutures. The fourth residual block focuses on extracting the mucosal thickness distribution, particularly the mucosal thickening patterns in the pharyngeal recess region and around the torus tubarius. The last residual block specifically identifies the adenoid contour and calculates its projected area in the sagittal plane of the nasopharynx. When these feature values ​​are matched with standard curves for children's growth and development, the system references bone age assessment indicators in the WHO child growth criteria, mapping continuous feature values ​​to discrete age intervals.

[0020] For example, when processing the imaging data of a 4-year-old child, the ResNet-50 extracted ossification value of the sphenoid ridge was 0.25, the adenoids occupied 0.75% of the nasopharynx, and the posterior pharyngeal wall mucosal thickness was 8 mm. Based on preset first thresholds of 0.3 and second thresholds of 0.6, the system labeled the child as an infant. This age-stratified label not only reflects the physiological developmental status but also implicitly contains information on differences in lesion growth rate and treatment sensitivity.

[0021] In one possible implementation, the differentiated image processing strategy fully considers the varying levels of cooperation among children of different ages. Early childhood patients, unable to remain still for extended periods, exhibit motion artifacts in their images primarily manifesting as blurred edges and striped interference. Edge-preserving filtering with a 3×3 convolutional kernel calculates the gradient difference between the central pixel and its eight surrounding pixels, smoothing only areas with gentle gradient changes, thus preserving the true contours of the lesion edges. For school-aged children, image noise mainly originates from salt-and-pepper noise generated by low-dose scanning; a 5×5 convolutional kernel can perform median filtering within a larger receptive field, effectively suppressing random noise.

[0022] It should be noted that during the conversion of the processed image sequence into a feature matrix, each slice is encoded into a 256-dimensional feature vector after model processing, containing texture features, morphological features, and grayscale distribution features. When converting text records into vectors, word embedding technology is used to map descriptions such as "nasal congestion for 3 days" into triple vectors containing symptom type, severity, and duration. The final preliminary multimodal dataset integrates the image feature matrix, text vectors, and age-stratified labels into a unified data structure through tensor concatenation, laying the data foundation for subsequent lesion identification and report generation.

[0023] Step S102: The noise and blurred regions in the original image sequence are processed using image enhancement methods from the preliminary multimodal dataset. The incomplete parts are identified by combining missing pattern detection and mask region accurate localization to obtain the enhanced image sequence.

[0024] Image sequences were extracted from a preliminary multimodal dataset. A nonlocal mean filtering algorithm was used to process Gaussian noise in CT slices. The weighted average of each pixel and its neighboring similar structural blocks was calculated, and different weights were assigned based on structural similarity. If the similarity exceeded a preset threshold, the weight coefficient was increased. This process preserved lesion texture details while suppressing background noise, resulting in denoised image data. For this denoised image data, blurred regions were identified by calculating the local gradient magnitude of each slice. Wiener filtering was used for deconvolution in the frequency domain, and the filtering parameters were adaptively adjusted according to the signal-to-noise ratio to restore the edge blur caused by the child's head movements. Simultaneously, the information entropy value of each slice was detected. If the entropy value was lower than a preset threshold, the slice region was marked as a data missing location, resulting in an edge restoration image and missing region markers. A binary mask image was generated based on the missing region markers. Morphological dilation was used to extend the missing boundary outwards by three pixels to determine the compensation range. Gray values ​​at corresponding spatial locations were extracted from the three slices preceding and following the edge restoration image. Trilinear interpolation was used to calculate the pixel values ​​at the missing locations, and the interpolation results were filled into the corresponding positions in the edge restoration image to obtain an enhanced image sequence.

[0025] Specifically, in one implementation, the image sequence extracted from the initial multimodal dataset contains CT slices with different scan slice thicknesses, each with varying degrees of Gaussian noise. When processing this noise, the nonlocal mean filtering algorithm first defines a search window size of 21×21 pixels and a neighborhood block size of 7×7 pixels. For the pixel to be processed, the algorithm searches for all possible neighborhood blocks within the search window, calculates the Euclidean distance between the current neighborhood block and candidate neighborhood blocks, and converts the distance into similarity weights using a Gaussian weighted function. When the texture similarity between two neighborhood blocks exceeds a preset threshold of 0.75, the weight coefficient increases from the default 0.3 to 0.8, thereby smoothing the noise while preserving the subtle texture features of nasopharyngeal lesions.

[0026] Specifically, the Wiener filtering process for blurry regions involves several key steps. First, the system performs a two-dimensional Fast Fourier Transform (FFT) on the denoised image data, converting the spatial domain image to the frequency domain. In the frequency domain, the direction and degree of motion blur are identified by analyzing the energy distribution of the spectrum; motion blur manifests as striped dark bands in a specific direction. The Wiener filter is constructed based on the minimum mean square error criterion, with its transfer function being H*(u,v) / [|H(u,v)|²+K], where H(u,v) is the frequency domain representation of the point spread function, and K is a regularization parameter adaptively adjusted according to the signal-to-noise ratio (SNR). When the SNR is below 10dB, K is set to 0.01 to enhance the denoising effect; when the SNR is above 20dB, K is reduced to 0.001 to retain more detail. Finally, an inverse Fourier transform is used to convert the processed frequency domain image back to the spatial domain, achieving effective edge recovery.

[0027] It should be noted that the information entropy is calculated using the Shannon entropy formula. For each 8×8 image block, its grayscale histogram is calculated, and regions with entropy values ​​below 4.5 bits are considered to have insufficient information and are marked as potential data loss locations. These losses may be caused by metal artifacts during the scanning process, sudden movement of the child, or equipment malfunction.

[0028] In one possible implementation, the morphological dilation operation uses a 3×3 structuring element to perform three iterative dilations on the binary mask image, ensuring that the missing boundary expands outward by exactly three pixels. This expansion range is set based on clinical experience, covering uncertain areas at the edges without unduly affecting the display of normal tissue.

[0029] Preferably, in the implementation of bilinear interpolation, the system extracts the gray values ​​of spatially corresponding positions from three slices before and after the edge-reconstructed image as references. For the missing position (x, y), the interpolation formula considers the neighboring pixels in the axial, coronal, and sagittal directions, and calculates the filling value by weighted averaging. The weight coefficients are determined according to the reciprocal of the spatial distance; the closer the reference point, the greater the weight, thereby ensuring the continuity of the compensated pixel value with the surrounding tissue.

[0030] For example, when processing a slice containing calcified cervical lymph nodes, the calcified area produces radial artifacts in the original image, resulting in the loss of information within a 5-7 pixel range. The system, through the aforementioned process, first identifies these artifact areas, then extracts tissue density information of the corresponding anatomical location from adjacent artifact-free slices, and finally reconstructs the soft tissue structure obscured by the artifacts through interpolation, enabling physicians to accurately assess the infiltration around the lymph nodes.

[0031] Step S103: Extract edge and volume features of lesion regions from enhanced image sequences using convolutional neural networks, perform differential processing based on age stratification, and determine candidate lesion regions.

[0032] Features are extracted from the enhanced image sequence using a VGG-16 convolutional neural network. The grayscale matrix of each slice is input to the first five convolutional layers of the network, and a 256-dimensional feature map is output. The gradient magnitudes in the horizontal and vertical directions are calculated using the Sobel operator, and the gradient difference between adjacent pixels is calculated. If the difference exceeds a preset threshold and remains stable within three consecutive pixels, it is marked as a potential edge point. The edge points are connected to form a closed contour, thus obtaining the lesion edge feature map. For the internal region of the lesion edge feature map, the single-layer volume value is obtained by calculating the number of pixels contained in the closed contour in each slice and multiplying it by the slice thickness. The volumes of all slices containing lesions are accumulated. At the same time, the volume calculation weight is adjusted according to the age stratification label. For early childhood, a first preset weight coefficient of 1.15-1.25 is used to compensate for tissue density differences. The first preset weight coefficient is based on the clinical statistical result that the nasopharyngeal tissue density in early childhood is 15-25% lower than that in adults. It is used to compensate for the volume underestimation caused by density differences. For preschool age, a third preset weight coefficient of 0.95-1.05 is used to compensate for growth and development differences. The third preset weight coefficient is based on the anatomical characteristics of decreasing mucosal thickness and moderate ossification in preschool age. It is obtained by calibrating the error between the volume measurement values ​​of 300 cases and the actual pathological volume. For school age, a second preset weight coefficient of 0.85-0.95 is used to correct for the influence of ossification. The second preset weight coefficient is based on the clinical phenomenon that skull base ossification in school age causes lesion edge obscuring. It is obtained by fitting data from 200 cases to obtain age-corrected volume feature data. Based on the age-corrected volumetric feature data and the lesion edge feature map, the circularity is obtained by calculating the ratio of the square of the perimeter of the lesion outline to its area, the eccentricity is obtained by calculating the ratio of the major axis to the minor axis, and the compactness is obtained by calculating the ratio of the area to the area of ​​the circumscribed rectangle. If the circularity is lower than the first threshold and the volume exceeds the second threshold, it is marked as an abnormal region. If the edge is lobed and the standard deviation of the density distribution exceeds the third threshold, it is marked as a potential lesion region. The spatial coordinates of the abnormal region and the potential lesion region are combined to determine the lesion candidate region. The first gradient threshold is set to 20 gray levels, the second gradient threshold is set to 35 gray levels, and the third gradient threshold is set to 50 gray levels. Through orthogonal experiments for optimization, the lesion edge recognition accuracy is highest when the gradient thresholds are 20, 35, and 50, respectively.

[0033] Specifically, in one implementation, the VGG-16 convolutional neural network employs a transfer learning strategy to load weight parameters pre-trained on the ImageNet dataset when processing enhanced image sequences. The first five convolutional blocks of the network contain 13 convolutional layers, each using a 3×3 convolutional kernel, progressively extracting features from low to high levels through ReLU activation functions and max-pooling layers. Each input CT slice is first resized to a standard size of 224×224 pixels, normalized to the 0-1 range, and then forward-propagated through the network to obtain a 256-dimensional feature map. These feature maps contain information on the texture, shape, and intensity distribution of lesions.

[0034] Specifically, the Sobel operator uses 3×3 convolution kernels in both the horizontal and vertical directions to calculate gradients. The horizontal gradient Gx is obtained by convolving a matrix of [-1,0,1;-2,0,2;-1,0,1], and the vertical gradient Gy is calculated using its transpose. The gradient magnitude is calculated using sqrt(Gx²+Gy²). When the gradient difference between adjacent pixels exceeds a preset threshold of 30 gray levels, it indicates the presence of a clear tissue boundary. Continuity is determined using a sliding window; if the rate of change of gradient magnitude between three consecutive pixels is less than 15%, the edge is considered stable.

[0035] It should be noted that the age-corrected weights are set based on the physiological characteristics of nasopharyngeal development in children. The nasopharynx is relatively narrow in early childhood, and physiological hypertrophy of the adenoids can occupy a significant amount of space, easily leading to an underestimation of the actual lesion volume. Therefore, the first preset weighting coefficient is set in the range of 1.15-1.25. In school-aged children, the degree of skull base ossification increases, and the high-density signal of bony structures on CT scans may obscure part of the lesion margins. The second preset weighting coefficient is set in the range of 0.85-0.95 to compensate for this effect.

[0036] For example, roundness is calculated using the formula 4πA / P², where A is the area enclosed by the lesion outline and P is the perimeter of the outline. Normal lymph nodes typically have a roundness greater than 0.7, while malignant tumors, due to their invasive growth leading to irregular edges, often have a roundness less than 0.5. Eccentricity is calculated by fitting the least circumscribed ellipse of the lesion outline; a ratio of major axis to minor axis exceeding 2.0 suggests a spindle-shaped or irregular lesion.

[0037] In one possible implementation, density distribution is assessed by calculating the standard deviation of the CT values ​​of all pixels within the lesion region. The standard deviation for homogeneous benign lesions is typically less than 20 HU, while malignant lesions with internal necrosis, hemorrhage, or calcification exhibit non-uniform density distribution with a standard deviation reaching 40-60 HU. Identification of lobulated edges is achieved through convex hull analysis, calculating the ratio of the actual contour area to the convex hull area; a ratio less than 0.8 indicates a positive lobulation sign.

[0038] Preferably, the spatial coordinates of abnormal areas and potential lesion areas are merged using a Boolean union operation to ensure that all suspicious areas are included in the lesion candidate region, avoiding missed diagnoses. The finally determined lesion candidate region generates a 3D bounding box, recording its coordinate range in the axial, coronal, and sagittal planes. Subsequently, precise localization analysis is performed based on this bounding box, and a diagnostic report is generated, providing spatial reference and clinical recommendations.

[0039] In step S104, for the candidate lesion region, if there are parts with unclear boundaries, the mask is dynamically adjusted through the attention mechanism to focus on small-volume features and local details, and age-related features are combined for screening. If the boundaries are clear, the candidate lesion region is directly used to determine the location of potential lesions.

[0040] For candidate lesion regions, the grayscale gradient difference between each pixel and its adjacent pixels is calculated, and the gradient direction change angle is statistically analyzed. If the gradient direction change of multiple consecutive pixels exceeds a preset angle threshold, they are marked as points with unclear boundaries. The proportion of these unclear points to the total contour length is calculated. If the proportion exceeds a first preset threshold, the boundary is determined to be blurred; otherwise, the boundary is determined to be clear, resulting in a boundary clarity identifier and a blurred region distribution map. If the boundary clarity identifier is blurred, a region weight mask is generated based on the blurred region distribution map. A self-attention mechanism is constructed, extracting the query matrix, key matrix, and value matrix from the feature vector of the candidate lesion region. The key matrix is ​​adjusted using the weight mask to highlight the blurred region. The product of the query matrix and the transpose of the adjusted key matrix is ​​calculated and scaled. The attention weights are obtained by normalization using the softmax function. The weights are multiplied by the value matrix to obtain weighted features. Simultaneously, different numbers of attention head parameters are set according to age-level labels. For early childhood, an attention head higher than that for school-age children is used to capture subtle changes, while for school-age children, an attention head lower than that for early childhood children is used to focus on major features, resulting in an age-adaptive feature map. A dynamic binary mask is generated based on the age-adaptive feature map. Morphological dilation is used to expand the mask boundary, covering blurred areas and their neighborhoods. Local feature vectors within the mask area are extracted. The cosine similarity between these local feature vectors and pre-established typical lesion feature templates for the same age group is calculated. If the similarity exceeds a second preset threshold, the area is retained as a valid lesion. The valid lesion portions are then fused to obtain a refined boundary. If the boundary clarity is indicated as clear, the original lesion candidate region boundary is directly used; otherwise, the refined boundary is used. By calculating the centroid coordinates, maximum diameter, and spatial bounding box parameters of the lesion region, the start and end coordinates of the lesion in the axial, coronal, and sagittal planes are recorded to determine the location of potential lesions.

[0041] Specifically, in one implementation, boundary sharpness is determined based on statistical analysis of gradient directions. The system first samples the contour of the lesion candidate region, extracting a sampling point every two pixels and calculating the gradient vector at each sampling point. The gradient vector is calculated using the Sobel operator within a 3×3 neighborhood, containing both magnitude and direction components. The gradient direction difference between adjacent sampling points is calculated using an inverse cosine function. When the directional change angle of five or more consecutive sampling points exceeds 30 degrees, the contour segment is marked as unclear. The total length of all unclear segments is calculated. If its proportion to the total contour length exceeds a preset first threshold of 0.3, the entire lesion candidate region is determined to have blurred boundaries, requiring subsequent attention mechanism processing.

[0042] Specifically, the construction of the self-attention mechanism involves several key steps. The lesion candidate region is first divided into 8×8 image patches, and each patch extracts a 256-dimensional feature vector through convolution. These feature vectors constitute the input sequence, with a length of 64. The query matrix Q is obtained by multiplying the feature vectors with the learnable query weight matrix WQ, resulting in a dimension of 64×256. The key matrix K and value matrix V are generated in the same way using their corresponding weight matrices WK and WV. The attention score is obtained by calculating the matrix product of Q and the transpose of K, and the result is scaled by dividing by the square root of the dimension to prevent gradient vanishing. The scaled score is normalized using the softmax function to obtain the attention weight matrix, with a dimension of 64×64, representing the degree of attention each image patch gives to all other patches. The final attention output is obtained by multiplying the weight matrix with the value matrix V, achieving weighted aggregation of features.

[0043] It should be noted that the age-adaptive parameters were set to fully consider the differences in pathological characteristics at different developmental stages of children. In early childhood, the nasopharyngeal tissues are soft, and lesion boundaries often exhibit an infiltrative growth pattern; subtle textural changes are crucial for diagnosis. Therefore, eight attention heads were used for early childhood, each focusing on features at different scales: two heads specifically capturing minute textures within the 1-3 pixel range, three heads focusing on medium-scale structures of 5-10 pixels, and three heads handling overall morphology larger than 15 pixels. Six attention heads were used for preschool children. In contrast, school-aged children have relatively mature tissue structures and clearer lesion boundaries; four attention heads were sufficient to extract features, corresponding to the four main feature dimensions of texture, edge, shape, and density. This differentiated setting allows the model to automatically adjust the granularity and focus of feature extraction according to the child's age.

[0044] For example, the generation process of the dynamic binary mask begins with threshold segmentation of the age-adaptive feature map. The activation value of each pixel in the feature map reflects the probability that the location belongs to a lesion. The segmentation threshold is determined using the Otsu adaptive thresholding method to generate the initial binary mask. The morphological dilation operation uses a circular structuring element, with the radius dynamically adjusted according to the size of the lesion, typically one-tenth the length of the shorter side of the lesion's smallest bounding rectangle. The dilation operation is performed three times to ensure that the mask fully covers the blurred boundary region and its neighborhood. Local features within the mask-covered region are extracted through pooling operations, with each 5×5 local window generating a 128-dimensional feature vector.

[0045] In one possible implementation, the feature templates are constructed based on statistical analysis of a large number of nasopharyngeal carcinoma cases in children of the same age group. The system pre-collects typical lesion samples from different age groups and extracts the most representative feature patterns through principal component analysis. The early childhood template focuses on encoding abnormal hyperplasia patterns around the adenoids, asymmetry of the pharyngeal recess, and early signs of skull base bone destruction. The school-age template focuses more on infiltration of the parapharyngeal space, cervical lymph node metastasis patterns, and the mass effect of the nasopharynx.

[0046] Preferably, the cosine similarity calculation employs normalization to ensure comparability of features at different scales. Local feature vectors are first normalized using the L2 norm to a magnitude of 1, and then the dot product with the feature template is calculated. The similarity threshold is dynamically adjusted based on clinical needs; a second preset threshold is set to 0.6 for high-sensitivity scenarios and increased to 0.75 for high-specificity scenarios.

[0047] Understandably, the handling strategies for both clear and blurred boundaries reflect the algorithm's adaptability. When the boundary clarity indicator shows a clear boundary, the system directly uses the original lesion candidate region boundary to avoid unnecessary computational overhead. This situation typically occurs when there is a significant difference in density between the lesion and normal tissue, the child's cooperation is high, and the image quality is excellent. Furthermore, the final determination of the lesion location integrates multiple spatial parameters. The centroid coordinates are obtained by calculating the weighted average of the coordinates of all pixels within the lesion region, with the weight being the grayscale value of the corresponding pixel. The maximum diameter is calculated by traversing all possible endpoint pairs, selecting the pair with the largest Euclidean distance as the major axis endpoints. The spatial bounding box uses an axis-aligned bounding box algorithm, recording the minimum and maximum coordinate values ​​in the X, Y, and Z directions respectively.

[0048] For example, when processing nasopharyngeal MRI images of a 3-year-old child, the system first detected an abnormal signal area of ​​15mm × 12mm in the pharyngeal recess region. Due to slight head movement during the examination, approximately 40% of the lower edge of the lesion became blurred. The system automatically identified this as a case of blurred boundaries and activated the 8-head attention mechanism. Through attention weighting, the system focused on the clearly defined upper edge of the lesion and its internal signal characteristics. Combining this with early childhood feature template matching, the system accurately delineated the complete outline of the lesion, determining its spatial location as the posterior wall of the nasopharynx, with coordinates ranging from axial Z: 45-52mm, coronal Y: 28-40mm, and sagittal X: 32-47mm. This precise spatial localization provides clinicians with a reliable reference for developing radiotherapy plans.

[0049] Step S105: Obtain the potential lesion location and combine it with auxiliary clinical description and edge feature volume features to perform attention mechanism multimodal fusion weight cross-validation. Incomplete parts are processed through cross-modal information compensation and missing data weight allocation to obtain the validated lesion information.

[0050] Three-dimensional coordinate data of potential lesion locations are obtained from edge feature maps and volumetric feature data. Auxiliary clinical descriptions, including the degree of nasal congestion, runny nose characteristics, hearing loss records, and neck palpation results, are extracted from electronic medical records. Simultaneously, gradient distribution data from the edge feature map and spatial occupancy information from the volumetric feature data are read. The text description is converted into a semantic vector of a preset dimension using word embedding technology. This semantic vector is then dimensionally aligned with the image feature vector to obtain a multimodal feature set. For this multimodal feature set, a Transformer encoder is used to construct a cross-modal attention matrix. The input is the aligned text vector and image vector. A multi-head attention mechanism is used to calculate the correlation score between the text modality and the image modality. If data in a certain modality is missing, feature values ​​at the corresponding location are extracted from other modalities as compensation based on the correlation score. The correlation score is normalized using a softmax function to obtain fusion weights. The image features and text features are then weighted and summed according to the fusion weights to obtain the fused feature representation. Cross-validation is performed based on the fused feature representation. Marginal gradient values ​​and symptom severity values ​​from the fused features are extracted, and their Pearson correlation coefficient is calculated. Volume measurements and nasopharyngeal ventilation scores are also extracted from the fused features, and their correlation coefficient is calculated. If the correlation coefficient exceeds a preset threshold, the weight of the corresponding feature pair in the fused features is increased; otherwise, the weight is decreased. The contribution of each modality is iteratively adjusted using gradient descent until convergence, resulting in a modality weight allocation matrix. Multi-source features are recombined based on the modality weight allocation matrix. Missing data locations are zero-padded and multiplied by the mean of adjacent complete data as compensation. Complete data retains its original value and is multiplied by its corresponding weight. A weighted average is used to calculate the lesion malignancy score and clinical symptom correlation. The lesion spatial coordinates, malignancy score, and clinical correlation are fused to obtain the validated lesion information.

[0051] Specifically, in one implementation, the extraction and alignment of multimodal features begins with the standardization of heterogeneous data sources. The three-dimensional coordinate data of potential lesion locations are stored as floating-point arrays, containing six values: axial, coronal, and sagittal start and end coordinates. Clinical descriptions in the electronic medical record extract key information through natural language processing: nasal congestion is quantified on a scale of 0-10; runny nose characteristics are coded as clear, mucoid, and purulent; hearing loss is converted to decibel values; and neck palpation results are used to mark the location, size, and mobility of lymph nodes. The gradient value of each pixel in the edge feature map ranges from 0-255, and volumetric feature data records the three-dimensional dimensions and occupancy percentage of the lesion.

[0052] Specifically, word embedding technology uses the Word2Vec model to convert clinical text into semantic vectors. Each medical term is first processed through word segmentation, and then its corresponding vector representation is looked up in a pre-trained medical vocabulary. Dimension alignment is achieved through a linear transformation matrix, mapping feature vectors of different dimensions to a unified 256-dimensional space. The image feature vectors come from the output of the convolutional neural network and are already in 256-dimensional format. The text semantic vectors have their dimensions adjusted through fully connected layers to ensure that both modalities are represented in the same feature space.

[0053] It's important to note that the Transformer encoder employs a unique positional encoding strategy when constructing the cross-modal attention matrix. Positional encoding in text sequences reflects word order, while in image sequences it corresponds to spatial location information. The encoder consists of six identical layers, each comprising a multi-head attention sublayer and a feedforward neural network sublayer. The multi-head attention mechanism uses eight attention heads, each with a dimension of 32, each focusing on different types of cross-modal associations. The first head focuses on the correspondence between lesion location and symptom description; the second head focuses on the relationship between edge features and infiltration degree; the third head handles the correlation between volume size and nasal congestion severity; and the remaining heads capture the association patterns between texture, density, morphology, and other features with clinical manifestations. The correlation score is calculated by the dot product of the query vector and the key vector, ranging from -1 to 1, with positive values ​​indicating positive correlation and negative values ​​indicating negative correlation. The absolute value reflects the degree of correlation.

[0054] For example, when data on the adenoid region is missing in the image modality, the system extracts compensation information from the text modality using the established correlation score. If the clinical description records "severe adenoid hypertrophy, almost complete obstruction of the nasopharynx," with a correlation score of 0.85, the system will infer from the text features that the volume of the adenoids accounts for approximately 80-90% of the nasopharynx based on this high correlation, and use this estimate to fill in the missing image features.

[0055] In one possible implementation, the cross-validation process employs a two-way matching mechanism. Marginal gradient values ​​typically range from 20 to 150, with higher gradient values ​​for clear boundaries and lower gradient values ​​for ambiguous boundaries. Symptom severity is rated on a standardized 0-10 scale, with nasal congestion, hearing loss, and neck discomfort weighted at 0.5, 0.3, and 0.2, respectively. The Pearson correlation coefficient is calculated based on the ratio of the covariance to the standard deviation of each set of values. Volumetric measurements are in cubic millimeters, and nasopharyngeal ventilation scores are obtained via nasopharyngoscopy using a Brodsky grading system of 0-4.

[0056] Preferably, the gradient descent method uses an adaptive learning rate in its iterative adjustment process. The initial learning rate is 0.01, and it is halved every 10 iterations if the loss function decreases by less than a threshold. The loss function is defined as the mean squared error between the predicted value and the actual clinical performance, plus a regularization term. The convergence condition is set as the weight change being less than 0.001 for five consecutive iterations or reaching the maximum number of iterations (100).

[0057] Understandably, the modality weighting matrix is ​​a 3×3 matrix, with rows corresponding to three input modalities (image, text, and physical signs) and columns corresponding to three output feature types (spatial, morphological, and clinical). Each element in the matrix represents the contribution weight of a specific input modality to a specific output feature, with weight values ​​between 0 and 1, and the sum of the weights in each column is 1. Furthermore, the compensation strategy for missing data is dynamically adjusted based on the missing pattern. For consecutively missing image slices, the system uses linear interpolation to calculate the missing portion in the middle based on the feature values ​​of the preceding and following complete slices. For scattered missing text fields, the system uses the statistical mean of similar cases as the imputation value. The calculation of the compensation value considers the spatiotemporal continuity of the data; the mean of adjacent complete data is multiplied by an attenuation coefficient to obtain the final compensation value, which is determined based on the distance between the missing location and the complete data.

[0058] For example, in processing the case of a 5-year-old child, CT images showed some missing data due to artifacts caused by a metal implant, while the clinical record showed severe nasal obstruction and hearing loss. The system first extracted the lesion volume from the complete image area, e.g., 2800 cubic millimeters, with a mean edge gradient of 45. Text features showed a nasal obstruction score of 9 and a hearing loss of 25 decibels. Through cross-validation, the correlation coefficient between volume and the degree of nasal obstruction was 0.78, exceeding the threshold of 0.7, and the corresponding weight was increased to 0.85. The correlation coefficient between edge gradient and symptom description was 0.62, slightly below the threshold, and the weight was adjusted to 0.65. The final fused validated lesion information included: spatial coordinates indicating the lesion was located on the right side of the posterior wall of the nasopharynx; a malignancy score of 6.5; and a clinical correlation of 0.82, suggesting that the lesion was highly correlated with the child's symptoms and required close monitoring.

[0059] Step S106: For verifying lesion information, transfer learning is used to analyze intermodal dependencies across age groups, handle missing image segments in data-sparse scenarios, and generate complete lesion descriptions.

[0060] The feature sets of nasopharyngeal carcinoma lesions in children of different age groups were used as the source task pre-training feature library for transfer learning. The source task was to learn the accumulated typical lesion features of each age group, and the target task was to process lesions with sparse data or missing images. To verify lesion information, the morphological parameters, density distribution values, and growth rate data of typical lesions in early childhood, preschool age, and school age were extracted from the constructed feature sets of nasopharyngeal carcinoma lesions in children of different age groups. The cosine similarity between the current lesion feature vector and the feature vectors of each age group was calculated. The age group with the highest similarity was selected, and the feature mean and variance of the group were extracted as reference ranges to obtain the age feature mapping relationship. Based on the age feature mapping relationship, an intermodal dependency graph is constructed. The source task features of the transfer are used to optimize the calculation of modal dependency strength. Feature units from image, text, and physical characteristics are used as graph nodes. Edge weights are assigned based on the mutual information values ​​between features to represent dependency strength. Feature propagation is achieved by multiplying the adjacency matrix and the feature matrix. If a node has sparse data, feature values ​​are aggregated from neighboring nodes with weights greater than a threshold. For each additional hop in the dependency path, the weight is multiplied by a preset attenuation factor to obtain a cross-modal feature completion matrix. This cross-modal feature completion matrix is ​​used to process image gaps. For consecutively missing slices, grayscale values ​​and texture features are extracted from the preceding and following complete slices. Combined with the predicted values ​​at corresponding positions in the completion matrix, the pixel values ​​of the missing slices are calculated using a weighted average. For scattered missing regions, image blocks with the highest texture similarity from the complete regions of the same slice are selected for copying and filling, resulting in reconstructed complete image data. By integrating the reconstructed complete image data with the original verified lesion information, the data is organized according to six dimensions: lesion location, size, shape, edge, density, and enhancement. The location coordinates are converted into anatomical localization descriptions, the volume values ​​are converted into diameter ranges, the morphological parameters are converted into regular or irregular descriptions, and the edge gradients are converted into clear or blurred descriptions, thus forming a complete lesion description.

[0061] Specifically, in one implementation, the age feature mapping relationship is constructed based on a large-scale pediatric nasopharyngeal carcinoma case database. This database is divided into three age groups according to children's growth and development patterns: the early childhood stage includes lesion data from children aged 2-5 years; the preschool stage covers children aged 6-8 years; and the school-age stage includes children aged 9-14 years. The feature set for each age group includes lesion morphological parameters, density distribution values, and growth rate data. Morphological parameters cover five dimensions: major axis, minor axis, volume, roundness, and eccentricity. Density distribution values ​​record the mean, standard deviation, and kurtosis of CT values. The growth rate is calculated by comparing images at different time points to determine the average monthly percentage increase.

[0062] Specifically, the calculation of cosine similarity involves the standardization of feature vectors. The feature vector of the current lesion is first standardized using Z-scores, ensuring that each dimension has zero mean and unit variance. The reference feature vectors for each age group are also standardized, and then the dot product of the two vectors is calculated and divided by the product of their magnitudes. The similarity value ranges from 0 to 1, with values ​​closer to 1 indicating greater similarity. The system selects the age group with the highest similarity, extracts the mean of all case features in that group as the central reference value, and uses the variance as the allowable deviation range. These statistics constitute the core of the age feature mapping relationship.

[0063] It should be noted that the construction of the intermodal dependency graph adopts the basic principles of graph neural networks. The nodes in the graph correspond to feature units of the three modalities: the image modality contains 256 nodes corresponding to various dimensions of the feature vector; the text modality contains 128 nodes corresponding to the semantic vector dimension; and the vital signs modality contains 64 nodes corresponding to physiological parameters. Edges between nodes are determined through mutual information calculation, which reflects the degree of interdependence between two feature variables. Specifically, the joint probability distribution and marginal probability distribution of each pair of features are first estimated, and then the difference between the joint entropy and the marginal entropy is calculated. Edges are established between node pairs with mutual information values ​​greater than 0.3, and the weight of the edge is equal to the normalized mutual information value. This construction method ensures that dependencies are only established between features with strong correlations.

[0064] For example, the feature propagation process is implemented through matrix multiplication of the adjacency matrix and the feature matrix. The adjacency matrix A is a 448×448 sparse matrix, where non-zero elements represent the connection weights between nodes. The feature matrix F has a dimension of 448×1 and contains the current feature values ​​of all nodes. The first propagation calculates A×F to obtain the aggregated features of first-order neighbors, and the second propagation calculates A²×F to obtain the contributions of second-order neighbors. For each additional hop, the propagation weights are multiplied by a decay factor of 0.8 to prevent excessive influence from distant nodes. For nodes with sparse data, their feature values ​​are initially 0, and features are aggregated from neighboring nodes through three propagation iterations. The final value is a weighted sum of the propagation results at each order.

[0065] In one possible implementation, the processing of consecutively missing slices employs an interpolation method based on spatiotemporal continuity. The system first identifies the range of consecutively missing slice numbers, and then extracts three complete slices before and after the missing segment. The features extracted from each complete slice include the average gray value of the lesion region, texture complexity, and edge sharpness. The gray value is calculated directly from the pixel intensity, the texture complexity is characterized by the entropy value of the gray-level co-occurrence matrix, and the edge sharpness is reflected by the standard deviation of the gradient magnitude.

[0066] Preferably, the predicted values ​​in the cross-modal feature completion matrix are determined through weighted fusion. For the i-th missing slice, its predicted gray value Gi = w1 × Gi-1 + w2 × Gi+1 + w3 × Mi, where Gi-1 and Gi+1 are the gray values ​​of the preceding and following complete slices, respectively, Mi is the predicted value at the corresponding position in the completion matrix, and the weights w1, w2, and w3 are determined based on the inverse distance ratio and sum to 1. This ternary fusion method considers both spatial continuity and utilizes cross-modal information.

[0067] Understandably, filling scattered missing areas requires more refined processing. The system constructs 5×5 pixel sliding windows within the complete area of ​​the same slice, calculating the texture similarity between each window and the surrounding missing area. Texture similarity is measured by the chi-square distance of a local binary pattern histogram; the smaller the distance, the more similar the textures. The top three image patches with the highest similarity are selected, and their median pixel values ​​are used as the filling value. This method maintains the consistency of local texture. Furthermore, the tissue described for complete lesions follows a standardized medical report format. Anatomical localization of location coordinates is converted according to the nasopharyngeal regional standard, such as Z: 45-52mm corresponding to "posterior wall of the nasopharynx," and Y: 28-40mm corresponding to "pharyngeal recess region." Volume values ​​are converted to three radial lengths using the ellipsoid formula, such as 2800 cubic millimeters being converted to "maximum diameter approximately 18mm." Morphological parameters are described using qualitative terminology: roundness greater than 0.7 is described as "quasi-circular," 0.4-0.7 as "elliptical," and less than 0.4 as "irregular."

[0068] For example, in processing a case of a 4-year-old child, the lesion information was verified to show spatial coordinates of 45-50mm in the axial plane, 30-38mm in the coronal plane, and 35-42mm in the sagittal plane, with a malignancy score of 6.8 and a clinical correlation of 0.85. The system first extracted reference data from the early childhood feature set and found that the similarity between the current lesion and this age group was 0.82. The constructed dependency graph showed that the dependency weight between image density features and clinical nasal congestion symptoms was 0.75. Due to the missing data in the 47mm slice, the system extracted gray values ​​135 and 142 from the 46mm and 48mm slices, combined with the completion matrix prediction value of 138, and calculated the fill value of 139. The final complete lesion description was: "The lesion is located on the right side of the posterior wall of the nasopharynx, in the pharyngeal recess region, with a maximum diameter of approximately 16mm, an elliptical shape, unclear edges, uneven density, mild enhancement, and is considered highly malignant, closely related to clinical symptoms." This structured description provides clinicians with clear and complete diagnostic evidence.

[0069] Step S107: Generate a structured diagnostic report template based on the complete lesion description, incorporate the automatic classification results of age attributes, and generate a multimodal diagnostic report for small lesions of nasopharyngeal carcinoma in children.

[0070] Based on the complete lesion description, six key diagnostic elements—lesion location, size, shape, margin, density, and enhancement—were extracted. Following the standard radiology diagnostic report format, the report was organized into three sections: Examination Findings, Imaging Manifestations, and Diagnostic Opinion. The Examination Findings section was filled with the child's name, age, examination date, and scan sequence parameters. The Imaging Manifestations section inserted the location, shape, and density characteristics of the lesion, resulting in a preliminary report framework. The age-based classification results were embedded into the Diagnostic Opinion section of this preliminary report framework. Depending on the child's age, if it is early childhood, differentiation from adenoid hypertrophy should be considered, considering imaging features such as nasopharyngeal soft tissue thickening, which may be accompanied by recurrent nasal congestion and snoring. If it is preschool age, the possibility of chronic nasopharyngeal inflammation or allergic rhinitis should be considered, considering the indistinct borders of small lesions on imaging, which may be accompanied by increased nasal discharge and mild hearing loss. If it is school age, the risk of cervical lymph node metastasis should be assessed, considering the enhancement characteristics of the lesions on imaging, which may be accompanied by neck masses and tinnitus. By integrating descriptions of imaging features with clinical symptoms, a multimodal diagnostic report on small lesions of nasopharyngeal carcinoma in children is output, containing complete imaging information and age-specific diagnostic suggestions.

[0071] Specifically, in one implementation, the radiology diagnostic report standard requires that the report content be organized into three standard chapters. The "Examination Findings" chapter records objective scan information, including the child's basic information such as name, gender, age accurate to the month, examination date and time, and technical parameters such as scan sequence type, slice thickness, and contrast agent dosage. The "Imaging Findings" chapter describes the imaging characteristics of the lesion in detail, proceeding from overall to local, and from localization to characterization. The "Diagnostic Opinion" chapter synthesizes the imaging findings and clinical information, providing a diagnostic conclusion and recommendations.

[0072] It should be noted that the inclusion of age-specific content needs to consider the anatomical characteristics of children at different developmental stages. Physiological adenoid hypertrophy is common in early childhood, peaking between 2 and 6 years of age; therefore, the diagnostic opinion should clearly state "malignancy is suspected, differentiation from physiological adenoid hypertrophy is necessary, and nasopharyngoscopy is recommended." School-aged children have a relatively developed lymphatic system and a higher risk of cervical lymph node metastasis; therefore, the diagnostic opinion should include a suggestion to "perform a contrast-enhanced cervical CT scan to assess lymph node status."

[0073] Specifically, the age group adjustment of medical terminology involves a change in expression. In adult reports, "space-occupying lesion" is changed to "abnormal tissue proliferation" in children's reports, and "infiltrative growth" is changed to "unclear-bordered lesion expansion," making it easier for parents to understand the condition.

[0074] The specific embodiments described above further illustrate the purpose, technical solution, and beneficial effects of the present invention. It should be understood that the above description is only a specific embodiment of the present invention and is not intended to limit the scope of protection of the present invention. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the scope of protection of the present invention.

Claims

1. A method for generating multimodal nasopharyngeal carcinoma reports under modality loss, characterized in that, The method includes: The original image sequences and auxiliary clinical descriptions of pediatric nasopharyngeal carcinoma patients were obtained from imaging equipment and text records through a multimodal data acquisition module. Convolutional neural networks were used to model age attributes based on physiological differences in children and automatically classify and label data to obtain a preliminary multimodal dataset. The image enhancement method is used to process the noise and blurred regions in the original image sequence from the preliminary multimodal dataset. The incomplete parts are identified by combining missing pattern detection and mask region accurate localization to obtain the enhanced image sequence. Convolutional neural networks are used to extract edge and volume features of lesion regions from enhanced image sequences, and differential processing is performed based on age stratification to determine candidate lesion regions. For candidate lesion regions, if there are parts with unclear boundaries, the mask is dynamically adjusted through an attention mechanism to focus on small-volume features and local details, and age-related features are combined for screening. If the boundaries are clear, the candidate lesion region is directly used to determine the location of potential lesions. The location of potential lesions is obtained by combining auxiliary clinical descriptions, edge features and volume features with attention mechanism multimodal fusion weight cross-validation. Incomplete parts are processed by cross-modal information compensation and missing data weight allocation to obtain the validated lesion information. To verify lesion information, transfer learning was used to analyze intermodal dependencies across age groups, and missing image segments were handled in data-sparse scenarios to generate complete lesion descriptions. A structured diagnostic report template is generated based on a complete lesion description, and the results of automatic classification based on age attributes are incorporated to generate a multimodal diagnostic report for small lesions of nasopharyngeal carcinoma in children.

2. The method for generating multimodal nasopharyngeal carcinoma reports under modality loss according to claim 1, characterized in that, The process involves acquiring raw image sequences and auxiliary clinical descriptions of pediatric nasopharyngeal carcinoma patients from imaging equipment and text records using a multimodal data acquisition module. Based on children's physiological differences, a convolutional neural network is used to model age attributes for automatic classification and data labeling, resulting in a preliminary multimodal dataset, including: The DICOM interface is used to read axial, coronal and sagittal image sequences of the nasopharynx of children from CT and MRI equipment. At the same time, the child's date of birth, first visit time, chief symptoms and past medical history are extracted from the electronic medical record system. The data is aligned according to the scanning parameters in the image header file and the timestamps in the text to obtain the original multimodal data with time matching. For the image sequences in the original multimodal data, a ResNet-50 convolutional neural network was used to extract three types of physiological features for each slice: the degree of ossification of the nasopharyngeal bones, the distribution of mucosal thickness, and the size of the adenoids. The correspondence between the feature values ​​and the age range was calculated based on the standard curve of children's growth and development to obtain age-stratified label data. The age-stratified label data is obtained, and the original multimodal data is differentiated according to the nasopharyngeal anatomy characteristics of different age groups. The feature matrix, text vector and age-stratified label data are then fused to construct a preliminary multimodal dataset.

3. The method for generating multimodal nasopharyngeal carcinoma reports under modality loss according to claim 1, characterized in that, The process involves processing noise and blurred regions in the original image sequence from the initial multimodal dataset using image enhancement methods, and combining missing pattern detection and masked region precise location to identify incomplete parts, resulting in an enhanced image sequence, including: Image sequences were extracted from the preliminary multimodal dataset. Gaussian noise in CT slices was processed using a nonlocal mean filtering algorithm. By calculating the weighted average of each pixel and similar structural blocks in its neighborhood, the texture details of the lesion were preserved, resulting in denoised image data. For the denoised image data, the blurred regions are identified by calculating the local gradient magnitude of each slice, and the edge blur is restored by performing deconvolution operation using Wiener filtering in the frequency domain. At the same time, the information entropy value of each slice is detected to mark the missing data positions, thus obtaining the edge restored image and the missing region marking. A binary mask image is generated based on the missing region markers. The missing boundary is expanded by morphological dilation. Gray values ​​of corresponding spatial locations are extracted from the front and back slices of the edge-recovered image. The pixel values ​​of the missing locations are calculated by trilinear interpolation to obtain an enhanced image sequence.

4. The method for generating multimodal nasopharyngeal carcinoma reports under modality loss according to claim 1, characterized in that, The step of extracting edge and volume features of lesion regions from enhanced image sequences using convolutional neural networks, performing differential processing based on age stratification, and determining candidate lesion regions includes: Features are extracted from enhanced image sequences using a VGG-16 convolutional neural network, and a feature map is output. The gradient magnitude is calculated using the Sobel operator to mark potential edge points, and the edge points are connected to form a closed contour to obtain the lesion edge feature map. The lesion edge feature map is differentially optimized according to the age stratification label. For the early childhood label, the first gradient threshold is used to screen potential edge points and strengthen the connection weight of continuous edge points. For the preschool age label, the second gradient threshold is used to screen edge points. For the school age label, the third gradient threshold is used to screen edge points and weaken the edge point weight of the bone structure interference. The age-corrected edge feature map is obtained. The second gradient threshold is higher than the first gradient threshold, and the third gradient threshold is higher than the second gradient threshold. For the internal region of the lesion edge feature map after age correction, the single-layer volume value is calculated and accumulated. The volume calculation weight is adjusted according to the age stratification label. The first preset weight coefficient is used to compensate for the difference in tissue density in early childhood. The third preset weight coefficient is used to adapt to the anatomical characteristics of the mucosa-ossification transition stage in the preschool age. The second preset weight coefficient is used to correct the influence of ossification in school age, so as to obtain the age-corrected volume feature data. Based on the age-corrected volumetric feature data and the age-corrected lesion edge feature map, the roundness, eccentricity, and compactness are calculated, abnormal areas and potential lesion areas are marked, spatial coordinates are merged, and candidate lesion areas are determined.

5. The method for generating multimodal nasopharyngeal carcinoma reports under modality loss according to claim 1, characterized in that, For the candidate lesion region, if there are areas with unclear boundaries, an attention mechanism is used to dynamically adjust the mask to focus on small-volume features and local details, combined with age-related features for screening. If the boundaries are clear, the candidate lesion region is directly used to determine the location of potential lesions, including: For candidate lesion regions, the gray-level gradient difference and gradient direction change angle are calculated to determine the boundary clarity, and the boundary clarity identifier and blurry region distribution map are obtained. If the boundary clarity is marked as blurry, a self-attention mechanism is constructed to calculate attention weights. Attention head parameters are set according to age-level labels. In early childhood, a preset first number of attention heads are used to capture subtle changes. In preschool age, a preset second number of attention heads are used to balance detail capture and main feature extraction. In school age, a preset third number of attention heads are used to focus on main features, resulting in an age-adaptive feature map. The preset first number is greater than the preset second number, which is greater than the preset third number. Based on the age-adaptive feature map, a dynamic binary mask is generated, local feature vectors are extracted, cosine similarity is calculated, and the effective lesion part is fused to obtain a refined boundary. If the boundary clarity indicator is clear, the original lesion candidate region boundary is directly used to calculate the centroid coordinates and spatial bounding box parameters to determine the location of the potential lesion.

6. The method for generating multimodal nasopharyngeal carcinoma reports under modality loss according to claim 1, characterized in that, The process of obtaining potential lesion locations, combined with auxiliary clinical descriptions, edge features, and volumetric features, employs an attention-based multimodal fusion weighted cross-validation mechanism. Incomplete portions are addressed through cross-modal information compensation and weighted allocation of missing data to obtain validated lesion information, including: The three-dimensional coordinate data of potential lesion locations are obtained. Auxiliary clinical descriptions, including the degree of nasal congestion, runny nose, hearing loss records, and neck palpation results in the chief complaint, are extracted from the electronic medical record. At the same time, the gradient distribution data in the edge feature map and the spatial occupancy information in the volume feature data are read. The text description is converted into a semantic vector through word embedding. The semantic vector is aligned with the image feature vector in dimension to obtain a multimodal feature set. For the aforementioned multimodal feature set, a cross-modal attention matrix is ​​constructed using a Transformer encoder. The correlation score between the text modality and the image modality is calculated using a multi-head attention mechanism. Feature values ​​at corresponding positions are extracted from other modalities as compensation. The correlation score is normalized using a softmax function to obtain fusion weights. The image features and text features are weighted and summed according to the fusion weights to obtain the fused feature representation. Cross-validation is performed based on the fused feature representation, and the correlation coefficient is calculated to adjust the contribution of each mode, resulting in a mode weight allocation matrix. Based on the modality weight allocation matrix, the multi-source features are recombined, and the spatial coordinates of the lesion, the malignancy score, and the clinical correlation are fused to obtain the verification lesion information.

7. The method for generating multimodal nasopharyngeal carcinoma reports under modality loss according to claim 1, characterized in that, The method employs transfer learning to analyze inter-modal dependencies across age groups to verify lesion information, handles missing image segments in data-sparse scenarios, and generates complete lesion descriptions, including: The feature set of nasopharyngeal carcinoma lesions in children of each age group is used as the source task pre-training feature library for transfer learning. For verifying lesion information, the cosine similarity between the current lesion feature vector and the feature vector of each age group is calculated. The age group with the highest similarity is selected to obtain the age feature mapping relationship. Based on the age feature mapping relationship, an intermodal dependency graph is constructed. The source task features of the transfer are used to optimize the calculation of modal dependency strength. Feature propagation is achieved by multiplying the adjacency matrix and the feature matrix to obtain the cross-modal feature completion matrix. The cross-modal feature completion matrix is ​​used to process image missing data, and the pixel values ​​of the missing slices are calculated by weighted averaging to obtain reconstructed complete image data; By integrating the reconstructed complete image data with the original verified lesion information, the data is organized according to six dimensions: lesion location, size, shape, edge, density, and enhancement, and combined to form a complete lesion description.

8. The method for generating multimodal nasopharyngeal carcinoma reports under modality loss according to claim 1, characterized in that, The process involves generating a structured diagnostic report template based on a complete lesion description, incorporating automatic age-based classification results, and generating a multimodal diagnostic report for small lesions in pediatric nasopharyngeal carcinoma, including: Based on the complete lesion description, extract six key diagnostic elements: lesion location, size, shape, margin, density, and enhancement. Organize these elements into three sections: examination findings, imaging manifestations, and diagnostic opinions. Fill in the child's name, age, examination date, and scan sequence parameters, and insert the content of the lesion description to obtain a preliminary report framework. The age attribute classification results are embedded into the diagnostic opinion section of the preliminary report framework. Differential diagnosis prompts for the corresponding age group are added. If it is early childhood, adenoid hypertrophy is added as a differential diagnosis prompt. If it is preschool age, chronic nasopharyngitis and early tumor differentiation prompts are added. If it is school age, cervical lymph node metastasis risk assessment is added. Medical terminology is adjusted, and the correlation between imaging feature descriptions and clinical symptoms is integrated to output a multimodal diagnostic report for small lesions of nasopharyngeal carcinoma in children.

Images