Chest radiograph image recognition method and device

Abstract

The invention provides a chest radiograph image recognition method and device, and relates to the technical field of image processing. The method comprises the following steps: acquiring a chest radiograph image to be recognized; inputting the chest radiograph image into a pre-trained identification model to obtain an identification probability of each preset disease tag corresponding to the chest radiograph image; the recognition model is used for performing recursive multi-scale feature division and spatial reconstruction on the chest radiograph image, and performing interactive updating between a preset disease tag and features in the feature division process of each scale level to obtain tag semantics with global perception; and determining the recognition probability of each preset disease tag corresponding to the chest radiograph image based on the tag semantics with global perception. According to the invention, the recognition precision of complex focuses with variable scales and different shapes in the chest radiograph can be improved.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of image processing, and in particular to a chest radiograph image recognition method and device. BACKGROUND

[0002] As an important task in medical image analysis, chest radiograph image recognition has been widely applied in intelligent screening and clinical auxiliary diagnosis of lung diseases. The chest radiograph image recognition process is mainly a multi-label classification process of chest radiograph images. With the publication of large-scale public medical image datasets such as CheXpert and ChestX-ray14, multi-label classification models based on deep learning have made significant progress in chest radiograph image recognition, providing a new technical path for the automatic screening of clinical lung diseases.

[0003] In related technologies, multi-label learning, attention mechanisms, Transformer structures, and graph neural networks are mostly used to achieve multi-label classification of chest radiograph images. In the research of multi-label chest radiograph image classification, how to accurately identify complex lesions of varying scales and different shapes in chest radiograph images has always been a core challenge. Early models based on convolutional neural networks, such as CheXNet and its subsequent improvements, mainly rely on deep convolutional backbone networks (such as DenseNet121) to extract global image features, and consider the multi-label classification task as multiple independent binary classification problems. Although this method achieves preliminary performance in chest radiograph image recognition, its fixed receptive field cannot adaptively capture discriminative features of different scale lesions from small nodules to large exudates.

[0004] To overcome the above limitations, researchers have introduced multi-scale feature fusion and attention mechanisms. For example, dual-branch network structures attempt to extract global and local features in parallel, and various attention models (such as DualAttNet) try to enhance the model's attention to suspicious areas by calculating spatial weight maps. These methods have enriched the perception of lesions to some extent. However, the inventors have found through research that the above methods still rely on pure visual signal driving and lack the use of high-level semantic priors. Their process usually separates image encoding and label prediction, i.e., first extracting visual features and then fusing or calculating the relevance of label embeddings. This "visual first, semantic second" paradigm makes it difficult for the model to achieve accurate visual-label alignment when dealing with complex scenarios where visual manifestations are ambiguous, different lesions have significant semantic co-occurrence or exclusion, thereby limiting its final recognition accuracy.

[0005] In summary, the existing chest radiograph image recognition technology generally has the problem of low recognition accuracy for complex lesions of varying scales and different shapes. SUMMARY

[0006] Embodiments of the present application provide a chest radiograph image recognition method and device to solve the problem of low recognition accuracy of complex lesions with varying scales and shapes in chest radiograph images in the related art.

[0007] In a first aspect, the embodiments of the present application provide a chest radiograph image recognition method, comprising: obtaining a chest radiograph image to be recognized; inputting the chest radiograph image into a pre-trained recognition model to obtain recognition probabilities of each preset disease label corresponding to the chest radiograph image; The recognition model is configured to: perform recursive multi-scale feature division and spatial reconstruction on the chest radiograph image, and perform interactive updating between preset disease labels and features in the feature division process at each scale level to obtain label semantics with global perception; and determine the recognition probabilities of each preset disease label corresponding to the chest radiograph image based on the label semantics with global perception.

[0008] In a possible implementation, the recursive multi-scale feature division and spatial reconstruction on the chest radiograph image, and the interactive updating between the preset disease labels and the features in the feature division process at each scale level to obtain the label semantics with global perception, comprise: determining a current scale level based on a current iteration number, and performing feature division on the chest radiograph image according to the current scale level to obtain a visual block sequence; obtaining a label embedding matrix corresponding to the preset disease label at the current iteration number, and performing interactive updating on the label embedding matrix and the visual block sequence to obtain an updated label embedding matrix and visual block sequence; reconstructing the updated visual block sequence to obtain a reconstructed chest radiograph image; determining a label embedding matrix corresponding to the preset disease label at the next iteration based on the updated label embedding matrix; updating the iteration number, and taking the reconstructed chest radiograph image as a new chest radiograph image to jump to the step of determining the current scale level based on the current iteration number, until a set iteration number is reached to obtain the label embedding matrix corresponding to the preset disease label at the set iteration number; determining the label embedding matrix corresponding to the preset disease label at the set iteration number as the label semantics with global perception.

[0009] In a possible implementation, the interactive updating on the label embedding matrix and the visual block sequence to obtain the updated label embedding matrix and visual block sequence comprises: splicing the label embedding matrix and the visual block sequence to obtain a cross-modal input sequence; updating the cross-modal input sequence by using an encoder to obtain an updated label embedding matrix and a visual block sequence.

[0010] In a possible implementation, the determining, based on the updated label embedding matrix, of the label embedding matrix corresponding to the preset disease label in the next iteration includes: linearly mapping the updated label embedding matrix to obtain the label embedding matrix corresponding to the preset disease label in the next iteration.

[0011] In a possible implementation, the feature division of the chest radiograph image according to the current scale level to obtain the visual block sequence includes: determining the size of the visual block based on the current scale level; the higher the scale level, the smaller the size of the visual block; dividing the chest radiograph image according to the size by using a soft division strategy to obtain a plurality of visual blocks; mapping the plurality of visual blocks to obtain the visual block sequence.

[0012] In a possible implementation, the determining, based on the label semantics with global awareness, of the recognition probability of each preset disease label corresponding to the chest radiograph image includes: obtaining patient attribute values of a patient corresponding to the chest radiograph image in each preset attribute dimension; determining a node vector based on the label semantics with global awareness; determining an adjacency matrix based on the patient attribute values; performing graph convolution operation based on the node vector and the adjacency matrix to determine the recognition probability of each preset disease label corresponding to the chest radiograph image.

[0013] In a possible implementation, the determining, based on the patient attribute values, of the adjacency matrix includes: for each patient attribute value, determining a label co-occurrence matrix corresponding to the patient attribute value based on a preset label co-occurrence set; the label co-occurrence set includes label co-occurrence matrices corresponding to different patient attribute values in each preset attribute dimension; determining the label co-occurrence matrices corresponding to the patient attribute values as the adjacency matrix.

[0014] In a possible implementation, before the determining, based on the patient attribute values, of the adjacency matrix, the method further includes: dividing a chest radiograph image sample set based on each preset attribute dimension and a corresponding patient attribute value range of the preset attribute dimension to determine a sample subset of each preset attribute dimension in each patient attribute value; For each sample subset of each preset attribute dimension at each patient attribute value, the co-occurrence frequency corresponding to each two preset disease labels is counted respectively. The co-occurrence frequency is normalized and sparsified to determine the label co-occurrence matrix of the preset attribute dimension at the patient attribute value.

[0015] In a possible implementation, the graph convolution operation is performed based on the node vector and the adjacency matrix to determine the recognition probability of each preset disease label corresponding to the chest radiograph, including: The graph convolution operation is performed based on the node vector and the adjacency matrix to obtain an attribute-enhanced label embedding matrix. The label joint vector is obtained by fusing the label semantics with global perception and the attribute-enhanced label embedding matrix. The recognition probability of each preset disease label corresponding to the chest radiograph is determined based on the label joint vector.

[0016] In a second aspect, the embodiments of the present application provide a chest radiograph recognition device, including a memory and a processor, the memory stores a computer program, and the processor implements the method in the first aspect or any possible implementation manner of the first aspect when executing the computer program.

[0017] In the embodiments of the present application, the recognition model can obtain local lesion structure features of different fine granularities in the image through the recursive multi-scale feature division and spatial reconstruction mechanism, thereby effectively enhancing the expression ability of the model to lesion regions of different scales (such as micro-nodules and large-scale exudation). While retaining global structure information, the expression of fine-grained lesions is strengthened, thereby improving the recognition ability of the model to heterogeneous lesions of different scales; and in the feature division process, the disease label is introduced, the disease label is embedded into the visual coding process (i.e., the feature division process) as prior semantic signals, and through the interaction between the label and the feature, high-order cross-modal alignment between the image features and the semantic labels can be realized, the consistency between the visual features (i.e., the lesion structure features in the image) and the disease labels is improved, and finally the purpose of improving the recognition accuracy of complex lesions is achieved. BRIEF DESCRIPTION OF DRAWINGS

[0018] Figure 1 is an implementation flowchart of the chest radiograph recognition method provided by the embodiments of the present application; Figure 2 is an implementation flowchart for determining the label semantics with global perception provided by the embodiments of the present application; Figure 3 is a processing flowchart of the MFPR module in the recognition model provided by the embodiments of the present application; Figure 4is an implementation flowchart of the A-GCN module provided by the embodiment of the present application for determining the recognition probability of each preset disease label corresponding to the chest radiograph image; Figure 5 is an image processing flowchart of the recognition model provided by the embodiment of the present application; Figure 6 is a structural schematic diagram of the chest radiograph image recognition device provided by the embodiment of the present application. DETAILED DESCRIPTION

[0019] The embodiment of the present application will be described in detail below with reference to the accompanying drawings.

[0020] In the related art, when a convolutional neural network is used to recognize a chest radiograph image (i.e., to perform multi-label classification on the chest radiograph image), it is difficult to adaptively capture the lesion features of different scales from a small nodule to a large range of exudation. Although researchers use multi-scale feature fusion and attention mechanisms to enrich the lesion perception ability in the chest radiograph image to some extent, the present inventors have found through research that this approach still essentially relies on pure visual signal driving and lacks the use of high-level semantic priors. This "visual first, semantic second" paradigm makes it difficult for the model to achieve accurate visual-label alignment when dealing with complex lesions with ambiguous visual manifestations, significant semantic co-occurrence or exclusion between different lesions, thereby limiting the final recognition accuracy.

[0021] To improve the recognition accuracy of complex lesions with varying scales and different morphologies in the chest radiograph image, the embodiment of the present application uses a recognition model to recognize the chest radiograph image. The recognition model can obtain local lesion structure features of different fine granularities in the image through a recursive multi-scale feature division and spatial reconstruction mechanism, thereby effectively enhancing the expression ability of the model for different scale lesion regions (such as a small nodule and a large range of exudation), and improving the recognition ability of the model for different scale heterogeneous lesions. In addition, the present application introduces disease labels in the feature division process, embeds the disease labels as prior semantic signals into the visual coding process (i.e., the feature division process), and through the interaction and update between the labels and the features, achieves high-order cross-modal alignment between the image features and the semantic labels, improves the consistency between the visual features (i.e., the lesion structure features in the image) and the disease labels, and ultimately achieves the purpose of improving the recognition accuracy of complex lesions.

[0022] Referring to Figure 1 which shows an implementation flowchart of the chest radiograph image recognition method provided by the embodiment of the present application, and is described in detail as follows: Step 101, obtaining a chest radiograph image to be recognized.

[0023] Here, the chest radiograph image can be an X-ray image, a CT image, or a nuclear magnetic resonance image taken for a chest cavity region. Based on obtaining the chest radiograph image to be identified, the embodiment of the present application can perform preprocessing operations such as image size normalization and grayscale standardization on the chest radiograph image, so as to improve the chest radiograph image quality and enhance the recognition accuracy of the recognition model. The stability and generalization ability of the recognition model are enhanced.

[0024] In step 102, the chest radiograph image is input into the pre-trained recognition model to obtain the recognition probability of each preset disease label corresponding to the chest radiograph image.

[0025] The embodiment of the present application can input the preprocessed chest radiograph image into the pre-trained recognition model, and the recognition model can output the recognition probability of each preset disease label corresponding to the chest radiograph image. In essence, the recognition process of the chest radiograph image is a multi-label classification process of the chest radiograph image. The recognition model in the embodiment of the present application is a multi-label classification model for the chest radiograph image.

[0026] The recognition model is used to: perform recursive multi-scale feature division and spatial reconstruction on the chest radiograph image, and perform interactive updating between the preset disease label and the feature in the feature division process at each scale level to obtain a label semantic with global perception; based on the label semantic with global perception, determine the recognition probability of each preset disease label corresponding to the chest radiograph image.

[0027] Here, the recursive multi-scale feature division and spatial reconstruction can be simply understood as iteratively performing the “feature division-spatial reconstruction” operation. Here, it should be noted that the scale level corresponding to the feature division process in each iteration is different, and the more the number of iterations, the higher the scale level. Here, the scale level is used to represent the feature division fineness in the feature division process, and the higher the scale level, the higher the feature division fineness.

[0028] The recognition model adopts the recursive “feature division-spatial reconstruction” mechanism, which can gradually extract and fuse image representations of different semantic granularities at each scale level, thereby effectively capturing local structure and global context information and improving the perception ability of the model to different size lesion regions.

[0029] Moreover, in each iteration process, the embodiment of the present application can perform interactive updating between the divided feature and the preset disease label, thereby realizing multi-level fusion of the disease label in the visual feature space, so that the disease label can be aligned with image features of different scales, and finally realizing high-order cross-modal alignment between image features and label semantics, and obtaining a label semantic with global perception.

[0030] The recognition model can perform graph convolution operation based on the label semantics with global perception, thereby realizing multi-label classification of the chest image, obtaining recognition probabilities of each preset disease label corresponding to the chest image, and determining the lesion type in the chest image.

[0031] It can be understood that the recognition model in the embodiments of the present application can also be used for multi-label classification of medical images of other parts. For example, skin images, torso images, etc. The types of images include, but are not limited to, X-ray images, CT images, and nuclear magnetic resonance images, etc.

[0032] The recognition model in the embodiments of the present application can obtain local lesion structure features of different fine granularities in the image through the recursive multi-scale feature partitioning and spatial reconstruction mechanism, thereby effectively enhancing the expression ability of the model to different scale lesion regions (such as small nodules and large range exudation). While preserving the global structure information, the expression of fine-grained lesions is strengthened, thereby improving the recognition ability of the model to heterogeneous lesions of different scales; and the disease label is introduced in the feature partitioning process, the disease label is embedded into the visual coding process (i.e. feature partitioning process) as prior semantic signal, through the interaction update between the label and the feature, the high-order cross-modal alignment between the image features and the semantic labels can be realized, the consistency between the visual features (i.e. lesion structure features in the image) and the disease labels is improved, and finally the purpose of improving the recognition accuracy of complex lesions is achieved.

[0033] The following will be described in combination with Figures 2 to 5 The processing process of the recognition model for the chest image will be described.

[0034] The recognition model has a multi-scale feature partitioning and reconstruction (MFPR) module and a label-guided multi-scale semantic alignment (LMSA) module built-in. After receiving the preprocessed chest image, the MFPR module performs recursive multi-scale feature partitioning and spatial reconstruction on the chest image. In addition, the LMSA module performs interactive update between the preset disease label and the feature in the feature partitioning process at each scale level, to obtain label semantics with global perception. Referring to Figure 2 The specific process is as follows: Step 201, based on the current iteration number, determine the current scale level, and perform feature partitioning on the chest image according to the current scale level, to obtain a visual block sequence.

[0035] In the recursive multi-scale feature partitioning and spatial reconstruction process, the more iterations and the higher the scale level, the more refined the feature partitioning. In each feature partitioning process, the MFPR module can first determine the current scale level based on the current iteration number, and then perform partitioning based on the current scale level to obtain a visual block sequence.

[0036] In some embodiments, the size of the visual blocks can be determined based on the current scale level; then, a soft partitioning strategy is used to divide the chest X-ray image according to the size to obtain multiple visual blocks; finally, the multiple visual blocks are mapped to obtain a visual block sequence.

[0037] Here, the size of the visual block is used to characterize the fineness of feature segmentation. The smaller the size of the visual block, the higher the fineness of feature segmentation. In this embodiment of the invention, the higher the scale level, the smaller the size of the visual block.

[0038] For the Chest X-ray image at the next iteration .in, , and These represent the height, width, and number of channels of the chest X-ray image, respectively. The MFPR module uses a soft partitioning strategy to divide... It is divided into multiple overlapping visual blocks. The size of each visual block is [size missing]. Step size is The edge regions are filled with zero padding. After partitioning, the first... The total number of visual blocks at the next iteration is:

[0039] in, Indicates the first The total number of visual blocks in the next iteration This indicates the number of pixels that are zero-padded. The value of decreases continuously as the scale level increases.

[0040] Here, each visual block can be mapped to a fixed dimension. The vectors are used to form a sequence of visual blocks. This ultimately yields the first... All visual patch sequences generated in the next iteration For example, embodiments of the present invention can utilize the flatten operation to map each visual block into a vector of fixed dimensions.

[0041] Step 202: Obtain the label embedding matrix corresponding to the preset disease label at the current iteration number, and interactively update the label embedding matrix and visual block sequence to obtain the updated label embedding matrix and visual block sequence.

[0042] The preset disease label can be represented as: . Wherein, represents a preset disease label set, , and respectively represent different preset disease labels, represents the total number of preset disease labels.

[0043] The embodiment of the application can use a pre-trained language model (for example, a GloVe model) to perform semantic coding on each preset disease label to obtain an initial label embedding matrix:

[0044] Wherein, represents the initial label embedding matrix, that is, the label embedding matrix corresponding to the first iteration, , and respectively represent the embedding vectors of different preset disease labels (that is, the label semantics used to embed the visual block sequence), , represents the embedding dimension.

[0045] It should be noted that the embedding dimension of the label is the same as the dimension of the visual block sequence, so the same letter parameter is used in the embodiment of the application to represent the embedding dimension and the dimension of the visual block sequence.

[0046] Here, the LMSA module can use the encoder to interactively update the label embedding matrix and the visual block sequence, thereby embedding the label semantics on the basis of the visual features of the current scale level, and realizing the explicit alignment of the label semantics and the visual features.

[0047] In some embodiments, the LMSA module can first splice the label embedding matrix and the visual block sequence to obtain a cross-modal input sequence; then, the encoder is used to encode and update the cross-modal input sequence to obtain an updated label embedding matrix and visual block sequence.

[0048] The cross-modal input sequence corresponding to the first iteration can be represented as: .

[0049] Wherein, represents the cross-modal input sequence corresponding to the first iteration, represents the label embedding matrix corresponding to the first iteration.

[0050] The LMSA module uses an encoder (for example, a Transformer Encoder) to encode and update the cross-modal input sequence The encoding is performed to realize the interactive update between the label embedding matrix and the visual block sequence. The core of the Transformer Encoder is the multi-head self-attention mechanism inside it, which effectively captures the high-order interaction relationship between visual regions, labels and both by modeling the context dependence in multiple subspaces in parallel.

[0051] In the Transformer Encoder, the cross-modal input sequence is first mapped into a query matrix , a key matrix and a value matrix :

[0052] where , and represent the corresponding weight matrix of the query matrix , the corresponding weight matrix of the key matrix , and the corresponding weight matrix of the value matrix .

[0053] Then, the scaled dot-product self-attention mechanism is applied to model the cross-modal context relationship:

[0054] where denotes the self-attention mechanism, denotes the transpose factor.

[0055] Finally, the updated visual and semantic representations (i.e., the visual block sequence and the label embedding matrix) are output by parallel modeling and splicing of multiple attention heads:

[0056] where denotes the updated visual block sequence, which integrates the label semantic context, denotes the updated label embedding matrix, which integrates the visual context, denotes the output result of the self-attention mechanism when is input into the self-attention mechanism.

[0057] This invention incorporates an LMSA module within the recognition model. This module, based on the multi-scale visual features generated by the MFPR module, introduces label semantic embedding to construct a unified cross-modal modeling space. Semantic-aware modeling is then performed using a TransformerEncoder, enabling label-guided explicit alignment and multi-level fusion. The introduction of the LMSA module effectively enhances the recognition model's semantic perception of lesion regions and fully explores the cross-modal dependencies between image regions and label semantics, thereby improving the quality of semantic representation and class discrimination ability in multi-label classification tasks.

[0058] Step 203: Reconstruct the updated visual block sequence to obtain the reconstructed chest X-ray image.

[0059] With the first Taking the next iteration as an example, after completing the interactive update with the label embedding matrix, the visual block sequence Updated to The MFPR module reconstructs the updated visual patch sequence, remapping it to a chest X-ray image to restore the integrity of its spatial structure and local location information, facilitating subsequent iterative segmentation. The reconstruction operation can be expressed by the formula:

[0060] in, This represents the chest X-ray image reconstructed after the Reshape operation, i.e., the first... Chest X-ray image at the next iteration , , and These represent the height, width, and number of channels of the reconstructed chest X-ray image, respectively. This represents the Reshape function. , .

[0061] Step 204: Based on the updated label embedding matrix, determine the label embedding matrix corresponding to the preset disease label in the next iteration.

[0062] To enable the dynamic transfer of semantic information across multiple scales, in some embodiments, the updated label embedding matrix is ​​linearly mapped to obtain the label embedding matrix corresponding to the preset disease label in the next iteration.

[0063] With the first Taking the second iteration as an example, the LMSA module will use the current scale level (i.e., the first iteration) The output of the next iteration After linear mapping, the next scale level (i.e., the ) is generated. The label embedding matrix for the next iteration: .

[0064] in, Indicates the first In the next iteration, a label embedding matrix corresponding to the disease label is preset. This represents the weight matrix corresponding to the linear mapping.

[0065] Step 205: Update the iteration count, and use the reconstructed chest X-ray image as the new chest X-ray image. Jump to the "Determine the current scale level based on the current iteration count" step until the set iteration count is reached, and obtain the label embedding matrix of the preset disease label under the set iteration count.

[0066] The above partitioning and reconstruction process can be executed recursively multiple times, forming the core path of the MFPR module. The MFPR module first processes the initial chest X-ray image... Divided into visual block sequences After interacting with the tag semantics (i.e., the tag embedding matrix), it is reconstructed into This process continues in each subsequent iteration, such as... Figure 3 As shown, the first In this iteration, chest X-ray images Divided into visual block sequences After interactive updates, it was reconstructed into Reconstructed chest X-ray image It was re-divided into visual block sequences Through a recursive process of "segmentation → interaction → reconstruction", the MFPR module can effectively improve the representation ability of lesion areas by extracting and fusing semantic information at different scales layer by layer.

[0067] The MFPR module obtains fine-grained lesion structure features in images through a recursive local region segmentation mechanism, and uses a spatial structure reconstruction strategy to achieve efficient fusion between local and global semantics, thereby effectively enhancing the model's ability to express lesion regions of different scales (such as micro nodules and large-scale exudation).

[0068] Step 206: The tag embedding matrix corresponding to the preset disease tag at a set number of iterations is determined as a tag semantic with global awareness.

[0069] The LMSA module repeats the above label semantic embedding process at different scale levels, enabling multi-level fusion of label semantic information in the visual feature space. This continues until a set number of iterations is reached, yielding the label embedding matrix corresponding to that set number of iterations, which represents globally perceptive label semantics. . This is used to capture structured semantic dependencies between categories, thereby improving the recognition accuracy of the identification model for complex lesions.

[0070] In this embodiment of the invention, the MFPR module obtains fine-grained lesion structure features in the image through a recursive local region partitioning mechanism, and utilizes a spatial structure reconstruction strategy to achieve efficient fusion between local and global semantics, thereby effectively enhancing the model's ability to express lesion regions of different scales (such as micro nodules and large-scale exudations). To improve the consistency modeling effect between visual features and label semantics, this embodiment of the invention introduces a label guidance mechanism in the feature extraction process, designs an LMSA module based on Transformer Encoder, embeds label embedding as a prior semantic signal into the visual encoding process, and explicitly models high-order cross-modal semantic dependencies through a cross-attention mechanism between labels and image visual blocks, thereby achieving joint optimization modeling of visual-semantic information.

[0071] In the multi-scale segmentation and reconstruction process, the embodiments of the present invention combine a label embedding mechanism, which can not only enhance the recognition model's ability to perceive lesion regions at different scales, but also achieve explicit alignment between label semantics and visual features at different scales, enhance the correspondence between label semantics and image visual blocks at different scales, thereby improving the overall recognition accuracy of the recognition model for complex lesions.

[0072] To enhance the recognition model's ability to model multi-label semantic structures, this embodiment of the invention includes an attribute-aware graph convolution (A-GCN) module within the recognition model. This module is used to model the globally aware label semantics described above. Perform graph convolution operations to determine the recognition probability of each preset disease label corresponding to the chest X-ray image.

[0073] Building upon traditional graph convolutional networks, the A-GCN module introduces an attribute mechanism at two levels: label relationship modeling and graph convolution propagation. This allows the graph convolutional network to be dynamically adjusted based on patient attributes, enabling it to adapt to these attributes and ultimately enhancing the recognition model's adaptability and allocation accuracy for heterogeneous populations (i.e., populations with different attributes).

[0074] The following is combined Figure 4 The specific process by which the A-GCN module determines the recognition probability of each preset disease label corresponding to a chest X-ray image is described in detail: Step 401: Obtain the patient attribute values ​​corresponding to the chest X-ray image under each preset attribute dimension.

[0075] Here, the preset attribute dimensions can include multiple attribute dimensions such as gender and age. These preset attribute dimensions can form an attribute space: .

[0076] in, Represents the attribute space. 、 and represent different preset attribute dimensions, for example, gender, age, etc. represents the total number of preset attribute dimensions.

[0077] Each preset attribute dimension contains a plurality of patient attribute values, denoted as .

[0078] wherein, represents the preset attribute dimension corresponding patient attribute value set, and represent different patient attribute values corresponding to the preset attribute dimension . For example, the "gender" dimension can correspond to {male, female}, and the "age" dimension corresponds to {child, young, middle-aged, elderly}. The embodiment of the present application exemplarily divides the patient age into childhood (0-14 years old), youth (15-40 years old), middle age (41-65 years old), and old age (more than 65 years old).

[0079] In the process of identifying the chest radiograph image of the patient, the embodiment of the present application can obtain the patient attribute values of the patient in each preset attribute dimension, for example, the gender of the patient and the age of the patient, etc.

[0080] Step 402, determining a node vector based on the label semantics with global perception.

[0081] In the embodiment of the present application, the label semantics with global perception can be directly used as a node vector for input into an A-GCN module to perform graph convolution operation.

[0082] Step 403, determining an adjacency matrix based on the patient attribute values.

[0083] In the embodiment of the present application, for each patient attribute value, a label co-occurrence matrix corresponding to the patient attribute value can be determined based on a preset label co-occurrence set, and the label co-occurrence matrices corresponding to each patient attribute value are determined as the adjacency matrix.

[0084] Here, the label co-occurrence set includes the label co-occurrence matrices corresponding to each preset attribute dimension under different patient attribute values.

[0085] In multi-label chest X-ray images, structured co-occurrence relationships are prevalent among labels. However, this co-occurrence structure is significantly modulated by individual attributes (such as gender and age), resulting in significant heterogeneity across different attribute subspaces. To characterize this attribute-driven structural variation, this invention constructs a label co-occurrence set, which dynamically determines the label co-occurrence matrix based on patient attribute values, enabling explicit modeling of label dependency structures within attribute contexts.

[0086] In some embodiments, the process of constructing the tag co-occurrence set is as follows: First, based on each preset attribute dimension and its corresponding patient attribute value range, the chest X-ray image sample set is divided to determine the sample subset for each preset attribute dimension under each patient attribute value. Next, for each preset attribute dimension sample subset under each attribute value, the co-occurrence frequency corresponding to each pair of preset disease labels is counted. Finally, the co-occurrence frequency is normalized and sparsified to determine the label co-occurrence matrix of the preset attribute dimension under the patient attribute value.

[0087] Specifically, for each preset attribute dimension and their corresponding patient attribute values The chest X-ray image sample set is divided into those that satisfy... (i.e., preset attribute dimensions) The patient attribute value is (sample subset) And within each sample subset, statistical analysis of arbitrary label pairs is performed. Co-occurrence frequency:

[0088] in, Indicates label pairs Co-occurrence frequency, Indicates the preset attribute dimension. This indicates the values ​​that the patient attributes can take.

[0089] Using the above method, the co-occurrence frequency of all tag pairs can be determined. Normalizing these co-occurrence frequencies yields the tag co-occurrence matrix under attribute conditions: .

[0090] in, Indicates the first element in the label co-occurrence matrix. Line number The element values ​​of the column, i.e., the sample subset Chinese label pair The normalized value of the co-occurrence frequency, .

[0091] Indicates that under given attribute conditions Lower, label Relative to the label The conditional co-occurrence probability characterizes the semantic dependence strength between labels in the attribute context.

[0092] Considering that low-frequency co-occurrence may introduce noise and affect the stability of structure modeling, the embodiment of the application introduces a confidence threshold The elements with conditional co-occurrence probability lower than the confidence threshold are sparsified:

[0093] The above sparsification strategy effectively eliminates non-significant label relationships, improving the structure recognition and robustness of the label co-occurrence matrix. After sparsifying each element in the label co-occurrence matrix, the embodiment of the application obtains the final label co-occurrence matrix.

[0094] Based on the above method, the embodiment of the application can construct a set of label contribution subsets for each preset attribute dimension . Among them, denotes the corresponding label co-occurrence matrix.

[0095] The embodiment of the application determines the final label co-occurrence set by collecting the label co-occurrence subsets corresponding to each preset attribute dimension.

[0096] The embodiment of the application can determine the label co-occurrence matrix of the patient under each preset attribute dimension based on the patient attribute values corresponding to the chest radiograph image to be recognized. In the subsequent graph convolution process, the label co-occurrence matrix under each preset attribute dimension can be used as an adjacency matrix to define the graph structure, thereby realizing personalized regulation of semantic propagation paths and enhancing the context adaptability of label embedding.

[0097] The embodiment of the application pre-constructs a label co-occurrence set according to preset attribute dimensions such as age and gender, and dynamically selects the corresponding label graph path (i.e. adjacency matrix) according to the patient attribute features carried by the chest radiograph image during reasoning, completes the semantic relationship propagation process for individuals, and improves the adaptation ability and classification accuracy of the model in heterogeneous populations Step 404, performing graph convolution operation based on node vector and adjacency matrix to determine the recognition probability of each preset disease label corresponding to the chest radiograph image.

[0098] On the basis of the label co-occurrence matrix constructed under the above attribute condition, an attribute-driven graph convolution network is arranged in the A-GCN module in the embodiment of the application to realize dynamic propagation and structure adaptive modeling of label embedding in different attribute subspaces. Based on the patient attribute values corresponding to the chest image, the network can select the corresponding propagation path in multiple attribute perception-based graph structures, guide the label embedding to be personalized updated according to the context, and thus enhance the recognition accuracy for heterogeneous populations.

[0099] The graph convolution network is a classical graph neural network architecture that guides the aggregation and propagation of node features in the local neighborhood through the adjacency structure and is widely used in structured semantic modeling tasks. The general processing flow of the graph convolution network is as follows: if the number of label categories is , the adjacency matrix is , and the unit matrix is . After introducing the self-connection, the graph convolution network is obtained , and the corresponding degree matrix is . After symmetric normalization processing, the graph convolution network can be obtained .

[0100] Among them, denotes the adjacency matrix after adding the self-connection, denotes the degree of node in the graph structure, denotes the edge weight between node and node , that is, the element value of the row and the column in the adjacency matrix, denotes the normalized adjacency matrix, denotes the degree matrix corresponding to the adjacency matrix after adding the self-connection.

[0101] The node vector of the layer in the graph convolution network can be represented as , and the corresponding learnable parameter is . The forward propagation process of the graph convolution network can be represented as:

[0102] Among them, denotes the node vector of the layer in the graph convolution network, denotes a nonlinear activation function.

[0103] According to the above, the traditional graph convolution network is mostly based on the node vector and adopts a fixed adjacency matrix to perform the graph convolution operation to achieve the purpose of image classification.

[0104] However, the inventors have found through research that in a multi-label chest radiograph image classification task, using a fixed graph structure (the graph structure is fixed when the adjacency matrix is fixed) for feature propagation cannot fully capture the semantic differences of label relationships in different attribute contexts. Therefore, the embodiment of the present application introduces an attribute selection mechanism into the traditional graph convolutional network, so that the graph convolutional layer can dynamically adjust the adjacency matrix according to the patient attribute value of the current chest radiograph image during the propagation process, thereby dynamically adjusting the graph structure.

[0105] Specifically, each graph convolutional layer in the attribute-driven graph neural network in the A-GCN module is associated with a preset attribute dimension If the patient attribute value of the preset attribute dimension associated with the graph convolutional layer is (i.e. ), the graph convolutional layer will select the corresponding label co-occurrence matrix as the adjacency matrix and participate in the graph convolution process.

[0106] In the embodiment of the present application, the label semantics output by the LMSA module is input to the attribute-driven graph neural network as an initial node vector, denoted as . The attribute-driven graph neural network performs forward propagation based on the initial node vector and the adjacency matrix corresponding to each graph convolutional layer. The forward propagation process can be represented as: .

[0107] wherein denotes the normalized label co-occurrence matrix.

[0108] The attribute-driven graph neural network outputs the node vector of the layer graph convolutional layer through the above forward propagation process. Here, the total number of graph convolutional layers is the same as the total number of preset attribute dimensions, i.e. .

[0109] The attribute-driven graph convolutional network can integrate the dynamic association structure between labels in multiple attribute contexts layer by layer by stacking multiple layers of graph convolution based on different attribute dimensions, and realize hierarchical refinement and personalized modeling of label semantics in attribute space.

[0110] Based on the attribute-driven graph convolutional network, the A-GCN module can perform graph convolution operation based on the node vector and the adjacency matrix to obtain an attribute-enhanced label embedding matrix when determining the recognition probability of each preset disease label corresponding to the chest radiograph image; then, the label semantics with global perception and the attribute-enhanced label embedding matrix are fused to obtain a label joint vector; finally, based on the label joint vector, the recognition probability of each preset disease label corresponding to the chest radiograph image is determined.

[0111] Here, the specific process of performing graph convolution operations using an attribute-driven graph convolutional network based on node vectors and adjacency matrices is detailed above and will not be repeated here. The attribute-driven graph convolutional network outputs... This is the attribute-enhanced tag embedding matrix mentioned above.

[0112] To fully utilize spatial structural information and semantic dependencies, the A-GCN module modifies the tag semantics output by the LMSA module. And the aforementioned attribute-enhanced tag embedding matrix The fusion is performed to obtain the joint vector. Joint vector The multi-label prediction probability (i.e., the recognition probability of each preset disease label corresponding to the chest X-ray image) is obtained after passing through a fully connected layer: .

[0113] in, Indicates the multi-label prediction probability. Represents a logical function. This represents the weight matrix corresponding to the fully connected layer. This indicates the bias term.

[0114] In a multi-label classification task for chest X-ray images, the recognition probability of each preset disease label corresponding to each chest X-ray image can be represented as a single-dimensional classification system. binary vector: .

[0115] in, , and These represent preset disease labels. , and The recognition probability, In this embodiment of the invention, the recognition probability of a preset disease label can be 0 or 1, used to characterize whether the preset disease label exists in the chest X-ray image. For example, for each preset disease label, when the recognition probability of the preset disease label is 0, it is determined that the preset disease label does not exist in the chest X-ray image. When the recognition probability of the preset disease label is 1, it is determined that the preset disease label exists in the chest X-ray image.

[0116] Here, the A-GCN module mainly includes two core designs: First, constructing a label co-occurrence matrix under attribute conditions to reveal the differences in label semantic structure as patient attributes change; second, dynamically selecting the graph structure based on patient attributes to guide label representation. Propagation and integration in different contexts. In this embodiment of the invention, by setting the A-GCN module in the recognition model, the recognition accuracy of the recognition model for heterogeneous groups can be enhanced.

[0117] On the basis of the above embodiments, hereinafter Figure 5 The image processing flow of the recognition model is introduced uniformly: Referring to Figure 5 , the chest radiograph image to be identified is input to the MFPR module for recursive feature division and reconstruction. The LMSA module uses the GloVe model to perform semantic coding on the preset disease labels to obtain an initial label embedding matrix. In each feature division of the MFPR module, the LMSA module uses the Transformer Encoder to update the label embedding matrix and the visual block sequence interactively. The MFPR module reconstructs the chest radiograph image based on the updated visual block sequence, and the LMSA module determines the label embedding matrix corresponding to the next iteration based on the updated label embedding matrix to perform the next iteration. After going through the iteration process for a set number of iterations, the final label semantics with global perception (i.e., the label embedding matrix under the set number of iterations) is obtained. The label semantics and patient attribute values are input to the A-GCN module, the A-GCN module determines the adjacency matrix corresponding to each graph convolution layer based on the patient attribute values, and determines the initial node vector based on the above-mentioned label semantics to perform graph convolution operation. The output result of the graph convolution operation is fused with the above-mentioned label semantics, and the fusion result is mapped through a fully connected layer to obtain the recognition probability of each preset disease label.

[0118] Here, Figure 5 Only age and gender are exemplarily shown as two attribute dimensions, and correspondingly, two layers of graph convolution layers are provided in the A-GCN module.

[0119] The recognition model in the embodiment of the application first performs recursive local division and spatial reconstruction on the chest radiograph image through the MFPR module, retains the global structure information while strengthening the expression of fine-grained lesions, thereby improving the recognition ability of the model for scale-heterogeneous lesions; secondly, the LMSA module is used to explicitly introduce label embedding into the feature division process, and the high-order cross-modal alignment between image regions and semantic labels is realized based on the Transformer structure, thereby enhancing the semantic consistency of the model under complex label combinations; finally, the A-GCN module is used to construct the label co-occurrence matrix under the condition of patient attribute information, and a dynamic attribute regulation mechanism is introduced in the label semantic propagation process, thereby effectively improving the personalized label reasoning ability of the model in a heterogeneous population.

[0120] Before the chest radiograph image is identified by the recognition model, the recognition model can be pre-trained. Each chest radiograph image in the training set is labeled by . Among them, , and respectively represent the preset disease labels , and The mark, In this embodiment of the invention, the preset disease label can be either 0 or 1, used to indicate whether the preset disease label exists in the chest X-ray image. For example, for each preset disease label, when the label is 0, it is determined that the preset disease label does not exist in the chest X-ray image. When the label is 1, it is determined that the preset disease label exists in the chest X-ray image.

[0121] Considering the significant class imbalance problem commonly found in medical image labeling, where most disease labels appear negative, this invention introduces a class-sensitive weighted binary cross-entropy loss function to improve the model's ability to identify minority class labels. Specifically, for preset disease labels... The mark ,like Then the positive loss term is weighted by the coefficient. Magnified, among which, and These represent the number of positive and negative labeled samples in the current batch, respectively. The loss function for the recognition model is defined as follows:

[0122] in, This represents the loss function corresponding to the recognition model. Indicates the primary classification loss. This represents the loss function corresponding to the LMSA module. This represents a hyperparameter used to adjust the relative weights between two loss functions. , Indicates preset disease labels The recognition probability, This indicates that the preset disease labels are embedded in the label matrix. The corresponding embedding vector, after linear mapping.

[0123] The embodiments of the present invention are set It can explicitly supervise the fusion process of label semantics and visual features, guide the LMSA module to learn label representations with strong semantic consistency and high discriminativeness, and improve the multi-label recognition performance in complex scenarios.

[0124] To evaluate the generalization and adaptation ability of the recognition model in the embodiments of the present application under different data sources, here, the ChestX-ray14 and CheXpert two chest radiograph image datasets are selected for experimental verification. Both datasets use frontal chest radiographs as input, have multi-label annotations of 14 common chest diseases, and contain patient attributes such as gender and age, which can support multi-disease co-occurrence analysis and attribute-aware modeling research. ChestX-ray14 contains 112,120 images from 30,805 patients, and the labels are extracted from radiology reports through natural language processing, with an accuracy of more than 90%. Among them, about 46% of the images contain at least one positive label. CheXpert contains 224,316 images covering 65,240 patients, and the label supports uncertainty modeling (1 represents positive, 0 represents negative, and -1 represents uncertain).

[0125] The embodiments of the present application adopt two processing methods: regarding the uncertain label as positive (CheXpert_1s) or negative (CheXpert_0s). The present application adopts CheXpert_1s and CheXpert_0s two settings, wherein CheXpert_1s regards the uncertain label as positive, and CheXpert_0s regards it as negative. The present application selects AUC and ROC curve as the main evaluation index, which is particularly suitable for medical image scenes with serious label distribution imbalance. The ROC curve measures the overall classification ability of the model by depicting the relationship between the false positive rate (FPR) and the true positive rate (TPR) at different discrimination thresholds, which is defined as follows:

[0126]

[0127] Among them, , , and represent the number of true positive, false positive, true negative and false negative samples respectively. AUC represents the area under the ROC curve, and the closer AUC is to 1, the stronger the model's ability to distinguish between positive and negative samples. The embodiments of the present application calculate AUC for each label separately, and take the average value as a comprehensive indicator of the overall performance of the model.

[0128] Comparative experiments with existing representative methods. The experimental results are shown in Tables 1, 2 and 3.

[0129] Table 1 AUC (%) of each label in ChestX-ray14

[0130] Table 2 AUC (%) of each label on CheXpert in the CheXpert 1s setting

[0131] Table 3 AUC (%) of each label on CheXpert in the CheXpert 0s setting

[0132] The above Tables 1-3 show the AUC indicators of different labels under different models. According to the above Tables 1-3, on the ChestX-ray14 dataset, the average AUC of the recognition model in the embodiment of the present application on the 14 disease label reaches 84.2%, which is significantly better than the mainstream convolutional network (CheXNet model: 80.7%), the Transformer structure (LT-ViT model: 82.8%) and the graph neural network (such as CheXGCN model: 82.6%, ImageGCN model: 83.2%) and other methods, which verifies the technical advantages of the embodiment of the present application in the lesion scale heterogeneity modeling and semantic consistency expression. Especially in the lesion type with spatial overlap or fuzzy semantic boundary (such as Effusion label, Consolidation label, Cardiomegaly label), the optimal indicators are obtained, which shows that the recognition model in the embodiment of the present application has stronger discrimination ability in complex image context. On the CheXpert dataset, considering the characteristics of the label containing uncertainty setting, two mainstream processing strategies (CheXpert_1s and CheXpert_0s) are used for testing. The experimental results show that the embodiment of the present application realizes the current optimal or top optimal performance under the two settings, and the average AUC reaches 83.8% under the CheXpert_0s setting, which is further superior to the existing similar models. The results show that the recognition model in the embodiment of the present application has strong label noise robustness and can adapt to the discrimination challenge brought by the annotation uncertainty in the real clinical scene.

[0133] In summary, the recognition model proposed in the embodiment of the present application is superior to the current optimal technology in accuracy, stability and adaptability, especially when facing complex scenes such as lesion overlap, label co-occurrence and population heterogeneity, which has the potential to be widely applied to intelligent auxiliary diagnosis systems.

[0134] It should be understood that the size of the serial number of each step in the above embodiment does not mean the order of execution, and the execution order of each process should be determined according to its function and inherent logic, and should not constitute any limitation on the implementation process of the embodiment of the present application.

[0135] The following is an embodiment of the device of the present application, for details not described in detail, please refer to the corresponding method embodiment described above.

[0136] Figure 6 The structure diagram of the chest radiograph image recognition device provided by the embodiment of the present application is shown, only the part related to the embodiment of the present application is shown for the convenience of description, and the details are described as follows: As Figure 6 The chest radiograph image recognition device 6 includes an acquisition module 61 and a recognition module 62.

[0137] The acquisition module 61 is configured to acquire a chest radiograph image to be recognized. The recognition module 62 is configured to input the chest radiograph image into a pre-trained recognition model to obtain recognition probabilities of each preset disease label corresponding to the chest radiograph image. The recognition model is configured to: perform recursive multi-scale feature division and spatial reconstruction on the chest radiograph image, and perform interactive updating between the preset disease label and the feature in the feature division process at each scale level to obtain a label semantic with global perception; and determine the recognition probabilities of each preset disease label corresponding to the chest radiograph image based on the label semantic with global perception.

[0138] In a possible implementation, the recognition module 62 is specifically configured to: determine a current scale level based on a current iteration number, perform feature division on the chest radiograph image according to the current scale level to obtain a visual block sequence; obtain a label embedding matrix corresponding to the preset disease label at the current iteration number, and perform interactive updating on the label embedding matrix and the visual block sequence to obtain an updated label embedding matrix and an updated visual block sequence; reconstruct the updated visual block sequence to obtain a reconstructed chest radiograph image; determine a label embedding matrix corresponding to the preset disease label at the next iteration based on the updated label embedding matrix; update the iteration number, and take the reconstructed chest radiograph image as a new chest radiograph image, jump to execute the step of determining the current scale level based on the current iteration number, until a set iteration number is reached, and obtain a label embedding matrix corresponding to the preset disease label at the set iteration number; determine the label embedding matrix corresponding to the preset disease label at the set iteration number as the label semantic with global perception.

[0139] In a possible implementation, the recognition module 62 is specifically configured to: splice the label embedding matrix and the visual block sequence to obtain a cross-modal input sequence; The encoder is used to encode and update the cross-modal input sequence to obtain an updated label embedding matrix and a visual block sequence.

[0140] In a possible implementation, the identification module 62 is specifically configured to perform linear mapping on the updated label embedding matrix to obtain a label embedding matrix corresponding to the preset disease label in the next iteration.

[0141] In a possible implementation, the identification module 62 is specifically configured to determine the size of the visual block based on the current scale level; the higher the scale level, the smaller the size of the visual block. The soft division strategy is adopted to divide the chest radiograph image according to the size to obtain a plurality of visual blocks. The plurality of visual blocks are mapped to obtain a visual block sequence.

[0142] In a possible implementation, the identification module 62 is specifically configured to: Obtain patient attribute values of a patient corresponding to the chest radiograph image in each preset attribute dimension; Determine a node vector based on the label semantics with global perception; Determine an adjacency matrix based on the patient attribute values; Perform graph convolution operation based on the node vector and the adjacency matrix to determine the identification probability of each preset disease label corresponding to the chest radiograph image.

[0143] In a possible implementation, the identification module 62 is specifically configured to: For each patient attribute value, determine a label co-occurrence matrix corresponding to the patient attribute value based on a preset label co-occurrence set; the label co-occurrence set includes the label co-occurrence matrix corresponding to each preset attribute dimension under different patient attribute values. Determine the label co-occurrence matrix corresponding to each patient attribute value as the adjacency matrix.

[0144] In a possible implementation, the identification module 62 is further configured to: Divide the chest radiograph image sample set based on each preset attribute dimension and the corresponding patient attribute value range to determine a sample subset of each preset attribute dimension under each patient attribute value. For each sample subset of each preset attribute dimension under each patient attribute value, respectively count the co-occurrence frequency corresponding to each pair of preset disease labels. Perform normalization processing and sparsification processing on the co-occurrence frequency to determine the label co-occurrence matrix of the preset attribute dimension under the patient attribute value.

[0145] In a possible implementation, the identification module 62 is specifically configured to: The graph convolution operation is performed based on the node vector and the adjacency matrix to obtain an attribute-enhanced label embedding matrix; The label joint vector is obtained by fusing the label semantics with global perception and the attribute-enhanced label embedding matrix; Based on the label joint vector, the recognition probability of each preset disease label corresponding to the chest radiograph image is determined.

[0146] The embodiment of the present application also provides a chest radiograph image recognition device, which comprises a memory and a processor, the memory stores a computer program, and the processor implements the method in the above method embodiment when executing the computer program.

[0147] The above-described embodiments are only used to illustrate the technical solutions of the present application, rather than limit them; although the present application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that the technical solutions recorded in the foregoing embodiments can be modified, or some technical features can be replaced by equivalents; and these modifications or replacements do not make the essence of the corresponding technical solutions deviate from the spirit and scope of the technical solutions of the embodiments of the present application, and should be included in the protection scope of the present application.

Claims

1. A method for recognizing chest X-ray images, characterized in that, include: Obtain the chest X-ray image to be identified; The chest X-ray image is input into a pre-trained recognition model to obtain the recognition probability of each preset disease label corresponding to the chest X-ray image; The recognition model is used to: perform recursive multi-scale feature segmentation and spatial reconstruction on the chest X-ray image, and perform interactive updates between preset disease labels and features during the feature segmentation process at each scale level to obtain globally aware label semantics; and determine the recognition probability of each preset disease label corresponding to the chest X-ray image based on the globally aware label semantics.

2. The method for recognizing chest X-ray images according to claim 1, characterized in that, The process involves recursively performing multi-scale feature segmentation and spatial reconstruction on the chest X-ray image, and updating the interaction between preset disease labels and features during feature segmentation at each scale level to obtain globally aware label semantics, including: Based on the current number of iterations, the current scale level is determined, and the chest X-ray image is segmented according to the current scale level to obtain a visual block sequence; Obtain the label embedding matrix corresponding to the preset disease label at the current iteration number, and interactively update the label embedding matrix and the visual block sequence to obtain the updated label embedding matrix and visual block sequence; The updated visual block sequence is reconstructed to obtain the reconstructed chest X-ray image; Based on the updated label embedding matrix, determine the label embedding matrix corresponding to the preset disease label in the next iteration; Update the iteration count, and use the reconstructed chest X-ray image as the new chest X-ray image. Jump to the "determine the current scale level based on the current iteration count" step until the set iteration count is reached, and obtain the label embedding matrix corresponding to the preset disease label under the set iteration count. The label embedding matrix corresponding to the preset disease label at the set number of iterations is determined as the globally aware label semantics.

3. The method for recognizing chest X-ray images according to claim 2, characterized in that, The step of interactively updating the label embedding matrix and the visual block sequence to obtain the updated label embedding matrix and visual block sequence includes: The label embedding matrix and the visual block sequence are concatenated to obtain a cross-modal input sequence; The cross-modal input sequence is encoded and updated using an encoder to obtain the updated label embedding matrix and visual block sequence.

4. The method for recognizing chest X-ray images according to claim 2 or 3, characterized in that, The step of determining the label embedding matrix corresponding to the preset disease label in the next iteration based on the updated label embedding matrix includes: The updated label embedding matrix is ​​linearly mapped to obtain the label embedding matrix corresponding to the preset disease label in the next iteration.

5. The method for recognizing chest X-ray images according to claim 2 or 3, characterized in that, The step of segmenting the chest X-ray image according to the current scale level to obtain a visual block sequence includes: The size of the visual block is determined based on the current scale level; the higher the scale level, the smaller the size of the visual block. A soft partitioning strategy is adopted to divide the chest X-ray image according to the specified size, resulting in multiple visual blocks; The multiple visual blocks are mapped to obtain the visual block sequence.

6. The method for recognizing chest X-ray images according to claim 1 or 2, characterized in that, The step of determining the recognition probability of each preset disease label corresponding to the chest X-ray image based on globally aware label semantics includes: Obtain the patient attribute values ​​corresponding to the chest X-ray image under each preset attribute dimension; Based on the globally aware label semantics, determine the node vector; Based on the patient attribute values, determine the adjacency matrix; Graph convolution is performed based on the node vector and the adjacency matrix to determine the recognition probability of each preset disease label corresponding to the chest X-ray image.

7. The method for recognizing chest X-ray images according to claim 6, characterized in that, The process of determining the adjacency matrix based on the patient attribute values ​​includes: For each patient attribute value, a label co-occurrence matrix corresponding to that patient attribute value is determined based on a preset label co-occurrence set; the label co-occurrence set includes the label co-occurrence matrices corresponding to each preset attribute dimension under different patient attribute values; The co-occurrence matrix of labels corresponding to the attribute values ​​of each patient is determined as the adjacency matrix.

8. The method for recognizing chest X-ray images according to claim 6, characterized in that, Before determining the adjacency matrix based on the patient attribute values, the method further includes: Based on each preset attribute dimension and its corresponding patient attribute value range, the chest X-ray image sample set is divided to determine the sample subset of each preset attribute dimension under each patient attribute value. For each preset attribute dimension, under each patient attribute value, the co-occurrence frequency of each pair of preset disease labels is counted. The co-occurrence frequencies are normalized and sparsified to determine the label co-occurrence matrix of the preset attribute dimension under the patient's attribute value.

9. The method for recognizing chest X-ray images according to claim 6, characterized in that, Based on the node vectors and the adjacency matrix, a graph convolution operation is performed to determine the recognition probability of each preset disease label corresponding to the chest X-ray image, including: A graph convolution operation is performed based on the node vectors and the adjacency matrix to obtain the attribute-enhanced label embedding matrix; The globally aware tag semantics and the attribute-enhanced tag embedding matrix are fused to obtain a joint tag vector; Based on the joint vector of the labels, the recognition probability of each preset disease label corresponding to the chest X-ray image is determined.

10. A chest X-ray image recognition device, characterized in that, It includes a memory and a processor, the memory storing a computer program, and the processor executing the computer program to implement the method as described in any one of claims 1 to 9.

Images