An individual identity recognition method sharing cross-modal fusion of bone information

Abstract

This invention relates to an individual identification method based on cross-modal fusion of shared skeletal information, belonging to the field of medical image retrieval technology. First, a cross-domain bidirectional image transformation network is used to convert VRT images in the original data into CXR images to achieve modal completion. Then, CXR-VRT data pairs are used to train a cross-modal fusion model based on shared skeletal information. After training, a neural bone pattern retrieval database is constructed based on a chest radiograph identity retrieval database. First, the VRT image to be retrieved is processed through the cross-domain bidirectional image transformation network to obtain CXR-VRT image pairs. Then, the corresponding neural bone pattern is obtained through the cross-modal fusion model based on shared skeletal information. Finally, a nearest neighbor search method is used to search the neural bone pattern retrieval database to determine the corresponding individual identity. This achieves automatic individual information identification directly based on skeletal imaging data. It not only provides strong support for practical applications in fields such as forensic medicine but also promotes the development of related research fields such as cross-modal retrieval.

Description

Technical Field

[0001] This invention belongs to the field of medical image retrieval technology, specifically relating to an individual identity recognition method based on cross-modal fusion of shared skeletal information. Background Technology

[0002] Individual identification plays a crucial role in accident and crime investigations. By analyzing and comparing biological evidence (such as DNA, fingerprints, and skeletal remains), an individual's identity can be determined, aiding in investigations. However, when a body is severely decomposed, intentionally damaged, or charred, soft tissue markers such as facial features and fingerprints are destroyed and rendered unusable for identification. Furthermore, extracting DNA from such remains is extremely challenging due to its degradation over time. Even when DNA is successfully extracted, the identification process is typically limited by time and economic costs, as well as the size of DNA databases. Without prior DNA sequencing and storage in the database for the individual or their close relatives, identification cannot be confirmed.

[0003] Compared to DNA and soft tissue markers, bones are less susceptible to damage and have a longer preservation time. In forensic medicine, it is common to manually compare features or data from skeletal imaging materials from before and after death to identify individuals. For example, the thoracic skeleton, including ribs, vertebrae, and sternum, has its imaging data (such as chest X-rays and CT scans) used for manual comparison to estimate sex and age due to its complex morphology and identifiable boundaries. However, since these methods are usually performed manually by forensic scientists, they are inevitably limited by measurement methods and expert experience. Therefore, how to directly and automatically identify individual information based on skeletal imaging data is of great research value.

[0004] Chest X-ray (CXR) is a routine, low-cost, and low-radiation medical examination, often central to the screening, diagnosis, and management of various chest diseases. More importantly, chest X-ray images typically contain all the skeletal elements of the chest and clear identification information, naturally forming a massive chest X-ray identity retrieval database. For unidentified cadavers, to avoid the impact of soft tissue and organ decomposition on skeletal observation, forensic pathologists typically use Volume Rendering Technology (VRT) to preprocess their chest computed tomography (CT) images, thereby obtaining clear three-dimensional images of the sternum. Therefore, using the clear thoracic skeletal structure from VRT images for identity retrieval in a massive chest X-ray database is feasible and has significant social implications.

[0005] Therefore, at this stage, it is necessary to design an individual identity recognition method, system, and storage medium that uses cross-modal fusion of shared skeletal information to solve the above problems. Summary of the Invention

[0006] The purpose of this invention is to provide a method, system, and storage medium for individual identity recognition through cross-modal fusion of shared skeletal information, which solves the technical problems existing in the prior art. Combining the feature that chest X-ray databases can provide massive amounts of identity information, this invention uses VRT images with clear chest skeletal structures to perform cross-modal retrieval in the chest X-ray database to determine individual identity, thereby achieving the purpose of automatic individual identity recognition directly based on skeletal imaging data, and improving the efficiency and accuracy of individual identity recognition.

[0007] To achieve the above objectives, the technical solution of the present invention is as follows:

[0008] A method for individual identity recognition through cross-modal fusion of shared skeletal information includes the following steps:

[0009] S1: Construction of the original dataset: Collect CXR-VRT image pairs from the same individual as samples;

[0010] S2: Construction of the chest X-ray identity retrieval library: Collect a large number of single chest X-rays with different identity sources from the original dataset in S1 as interference items, and combine them with the chest X-ray data of the test set part of the original dataset in S1 to form the chest X-ray identity retrieval library.

[0011] S3: Modal Completion: Using a cross-domain bidirectional image transformation network, real chest X-ray images from the original dataset are converted into generated VRT images, and VRT images from the original dataset are converted into generated chest X-ray images. The dataset is then completed using real image pairs and generated image pairs from each individual.

[0012] S4: Data Augmentation: All training samples are preprocessed using the following augmentation methods in sequence: size adjustment, histogram equalization, contrast adjustment, probabilistic color jittering, probabilistic horizontal flipping, probabilistic random rotation, and normalization.

[0013] S5: Construct a cross-modal fusion model for shared skeletal information in CXR-VRT: For chest X-ray images and VRT images, two encoder-decoder modules are used to extract fine-grained features of the images respectively; for the extracted features of the corresponding chest X-ray images and VRT images, a feature fusion module is used to fuse the features;

[0014] S6: Training the CXR-VRT shared skeletal information cross-modal fusion model: Using image pairs from the constructed dataset as input, for the reconstructed image output by the decoder, the reconstruction loss of the original image is used to learn more fine-grained features in the image. For the image features output by the encoder, each pair of features is processed by the feature fusion module and then contrastive learning is used to minimize intra-class distance and maximize inter-class distance. The overall training objective is the sum of the reconstruction loss of the chest X-ray image and the VRT image and the contrastive loss between the fused features.

[0015] S7: Construct a neural bone pattern retrieval database based on the chest X-ray identity retrieval database: First, convert the chest X-ray data into VRT images using a cross-domain bidirectional image conversion network. Then, use the CXR-VRT shared skeletal information cross-modal fusion model to obtain the corresponding fusion features. Finally, use the fusion features, i.e., neural bone pattern, as the key and the corresponding individual identity information as the value to construct a search table as the neural bone pattern retrieval database.

[0016] S8: Using VRT images to identify individuals: The VRT image to be retrieved is converted into a CXR image using a cross-domain bidirectional image transformation network. After obtaining the CXR-VRT image pair, the neural bone pattern is obtained as the retrieval item using the CXR-VRT shared skeletal information cross-modal fusion model. Then, the individual identity is retrieved in the neural bone pattern retrieval database using the nearest neighbor search method.

[0017] Furthermore, step S1 is detailed as follows:

[0018] The original dataset collects CXR-VRT image pairs from the same individual, totaling 1315 individuals. Each individual contains only one pair of CXR-VRT images, resulting in a total of 1315 pairs of CXR-VRT data. The CXR and VRT images are of different sizes. The training and test sets are divided in an 8:2 ratio, containing 1052 and 263 sample pairs, respectively.

[0019] Furthermore, step S2 is detailed as follows:

[0020] A large number of single chest X-rays with different identity sources from those in the original dataset were collected as noise items, and these were combined with all chest X-rays in the test set of the original dataset to form an identity retrieval library; a total of 876 chest X-ray image samples were collected as noise items, and these, together with 263 chest X-rays from the test set in the original dataset, formed a chest X-ray identity retrieval library with a sample size of 1139.

[0021] Furthermore, step S4 is detailed as follows:

[0022] During training, the training samples were preprocessed using the following augmentation methods: First, bilinear interpolation was used to resize all images to 256*256. Then, histogram equalization was applied to the images. Next, the image contrast was adjusted with a parameter setting of 1.8, color jitter was performed with a probability of 0.2, and the images were horizontally flipped with a random probability of 0.5. Random rotations of (-20, +20) degrees were performed with a random probability of 0.3. In addition, all samples were normalized. For VRT images, the mean was set to 0.3817 and the standard deviation to 0.3180. For chest X-ray images, the mean was set to 0.6425 and the standard deviation to 0.1613.

[0023] Furthermore, step S5 is detailed as follows:

[0024] The cross-modal fusion model consists of two encoder-decoder modules for fine-grained feature extraction from VRT images and chest radiographs, respectively, and a feature fusion module.

[0025] The encoder-decoder module is a U-Net structure with skip connections. The encoder part contains four encoding modules, each containing two 3×3 convolutional layers, a BN layer, and a ReLU activation layer. Each encoding module is connected to a downsampling layer with a 2×2 convolutional kernel. The decoder part contains four decoding modules, each containing two 3×3 convolutional layers, a BN layer, and a ReLU activation layer. Each decoding module is connected to a deconvolutional layer with a 2×2 convolutional kernel. Finally, the reconstructed image is obtained through a 1×1 deconvolutional layer.

[0026] The feature fusion module is a fully connected layer. The input is a vector concatenated from the two images after passing through the encoder module. The vector has a dimension of 2×1024×16×16. The output is a fused feature vector with a dimension of 256.

[0027] Furthermore, step S6 is detailed as follows:

[0028] Let the original VRT image and CXR image be denoted as x and y, respectively, and let the two transformations of the cross-domain bidirectional image transformation network, VRT to CXR and CXR to VRT, be denoted as T. xy and T yx The converted CXR and VRT are denoted as T respectively. xy (x) and T yx (y), the encoder modules corresponding to the VRT image and the CXR image are denoted as f and f′ respectively, the decoder modules are denoted as g and g′ respectively, and the feature fusion module is denoted as F;

[0029] The CXR and VRT are fed into their respective encoder and decoder modules, and the reconstruction loss is calculated as shown below to ensure the acquisition of fine-grained features.

[0030]

[0031]

[0032]

[0033] in and These represent the VRT and CXR images reconstructed after passing through the encoder-decoder module, respectively. and These are the representations of the two reconstruction functions;

[0034] The feature fusion module is trained concurrently with the encoder-decoder module. Fine-grained feature maps of VRT and CXR obtained from the encoder are used as inputs to the feature fusion module, and then the contrastive loss is calculated for learning.

[0035]

[0036] c = F(f(x), f′(y))

[0037] c′=F(f(T yx (y)), f′(T) xy (x)))

[0038] Where c and c′ represent the fusion features of the real data pair and the data pair obtained through the image transformation network, respectively. i c′ represents the fusion feature of real data pairs from the same individual. i This represents the fused features of the generated data pair transformed from the real data, serving as a positive sample, while c j This represents the fusion feature of real data pairs from other individuals, used as a negative sample;

[0039] The overall loss function is shown below:

[0040]

[0041] Where λ re Indicates reconstruction losses Weighting.

[0042] Furthermore, step S7 is detailed as follows:

[0043] First, a cross-domain bidirectional image transformation network is used to process chest radio images ({y) from the chest radio identity retrieval database. i}) converted to VRT image({T) yx (y iThen, the CXR-VRT shared skeletal information cross-modal fusion model is used to obtain the corresponding fusion features; finally, using the fusion features, i.e., neural bone pattern, as the key and the corresponding individual identity information as the value, a search table is constructed as the neural bone pattern retrieval library (C), specifically represented as follows:

[0044]

[0045] Furthermore, step S8 is detailed as follows:

[0046] First, the VRT image (x) to be retrieved is converted into a chest X-ray image (T) through a cross-domain bidirectional image transformation network. xy (x) is then used to obtain the corresponding neural bone pattern (c) by using the CXR-VRT shared skeletal information cross-modal fusion model. Finally, the corresponding individual identity is retrieved in the neural bone pattern retrieval library (C) by nearest neighbor search. The specific formula is as follows:

[0047] c = F(f(x), f′(T) xy (x)))

[0048]

[0049] A cross-modal fusion individual identity recognition system based on shared skeletal information is provided, which employs the aforementioned cross-modal fusion individual identity recognition method based on shared skeletal information for individual identity recognition.

[0050] A storage medium, characterized in that a computer program is stored on the storage medium, and the computer program, when run, executes an individual identity recognition method for cross-modal fusion of shared skeletal information as described above.

[0051] Compared with the prior art, the beneficial effects of the present invention are as follows:

[0052] This invention addresses the problems inherent in manual individual identification based on skeletal imaging data by proposing a cross-modal fusion method for individual identification using shared skeletal information. This method enables automatic individual information identification directly based on skeletal imaging data. This invention not only provides strong support for practical applications in fields such as forensic medicine but also promotes the development of related research areas such as cross-modal retrieval. Attached Figure Description

[0053] Figure 1 This is a flowchart illustrating the method proposed in this invention.

[0054] Figure 2 This is an overall framework diagram of the method proposed in this invention.

[0055] Figure 3This is a schematic diagram of the CXR-VRT shared skeletal information cross-modal fusion model in the method framework proposed in this invention.

[0056] Figure 4 This is a schematic diagram of the neural bone pattern retrieval library construction module in the method framework proposed in this invention.

[0057] Figure 5 This is a visualization of the retrieval results of the method proposed in this invention.

[0058] Figure 6 This is a comparison chart of the retrieval results indicators of the method proposed in this invention and other methods. Detailed Implementation

[0059] Example:

[0060] A method for individual identity recognition through cross-modal fusion of shared skeletal information includes the following steps:

[0061] S1: Construction of the original dataset: Collect CXR-VRT image pairs from the same individual as samples;

[0062] S2: Construction of the chest X-ray identity retrieval library: Collect a large number of single chest X-rays with different identity sources from the original dataset in S1 as interference items, and combine them with the chest X-ray data of the test set part of the original dataset in S1 to form the chest X-ray identity retrieval library.

[0063] S3: Modal Completion: Using a cross-domain bidirectional image transformation network, real chest X-ray images from the original dataset are converted into generated VRT images, and VRT images from the original dataset are converted into generated chest X-ray images. The dataset is then completed using real image pairs and generated image pairs from each individual.

[0064] S4: Data Augmentation: All training samples are preprocessed using the following augmentation methods in sequence: size adjustment, histogram equalization, contrast adjustment, probabilistic color jittering, probabilistic horizontal flipping, probabilistic random rotation, and normalization.

[0065] S5: Construct a cross-modal fusion model for shared skeletal information in CXR-VRT: For chest X-ray images and VRT images, two encoder-decoder modules are used to extract fine-grained features of the images respectively; for the extracted features of the corresponding chest X-ray images and VRT images, a feature fusion module is used to fuse the features;

[0066] S6: Training the CXR-VRT shared skeletal information cross-modal fusion model: Using image pairs from the constructed dataset as input, for the reconstructed image output by the decoder, the reconstruction loss of the original image is used to learn more fine-grained features in the image. For the image features output by the encoder, each pair of features is processed by the feature fusion module and then contrastive learning is used to minimize intra-class distance and maximize inter-class distance. The overall training objective is the sum of the reconstruction loss of the chest X-ray image and the VRT image and the contrastive loss between the fused features.

[0067] S7: Construct a neural bone pattern retrieval database based on the chest X-ray identity retrieval database: First, convert the chest X-ray data into VRT images using a cross-domain bidirectional image conversion network. Then, use the CXR-VRT shared skeletal information cross-modal fusion model to obtain the corresponding fusion features. Finally, use the fusion features, i.e., neural bone pattern, as the key and the corresponding individual identity information as the value to construct a search table as the neural bone pattern retrieval database.

[0068] S8: Using VRT images to identify individuals: The VRT image to be retrieved is converted into a CXR image using a cross-domain bidirectional image transformation network. After obtaining the CXR-VRT image pair, the neural bone pattern is obtained as the retrieval item using the CXR-VRT shared skeletal information cross-modal fusion model. Then, the individual identity is retrieved in the neural bone pattern retrieval database using the nearest neighbor search method.

[0069] Furthermore, step S1 is detailed as follows:

[0070] The original dataset collects CXR-VRT image pairs from the same individual, totaling 1315 individuals. Each individual contains only one pair of CXR-VRT images, resulting in a total of 1315 pairs of CXR-VRT data. The CXR and VRT images are of different sizes. The training and test sets are divided in an 8:2 ratio, containing 1052 and 263 sample pairs, respectively.

[0071] Furthermore, step S2 is detailed as follows:

[0072] A large number of single chest X-rays with different identity sources from those in the original dataset were collected as noise items, and these were combined with all chest X-rays in the test set of the original dataset to form an identity retrieval library; a total of 876 chest X-ray image samples were collected as noise items, and these, together with 263 chest X-rays from the test set in the original dataset, formed a chest X-ray identity retrieval library with a sample size of 1139.

[0073] Furthermore, step S4 is detailed as follows:

[0074] During training, the training samples were preprocessed using the following augmentation methods: First, bilinear interpolation was used to resize all images to 256*256. Then, histogram equalization was applied to the images. Next, the image contrast was adjusted with a parameter setting of 1.8, color jitter was performed with a probability of 0.2, and the images were horizontally flipped with a random probability of 0.5. Random rotations of (-20, +20) degrees were performed with a random probability of 0.3. In addition, all samples were normalized. For VRT images, the mean was set to 0.3817 and the standard deviation to 0.3180. For chest X-ray images, the mean was set to 0.6425 and the standard deviation to 0.1613.

[0075] Furthermore, step S5 is detailed as follows:

[0076] The cross-modal fusion model consists of two encoder-decoder modules for fine-grained feature extraction from VRT images and chest radiographs, respectively, and a feature fusion module.

[0077] The encoder-decoder module is a U-Net structure with skip connections. The encoder part contains four encoding modules, each containing two 3×3 convolutional layers, a BN layer, and a ReLU activation layer. Each encoding module is connected to a downsampling layer with a 2×2 convolutional kernel. The decoder part contains four decoding modules, each containing two 3×3 convolutional layers, a BN layer, and a ReLU activation layer. Each decoding module is connected to a deconvolutional layer with a 2×2 convolutional kernel. Finally, the reconstructed image is obtained through a 1×1 deconvolutional layer.

[0078] The feature fusion module is a fully connected layer. The input is a vector concatenated from the two images after passing through the encoder module. The vector has a dimension of 2×1024×16×16. The output is a fused feature vector with a dimension of 256.

[0079] Furthermore, step S6 is detailed as follows:

[0080] Let the original VRT image and CXR image be denoted as x and y, respectively, and let the two transformations of the cross-domain bidirectional image transformation network, VRT to CXR and CXR to VRT, be denoted as T. xy and T yx The converted CXR and VRT are denoted as T respectively. xy (x) and T yx (y), the encoder modules corresponding to the VRT image and the CXR image are denoted as f and f′ respectively, the decoder modules are denoted as g and g′ respectively, and the feature fusion module is denoted as F;

[0081] The CXR and VRT are fed into their respective encoder and decoder modules, and the reconstruction loss is calculated as shown below to ensure the acquisition of fine-grained features.

[0082]

[0083]

[0084]

[0085] in and These represent the VRT and CXR images reconstructed after passing through the encoder-decoder module, respectively. and These are the representations of the two reconstruction functions;

[0086] The feature fusion module is trained concurrently with the encoder-decoder module. Fine-grained feature maps of VRT and CXR obtained from the encoder are used as inputs to the feature fusion module, and then the contrastive loss is calculated for learning.

[0087]

[0088] c = F(f(x), f′(y))

[0089] c′=F(f(T yx (y)), f′(T) xy (x)))

[0090] Where c and c′ represent the fusion features of the real data pair and the data pair obtained through the image transformation network, respectively. i c′ represents the fusion feature of real data pairs from the same individual. i This represents the fused features of the generated data pair transformed from the real data, serving as a positive sample, while c j This represents the fusion feature of real data pairs from other individuals, used as a negative sample;

[0091] The overall loss function is shown below:

[0092]

[0093] Where λ re Indicates reconstruction losses Weighting.

[0094] Furthermore, step S7 is detailed as follows:

[0095] First, a cross-domain bidirectional image transformation network is used to process chest radio images ({y) from the chest radio identity retrieval database. i}) converted to VRT image({T) yx (y iThen, the CXR-VRT shared skeletal information cross-modal fusion model is used to obtain the corresponding fusion features; finally, using the fusion features, i.e., neural bone pattern, as the key and the corresponding individual identity information as the value, a search table is constructed as the neural bone pattern retrieval library (C), specifically represented as follows:

[0096]

[0097] Furthermore, step S8 is detailed as follows:

[0098] First, the VRT image (x) to be retrieved is converted into a chest X-ray image (T) through a cross-domain bidirectional image transformation network. xy (x) is then used to obtain the corresponding neural bone pattern (c) by using the CXR-VRT shared skeletal information cross-modal fusion model. Finally, the corresponding individual identity is retrieved in the neural bone pattern retrieval library (C) by nearest neighbor search. The specific formula is as follows:

[0099] c = F(f(x), f′(T) xy (x)))

[0100]

[0101] A cross-modal fusion individual identity recognition system based on shared skeletal information is provided, which employs the aforementioned cross-modal fusion individual identity recognition method based on shared skeletal information for individual identity recognition.

[0102] A storage medium, characterized in that a computer program is stored on the storage medium, and the computer program, when run, executes an individual identity recognition method for cross-modal fusion of shared skeletal information as described above.

Claims

1. A method for individual identity recognition based on cross-modal fusion of shared skeletal information, characterized in that, Includes the following steps: S1: Construction of the original dataset: Collect CXR-VRT image pairs from the same individual as samples; S2: Construction of the chest X-ray identity retrieval library: Collect a large number of single chest X-rays with different identity sources from the original dataset in S1 as interference items, and combine them with the chest X-ray data of the test set part of the original dataset in S1 to form the chest X-ray identity retrieval library. S3: Modal Completion: Using a cross-domain bidirectional image transformation network, real chest X-ray images from the original dataset are converted into generated VRT images, and VRT images from the original dataset are converted into generated chest X-ray images. The dataset is then completed using real image pairs and generated image pairs from each individual. S4: Data Augmentation: All training samples are preprocessed using the following augmentation methods in sequence: size adjustment, histogram equalization, contrast adjustment, probabilistic color jittering, probabilistic horizontal flipping, probabilistic random rotation, and normalization. S5: Construct a cross-modal fusion model for shared skeletal information in CXR-VRT: For chest X-ray images and VRT images, two encoder-decoder modules are used to extract fine-grained features of the images respectively; for the extracted features of the corresponding chest X-ray images and VRT images, a feature fusion module is used to fuse the features; S6: Training the CXR-VRT shared skeletal information cross-modal fusion model: Using image pairs from the constructed dataset as input, for the reconstructed image output by the decoder, the reconstruction loss of the original image is used to learn more fine-grained features in the image. For the image features output by the encoder, each pair of features is processed by the feature fusion module and then contrastive learning is used to minimize intra-class distance and maximize inter-class distance. The overall training objective is the sum of the reconstruction loss of the chest X-ray image and the VRT image and the contrastive loss between the fused features. S7: Construct a neural bone pattern retrieval database based on the chest X-ray identity retrieval database: First, convert the chest X-ray data into VRT images using a cross-domain bidirectional image conversion network. Then, use the CXR-VRT shared skeletal information cross-modal fusion model to obtain the corresponding fusion features. Finally, use the fusion features, i.e., neural bone pattern, as the key and the corresponding individual identity information as the value to construct a search table as the neural bone pattern retrieval database. S8: Using VRT images to identify individuals: The VRT image to be retrieved is converted into a CXR image using a cross-domain bidirectional image transformation network. After obtaining the CXR-VRT image pair, the neural bone pattern is obtained as the retrieval item using the CXR-VRT shared skeletal information cross-modal fusion model. Then, the individual identity is retrieved in the neural bone pattern retrieval database using the nearest neighbor search method.

2. The individual identity recognition method based on cross-modal fusion of shared skeletal information according to claim 1, characterized in that, Step S1 is as follows: The original dataset collects CXR-VRT image pairs from the same individual, totaling 1315 individuals. Each individual contains only one pair of CXR-VRT images, resulting in a total of 1315 pairs of CXR-VRT data. The CXR and VRT images are of different sizes. The training and test sets are divided in an 8:2 ratio, containing 1052 and 263 sample pairs, respectively.

3. The individual identity recognition method based on cross-modal fusion of shared skeletal information according to claim 2, characterized in that, Step S2 is as follows: A large number of single chest X-rays with different identity sources from those in the original dataset were collected as noise items, and these were combined with all chest X-rays in the test set of the original dataset to form an identity retrieval library; a total of 876 chest X-ray image samples were collected as noise items, and these, together with 263 chest X-rays from the test set in the original dataset, formed a chest X-ray identity retrieval library with a sample size of 1139.

4. The individual identity recognition method based on cross-modal fusion of shared skeletal information according to claim 2, characterized in that, Step S4 is as follows: During training, the training samples were preprocessed using the following augmentation methods: First, bilinear interpolation was used to resize all images to 256*256. Then, histogram equalization was applied to the images. Next, the image contrast was adjusted with a parameter setting of 1.8, color jitter was performed with a probability of 0.2, and the images were horizontally flipped with a random probability of 0.

5. Random rotations of (-20, +20) degrees were performed with a random probability of 0.

3. In addition, all samples were normalized. For VRT images, the mean was set to 0.3817 and the standard deviation to 0.3180. For chest X-ray images, the mean was set to 0.6425 and the standard deviation to 0.1613.

5. The individual identity recognition method based on cross-modal fusion of shared skeletal information according to claim 1, characterized in that, Step S5 is as follows: The cross-modal fusion model consists of two encoder-decoder modules for fine-grained feature extraction from VRT images and chest radiographs, respectively, and a feature fusion module. The encoder-decoder module is a U-Net structure with skip connections. The encoder part contains four encoding modules, each containing two 3×3 convolutional layers, a BN layer, and a ReLU activation layer. Each encoding module is connected to a downsampling layer with a 2×2 convolutional kernel. The decoder part contains four decoding modules, each containing two 3×3 convolutional layers, a BN layer, and a ReLU activation layer. Each decoding module is connected to a deconvolutional layer with a 2×2 convolutional kernel. Finally, the reconstructed image is obtained through a 1×1 deconvolutional layer. The feature fusion module is a fully connected layer. The input is a vector concatenated from the two images after passing through the encoder module. The vector has a dimension of 2×1024×16×16. The output is a fused feature vector with a dimension of 256.

6. The individual identity recognition method based on cross-modal fusion of shared skeletal information according to claim 1, characterized in that, Step S6 is as follows: Let the original VRT image and CXR image be denoted as x and y, respectively, and let the two transformations of the cross-domain bidirectional image transformation network, VRT to CXR and CXR to VRT, be denoted as T. xy and T yx The converted CXR and VRT are denoted as T. yx (x) and T yx (y), the encoder modules corresponding to the VRT image and the CXR image are denoted as f and f′ respectively, the decoder modules are denoted as g and g′ respectively, and the feature fusion module is denoted as F; The CXR and VRT are fed into their respective encoder and decoder modules, and the reconstruction loss is calculated as shown below to ensure the acquisition of fine-grained features. in and These represent the VRT and CXR images reconstructed after passing through the encoder-decoder module, respectively. and These are the representations of the two reconstruction functions; The feature fusion module is trained concurrently with the encoder-decoder module. Fine-grained feature maps of VRT and CXR obtained from the encoder are used as inputs to the feature fusion module, and then the contrastive loss is calculated for learning. c = F(f(x), f′(y)) c′=F(f(T yx (y)),f′(T xy (x))) Where c and c′ represent the fusion features of the real data pair and the data pair obtained through the image transformation network, respectively. i c′ represents the fusion feature of real data pairs from the same individual. i This represents the fused features of the generated data pair transformed from the real data, serving as a positive sample, while c j This represents the fusion feature of real data pairs from other individuals, used as a negative sample; The overall loss function is shown below: Where λ re Indicates reconstruction losses Weighting.

7. The individual identity recognition method based on cross-modal fusion of shared skeletal information according to claim 1, characterized in that, Step S7 is as follows: First, a cross-domain bidirectional image transformation network is used to process chest radio images ({y) from the chest radio identity retrieval database. i }) converted to VRT image({T) yx (y i Then, the CXR-VRT shared skeletal information cross-modal fusion model is used to obtain the corresponding fusion features; finally, using the fusion features, i.e., neural bone pattern, as the key and the corresponding individual identity information as the value, a search table is constructed as the neural bone pattern retrieval library (C), specifically represented as follows:

8. The individual identity recognition method based on cross-modal fusion of shared skeletal information according to claim 1, characterized in that, Step S8 is as follows: First, the VRT image (x) to be retrieved is converted into a chest X-ray image (T) through a cross-domain bidirectional image transformation network. xy (x) is then used to obtain the corresponding neural bone pattern (c) by using the CXR-VRT shared skeletal information cross-modal fusion model. Finally, the corresponding individual identity is retrieved in the neural bone pattern retrieval library (C) by nearest neighbor search. The specific formula is as follows: c=F(f(x),f′(T xy (x))) 9. A cross-modal fusion individual identity recognition system based on shared skeletal information, characterized in that, Individual identity recognition is performed using a cross-modal fusion method based on shared skeletal information as described in any one of claims 1-8.

10. A storage medium, characterized in that, The storage medium stores a computer program, which, when executed, performs an individual identity recognition method based on cross-modal fusion of shared skeletal information as described in any one of claims 1-8.

Images