Information processing method, device and equipment suitable for fundus image and medium

Abstract

This application discloses a method, apparatus, device, and medium for information processing of fundus images, relating to the field of information processing technology. The method includes: acquiring fundus images and vital sign data of a target object; extracting visual feature sequences from the fundus images using a preset visual feature extraction network; mapping the vital sign data into feature words based on a preset feature word converter, and generating corresponding semantic feature sequences through a self-attention mechanism; extracting target visual feature sequences that meet preset correlation conditions from the visual feature sequences based on a preset fusion classification network and according to the semantic feature sequences, and fusing the semantic feature sequences and target visual feature sequences to obtain a fused feature vector; and generating and outputting an acute mountain sickness risk assessment result based on the preset fusion classification network and the fused feature vector. This simulates the expert decision-making process integrating medical history, vital signs, and images, achieving automated early assessment of acute mountain sickness risk.

Description

Technical Field

[0001] This invention relates to the field of information processing technology, and in particular to information processing methods, apparatus, devices and media applicable to fundus images. Background Technology

[0002] Currently, in practical scenarios of high-altitude emergency medical care and medical support at all levels, there is a significant technological gap in the early screening of acute altitude sickness (AMS). Due to the extreme scarcity of professional ophthalmologists and altitude sickness specialists in high-altitude areas, grassroots health sentinel sites or portable mobile clinics often lack the professional personnel capable of independently and accurately diagnosing AMS. Therefore, there is an urgent need for an intelligent auxiliary screening method that can replace expert experience to solve the operational challenge of quickly identifying high-risk patients and guiding transfer decisions in harsh environments.

[0003] However, traditional screening processes have significant limitations and cannot meet the needs of precise prevention and control. On the one hand, the currently used Lake Louise scoring system mainly relies on patients' subjective self-reports (such as the degree of headache, whether they feel nauseous or vomiting). This single-dimensional approach to diagnosis is easily influenced by factors such as patients' psychological fear, common cold, or fatigue, leading to a high rate of misdiagnosis and a lack of objective physiological evidence. On the other hand, although fundus examination can objectively reflect changes in intracranial pressure, in practice, it has been found that relying solely on fundus images for isolated judgments is prone to missed diagnoses or misjudgments. This is because different populations have vastly different physiological tolerances to hypoxia in different environments. For example, the vascular responses of people from plains areas and people who have lived in high-altitude areas are completely different at the same altitude, and early, subtle papilledema features are easily overlooked by algorithms without the assistance of clinical symptoms.

[0004] Therefore, how to cross-validate multi-source information (fundus images and vital signs data) to comprehensively analyze and determine the risk assessment results of acute mountain sickness reflected by fundus images is a technical problem that needs to be solved in this field. Summary of the Invention

[0005] In view of this, the purpose of this invention is to provide a method, apparatus, device, and medium for processing information from fundus images, capable of cross-validating multi-source information (fundus images and vital sign data) to comprehensively analyze and determine the risk assessment results of acute mountain sickness fed back from the fundus images. The specific solution is as follows: In a first aspect, this application discloses an information processing method for fundus images, applied to a computer device, comprising: Acquire fundus images and vital sign data of the target object; The visual feature sequence of the fundus image is extracted using a preset visual feature extraction network; Based on a preset feature word converter, the vital signs data are mapped to feature words respectively, and corresponding semantic feature sequences are generated through a self-attention mechanism. Based on a preset fusion classification network, a target visual feature sequence that meets a preset correlation condition is extracted from the visual feature sequence according to the semantic feature sequence, and the semantic feature sequence and the target visual feature sequence are fused to obtain a fused feature vector. The acute mountain sickness risk assessment results are generated and output through the preset fusion classification network and based on the fusion feature vector.

[0006] Optionally, the vital signs data include the target's place of origin, the target's current altitude, the target's current headache level, and the target's vomiting level.

[0007] Optionally, the step of mapping the vital sign data into feature words based on a preset feature word converter includes: Determine the data type of each data variable in the vital signs data; If the data variable is the current altitude information or the current headache level information, then the data variable is determined to be a continuous variable. Then, a preset linear projection layer is used to perform projection mapping processing on the continuous variable to obtain feature words. If the data variable is the place of origin information or the vomiting degree information, then the data variable is determined to be a discrete variable. Then, the discrete variable is binned and embedded according to the preset embedding table to obtain feature words.

[0008] Optionally, the preset visual feature extraction network includes a preset fundus image recognition model and a feature pyramid network; Accordingly, the step of extracting the visual feature sequence of the fundus image using a preset visual feature extraction network includes: Based on the preset fundus image recognition model, image features at each level of the fundus image are extracted; wherein, the image features include optic disc morphology information, retinal vessel course and diameter information, and retinal hemorrhage or exudation changes information; The feature pyramid network is used to perform cross-level fusion processing on the semantic and texture information in the image features to generate a visual feature sequence.

[0009] Optionally, generating the corresponding semantic feature sequence through the self-attention mechanism includes: Each set of feature words is input into the self-attention layer to calculate the association weights between each feature word, and the vector representation of each feature word is updated according to the association weights to generate a semantic feature sequence containing contextual semantic information between vital signs data.

[0010] Optionally, the step of extracting a target visual feature sequence that meets a preset relevance condition from the visual feature sequence based on a preset fusion classification network and according to the semantic feature sequence, and fusing the semantic feature sequence and the target visual feature sequence to obtain a fused feature vector, includes: Based on a preset fusion classification network, the semantic feature sequence and the visual feature sequence are used to perform dot product similarity calculation to obtain a similarity score; The similarity scores are normalized using the softmax function of the preset fusion classification network to obtain an attention weight matrix; wherein the attention weight matrix represents the correlation condition between each feature in the visual feature sequence and the semantic feature sequence. The visual feature sequence is weighted and summed using the preset fusion classification network and the attention weight matrix to extract the target visual feature sequence. The semantic feature sequence and the target visual feature sequence are then fused to obtain a fused feature vector.

[0011] Optionally, information processing methods suitable for fundus images also include: Obtain the training sample set; each training sample includes historical fundus images, historical vital sign data, and corresponding acute mountain sickness risk level labels; Construct a multimodal neural network to be trained; wherein the multimodal neural network to be trained includes: an initial visual feature extraction network, an initial feature word converter, and an initial fusion classification network; Construct a joint loss function that includes a main loss function for optimizing risk classification and an auxiliary contrastive loss function for narrowing the distance between high-risk image-high-risk feature training sample pairs in the feature space; Freeze all network parameters of the initial visual feature extraction network, and use the training sample set and the joint loss function to train and optimize the network parameters of the initial feature word converter and the initial fusion classification network; Unfreeze the target network parameters in the initial visual feature extraction network, and use the training sample set and the joint loss function to jointly fine-tune all the training parameters of the multimodal neural network to be trained, so as to obtain a preset visual feature extraction network, a preset feature word converter, and a preset fusion classification network.

[0012] Secondly, this application discloses an information processing device suitable for fundus images, comprising: The data acquisition module is used to acquire fundus images and vital sign data of the target object; The sequence extraction module is used to extract the visual feature sequence of the fundus image using a preset visual feature extraction network; The sequence generation module is used to map the vital sign data into feature words based on a preset feature word converter, and generate corresponding semantic feature sequences through a self-attention mechanism. The fusion module is used to extract a target visual feature sequence that meets a preset correlation condition from the visual feature sequence based on a preset fusion classification network and according to the semantic feature sequence, and to fuse the semantic feature sequence and the target visual feature sequence to obtain a fusion feature vector. The classification module is used to generate and output the acute mountain sickness risk assessment result through the preset fusion classification network and based on the fusion feature vector.

[0013] Thirdly, this application discloses an electronic device, including: Memory, used to store computer programs; A processor is configured to execute the computer program to implement the steps of the aforementioned disclosed information processing method for fundus images.

[0014] Fourthly, this application discloses a computer-readable storage medium for storing a computer program; wherein, when the computer program is executed by a processor, it implements the steps of the aforementioned disclosed information processing method for fundus images.

[0015] As can be seen, this application discloses the following: acquiring fundus images and vital sign data of a target object; extracting visual feature sequences from the fundus images using a preset visual feature extraction network; mapping the vital sign data into feature words based on a preset feature word converter, and generating corresponding semantic feature sequences through a self-attention mechanism; extracting target visual feature sequences that meet preset correlation conditions from the visual feature sequences based on a preset fusion classification network and according to the semantic feature sequences, and fusing the semantic feature sequences with the target visual feature sequences to obtain a fused feature vector; generating and outputting acute mountain sickness risk assessment results based on the preset fusion classification network and according to the fused feature vector. Thus, by using a visual feature extraction network and a feature word converter to extract high-order features from fundus images and multivariate vital sign data respectively, and then using an attention mechanism guided by the semantics of vital signs to screen out target features highly correlated with clinical indicators from the fundus visual sequences for deep fusion, this process simulates the expert's decision-making process based on medical history, vital signs, and images, achieving comprehensive processing, analysis, and evaluation of the integrated feature data and fundus images. Attached Figure Description

[0016] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on the provided drawings without creative effort.

[0017] Figure 1 This application discloses a flowchart of an information processing method for fundus images. Figure 2 This is a schematic diagram of the structure of an information processing device for fundus images disclosed in this application; Figure 3 This is a structural diagram of an electronic device disclosed in this application. Detailed Implementation

[0018] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those of ordinary skill in the art without creative effort are within the scope of protection of the present invention.

[0019] Currently, in practical scenarios of high-altitude emergency medical care and medical support at all levels, there is a significant technological gap in the early screening of acute altitude sickness (AMS). Due to the extreme scarcity of professional ophthalmologists and altitude sickness specialists in high-altitude areas, grassroots health sentinel sites or portable mobile clinics often lack the professional personnel capable of independently and accurately diagnosing AMS. Therefore, there is an urgent need for an intelligent auxiliary screening method that can replace expert experience to solve the operational challenge of quickly identifying high-risk patients and guiding transfer decisions in harsh environments.

[0020] However, traditional screening processes have significant limitations and cannot meet the needs of precise prevention and control. On the one hand, the currently used Lake Louise scoring system mainly relies on patients' subjective self-reports (such as the degree of headache, whether they feel nauseous or vomiting). This single-dimensional approach to diagnosis is easily influenced by factors such as patients' psychological fear, common cold, or fatigue, leading to a high rate of misdiagnosis and a lack of objective physiological evidence. On the other hand, although fundus examination can objectively reflect changes in intracranial pressure, in practice, it has been found that relying solely on fundus images for isolated judgments is prone to missed diagnoses or misjudgments. This is because different populations have vastly different physiological tolerances to hypoxia in different environments. For example, the vascular responses of people from plains areas and people who have lived in high-altitude areas are completely different at the same altitude, and early, subtle papilledema features are easily overlooked by algorithms without the assistance of clinical symptoms.

[0021] Therefore, the present invention provides an information processing scheme suitable for fundus images, which can cross-verify multi-source information (fundus images and vital sign data) to comprehensively analyze and determine the risk assessment results of acute mountain sickness fed back by fundus images.

[0022] like Figure 1 As shown, the present invention provides an information processing method for fundus images, applied to a computer device, comprising: Step S11: Obtain fundus images and vital sign data of the target object.

[0023] In this embodiment, fundus images and vital sign data are acquired through a sensing front-end of a computer device. The vital sign data includes the target subject's place of origin, current altitude, current headache level, and vomiting level. Specifically, fundus images are acquired using a handheld, non-mydriatic fundus camera. This device features autofocus and low-light imaging capabilities, capturing high-resolution images of the patient's retina and transmitting them losslessly to the computing terminal via USB 3.0 or Wi-Fi Direct. Simultaneously, the acquisition of vital sign and environmental data is integrated into the human-computer interaction interface module of the sensing terminal. This module automatically acquires latitude, longitude, and altitude data using the terminal's built-in GPS sensor and allows medical personnel to input structured text information such as the target subject's place of origin, duration of stay at high altitude, and headache and vomiting levels through a touchscreen form interface, thus forming a complete audiovisual multimodal input stream. Furthermore, the sensing front-end is physically connected to the core computing terminal via a high-speed data interface.

[0024] Step S12: Extract the visual feature sequence of the fundus image using a preset visual feature extraction network.

[0025] In this embodiment, the core computing terminal is a highly integrated edge computing physical architecture. Specifically, the core computing terminal is a medical-grade ruggedized mobile computing terminal equipped with a high-performance independent graphics acceleration chip (GPU) or an embedded AI accelerator (such as an NPU). This terminal not only serves as the control center of the entire computing device but also directly carries the lightweight, adapted AMS-Informer multimodal AI model. This ensures that even in extreme environments with no network or poor connectivity, it can independently and in real-time complete the entire process from data reading and image preprocessing to complex model inference, eliminating strong dependence on cloud servers and guaranteeing diagnostic efficiency during the critical rescue time. Specifically, the visual feature sequence of the fundus image is extracted through a preset visual feature extraction network of the core computing terminal. Specifically, the preset visual feature extraction network includes a preset fundus image recognition model and a feature pyramid network. Correspondingly, the extraction of the visual feature sequence of the fundus image using the preset visual feature extraction network includes: extracting image features at each level of the fundus image based on the preset fundus image recognition model; wherein, the image features include optic disc morphology information, retinal vessel course and diameter information, and retinal hemorrhage or exudation changes information; and performing cross-level fusion processing on the semantic and texture information in the image features through the feature pyramid network to generate the visual feature sequence. It can be understood that the preset visual feature extraction network uses the RetiZero (preset fundus image recognition model) pre-trained fundus image model based on the Vision Transformer architecture as the backbone network, and combines it with a Feature Pyramid Network (FPN) to construct multi-scale visual feature sequences. The specific generation process of the video feature sequence is as follows: In the inference phase of this embodiment, the preset visual feature extraction network has completed parameter optimization through the aforementioned training process. Specifically: the general feature extraction capability of the RetiZero backbone network, pre-trained on a large-scale fundus dataset, has been fully retained, and the lightweight adaptation parameters introduced through low-rank adaptation (LoRA) technology have been trained and solidified. During inference, all parameters of RetiZero (including pre-trained weights and LoRA adaptation parameters) are frozen, and only forward propagation calculations are performed without parameter updates. The 1×1 convolutional layers, upsampling fusion paths, and optional downsampling convolutional layers of the feature pyramid network (FPN) have all completed parameter optimization through end-to-end training. During inference, the parameters of each layer of the FPN are also frozen, and the learned feature fusion strategy is directly executed. When a fundus image to be evaluated is input, the visual feature extraction module performs forward inference calculations: The original fundus image size is adjusted to the desired input size of the RetiZero network; in this embodiment, it is set to 640×480 pixels, and pixel value normalization is performed. The preprocessed image tensor is fed into the RetiZero backbone network. The trained RetiZero network performs forward propagation on the input image tensor. During the computation, the network outputs feature maps at preset intermediate layer positions. In this embodiment, feature maps are extracted from the following four layers of RetiZero: Layer 3 output (shallow feature map): size 40×30, 256 channels. This feature map retains rich spatial detail information, specifically the morphological contours of retinal vessels, the boundary texture of hemorrhages, and the local gray-scale gradients at the optic disc edge. These detailed features are important visual evidence for identifying early microcirculatory disturbances.

[0026] Layer 6 output (middle-layer feature map): size 20×15, number of channels 384. This feature map aggregates local textures while compressing spatial resolution, forming the response patterns of local structural units, including the course of blood vessel segments, the continuous contour of the optic disc boundary, and the topological morphology of blood vessel bifurcation points.

[0027] The outputs of layers 9 and 12 (deep feature maps) have sizes of 10×8 and 5×4, respectively, and 768 channels each. These two feature maps have global receptive fields, and their activation values ​​represent high-level semantic concepts, specifically including the overall edema score of the optic disc, diffuse color shift in the retinal background, and the tortuosity and dilation index of the main veins.

[0028] The four feature maps mentioned above constitute the input data for subsequent fusion processing. It should be noted that, since RetiZero has been trained and frozen, the feature maps generated by each forward propagation for the same patient under the same shooting conditions are deterministic and reproducible.

[0029] When the trained FPN receives the above four feature maps, it performs a top-down cross-scale fusion calculation: the number of channels of the four feature maps at different levels is uniformly compressed to 256 dimensions through the pre-trained 1×1 convolutional layer, and the compressed feature map is output. The 1×1 convolutional layer has learned to perform semantic recalibration on the original features during the training phase, which can enhance the response patterns related to the pathology of altitude sickness and suppress the noise response that is irrelevant to the task.

[0030] Starting with the deep feature map (output of layer 12), additive fusion is performed layer by layer with lower-level features. Specifically, the compressed feature map of layer 12 is first upsampled by 2x bilinearly, expanding its size to 10×8, and then added element-wise with the compressed feature map of layer 9 to generate a new fused deep feature map. This new fused deep feature map is then upsampled by 2x bilinearly, expanding its size to 20×15, and then added element-wise with the compressed feature map of layer 6 to generate a new fused mid-level feature map. This process is repeated until a new fused shallow feature map is obtained. Since the upsampling operation uses bilinear interpolation, its interpolation weights are fixed and do not require learning. Additive fusion directly adds the values ​​at each spatial location of the two feature maps. Because the FPN has already been trained, the contribution ratio of each layer's features during fusion is implicitly encoded in the parameters of the 1×1 convolution, requiring no additional adjustment during inference. Then, the new fused shallow feature map is downsampled by a 3×3 convolutional layer with a stride of 2 to generate a downsampled feature map with a size of 20×15 and 256 channels. Since this convolutional layer has learned to abstract global background features from local details during the training phase, it is used to characterize macroscopic visual cues such as the overall color of the retina and the morphology of retinal blood vessels.

[0031] The above steps generate a set of multi-scale feature maps (new fused deep feature map, new fused mid-level feature map, new fused shallow feature map, and downsampled feature map), whose spatial dimensions and channel number are aligned to 256 dimensions. To adapt to the format requirements of the subsequent cross-modal cross-attention module for serialized input, the following serialization transformation is performed: Each feature map is flattened into a one-dimensional sequence based on its spatial dimension (height × width). Specifically: the new fused shallow feature map is flattened into a sequence of length 1200; the new fused mid-layer feature map is flattened into a sequence of length 300; the new fused deep feature map is flattened into a sequence of length 80; and the downsampled feature map is flattened into a sequence of length 300. These four flattened sequences are then concatenated sequentially along their length dimension to form a visual feature sequence of total length 1880. Each position in this sequence corresponds to a spatial region on the original fundus image: the first 1200 positions correspond to various local regions in the original 40×30 grid (small receptive field, focusing on details such as microvessels and optic disc edges); the middle 300 positions correspond to a 20×15 grid (moderate receptive field, representing vascular segments and local optic disc contours); the last 80 positions correspond to a 10×8 grid (larger receptive field, representing the overall morphology of the optic disc); and the last 300 positions correspond to the global context features of the 20×15 grid (receptive field covering the entire image, representing background color). This arrangement preserves the spatial topological relationship of the feature maps, and features of different scales are arranged in segments and continuously in the sequence, providing positional awareness for subsequent attention calculations.

[0032] In this way, since the model parameters are completely frozen after training, stable and repeatable visual feature sequences can be generated for the same input image, ensuring the consistency and reliability of diagnostic results. The conversion from a single fundus image to a 1880×256-dimensional visual feature sequence can be completed in milliseconds on an edge computing terminal equipped with a GPU or NPU, meeting the real-time requirements of emergency scenarios. The output visual feature sequence has a fixed dimensional format and semantic meaning, which can be directly used as the input of the subsequent cross-modal attention module without additional adaptation or post-processing.

[0033] Step S13: Based on the preset feature word converter, the vital signs data are mapped to feature words respectively, and corresponding semantic feature sequences are generated through the self-attention mechanism.

[0034] In this embodiment, the data type of each data variable of the vital signs data is determined. If the data variable is the current altitude information or the current headache level information, it is determined that the data variable is a continuous variable. Then, a preset linear projection layer is used to perform projection mapping processing on the continuous variable to obtain feature tokens. If the data variable is the place of origin information or the vomiting level information, it is determined that the data variable is a discrete variable. Then, the discrete variable is binned and embedded according to a preset embedding table to obtain feature tokens. It can be understood that the preset feature token converter adopts the FT-Transformer model based on the Feature Tokenizer architecture. This model has completed end-to-end training and is deployed on an edge computing terminal. After the module has completed training and deployment, the specific process of processing the input vital signs data and generating semantic feature sequences in the inference stage is as follows: In the inference stage, the vital signs data of the target object is received through the human-computer interaction interface or external interface. In this embodiment, the vital signs data includes at least the following four core fields, as shown in Table 1: Table 1. Vital Signs Data and Explanations

[0035] It's important to note that a field type mapping table is generated internally within the computer device. This table is fixed during the model training phase and used to quickly determine the data type of each input variable during inference. Based on this mapping table, the following are automatically identified: continuous variables: current altitude, headache VAS score; discrete variables: place of origin, vomiting severity. Thus, when the data type corresponding to a data variable is identified, it is directly mapped using a pre-defined linear projection layer or a pre-defined embedding table. Specifically, for continuous variables, a pre-defined linear projection layer is used for mapping. This linear projection layer is a single-layer fully connected neural network whose weight matrix and bias vector have been optimized through end-to-end backpropagation during training and are completely frozen during inference. Taking the current altitude of 4500 meters as an example, the value is first standardized preprocessed. Based on the mean and standard deviation of the training set statistics, the original altitude value is converted into a standardized score. The standardized scalar value is input into the linear projection layer, and the corresponding feature words are generated through the following calculations: ; in, Characteristic terms representing continuous variables Represents standard scores, Represents the weight matrix. This represents the bias vector.

[0036] In this way, a single scalar value is directly mapped to a dense vector of 256 dimensions (set in this embodiment). Since the weight matrix has learned the non-linear relationship between altitude values ​​and altitude sickness risk during training, the magnitude of the value will linearly affect the magnitude and activation direction of the output vector. That is, the higher the altitude, the greater the response strength of the vector in a specific semantic dimension. The gradient information of continuous variable values ​​is fully preserved, avoiding the information loss and boundary ambiguity problems caused by traditional bucketing strategies.

[0037] For discrete variables, a pre-defined embedding table is used for mapping. The embedding table is essentially a trainable lookup matrix with an embedding dimension of 256. This matrix is ​​optimized through backpropagation during training and is completely frozen during inference. Taking the place of origin variable as an example, this variable contains two categories: plains and plateau. An embedding table of size is maintained. When the input value is "plains," the index corresponding to that category (e.g., 0) is automatically looked up, and the 0th row vector is retrieved from the embedding table as the feature words for that variable. ; in, This is the vector of the 0th row of the embedded table. These are the feature words of a discrete vector.

[0038] A discrete vector is a dense, fixed-dimensional semantic representation whose numerical distribution encodes prior knowledge of the category in altitude sickness risk assessment. For example, the embedding vector of the plains category after training may have high activation values ​​in a specific dimension, representing the semantic attribute of low tolerance. Similarly, the vomiting severity variable (level 0-3) generates feature terms by looking up the row vector corresponding to the input level (e.g., level 2) through its dedicated embedding table. This will not be elaborated further.

[0039] In this embodiment, each set of feature words is input into the self-attention layer to calculate the association weights between each feature word, and the vector representation of each feature word is updated according to the association weights to generate a semantic feature sequence containing contextual semantic information between vital sign data. It is understood that, to capture the nonlinear associations and synergistic effects between vital sign variables, the feature word sequence is input into a trained Transformer self-attention layer. This self-attention layer consists of multiple stacked encoder blocks; this embodiment uses two layers, each with eight attention heads, and all parameters are completely frozen during the inference phase. Since there is no natural sequence order among vital sign variables, but to help the self-attention mechanism distinguish different variables, this embodiment adds a learnable positional encoding vector to each word before inputting it into the self-attention layer. This positional encoding is jointly optimized with other model parameters during the training phase, and the fixed value is directly used during inference to obtain the initial feature word sequence after adding the positional encoding. The initial feature word sequence after adding the positional encoding is input into the first Transformer encoder layer, each encoder containing a multi-head self-attention sublayer and a feedforward neural network sublayer. Taking single-head attention as an example, the calculation process is as follows: The input sequence is processed through three linear projection matrices to generate a query matrix, a key matrix, and a value matrix. Then, an attention weight matrix is ​​calculated to output a weighted feature representation. Multi-head attention concatenates and projects the weighted feature representations from multiple heads, capturing patterns of association between variables in different subspaces. Through self-attention calculation, the vector representation of each word is dynamically updated based on its relevance to other words in the sequence. For example, when the current altitude is high (e.g., 4500 meters), the self-attention mechanism amplifies the attention weight of the headache VAS score word on the continuous variable (altitude), incorporating the contextual information of the severity of headache symptoms at high altitudes into the updated vector. Simultaneously, the vector representation of place of origin (e.g., plains) further modulates this association; if the place of origin is plains, the model assigns a higher contribution weight to the discrete variable (plains); if the place of origin is plateau, the weight decreases accordingly. After processing by two Transformer encoder layers, the vector of each word has aggregated global contextual information from all other relevant variables. The output sequence at this point is the semantic feature sequence. Each feature term in this sequence is no longer an independent encoding of an isolated variable, but rather an embedding vector containing rich clinical semantic context. The entire sequence will be used as a query vector input to the subsequent cross-modal cross-attention module for deep fusion with the visual feature sequence.

[0040] In this way, to address the heterogeneity between continuous and discrete variables in the vital signs data, linear projection and embedding tables are used for feature lexical generation, respectively, preserving the gradient information of continuous values ​​while assigning learnable semantic representations to discrete categories. Through a self-attention mechanism, the feature representation of each vital sign variable is integrated in real time with the global clinical context of other relevant variables, simulating the cognitive process of experts making judgments based on multi-dimensional information. The output semantic feature sequence has a fixed length and dimension, a uniform format, and stable semantics, and can be directly used as the query vector for subsequent cross-modal cross-attention modules without additional adaptation.

[0041] Step S14: Based on a preset fusion classification network and according to the semantic feature sequence, extract a target visual feature sequence that meets the preset correlation condition from the visual feature sequence, and perform fusion processing on the semantic feature sequence and the target visual feature sequence to obtain a fusion feature vector.

[0042] In this embodiment, a similarity score is obtained by calculating the dot product similarity between the semantic feature sequence and the visual feature sequence based on a preset fusion classification network. The similarity score is then normalized using the softmax function of the preset fusion classification network to obtain an attention weight matrix. This attention weight matrix represents the relevance between each feature in the visual feature sequence and the semantic feature sequence. The visual feature sequence is then weighted and summed using the preset fusion classification network and the attention weight matrix to extract a target visual feature sequence. Finally, the semantic feature sequence and the target visual feature sequence are fused to obtain a fused feature vector. It is understood that the preset fusion classification network includes a cross-modal cross-attention layer and a gated multimodal unit (GMU). This network has been trained end-to-end and deployed on an edge computing terminal. After training and deployment, the module receives visual and semantic feature sequences, performs deep fusion, and generates a fused feature vector during the actual inference phase. The specific process is as follows: Since the visual and semantic feature sequences can be directly multiplied in the current dimensional space (both 256 dimensions), this embodiment does not require additional linear projection to perform subsequent attention calculations. In other embodiments, if the dimensions of the two sequences are inconsistent, they need to be mapped to the same hidden layer space through a trained linear projection layer. In this embodiment, the dimensions of the two sequences are naturally aligned, thus entering the cross-attention calculation stage.

[0043] In this embodiment, a cross-attention mechanism is used, employing semantic feature sequences as queries and visual feature sequences as keys and values. This mechanism simulates the diagnostic thinking of a clinical expert reviewing fundus images based on the patient's chief complaint. First, query, key, and value matrices are generated, with the semantic feature sequences directly used as the query matrix. The visual feature sequences are used as the key and value matrices, respectively. All matrices directly use the intermediate results of forward propagation during inference without any additional transformation. For each query vector in the query matrix, a dot product operation is performed with each key vector in the key matrix to obtain a similarity score matrix. The similarity score represents the degree of matching between an individual's semantic features and a visual region in the feature space; a higher score indicates a stronger correlation between the visual region and the current clinical indication. Then, the Softmax function is applied to normalize each row of the similarity score matrix (i.e., the scores of all visual positions corresponding to each query vector) to obtain an attention weight matrix. Each element in this weight matrix represents the degree of attention paid to the visual region by the feature terms of the vital sign data. Positions with high weights are determined by the model to be target visual regions highly correlated with the current clinical indication. Then, for each query vector, the value matrix is ​​weighted and summed using its corresponding attention weight to obtain a new feature vector. This vector represents the visual features extracted from the fundus image and weighted by correlation for a specific vital sign data. All the new feature vectors are concatenated in the original query order to obtain the target visual feature sequence.

[0044] To further balance the contributions of visual and semantic modalities in decision-making, the target visual feature sequence and the original semantic feature sequence are jointly input into a gated multimodal unit for adaptive fusion. First, the two are concatenated along the sequence length dimension to obtain a joint feature sequence. This joint feature sequence is then input into a pre-trained single-layer fully connected network, which outputs a scalar gating coefficient. The specific calculation is as follows: global average pooling is applied to the joint feature sequence, compressing it into a 256-dimensional global vector; this global vector is input into the fully connected layer, outputting a scalar; the scalar is then compressed to a preset interval [0,1] using the sigmoid function to obtain the gating coefficient. Specifically, this gating coefficient represents the credibility of the visual modality in the current sample. When the fundus image is clear and the lesion features are significant, the gating coefficient approaches 1; when the image is blurry, the illumination is uneven, or the lesion features are not obvious, the gating coefficient approaches 0. The target visual feature sequence and the semantic feature sequence are then weighted and summed using the gating coefficient to obtain the final fused feature vector.

[0045] Step S15: Generate and output the acute mountain sickness risk assessment result through the preset fusion classification network and based on the fusion feature vector.

[0046] In this embodiment, the final generated fused feature vector will be passed to the classification head of the preset fusion classification network. The classification head consists of two fully connected layers and is used to output the probability distribution of the risk level of acute mountain sickness (e.g., low risk, medium risk, high risk). After the probability distribution is normalized by Softmax, it is combined with a preset risk threshold to generate the final risk assessment result and output it.

[0047] In this embodiment, the training process for each part of the core computing terminal is as follows: A training sample set is obtained; each training sample includes historical fundus images, historical vital sign data, and corresponding acute mountain sickness risk level labels; a multimodal neural network to be trained is constructed; the multimodal neural network to be trained includes: an initial visual feature extraction network, an initial feature word converter, and an initial fusion classification network; a joint loss function is constructed, including a main loss function for optimizing risk classification and an auxiliary contrast loss function for narrowing the distance between high-risk image-high-risk vital sign training sample pairs in the feature space; all network parameters of the initial visual feature extraction network are frozen, and the network parameters of the initial feature word converter and the initial fusion classification network are trained and optimized using the training sample set and the joint loss function; the target network parameters in the initial visual feature extraction network are unfrozen, and all training parameters of the multimodal neural network to be trained are jointly fine-tuned using the training sample set and the joint loss function to obtain a preset visual feature extraction network, a preset feature word converter, and a preset fusion classification network. It is understood that the preset visual feature extraction network, preset feature term converter, cross-modal fusion module, and classification head constitute an end-to-end multimodal neural network, namely AMS-Informer, which is the core computing terminal part. This network needs to undergo a phased, multi-task joint optimization training process to achieve optimal performance in high-altitude disease screening scenarios with small samples, class imbalance, and difficulties in multimodal alignment. The training dataset used in this embodiment comes from real clinical collections from multiple medical sites in high-altitude areas. Each training sample includes: fundus image: taken by a portable non-mydriatic fundus camera, uniformly preprocessed into a 640×480 pixel RGB image; vital sign data vector: including place of origin (binary classification), current altitude (continuous value, unit: meters), duration of stay in high-altitude areas (continuous value, unit: hours), headache VAS score (continuous value, 0-10), vomiting severity (four-category level 0-3); risk level label: jointly determined by at least two high-altitude medical experts based on the Lake Louise scoring system and clinical outcomes, divided into three levels: low risk, medium risk, and high risk. This embodiment merges medium-risk and high-risk samples into positive samples and low-risk samples into negative samples for binary classification training; at the same time, it retains the tertiary labels for fine-grained evaluation. Addressing the limited sample size of fundus images (approximately 800 cases) and the fact that positive samples account for less than 30%, this embodiment employs the following strategies to expand and balance the dataset: Image data augmentation: Online augmentation methods such as random horizontal flipping, small-angle rotation ±10°, brightness and contrast fine-tuning, and Gaussian noise injection are applied to the fundus images in the training set, dynamically generating augmented views in each iteration; Class-balanced sampling: A weighted random sampler is used to assign higher sampling weights to positive samples, ensuring that the proportion of positive samples in each batch is no less than 40%.

[0048] The backbone network RetiZero uses weights pre-trained through self-supervised training on a large-scale public fundus dataset (containing multiple diseases such as diabetic retinopathy, glaucoma, and age-related macular degeneration) as a general fundus feature extractor. This pre-trained model already possesses strong generalization ability to recognize anatomical structures such as the optic disc, blood vessels, and macula.

[0049] The Low-Rank Adaptation (LoRA) module inserts two low-rank decomposition matrices in parallel alongside the query and value projection matrices of each Transformer encoding layer in RetiZero. The original pre-trained weights are completely frozen, and only the low-rank matrix parameters introduced by LoRA (approximately 0.5% of the total parameters) are labeled as trainable. This design allows the model to update only a very small number of parameters during fine-tuning, effectively preventing catastrophic forgetting in small sample sizes.

[0050] Feature Pyramid Network (FPN): Its horizontally connected 1×1 convolutional layers, upsampling paths, and optional downsampling convolutional layers are all randomly initialized, and the parameters follow a Kaiming normal distribution.

[0051] Continuous variable linear projection layer: A single-layer fully connected network is constructed for each continuous variable (altitude, duration of stay at high altitude, and headache VAS score), with an input dimension of 1 and an output dimension of 256. The weight matrix is ​​initialized using Xavier, and the bias is initialized to 0.

[0052] Discrete variable embedding table: An embedding lookup matrix is ​​constructed for each discrete variable (place of origin, degree of vomiting), with the number of rows equal to the number of categories and the number of columns being 256. The embedding vectors are randomly initialized using a normal distribution with a mean of 0 and a standard deviation of 0.1.

[0053] Self-attention encoder: Employs a stack of two Transformer encoder layers, each containing eight attention heads. The feedforward network has a hidden layer dimension of 1024 and an output dimension of 256. All parameters are initialized using Xavier.

[0054] Cross-modal attention layer: In this embodiment, the visual and semantic feature dimensions are naturally aligned (both are 256), so no additional projection layer is needed. Attention calculation uses scaled dot product attention, without adding any new trainable parameters.

[0055] The gated multimodal unit (GMU) consists of a global average pooling layer, a fully connected layer (input 256, output 1), and a sigmoid activation function. The weights of the fully connected layer are initialized using Xavier, and the biases are initialized to 0.

[0056] The classification head consists of two fully connected layers, each followed by ReLU activation and Dropout (dropout rate of 0.3). The last layer outputs 3-dimensional logits, corresponding to low, medium, and high risk levels, with weights initialized using Xavier.

[0057] This embodiment employs a multi-task joint loss function to simultaneously optimize risk classification accuracy and multi-modal feature alignment quality. The total loss function is defined as: To address the issues of a low proportion of positive samples in the training set and the difficulty in classifying some positive samples (early-stage patients) due to weak fundus signs, this embodiment employs focal loss as the main loss function. This loss function reduces the loss contribution of easily classified negative samples, directing the training focus to difficult-to-classify positive samples and effectively alleviating the class imbalance problem. InfoNCE loss is selected as the auxiliary contrast loss.

[0058] This embodiment constructs a multimodal contrastive learning task: visual and semantic features of the same patient are used as positive sample pairs, and visual and semantic features of different patients are randomly combined as negative sample pairs. The InfoNCE loss function is used to shorten the distance between positive sample pairs and widen the distance between negative sample pairs: in this embodiment, the semantic features of all other patients within a batch are used as negative samples (batch size = 32, i.e., each batch contains 31 negative samples). This contrastive loss-forced model aligns the image features and vital signs of high-risk patients in the feature space, enabling the model to activate corresponding visual modes through vital sign cues even if image signs are mild, thereby improving the sensitivity of early diagnosis.

[0059] To maximize the retention of pre-trained knowledge and task adaptation with limited samples, this embodiment employs a two-stage course learning strategy for end-to-end training. All training processes utilize the AdamW optimizer with weight decay set to 1e-4.

[0060] Phase 1: Semantic and Fusion Pre-training. Objective: While freezing the visual backbone, prioritize training the semantic feature embedding network, cross-modal fusion module, and classification head to quickly converge to a reasonable decision boundary. Hyperparameter configuration: Optimizer: AdamW, initial learning rate 1e-3; batch size: 32; training epochs: 50; learning rate scheduling: cosine annealing, 5 warm-up epochs. Trainable parameter set: continuous variable linear projection layer, discrete variable embedding table, self-attention encoder, gated multimodal unit (GMU), classification head; while the RetiZero backbone network, the LoRA low-rank matrix in RetiZero, and the Feature Pyramid Network (FPN) are completely frozen. The current stage objective is to enable the model to achieve basic classification performance relying solely on vital sign data and the fusion module, while allowing the cross-modal attention mechanism to initially learn to associate vital sign queries with visual features (although the visual features are not yet optimized for altitude sickness at this stage).

[0061] Phase 2: Visual Flow Adaptation and Joint Fine-Tuning. Objective: To unfreeze the lightweight adaptation parameters (LoRA) in the visual feature extraction network, shifting visual features towards the pathological representation of altitude sickness, while simultaneously optimizing in conjunction with the converged semantic and fusion modules. Hyperparameter configuration: Optimizer: AdamW, initial learning rate 5e-5 (visual flow) / 1e-4 (other modules); batch size: 32; training epochs: 100; learning rate scheduling: cosine annealing, 10 warm-up epochs; LoRA rank $r=8$, low-rank matrix parameters are randomly initialized using a Gaussian distribution. The trainable parameter set includes: the LoRA low-rank matrix (newly unfrozen) in the RetiZero backbone network, the Feature Pyramid Network (FPN) (newly unfrozen), the continuous variable linear projection layer (maintained training), the discrete variable embedding table (maintained training), the self-attention encoder (maintained training), the gated multimodal unit (GMU) (maintained training), and the classification head (maintained training), while the original pre-trained weights of the RetiZero backbone network (maintained frozen). The goal at this stage is to fine-tune the visual feature extraction network with a small learning rate, gradually enhancing its response sensitivity to pathological features related to altitude sickness, such as papilledema, venous dilation, and microhemorrhages, without forgetting general fundus structure knowledge. Simultaneously, the optimization of visual features will feed back into the representation of semantic features and the fusion layer, achieving end-to-end global optimization.

[0062] As can be seen, this application discloses the following: acquiring fundus images and vital sign data of a target object; extracting visual feature sequences from the fundus images using a preset visual feature extraction network; mapping the vital sign data into feature words based on a preset feature word converter, and generating corresponding semantic feature sequences through a self-attention mechanism; extracting target visual feature sequences that meet preset correlation conditions from the visual feature sequences based on a preset fusion classification network and according to the semantic feature sequences, and fusing the semantic feature sequences with the target visual feature sequences to obtain a fused feature vector; generating and outputting acute mountain sickness risk assessment results based on the preset fusion classification network and according to the fused feature vector. Thus, by using a visual feature extraction network and a feature word converter to extract high-order features from fundus images and multivariate vital sign data respectively, and then using an attention mechanism guided by the semantics of vital signs to screen out target features highly correlated with clinical indicators from the fundus visual sequences for deep fusion, this process simulates the expert's decision-making process based on medical history, vital signs, and images, achieving comprehensive processing, analysis, and evaluation of the integrated feature data and fundus images.

[0063] like Figure 2 As shown, the present invention also discloses an information processing device suitable for fundus images, comprising: Data acquisition module 11 is used to acquire fundus images and vital sign data of the target object; The sequence extraction module 12 is used to extract the visual feature sequence of the fundus image using a preset visual feature extraction network; Sequence generation module 13 is used to map the vital signs data into feature words based on a preset feature word converter, and generate corresponding semantic feature sequences through a self-attention mechanism; The fusion module 14 is used to extract a target visual feature sequence that meets a preset correlation condition from the visual feature sequence based on a preset fusion classification network and according to the semantic feature sequence, and to fuse the semantic feature sequence and the target visual feature sequence to obtain a fusion feature vector. The classification module 15 is used to generate and output the acute mountain sickness risk assessment result through the preset fusion classification network and based on the fusion feature vector.

[0064] It is evident that by cascading the sequence extraction module, sequence generation module, and fusion module, and using semantic feature sequences as the query guide, the system dynamically retrieves and extracts target visual regions highly correlated with clinical indications from visual feature sequences. This overcomes the technical bottleneck of traditional methods that separate image analysis from medical history, and improves the sensitivity of identifying early and subtle pathological changes.

[0065] Furthermore, embodiments of this application also disclose an electronic device, Figure 3 This is a structural diagram of an electronic device 20 according to an exemplary embodiment. The content of the diagram should not be construed as limiting the scope of this application.

[0066] Figure 3 This is a schematic diagram of the structure of an electronic device 20 provided in an embodiment of this application. Specifically, the electronic device 20 may include: at least one processor 21, at least one memory 22, a power supply 23, a communication interface 24, an input / output interface 25, and a communication bus 26. The memory 22 stores a computer program, which is loaded and executed by the processor 21 to implement the relevant steps in the information processing method for fundus images disclosed in any of the foregoing embodiments. Alternatively, the electronic device 20 in this embodiment may specifically be an electronic computer.

[0067] In this embodiment, the power supply 23 is used to provide operating voltage for each hardware device on the electronic device 20; the communication interface 24 can create a data transmission channel between the electronic device 20 and external devices, and the communication protocol it follows can be any communication protocol applicable to the technical solution of this application, and is not specifically limited here; the input / output interface 25 is used to acquire external input data or output data to the outside world, and its specific interface type can be selected according to specific application needs, and is not specifically limited here.

[0068] The processor 21 may include one or more processing cores, such as a quad-core processor or an octa-core processor. The processor 21 may be implemented using at least one hardware form selected from DSP (Digital Signal Processing), FPGA (Field-Programmable Gate Array), and PLA (Programmable Logic Array). The processor 21 may also include a main processor and a coprocessor. The main processor, also known as a CPU (Central Processing Unit), is used to process data in the wake-up state; the coprocessor is a low-power processor used to process data in the standby state. In some embodiments, the processor 21 may integrate a GPU (Graphics Processing Unit), which is responsible for rendering and drawing the content to be displayed on the screen. In some embodiments, the processor 21 may also include an AI (Artificial Intelligence) processor, which is used to handle computational operations related to machine learning.

[0069] In addition, the memory 22, as a carrier for resource storage, can be a read-only memory, random access memory, disk or optical disk, etc. The resources stored thereon can include operating system 221, computer program 222, etc., and the storage method can be temporary storage or permanent storage.

[0070] The operating system 221 manages and controls the various hardware devices and computer programs 222 on the electronic device 20 to enable the processor 21 to perform calculations and processing on the massive amounts of data 223 in the memory 22. The operating system 221 can be Windows Server, Netware, Unix, Linux, etc. The computer program 222, in addition to including a computer program capable of performing the information processing method for fundus images executed by the electronic device 20 as disclosed in any of the foregoing embodiments, may further include computer programs capable of performing other specific tasks. The data 223 may include data received by the electronic device from external devices, as well as data collected by its own input / output interface 25.

[0071] Furthermore, this application also discloses a computer-readable storage medium for storing a computer program; wherein, when the computer program is executed by a processor, it implements the aforementioned information processing method for fundus images. Specific steps of this method can be found in the corresponding content disclosed in the foregoing embodiments, and will not be repeated here.

[0072] The various embodiments in this specification are described in a progressive manner, with each embodiment focusing on its differences from other embodiments. Similar or identical parts between embodiments can be referred to interchangeably. For the apparatus disclosed in the embodiments, since it corresponds to the method disclosed in the embodiments, the description is relatively simple; relevant parts can be referred to in the method section.

[0073] Those skilled in the art will further recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, computer software, or a combination of both. To clearly illustrate the interchangeability of hardware and software, the components and steps of the various examples have been generally described in terms of functionality in the foregoing description. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application. The steps of the methods or algorithms described in conjunction with the embodiments disclosed herein can be implemented directly in hardware, software modules executed by a processor, or a combination of both. The software module may be located in random access memory (RAM), memory, read-only memory (ROM), electrically programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), registers, hard disks, removable disks, CD-ROMs (Compact Disc-Read Only Memory), or any other form of storage medium known in the art.

[0074] Finally, it should be noted that in this document, relational terms such as "first" and "second" are used only to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.

[0075] The solution provided by the present invention has been described in detail above. Specific examples have been used to illustrate the principles and implementation methods of the present invention. The description of the above embodiments is only for helping to understand the method and core ideas of the present invention. At the same time, for those skilled in the art, there will be changes in the specific implementation methods and application scope based on the ideas of the present invention. Therefore, the content of this specification should not be construed as a limitation of the present invention.

Claims

1. An information processing method suitable for a fundus image, characterized by, Applied to computer devices, including: Acquire fundus images and vital sign data of the target object; The visual feature sequence of the fundus image is extracted using a preset visual feature extraction network; Based on a preset feature word converter, the vital signs data are mapped to feature words respectively, and corresponding semantic feature sequences are generated through a self-attention mechanism. Based on a preset fusion classification network, a target visual feature sequence that meets a preset correlation condition is extracted from the visual feature sequence according to the semantic feature sequence, and the semantic feature sequence and the target visual feature sequence are fused to obtain a fused feature vector. The acute mountain sickness risk assessment results are generated and output through the preset fusion classification network and based on the fusion feature vector.

2. The information processing method suitable for fundus images according to claim 1, characterized by, The vital signs data include the target's place of origin, the target's current altitude, the target's current headache level, and the target's vomiting level.

3. The information processing method for fundus images according to claim 2, characterized in that, The step of mapping the vital sign data into feature words based on a preset feature word converter includes: Determine the data type of each data variable in the vital signs data; If the data variable is the current altitude information or the current headache level information, then the data variable is determined to be a continuous variable. Then, a preset linear projection layer is used to perform projection mapping processing on the continuous variable to obtain feature words. If the data variable is the place of origin information or the vomiting degree information, then the data variable is determined to be a discrete variable. Then, the discrete variable is binned and embedded according to the preset embedding table to obtain feature words.

4. The information processing method for fundus images according to claim 1, characterized in that, The preset visual feature extraction network includes a preset fundus image recognition model and a feature pyramid network; Accordingly, the step of extracting the visual feature sequence of the fundus image using a preset visual feature extraction network includes: Based on the preset fundus image recognition model, image features at each level of the fundus image are extracted; wherein, the image features include optic disc morphology information, retinal vessel course and diameter information, and retinal hemorrhage or exudation changes information; The feature pyramid network is used to perform cross-level fusion processing on the semantic and texture information in the image features to generate a visual feature sequence.

5. The information processing method for fundus images according to claim 1, characterized in that, The generation of the corresponding semantic feature sequence through the self-attention mechanism includes: Each set of feature words is input into the self-attention layer to calculate the association weights between each feature word, and the vector representation of each feature word is updated according to the association weights to generate a semantic feature sequence containing contextual semantic information between vital signs data.

6. The information processing method for fundus images according to claim 1, characterized in that, The process of extracting a target visual feature sequence that meets a preset relevance condition from the visual feature sequence based on a preset fusion classification network and according to the semantic feature sequence, and then fusing the semantic feature sequence with the target visual feature sequence to obtain a fused feature vector, includes: Based on a preset fusion classification network, the semantic feature sequence and the visual feature sequence are used to perform dot product similarity calculation to obtain a similarity score; The similarity scores are normalized using the softmax function of the preset fusion classification network to obtain an attention weight matrix; wherein the attention weight matrix represents the correlation condition between each feature in the visual feature sequence and the semantic feature sequence. The visual feature sequence is weighted and summed using the preset fusion classification network and the attention weight matrix to extract the target visual feature sequence. The semantic feature sequence and the target visual feature sequence are then fused to obtain a fused feature vector.

7. The information processing method for fundus images according to claim 1, characterized in that, Also includes: Obtain the training sample set; each training sample includes historical fundus images, historical vital sign data, and corresponding acute mountain sickness risk level labels; Construct a multimodal neural network to be trained; wherein the multimodal neural network to be trained includes: an initial visual feature extraction network, an initial feature word converter, and an initial fusion classification network; Construct a joint loss function that includes a main loss function for optimizing risk classification and an auxiliary contrastive loss function for narrowing the distance between high-risk image-high-risk feature training sample pairs in the feature space; Freeze all network parameters of the initial visual feature extraction network, and use the training sample set and the joint loss function to train and optimize the network parameters of the initial feature word converter and the initial fusion classification network; Unfreeze the target network parameters in the initial visual feature extraction network, and use the training sample set and the joint loss function to jointly fine-tune all the training parameters of the multimodal neural network to be trained, so as to obtain a preset visual feature extraction network, a preset feature word converter, and a preset fusion classification network.

8. An information processing device suitable for fundus images, characterized in that, include: The data acquisition module is used to acquire fundus images and vital sign data of the target object; The sequence extraction module is used to extract the visual feature sequence of the fundus image using a preset visual feature extraction network; The sequence generation module is used to map the vital sign data into feature words based on a preset feature word converter, and generate corresponding semantic feature sequences through a self-attention mechanism. The fusion module is used to extract a target visual feature sequence that meets a preset correlation condition from the visual feature sequence based on a preset fusion classification network and according to the semantic feature sequence, and to fuse the semantic feature sequence and the target visual feature sequence to obtain a fusion feature vector. The classification module is used to generate and output the acute mountain sickness risk assessment result through the preset fusion classification network and based on the fusion feature vector.

9. An electronic device, characterized in that, include: Memory, used to store computer programs; A processor for executing the computer program to implement the steps of the information processing method for fundus images as described in any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that, Used to store a computer program; wherein, when executed by a processor, the computer program implements the steps of the information processing method for fundus images as described in any one of claims 1 to 7.

Images