Intelligent pre-examination triage method based on multi-modal data and edge calculation

Abstract

The invention provides an intelligent pre-examination triage method based on multi-modal data and edge calculation, and relates to the technical field of multi-modal data, and the method comprises the steps: collecting the facial expression, wound part, sound and physiological signals of a patient through medical equipment; multi-modal features are extracted by using a deep learning network and dynamic weighted fusion is carried out to obtain a patient state evaluation result; performing data time sequence alignment and feature extraction based on edge nodes, dynamically adjusting calculation precision and realizing task migration; multi-source data are collected for time sequence alignment calibration and feature fusion, the emergency degree is calculated, and group damage distribution is predicted. According to the method, real-time processing and dynamic optimization of multi-modal data are realized, and the accuracy and efficiency of pre-examination triage are improved.

Description

Technical Field

[0001] The present invention relates to multimodal data technology, and in particular to an intelligent pre-examination and triage method based on multimodal data and edge computing. Background Art

[0002] With the rapid development of medical informatization, pre-examination and triage, as an important part of the hospital emergency department, has a direct impact on the rational allocation of medical resources and the efficiency of patient treatment. At present, pre-examination and triage mainly rely on medical staff to observe the patient's symptoms and ask about the medical history to determine the severity of the disease and perform graded treatment. At the same time, with the development of the Internet of Things and artificial intelligence technology, intelligent pre-examination and triage systems based on multimodal data have gradually been applied. By collecting multi-source data such as the patient's physiological signs, images, and voice, combined with edge computing, rapid analysis and decision-making can be achieved.

[0003] The existing pre-examination and triage technologies mainly have the following problems: Existing technologies mainly rely on a single data source for analysis and lack the ability to comprehensively analyze multimodal data such as patients' facial expressions, trauma images, voice, and physiological signals. This makes it difficult to comprehensively and accurately assess the patient's condition and can easily lead to deviations in triage results.

[0004] Most of the current pre-examination and triage systems use a centralized cloud computing architecture, and all data needs to be uploaded to the cloud for processing. This has problems such as large network bandwidth usage and high processing delays, making it difficult to meet the real-time requirements in emergency scenarios and affecting treatment efficiency.

[0005] Existing technologies lack the ability to conduct overall analysis and resource scheduling in group injury scenarios, and are unable to optimize the allocation of medical resources based on the urgency and injury distribution characteristics of multiple patients, which can easily lead to waste of resources or insufficient supply. Summary of the invention

[0006] The embodiments of the present invention provide an intelligent pre-examination and triage method based on multimodal data and edge computing, which can solve the problems in the prior art.

[0007] According to a first aspect of the embodiments of the present invention, Provides intelligent pre-diagnosis and triage methods based on multimodal data and edge computing, including: Medical-grade equipment is used to collect the patient's facial expressions, injury sites, sounds, and physiological signals. The deep learning network is used to extract the image features of facial expressions and injury sites to calculate the intensity of expression and the degree of injury. The neural network is used to analyze the sound features to obtain the pain score and breathing score. The physiological signals are combined to calculate the cardiac function and blood pressure abnormality. The expression intensity, injury degree, pain score, breathing score, cardiac function, and blood pressure abnormality are dynamically weighted and fused to obtain the patient's status assessment result. Acquire multimodal data from medical devices and store them by type to edge nodes for time alignment, build vector allocation tasks based on the resource status of edge nodes, perform data preprocessing and feature extraction, dynamically adjust calculation accuracy, transmit compressed features to the cloud for analysis, and migrate tasks to nodes below the preset load threshold when processing delay exceeds the preset threshold to ensure real-time data processing; Multi-source sensors are used to collect the patient's physiological, visual, and audio data and perform time alignment calibration. Multimodal features are extracted from the calibrated data and adaptively fused. The patient's urgency is calculated based on the multimodal features and the distribution of group injuries is predicted. A resource scheduling plan is generated based on the distribution of group injuries. Parameters are dynamically optimized and adjusted through real-time evaluation of system performance.

[0008] Medical-grade equipment is used to collect the patient's facial expressions, injury sites, sounds, and physiological signals. The deep learning network is used to extract the image features of facial expressions and injury sites to calculate the intensity of expression and the degree of injury. The neural network is used to analyze the sound features to obtain the pain score and breathing score. The cardiac function and blood pressure abnormality are calculated in combination with physiological signals. The expression intensity, degree of injury, pain score, breathing score, cardiac function, and blood pressure abnormality are dynamically weighted and fused to obtain the patient status assessment results, including: Acquire multimodal physiological data of patients through medical-grade acquisition equipment, including acquiring facial expression images and trauma site images through medical-grade camera systems, acquiring sound signals through medical-grade directional microphone arrays, and acquiring electrocardiogram signals, blood pressure signals, body temperature signals, and blood oxygen saturation signals through physiological signal acquisition equipment; The preprocessed images of the wound site and facial expression were preprocessed to obtain preprocessed images. The preprocessed images were input into the U-Net network for regional segmentation and the wound severity score was calculated through the multi-task learning network. The preprocessed facial expression images were input into the VGG network to extract facial features and calculate the expression intensity score. The preprocessed wound site images were input into the multi-scale feature pyramid network to extract wound surface features and calculate the injury severity. The sound signal is subjected to noise suppression and enhancement processing to obtain an enhanced signal, the sound features are extracted from the enhanced signal and the pain degree score is calculated through the long short-term memory network, the enhanced signal is subjected to frequency band decomposition to obtain the breath sound features and the degree of dyspnea is calculated; the acquired electrocardiogram signal is subjected to wave group analysis to extract features and calculate the cardiac function score, the acquired blood pressure signal is subjected to trend analysis to obtain the degree of blood pressure abnormality, and the acquired body temperature signal and blood oxygen saturation signal are subjected to normalization processing; A multimodal feature vector is constructed by the trauma severity score, expression intensity score, injury severity, pain degree score, dyspnea degree, cardiac function score and blood pressure abnormality degree. The multimodal feature vector is input into the feature weight generation network, and the feature weights are dynamically adjusted based on historical data to obtain weighted features. The weighted features are combined with historical status information to generate the patient's current status assessment results.

[0009] The preprocessed images of the wound site and facial expression images are preprocessed to obtain preprocessed images. The preprocessed images are input into the U-Net network for regional segmentation and the wound severity score is calculated through the multi-task learning network. The preprocessed facial expression images are input into the VGG network to extract facial features and calculate the expression intensity score. The preprocessed wound site images are input into the multi-scale feature pyramid network to extract wound surface features. The calculation of the injury severity includes: Performing image preprocessing on the image data of the wound site to obtain a preprocessed wound image, wherein the image preprocessing includes image denoising, contrast enhancement and geometric correction; inputting the preprocessed wound image into a wound region segmentation network to perform wound region segmentation to obtain a wound region segmentation result; The preprocessed trauma image and trauma area segmentation results are input into a multi-task learning network. The multi-task learning network includes a trauma classification branch, a trauma depth estimation branch, and a trauma edge detection branch. The trauma classification branch uses densely connected convolution blocks to extract multi-scale features for trauma type classification. The trauma depth estimation branch predicts trauma depth information based on a deep regression network. The trauma edge detection branch uses an edge detection operator to extract trauma edge contour features. The trauma type classification, trauma depth information and trauma edge contour features are input into the feature fusion module. The feature fusion module uses an adaptive weight method to fuse multi-task features to obtain fusion features; the trauma severity score is calculated based on the fusion features.

[0010] Acquire multimodal data from medical devices and store them by type to edge nodes for time alignment, build vector allocation tasks based on the resource status of edge nodes, perform data preprocessing and feature extraction, dynamically adjust calculation accuracy, transmit compressed features to the cloud for analysis, and migrate tasks to nodes below the preset load threshold when processing delay exceeds the preset threshold to ensure real-time data processing, including: The multimodal data are stored in the corresponding edge node buffer area according to the data type, and the multimodal data are time-series aligned based on the acquisition timestamp to generate aligned multimodal data; Construct a two-layer architecture of macro task allocation layer and micro resource scheduling layer to realize bidirectional information transmission between layers, perform multi-dimensional priority evaluation on tasks to obtain a comprehensive reward function, input the comprehensive reward function into the dual Q network structure and combine it with attention mechanism and priority experience playback for training, and finally output the task scheduling strategy; Monitor the resources of the edge node cache area, obtain the computing load, memory occupancy rate and network bandwidth status of the edge node cache area, and construct a resource state vector based on the computing load, memory occupancy rate and network bandwidth status. The resource state vector is used to characterize the real-time resource status of the edge node; input the resource state vector and the aligned multimodal data into the task scheduling model, which is obtained based on deep reinforcement learning training. The task scheduling model outputs a task allocation strategy based on the processing requirements of the resource state vector and the task scheduling strategy, and allocates the data processing task to the corresponding edge node according to the task allocation strategy; Perform data preprocessing and feature extraction on the edge node, perform noise reduction and standardization on the multimodal data to obtain preprocessed data, group the preprocessed data according to a preset batch size to obtain data batches, and input the data batches into a lightweight feature extraction network to extract features to obtain feature data; dynamically adjust the calculation accuracy of the lightweight feature extraction network based on the resource state vector of the edge node, reduce the network quantization bit width and calculation accuracy when the resource state vector indicates that the resources are tight, and improve the calculation accuracy when the resource state vector indicates that the resources are sufficient, and compress and encode the feature data to obtain compressed features; The compressed features are transmitted to the cloud server through an encrypted channel, the compressed features are decoded on the cloud server to obtain decoded features, the decoded features are analyzed and processed to generate analysis results; the data processing delay of the edge node is monitored in real time. When the data processing delay exceeds the preset delay threshold, the target edge node with a load lower than the preset load threshold is selected based on the resource state vector, and the data processing task to be processed is migrated to the target edge node to ensure the real-time performance of data processing.

[0011] A two-layer architecture of macro task allocation layer and micro resource scheduling layer is constructed to realize bidirectional information transmission between layers. Multi-dimensional priority evaluation of tasks is performed to obtain a comprehensive reward function. The comprehensive reward function is input into the dual Q network structure and trained in combination with attention mechanism and priority experience playback. The final output task scheduling strategy includes: A two-layer reinforcement learning architecture including a macro task allocation layer and a micro resource scheduling layer is constructed. The state space of the macro task allocation layer includes the resource state matrix, the set of pending tasks and the task queue state. The action space of the macro task allocation layer includes the task grouping strategy and the node selection strategy. The state space of the micro resource scheduling layer includes the state space of the macro task allocation layer, the local resource utilization rate and the task dependency graph. The action space of the micro resource scheduling layer includes the resource allocation ratio, the task priority adjustment and the scheduling sequence arrangement. The tasks assigned by the macro task allocation layer are specifically scheduled through the micro resource scheduling layer. The decision results of the macro task allocation layer are converted into the constraints of the micro resource scheduling layer through the strategy mapping function. The state information of the micro resource scheduling layer is compressed and transmitted to the macro task allocation layer through the state aggregation function, so as to realize the two-way transmission of information between layers. Perform multi-dimensional priority evaluation on the tasks received by the micro-resource scheduling layer, calculate the task urgency based on the task deadline, calculate the resource demand based on the demand for various resources, multiply the task urgency by the task execution efficiency reward to obtain the first product, multiply the resource demand by the task resource utilization reward to obtain the second product, and add the first product and the second product to obtain the comprehensive reward function of the task; A dual Q network structure consisting of a main network and a target network is constructed. The main network and the target network use the same network structure. The comprehensive reward function is input into the main network. The loss function is constructed based on the temporal difference error to train the main network, and the network parameters of the main network are periodically copied to the target network. The state features in the state space of the micro-resource scheduling layer are input into the attention mechanism module, and the importance weights of the state features are calculated through the attention mechanism module, and the state features are weighted according to the importance weights. The training samples are stored in the experience replay pool according to the priority determined by the temporal difference error, and the main network is sampled from the experience replay pool according to the sample priority to train the main network, and the scheduling strategy of the task is output based on the trained main network.

[0012] The patient's physiological, visual and audio data are collected through multi-source sensors and time-series alignment and calibration is performed. Multimodal features are extracted from the calibrated data and adaptively fused. The patient's urgency is calculated based on the multimodal features and the distribution of group injuries is predicted. A resource scheduling plan is generated based on the distribution of group injuries. The parameters are dynamically optimized and adjusted by real-time evaluation of system performance, including: The patient's physiological data, visual data and audio data are collected in real time through multi-source sensors, an alignment weight coefficient is generated according to the timestamp, and a temporal alignment loss function is calculated based on the alignment weight coefficient to calibrate the physiological data, visual data and audio data to obtain calibrated multimodal data; Feature extraction is performed on the calibrated multimodal data. Multi-scale spectrum analysis of physiological data is performed through wavelet transform to obtain time domain and frequency domain features. Facial micro-expression features are extracted from visual data through VGG network. Acoustic features are extracted from audio data through time-frequency attention mechanism to obtain multimodal feature data. The multimodal feature data is input into the cross-modal attention fusion mechanism for adaptive weight fusion to obtain fusion features, and the fusion features are input into the multi-level feature extraction network to extract physiological features, behavioral features and semantic features respectively. Different evaluation indicators are comprehensively scored based on the dynamic weight coefficient to obtain the urgency score; Patients are classified according to their urgency scores. In group injury scenarios, multi-source information fusion is performed based on multimodal feature data to estimate the number of injured and predict the distribution of injuries. A resource scheduling optimization objective function is constructed based on the number of injured and the distribution of injuries, and the optimal resource scheduling plan is generated by optimizing the resource scheduling optimization objective function. A multi-dimensional performance evaluation indicator system is constructed to monitor the execution status of the optimal resource scheduling solution in real time. The performance improvement metrics are input into the deep reinforcement learning model to obtain the optimization strategy. The optimization knowledge base is updated based on the execution effect and the strategy network parameters are dynamically adjusted. Combined with the fault recovery mechanism, dynamic balance optimization of system performance and reliability is achieved.

[0013] Construct a multi-dimensional performance evaluation index system to monitor the execution status of the optimal resource scheduling solution in real time, input the performance improvement metrics into the deep reinforcement learning model to obtain the optimization strategy, update the optimization knowledge base based on the execution effect and dynamically adjust the policy network parameters, and combine the fault recovery mechanism to achieve dynamic balance optimization of system performance and reliability, including: Build a multi-dimensional performance evaluation indicator system to collect data processing delay, computing load, network bandwidth occupancy, and memory usage of each edge node. Calculate end-to-end processing delay, system throughput, and resource utilization efficiency based on data processing delay, computing load, network bandwidth occupancy, and memory utilization, and construct end-to-end processing delay, system throughput, and resource utilization efficiency as performance evaluation vectors. Monitor the performance evaluation vector in real time and calculate the performance improvement metrics before and after optimization. The performance improvement metrics include latency improvement rate, throughput improvement rate, and resource saving rate. Input the performance improvement metrics into the deep reinforcement learning model for training to obtain the performance optimization strategy. Based on the performance optimization strategy, the task allocation and resource scheduling of edge nodes are executed, and the state feature vector, optimization decision vector and performance improvement vector in the execution process are recorded and stored in the optimization knowledge base, where the state feature vector includes resource utilization and load level, the optimization decision vector includes resource allocation and task migration decision, and the performance improvement vector includes the improvement degree of each performance indicator; Calculate the cosine similarity between the current state feature vector and the historical state feature vector in the optimization knowledge base, build a comprehensive performance reward function based on the cosine similarity and the current execution effect, use the comprehensive performance reward function to calculate the policy gradient and update the policy network parameters of the deep reinforcement learning model to generate a new performance optimization strategy; Real-time monitoring of the execution effect of performance optimization strategies. When the fluctuation coefficient of the performance indicator exceeds the preset fluctuation threshold or the system stability time exceeds the preset time threshold, the optimization decision vector with the highest historical similarity is selected from the optimization knowledge base as an alternative strategy and switched to ensure the stability of system performance. Monitor the fault status of edge nodes, record the fault recovery time and calculate the average fault recovery time. Combine the system average failure time to calculate the system availability index. Use the system availability index as a constraint for selecting performance optimization strategies to achieve dynamic balance optimization of system performance and reliability.

[0014] According to a second aspect of the embodiments of the present invention, An electronic device is provided, comprising: processor; a memory for storing processor-executable instructions; The processor is configured to call instructions stored in the memory to execute the aforementioned method.

[0015] According to a third aspect of the embodiments of the present invention, A computer-readable storage medium is provided, on which computer program instructions are stored. When the computer program instructions are executed by a processor, the aforementioned method is implemented.

[0016] The beneficial effects of this application are as follows: The present invention collects multimodal data through medical-grade equipment and combines it with a deep learning network for feature extraction and fusion analysis. It can comprehensively and accurately assess the patient's condition, improve the objectivity and accuracy of pre-examination and triage, and effectively avoid the subjectivity and uncertainty of manual evaluation.

[0017] The present invention adopts edge computing architecture to perform distributed processing and dynamic task scheduling of multimodal data, ensures the real-time performance of data processing through resource status perception and task migration mechanism, optimizes network bandwidth occupancy, and improves system reliability and scalability.

[0018] Based on the time series alignment and feature fusion of multi-source sensor data, the present invention realizes the precise assessment of the patient's urgency and the accurate prediction of the distribution of group injuries. By dynamically optimizing parameters, the system performance is improved, a reliable basis is provided for the rational scheduling of medical resources, and the treatment efficiency is effectively improved. BRIEF DESCRIPTION OF THE DRAWINGS

[0019] Figure 1It is a flowchart of an intelligent pre-examination and triage method based on multimodal data and edge computing according to an embodiment of the present invention; Figure 2 This is a schematic diagram of multi-dimensional evaluation performance indicators of an embodiment of the present invention; Figure 3 This is a schematic diagram of a real-time monitoring panel of a double-layer task scheduling system according to an embodiment of the present invention; Figure 4 This is a heat map of resource load distribution in an embodiment of the present invention; Figure 5 It is a schematic diagram of the processing efficiency under dynamic load changes of an embodiment of the present invention. DETAILED DESCRIPTION

[0020] In order to make the purpose, technical solution and advantages of the embodiments of the present invention clearer, the technical solution in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0021] The technical solution of the present invention is described in detail with specific embodiments below. The following specific embodiments can be combined with each other, and the same or similar concepts or processes may not be described in detail in some embodiments.

[0022] Figure 1 Schematic diagram of a flow chart of an intelligent pre-diagnosis and triage method based on multimodal data and edge computing according to an embodiment of the present invention. Figure 1 As shown, the method includes: Medical-grade equipment is used to collect the patient's facial expressions, injury sites, sounds, and physiological signals. The deep learning network is used to extract the image features of facial expressions and injury sites to calculate the intensity of expression and the degree of injury. The neural network is used to analyze the sound features to obtain the pain score and breathing score. The physiological signals are combined to calculate the cardiac function and blood pressure abnormality. The expression intensity, injury degree, pain score, breathing score, cardiac function, and blood pressure abnormality are dynamically weighted and fused to obtain the patient's status assessment result. Acquire multimodal data from medical devices and store them by type to edge nodes for time alignment, build vector allocation tasks based on the resource status of edge nodes, perform data preprocessing and feature extraction, dynamically adjust calculation accuracy, transmit compressed features to the cloud for analysis, and migrate tasks to nodes below the preset load threshold when processing delay exceeds the preset threshold to ensure real-time data processing; Multi-source sensors are used to collect the patient's physiological, visual, and audio data and perform time alignment calibration. Multimodal features are extracted from the calibrated data and adaptively fused. The patient's urgency is calculated based on the multimodal features and the distribution of group injuries is predicted. A resource scheduling plan is generated based on the distribution of group injuries. Parameters are dynamically optimized and adjusted through real-time evaluation of system performance.

[0023] In an optional implementation, facial expressions, wound sites, sounds and physiological signals of patients are collected by medical-grade equipment, and image features of facial expressions and wound sites are extracted by deep learning networks to calculate expression intensity and injury severity, and pain scores and respiration scores are obtained by analyzing sound features by neural networks. Cardiac function and blood pressure abnormality are calculated in combination with physiological signals, and the expression intensity, injury severity, pain score, respiration score, cardiac function and blood pressure abnormality are dynamically weighted and fused to obtain patient status assessment results including: Acquire multimodal physiological data of patients through medical-grade acquisition equipment, including acquiring facial expression images and trauma site images through medical-grade camera systems, acquiring sound signals through medical-grade directional microphone arrays, and acquiring electrocardiogram signals, blood pressure signals, body temperature signals, and blood oxygen saturation signals through physiological signal acquisition equipment; The preprocessed images of the wound site and facial expression were preprocessed to obtain preprocessed images. The preprocessed images were input into the U-Net network for regional segmentation and the wound severity score was calculated through the multi-task learning network. The preprocessed facial expression images were input into the VGG network to extract facial features and calculate the expression intensity score. The preprocessed wound site images were input into the multi-scale feature pyramid network to extract wound surface features and calculate the injury severity. The sound signal is subjected to noise suppression and enhancement processing to obtain an enhanced signal, the sound features are extracted from the enhanced signal and the pain degree score is calculated through the long short-term memory network, the enhanced signal is subjected to frequency band decomposition to obtain the breath sound features and the degree of dyspnea is calculated; the acquired electrocardiogram signal is subjected to wave group analysis to extract features and calculate the cardiac function score, the acquired blood pressure signal is subjected to trend analysis to obtain the degree of blood pressure abnormality, and the acquired body temperature signal and blood oxygen saturation signal are subjected to normalization processing; A multimodal feature vector is constructed by the trauma severity score, expression intensity score, injury severity, pain degree score, dyspnea degree, cardiac function score and blood pressure abnormality degree. The multimodal feature vector is input into the feature weight generation network, and the feature weights are dynamically adjusted based on historical data to obtain weighted features. The weighted features are combined with historical status information to generate the patient's current status assessment results.

[0024] First, the multimodal physiological data of the patient is collected through medical-grade equipment. The medical-grade camera system uses a binocular camera with a resolution of 4K and a frame rate of 60fps to collect images of the patient's facial expressions and trauma sites. The medical-grade directional microphone array consists of 8 microphones with a sampling rate of 48kHz to collect patient voice signals. The physiological signal acquisition equipment includes an electrocardiogram acquisition module, a blood pressure acquisition module, a body temperature acquisition module, and a blood oxygen saturation acquisition module, with sampling rates of 1kHz, 200Hz, 10Hz, and 100Hz, respectively.

[0025] The collected images were preprocessed, including image denoising, illumination compensation and size normalization. The preprocessed images of the wound site were input into the U-Net network, which contained 4 layers of encoders and 4 layers of decoders. Each layer used a 3×3 convolution kernel to perform region segmentation to obtain the wound area mask. The mask was combined with the original image and input into the multi-task learning network to extract the wound features and calculate the severity score, with a score range of 0-10. The preprocessed facial expression images were input into the 16-layer VGG network to extract the features of 68 key points on the face and calculate the expression intensity score, with a score range of 0-5. The wound site images were input into the 5-layer feature pyramid network to extract multi-scale features to calculate the injury severity, with a score range of 0-10.

[0026] The sound signal was denoised by wavelet and enhanced by spectral subtraction, and the signal-to-noise ratio was improved by 10dB. 39-dimensional MFCC features were extracted from the enhanced signal and input into the bidirectional long short-term memory network to calculate the pain score, with a score range of 0-10. The enhanced signal was bandpass filtered to obtain breathing sounds, and the frequency domain features were extracted to calculate the degree of dyspnea, with a score range of 0-5. QRS wave detection was performed on the ECG signal, and features such as RR interval were extracted to calculate the cardiac function score, with a score range of 0-10. Trend analysis was performed on the blood pressure signal to obtain the blood pressure abnormality, with a score range of 0-5. The body temperature and blood oxygen saturation signals were normalized to the maximum and minimum values.

[0027] Each score is constructed into a 28-dimensional feature vector and input into a 3-layer fully connected network to dynamically generate feature weights. The weight generation network is trained based on 3,000 historical data. The weighted features are combined with the state information of the previous 5 frames, and the patient's current state assessment results are generated through a gated recurrent unit network, including three levels: critical, severe, and mild.

[0028] The accuracy and reliability of patient status assessment are improved through the fusion analysis of multimodal physiological signals, and the assessment accuracy rate reaches more than 95%. The deep learning method is used to automatically extract multimodal features, which reduces the subjectivity of manual feature extraction and makes the assessment results more objective. The dynamic weight fusion strategy can adaptively adjust the importance of different features and improve the adaptability of the assessment system to the individual differences of different patients.

[0029] In an optional embodiment, the wound site image and the facial expression image are preprocessed to obtain a preprocessed image, the preprocessed image is input into a U-Net network for region segmentation and a wound severity score is calculated through a multi-task learning network, the preprocessed facial expression image is input into a VGG network to extract facial features and calculate the expression intensity score, and the preprocessed wound site image is input into a multi-scale feature pyramid network to extract wound surface features and calculate the injury severity, including: Performing image preprocessing on the image data of the wound site to obtain a preprocessed wound image, wherein the image preprocessing includes image denoising, contrast enhancement and geometric correction; inputting the preprocessed wound image into a wound region segmentation network to perform wound region segmentation to obtain a wound region segmentation result; The preprocessed trauma image and trauma area segmentation results are input into a multi-task learning network. The multi-task learning network includes a trauma classification branch, a trauma depth estimation branch, and a trauma edge detection branch. The trauma classification branch uses densely connected convolution blocks to extract multi-scale features for trauma type classification. The trauma depth estimation branch predicts trauma depth information based on a deep regression network. The trauma edge detection branch uses an edge detection operator to extract trauma edge contour features. The trauma type classification, trauma depth information and trauma edge contour features are input into the feature fusion module. The feature fusion module uses an adaptive weight method to fuse multi-task features to obtain fusion features; the trauma severity score is calculated based on the fusion features.

[0030] When preprocessing the wound site image, bilateral filtering was first used for image denoising, with the filter window size set to 5×5 pixels, the spatial domain standard deviation set to 3, and the grayscale domain standard deviation set to 25. Then the image contrast was enhanced by the histogram equalization method, and the image grayscale value was remapped to the range of 0-255. Finally, geometric correction was performed based on SIFT feature point matching, the feature points in the image were extracted and the corresponding relationship was established, and the image geometric deformation was corrected using the affine transformation matrix.

[0031] The preprocessed trauma image is input into the U-Net network for regional segmentation. The U-Net network consists of two parts: an encoder and a decoder. The encoder consists of four downsampling blocks, each of which contains two 3×3 convolutional layers and a maximum pooling layer. The decoder consists of four upsampling blocks, each of which contains a deconvolution layer and two 3×3 convolutional layers. The jump connection concatenates the feature map of the corresponding layer of the encoder with the feature map of the decoder to enhance the detail information. The network outputs a binary segmentation map of the trauma area.

[0032] In the multi-task learning network, the wound classification branch adopts the DenseNet structure, which contains 4 dense blocks, each dense block contains 6 dense connection layers, and the growth rate is set to 32. The wound depth estimation branch adopts the ResNet structure, which contains 5 residual blocks for feature extraction and outputs the wound depth value. The wound edge detection branch first uses the Sobel operator to extract the horizontal and vertical gradients, and then obtains the edge contour through non-maximum suppression.

[0033] The feature fusion module uses the attention mechanism to fuse the three branch features. The spatial attention weight and channel attention weight are first calculated for each branch feature, and then the weighted features are element-wise added to obtain the fused feature. Finally, the fused feature is mapped to a trauma severity score of 0-100 through a fully connected layer.

[0034] For facial expression images, the VGG16 network is used to extract features. First, the image is scaled to 224×224 size, and hierarchical features are extracted through 5 convolution blocks. Each convolution block contains 2-3 3×3 convolution layers and a maximum pooling layer. Finally, the expression intensity score is obtained through 3 fully connected layers.

[0035] The wound surface feature extraction uses FPN network, which includes two feature extraction paths: bottom-up and top-down. The bottom-up path uses ResNet50 as the backbone network to extract multi-scale features, and the top-down path fuses different scale features through upsampling and lateral connection, and finally outputs the wound severity score.

[0036] Figure 2 This is a schematic diagram of multi-dimensional evaluation performance indicators of an embodiment of the present invention: This technical solution adopts an innovative three-stage hybrid architecture: first, the U-Net network is used to perform accurate trauma area segmentation, then multi-scale features are extracted through the improved VGG network, and finally a multi-task learning framework is used to complete the comprehensive evaluation, achieving comprehensive leadership in various indicators; the traditional CNN method uses a single deep network architecture and lacks a dedicated regional segmentation and feature fusion mechanism; the rule-based method relies entirely on preset medical rules and basic image processing algorithms. Performance tests show that the various indicators of this technical solution are as follows: accuracy 95.8%, recall 94.2%, F1-score 95.0%, processing speed 90.5%, ROC-AUC 96.8%; the various indicators of the traditional CNN method are: accuracy 88.3%, recall 86.7%, F1-score 87.5%, processing speed 82.3%, ROC-AUC 89.2%; the rule-based method performs the worst: accuracy 82.1%, recall 80.5%, F1-score 81.3%, processing speed 75.8%, ROC-AUC 84.5%. It can be seen from the experimental data that this technical solution not only has significant advantages in accuracy and processing efficiency, but also achieves comprehensive performance improvement in various evaluation indicators. In particular, the ROC-AUC indicator reached a high level of 96.8%, which is 7.6 and 12.3 percentage points higher than the other two methods, respectively, which fully proves the comprehensive advantages of this method in trauma assessment tasks. Experimental results show that this technical solution successfully solves the limitations of traditional methods in terms of accuracy, efficiency and generalization ability through innovative algorithm architecture design and multi-task learning strategy, providing more reliable technical support for clinical trauma assessment.

[0037] In response to the problems existing in traditional trauma assessment methods, such as rule-based methods that rely on manually preset rules and lack adaptability, and traditional CNN methods that use a single network architecture that is difficult to meet complex assessment needs, this application proposes a trauma assessment solution based on a multi-stage hybrid architecture. Compared with the existing technology, the innovation of this solution is: first, by introducing the U-Net network to replace the traditional segmentation method based on threshold or region growing, the accuracy of trauma area identification is improved; second, the improved VGG network is used for multi-scale feature extraction, which better captures the multi-dimensional characteristics of trauma compared to the single feature expression ability of traditional CNN; finally, a multi-task learning framework is innovatively constructed to transform tasks such as trauma classification, depth estimation, and edge detection that originally required serial processing into a parallel processing mechanism.

[0038] The technical effects of this solution are mainly reflected in the following aspects: First, through the precise segmentation capability of the U-Net network, the accuracy of trauma area recognition is significantly improved, solving the problem that traditional methods are easily affected by image quality and lighting conditions; second, with the help of the improved VGG network multi-scale feature extraction capability, the feature expression of different types and degrees of trauma is enhanced, and the robustness of the evaluation is improved; third, the parallel processing mechanism of the multi-task learning framework greatly improves the evaluation efficiency, while the feature sharing and complementary effects between tasks improve the overall evaluation performance; finally, through the comprehensive improvement of multi-dimensional evaluation indicators, the comprehensive advantages of this solution in terms of accuracy, efficiency and generalization ability are verified, providing more reliable technical support for clinical trauma evaluation. The experimental results show that compared with the existing technical solutions, the evaluation method proposed in this application has achieved significant improvements in various indicators such as evaluation accuracy, processing efficiency and generalization ability, which fully proves the technical innovation value and practical application potential of this method.

[0039] In an optional implementation, the multimodal data of the medical device is obtained and stored in the edge node by type for time alignment, a vector allocation task is constructed based on the resource status of the edge node, data preprocessing and feature extraction are performed, the calculation accuracy is dynamically adjusted, and the compressed features are transmitted to the cloud for analysis. When the processing delay exceeds a preset threshold, the task is migrated to a node below a preset load threshold to ensure the real-time performance of data processing, including: The multimodal data are stored in the corresponding edge node buffer area according to the data type, and the multimodal data are time-series aligned based on the acquisition timestamp to generate aligned multimodal data; Construct a two-layer architecture of macro task allocation layer and micro resource scheduling layer to realize bidirectional information transmission between layers, perform multi-dimensional priority evaluation on tasks to obtain a comprehensive reward function, input the comprehensive reward function into the dual Q network structure and combine it with attention mechanism and priority experience playback for training, and finally output the task scheduling strategy; Monitor the resources of the edge node cache area, obtain the computing load, memory occupancy rate and network bandwidth status of the edge node cache area, and construct a resource state vector based on the computing load, memory occupancy rate and network bandwidth status. The resource state vector is used to characterize the real-time resource status of the edge node; input the resource state vector and the aligned multimodal data into the task scheduling model, which is obtained based on deep reinforcement learning training. The task scheduling model outputs a task allocation strategy based on the processing requirements of the resource state vector and the task scheduling strategy, and allocates the data processing task to the corresponding edge node according to the task allocation strategy; Perform data preprocessing and feature extraction on the edge node, perform noise reduction and standardization on the multimodal data to obtain preprocessed data, group the preprocessed data according to a preset batch size to obtain data batches, and input the data batches into a lightweight feature extraction network to extract features to obtain feature data; dynamically adjust the calculation accuracy of the lightweight feature extraction network based on the resource state vector of the edge node, reduce the network quantization bit width and calculation accuracy when the resource state vector indicates that the resources are tight, and improve the calculation accuracy when the resource state vector indicates that the resources are sufficient, and compress and encode the feature data to obtain compressed features; The compressed features are transmitted to the cloud server through an encrypted channel, the compressed features are decoded on the cloud server to obtain decoded features, the decoded features are analyzed and processed to generate analysis results; the data processing delay of the edge node is monitored in real time. When the data processing delay exceeds the preset delay threshold, the target edge node with a load lower than the preset load threshold is selected based on the resource state vector, and the data processing task to be processed is migrated to the target edge node to ensure the real-time performance of data processing.

[0040] The edge computing processing method for multimodal medical data first receives multimodal data from medical equipment. Physiological signal data is collected at a sampling frequency of 256Hz, including signal type identification, acquisition timestamp and signal value such as ECG, EEG, and blood oxygen; medical image data includes image frame numbers, shooting timestamps and 1920×1080 resolution pixel matrices such as ultrasound and X-ray; device status data includes device number, operating status code, alarm information and recording timestamp.

[0041] Different types of data are stored in dedicated cache areas. Physiological signal data is stored in high-speed cache areas using a circular buffer structure for fast reading and writing, with a buffer size of 1GB; medical image data is stored in large-capacity cache areas using a block storage mechanism, with a single block size of 256MB; device status data is stored in status monitoring cache areas using key-value pair storage, with a cache capacity of 100MB. Data alignment is performed based on timestamp information, with a 1-second time window set, a window sliding step of 100ms, and an alignment accuracy of milliseconds to ensure the temporal consistency of data in different modalities.

[0042] A two-layer task scheduling architecture is constructed. The macro task allocation layer is responsible for global resource allocation and task priority assessment, and the micro resource scheduling layer performs specific task allocation and dynamic resource adjustment. Priority assessment considers three dimensions: task urgency, data importance, and resource demand: the task deadline and the current time difference less than 100ms is the highest priority, and the difference between 100-500ms is medium priority; data anomalies exceeding three standard deviations are defined as high importance; resource demand is evaluated based on the processing algorithm type and data scale, generating a priority score of 0-100.

[0043] The dual Q network structure is used to implement task scheduling strategy learning. The network input layer receives the task priority score, resource state vector and historical scheduling record. The feature extraction layer contains three layers of fully connected networks, with 256, 128 and 64 hidden nodes respectively. The attention mechanism layer calculates the feature weight according to the task priority, and high-priority tasks get larger weight values. The priority experience playback mechanism gives priority to the scheduling experience of high-scoring tasks for training to improve the scheduling effect of key tasks.

[0044] Monitor the resource status of edge nodes in real time, with CPU usage sampled every 100ms, memory usage sampled every 500ms, and network bandwidth sampled every 1s. Construct a 16-dimensional resource status vector, including computing load, memory usage, cache usage, and network status indicators. When the task processing delay exceeds 200ms, the task migration mechanism is triggered, and nodes with CPU usage below 50% and memory usage below 60% are selected as migration targets.

[0045] Data preprocessing is performed at the edge node. A 15-point median filter is used to remove baseline drift for physiological signals, a 3×3 Gaussian filter is used to eliminate noise for image data, and outlier detection is performed on device status data. Standardization processing maps various types of data to the 0-1 interval. Preprocessed data is grouped by batches, with a physiological signal batch size of 1024 points, an image batch of 64 frames, and a status data batch of 128 items.

[0046] The lightweight feature extraction network adopts a deep separable convolution structure and contains 5 feature extraction modules. When the CPU usage rate exceeds 80%, the network is quantized to 8-bit fixed-point representation, and the number of feature channels is reduced to 1 / 4 of the original; when resources are sufficient, 32-bit floating-point calculation is restored. The extracted features are reduced in dimension through principal component analysis and compressed using run-length encoding, with a compression ratio of 4:1. The compressed features are transmitted to the cloud server through an SSL encrypted channel for decoding and analysis.

[0047] Improved data management efficiency: Dedicated cache design and precise timing alignment mechanism enable efficient storage and collaborative processing of multimodal data, reduce data access latency, and improve storage utilization. Pipeline processing of preprocessing and feature extraction improves data processing throughput. Optimized task scheduling performance: The two-layer architecture combined with deep reinforcement learning enables intelligent and adaptive task scheduling, and the priority evaluation and experience replay mechanism ensure the timely processing of key tasks. The dynamic precision adjustment strategy optimizes computing resource utilization while ensuring processing quality. System reliability assurance: Real-time resource monitoring and task migration mechanisms improve system stability and effectively avoid single point failures. Feature compression and secure transmission reduce communication overhead, and multi-level caching and parallel processing improve system response speed and significantly improve the overall service quality.

[0048] In an optional implementation, a two-layer architecture of a macro task allocation layer and a micro resource scheduling layer is constructed to realize bidirectional information transmission between layers, and a multi-dimensional priority evaluation is performed on the tasks to obtain a comprehensive reward function. The comprehensive reward function is input into the dual Q network structure and trained in combination with the attention mechanism and priority experience playback. The final output task scheduling strategy includes: A two-layer reinforcement learning architecture including a macro task allocation layer and a micro resource scheduling layer is constructed. The state space of the macro task allocation layer includes the resource state matrix, the set of pending tasks and the task queue state. The action space of the macro task allocation layer includes the task grouping strategy and the node selection strategy. The state space of the micro resource scheduling layer includes the state space of the macro task allocation layer, the local resource utilization rate and the task dependency graph. The action space of the micro resource scheduling layer includes the resource allocation ratio, the task priority adjustment and the scheduling sequence arrangement. The tasks assigned by the macro task allocation layer are specifically scheduled through the micro resource scheduling layer. The decision results of the macro task allocation layer are converted into the constraints of the micro resource scheduling layer through the strategy mapping function. The state information of the micro resource scheduling layer is compressed and transmitted to the macro task allocation layer through the state aggregation function, so as to realize the two-way transmission of information between layers. Perform multi-dimensional priority evaluation on the tasks received by the micro-resource scheduling layer, calculate the task urgency based on the task deadline, calculate the resource demand based on the demand for various resources, multiply the task urgency by the task execution efficiency reward to obtain the first product, multiply the resource demand by the task resource utilization reward to obtain the second product, and add the first product and the second product to obtain the comprehensive reward function of the task; A dual Q network structure consisting of a main network and a target network is constructed. The main network and the target network use the same network structure. The comprehensive reward function is input into the main network. The loss function is constructed based on the temporal difference error to train the main network, and the network parameters of the main network are periodically copied to the target network. The state features in the state space of the micro-resource scheduling layer are input into the attention mechanism module, and the importance weights of the state features are calculated through the attention mechanism module, and the state features are weighted according to the importance weights. The training samples are stored in the experience replay pool according to the priority determined by the temporal difference error, and the main network is sampled from the experience replay pool according to the sample priority to train the main network, and the scheduling strategy of the task is output based on the trained main network.

[0049] First, a two-layer reinforcement learning architecture is constructed. The state space of the macro task allocation layer includes the resource state matrix, the set of pending tasks, and the task queue state. The resource state matrix records the CPU, memory, bandwidth and other resource usage of each node. For example, the CPU usage of a node is 80%, the memory usage is 60%, and the bandwidth usage is 40%. The set of pending tasks contains information such as task ID, resource demand, and deadline. For example, task 1 requires a 2-core CPU, 4GB of memory, and the deadline is 10 minutes later. The task queue state records the number of waiting tasks in each queue. The macro-level action space includes task grouping strategies and node selection strategies, such as grouping tasks with similar resource requirements into a group and selecting nodes with sufficient resources.

[0050] The state space of the micro-resource scheduling layer adds local resource utilization and task dependency graphs on the basis of the macro-layer. Local resource utilization records the fine-grained resource usage of the selected node, such as the load of each CPU core. The task dependency graph describes the forward and backward dependencies between tasks. The micro-layer action space includes specific resource allocation ratios, task priority adjustments, and scheduling timing arrangements.

[0051] The macro-level decision is converted into micro-level constraints through the policy mapping function. For example, after the macro-level selects a node, the micro-level can only allocate resources within the scope of the node. The micro-level state information is compressed and passed to the macro-level through the state aggregation function, for example, the fine-grained resource usage is aggregated into resource utilization.

[0052] Perform multi-dimensional priority assessment on tasks. Calculate urgency based on task deadlines. The closer to the deadline, the higher the urgency. Calculate resource requirements based on CPU, memory and other resource requirements. Multiply urgency by execution efficiency reward to get the first product, multiply resource requirement by resource utilization reward to get the second product, and add the two together to get the comprehensive reward function.

[0053] Construct a dual Q network structure. The main network and the target network use the same three-layer neural network structure. The input layer receives state features, the hidden layer contains 128 neurons, and the output layer corresponds to the action space dimension. The comprehensive reward function is input into the main network for training, and the main network parameters are copied to the target network every 1000 steps. The attention mechanism module calculates the importance weight of the state features and gives higher weight to key features such as CPU usage. The training samples are stored in the experience replay pool with a capacity of 10,000 according to priority, and 256 samples are randomly sampled each time to train the network.

[0054] Figure 3 This is a schematic diagram of a real-time monitoring panel of a double-layer task scheduling system according to an embodiment of the present invention: The figure shows that the system is currently running well and all core indicators are performing well. In terms of top-level monitoring, the system is in normal operation, with 324 pending tasks in the task queue, an overall resource utilization rate of 87.6%, and a scheduling efficiency of 96.4%, indicating that the system is running stably and efficiently. At the macro task allocation level, the resource status matrix shows that the CPU utilization rate is 80%, the memory utilization rate is 60%, and the bandwidth utilization rate is 40%, and the resource allocation is reasonable; the task grouping strategy includes 12 high-priority tasks, 45 medium-priority tasks, and 267 low-priority tasks, realizing hierarchical management of tasks. The micro resource scheduling layer shows that the loads of the four processor cores are 85%, 78%, 92%, and 65%, respectively. Although there is a certain degree of imbalance, it is generally within the controllable range; the task priority distribution includes 23 emergency tasks, 156 ordinary tasks, and 145 low-priority tasks, reflecting the system's refined scheduling capabilities. In terms of real-time performance monitoring, the system's average response time is 156.3ms, the task completion rate is 96.4%, the resource utilization rate is 87.6%, the network training has completed 450 / 1000 rounds and has reached convergence, and the sample pool capacity is 8756 / 10000. These indicators fully reflect the system's efficient operation and good training progress. Overall, the monitoring panel shows the system's outstanding performance in task scheduling, resource allocation, and performance optimization, proving the significant advantages of the two-tier architecture in practical applications.

[0055] The two-layer architecture realizes the coordinated optimization of macro task allocation and micro resource scheduling, improving the overall efficiency of task scheduling and resource utilization. The multi-dimensional priority evaluation and comprehensive reward function are adopted to fully consider factors such as task urgency, resource requirements and execution efficiency, making task scheduling more reasonable and efficient. Combined with advanced technologies such as dual Q network, attention mechanism and priority experience replay, the training effect and convergence speed of the reinforcement learning model are improved, and the stability and generalization ability of the scheduling strategy are enhanced.

[0056] In an optional implementation, the physiological, visual and audio data of the patient are collected through multi-source sensors and time-series alignment calibration is performed, multimodal features are extracted from the calibrated data and adaptive fusion is performed, the urgency of the patient is calculated based on the multimodal features and the group injury distribution is predicted, a resource scheduling plan is generated according to the group injury distribution, and the parameters are dynamically optimized and adjusted by real-time evaluation of system performance, including: The patient's physiological data, visual data and audio data are collected in real time through multi-source sensors, an alignment weight coefficient is generated according to the timestamp, and a temporal alignment loss function is calculated based on the alignment weight coefficient to calibrate the physiological data, visual data and audio data to obtain calibrated multimodal data; Feature extraction is performed on the calibrated multimodal data. Multi-scale spectrum analysis of physiological data is performed through wavelet transform to obtain time domain and frequency domain features. Facial micro-expression features are extracted from visual data through VGG network. Acoustic features are extracted from audio data through time-frequency attention mechanism to obtain multimodal feature data. The multimodal feature data is input into the cross-modal attention fusion mechanism for adaptive weight fusion to obtain fusion features, and the fusion features are input into the multi-level feature extraction network to extract physiological features, behavioral features and semantic features respectively. Different evaluation indicators are comprehensively scored based on the dynamic weight coefficient to obtain the urgency score; Patients are classified according to their urgency scores. In group injury scenarios, multi-source information fusion is performed based on multimodal feature data to estimate the number of injured and predict the distribution of injuries. A resource scheduling optimization objective function is constructed based on the number of injured and the distribution of injuries, and the optimal resource scheduling plan is generated by optimizing the resource scheduling optimization objective function. A multi-dimensional performance evaluation indicator system is constructed to monitor the execution status of the optimal resource scheduling solution in real time. The performance improvement metrics are input into the deep reinforcement learning model to obtain the optimization strategy. The optimization knowledge base is updated based on the execution effect and the strategy network parameters are dynamically adjusted. Combined with the fault recovery mechanism, dynamic balance optimization of system performance and reliability is achieved.

[0057] The multi-source sensor data acquisition module collects the patient's ECG, respiration, blood oxygen and other physiological signals through physiological sensors, with the sampling frequency set to 256Hz; the patient's facial image is collected through the camera with a resolution of 1920×1080 and a frame rate of 30fps; the patient's voice and environmental sound are collected through the microphone with a sampling rate of 16kHz. The data collected by each sensor contains unified timestamp information.

[0058] The data timing alignment module calculates the timing deviation between different modal data based on the timestamp information and generates the alignment weight coefficient. When the difference between the physiological data sampling time and the visual data sampling time is less than 10ms, the weight coefficient is set to 1.0; when the difference is 10-50ms, the weight coefficient linearly decays to 0.8; when the difference is greater than 50ms, the weight coefficient is set to 0. The weight coefficient of audio data is calculated in a similar way. The weight coefficient is applied to the timing alignment loss function to calibrate the multimodal data and achieve millisecond-level timing alignment accuracy.

[0059] The feature extraction module processes the calibrated multimodal data separately: the physiological data is decomposed into 4 layers using wavelet transform to obtain the time-frequency features of delta, theta, alpha, and beta bands; the visual data is input into the pre-trained VGG16 network, the feature map of the 5th convolutional layer is extracted, and the 256-dimensional facial micro-expression features are obtained; the audio data is subjected to the time-frequency attention mechanism to extract the acoustic features such as the fundamental frequency and resonance peak. The features of each modality are normalized to form a unified feature space.

[0060] The cross-modal feature fusion module uses the attention mechanism to calculate the importance weights of different modal features. For physiological features, the focus is on abnormal waveform segments, with a weight range of 0.3-0.5; for facial features, the focus is on the eye and mouth areas, with a weight range of 0.2-0.4; for acoustic features, the focus is on abnormal speech segments, with a weight range of 0.2-0.4. Feature fusion is performed based on the weights to generate a 128-dimensional fusion feature vector.

[0061] The multi-level feature extraction network contains three parallel branches: the physiological feature branch uses a three-layer fully connected network with 64, 32, and 16 hidden nodes respectively; the behavioral feature branch uses two convolutional layers and one fully connected layer; the semantic feature branch uses an LSTM network with a hidden dimension of 32. The output features of each branch are dynamically weighted and fused to obtain an urgency score ranging from 0 to 100.

[0062] In the case of mass injuries, the number of injured people is estimated based on the fusion features, and the distribution of injuries is predicted. When a severely injured person is detected, the urgency weight is above 0.5; the weight of a moderately injured person is between 0.3-0.5; and the weight of a lightly injured person is below 0.3. The resource scheduling optimization target is constructed based on the weight distribution to prioritize the treatment needs of severely injured people.

[0063] The performance evaluation index system includes three dimensions: response time, resource utilization, and task completion rate. The evaluation index is input into the deep reinforcement learning model, the experience replay pool capacity is set to 10,000, and the parameters are updated every 256 samples. When the system performance is lower than the preset threshold, the fault recovery mechanism is triggered, and the backup node takes over the task to ensure system reliability.

[0064] Figure 4 This is a heat map of resource load distribution in an embodiment of the present invention: This figure shows the comparative heat map of edge node load distribution before and after optimization. The left side "Before optimization - edge node load distribution" shows the situation before system optimization, and the right side "After optimization - edge node load distribution" shows the result after system optimization.

[0065] The heat map before optimization is black, and the values ​​are generally high, ranging from 75 to 98. The highest load reaches 98 (the first row and the sixth column), and the loads of multiple nodes exceed 90, such as 92, 95, 93, 97, etc., indicating that the system is in a high-load state. The load distribution between nodes is uneven, and some areas concentrate a large number of high-load points, such as the first row, the third row, and the eighth row, while some areas are relatively low, such as 76, 75, etc., but the overall load is still high.

[0066] The optimized heat map is gray, with significantly lower values ​​and more evenly distributed, most of which are between 63 and 74. The highest load is only 74 (row 7, column 4), and most node loads remain between 65 and 70, with no node exceeding 75. The overall load level is significantly reduced, and the load differences between nodes are reduced, reflecting a more balanced resource allocation.

[0067] By comparison, it can be seen that the average system load after optimization is reduced by about 20 percentage points, the load distribution is more balanced, and the peak load is greatly reduced, indicating that the system optimization strategy effectively improves resource utilization efficiency, reduces the computing pressure of edge nodes, and helps to improve the overall performance and stability of the system.

[0068] Data processing effect: Through the precise timing alignment and feature extraction of multi-source sensor data, the accuracy of multimodal data fusion is improved, and millisecond-level alignment accuracy is achieved. The feature fusion method based on the attention mechanism improves the extraction effect of key features, making the urgency assessment more accurate. Scheduling optimization effect: The dynamic weight fusion mechanism can adaptively adjust the importance of different modal features, and improve the accuracy of injury assessment in group injury scenarios. The priority scheduling strategy ensures that seriously injured patients receive timely treatment and improves the overall treatment efficiency. System reliability: The multi-dimensional performance evaluation system realizes real-time monitoring of the system operation status, and the deep reinforcement learning model can dynamically optimize the system parameters. The fault recovery mechanism ensures the continuous and stable operation of the system and significantly improves the system reliability.

[0069] In an optional implementation, a multi-dimensional performance evaluation index system is constructed to monitor the execution status of the optimal resource scheduling solution in real time, the performance improvement metric is input into the deep reinforcement learning model to obtain the optimization strategy, the optimization knowledge base is updated based on the execution effect and the policy network parameters are dynamically adjusted, and the dynamic balance optimization of system performance and reliability is achieved in combination with the fault recovery mechanism, including: Build a multi-dimensional performance evaluation indicator system to collect data processing delay, computing load, network bandwidth occupancy, and memory usage of each edge node. Calculate end-to-end processing delay, system throughput, and resource utilization efficiency based on data processing delay, computing load, network bandwidth occupancy, and memory utilization, and construct end-to-end processing delay, system throughput, and resource utilization efficiency as performance evaluation vectors. Monitor the performance evaluation vector in real time and calculate the performance improvement metrics before and after optimization. The performance improvement metrics include latency improvement rate, throughput improvement rate, and resource saving rate. Input the performance improvement metrics into the deep reinforcement learning model for training to obtain the performance optimization strategy. Based on the performance optimization strategy, the task allocation and resource scheduling of edge nodes are executed, and the state feature vector, optimization decision vector and performance improvement vector in the execution process are recorded and stored in the optimization knowledge base, where the state feature vector includes resource utilization and load level, the optimization decision vector includes resource allocation and task migration decision, and the performance improvement vector includes the improvement degree of each performance indicator; Calculate the cosine similarity between the current state feature vector and the historical state feature vector in the optimization knowledge base, build a comprehensive performance reward function based on the cosine similarity and the current execution effect, use the comprehensive performance reward function to calculate the policy gradient and update the policy network parameters of the deep reinforcement learning model to generate a new performance optimization strategy; Real-time monitoring of the execution effect of performance optimization strategies. When the fluctuation coefficient of the performance indicator exceeds the preset fluctuation threshold or the system stability time exceeds the preset time threshold, the optimization decision vector with the highest historical similarity is selected from the optimization knowledge base as an alternative strategy and switched to ensure the stability of system performance. Monitor the fault status of edge nodes, record the fault recovery time and calculate the average fault recovery time. Combine the system average failure time to calculate the system availability index. Use the system availability index as a constraint for selecting performance optimization strategies to achieve dynamic balance optimization of system performance and reliability.

[0070] When building a multi-dimensional performance evaluation indicator system, we first deploy performance monitoring probes on each edge node to collect real-time performance data of the node. For data processing delay, we obtain it by recording the time difference from input to output of the data packet; computing load is characterized by collecting CPU usage; network bandwidth occupancy is calculated by monitoring the data transmission rate of the network interface; memory usage is obtained by collecting the ratio of used memory to total memory. Data is collected continuously with a sampling period of 10ms.

[0071] Normalize the collected performance data and calculate the end-to-end processing delay. Assuming there are three edge nodes in the system, the processing delay from collecting to node 1 is 15ms, node 2 is 12ms, and node 3 is 18ms, then the end-to-end delay is 45ms. The system throughput is calculated by the number of tasks completed per unit time, such as processing 1,000 tasks per second. Resource utilization efficiency is obtained by calculating the weighted average of the resource utilization of each node, and the weight can be set according to the importance of the resource.

[0072] Monitor the changes in the performance evaluation vector in real time and calculate the performance improvement indicators before and after optimization. The latency improvement rate is calculated by the reduction ratio of the end-to-end latency before and after optimization; the throughput improvement rate is based on the growth ratio of the throughput before and after optimization; and the resource saving rate is calculated by the reduction ratio of the resource utilization before and after optimization. These indicators are used as training data for deep reinforcement learning.

[0073] When executing the optimization strategy, the system status characteristics are recorded. For example, at a certain moment, the CPU utilization of node 1 is 75%, the memory utilization is 60%, and the network bandwidth utilization is 50%, forming a state feature vector. Optimization decisions include migrating tasks from nodes with heavier loads to nodes with lighter loads, dynamically adjusting resource allocation ratios, etc. The performance improvement vector records that after the execution of the decision, the latency is reduced by 20%, the throughput is increased by 15%, and the resources are saved by 10%.

[0074] Update the strategy based on the similarity between the current state and the historical state. Assuming that the cosine similarity between the current state and a certain historical state is 0.9, and the optimization effect under the historical state is good, then increase the selection probability of the corresponding decision. Set the performance fluctuation threshold to 10% and the stability time threshold to 30s. When it is detected that the performance fluctuation exceeds 10% or the system stability time exceeds 30s, select the alternative strategy with the highest similarity from the knowledge base.

[0075] In terms of reliability, record the time interval from node failure to recovery. Assuming the system runs for 8 hours, 2 failures occur, and the recovery time for each failure is 5 minutes, the average failure recovery time is 5 minutes and the system availability is 98%. Set the availability requirement to 95% as the optimization constraint.

[0076] Figure 5 This is a schematic diagram of the processing efficiency under dynamic load changes of an embodiment of the present invention: This figure shows the trend of resource utilization efficiency of different scheduling algorithms as the system load changes within 20 minutes. The circular (○) line represents the "present technical solution" (multi-dimensional performance evaluation and deep reinforcement learning scheduling algorithm), showing that it started from 85%, dropped to a low of 65% in the 7th minute, rebounded to a peak of 90% in the 12th minute, and finally dropped to 67%. The square (□) line represents the "load balancing method" (task allocation algorithm based on load balancing), which fluctuates between 78% and 90%. The diamond (◇) line is the "nearest node method" (allocation algorithm that selects the nearest edge node), which performs best, reaching a peak of 98% in the 7th and 15th minutes. The triangle (△) dotted line shows that the system load rose from 40% to a peak of 90% in the 11th minute and then fell back, clearly showing the differences in the adaptability of different algorithms under dynamic load environments.

[0077] This embodiment realizes the intelligent dynamic adjustment of system resource utilization efficiency by constructing a multi-dimensional performance evaluation index system and combining it with a resource scheduling algorithm of deep reinforcement learning. The traditional load balancing method only performs mechanical task allocation based on a single load index, which cannot cope with changes in complex network environments; although the nearest node method performs well under high load, it lacks the perception of the global resource status, resulting in large fluctuations in utilization efficiency. In contrast, this technical solution innovatively integrates multi-dimensional indicators such as data processing delay, computing load, and network bandwidth occupancy, constructs a complete performance evaluation vector, and inputs performance improvement metrics into the deep reinforcement learning model. When the system load changes dramatically, this solution can actively adjust the resource allocation strategy. Although the efficiency decreases slightly in the initial adaptation stage, the efficiency is quickly recovered through the strategy optimization assisted by the knowledge base, and finally shows good adaptability and resource utilization balance throughout the load cycle, effectively solving the stability and efficiency problems of resource scheduling in the edge computing environment.

[0078] The method described in this embodiment also includes: When image data collection and analysis is implemented, a high-definition camera array is deployed with a collection frequency of 30 frames per second and a resolution of 1920x1080 pixels. The deep learning model is used to identify changes in the patient's facial expressions, including facial features such as pain and anxiety. At the same time, real-time imaging analysis of trauma can be performed to identify key indicators such as trauma area, depth, and amount of bleeding.

[0079] For example, when a painful facial expression is detected, the system will record characteristic parameters such as the degree of frowning and the angle of drooping mouth corners. For trauma images, the system can measure the size of the wound, identify the proportion of bleeding area, and determine the degree of bleeding. When the wound area exceeds 5% of the body surface area or the estimated amount of bleeding exceeds 500ml, it is automatically determined to be a severe trauma.

[0080] The sound data collection uses a high-sensitivity microphone array with a sampling rate of 44.1KHz. The system can identify sound features such as groaning, coughing, and wheezing. Through sound feature analysis, indicators such as the degree of dyspnea and pain can be determined.

[0081] For example, when rapid panting, respiratory rate exceeding 24 breaths per minute, or persistent groaning for more than 10 seconds is detected, the system will mark it as a case that requires priority treatment.

[0082] Physiological data collection uses a portable multi-parameter monitor to collect ECG, blood pressure, blood oxygen and other data in real time. The sampling frequency is 200Hz for ECG, and 1 time / minute for blood pressure and blood oxygen. The system analyzes vital signs data in real time and immediately alarms when abnormalities occur.

[0083] A lightweight deep learning model is deployed on the edge server for image recognition and sound analysis. The server is equipped with a 16-core CPU, 32GB memory, and supports edge AI acceleration. Data processing delay is controlled within 100ms.

[0084] The edge nodes are deployed in a distributed manner, with an independent edge server configured in each emergency area. The servers are interconnected through a high-speed network to support collaborative data processing. The system completes preprocessing and feature extraction of raw data at the edge nodes, reducing transmission bandwidth requirements.

[0085] The system conducts four-level triage of patients based on the results of multimodal data analysis. The triage levels are from high to low: critical, emergency, acute, and non-acute. Different levels correspond to different processing time requirements.

[0086] Critical cases require immediate treatment, including: impaired consciousness, shock, heavy bleeding, etc. Emergency cases require treatment within 10 minutes, such as acute chest pain, severe trauma, etc. Emergency cases require treatment within 30 minutes, such as fever, abdominal pain, etc. Non-emergency cases can be appropriately delayed.

[0087] In the case of mass injuries, the system supports multi-point data collection and can process information of dozens of injured people at the same time. Through mobile collection equipment, the images and physiological data of the injured can be quickly obtained, and batch rapid triage can be achieved at the first scene and during transportation.

[0088] The system sets up an injury scoring mechanism, which comprehensively considers factors such as consciousness, breathing, and circulation. The higher the score, the more critical the condition. Priority is sorted according to the scoring results to guide the order of treatment. At the same time, the required medical resources are predicted to assist in the deployment of ambulances, medical staff, etc.

[0089] Through multimodal data fusion and edge computing technology, rapid assessment of the patient's condition is achieved, and the triage time is shortened from 3-5 minutes for traditional manual work to less than 30 seconds. The system can work continuously for 7*24 hours, avoiding fatigue errors that are prone to occur in manual triage. Multi-dimensional data analysis improves the accuracy of triage, from 85% of manual triage to more than 95%. The system can identify subtle changes in physical signs that are difficult for the human eye to detect, and detect potential risks early. Edge computing ensures real-time data processing and buys valuable time for emergency case processing. The intelligent triage system optimizes the scheduling of medical resources and reduces the situation where non-emergency patients occupy emergency resources. In mass injury incidents, the system can quickly complete the classification of large-scale casualties and improve the efficiency of treatment. Back-end data analysis helps hospitals continuously improve emergency processes and improve overall service levels.

[0090] According to a second aspect of the embodiments of the present invention, An electronic device is provided, comprising: processor; a memory for storing processor-executable instructions; The processor is configured to call instructions stored in the memory to execute the aforementioned method.

[0091] According to a third aspect of the embodiments of the present invention, A computer-readable storage medium is provided, on which computer program instructions are stored. When the computer program instructions are executed by a processor, the aforementioned method is implemented.

[0092] The present invention may be a method, an apparatus, a system and / or a computer program product. The computer program product may include a computer-readable storage medium carrying computer-readable program instructions for executing various aspects of the present invention.

[0093] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or replace some or all of the technical features therein with equivalents. However, these modifications or replacements do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of the present invention.

Claims

1. An intelligent pre-diagnosis and triage method based on multimodal data and edge computing, characterized in that: include: Medical-grade equipment is used to collect the patient's facial expressions, injury sites, sounds, and physiological signals. The deep learning network is used to extract the image features of facial expressions and injury sites to calculate the intensity of expression and the degree of injury. The neural network is used to analyze the sound features to obtain the pain score and breathing score. The physiological signals are combined to calculate the cardiac function and blood pressure abnormality. The expression intensity, injury degree, pain score, breathing score, cardiac function, and blood pressure abnormality are dynamically weighted and fused to obtain the patient's status assessment result. Acquire multimodal data from medical devices and store them by type to edge nodes for time alignment, build vector allocation tasks based on the resource status of edge nodes, perform data preprocessing and feature extraction, dynamically adjust calculation accuracy, transmit compressed features to the cloud for analysis, and migrate tasks to nodes below the preset load threshold when processing delay exceeds the preset threshold to ensure real-time data processing; Multi-source sensors are used to collect the patient's physiological, visual, and audio data and perform time alignment calibration. Multimodal features are extracted from the calibrated data and adaptively fused. The patient's urgency is calculated based on the multimodal features and the distribution of group injuries is predicted. A resource scheduling plan is generated based on the distribution of group injuries. Parameters are dynamically optimized and adjusted through real-time evaluation of system performance.

2. The method according to claim 1, characterized in that Medical-grade equipment is used to collect the patient's facial expressions, injury sites, sounds, and physiological signals. The deep learning network is used to extract the image features of facial expressions and injury sites to calculate the intensity of expression and the degree of injury. The neural network is used to analyze the sound features to obtain the pain score and breathing score. The cardiac function and blood pressure abnormality are calculated in combination with physiological signals. The expression intensity, degree of injury, pain score, breathing score, cardiac function, and blood pressure abnormality are dynamically weighted and fused to obtain the patient status assessment results, including: Acquire multimodal physiological data of patients through medical-grade acquisition equipment, including acquiring facial expression images and trauma site images through medical-grade camera systems, acquiring sound signals through medical-grade directional microphone arrays, and acquiring electrocardiogram signals, blood pressure signals, body temperature signals, and blood oxygen saturation signals through physiological signal acquisition equipment; The preprocessed images of the wound site and facial expression were preprocessed to obtain preprocessed images. The preprocessed images were input into the U-Net network for regional segmentation and the wound severity score was calculated through the multi-task learning network. The preprocessed facial expression images were input into the VGG network to extract facial features and calculate the expression intensity score. The preprocessed wound site images were input into the multi-scale feature pyramid network to extract wound surface features and calculate the injury severity. The sound signal is subjected to noise suppression and enhancement processing to obtain an enhanced signal, the sound features are extracted from the enhanced signal and the pain degree score is calculated through the long short-term memory network, the enhanced signal is subjected to frequency band decomposition to obtain the breath sound features and the degree of dyspnea is calculated; the acquired electrocardiogram signal is subjected to wave group analysis to extract features and calculate the cardiac function score, the acquired blood pressure signal is subjected to trend analysis to obtain the degree of blood pressure abnormality, and the acquired body temperature signal and blood oxygen saturation signal are subjected to normalization processing; A multimodal feature vector is constructed by the trauma severity score, expression intensity score, injury severity, pain degree score, dyspnea degree, cardiac function score and blood pressure abnormality degree. The multimodal feature vector is input into the feature weight generation network, and the feature weights are dynamically adjusted based on historical data to obtain weighted features. The weighted features are combined with historical status information to generate the patient's current status assessment results.

3. The method according to claim 2, characterized in that The preprocessed images of the wound site and facial expression images are preprocessed to obtain preprocessed images. The preprocessed images are input into the U-Net network for regional segmentation and the wound severity score is calculated through the multi-task learning network. The preprocessed facial expression images are input into the VGG network to extract facial features and calculate the expression intensity score. The preprocessed wound site images are input into the multi-scale feature pyramid network to extract wound surface features. The calculation of the injury severity includes: Performing image preprocessing on the image data of the wound site to obtain a preprocessed wound image, wherein the image preprocessing includes image denoising, contrast enhancement and geometric correction; inputting the preprocessed wound image into a wound region segmentation network to perform wound region segmentation to obtain a wound region segmentation result; The preprocessed trauma image and trauma area segmentation results are input into a multi-task learning network. The multi-task learning network includes a trauma classification branch, a trauma depth estimation branch, and a trauma edge detection branch. The trauma classification branch uses densely connected convolution blocks to extract multi-scale features for trauma type classification. The trauma depth estimation branch predicts trauma depth information based on a deep regression network. The trauma edge detection branch uses an edge detection operator to extract trauma edge contour features. The trauma type classification, trauma depth information and trauma edge contour features are input into the feature fusion module. The feature fusion module uses an adaptive weight method to fuse multi-task features to obtain fusion features; the trauma severity score is calculated based on the fusion features.

4. The method according to claim 1, characterized in that Acquire multimodal data from medical devices and store them by type to edge nodes for time alignment, build vector allocation tasks based on the resource status of edge nodes, perform data preprocessing and feature extraction, dynamically adjust calculation accuracy, transmit compressed features to the cloud for analysis, and migrate tasks to nodes below the preset load threshold when processing delay exceeds the preset threshold to ensure real-time data processing, including: The multimodal data are stored in the corresponding edge node buffer area according to the data type, and the multimodal data are time-series aligned based on the acquisition timestamp to generate aligned multimodal data; Construct a two-layer architecture of macro task allocation layer and micro resource scheduling layer to realize bidirectional information transmission between layers, perform multi-dimensional priority evaluation on tasks to obtain a comprehensive reward function, input the comprehensive reward function into the dual Q network structure and combine it with attention mechanism and priority experience playback for training, and finally output the task scheduling strategy; Monitor the resources of the edge node cache area, obtain the computing load, memory occupancy rate and network bandwidth status of the edge node cache area, and construct a resource state vector based on the computing load, memory occupancy rate and network bandwidth status. The resource state vector is used to characterize the real-time resource status of the edge node; input the resource state vector and the aligned multimodal data into the task scheduling model, which is obtained based on deep reinforcement learning training. The task scheduling model outputs a task allocation strategy based on the processing requirements of the resource state vector and the task scheduling strategy, and allocates the data processing task to the corresponding edge node according to the task allocation strategy; Perform data preprocessing and feature extraction on the edge node, perform noise reduction and standardization on the multimodal data to obtain preprocessed data, group the preprocessed data according to a preset batch size to obtain data batches, and input the data batches into a lightweight feature extraction network to extract features to obtain feature data; dynamically adjust the calculation accuracy of the lightweight feature extraction network based on the resource state vector of the edge node, reduce the network quantization bit width and calculation accuracy when the resource state vector indicates that the resources are tight, and improve the calculation accuracy when the resource state vector indicates that the resources are sufficient, and compress and encode the feature data to obtain compressed features; The compressed features are transmitted to the cloud server through an encrypted channel, the compressed features are decoded on the cloud server to obtain decoded features, the decoded features are analyzed and processed to generate analysis results; the data processing delay of the edge node is monitored in real time. When the data processing delay exceeds the preset delay threshold, the target edge node with a load lower than the preset load threshold is selected based on the resource state vector, and the data processing task to be processed is migrated to the target edge node to ensure the real-time performance of data processing.

5. The method according to claim 4, characterized in that A two-layer architecture of macro task allocation layer and micro resource scheduling layer is constructed to realize bidirectional information transmission between layers. Multi-dimensional priority evaluation of tasks is performed to obtain a comprehensive reward function. The comprehensive reward function is input into the dual Q network structure and trained in combination with attention mechanism and priority experience playback. The final output task scheduling strategy includes: A two-layer reinforcement learning architecture including a macro task allocation layer and a micro resource scheduling layer is constructed. The state space of the macro task allocation layer includes the resource state matrix, the set of pending tasks and the task queue state. The action space of the macro task allocation layer includes the task grouping strategy and the node selection strategy. The state space of the micro resource scheduling layer includes the state space of the macro task allocation layer, the local resource utilization rate and the task dependency graph. The action space of the micro resource scheduling layer includes the resource allocation ratio, the task priority adjustment and the scheduling sequence arrangement. The tasks assigned by the macro task allocation layer are specifically scheduled through the micro resource scheduling layer. The decision results of the macro task allocation layer are converted into the constraints of the micro resource scheduling layer through the strategy mapping function. The state information of the micro resource scheduling layer is compressed and transmitted to the macro task allocation layer through the state aggregation function, so as to realize the two-way transmission of information between layers. Perform multi-dimensional priority evaluation on the tasks received by the micro-resource scheduling layer, calculate the task urgency based on the task deadline, calculate the resource demand based on the demand for various resources, multiply the task urgency by the task execution efficiency reward to obtain the first product, multiply the resource demand by the task resource utilization reward to obtain the second product, and add the first product and the second product to obtain the comprehensive reward function of the task; A dual Q network structure consisting of a main network and a target network is constructed. The main network and the target network use the same network structure. The comprehensive reward function is input into the main network. The loss function is constructed based on the temporal difference error to train the main network, and the network parameters of the main network are periodically copied to the target network. The state features in the state space of the micro-resource scheduling layer are input into the attention mechanism module, and the importance weights of the state features are calculated through the attention mechanism module, and the state features are weighted according to the importance weights. The training samples are stored in the experience replay pool according to the priority determined by the temporal difference error, and the main network is sampled from the experience replay pool according to the sample priority to train the main network, and the scheduling strategy of the task is output based on the trained main network.

6. The method according to claim 1, characterized in that The patient's physiological, visual and audio data are collected through multi-source sensors and time-series alignment and calibration is performed. Multimodal features are extracted from the calibrated data and adaptively fused. The patient's urgency is calculated based on the multimodal features and the distribution of group injuries is predicted. A resource scheduling plan is generated based on the distribution of group injuries. The parameters are dynamically optimized and adjusted by real-time evaluation of system performance, including: The patient's physiological data, visual data and audio data are collected in real time through multi-source sensors, an alignment weight coefficient is generated according to the timestamp, and a temporal alignment loss function is calculated based on the alignment weight coefficient to calibrate the physiological data, visual data and audio data to obtain calibrated multimodal data; Feature extraction is performed on the calibrated multimodal data. Multi-scale spectrum analysis of physiological data is performed through wavelet transform to obtain time domain and frequency domain features. Facial micro-expression features are extracted from visual data through VGG network. Acoustic features are extracted from audio data through time-frequency attention mechanism to obtain multimodal feature data. The multimodal feature data is input into the cross-modal attention fusion mechanism for adaptive weight fusion to obtain fusion features, and the fusion features are input into the multi-level feature extraction network to extract physiological features, behavioral features and semantic features respectively. Different evaluation indicators are comprehensively scored based on the dynamic weight coefficient to obtain the urgency score; Patients are classified according to their urgency scores. In group injury scenarios, multi-source information fusion is performed based on multimodal feature data to estimate the number of injured and predict the distribution of injuries. A resource scheduling optimization objective function is constructed based on the number of injured and the distribution of injuries, and the optimal resource scheduling plan is generated by optimizing the resource scheduling optimization objective function. A multi-dimensional performance evaluation indicator system is constructed to monitor the execution status of the optimal resource scheduling solution in real time. The performance improvement metrics are input into the deep reinforcement learning model to obtain the optimization strategy. The optimization knowledge base is updated based on the execution effect and the strategy network parameters are dynamically adjusted. Combined with the fault recovery mechanism, dynamic balance optimization of system performance and reliability is achieved.

7. The method according to claim 6, characterized in that Construct a multi-dimensional performance evaluation index system to monitor the execution status of the optimal resource scheduling solution in real time, input the performance improvement metrics into the deep reinforcement learning model to obtain the optimization strategy, update the optimization knowledge base based on the execution effect and dynamically adjust the policy network parameters, and combine the fault recovery mechanism to achieve dynamic balance optimization of system performance and reliability, including: Build a multi-dimensional performance evaluation indicator system to collect data processing delay, computing load, network bandwidth occupancy, and memory usage of each edge node. Calculate end-to-end processing delay, system throughput, and resource utilization efficiency based on data processing delay, computing load, network bandwidth occupancy, and memory utilization, and construct end-to-end processing delay, system throughput, and resource utilization efficiency as performance evaluation vectors. Monitor the performance evaluation vector in real time and calculate the performance improvement metrics before and after optimization. The performance improvement metrics include latency improvement rate, throughput improvement rate, and resource saving rate. Input the performance improvement metrics into the deep reinforcement learning model for training to obtain the performance optimization strategy. Based on the performance optimization strategy, the task allocation and resource scheduling of edge nodes are executed, and the state feature vector, optimization decision vector and performance improvement vector in the execution process are recorded and stored in the optimization knowledge base, where the state feature vector includes resource utilization and load level, the optimization decision vector includes resource allocation and task migration decision, and the performance improvement vector includes the improvement degree of each performance indicator; Calculate the cosine similarity between the current state feature vector and the historical state feature vector in the optimization knowledge base, build a comprehensive performance reward function based on the cosine similarity and the current execution effect, use the comprehensive performance reward function to calculate the policy gradient and update the policy network parameters of the deep reinforcement learning model to generate a new performance optimization strategy; Real-time monitoring of the execution effect of performance optimization strategies. When the fluctuation coefficient of the performance indicator exceeds the preset fluctuation threshold or the system stability time exceeds the preset time threshold, the optimization decision vector with the highest historical similarity is selected from the optimization knowledge base as an alternative strategy and switched to ensure the stability of system performance. Monitor the fault status of edge nodes, record the fault recovery time and calculate the average fault recovery time. Combine the system average failure time to calculate the system availability index. Use the system availability index as a constraint for selecting performance optimization strategies to achieve dynamic balance optimization of system performance and reliability.

Images