Batch injury classification system and method based on multi-modal perception and ai decision

Abstract

This invention discloses a batch damage classification system and method based on multimodal perception and AI decision-making, belonging to the field of data processing technology. The batch damage classification method based on multimodal perception and AI decision-making includes: S1, collecting multi-source damage data, preprocessing and extracting features, storing and constructing a multi-source damage database; S2, performing visual feature preservation and pruning strategy adjustments based on fidelity analysis results, and outputting fidelity-limited region labels; S3, performing cross-modal fine-grained evidence compensation analysis based on multimodal feature data of fidelity-limited candidate regions; and S4, performing damage classification control and adjustment based on boundary-sensitive discriminant analysis results and outputting damage classification results. This method solves the problems of loss of fine-grained visual evidence, unstable cross-modal fusion, and fluctuating classification results in batch damage classification under conditions of low contrast, occlusion, and motion blur.

Description

Technical Field

[0001] This invention relates to the field of data processing technology, specifically to a batch damage classification system and method based on multimodal perception and AI decision-making. Background Technology

[0002] In various public safety emergencies, rapid and accurate triage of large numbers of injured individuals is crucial for improving overall treatment efficiency and reducing casualties. Its core requirement is to determine the severity of injuries of multiple injured individuals within a short period, accurately classifying injuries into four levels: red, yellow, green, and black, providing a reliable basis for emergency resource allocation and treatment prioritization. With the application of intelligent sensing and artificial intelligence technologies in the field of emergency medical care, existing triage techniques are gradually evolving from manual judgment to intelligent sensing combined with algorithmic decision-making. Simultaneously, environmental data is being introduced to assist in calibrating the triage results, attempting to adapt to complex on-site rescue environments.

[0003] For example, invention patent CN106650831B discloses a damage detection method and apparatus based on feature fusion, comprising: extracting multiple basic features of a sample to be detected to obtain vectors corresponding to each basic feature of the sample; using a BP neural network classifier to learn the sample based on the vectors corresponding to each basic feature of the sample and a BP neural network model to obtain a damage type label for the sample, wherein the BP neural network model is obtained by the BP neural network classifier based on the damage type labels of training samples and multiple basic features of each training sample; and determining the damage type of the sample based on the damage type label. The damage detection method and apparatus based on feature fusion of this invention can quickly and accurately detect different damage types, and can also determine the damage type of structures that simultaneously possess multiple damage types.

[0004] For example, invention patent CN115458148A discloses an intelligent selection method and device for triage methods. The intelligent selection method includes: forming a training dataset based on real triage data throughout the entire triage cycle; training an artificial neural network using the training dataset to form an intelligent triage method selection model; activating the intelligent triage method selection model based on changes in data dimensions throughout the entire triage cycle to receive current patient information and determine the triage method for the current stage; selecting the triage method based on the data acquisition status. The selection of the triage method for the current stage allows for sufficient processing of the current patient status data, enabling switching to an appropriate triage method based on the stage data. This ensures the reliability of the triage assessment results and effectively promotes the selection of triage methods based on the knowledge rules of professional physicians.

[0005] However, existing batch triage technologies still have many technical shortcomings in practical applications, making it difficult to meet the demands of large-scale, high-precision, and high-stability triage. Firstly, the visual feature processing stage lacks a targeted fine-grained evidence protection mechanism. When performing patch sequence compression and pruning on visual data such as wounds, bleeding areas, and limb postures, fine-grained injury features such as laceration edges and slight posture deviations are easily lost due to fixed processing strategies. Especially in scenarios with image occlusion, blurring, or shooting distance deviations, the effectiveness of fine-grained visual evidence is significantly reduced, directly affecting the accuracy of injury assessment. Secondly, the compensation mechanism for the failure of single-modal evidence during multimodal feature fusion is imperfect. When the fidelity of core modal features such as vision is insufficient, accurate and effective cross-modal evidence compensation cannot be achieved through other modal features such as physiological and speech features, hindering the coordination of multimodal data. Third, the boundary stability control of injury grading is insufficient. For samples at the boundary of adjacent injury grades, there is a lack of a sensitive discrimination mechanism that combines the grading boundary interval and temporal grade shift. This can easily lead to the same patient experiencing a jump in injury grade within adjacent observation time windows. At the same time, the disturbance of environmental calibration is not reasonably constrained, further reducing the stability of grading results. Fourth, the processing and association management of multi-source data lacks standardized procedures. The temporal alignment accuracy of visual, physiological, speech, and environmental data is insufficient. The adaptability of feature extraction dimensions to injury judgment is poor. The fusion and connection of various feature values ​​with the grading model is not smooth, making it difficult to form an efficient and unified batch triage data processing system.

[0006] Therefore, in order to address the above problems, there is an urgent need for a batch damage classification system and method based on multimodal perception and AI decision-making. Summary of the Invention

[0007] Technical problems to be solved To address the shortcomings of existing technologies, this invention provides a batch damage classification system and method based on multimodal perception and AI decision-making, which solves the problems of loss of fine-grained visual evidence, unstable cross-modal fusion, and fluctuating classification results in batch damage classification under conditions of low contrast, occlusion, and motion blur.

[0008] Technical solution To achieve the above objectives, the present invention provides the following technical solution: a batch damage classification method based on multimodal perception and AI decision-making, comprising: S1, collecting multi-source damage data, preprocessing and extracting features from the multi-source damage data, storing the data, and constructing a multi-source damage database; S2, performing fine-grained visual evidence fidelity analysis on the visual feature data, performing visual feature preservation and pruning strategy adjustments based on the fidelity analysis results, and outputting fidelity-limited region markers; S3, performing cross-modal fine-grained evidence compensation analysis based on the multimodal feature data of fidelity-limited candidate regions, and performing evidence enhancement and compensation operations based on the compensation analysis results; S4, performing boundary-sensitive discriminant analysis based on multi-source fusion features, evidence compensation values, grading boundary intervals, and temporal offsets, performing damage classification control and adjustment based on the boundary-sensitive discriminant analysis results, and outputting damage classification results.

[0009] Furthermore, the specific steps for collecting multi-source triage data, including visual data, physiological data, speech data, and environmental data, are as follows: Visual data collection: Image acquisition unit obtains images of the wound area, bleeding area, limb posture, and original patch sequences of candidate regions of the wounded. Physiological data collection: Blood oxygenation sequence, respiratory sequence, electrocardiogram sequence, and blood pressure sequence are obtained. Speech data collection: Speech acquisition unit obtains speech segments describing pain, bleeding, and limited mobility, and performs semantic parsing, keyword extraction, and semantic intensity statistics to obtain semantic impairment support values. Environmental data collection: Natural disaster intensity values, battlefield hazard values, and the supply-demand ratio of medical resources are obtained. Based on the triage grading rules, grade boundary values ​​corresponding to red, yellow, green, and black injury levels are set, and an adjacent grade boundary index corresponding to the current sample is established.

[0010] Further, the specific steps for preprocessing and feature extraction of multi-source triage data, and then storing and constructing a multi-source triage database, are as follows: Sliding window aggregation, median filtering for noise reduction, outlier removal, and missing value interpolation are performed on the collected visual, physiological, speech, and environmental data to form triage observation records, which are then scaled to a uniform numerical range using the minimum-maximum normalization method; edges are extracted from wound and bleeding area images, and the connectivity ratio of eight-neighborhood pixels of candidate region edges is calculated and normalized to obtain local edge continuity values; local texture gradient changes are statistically analyzed, and the mean gray-level difference is calculated to obtain local texture abrupt change values; key points are extracted from limb end posture images, and the mean Euclidean distance to reference posture key points is calculated and normalized to obtain local posture offset values; the distribution of candidate regions in the original patch sequence is statistically analyzed, and the ratio of effective to all patch counts is calculated to obtain candidate patch distribution density values; occlusion areas are segmented from wound and bleeding area images, and the ratio of occlusion area to candidate region area is calculated to obtain occlusion influence values; for limb end... Edge sharpness attenuation analysis was performed on posture and wound area images, and the Laplacian response mean attenuation ratio was calculated to obtain the blur effect value. Imaging scale and reference distance were compared for candidate region images, and the relative deviation from the standard imaging scale was calculated to obtain the distance effect value. Time alignment, denoising, and abnormal segment extraction were performed on blood oxygen, respiration, ECG, and blood pressure sequences, and the weighted single-dimensional abnormal amplitude was used to obtain the physiological abnormality support value. Normalization and aggregation calculations were performed on natural disaster intensity, battlefield danger, and the supply-demand ratio of medical resources to generate the environmental threat level for the current observation time window. The basic classification value was input into the environmental calibration unit, and the environmental calibration classification value was obtained according to the environmental threat level and calibration rules. The environmental calibration perturbation value was then obtained based on the difference between the environmental calibration classification value and the basic classification value. A unified time index alignment was performed on the multi-source triage data to form visual, physiological, and semantic features corresponding to the candidate regions. The multi-source triage data was stored in association according to the casualty number, observation time window number, and candidate region location index, and a batch multi-source triage database was constructed.

[0011] Further, the specific steps for fine-grained visual evidence fidelity analysis of visual feature data are as follows: Obtain the local edge continuity value, local texture abrupt change value, local pose shift value, candidate patch distribution density value, occlusion influence value, blur influence value, and distance influence value in the t-th observation time window of the i-th injured person; add the local pose shift value to one in the t-th observation time window of the i-th injured person to obtain the pose correction term; multiply the local edge continuity value, local texture abrupt change value, and pose correction term to obtain the visual joint product term; take the cube root of the visual joint product term to obtain the visual amplitude term; perform a power operation on the candidate patch distribution density value to obtain the retained density adjustment term; sum the occlusion, blur, and distance influence values ​​to obtain the imaging attenuation input term; take the negative value of the imaging attenuation input term and perform an exponential operation to obtain the imaging attenuation term; multiply the visual amplitude term, retained density adjustment term, and imaging attenuation term to obtain the visual fidelity value.

[0012] Furthermore, based on the fidelity analysis results, the specific steps for performing visual feature preservation and pruning strategy adjustments, and outputting fidelity-limited region markers, are as follows: By comparing the visual fidelity value with the fidelity threshold in real time, when the visual fidelity value is less than the fidelity threshold, the retention density coefficient is increased to k to improve the retention ratio of candidate patch fragments. A candidate region layered contraction strategy is adopted to shrink the pruning span corresponding to the patch fragments at the tear edge, bleeding edge, and end pose position to a fraction of the original pruning span. times, Double The algorithm multiplies the data by 100%, performs bilinear interpolation resampling on the Patch raster of the candidate region image, and recalculates the visual fidelity value. If the visual fidelity value is still less than the fidelity threshold, the current candidate region is marked as a fidelity-limited candidate region. When the visual fidelity value is greater than or equal to the fidelity threshold, it is marked as a visually fidelity-sufficient region. The current compressed token sequence and visual fidelity value are then output to the boundary stability judgment module.

[0013] Furthermore, the specific steps for cross-modal fine-grained evidence compensation analysis based on multimodal feature data of fidelity-limited candidate regions are as follows: Obtain the visual fidelity value, physiological abnormality support value, and semantic impairment support value for the i-th injured person in the t-th observation time window within the fidelity-limited candidate region. Perform time alignment, association mapping, and difference measurement on visual changes, physiological changes, and semantic changes under a unified time index. Construct a cross-modal evidence association model based on a multi-head cross-attention mechanism and output the cross-modal offset value. Multiply the physiological abnormality support value and the semantic impairment support value to obtain a joint support term. Add the joint support term to one to obtain a logarithmic input term. Perform natural logarithmic operation on the logarithmic input term to obtain a support gain term. Add the support gain term to one to obtain a compensation amplification term. Square the cross-modal offset value to obtain a squared offset term. Add the squared offset term to one to obtain a offset suppression term. Multiply the visual fidelity value, the compensation amplification term, and the offset suppression term to obtain the evidence compensation value.

[0014] Furthermore, the specific steps for performing evidence enhancement and compensation operations based on the compensation analysis results are as follows: By comparing the evidence compensation value with the compensation threshold in real time, when the evidence compensation value is less than the compensation threshold, the current candidate region is marked as a compensation failure candidate region, the basic fusion feature sequence is retained, and it is archived to the multi-source database of damage detection; when the evidence compensation value is greater than or equal to the compensation threshold, the candidate region visual token sequence, physiological abnormality support value, semantic damage support value, evidence compensation value, and candidate region location index are used as inputs, and a fine-grained evidence enhancement model is constructed using a gated attention algorithm, outputting the compensation enhancement fusion feature sequence, and the compensation enhancement fusion feature sequence and the evidence compensation value are output together to the boundary stability judgment module.

[0015] Furthermore, the specific steps for boundary-sensitive discriminant analysis based on multi-source fusion features, evidence compensation values, hierarchical boundary intervals, and temporal offsets are as follows: Obtain the visual fidelity value and basic fusion feature sequence, successful compensation candidate region markers, compensation-enhanced fusion feature sequence, evidence compensation value, candidate region location index, and compensation failure candidate region markers, basic fusion feature sequence, and evidence compensation value corresponding to the visually fidelity-sufficient region; input the basic fusion feature sequence of the visually fidelity-sufficient region into the basic hierarchical network, which adopts a multi-layer fully connected discriminant structure to perform feature concatenation, linear transformation, and nonlinear activation calculations on visual features, physiological features, and semantic features, outputting a basic hierarchical probability vector and generating a basic hierarchical value based on the highest probability category; calculate the difference between the previous and current observation time window basic hierarchical values ​​to obtain the hierarchical offset value. The classification boundary interval is obtained based on the difference between the scores of the highest and second-highest probability categories in the triage classification. The compensated enhancement fusion feature sequence and the basic fusion feature sequence are input into the basic classification network, and the compensated correction classification value and the failure backoff classification value are output. The corresponding classification value is selected according to the sample source region to obtain the current classification value. The classification boundary interval value, the level offset value and the environmental calibration perturbation value are obtained. The classification boundary interval value is added to the boundary buffer constant to obtain the boundary correction term. The evidence compensation value is divided by the boundary correction term to obtain the boundary sensitive compensation term. The temporal suppression coefficient is multiplied by the level offset value to obtain the temporal suppression input term. The temporal suppression term is obtained by exponential operation after taking the negative value of the temporal suppression input term. The environmental calibration perturbation value is increased by one and the reciprocal is taken to obtain the environmental suppression term. The boundary sensitive compensation term, the temporal suppression term and the environmental suppression term are multiplied to obtain the boundary sensitive discrimination value.

[0016] Furthermore, the specific steps for performing flaw detection grading control and adjustment based on the boundary sensitivity discriminant analysis results and outputting the flaw detection grading results are as follows: By comparing the boundary sensitivity discriminant value with the sensitivity threshold in real time, when the boundary sensitivity discriminant value is less than the sensitivity threshold, the environmental calibration grading value is determined as the final flaw detection grading value; when the boundary sensitivity discriminant value is greater than or equal to the sensitivity threshold, it is marked as a boundary sensitive sample and enters the flaw detection grading control process: For candidate areas with successful compensation, the environmental calibration grading value is used unchanged as the final flaw detection grading value; for candidate areas with failed compensation, the calibration grading value is constrained by the level offset value to perform a jump constraint. If the level offset value is greater than or equal to the sensitivity threshold, the environmental calibration grading value is used as the final flaw detection grading value. If the shift value is greater than the offset threshold, it is judged as an abnormal jump, and the current basic classification value is forcibly rolled back to the final classification value of the previous observation time window; if the classification shift value is less than or equal to the offset threshold, the environmental calibration classification value is kept unchanged as the final damage detection classification value; for areas with sufficient visual fidelity, the smoothing coefficient P is determined by the negative exponential decay relationship between the visual fidelity value and the classification shift value, the classification probability vector of adjacent windows is weighted and averaged, and the threshold is adjusted in combination with the environmental threat level before the final damage detection classification value is output; the final damage detection classification value, boundary sensitive sample label, evidence compensation value and classification boundary interval value are output together to the batch damage detection multi-source database.

[0017] Furthermore, a second aspect of the present invention provides a batch damage classification system based on multimodal perception and AI decision-making, applying a batch damage classification method based on multimodal perception and AI decision-making, comprising: a multi-source damage data acquisition and preprocessing module, used to acquire multi-source damage data, preprocess and extract features from the multi-source damage data, store and construct a multi-source damage database; a visual evidence fidelity assessment module, used to perform fine-grained visual evidence fidelity analysis on visual feature data, perform visual feature preservation and pruning strategy adjustment operations based on the fidelity analysis results, and output fidelity-limited region markers; an evidence compensation module, used to perform cross-modal fine-grained evidence compensation analysis based on multimodal feature data of fidelity-limited candidate regions, and perform evidence enhancement and compensation operations based on the compensation analysis results; and a boundary stability judgment module, used to perform boundary sensitivity discrimination analysis based on multi-source fusion features, evidence compensation values, hierarchical boundary intervals and temporal offsets, perform damage classification control and adjustment based on the boundary sensitivity discrimination analysis results, and output damage classification results.

[0018] Beneficial effects The present invention has the following beneficial effects: (1) This invention performs a fidelity evaluation of fine-grained visual features and dynamically adjusts the patch retention ratio and pruning span based on the fidelity results. It also optimizes the resampling of key areas, thereby achieving the effect of complete preservation of fine-grained visual evidence such as wounds, bleeding and posture. This effectively solves the problems of excessive visual compression pruning, easy loss of key injury features, and poor imaging quality leading to distorted judgment in the prior art.

[0019] (2) This invention realizes the time alignment and correlation mapping of multimodal features based on the multi-head cross attention mechanism, constructs a cross-modal evidence compensation model to enhance evidence in areas with insufficient fidelity, and thus achieves the effect of multimodal complementary support and improved reliability of injury determination when a single modality fails. It effectively solves the problem of lack of effective cross-modal compensation mechanism and the failure of overall assessment caused by single-modal distortion in the prior art.

[0020] (3) This invention uses the boundary sensitivity discrimination value calculated by combining the hierarchical boundary interval, temporal offset and environmental disturbance to constrain the stability of the hierarchical results, thereby achieving the effect of stable judgment of adjacent level boundary samples and avoiding abnormal level jumps. It effectively solves the problems of easy misjudgment of boundary samples, large fluctuation of temporal hierarchical results and unstable hierarchical results caused by environmental disturbance in the prior art.

[0021] (4) This invention integrates multimodal perception, fidelity assessment, evidence compensation and boundary stability discrimination to form an end-to-end batch triage intelligent decision-making process, thereby achieving the effect of rapid, orderly and accurate triage of a large number of injured persons, effectively solving the problems of incomplete intelligent decision-making chain, low batch processing efficiency and insufficient adaptability to emergency scenarios in the existing technology.

[0022] Of course, any product implementing this invention does not necessarily need to achieve all of the advantages described above at the same time. Attached Figure Description

[0023] Figure 1 This is a flowchart of the batch damage detection and classification method based on multimodal perception and AI decision-making of the present invention; Figure 2 This is a structural diagram of the batch damage classification system based on multimodal perception and AI decision-making of the present invention; Figure 3 This is a flowchart of the visual fidelity layering enhancement and adaptive judgment process of the present invention; Figure 4 This is a bar chart showing the sensitive discrimination distribution of the injury triage boundary based on multimodal evidence fusion in this invention. Detailed Implementation

[0024] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0025] Please see Figures 1-4 This invention provides a technical solution: a batch damage detection classification method based on multimodal perception and AI decision-making, comprising: S1, collecting multi-source damage detection data, preprocessing and extracting features from the multi-source damage detection data, storing the data, and constructing a multi-source damage detection database; S2, performing fine-grained visual evidence fidelity analysis on visual feature data, performing visual feature preservation and pruning strategy adjustments based on the fidelity analysis results, and outputting fidelity-limited region markers; S3, performing cross-modal fine-grained evidence compensation analysis based on multimodal feature data of fidelity-limited candidate regions, and performing evidence enhancement and compensation operations based on the compensation analysis results; S4, performing boundary-sensitive discriminant analysis based on multi-source fusion features, evidence compensation values, grading boundary intervals, and temporal offsets, performing damage detection grading control and adjustment based on the boundary-sensitive discriminant analysis results, and outputting damage detection grading results.

[0026] Specifically, the steps for collecting multi-source triage data, including visual data, physiological data, voice data, and environmental data, are as follows: Visual data acquisition: Images of the wound area, bleeding area, limb posture, and original patch sequences of candidate regions for the i-th injured person within the t-th observation time window are acquired via an image acquisition unit. The observation time window length is set to 2-5 seconds, and the sliding window step size is consistent with the time window length. The original patch sequences of candidate regions are generated using a non-overlapping segmentation strategy with a fixed size of 32×32 pixels. The subsequently compressed token sequence has a one-to-one mapping relationship with the patch sequence, and each patch generates a unique corresponding visual token after feature encoding. Physiological data acquisition: Blood oxygen sequence, respiration sequence, electrocardiogram sequence, and blood pressure sequence for the i-th injured person within the t-th observation time window are acquired. Voice data acquisition: Voice data of the i-th injured person is acquired via a voice acquisition unit. Within the t-th observation time window, descriptive audio segments of pain, bleeding, and limited mobility are analyzed, semantic parsing, keyword extraction, and semantic intensity statistics are performed to obtain semantic impairment support values. Environmental data are collected: natural disaster intensity values, battlefield hazard values, and the supply-demand ratio of medical resources are obtained. Based on the internationally accepted triage rules of START and JumpSTART, four levels of injury are set: red (severe injury), yellow (moderate injury), green (minor injury), and black (death). Red corresponds to critical vital signs requiring immediate treatment, yellow corresponds to more severe injuries requiring priority treatment, green corresponds to minor injuries that can be treated later, and black corresponds to no vital signs and no on-site treatment. The threshold values ​​of each level are defined in the form of score intervals, and an index of adjacent level boundaries corresponding to the current sample is established.

[0027] This implementation plan standardizes the observation time window, patch segmentation size, and token mapping relationship, and clearly adopts internationally accepted triage rules to set the level boundaries. This achieves standardization of multi-source data acquisition and clarification of quantitative parameter definitions, ensuring the temporal consistency and repeatability of visual, physiological, speech, and environmental data of the injured. In turn, it achieves the effect of standardized batch triage data acquisition and objective and rigorous grading basis, effectively solving the problems of insufficient reliability of judgment caused by vague parameter definitions, subjective grading rules, and unclear data sources and encoding methods in the existing technology.

[0028] Specifically, the steps for preprocessing and extracting features from multi-source damage detection data, and then storing and constructing a multi-source damage detection database are as follows: The collected visual, physiological, speech, and environmental data were processed using sliding window aggregation, median filtering for noise reduction, outlier removal, and missing value interpolation to form a triage observation record for the i-th injured person at the t-th observation time window. The min-max normalization method was used to scale all feature values ​​to a uniform range of 0 to 1. Edges were extracted from wound and bleeding area images, and the connectivity ratio of eight-neighborhood pixels in candidate regions was calculated and normalized to obtain local edge continuity values. Local texture gradient changes were statistically analyzed, and the mean gray-level difference was calculated to obtain local texture abrupt change values. Key points were extracted from limb end-effector pose images, and the relationship with the reference pose was calculated. The Euclidean distance of key points is averaged and normalized to obtain the local pose offset value; the distribution of candidate regions in the original patch sequence is statistically analyzed, and the ratio of the number of effective patches to the total number of patches is calculated to obtain the candidate patch distribution density value; occlusion regions are segmented in the wound and bleeding area images, and the ratio of occlusion area to candidate region area is calculated to obtain the occlusion impact value; edge sharpness attenuation analysis is performed on the limb end pose and wound area images, and the attenuation ratio of the mean Laplacian response is calculated to obtain the blur impact value; the imaging scale and reference distance of the candidate region images are compared, and the relative deviation value with the standard imaging scale is calculated to obtain the distance impact value; Time alignment, denoising, and abnormal segment extraction were performed on blood oxygen, respiration, electrocardiogram, and blood pressure sequences. Weighted single-dimensional abnormal amplitude was used to obtain physiological abnormality support values. Normalization and aggregation calculations were performed on natural disaster intensity, battlefield danger, and the supply-demand ratio of medical resources to generate the environmental threat level for the current observation time window. The basic classification value was input into an environmental calibration unit, which used a linear weighted calibration rule, employing the environmental threat level as a weighting coefficient to constrain and correct the basic classification value, resulting in an environmental calibration classification value. The difference between the environmental calibration classification value and the basic classification value was then calculated, and the absolute value was taken to obtain the environmental calibration perturbation value. Furthermore, multi-source triage data was aligned using a unified time index based on the observation time window to form visual, physiological, and semantic features corresponding to candidate regions. Multi-source triage data was stored in association according to the casualty number, observation time window number, and candidate region location index, and a batch triage multi-source database was constructed.

[0029] In this implementation plan, standardized preprocessing and refined feature extraction are performed on multi-source flaw detection data, the numerical range is unified, and time alignment and associated storage are completed. At the same time, the linear weighting rules of the environmental calibration unit are clarified, realizing the standardized calculation and reliable management of various visual quantitative indicators, physiological support values ​​and environmental disturbance parameters. This achieves the effect of complete flaw detection feature dimensions, consistent data time sequence, and objective and evidence-based environmental correction, effectively solving the problems of insufficient classification accuracy caused by inconsistent feature calculation standards, chaotic data association, and fuzzy environmental calibration rules in the existing technology.

[0030] Specifically, the steps for fine-grained visual evidence fidelity analysis of visual feature data are as follows: Obtain the local edge continuity value, local texture abrupt change value, local pose shift value, candidate patch distribution density value, occlusion influence value, blur influence value, and distance influence value for the i-th injured person during the t-th observation time window. All indicators are normalized dimensionless values, which can objectively reflect the completeness of fine-grained visual signs and the extent of imaging interference. Add the local pose shift value to one to obtain a pose correction term, thereby weakening the interference of abnormal limb posture on visual feature assessment. Multiply the local edge continuity value, local texture abrupt change value, and pose correction term to obtain a visual joint product term. Take the cube root of the visual joint product term to obtain the visual... The amplitude term comprehensively reflects the overall effectiveness of visual features related to wounds, bleeding, and limb posture. The retention density adjustment term is obtained by exponentiation of the retention density adjustment coefficient on the candidate patch distribution density value, where the retention density adjustment coefficient is a pre-set normal number used to reasonably control the degree of influence of candidate patch retention on visual features. The imaging attenuation input term is obtained by summing the occlusion, blur, and distance influence values. The imaging attenuation term is obtained by exponential operation after taking the negative value of the imaging attenuation input term, which is used to quantify the attenuation effect of various imaging adverse factors on visual evidence. The visual amplitude term, retention density adjustment term, and imaging attenuation term are multiplied to finally obtain the visual fidelity value of the i-th injured person in the t-th observation time window.

[0031] The specific formula for visual fidelity is as follows: ; In the formula, Indicates the first The wounded were in the first Visual fidelity values ​​within a single observation time window; This represents the local edge continuity value, used to characterize the degree of continuity between the tear edge and the bleeding edge; This represents a local texture abrupt change value, used to characterize the degree of difference between the texture of the wound area and the background texture; This represents the local posture deviation value, used to characterize the degree of local abnormalities in slight external rotation, flexion, and hanging of the limb extremities; This represents the candidate patch distribution density value, used to characterize the proportion of candidate regions retained in the compressed sequence; This indicates the impact value of occlusion, reflecting the degree of occlusion in the window injury area; This represents the blur effect value, reflecting the degree of blur in the imaging within the time window; This indicates the impact of distance, reflecting the degree of image degradation caused by the shooting distance within the time window; This represents the retention density adjustment coefficient, used to control the intensity of the influence of the candidate patch retention ratio on the visual fidelity value.

[0032] In this implementation scheme, by performing standardized combination calculations on multiple visual feature indicators and introducing posture correction, patch retention control and imaging interference attenuation mechanisms, a precise quantitative assessment of the fidelity of fine-grained visual signs of the wounded is achieved. This results in objectively measurable quality of visual evidence, effective elimination of various adverse imaging factors, and stable and reliable visual feature assessment. It effectively solves the problems in the prior art where the quality of visual evidence cannot be quantified and imaging interference is difficult to eliminate, leading to distortion in the determination of visual signs.

[0033] Specifically, the steps for performing visual feature preservation and pruning strategy adjustments based on the fidelity analysis results, and outputting fidelity-limited region labels, are as follows: By comparing visual fidelity values ​​with fidelity thresholds in real time, the fidelity threshold is calculated statistically from the optimal boundary point of the ROC curve of a massive batch of visual samples from injury detection, and is fixed at 0.65. When the visual fidelity value is less than the fidelity threshold, the retention density coefficient is increased to k. k is obtained by fitting a comparative experiment between the candidate patch retention rate and the accuracy of visual fine-grained feature recognition for patients with different injury levels, and is fixed at 1.5, thereby improving the retention rate of candidate patch fragments. A candidate region layered contraction strategy is adopted to contract the pruning span corresponding to the patch fragments at the tear edge, bleeding edge, and end posture position to the original pruning span. times, Double times, of which, , and Based on experiments on the sensitivity of fine-grained injury features in different regions, and combined with the balance between feature preservation integrity and data transmission efficiency, the values ​​of γ1, γ2, and γ3 are calculated. The values ​​range from 0.3 to 0.5, γ2 from 0.4 to 0.6, and γ3 from 0.5 to 0.7. The candidate region image Patch raster is resampled by bilinear interpolation, and the visual fidelity value is recalculated. If the visual fidelity value is still less than the fidelity threshold, the current candidate region is marked as a fidelity-limited candidate region. When the visual fidelity value is greater than or equal to the fidelity threshold, it is marked as a visually fidelity-sufficient region. The current compressed token sequence and visual fidelity value are output to the boundary stability judgment module.

[0034] In this implementation scheme, visual fidelity is judged and adaptively adjusted in real time. Candidate patches are retained to strengthen areas with insufficient fidelity, and hierarchical pruning, shrinkage and resampling optimization are performed on key injury areas. This accurately distinguishes between areas with limited fidelity and areas with sufficient visual fidelity, thereby achieving the effect of fine-grained adaptive protection of visual injury features and objective and controllable visual feature processing strategies. It effectively solves the problems of lack of objective basis for visual feature pruning, easy loss of key injury areas and imperfect visual evidence quality judgment and optimization mechanism in the existing technology.

[0035] Specifically, the steps for cross-modal fine-grained evidence compensation analysis based on multimodal feature data of fidelity-limited candidate regions are as follows: The visual fidelity value, physiological anomaly support value, and semantic impairment support value of the i-th injured person in the t-th observation time window within the fidelity-limited candidate region are obtained. These three values ​​are normalized scalars and can be directly used for cross-modal association calculations. Time alignment, association mapping, and difference measurement are performed on visual changes, physiological changes, and semantic changes under a unified time index. A cross-modal evidence association model is constructed based on a multi-head cross-attention mechanism. In this model, the query features for attention are derived from the visual features corresponding to the visual fidelity value, the key features from the physiological features corresponding to the physiological anomaly support value, and the value features from the semantic impairment support value. Semantic features are defined as follows: the cross-modal offset value is obtained by calculating the weighted sum of the attention weight difference and the inter-modal feature alignment error, and then outputting the cross-modal offset value; the physiological abnormality support value is multiplied by the semantic impairment support value to obtain the support joint term; the support joint term is added to one to obtain the logarithmic input term; the logarithmic input term is subjected to natural logarithmic operation to obtain the support gain term; the support gain term is added to one to obtain the compensation amplification term; the cross-modal offset value is squared to obtain the offset squared term; the offset squared term is added to one to obtain the offset suppression term; the visual fidelity value, the compensation amplification term, and the offset suppression term are multiplied to obtain the evidence compensation value.

[0036] The specific formula for the evidence compensation value is as follows: ; In the formula, Indicates the first The wounded were in the first Evidence compensation value within each observation time window; Indicates visual fidelity; This represents the physiological abnormality support value, used to characterize the degree to which abnormal physical signs support local visual signs. The semantic impairment support value represents the degree to which the semantic description supports the local visual impairment. This represents the cross-modal offset value, used to characterize the degree of synchronization deviation among visual, physiological, and semantic evidence.

[0037] In this implementation scheme, for candidate regions with limited fidelity, a multi-head cross-attention mechanism with limited feature sources is used to achieve accurate alignment and association analysis of visual, physiological, and semantic multimodal features based on normalized scalars. Cross-modal offset values ​​are obtained by weighting the difference in attention weights and feature alignment errors, and evidence compensation values ​​are obtained by combining support joint operations and offset suppression operations. This achieves the effect of multimodal feature complementarity enhancement and accurate cross-modal evidence compensation when visual evidence is insufficient. It effectively solves the problems of generalized application of cross-modal attention mechanisms, unclear intermodal associations, and lack of reliable compensation basis after the failure of single modal evidence in existing technologies.

[0038] Specifically, the steps for performing evidence enhancement and compensation operations based on the compensation analysis results are as follows: After obtaining the evidence compensation value calculated from visual fidelity, physiological abnormality support, and semantic impairment support through cross-modal offset measurement, the fidelity assessment and adaptive enhancement of visual evidence are achieved by comparing the evidence compensation value with the compensation threshold in real time. Figure 3 This is a flowchart of the visual fidelity layering enhancement and adaptive judgment process in this embodiment. When the evidence compensation value is less than the compensation threshold, it indicates that the multimodal evidence quality of the current candidate region is insufficient and the cross-modal consistency is poor. At this time, the current candidate region is marked as a compensation failure candidate region. The basic fusion feature sequence obtained by basic fusion of visual token sequence, physiological features and semantic features is retained, and the basic fusion feature sequence, evidence compensation value and compensation failure mark are archived together to the multi-source database of fault detection as the basis for subsequent tracing and model optimization. When the evidence compensation value is greater than or equal to the compensation threshold, it indicates that the multimodal evidence quality of the current candidate region is insufficient and the cross-modal consistency is poor. The quality of the modal evidence is high and the cross-modal synchronization is good. At this time, the visual token sequence of the candidate region, the physiological abnormality support value, the semantic damage support value, the evidence compensation value, and the candidate region location index are used as inputs. A fine-grained evidence enhancement model is constructed using a gating attention algorithm. This model dynamically adjusts the contribution weights of different modal features through a gating mechanism and enhances the features of key locations such as the tear edge, the bleeding edge, and the end posture. It outputs a compensated enhancement fusion feature sequence and outputs the compensated enhancement fusion feature sequence and the evidence compensation value together to the boundary stability judgment module for subsequent boundary sensitivity discrimination and hierarchical adjustment.

[0039] In this implementation scheme, adaptive triage processing of candidate regions with limited fidelity is achieved by comparing the evidence compensation value and the compensation threshold in real time: when the evidence compensation value is lower than the compensation threshold, the current region is marked as a candidate region for compensation failure, the basic fusion feature sequence is retained and archived in the database, ensuring the traceability of low-quality evidence and the basis for subsequent optimization; when the evidence compensation value reaches or exceeds the compensation threshold, a gated attention enhancement mechanism is activated to fuse and enhance the visual token sequence, physiological abnormality support value, semantic damage support value, and location index, generating a compensated enhanced fusion feature sequence and sending it to the boundary stability judgment module. This step achieves differentiated processing of multimodal evidence based on the quality of the evidence, ensuring the integrity of low-quality evidence while providing more discriminative enhanced features for subsequent boundary-sensitive discrimination of high-quality evidence.

[0040] Specifically, the steps for boundary-sensitive discriminant analysis based on multi-source fusion features, evidence compensation values, hierarchical boundary intervals, and temporal offsets are as follows: The system acquires the visual fidelity value and basic fusion feature sequence, successful compensation candidate region marker, compensation-enhanced fusion feature sequence, evidence compensation value, and candidate region location index corresponding to the visually fidelity-sufficient region, as well as the compensation failure candidate region marker, basic fusion feature sequence, and evidence compensation value. All feature sequences and values ​​are standardized data after unified normalization. The basic fusion feature sequence of the visually fidelity-sufficient region is input into a basic grading network. This network employs a three-layer fully connected discriminant structure, sequentially performing feature concatenation, linear transformation, and nonlinear activation calculations of linear rectifier units on visual features, physiological features, and semantic features. It outputs a basic grading probability vector and generates a basic grading value based on the highest probability category. The absolute difference between the basic grading value of the previous observation time window and the current observation time window is calculated to obtain the grading offset value. The grading boundary interval value is obtained based on the score difference between the highest probability category and the second highest probability category in the damage grading results. The compensation-enhanced fusion feature sequence and the basic fusion feature sequence are then processed together. The basic hierarchical network is input separately, and the corresponding outputs are the compensation correction hierarchical value and the failure backoff hierarchical value. The corresponding hierarchical value is selected based on whether the sample belongs to a visually fidelity-sufficient region, a successful compensation candidate region, or a failed compensation candidate region, to obtain the current hierarchical value. The hierarchical boundary interval value, the hierarchical offset value, and the environmental calibration perturbation value are obtained. The hierarchical boundary interval value is added to the boundary buffer constant to obtain the boundary correction term. The boundary buffer constant is obtained by statistical optimization calculation based on the misjudgment rate of batch flaw detection boundary samples, and its value is 0.2. The evidence compensation value is divided by the boundary correction term to obtain the boundary sensitivity compensation term. The temporal suppression coefficient is multiplied by the hierarchical offset value to obtain the temporal suppression input term. The temporal suppression coefficient is determined by fitting multiple sets of temporal hierarchical fluctuation control experiments and its value is 0.8. The temporal suppression input term is negative and then exponentially calculated to obtain the temporal suppression term. The environmental calibration perturbation value is added by one and then the reciprocal is taken to obtain the environmental suppression term. The boundary sensitivity compensation term, the temporal suppression term, and the environmental suppression term are multiplied to finally obtain the boundary sensitivity discrimination value.

[0041] The specific formula for the boundary sensitivity discriminant value is as follows: ; In the formula, Indicates the first The wounded were in the first Boundary-sensitive discriminant values ​​within each observation time window; Indicates the value of evidence compensation; This represents the grading boundary interval value, used to characterize the distance between the current basic grading result and the boundary of the adjacent grading level; This represents the boundary buffer constant, used to prevent the denominator from approaching zero when the hierarchical boundary interval value is too small; Indicates the level offset value; This represents the temporal suppression coefficient, used to control the weakening effect of short-term level fluctuations on the stability discrimination value; This represents the environmental calibration disturbance value, reflecting the degree of interference of environmental factors on the triage calibration results.

[0042] Table 1 shows the boundary sensitivity discrimination data for triage based on multimodal evidence fusion in this embodiment. The evidence compensation value for the third observation time window is 0.89, the grading boundary interval is 0.39, the grading offset is 0, and the environmental disturbance value is 0.14, resulting in a calculated boundary sensitivity discrimination value of 0.78. The evidence compensation value for the fourth observation time window is 0.54, the grading boundary interval is 0.13, the grading offset is 1, and the environmental disturbance value is 0.15, resulting in a calculated boundary sensitivity discrimination value of 0.41. The evidence compensation value for the eighth observation time window is 0.49, the grading boundary interval is 0.11, the grading offset is 1, and the environmental disturbance value is 0.21, resulting in a calculated boundary sensitivity discrimination value of 0.33. The evidence compensation value for the 13 observation time windows is 0.51, the grade boundary interval is 0.14, the grade offset is 1, and the environmental disturbance value is 0.18, resulting in a calculated boundary sensitivity discriminant value of 0.38. The evidence compensation value for the 17th observation time window is 0.61, the grade boundary interval is 0.18, the grade offset is 1, and the environmental disturbance value is 0.24, resulting in a calculated boundary sensitivity discriminant value of 0.45. The evidence compensation value for the 19th observation time window is 0.86, the grade boundary interval is 0.40, the grade offset is 0, and the environmental disturbance value is 0.27, resulting in a calculated boundary sensitivity discriminant value of 0.69.

[0043] Table 1. Sensitive discrimination data for triage boundary based on multimodal evidence fusion. like Figure 4 The image shows a bar chart illustrating the distribution of sensitive discrimination criteria for triage boundaries in patient triage based on multimodal evidence fusion, as provided in this embodiment of the application. This is combined with the data in Table 1, which contains the characteristic parameters for sensitive discrimination criteria for patient boundaries. Figure 4As can be seen, the evidence compensation values ​​for the 3rd and 19th observation time windows in Table 1 are 0.89 and 0.86, respectively; the grading boundary intervals are 0.39 and 0.40, respectively; the grade offset values ​​are both 0; and the environmental disturbance values ​​are 0.14 and 0.27, respectively. The calculated boundary sensitivity discriminant values ​​are 0.78 and 0.69, respectively, all higher than the sensitivity threshold and distributed in the area with higher discriminant values ​​in the bar chart. This corresponds to the injured person's condition being in the minor injury or recovery stable period. At this time, the quality of multimodal evidence is high, the classification boundary is clear, and the system has sufficient confidence in the grading results, so there is no need to initiate boundary stabilization intervention. On the other hand, the 4th, 8th, 13th, and 17th observation time windows correspond to the injury condition changing from minor injury to moderate injury and from moderate injury to severe injury, respectively. At the boundary stage of transition from serious injury to moderate injury, and from moderate injury to minor injury, the evidence compensation value drops to between 0.49 and 0.61, the grading boundary interval shrinks to between 0.11 and 0.18, and the grade offset value is 1, indicating a change in grading. The environmental disturbance value fluctuates between 0.15 and 0.24. The calculated boundary sensitivity discriminant value drops to between 0.33 and 0.45, all below the sensitivity threshold. This corresponds to the area in the bar chart where the discriminant value drops significantly, indicating that near the grade boundary, the quality of multimodal evidence decreases, the uncertainty of the classifier increases, and the environmental disturbance affects the system's confidence in the grading results, triggering the boundary sensitivity discriminant mechanism. Evidence compensation or time-series stability correction needs to be introduced. This distribution pattern fully verifies the coupling effect of multiple parameters in the boundary sensitivity discriminant value formula: the evidence compensation value reflects the quality of multimodal evidence and dominates the basic level of the discriminant value; the grade boundary interval characterizes the determinism of the classifier and, together with the evidence compensation value, determines the magnitude of the boundary sensitivity item; the grade offset value amplifies the impact of short-term fluctuations through exponential decay; and the environmental disturbance value introduces environmental pressure in the reciprocal form, jointly shaping the final boundary sensitivity discriminant value, thereby accurately identifying boundary unstable samples that require key attention and providing a quantitative basis for dynamic damage classification.

[0044] In this implementation scheme, a well-structured multi-layer fully connected basic hierarchical network is used to calculate the hierarchical value. The uncertainty of the hierarchical classification is quantified by indicators such as level offset and hierarchical boundary interval. Combined with the boundary buffer constant and temporal suppression coefficient determined by experimental statistical optimization, boundary sensitive compensation, temporal suppression and environmental suppression operations are integrated to obtain the boundary sensitive discrimination value. At the same time, a differentiated hierarchical value selection strategy is adopted for regions with different fidelity and compensation states. This achieves the effects of accurate and reliable injury classification, efficient identification of boundary sensitive samples, and dual suppression of temporal fluctuations and environmental interference. It effectively solves the problems of unclear hierarchical network structure, easy misjudgment of adjacent level boundary samples, and insufficient stability of the classification results caused by temporal level jumps and environmental disturbances in the existing technology.

[0045] Specifically, the steps for performing triage control and adjustment based on the boundary sensitivity discriminant analysis results and outputting the triage results are as follows: By comparing the boundary sensitivity discrimination value with the sensitivity threshold in real time, when the boundary sensitivity discrimination value is less than the sensitivity threshold, the basic classification result after environmental calibration is used as the final classification value; when the boundary sensitivity discrimination value is greater than or equal to the sensitivity threshold, it is marked as a boundary sensitive sample and enters the classification control process. The smoothing coefficient P is defined as: P = exp(-c・level offset value), where c is the smoothing attenuation coefficient. This coefficient is determined through fitting multiple sets of time-series classification fluctuation experiments, with a value range of 0.7-0.9, and a commonly used fixed value of 0.8, used to balance the stability and accuracy of the classification results and avoid classification deviations caused by fluctuations in a single parameter. For areas with sufficient visual fidelity, the basic classification result is directly used without additional adjustment; for areas with successful compensation, the classification result is fine-tuned based on the environmental threat level to ensure that the classification matches the actual injury; for areas where compensation fails, the classification level is appropriately reduced to ensure reliability by referring to historical classification data and current environmental parameters. Simultaneously, the batch flaw detection multi-source database is updated in real time, and the final classification result, boundary sensitivity discrimination value, environmental calibration parameters, and other information are stored together to achieve traceability and verification of the classification process. This process effectively avoids the limitations of a single grading standard, solves the problems of large fluctuations in grading results and inaccurate determination of boundary samples, and further achieves stable, accurate and traceable grading results in batch triage scenarios, providing a scientific basis for subsequent emergency triage and resource allocation.

[0046] In this implementation scheme, by comparing the boundary-sensitive discriminant value with the sensitivity threshold in real time, a differentiated grading decision strategy is adopted for candidate regions under different compensation states and visual fidelity states. The functional composition of the smoothing coefficient and the constraint rules for grading value jumps are clarified, and precise control is implemented for boundary-sensitive samples. This effectively suppresses abnormal grading jumps and optimizes grading stability through temporal smoothing. At the same time, the complete grading results and key parameters are synchronously stored in the database. This achieves the effects of accurate damage detection grading, reliable boundary sample identification, and dual suppression of temporal fluctuations and abnormal jumps. It effectively solves the problems of easy misjudgment of boundary regions, large fluctuations in grading results, and insufficient grading credibility and stability caused by unclear constraint rules.

[0047] Specifically, this embodiment provides a batch defect classification system based on multimodal perception and AI decision-making, applied to a batch defect classification method based on multimodal perception and AI decision-making, including: The multi-source triage data acquisition and preprocessing module is used to collect multi-source triage data containing visual, physiological, semantic, and environmental information. It sequentially performs sliding window aggregation, median filtering for noise reduction, outlier removal, and missing value interpolation preprocessing on the multi-source triage data. It also extracts visual features such as local edge continuity values, local texture abrupt change values, local pose shift values, candidate patch distribution density values, occlusion influence values, blur influence values, and distance influence values, as well as physiological anomaly support values, semantic impairment support values, and environmental calibration-related parameters. After normalizing all features to a unified numerical range, the data is associated and stored according to the casualty number, observation time window number, and candidate area location index to construct a multi-source triage database. The visual evidence fidelity assessment module is used to perform fine-grained visual evidence fidelity analysis on visual feature data corresponding to wounds, bleeding, and limb postures, calculate visual fidelity values, adaptively adjust the candidate patch retention ratio and key region pruning span based on the fidelity analysis results, perform bilinear interpolation resampling on the candidate region image patch raster, and output fidelity-limited region markers and visually fidelity-sufficient region markers. The evidence compensation module is used to perform cross-modal fine-grained evidence compensation analysis based on visual, physiological and semantic multimodal feature data of candidate regions with limited fidelity. It adopts a multi-head cross-attention mechanism with specified feature sources, calculates cross-modal offset values ​​and evidence compensation values, performs cross-modal evidence enhancement and compensation operations on regions with insufficient visual evidence according to the compensation analysis results, and outputs the labels of successfully compensated candidate regions, the labels of failed compensated candidate regions and the corresponding fused feature sequences. The boundary stability judgment module is used to perform boundary sensitivity discrimination analysis based on multi-source fusion features, evidence compensation value, grade boundary interval value, grade offset value and environmental calibration disturbance value, calculate the boundary sensitivity discrimination value, and perform grade control and adjustment operations such as grade value jump constraint and time series smoothing weighting on the damage classification results of different regions according to the boundary sensitivity discrimination analysis results. Finally, it outputs stable and reliable damage classification results and synchronizes them to the multi-source damage database.

[0048] This implementation plan, through the establishment of four major functional modules, achieves standardized collection, refined feature extraction, and standardized storage of multi-source damage detection data. It also completes adaptive assessment and optimization of visual evidence quality, multi-modal and cross-modal evidence compensation enhancement, and hierarchical control and stable output of boundary-sensitive samples. This forms a closed-loop intelligent damage detection grading system, effectively solving the problems of disordered multi-source damage detection data, difficulty in quantifying visual evidence quality, lack of effective compensation when single-modal evidence is insufficient, easy misjudgment of boundary sample grading, and large fluctuations in results. It significantly improves the objectivity, accuracy, and stability of damage detection grading.

[0049] It should be noted that, in this document, relational terms such as "first" and "second" are used only to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such process, method, article, or apparatus.

[0050] The preferred embodiments of the present invention disclosed above are merely illustrative of the invention. These preferred embodiments do not exhaustively describe all details, nor do they limit the invention to the specific implementations described. Clearly, many modifications and variations can be made based on the content of this specification. This specification selects and specifically describes these embodiments to better explain the principles and practical applications of the invention, thereby enabling those skilled in the art to better understand and utilize the invention. The invention is limited only by the claims and their full scope and equivalents.

Claims

1. A batch triage classification method based on multi-modal perception and AI decision, characterized in that, Includes the following steps: S1. Collect multi-source damage detection data, preprocess and extract features from the multi-source damage detection data, store the data, and construct a multi-source damage detection database. S2 performs fine-grained visual evidence fidelity analysis on visual feature data, performs visual feature preservation and pruning strategy adjustment operations based on the fidelity analysis results, and outputs fidelity-limited area markers. S3, performs cross-modal fine-grained evidence compensation analysis based on multimodal feature data of the fidelity-limited candidate region, and performs evidence enhancement and compensation operations based on the compensation analysis results; S4 performs boundary-sensitive discriminant analysis based on multi-source fusion features, evidence compensation values, grading boundary intervals, and temporal offsets. Based on the boundary-sensitive discriminant analysis results, it performs triage control and adjustment and outputs triage results.

2. The batch defect classification method based on multimodal perception and AI decision-making according to claim 1, characterized in that: The specific steps for collecting multi-source damage detection data, which includes visual data, physiological data, voice data, and environmental data, are as follows: Visual data acquisition: Images of wound areas, bleeding areas, limb postures, and candidate region original patch sequences of the wounded are acquired through the image acquisition unit; Physiological data acquisition: Blood oxygenation sequence, respiratory sequence, electrocardiogram sequence, and blood pressure sequence are acquired; Speech data acquisition: Speech segments describing pain, bleeding, and limited mobility are acquired through the speech acquisition unit, and semantic parsing, keyword extraction, and semantic intensity statistics are performed to obtain semantic impairment support values; Environmental data acquisition: Natural disaster intensity values, battlefield danger values, and supply-demand ratio of medical resources are acquired, and level boundary values ​​corresponding to red, yellow, green, and black injury levels are set according to the triage grading rules, and adjacent level boundary indexes corresponding to the current sample are established.

3. The batch defect classification method based on multimodal perception and AI decision-making according to claim 1, characterized in that: The specific steps for preprocessing and extracting features from multi-source flaw detection data, storing the data, and constructing a multi-source flaw detection database are as follows: The collected visual, physiological, speech, and environmental data are processed by sliding window aggregation, median filtering for noise reduction, outlier removal, and missing value interpolation to form a defect detection observation record. The record is then scaled to a uniform numerical range using the min-max normalization method. Edges are extracted from images of wounds and bleeding areas, and the connectivity ratio of eight-neighborhood pixels of candidate region edge pixels is statistically analyzed and normalized to obtain local edge continuity values. The following methods are employed to obtain local texture abrupt change values: Statistical analysis of local texture gradient changes and calculation of the mean gray-level difference; extraction of key points from limb end pose images, calculation and normalization of the mean Euclidean distance to reference pose key points to obtain local pose offset values; statistical analysis of candidate region distribution in the original patch sequence, calculation of the ratio of effective to all patches to obtain candidate patch distribution density values; segmentation of occluded regions in wound and bleeding area images, calculation of the ratio of occluded to candidate region area to obtain occlusion impact values; edge sharpness attenuation analysis of limb end pose and wound area images, calculation of the Laplacian response mean attenuation ratio to obtain blur impact values; and comparison of candidate region images with the imaging scale and reference distance, calculation of the relative deviation from the standard imaging scale to obtain distance impact values. Time alignment, denoising, and abnormal segment extraction were performed on blood oxygen, respiration, electrocardiogram, and blood pressure sequences. The physiological abnormality support value was obtained by weighting the single-dimensional abnormality amplitude. Normalization and aggregation calculations are performed on the intensity of natural disasters, battlefield hazards, and the supply-demand ratio of medical resources to generate the environmental threat level for the current observation time window. The basic classification value is input into the environmental calibration unit, and the environmental calibration classification value is obtained according to the environmental threat level and calibration rules. Then, the environmental calibration disturbance value is obtained based on the difference between the environmental calibration classification value and the basic classification value. Furthermore, the multi-source damage detection data is aligned with a unified time index to form visual, physiological, and semantic features corresponding to candidate regions. The multi-source damage detection data is stored in association according to the casualty number, observation time window number, and candidate region location index, and a batch multi-source damage detection database is constructed.

4. The batch defect classification method based on multimodal perception and AI decision-making according to claim 1, characterized in that: The specific steps for performing fine-grained visual evidence fidelity analysis on visual feature data are as follows: Obtain the local edge continuity value, local texture abruptness value, local pose offset value, candidate patch distribution density value, occlusion influence value, blur influence value and distance influence value of the i-th wounded person in the t-th observation time window; The attitude correction term is obtained by adding the local attitude offset value of the i-th wounded soldier in the t-th observation time window to a number. The attitude correction term is obtained by multiplying the local edge continuity value and the local texture abruptness value with the attitude correction term. The visual joint product term is obtained by taking the cube root of the visual joint product term. The retention density adjustment term is obtained by exponentiation of the retention density adjustment coefficient of the candidate patch distribution density value. The imaging attenuation input term is obtained by summing the occlusion, blur, and distance influence values. The imaging attenuation term is obtained by exponential operation after taking the negative value of the imaging attenuation input term. The visual fidelity value is obtained by multiplying the visual amplitude term, the retention density adjustment term, and the imaging attenuation term.

5. The batch defect classification method based on multimodal perception and AI decision-making according to claim 1, characterized in that: The specific steps for performing visual feature preservation and pruning strategy adjustments based on the fidelity analysis results, and outputting fidelity-limited region markers, are as follows: By comparing the visual fidelity value with the fidelity threshold in real time, when the visual fidelity value is less than the fidelity threshold, the retention density coefficient is increased to k to improve the retention ratio of candidate patch fragments. A candidate region hierarchical shrinkage strategy is adopted to shrink the pruning span corresponding to the patch fragments at the tear edge, bleeding edge, and end pose position to the original pruning span, respectively. times, Double The candidate region image Patch grid is resampled using bilinear interpolation and the visual fidelity value is recalculated. If the visual fidelity value is still less than the fidelity threshold, the current candidate region is marked as a fidelity-restricted candidate region. When the visual fidelity value is greater than or equal to the fidelity threshold, it is marked as a region with sufficient visual fidelity, and the current compressed token sequence and visual fidelity value are output to the boundary stability judgment module.

6. The batch defect classification method based on multimodal perception and AI decision-making according to claim 1, characterized in that: The specific steps for cross-modal fine-grained evidence compensation analysis based on multimodal feature data of fidelity-constrained candidate regions are as follows: Obtain the visual fidelity value, physiological abnormality support value, and semantic impairment support value of the i-th injured person in the t-th observation time window in the fidelity-limited candidate region. Perform time alignment, association mapping, and difference measurement on visual changes, physiological changes, and semantic changes under a unified time index. Based on the construction of a cross-modal evidence association model using a multi-head cross-attention mechanism, output the cross-modal offset value. Multiply the physiological abnormality support value by the semantic impairment support value to obtain the support joint term; add the support joint term to one to obtain the logarithmic input term; perform natural logarithmic operation on the logarithmic input term to obtain the support gain term; add the support gain term to one to obtain the compensation amplification term; square the cross-modal offset value to obtain the offset squared term; add the offset squared term to one to obtain the offset suppression term; multiply the visual fidelity value, the compensation amplification term, and the offset suppression term to obtain the evidence compensation value.

7. The batch defect classification method based on multimodal perception and AI decision-making according to claim 1, characterized in that: The specific steps for performing evidence enhancement and compensation operations based on the compensation analysis results are as follows: By comparing the evidence compensation value with the compensation threshold in real time, when the evidence compensation value is less than the compensation threshold, the current candidate region is marked as a compensation failure candidate region, the basic fusion feature sequence is retained, and it is archived to the multi-source database of damage detection. When the evidence compensation value is greater than or equal to the compensation threshold, the candidate region visual token sequence, physiological abnormality support value, semantic damage support value, evidence compensation value and candidate region location index are used as inputs. A fine-grained evidence enhancement model is constructed using a gated attention algorithm, and the compensation enhancement fusion feature sequence is output. The compensation enhancement fusion feature sequence and the evidence compensation value are then output to the boundary stability judgment module.

8. The batch defect classification method based on multimodal perception and AI decision-making according to claim 1, characterized in that: The specific steps for boundary-sensitive discriminant analysis based on multi-source fusion features, evidence compensation values, hierarchical boundary intervals, and temporal offsets are as follows: Obtain the visual fidelity value and basic fusion feature sequence corresponding to the visually fidelity-sufficient region, the label of the successful compensation candidate region, the compensation-enhanced fusion feature sequence, the evidence compensation value, the candidate region location index, as well as the label of the compensation-failed candidate region, the basic fusion feature sequence, and the evidence compensation value; The basic fusion feature sequence of the visually fidelity-sufficient region is input into the basic hierarchical network. The basic hierarchical network adopts a multi-layer fully connected discriminant structure, performs feature concatenation, linear transformation, and nonlinear activation calculation on visual features, physiological features, and semantic features, outputs a basic hierarchical probability vector, and generates a basic hierarchical value based on the highest probability category. The difference between the basic hierarchical value of the previous and current observation time windows is calculated to obtain the level offset value. The hierarchical boundary interval value is obtained based on the difference between the scores of the highest and second-highest probability categories in the defect detection level. The compensated enhanced fusion feature sequence and the basic fusion feature sequence are input into the basic hierarchical network respectively, and the compensated correction hierarchical value and the failure backoff hierarchical value are output. The corresponding hierarchical value is selected according to the sample source region to obtain the current hierarchical value. Obtain the graded boundary interval value, grade offset value, and environmental calibration disturbance value. Add the graded boundary interval value to the boundary buffer constant to obtain the boundary correction term. Divide the evidence compensation value by the boundary correction term to obtain the boundary sensitivity compensation term. Multiply the time series suppression coefficient by the grade offset value to obtain the time series suppression input term. Take the negative value of the time series suppression input term and perform an exponential operation to obtain the time series suppression term. The environmental suppression term is obtained by adding one to the environmental calibration disturbance value and taking the reciprocal. The boundary sensitivity compensation term, the temporal suppression term, and the environmental suppression term are multiplied together to obtain the boundary sensitivity discrimination value.

9. The batch defect classification method based on multimodal perception and AI decision-making according to claim 1, characterized in that: The specific steps for performing triage control and adjustment based on the boundary sensitivity discriminant analysis results and outputting the triage results are as follows: By comparing the boundary sensitivity discrimination value with the sensitivity threshold in real time, when the boundary sensitivity discrimination value is less than the sensitivity threshold, the environmental calibration classification value is determined as the final damage classification value. When the boundary sensitivity discrimination value is greater than or equal to the sensitivity threshold, it is marked as a boundary sensitive sample and enters the defect grading control process: For candidate areas with successful compensation, the environmental calibration grading value is used as the final defect grading value; for candidate areas with failed compensation, the calibration grading value is constrained by the grade offset value to perform a jump constraint. If the grade offset value is greater than the offset threshold, it is judged as an abnormal jump, and the current basic grading value is forced to revert to the final grading value of the previous observation time window; if the grade offset value is less than or equal to the offset threshold, the environmental calibration grading value is kept unchanged as the final defect grading value; for areas with sufficient visual fidelity, the smoothing coefficient P is determined by the negative exponential decay relationship between the visual fidelity value and the grade offset value, the adjacent window grading probability vector is weighted and averaged, and the threshold is adjusted in combination with the environmental threat level before the final defect grading value is output; the final defect grading value, boundary sensitive sample marking, evidence compensation value, and grading boundary interval value are output together to the batch defect detection multi-source database.

10. A batch damage classification system based on multimodal perception and AI decision-making, employing a batch damage classification method based on multimodal perception and AI decision-making as described in any one of claims 1-9, characterized in that, include: The multi-source data acquisition and preprocessing module for flaw detection is used to acquire multi-source flaw detection data, preprocess and extract features from the multi-source flaw detection data, store the data, and build a multi-source flaw detection database. The visual evidence fidelity assessment module is used to perform fine-grained visual evidence fidelity analysis on visual feature data, perform visual feature preservation and pruning strategy adjustments based on the fidelity analysis results, and output fidelity-limited area markers. The evidence compensation module is used to perform cross-modal fine-grained evidence compensation analysis based on multimodal feature data of candidate regions with limited fidelity, and to perform evidence enhancement and compensation operations based on the compensation analysis results. The boundary stability judgment module is used to perform boundary sensitivity discrimination analysis based on multi-source fusion features, evidence compensation value, hierarchical boundary interval and temporal offset. Based on the boundary sensitivity discrimination analysis results, it performs triage control and adjustment and outputs triage results.

Images