Battlefield personnel injury condition intelligent discrimination method based on multi-source information fusion

Abstract

The invention belongs to the technical field of intelligent perception, and particularly discloses a battlefield personnel injury intelligent judgment method based on multi-source information fusion. The method comprises the following steps: firstly, acquiring image data and radar data in the same scene and at the same moment through time-space synchronization; inputting the image data into a wounded detection model to obtain a bounding box of the first target and a first detection result of the first target; the position and vital signs of a second target are extracted from the radar data, and a second detection result of the second target is judged based on the vital signs; performing target association on the first target and the second target based on the position of the second target and the bounding box of the first target; and finally fusing the first and second detection results of the associated target. According to the method, through a target correlation fusion image detection technology and a radar detection technology, the identification precision of the wounded person in complex environments such as multi-shielding and multi-interference is improved.

Description

Technical Field

[0001] This application belongs to the field of intelligent sensing technology, and more specifically, relates to an intelligent method for battlefield personnel injury assessment based on multi-source information fusion. Background Technology

[0002] In modern battlefield and disaster relief environments, quickly and accurately locating the wounded and assessing their injury severity is crucial for improving rescue efficiency and survival rates. Casualty identification and injury assessment are key aspects of battlefield medical rescue and disaster emergency response; the core issue lies in how to quickly and accurately pinpoint the location of the wounded and determine their injuries in complex environments.

[0003] Most existing methods rely on visual inspection or single-sensor approaches, such as using photoelectric imaging equipment for casualty identification or employing life-detecting radar to detect vital signs like breathing and heartbeat. However, these methods have significant limitations: photoelectric equipment is prone to failure in smoke, rain, fog, at night, or in low-visibility conditions, leading to decreased identification accuracy; while radar can mitigate the effects of harsh natural conditions to some extent, it is susceptible to interference from complex electromagnetic environments and ambient noise, resulting in misjudgments. Furthermore, most existing methods can only roughly determine whether a person is injured, lacking the ability to further assess the severity of injuries, making it difficult to meet the needs of priority dispatching and rational allocation of resources in complex environments. Therefore, there is an urgent need for a method that can accurately identify the severity of injuries in complex environments. Summary of the Invention

[0004] To address the shortcomings of existing technologies, the purpose of this application is to provide an intelligent method for battlefield personnel injury assessment based on multi-source information fusion, aiming to solve the technical problem of insufficient accuracy in existing casualty identification methods.

[0005] The first aspect of this application relates to an intelligent method for battlefield personnel injury assessment based on multi-source information fusion, including: Image and radar data of the same scene at the same time are acquired through spatiotemporal synchronization. The image data is input into the casualty detection model to obtain the bounding box of the first target and the first detection result of whether the first target is a casualty; the location and vital signs of the second target are extracted from the radar data, and the second detection result of whether the second target is a casualty is further determined based on the vital signs. The position of the second target is projected onto the coordinate system of the bounding box of the first target using a transformation matrix to obtain the projected position of the second target; the transformation matrix is ​​obtained in advance through joint calibration of the image acquisition device and the radar; If the bounding box of the first target covers the projected position of the second target, then the first target and the second target are associated as the same target; The detection results for the same target are obtained by combining the first detection result and the second detection result.

[0006] Preferably, the casualty detection model is trained based on the YOLOv5s network model, wherein a combination structure of spatial-to-depth layers plus 1×1 convolutional layers is used to replace the original strided convolutional layers in the YOLOv5s backbone network and neck network. The spatial-to-depth layer is used to divide the input feature map into four local spatial regions using a 2×2 grid, and then convert the four local spatial regions from spatial dimension to channel dimension to achieve spatial dimensionality reduction without information loss. The 1×1 convolutional layer is used to compress the channel dimension to one-quarter using a 1×1 convolutional kernel.

[0007] Preferably, the casualty detection model is trained based on a YOLOv5s network model, wherein a dual-branch decoupling head replaces the original coupled detection head in the YOLOv5s network model. The structure of the dual-branch decoupling head is as follows: The independent classification branch includes: 3 layers of 3×3 convolutional layers, batch normalization layers, SiLU activation layers, 1 layer of 1×1 convolutional layers, and Softmax activation layers, and independently uses the focus loss function to focus on learning category features; The independent regression branch includes: three 3×3 convolutional layers, a batch normalization layer, a SiLU activation layer, a 1×1 convolutional layer, and a Sigmoid activation layer. It independently employs the full intersection-over-union loss function to focus on learning the location features and confidence features of the bounding box.

[0008] Preferably, the casualty detection model is trained based on a YOLOv5s network model, wherein optimized anchor boxes replace the original general anchor boxes in the YOLOv5s network model. The optimized anchor boxes are obtained through the following method: The pixel dimensions of the bounding boxes of all wounded soldiers in the battlefield real-scene image set are statistically analyzed to obtain a size set; The size set was clustered using the K-means++ clustering algorithm to obtain 9 cluster centers as the optimized anchor boxes; The nine optimized anchor frames are divided into three groups according to their area, and the three groups are respectively assigned to the three detection layers of the YOLOv5s network model.

[0009] Preferably, the vital signs of the second target are dynamically adjusted based on the environment in which the second target is located, specifically as follows: ; ; in, It is the adjusted breathing rate. It is the adjusted heart rate. It is the breathing rate before adjustment. This is the heart rate before adjustment. It is the altitude of the second target. It is the ambient temperature of the second target. It is the respiratory regulation coefficient. It is the heart rate regulation coefficient.

[0010] Preferably, image data and radar data of the same scene at the same time are acquired through spatiotemporal synchronization, specifically by: rigidly connecting the radar and the image acquisition device to ensure that the beam center axis of the radar and the optical axis of the image acquisition device are parallel, and maximizing the overlap area of ​​the field of view of the radar and the image acquisition device; and synchronously acquiring image data and radar data of the same scene at the same time through the radar and the image acquisition device.

[0011] Preferably, the location and vital signs of the second target are extracted from the radar data, specifically as follows: The radar data is subjected to a Fourier transform along the fast time dimension to generate a two-dimensional data matrix containing range and slow time information; Spectral analysis is performed on the slow time series of each distance unit in the two-dimensional data matrix to detect whether there are significant spectral peaks in the preset vital signs frequency band. The distance units with significant spectral peaks are marked as distance units of interest, and the distance corresponding to the distance unit of interest is the radial distance of the second target. For the data received by the multi-channel receiving array on the range of interest cell, the angle of arrival is estimated by beamforming or angle-dimensional Fourier transform to obtain the azimuth angle of the second target. The position of the second target can then be determined by combining the radial distance. The phase change signal is extracted from the spatial unit where the second target is located, and the breathing signal and heartbeat signal are decoupled from the phase change signal using signal processing technology; The respiratory signal and heartbeat signal are respectively subjected to power spectral density estimation. The respiratory rate and heart rate of the second target can be obtained by detecting the position of the main peak of their respective spectra.

[0012] Preferably, the second detection result for further determining whether the second target is an injured person based on the vital signs is as follows: if the respiratory rate of the second target is greater than the respiratory abnormality threshold and the heart rate is greater than the heart rate abnormality threshold, then the second detection result is an injured person; otherwise, it is not an injured person.

[0013] Preferably, the transformation matrix is ​​obtained in advance through joint calibration of the image acquisition device and the radar, specifically as follows: Fix a corner reflector in the center of the chessboard calibration plate; The chessboard calibration board is moved to multiple different poses, and at each pose, the image acquisition device and radar synchronously acquire image data and radar data of the chessboard calibration board. For each pose, the two-dimensional coordinates of all corner points in the checkerboard calibration board are identified from the image data, and the two-dimensional coordinates of the corner reflector are further derived from the relative position of the corner reflector on the checkerboard calibration board; the three-dimensional coordinates of the corner reflector are identified from the radar data. Thus, multiple sets of three-dimensional coordinate-two-dimensional coordinate pairs of the corner reflector are obtained through multiple poses; The transformation matrix is ​​obtained by solving multiple sets of the three-dimensional coordinate-two-dimensional coordinate pairs.

[0014] Preferred options also include: Based on the aforementioned vital signs, the severity of injury is assessed: If satisfied If so, it is determined to be endangered; If satisfied If so, it is determined to be a serious injury; If satisfied If so, it is determined to be a moderate injury; If satisfied If so, it is determined to be a minor injury; If satisfied If so, it is determined to be death. in, Respiratory rate, Heart rate.

[0015] Overall, the technical solutions conceived in this application have the following beneficial effects compared with the prior art: (1) This application proposes a method for identifying injuries by multi-source information fusion. It uses spatiotemporal synchronization and joint calibration to fuse image data and radar data to detect injuries to people in complex environments. It comprehensively determines whether a person is injured by combining the detection results based on vital signs and the detection results of the image detection model. This avoids the problems that image data is easily obscured by complex scenes and natural conditions, and radar data is easily interfered with by electromagnetic environment, thus improving the accuracy of injury identification.

[0016] (2) This application proposes an image detection model for wounded personnel detection. The image detection model is based on the YOLOv5 network model, in which the YOLOv5 network structure is optimized. The original strided convolutional layer in the YOLOv5 backbone network and neck network is replaced by a combination of spatial-to-depth layer and 1×1 convolutional layer (SPD-Conv) structure. The original strided convolutional layer uses stride=2 strided convolution, which directly discards 50% of the pixels after convolution, resulting in the loss of edge and contour features of small-scale wounded personnel. The SPD-Conv structure adopted in this application retains all details through "spatial-to-depth" and then performs channel dimensionality reduction through convolution, which not only meets the downsampling requirements but also solves the problem of loss of small target features.

[0017] (3) This application proposes an image detection model for wounded soldier detection. The image detection model is based on the YOLOv5 network model, in which the YOLOv5 network structure is optimized and a dual-branch decoupling head is used to replace the original coupled detection head in the YOLOv5s network model. In the coupled detection head, the optimization objectives of the classification task and the regression task interfere with each other (e.g., the classification features of partially occluded wounded soldiers are strong but the localization features are weak, resulting in "accurate classification but biased localization"). The dual-branch decoupling head of this application is trained through independent branches and independent loss functions, so that the two tasks can reach their respective optimal values. It is especially suitable for wounded soldier identification with partial occlusion and pose distortion in the battlefield.

[0018] (4) This application proposes an image detection model for wounded personnel detection. This image detection model is based on the YOLOv5 network model, in which optimized anchor boxes are used to replace the general anchor boxes in the YOLOv5s network model. The general anchor boxes are suitable for general targets such as pedestrians and vehicles, but have poor matching degree for wounded personnel in the battlefield environment. In this application, the anchor boxes are optimized for wounded personnel targets in the battlefield environment, so that they have a better matching degree for wounded personnel in the battlefield environment.

[0019] (5) Based on the identification of wounded, this application proposes a wound severity classification model based on vital signs. This model can effectively classify the wounded based on vital signs and is more suitable for the needs of priority scheduling and rational allocation of resources in battlefield rescue.

[0020] (6) This application proposes a dynamic compensation mechanism for vital signs, which compensates the target’s respiratory rate and heart rate based on the target’s environment, and then judges the injury status of the target, thereby further improving the accuracy of injury identification. Attached Figure Description

[0021] Figure 1 This is a flowchart of the intelligent battlefield personnel injury assessment method based on multi-source information fusion provided in the embodiments of this application.

[0022] Figure 2 This is a schematic diagram of the injury detection model structure provided in the embodiments of this application.

[0023] Figure 3 This is a schematic diagram of the injury assessment process provided in the embodiments of this application.

[0024] Figure 4 This is a schematic diagram of the composition of an electronic device provided in an embodiment of this application. Detailed Implementation

[0025] To make the objectives, technical solutions, and advantages of this application clearer, the following detailed description is provided in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the scope of this application.

[0026] In this application, the terms "first" and "second," etc., are used to distinguish different objects, not to describe a specific order of objects. For example, "first response message" and "second response message," etc., are used to distinguish different response messages, not to describe a specific order of response messages.

[0027] In this application, the term "electrical connection" can refer to a direct circuit connection or a signal transmission via a communication protocol.

[0028] In the embodiments of this application, the terms "exemplary" or "for example" are used to indicate that something is an example, illustration, or description. Any embodiment or design that is described as "exemplary" or "for example" in the embodiments of this application should not be construed as being more preferred or advantageous than other embodiments or design. Specifically, the use of the terms "exemplary" or "for example" is intended to present the relevant concepts in a specific manner.

[0029] In the description of the embodiments of this application, unless otherwise stated, "multiple" means two or more, for example, multiple processing units means two or more processing units, multiple elements means two or more elements, etc.

[0030] First, the technical terms involved in the embodiments of this application will be introduced.

[0031] The YOLOv5s network model is a lightweight model in the YOLOv5 series, suitable for scenarios with limited computing resources. It maintains fast inference speed while ensuring detection accuracy, achieving efficient single-stage object detection by optimizing network depth and width, making it suitable for deployment on mobile or embedded devices.

[0032] The K-means++ clustering algorithm is an improvement on the K-means clustering algorithm, with its core focus on optimizing the selection of initial cluster centers. It first randomly selects a center point, and the probability of selecting each subsequent new center point is proportional to the square of its shortest distance to existing centers. This method effectively disperses the initial center points, significantly reducing the number of iterations and helping the algorithm obtain better and more stable clustering results.

[0033] The embodiments of this application are described below with reference to the accompanying drawings.

[0034] This embodiment introduces an intelligent method for assessing battlefield personnel injuries based on multi-source information fusion. This embodiment uses image data and radar data collected by an optoelectronic pod and a life detection radar, respectively, to assess personnel injuries in a battlefield environment.

[0035] The electro-optical pod and life detection radar can be installed on mobile vehicles such as search and rescue helicopters, drones, and vehicles. The mobile vehicle carries them to collect data on the target area. Before data collection, the electro-optical pod and life detection radar need to be synchronized in time and space and jointly calibrated. Spatiotemporal synchronization: The photoelectric pod is rigidly connected to the life detection radar to ensure that the beam center axis of the life detection radar and the optical axis of the photoelectric pod are parallel, and the field of view overlap area of ​​the life detection radar and the photoelectric pod is maximized; image data and radar data of the same scene at the same time are acquired synchronously by the life detection radar and the photoelectric pod.

[0036] Joint calibration: Fix a corner reflector in the center of the chessboard calibration plate; The chessboard calibration board is moved to multiple different poses, and in each pose, the life detection radar and the optoelectronic pod are used to simultaneously collect image data and radar data of the chessboard calibration board. For each pose, the two-dimensional coordinates of all corner points in the checkerboard calibration board are identified from the image data, and the two-dimensional coordinates of the corner reflector are further derived from the relative position of the corner reflector on the checkerboard calibration board; the three-dimensional coordinates of the corner reflector are identified from the radar data. Thus, multiple sets of three-dimensional coordinate-two-dimensional coordinate pairs of the corner reflector are obtained through multiple poses; The transformation matrix of life detection radar to photoelectric pod is obtained by solving multiple sets of three-dimensional coordinate-two-dimensional coordinate pairs.

[0037] like Figure 1 As shown, a method for intelligent assessment of battlefield personnel injuries based on multi-source information fusion specifically includes the following steps: S1. Acquire image and radar data of the same scene at the same time through spatiotemporal synchronization.

[0038] S2. Input the image data into the casualty detection model to obtain the bounding box of the first target and the first detection result of whether the first target is a casualty. The casualty detection model is trained based on a YOLOv5s network model. See [link to YOLOv5s network model results] for details. Figure 2 As shown.

[0039] S2.1, Constructing a dataset for the battlefield environment: Existing YOLOv5 casualty recognition technologies mostly use general public datasets (such as COCO subsets) or simple scene datasets, without considering the specificity of the environment, resulting in poor model generalization ability.

[0040] (1) Constructing a dataset: Collect 10,000+ real-world battlefield images (including smoke, rain, fog, and low-visibility nighttime scenes), covering different postures of the wounded (falling, curled up, partially obscured by equipment / vegetation) and different scales (clear at close range, small targets at long range). Use bounding boxes to accurately select the wounded's body area (avoid including too much background), and finally form a training set of 8,000 images and a validation set of 2,000 images.

[0041] (2) Pretreatment: Existing technologies often employ enhancement methods such as large-scale cropping and blurring, leading to distortion of small-scale wounded features. The enhancement strategy in this application can preserve the detailed features of wounded as much as possible. Lightweight data augmentation: employs random horizontal flipping (probability 0.5), brightness / contrast fine-tuning (±10%), and small-range random cropping (cropping ratio 0.8-1.0) to avoid excessive blurring of small target details (adapting to the feature preservation requirements of SPD-Conv). Unified format: scale the image to 640×640, normalize to [0,1], and convert the channels to BGR.

[0042] Thus, the dataset needed to train the casualty detection model is obtained through the above two steps.

[0043] S2.2 Optimize anchor frames for battlefield environments Existing YOLOv5 casualty detection technology directly uses the generic anchor boxes of the original YOLOv5s (designed for the COCO dataset, adapted to common targets such as pedestrians and vehicles). However, the aspect ratio of battlefield casualty targets (often "narrow and long" or "wide and short" when down) differs greatly from that of generic targets, resulting in low anchor box matching accuracy and a high rate of missed detections for small targets. The anchor box optimization in this application is based on a customized design using the casualty dataset, and the fusion process is as follows: (1) Calculate the pixel size of the bounding boxes of all wounded soldiers in the battlefield real-scene image set to obtain the size set.

[0044] (2) The size set is clustered using the K-means++ clustering algorithm to obtain 9 cluster centers as the optimized anchor boxes.

[0045] Verify the optimization effect: The aspect ratio (w / h) of the bounding boxes of all wounded soldiers in the battlefield real-scene image set was statistically analyzed, and the distribution range was found to be 0.3-2.5, with the core concentration in the range of 0.5-1.5 (typical aspect ratio of wounded soldiers on the ground). Statistical analysis of the aspect ratio (w / h) of the optimized anchor frames revealed that the distribution range of the aspect ratio of the optimized anchor frames matches the distribution range of the aspect ratio of all the bounding boxes labeled with injuries in the real-scene image set.

[0046] (3) Divide the nine optimized anchor frames into three groups according to their area, and assign the three groups to the three detection layers of the YOLOv5s network model respectively.

[0047] (4) Write the optimized anchor boxes into the YOLOv5s model network configuration file (replacing the original YOLOv5s anchor box parameters). During training, the model's detection branch learns directly based on these optimized anchor boxes, thereby improving the target matching accuracy.

[0048] S2.3 Optimize the structure of the network model for the battlefield environment S2.3.1 Replace the original strided convolutional layers (two strided convolutional layers in the backbone network and one downsampling layer in the neck network) in the YOLOv5s backbone network and neck network with a combined structure of spatial to depth layers plus 1×1 convolutional layers (SPD-Conv). The spatial-to-depth layer is used to divide the input feature map into four local spatial regions using a 2×2 grid, and then convert the four local spatial regions from spatial dimension to channel dimension (e.g., an 80×80×64 feature map is converted into a 40×40×256 feature map), achieving spatial dimensionality reduction without information loss (preserving all pixel details without information loss). The 1×1 convolutional layer is used to compress the channel dimension to one-quarter using a 1×1 convolutional kernel (the 40×40×256 feature map is transformed into a 40×40×64 feature map), thus completing the downsampling.

[0049] Compared with existing technologies: The original YOLOv5s uses stride=2 stride convolution, which directly discards 50% of the pixels, resulting in the loss of edge and contour features of small-scale wounded people; while SPD-Conv retains all details by "spatial to depth" and then reduces dimensions through convolution, which not only meets the downsampling requirements but also solves the problem of loss of features of small targets.

[0050] S2.3.2 A dual-branch decoupling head replaces the original coupling detection head in the YOLOv5s network model. The structure of the dual-branch decoupling head is as follows: Independent classification branch: 3 layers of 3×3 convolutional layers (Conv) + batch normalization layer (BN) + SiLU activation layer + 1 layer of 1×1 convolutional layer (Conv) + Softmax activation layer, independently using the focal loss function to focus on learning category (injured / uninjured) features; Independent regression branch: 3 layers of 3×3 convolutional layers (Conv) + batch normalization layer (BN) + SiLU activation layer + 1 layer of 1×1 convolutional layer (Conv) + Sigmoid activation layer, independently using the complete intersection-union loss function (CIoU Loss) to focus on learning the location features and confidence features of the bounding box.

[0051] Compared with existing technologies: In the original coupled detection head, the optimization objectives of the classification task and the regression task interfere with each other (e.g., the classification features of partially occluded wounded soldiers are strong but the localization features are weak, resulting in "accurate classification but biased localization"); the decoupled head is trained through independent branches and independent loss functions, so that the two tasks can reach their respective optimal levels, which is especially suitable for wounded soldier identification with partial occlusion and distorted posture in the battlefield.

[0052] S3. Extract the location and vital signs of the second target from the radar data, and further determine whether the second target is a wounded person based on the vital signs; S3.1. Extract the location and vital signs of the second target from the radar data, specifically as follows: The echo signal received by the radar is subjected to Fourier transform along the fast time (range dimension) to obtain a two-dimensional data matrix containing range and slow time information (i.e., data before the range-Doppler domain).

[0053] For each distance cell in the two-dimensional matrix, its complex slow time series is taken. Spectral analysis (such as Fourier transform) is performed on this series to detect whether there are significant spectral peaks within a preset vital sign frequency band (e.g., 0.1-0.8 Hz for respiration, 0.8-2.0 Hz for heartbeat). The distance cells with significant spectral peaks are marked as the distance cells of interest, and their corresponding distances are the radial distances to the second target.

[0054] For a given range cell of interest, the angle of arrival of the second target is estimated using data from a multi-channel receiving array on that range cell through beamforming (spatial filtering) or angle-dimensional Fourier transform (such as FFT). Combined with the obtained radial range, the position (range, angle) of the second target in the radar polar coordinate system can then be determined.

[0055] From the spatial unit where the target is located, extract the phase (or phase change of the baseband signal) of its complex slow time series. Since the chest cavity micro-motion modulation caused by breathing and heartbeat is in the phase, signal processing techniques (such as algorithms based on variational mode decomposition, or bandpass filtering combined with adaptive filtering, etc.) are needed to decouple the mixed phase change signals and separate the independent breathing signal and heartbeat signal components.

[0056] Power spectral density was estimated for the decoupled respiratory and heartbeat signals separately. By detecting the main peak positions of their respective power spectra within the vital signs frequency band, the respiratory rate of the second target could be estimated. and heart rate .

[0057] For the procedures on managing vital signs, please refer to [link / reference]. Figure 3 .

[0058] S3.2. Dynamically adjust the vital signs of the second target based on the environment in which the second target is located, specifically as follows: ; ; in, It is the adjusted breathing rate. It is the adjusted heart rate. It is the breathing rate before adjustment. This is the heart rate before adjustment. It is the altitude of the second target. It is the ambient temperature of the second target. It is the respiratory regulation coefficient. It is the heart rate regulation coefficient.

[0059] S3.3, Based on the vital signs, further determine whether the second target is a wounded person (second detection result), specifically as follows: Breath test result initialization

[0060] ; Heart rate monitoring result initialization

[0061] ; in, The threshold for abnormal breathing. This is the threshold for abnormal heart rate.

[0062] The second detection result is determined based on the following formula: ; like If the second test result is positive, then the patient is injured.

[0063] like If the second test result is negative, then the patient is not injured.

[0064] In the formula, Represents logical OR, This indicates a logical AND operation.

[0065] S4. Project the position of the second target onto the coordinate system containing the bounding box of the first target using a transformation matrix to obtain the projected position of the second target. If the bounding box of the first target covers the projected position of the second target, then associate the first target and the second target as the same target.

[0066] The transformation matrix is ​​obtained in advance through joint calibration of the life detection radar and the electro-optical pod.

[0067] In this step, it is found that the projection position of the second target is within the bounding box of the first target. Therefore, the first target and the second target are considered to be the same target, and the first detection result of the first target and the second detection result of the second target can be fused.

[0068] S5. The first detection result and the second detection result are fused to obtain the detection result of the same target. The specific fusion method is as follows:

[0069] in, For the detection results of the same target after fusion, The first test result is (1 for the injured and 0 for the non-injured). This represents the second test result (1 for the injured, 0 for the uninjured). In the formula, Represents logical OR, This indicates a logical AND operation.

[0070] S6. After the test results determine that the same target is a wounded person, further assessment of the injury is made based on vital signs: If satisfied If so, it is determined to be endangered; If satisfied If so, it is determined to be a serious injury; If satisfied If so, it is determined to be a moderate injury; If satisfied If so, it is determined to be a minor injury; If satisfied If so, it is determined to be death. in, The adjusted respiratory rate, This is the adjusted heart rate.

[0071] Based on the methods in the above embodiments, this application provides an electronic device, such as... Figure 4 As shown, the electronic device may include a processor, a communications interface, a memory, and a communication bus, wherein the processor, communications interface, and memory communicate with each other via the communication bus. The processor can invoke logical instructions stored in the memory to execute the methods described in the above embodiments.

[0072] Furthermore, the logical instructions in the aforementioned memory can be implemented as software functional units and, when sold or used as independent products, can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or a portion of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of this application.

[0073] Based on the methods in the above embodiments, this application provides a computer-readable storage medium storing a computer program that, when run on a processor, causes the processor to execute the methods in the above embodiments.

[0074] Based on the methods in the above embodiments, this application provides a computer program product that, when run on a processor, causes the processor to execute the methods in the above embodiments.

[0075] It is understood that the processor in the embodiments of this application can be a central processing unit (CPU), or other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, transistor logic devices, hardware components, or any combination thereof. A general-purpose processor can be a microprocessor or any conventional processor.

[0076] The method steps in this application embodiment can be implemented in hardware or by a processor executing software instructions. The software instructions can consist of corresponding software modules, which can be stored in random access memory (RAM), flash memory, read-only memory (ROM), programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), registers, hard disks, portable hard disks, CD-ROMs, or any other form of storage medium known in the art. An exemplary storage medium is coupled to the processor, enabling the processor to read information from and write information to the storage medium. Of course, the storage medium can also be a component of the processor. The processor and the storage medium can reside in an ASIC.

[0077] In the above embodiments, implementation can be achieved entirely or partially through software, hardware, firmware, or any combination thereof. When implemented using software, it can be implemented entirely or partially as a computer program product. The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, all or part of the processes or functions described in the embodiments of this application are generated. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions can be stored in a computer-readable storage medium or transmitted through the computer-readable storage medium. The computer instructions can be transmitted from one website, computer, server, or data center to another website, computer, server, or data center via wired (e.g., coaxial cable, fiber optic, digital subscriber line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) means. The computer-readable storage medium can be any available medium that a computer can access or a data storage device such as a server or data center that integrates one or more available media. The available medium can be a magnetic medium (e.g., floppy disk, hard disk, magnetic tape), an optical medium (e.g., DVD), or a semiconductor medium (e.g., solid-state disk (SSD)).

[0078] It is understood that the various numerical designations used in the embodiments of this application are merely for the convenience of description and are not intended to limit the scope of the embodiments of this application.

[0079] Those skilled in the art will readily understand that the above description is merely a preferred embodiment of this application and is not intended to limit this application. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of this application should be included within the scope of protection of this application.

Claims

1. A method for intelligent assessment of battlefield personnel injuries based on multi-source information fusion, characterized in that, include: Image and radar data of the same scene at the same time are acquired through spatiotemporal synchronization. The image data is input into the casualty detection model to obtain the bounding box of the first target and the first detection result of whether the first target is a casualty; The location and vital signs of the second target are extracted from the radar data, and based on the vital signs, a second detection result is made to determine whether the second target is a wounded person. The position of the second target is projected onto the coordinate system of the bounding box of the first target using a transformation matrix to obtain the projected position of the second target; The transformation matrix is ​​obtained in advance through joint calibration of the image acquisition device and the radar; If the bounding box of the first target covers the projected position of the second target, then the first target and the second target are associated as the same target; The detection results for the same target are obtained by combining the first detection result and the second detection result.

2. The intelligent battlefield personnel injury assessment method according to claim 1, characterized in that, The casualty detection model is trained based on the YOLOv5s network model, in which a combination structure of spatial-to-depth layers plus 1×1 convolutional layers is used to replace the original strided convolutional layers in the YOLOv5s backbone network and neck network. The spatial-to-depth layer is used to divide the input feature map into four local spatial regions using a 2×2 grid, and then convert the four local spatial regions from spatial dimension to channel dimension to achieve spatial dimensionality reduction without information loss. The 1×1 convolutional layer is used to compress the channel dimension to one-quarter using a 1×1 convolutional kernel.

3. The intelligent battlefield personnel injury assessment method according to claim 1, characterized in that, The casualty detection model is trained based on a YOLOv5s network model. A dual-branch decoupling head replaces the original coupled detection head in the YOLOv5s network model. The structure of the dual-branch decoupling head is as follows: The independent classification branch includes: 3 layers of 3×3 convolutional layers, batch normalization layers, SiLU activation layers, 1 layer of 1×1 convolutional layers, and Softmax activation layers, and independently uses the focus loss function to focus on learning category features; The independent regression branch includes: three 3×3 convolutional layers, a batch normalization layer, a SiLU activation layer, a 1×1 convolutional layer, and a Sigmoid activation layer. It independently employs the full intersection-over-union loss function to focus on learning the location features and confidence features of the bounding box.

4. The intelligent battlefield personnel injury assessment method according to claim 1, characterized in that, The casualty detection model is trained based on a YOLOv5s network model, wherein optimized anchor boxes are used to replace the original general anchor boxes in the YOLOv5s network model. The optimized anchor boxes are obtained through the following method: The pixel dimensions of the bounding boxes of all wounded soldiers in the battlefield real-scene image set are statistically analyzed to obtain a size set; The size set was clustered using the K-means++ clustering algorithm to obtain 9 cluster centers as the optimized anchor boxes; The nine optimized anchor frames are divided into three groups according to their area, and the three groups are respectively assigned to the three detection layers of the YOLOv5s network model.

5. The intelligent battlefield personnel injury assessment method according to claim 1, characterized in that, The vital signs of the second target are dynamically adjusted based on the environment in which the second target is located, specifically as follows: ； ； in, It is the adjusted breathing rate. It is the adjusted heart rate. It is the breathing rate before adjustment. This is the heart rate before adjustment. It is the altitude of the second target. It is the ambient temperature of the second target. It is the respiratory regulation coefficient. It is the heart rate regulation coefficient.

6. The intelligent battlefield personnel injury assessment method according to claim 1, characterized in that, To acquire image and radar data of the same scene at the same time through spatiotemporal synchronization, specifically: rigidly connect the radar and the image acquisition device to ensure that the radar beam center axis and the optical axis of the image acquisition device are parallel, and maximize the overlap area of ​​the field of view of the radar and the image acquisition device; and synchronously acquire image and radar data of the same scene at the same time through the radar and the image acquisition device.

7. The intelligent battlefield personnel injury assessment method according to claim 1, characterized in that, The location and vital signs of the second target are extracted from the radar data, specifically as follows: The radar data is subjected to a Fourier transform along the fast time dimension to generate a two-dimensional data matrix containing range and slow time information; Spectral analysis is performed on the slow time series of each distance unit in the two-dimensional data matrix to detect whether there are significant spectral peaks in the preset vital signs frequency band. The distance units with significant spectral peaks are marked as distance units of interest, and the distance corresponding to the distance unit of interest is the radial distance of the second target. For the data received by the multi-channel receiving array on the range of interest cell, the angle of arrival is estimated by beamforming or angle-dimensional Fourier transform to obtain the azimuth angle of the second target. The position of the second target can then be determined by combining the radial distance. The phase change signal is extracted from the spatial unit where the second target is located, and the breathing signal and heartbeat signal are decoupled from the phase change signal using signal processing technology; The respiratory signal and heartbeat signal are respectively subjected to power spectral density estimation. The respiratory rate and heart rate of the second target can be obtained by detecting the position of the main peak of their respective spectra.

8. The intelligent battlefield personnel injury assessment method according to claim 1, characterized in that, The second detection result, which further determines whether the second target is an injured person based on the vital signs, is as follows: if the respiratory rate of the second target is greater than the respiratory abnormality threshold and the heart rate is greater than the heart rate abnormality threshold, then the second detection result is that the target is an injured person; otherwise, the target is not an injured person.

9. The intelligent battlefield personnel injury assessment method according to claim 1, characterized in that, The transformation matrix is ​​obtained in advance through joint calibration of the image acquisition device and the radar, specifically as follows: Fix a corner reflector in the center of the chessboard calibration plate; The chessboard calibration board is moved to multiple different poses, and at each pose, the image acquisition device and radar synchronously acquire image data and radar data of the chessboard calibration board. For each pose, the two-dimensional coordinates of all corner points in the checkerboard calibration board are identified from the image data, and the two-dimensional coordinates of the corner reflector are further derived from the relative position of the corner reflector on the checkerboard calibration board; the three-dimensional coordinates of the corner reflector are identified from the radar data. Thus, multiple sets of three-dimensional coordinate-two-dimensional coordinate pairs of the corner reflector are obtained through multiple poses; The transformation matrix is ​​obtained by solving multiple sets of the three-dimensional coordinate-two-dimensional coordinate pairs.

10. The intelligent battlefield personnel injury assessment method according to claim 1, characterized in that, Also includes: Based on the aforementioned vital signs, the severity of injury is assessed: If satisfied If so, it is determined to be endangered; If satisfied If so, it is determined to be serious injury; If satisfied If so, it is determined to be a moderate injury; If satisfied If so, it is determined to be a minor injury; If satisfied If so, it is determined to be death. in, Respiratory rate, Heart rate.

Images