Reinforcement learning-based multi-modal post-fusion stealth method, electronic device, medium
By employing a multimodal post-fusion stealth method based on reinforcement learning, and utilizing an agent to generate and optimize the mask position, the robustness and modality inconsistency issues in multimodal data processing are resolved, achieving efficient stealth and security in complex environments.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- ZHEJIANG UNIV OF TECH
- Filing Date
- 2024-09-06
- Publication Date
- 2026-08-04
AI Technical Summary
Existing multimodal post-fusion methods suffer from low robustness and modal inconsistency in handling complex environments, leading to performance degradation and information loss, and failing to fully explore the potential relationships between different modalities.
A reinforcement learning-based approach is adopted, in which an agent generates an initial mask, performs masking on visible light and infrared images, optimizes the mask position using a depth deterministic policy gradient, obtains a confidence score through post-fusion processing, and updates parameters using a reward function to achieve stealth.
It improves concealment in complex environments, reduces the risk of being discovered, and enhances survivability, especially in ensuring the safety of rescue personnel during emergency rescue operations.
Smart Images
Figure CN119323706B_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the field of reinforcement learning, and more particularly to a multimodal post-fusion stealth method, electronic device, and medium based on reinforcement learning. Background Technology
[0002] Multimodal post-fusion cloaking is a complex field. Multimodal data processing still faces a series of challenges. Due to data heterogeneity, inconsistencies exist between different modalities, making fusion difficult. Existing multimodal post-fusion methods exhibit limitations in addressing these challenges, potentially leading to performance degradation and information loss. Post-fusion methods typically combine the outputs or predictions of multiple independent models after model training or inference. However, current multimodal post-fusion techniques suffer from low robustness to complex environments and shortcomings in handling modal inconsistencies. Traditional methods, relying solely on simple fusion, fail to fully exploit the potential relationships between different modalities, thus limiting performance in complex environments. Summary of the Invention
[0003] To address the shortcomings of existing technologies, this invention provides a multimodal post-fusion stealth method, electronic device, and medium based on reinforcement learning.
[0004] In a first aspect, embodiments of the present invention provide a multimodal post-fusion stealth method based on reinforcement learning, the method comprising:
[0005] Acquire a visible light image and an infrared image;
[0006] An initial mask is generated using an intelligent agent; initial masks are set for visible light and infrared images.
[0007] A masked visible light image is input into a visible light detector to obtain the confidence score of the target object in the visible light mode; an infrared image with a mask is input into an infrared detector to obtain the confidence score of the target object in the red mode.
[0008] The confidence scores of the target object in the visible light mode and the confidence scores of the target object in the red mode are post-fused to obtain the post-fused confidence score.
[0009] Compare the confidence scores of the target object in the visible light mode, the confidence scores of the target object in the red mode, and the post-fusion confidence scores with the threshold to complete one iteration;
[0010] When the confidence scores of the target object in the visible light mode, the confidence scores of the target object in the red mode, and the post-fusion confidence scores are all less than the threshold, the target object completes stealth.
[0011] Conversely, the reward function is calculated, the reinforcement learning parameters are updated, and the agent generates an updated mask. The above steps are repeated iteratively until the confidence score of the target object in the visible light mode, the confidence score of the target object in the red mode, and the post-fusion confidence score are all less than the threshold.
[0012] Secondly, embodiments of the present invention provide an electronic device, including a memory and a processor, wherein the memory is coupled to the processor; wherein the memory is used to store program data, and the processor is used to execute the program data to implement the above-described multimodal post-fusion stealth method based on reinforcement learning.
[0013] Thirdly, embodiments of the present invention provide a computer-readable storage medium having a computer program stored thereon, wherein the program, when executed by a processor, implements the above-described multimodal post-fusion stealth method based on reinforcement learning.
[0014] Fourthly, embodiments of the present invention provide a computer program product, including a computer program / instruction, which, when executed by a processor, implements the above-described reinforcement learning-based multimodal post-fusion stealth method.
[0015] Compared to other methods, this invention provides a multimodal post-fusion stealth method based on reinforcement learning. It involves setting optimally positioned masks for visible light and infrared images; inputting the masked visible light image to a visible light detector to obtain the target's confidence score in the visible light mode; inputting the masked infrared image to an infrared detector to obtain the target's confidence score in the red mode; performing post-fusion processing on the target's confidence scores in the visible light and red modes to obtain a post-fused confidence score; and achieving stealth when all three confidence scores are less than a threshold. This allows the method to be applied in civilian fields, particularly in emergency rescue operations, playing a crucial role in ensuring the safety of rescue personnel, improving concealment, reducing the risk of detection, and enhancing survivability in complex environments. Attached Figure Description
[0016] To more clearly illustrate the technical solutions in the embodiments of the present invention, the drawings used in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.
[0017] Figure 1 A flowchart of a multimodal post-fusion stealth method based on reinforcement learning provided in an embodiment of the present invention;
[0018] Figure 2A schematic diagram illustrating a multimodal post-fusion stealth method based on reinforcement learning provided in an embodiment of the present invention;
[0019] Figure 3 A schematic diagram of reinforcement learning provided in an embodiment of the present invention;
[0020] Figure 4 This is a schematic diagram of an electronic device provided in an embodiment of the present invention. Detailed Implementation
[0021] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.
[0022] It should be noted that, unless otherwise specified, the features in the following embodiments and implementation methods can be combined with each other.
[0023] like Figure 1 and Figure 2 As shown, this embodiment of the invention provides a multimodal post-fusion stealth method based on reinforcement learning, the method comprising the following steps:
[0024] Step S1: Acquire a visible light image and an infrared image.
[0025] Step S2: Generate an initial mask through the intelligent agent.
[0026] Step S3: Set masks for the visible light image and the infrared image respectively.
[0027] Furthermore, the process of setting masks for visible light images and infrared images includes:
[0028]
[0029] In the formula, ⊙ represents the Hadamard product, M represents the mask, and x vis Represents a visible light image, x inf Represents infrared images, This represents a visible light image with a mask. This represents an infrared image with a mask.
[0030] Where M∈{0,1} h×w The mask matrix (where h represents the height of the image and w represents the width of the image) is used to constrain the physical shape of the target.
[0031] Step S4: Input the visible light image with mask into the visible light detector to obtain the confidence score of the target object in the visible light mode; input the infrared image with mask into the infrared detector to obtain the confidence score of the target object in the red mode; perform post-fusion processing on the confidence scores of the target object in the visible light mode and the confidence scores of the target object in the red mode to obtain the post-fused confidence score.
[0032] Step S5: Compare the confidence scores of the target object in the visible light mode, the confidence scores of the target object in the red mode, and the post-fusion confidence scores with the threshold to complete one iteration;
[0033] When the confidence scores of the target object in the visible light mode, the target object in the red light mode, and the post-fusion confidence score are all less than the threshold, the target object completes stealth; the expression is as follows:
[0034]
[0035] In the formula, thre represents the threshold. This represents a visible light image with a mask. This represents an infrared image with a mask. This represents the confidence score of the target object in the visible light mode. R represents the confidence score of the target object in the red mode. fused (.) indicates a post-fusion operation (in this example, the post-fusion process uses the probEn model).
[0036] Conversely, calculate the reward function, update the reinforcement learning parameters, and enable the agent to generate an updated mask. Repeat steps S3 to S4 for iteration until the confidence score of the target object in the visible light mode, the confidence score of the target object in the red mode, and the post-fusion confidence score are all less than the threshold.
[0037] It's important to note that current policy gradient methods employ a stochastic strategy, requiring sampling of the distribution of the current optimal policy for each action. This example, however, utilizes the Deep Deterministic Policy Gradient (DDPG) method from reinforcement learning to optimize the patch placement. The agent randomly generates a mask, which is then cleverly embedded into a specific location within the image by the Actor network. Simultaneously, the Critic network evaluates the Actor's actions to deduce the optimal mask placement. To achieve this, this example introduces a reward function that optimizes the mask placement based on confidence levels.
[0038] Furthermore, the process of generating a mask through an intelligent agent includes:
[0039] The intelligent agent acquires state information from visible light images and infrared images;
[0040] The mask position is determined by employing deep deterministic policy gradients from reinforcement learning.
[0041] The estimation network for the Actor is defined as follows:
[0042] a = μ θ (s t |θ μ )+N t
[0043] In the formula, θ is the neural network parameter, s is the input state, and N is the input state. t This indicates a random location;
[0044] The current state and the position selected by the Actor network are then received by the Critic network, and the Q value is output. The Q value is used to evaluate the quality of the current state and action, that is, the position where the Actor network generates the mask.
[0045] The definition of the Critic value network is as follows:
[0046] Loss=-Q w (s, μ) θ (s))
[0047] Among them, Q w Let s represent the value function, s represent the input state, and μ represent the value function. θ Represents a policy neural network.
[0048] Furthermore, the process of updating reinforcement learning parameters includes:
[0049] The result of the post-fusion detection is used to optimize the mask position through a reward function, the definition of which is as follows:
[0050] r i = log(1 - correct)
[0051] In the formula, r i represents the reward value for the current i-th iteration, and correct represents the post-fusion confidence score;
[0052] The current reward result is combined with the TD target, and then the loss function is minimized to achieve the final goal. The definitions of the TD target and the loss function are as follows:
[0053] y i =r i +γQ'(s i+1 ,μ'(s i+1 |θ μ')|θ Q ')
[0054]
[0055] In the formula, r i γ represents the current reward outcome, γ represents the discount factor, and s i+1 This represents the optimized image state, where N represents the number of images.
[0056] Furthermore, in this example, average precision (AP) is used to measure performance. Precision is calculated on the test image within a single class, with overlapping true positive ground-truth bounding boxes. Calculating the average precision (AP) across all classes measures the performance of multi-class object detection. True positives are defined as detections with IoU > 0.5 and overlapping ground-truth bounding boxes.
[0057] Table 1
[0058]
[0059] The expression for calculating the IoU (Intersection over Union) ratio is as follows:
[0060]
[0061] In the formula, A represents the candidate bounding box of the original image, and B represents the candidate bounding box after passing through the detector.
[0062] A higher AP indicates a higher level of detection capability. The results show that the post-fusion stealth method provided in this example can achieve satisfactory stealth even with misaligned data, highlighting the vulnerability of the post-fusion model. To mitigate potential risks, appropriate security measures need to be implemented during the design and deployment of the post-fusion model, such as increasing the number of adversarial training iterations, implementing more reliable security deployments, and using encryption. Furthermore, for specific application scenarios, a balance needs to be struck between model performance and security to ensure the overall robustness of the system.
[0063] Furthermore, this example uses the FLIR dataset to simulate the stealth effect. The FLIR dataset contains a total of 9711 infrared training / validation images and 9233 visible light training / validation images. From these, 100 images that the target model can successfully identify with a high probability are selected. Therefore, the initial average precision (AP) is 100%. For the pedestrian detection task, YOLOv3 and SSD are chosen as the target detectors. For model training, pre-trained weights provided by the official documentation are used as initial weights, and then fine-tuning is performed on the selected data. The attack success rate (ASR) is used to represent the stealth success rate, and the performance is evaluated by the average precision drop (AP drop). The results are shown in Table 2 below.
[0064] Table 2
[0065]
[0066]
[0067] The results show that the method of the present invention can achieve a good stealth effect.
[0068] Accordingly, this application also provides an electronic device, including: one or more processors; a memory for storing one or more programs; and when the one or more programs are executed by the one or more processors, causing the one or more processors to implement the reinforcement learning-based multimodal post-fusion stealth method as described above. Figure 4 The diagram shown is a hardware structure diagram of any device with data processing capabilities for implementing the multimodal post-fusion stealth method based on reinforcement learning provided in this embodiment of the invention, except... Figure 4 In addition to the processor, memory, and network interface shown, any data processing device in the embodiment may also include other hardware depending on the actual function of the data processing device, which will not be described in detail here.
[0069] Accordingly, this application also provides a computer-readable storage medium storing computer instructions that, when executed by a processor, implement the reinforcement learning-based multimodal post-fusion stealth method described above. The computer-readable storage medium can be an internal storage unit of any data-processing device as described in any of the foregoing embodiments, such as a hard disk or memory. The computer-readable storage medium can also be an external storage device, such as a plug-in hard disk, smart media card (SMC), SD card, flash card, etc., equipped on the device. Furthermore, the computer-readable storage medium can include both internal storage units of any data-processing device and external storage devices. The computer-readable storage medium is used to store the computer program and other programs and data required by the data-processing device, and can also be used to temporarily store data that has been output or will be output.
[0070] The above embodiments are only used to illustrate the design concept and features of the present invention, and their purpose is to enable those skilled in the art to understand the content of the present invention and implement it accordingly. The protection scope of the present invention is not limited to the above embodiments. Therefore, all equivalent changes or modifications made based on the principles and design ideas disclosed in the present invention are within the protection scope of the present invention.
Claims
1. A multimodal post-fusion stealth method based on reinforcement learning, characterized in that, The method includes: Acquire a visible light image and an infrared image; An initial mask is generated using an intelligent agent; initial masks are set for visible light and infrared images. A masked visible light image is input into a visible light detector to obtain the confidence score of the target object in the visible light mode; an infrared image with a mask is input into an infrared detector to obtain the confidence score of the target object in the infrared mode. The confidence scores of the target object in the visible light mode and the confidence scores of the target object in the infrared mode are post-fused to obtain the post-fused confidence score. Compare the confidence scores of the target object in the visible light mode, the confidence scores of the target object in the infrared mode, and the post-fusion confidence scores with the threshold to complete one iteration; When the confidence scores of the target object in the visible light mode, the target object in the infrared mode, and the post-fusion confidence score are all less than the threshold, the target object completes stealth. Conversely, the reward function is calculated, the reinforcement learning parameters are updated, and the agent generates an updated mask. The above steps are repeated iteratively until the confidence score of the target object in the visible light mode, the confidence score of the target object in the infrared mode, and the post-fusion confidence score are all less than the threshold. The process of generating a mask through an intelligent agent includes: The intelligent agent acquires state information from visible light images and infrared images; The mask position is determined by employing deep deterministic policy gradients from reinforcement learning. The estimation network for the Actor is defined as follows: a=μ θ (s t |θ μ )+N t In the formula, θ is the neural network parameter, s is the input state, and N is the input state. t This indicates a random location; The current state and the position selected by the Actor network are then received by the Critic network, and the Q value is output. The Q value is used to evaluate the quality of the current state and action, that is, the position where the Actor network generates the mask. The definition of the Critic value network is as follows: Q w (s,μ θ (s)) Among them, Q w Let s represent the value function, s represent the input state, and μ represent the value function. θ Represents a policy neural network.
2. The multimodal post-fusion stealth method based on reinforcement learning according to claim 1, characterized in that, The process of setting a mask for visible light images and infrared images includes: In the formula, ⊙ represents the Hadamard product, M represents the mask, and x vis Represents a visible light image, x inf Represents an infrared image. This represents a visible light image with a mask. This represents an infrared image with a mask.
3. A multimodal post-fusion stealth method based on reinforcement learning according to claim 1 or 2, characterized in that, When the confidence scores of the target object in the visible light mode, the target object in the infrared mode, and the post-fusion confidence score are all less than the threshold, the expression is as follows: In the formula, thre represents the threshold. This represents a visible light image with a mask. This represents an infrared image with a mask. This represents the confidence score of the target object in the visible light mode. R represents the confidence score of the target object in the infrared mode. fused (.) indicates a post-fusion operation.
4. The multimodal post-fusion stealth method based on reinforcement learning according to claim 1, characterized in that, The post-fusion processing uses the probEn model.
5. The multimodal post-fusion stealth method based on reinforcement learning according to claim 1, characterized in that, The process of calculating the reward function and updating the reinforcement learning parameters includes: The results of the post-fusion detection are used to optimize the mask position using a reward function, which is defined as follows: r i =log(1-correct) In the formula, r i represents the reward value for the current i-th iteration, and correct represents the post-fusion confidence score; The current reward result is combined with the TD target, and then the loss function is minimized to achieve the final goal. The definitions of TD target and loss function are as follows: y i =r i +γQ′(s i+1 ,μ′(S i+1 |θ μ′ )|θ Q′ ) In the formula, r i γ represents the current reward outcome, γ represents the discount factor, and s i+1 This represents the optimized image state, where N represents the number of images.
6. An electronic device comprising a memory and a processor, characterized in that, The memory is coupled to the processor; wherein the memory is used to store program data, and the processor is used to execute the program data to implement the reinforcement learning-based multimodal post-fusion stealth method according to any one of claims 1-5.
7. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the program is executed by the processor, it implements the multimodal post-fusion stealth method based on reinforcement learning as described in any one of claims 1-5.
8. A computer program product comprising a computer program / instructions, characterized in that, When the computer program / instruction is executed by the processor, it implements the reinforcement learning-based multimodal post-fusion stealth method as described in any one of claims 1-5.