Multimodal post-fusion stealth method based on reinforcement learning, electronic equipment and medium

Abstract

The invention discloses a multi-mode post-fusion stealth method based on reinforcement learning, electronic equipment and a medium. The method comprises the following steps: acquiring a visible light image and an infrared image; generating an initial mask through the intelligent agent; setting initial masks for the visible light image and the infrared image; inputting the visible light image with the mask into a visible light detector to obtain a confidence score of the target object in a visible light mode; inputting the infrared image with the mask into an infrared detector to obtain a confidence score of the target object in the red mode; performing post-fusion processing on the scores to obtain a post-fusion confidence score; when the three confidence scores are all smaller than a threshold value, the target object is hidden; otherwise, calculating a reward function, updating reinforcement learning parameters, enabling the intelligent agent to generate an updated mask, and repeating the steps for iteration until the three confidence scores are all smaller than the threshold value.

Description

Technical Field

[0001] The present invention relates to the field of reinforcement learning, and in particular to a multimodal post-fusion stealth method, electronic equipment and medium based on reinforcement learning. Background Art

[0002] Multimodal post-fusion stealth is a complex field. The field of multimodal data processing still faces a series of challenges. Due to data heterogeneity, there is inconsistency in information between different modalities, which makes fusion difficult. Existing multimodal post-fusion methods show certain limitations in dealing with these challenges, which may lead to performance degradation and information loss. Post-fusion methods usually combine the outputs or prediction results of multiple independent models in a certain way after model training or inference is completed. However, the current multimodal post-fusion technology has certain shortcomings in terms of low robustness to complex environments and handling modal inconsistency. Traditional methods only use simple fusion methods and fail to fully explore the potential relationship between different modalities, thereby limiting the performance in complex environments. Summary of the invention

[0003] In view of the shortcomings of the prior art, the present invention provides a multi-modal post-fusion stealth method, electronic device, and medium based on reinforcement learning.

[0004] In a first aspect, an embodiment of the present invention provides a multimodal post-fusion stealth method based on reinforcement learning, the method comprising:

[0005] Acquire a visible light image and an infrared image;

[0006] Generate an initial mask through an intelligent agent; set the initial mask for the visible light image and the infrared image;

[0007] The visible light image with the mask is input into the visible light detector to obtain the confidence score of the target in the visible light mode; the infrared image with the mask is input into the infrared detector to obtain the confidence score of the target in the infrared mode;

[0008] Post-fusion processing is performed on the confidence score of the target object in the visible light mode and the confidence score of the target object in the red mode to obtain a post-fusion confidence score;

[0009] Compare the confidence score of the target in the visible light mode, the confidence score of the target in the red mode, the post-fusion confidence score and the threshold, and complete one iteration;

[0010] When the confidence score of the target in the visible light mode, the confidence score of the target in the red mode, and the post-fusion confidence score are all less than the threshold, the target is invisible;

[0011] On the contrary, the reward function is calculated and the reinforcement learning parameters are updated to enable the agent to generate an updated mask. The above steps are repeated iteratively until the confidence score of the target object in the visible light modality, the confidence score of the target object in the red modality, and the post-fusion confidence score are all less than the threshold.

[0012] In a second aspect, an embodiment of the present invention provides an electronic device, comprising a memory and a processor, wherein the memory is coupled to the processor; wherein the memory is used to store program data, and the processor is used to execute the program data to implement the above-mentioned multimodal post-fusion stealth method based on reinforcement learning.

[0013] In a third aspect, an embodiment of the present invention provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the above-mentioned multimodal post-fusion stealth method based on reinforcement learning.

[0014] In a fourth aspect, an embodiment of the present invention provides a computer program product, including a computer program / instruction, which, when executed by a processor, implements the above-mentioned multimodal post-fusion stealth method based on reinforcement learning.

[0015] Compared with the beneficial effects of the present invention: the present invention provides a multimodal post-fusion stealth method based on reinforcement learning, by setting a mask at the optimal position for the visible light image and the infrared image; inputting the visible light image with the mask into the visible light detector to obtain the confidence score of the target in the visible light mode; inputting the infrared image with the mask into the infrared detector to obtain the confidence score of the target in the red mode; performing post-fusion processing on the confidence score of the target in the visible light mode and the confidence score of the target in the red mode to obtain the post-fusion confidence score; when the above three confidence scores are all less than the threshold, the stealth is completed. The method of the present invention is applied to the civilian field, especially in emergency rescue operations, and plays an important role in ensuring the safety of rescuers, improving concealment, reducing the risk of being discovered, and enhancing survivability in complex environments. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the accompanying drawings required for use in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings in the following description are only some embodiments of the present invention. For ordinary technicians in this field, other accompanying drawings can be obtained based on these accompanying drawings without paying creative labor.

[0017] Figure 1 A flowchart of a multi-modal post-fusion stealth method based on reinforcement learning provided by an embodiment of the present invention;

[0018] Figure 2A schematic diagram of a multi-modal post-fusion stealth method based on reinforcement learning provided by an embodiment of the present invention;

[0019] Figure 3 A schematic diagram of reinforcement learning provided by an embodiment of the present invention;

[0020] Figure 4 A schematic diagram of an electronic device provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0021] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0022] It should be noted that, in the absence of conflict, the features in the following embodiments and implementations may be combined with each other.

[0023] like Figure 1 and Figure 2 As shown, an embodiment of the present invention provides a multi-modal post-fusion stealth method based on reinforcement learning, and the method includes the following steps:

[0024] Step S1, acquiring a visible light image and an infrared image.

[0025] Step S2, generating an initial mask through an agent.

[0026] Step S3, setting masks for the visible light image and the infrared image respectively.

[0027] Furthermore, the process of setting a mask for the visible light image and the infrared image includes:

[0028]

[0029] Where ⊙ represents the Hadamard product, M represents the mask, and x vis represents the visible light image, x inf represents an infrared image, represents the visible light image with mask, Represents an infrared image with a mask.

[0030] Where M∈{0,1} h×w Represents the mask matrix (where h represents the height of the image and w represents the width of the image), which is used to constrain the physical shape of the target.

[0031] Step S4: input the visible light image with the mask into the visible light detector to obtain the confidence score of the target in the visible light mode; input the infrared image with the mask into the infrared detector to obtain the confidence score of the target in the red mode; perform post-fusion processing on the confidence score of the target in the visible light mode and the confidence score of the target in the red mode to obtain the post-fusion confidence score.

[0032] Step S5, comparing the confidence score of the target object in the visible light mode, the confidence score of the target object in the red mode, the post-fusion confidence score and the threshold, completing one iteration;

[0033] When the confidence score of the target in the visible light mode, the confidence score of the target in the red mode, and the post-fusion confidence score are all less than the threshold, the target is invisible; the expression is as follows:

[0034]

[0035] In the formula, thre represents the threshold value, represents the visible light image with mask, represents an infrared image with a mask, represents the confidence score of the target in the visible light mode, Represents the confidence score of the target in the red mode, R fused (.) represents the post-fusion operation (in this example, the post-fusion processing adopts the probEn model).

[0036] On the contrary, the reward function is calculated, the reinforcement learning parameters are updated, the agent generates an updated mask, and the above steps S3 to S4 are repeated iteratively until the confidence score of the target object in the visible light modality, the confidence score of the target object in the red modality, and the post-fusion confidence score are all less than the threshold.

[0037] It should be noted that the current policy gradient adopts a random strategy. Each time an action is obtained, the distribution of the current optimal strategy needs to be sampled. In this example, the deep deterministic policy gradient (DDPG) method in reinforcement learning is used to optimize the position of the patch. The agent randomly generates a mask, and then the Actor network is responsible for cleverly embedding the mask into a certain position in the image. At the same time, the task of the Critic network is to evaluate the Actor's actions to derive the best position for the mask. To achieve this goal, this example introduces a reward function, which optimizes the position of the mask by confidence.

[0038] Furthermore, the process of generating a mask by an agent includes:

[0039] The agent obtains the visible light image and the status information of the infrared image;

[0040] Adopt deep deterministic policy gradient in reinforcement learning to determine the position of the mask;

[0041] The estimation network definition of an Actor is as follows:

[0042] a=μ θ (s t |θ μ )+N t

[0043] In the formula, θ is the neural network parameter, s is the input state, and N t It indicates a random position;

[0044] The Critic network then receives the current state and the position selected by the Actor network and outputs a Q value; the Q value is used to evaluate the quality of the current state and action, that is, the position of the mask generated by the Actor network;

[0045] Among them, the definition of the Critic value network is as follows:

[0046] Loss = -Q w (s,μ θ (s))

[0047] Among them, Q w represents the value function, s represents the input state, μ θ represents the policy neural network.

[0048] Furthermore, the process of updating reinforcement learning parameters includes:

[0049] The result of the post-fusion detection is used to optimize the position of the mask through the reward function, and the definition of the reward function is as follows:

[0050] r i =log(1-correct)

[0051] In the formula, r i represents the current reward value of the i-th time, and correct represents the post-fusion confidence score;

[0052] Combine the current reward result with the TD target, and then minimize the loss function to achieve the final goal. The definitions of TD target and loss function are as follows:

[0053] y i =r i +γQ'(s i+1 ,μ'(s i+1 |θ μ')|θ Q ')

[0054]

[0055] In the formula, r i represents the current reward result, γ represents the discount factor, s i+1 Indicates the image status after this optimization, and N indicates the number of images.

[0056] Furthermore, in this example, the average precision (AP) is used to measure the performance. The precision is calculated on test images within a single class with overlapping true positive ground-truth bounding boxes. Calculating the average precision (AP) over all classes can measure the performance of multi-class object detection. A true positive is defined as a detection with an IoU>0.5 overlap with the ground truth.

[0057] Table 1

[0058]

[0059] The expression for calculating the IoU intersection-over-union ratio is as follows:

[0060]

[0061] In the formula, A represents the candidate box of the original image, and B represents the candidate box after passing the detector.

[0062] The higher the AP, the easier it is to detect. From the analysis of the results, it can be concluded that the post-fusion stealth method provided in this example can also achieve the required stealth effect for non-aligned data, which shows the vulnerability of the post-fusion model. In order to reduce potential risks, it is necessary to take corresponding security measures in the design and deployment of the post-fusion model, such as increasing the number of adversarial training, performing more reliable security deployment, encryption, etc. At the same time, for specific application scenarios, it is necessary to balance the performance and security of the model to ensure the robustness of the overall system.

[0063] Furthermore, this example uses the FLIR dataset to simulate the stealth effect. The FLIR dataset contains a total of 9711 infrared training / validation images and 9233 visible light training / validation images. 100 images are selected from which the target model can successfully identify with a high probability. Therefore, the initial average precision (AP) is 100%. For the pedestrian detection task, YOLOV3 and SSD are selected as target detectors. For the training of the model, the official pre-trained weights are selected as the initialized weights, and then fine-tuning is performed on the data we selected. The attack success rate (ASR) is used to represent the stealth success rate, and the performance is evaluated by the average precision drop (AP drop). The results are shown in Table 2 below:

[0064] Table 2

[0065]

[0066]

[0067] It can be concluded from the result analysis that the method of the present invention can achieve a better stealth effect.

[0068] Accordingly, the present application also provides an electronic device, comprising: one or more processors; a memory for storing one or more programs; when the one or more programs are executed by the one or more processors, the one or more processors implement the above-mentioned multi-modal post-fusion stealth method based on reinforcement learning. Figure 4 As shown, a hardware structure diagram of any device with data processing capability in which the multi-modal post-fusion stealth method based on reinforcement learning provided by an embodiment of the present invention is located, except Figure 4 In addition to the processor, memory and network interface shown, any device with data processing capability in which the apparatus in the embodiment is located may also include other hardware, generally based on the actual functions of the device with data processing capability, which will not be described in detail.

[0069] Correspondingly, the present application also provides a computer-readable storage medium on which computer instructions are stored, and when the instructions are executed by the processor, the multimodal post-fusion stealth method based on reinforcement learning as described above is implemented. The computer-readable storage medium can be an internal storage unit of any device with data processing capabilities described in any of the aforementioned embodiments, such as a hard disk or a memory. The computer-readable storage medium can also be an external storage device, such as a plug-in hard disk, a smart memory card (Smart Media Card, SMC), an SD card, a flash card (Flash Card), etc. equipped on the device. Furthermore, the computer-readable storage medium can also include both an internal storage unit of any device with data processing capabilities and an external storage device. The computer-readable storage medium is used to store the computer program and other programs and data required by any device with data processing capabilities, and can also be used to temporarily store data that has been output or is to be output.

[0070] The above embodiments are only used to illustrate the design ideas and features of the present invention, and their purpose is to enable those skilled in the art to understand the content of the present invention and implement it accordingly. The protection scope of the present invention is not limited to the above embodiments. Therefore, any equivalent changes or modifications made based on the principles and design ideas disclosed by the present invention are within the protection scope of the present invention.

Claims

1. A multimodal post-fusion stealth method based on reinforcement learning, characterized in that: The method comprises: Acquire a visible light image and an infrared image; Generate an initial mask through an intelligent agent; set the initial mask for the visible light image and the infrared image; The visible light image with the mask is input into the visible light detector to obtain the confidence score of the target in the visible light mode; the infrared image with the mask is input into the infrared detector to obtain the confidence score of the target in the infrared mode; Post-fusion processing is performed on the confidence score of the target object in the visible light mode and the confidence score of the target object in the red mode to obtain a post-fusion confidence score; Compare the confidence score of the target in the visible light mode, the confidence score of the target in the red mode, the post-fusion confidence score and the threshold, and complete one iteration; When the confidence score of the target in the visible light mode, the confidence score of the target in the red mode, and the post-fusion confidence score are all less than the threshold, the target is invisible; On the contrary, the reward function is calculated and the reinforcement learning parameters are updated to enable the agent to generate an updated mask. The above steps are repeated iteratively until the confidence score of the target object in the visible light modality, the confidence score of the target object in the red modality, and the post-fusion confidence score are all less than the threshold.

2. According to the multimodal post-fusion stealth method based on reinforcement learning in claim 1, it is characterized in that: The process of setting a mask for a visible light image and an infrared image includes: Where ⊙ represents the Hadamard product, M represents the mask, and x vis represents the visible light image, x inf represents an infrared image, represents the visible light image with mask, Represents an infrared image with a mask.

3. A multimodal post-fusion stealth method based on reinforcement learning according to claim 1 or 2, characterized in that: When the confidence score of the target in the visible light mode, the confidence score of the target in the red mode, and the post-fusion confidence score are all less than the threshold, the expression is as follows: In the formula, thre represents the threshold value, represents the visible light image with mask, represents an infrared image with a mask, represents the confidence score of the target in the visible light mode, Represents the confidence score of the target in the red mode, R fused (.) indicates a post-fusion operation.

4. According to the multimodal post-fusion stealth method based on reinforcement learning in claim 1, it is characterized in that: The post-fusion processing adopts the probEn model.

5. According to the multimodal post-fusion stealth method based on reinforcement learning in claim 1, it is characterized in that: The process of generating masks through the agent includes: The agent obtains the visible light image and the status information of the infrared image; Adopt deep deterministic policy gradient in reinforcement learning to determine the position of the mask; The estimation network definition of an Actor is as follows: a=μ θ (s t |θ μ )+N t In the formula, θ is the neural network parameter, s is the input state, and N t It indicates a random position; The Critic network then receives the current state and the position selected by the Actor network and outputs a Q value; the Q value is used to evaluate the quality of the current state and action, that is, the position of the mask generated by the Actor network; Among them, the definition of the Critic value network is as follows: Loss=-Q w (s,μ θ (s)) Among them, Q w represents the value function, s represents the input state, μ θ represents the policy neural network.

6. According to claim 5, a multi-modal post-fusion stealth method based on reinforcement learning is characterized in that: The process of calculating the reward function and updating the reinforcement learning parameters includes: The result of the post-fusion detection is used to optimize the position of the mask through the reward function, and the definition of the reward function is as follows: r i =log(1-correct) In the formula, r i represents the current reward value of the i-th time, and correct represents the post-fusion confidence score; Combine the current reward result with the TD target, and then minimize the loss function to achieve the final goal. The definitions of TDtarget and loss function are as follows: y i =r i +γQ'(s i+1 ,m'(s i+1 |θ μ' )|θ Q' ) In the formula, r i represents the current reward result, γ represents the discount factor, s i+1 Indicates the image status after this optimization, and N indicates the number of images.

7. An electronic device comprising a memory and a processor, characterized in that: The memory is coupled to the processor; wherein the memory is used to store program data, and the processor is used to execute the program data to implement the multimodal post-fusion stealth method based on reinforcement learning as described in any one of claims 1-6 above.

8. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the program is executed by a processor, the multimodal post-fusion stealth method based on reinforcement learning is implemented as described in any one of claims 1-6.

9. A computer program product comprising a computer program / instructions, characterized in that When the computer program / instruction is executed by a processor, the multimodal post-fusion stealth method based on reinforcement learning described in any one of claims 1-6 is implemented.

Images