An adaptive adversarial attack method for cross-modal pedestrian search in an open environment
By designing a multi-granularity gradient collaboration module and a quality-aware pixel-level improvement module in the natural language cross-modal pedestrian search model, fine-grained perturbations are generated, which solves the problem of insufficient robustness of the model in open environments, achieves efficient attack effects and stealth, and is suitable for cross-modal pedestrian search in open environments.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- NORTHWESTERN POLYTECHNICAL UNIV
- Filing Date
- 2026-03-18
- Publication Date
- 2026-06-26
Smart Images
Figure CN122290170A_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the fields of computer vision, natural language processing, cross-modal retrieval, and adversarial attacks, specifically to an adaptive adversarial attack method for cross-modal pedestrian search in open environments. Background Technology
[0002] Natural language cross-modal person search technology refers to finding the image corresponding to a target pedestrian from an image database using natural language descriptions. It has significant practical implications and application value in fields such as security monitoring and intelligent video analysis. However, the robustness of this technology model has not yet been fully explored. In real-world open environments, attackers can add very small but carefully designed perturbations to the image during inference, severely disrupting cross-modal alignment and drastically reducing the model's retrieval performance, thus seriously hindering the practical application of natural language cross-modal person search technology.
[0003] It should be noted that the information disclosed in the background section above is only used to enhance the understanding of the background of the present invention, and therefore may include information that does not constitute prior art known to those skilled in the art. Summary of the Invention
[0004] This invention provides an adaptive adversarial attack method for cross-modal pedestrian search in open environments, aiming to solve the problems of insufficient robustness of natural language cross-modal pedestrian search models when deployed in open environments, and the susceptibility of cross-modal feature alignment relationships to being disrupted by minor perturbations added to images, resulting in a significant reduction in retrieval performance.
[0005] Other features and advantages of the invention will become apparent from the following detailed description, or may be learned in part by practice of the invention.
[0006] According to a first aspect of the present invention, an adaptive adversarial attack method for cross-modal pedestrian search in open environments is provided, the method comprising: Initialize the adversarial image by adding random noise to the original image; Construct attack targets for image modalities at the sample level. and attack targets targeting text modality ; Construct attack targets against image modality prototypes at the data distribution level. and attack targets targeting text modal prototypes ; Will As the primary target of attack ,Will With weighted , The sum of them serves as a secondary attack target. ; Will dominate the attack target and auxiliary attack targets Taking the partial derivatives with respect to the adversarial image, we obtain the dominant gradient map. and auxiliary gradient map And L1 normalize the two gradient maps; Calculate the dominant gradient graph based on the gradient graph quality factor calculation method. quality factor and auxiliary gradient map quality factor The quotient of the two is The fusion weights are used to obtain the fusion gradient map. ; Calculate the fusion gradient map mean amplitude At each pixel position gradient magnitude and The quotient is the scaling factor, which is applied to the basic perturbation step size. Scaling yields a pixel-level step matrix. ; Based on the step size matrix and fusion gradient map The perturbation is calculated, and a cropping function is used to ensure that the perturbation does not exceed the boundary. The adversarial image is then updated iteratively.
[0007] In some exemplary embodiments, the step of adding random noise to the original image to generate the initial adversarial image specifically involves:
[0008] in, Represents the original image. Represents the initial adversarial image. This represents a random perturbation obtained by sampling from a uniform distribution. It is the disturbance boundary, which limits the maximum range of the disturbance.
[0009] In some exemplary embodiments, the attack target targeting the image modality and attack targets targeting text modality The construction method is as follows:
[0010] in, , and These represent multiple adversarial image features, original image features, and text features for the same pedestrian identity; Represents the cosine distance; , , These represent an adversarial image feature, an original image feature, and a text feature, respectively, for the same pedestrian identity. and These represent the number of all image and text data under the same pedestrian identity.
[0011] In some exemplary embodiments, the attack target against the image modality prototype and attack targets targeting text modal prototypes The construction method is as follows:
[0012] in, and They are respectively The average image features and average text features for each pedestrian identity represent the image modality prototype and the text modality prototype, respectively. This is a predefined hyperparameter for the number of pedestrians.
[0013] In some exemplary embodiments, the gradient map quality factor calculation method specifically includes: For a gradient graph First, calculate its absolute value, then sum the three channels to convert it into a single-channel gradient map. Next, single-channel gradient plot Normalization is performed by subtracting the minimum value and then dividing by the maximum value. Finally, the mean value of the normalized gradient plot is calculated to obtain the quality factor. .
[0014] In some exemplary embodiments, the pixel-level step size matrix The calculation method is as follows:
[0015] in, It is the preset basic perturbation step size. It is the mean of the amplitude of the fused gradient map.
[0016] In some exemplary embodiments, the perturbation calculation and adversarial image update calculation formulas are as follows:
[0017] in, The function outputs only 0, 1, and -1, meaning it only takes the fused gradient map. The direction; It is a clipping function, ensuring Ensure that the infinity norm of the perturbation does not exceed the perturbation boundary. ; Indicates the first The adversarial image.
[0018] According to a second aspect of the present invention, a storage medium is provided having a computer program stored thereon, which, when executed by a processor, implements the adaptive adversarial attack method for cross-modal pedestrian search in open environments as described in the first aspect above.
[0019] According to a third aspect of the present invention, a computer program product is provided, on which a computer program is stored, wherein when the computer program is executed by a processor, it implements the adaptive adversarial attack method for cross-modal pedestrian search in open environments as described in the first aspect above.
[0020] According to a fourth aspect of the present invention, an electronic device is provided, comprising: Processor; and Memory for storing the executable instructions of the processor; The processor is configured to implement the adaptive adversarial attack method for cross-modal pedestrian search in open environments as described in the first aspect when executing the executable instructions.
[0021] This invention is the first to specifically study adversarial attacks against cross-modal pedestrian search technology in natural language, and proposes an adaptive adversarial attack method for cross-modal pedestrian search in open environments. Compared to previous adversarial attacks against traditional single-modal pedestrian search technology that only attack feature alignment relationships within the image modality, this method proposes a multi-granularity gradient collaboration module that simultaneously attacks feature alignment relationships across modalities and within the image modality. Furthermore, it conducts comprehensive attacks on semantics and modality at both the sample level and the data distribution level, enabling gradients of different granularities to collaborate and jointly guide subsequent perturbation calculations. Compared to previous adversarial attacks against cross-modal image-text retrieval technology that only use coarse-grained attacks with uniform perturbation intensity, this method proposes a quality-aware adaptive pixel-level improvement module that uses a fused gradient map to calculate a pixel-level step matrix, ultimately obtaining more targeted fine-grained perturbations. This method achieves significant performance improvements compared to traditional single-modal pedestrian search attack methods and cross-modal image-text retrieval attack methods, reaching the current international leading level.
[0022] Compared with existing technologies, it has the following beneficial effects: 1. Targeted attack, filling a technological gap: This invention designs an adversarial attack method specifically for natural language cross-modal pedestrian search technology. It is the first time that attack and defense research has been carried out specifically for this technology, filling the gap in existing technology research on adversarial attacks in this field, and providing important test basis for the robustness optimization of cross-modal pedestrian search models.
[0023] 2. Multi-granular gradient collaboration for more comprehensive and efficient attacks: A multi-granular gradient collaboration module is designed to construct attack targets at both the sample and data distribution levels, targeting both semantic and modal dimensions. Multi-granular gradient information is integrated to guide perturbation calculation. Compared with traditional single-modal attacks that only target the feature relationships within an image modality, this module achieves dual attacks on cross-modal and intra-modal feature alignment relationships, significantly improving the comprehensiveness and effectiveness of the attack.
[0024] 3. Pixel-level adaptive perturbation for precise and covert attacks: A quality-aware adaptive pixel-level improvement module is proposed, which achieves adaptive fusion of gradient maps through gradient quality factors and dynamically adjusts the contribution weights of different gradients. At the same time, a pixel-level step matrix is calculated so that the perturbation is precisely concentrated on the key area of the pedestrian subject in the image, while the perturbation in the background area is weakened. Compared with the uniform coarse-grained perturbation method of traditional cross-modal retrieval attacks, this not only improves the targeting of the attack, but also ensures that the perturbation is small. Human observers cannot distinguish the difference between the adversarial image and the original image, making the attack extremely covert.
[0025] 4. White-box strategy adapts to open environment and is highly practical: The white-box adversarial attack strategy can fully obtain the parameters, gradients and other data of the attacked model, which is suitable for the application scenarios of cross-modal pedestrian search models in open environment. The attack strategy is highly implementable and can effectively test the robustness of the model in actual deployment.
[0026] 5. Significantly improved performance, reaching the international leading level: Compared with traditional single-modal pedestrian search attack methods and cross-modal image and text retrieval attack methods, this method achieves a significant improvement in attack effect, effectively destroys cross-modal feature alignment relationships, and greatly reduces the retrieval performance of cross-modal pedestrian search models. The technical indicators have reached the current international leading level.
[0027] 6. Modular design with excellent scalability: The method adopts a design architecture with two core modules: multi-granularity gradient collaboration and quality-aware adaptive pixel-level improvement. The module parameters or structure can be fine-tuned for different application scenarios to adapt to the adversarial attack requirements of different cross-modal retrieval tasks, and has good scalability and generalization.
[0028] It should be understood that the above general description and the following detailed description are exemplary and explanatory only, and are not intended to limit the invention. Attached Figure Description
[0029] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the invention and, together with the description, serve to explain the principles of the invention. It is obvious that the drawings described below are merely some embodiments of the invention, and those skilled in the art can obtain other drawings based on these drawings without any inventive effort.
[0030] Figure 1 This is a schematic diagram of the adaptive adversarial attack method framework for cross-modal pedestrian search in open environments according to the present invention. Figure 2 This is a flowchart of an adaptive adversarial attack method for cross-modal pedestrian search in open environments, as proposed in this invention. Detailed Implementation
[0031] Exemplary embodiments will now be described more fully with reference to the accompanying drawings. However, these exemplary embodiments can be implemented in many forms and should not be construed as limited to the examples set forth herein; rather, they are provided so that the invention will be more comprehensive and complete, and will fully convey the concept of the exemplary embodiments to those skilled in the art. The described features, structures, or characteristics may be combined in any suitable manner in one or more embodiments.
[0032] Furthermore, the accompanying drawings are merely illustrative of the invention and are not necessarily drawn to scale. The same reference numerals in the drawings denote the same or similar parts, and therefore repeated descriptions of them will be omitted. Some block diagrams shown in the drawings are functional entities and do not necessarily correspond to physically or logically independent entities. These functional entities can be implemented in software, in one or more hardware modules or integrated circuits, or in different network and / or processor devices and / or microcontroller devices.
[0033] To address the shortcomings and deficiencies of existing technologies, this example implementation provides an adaptive adversarial attack method for cross-modal pedestrian search in open environments. This method adds only subtle perturbations to the image to avoid detection by observers, primarily focusing on white-box adversarial attack strategies. First, a multi-granularity gradient collaboration module is designed, constructing targets at both the sample level and the data distribution level to attack both semantic and modal aspects. The targets are backpropagated to obtain gradient maps, and these gradient maps of different granularities collaborate to comprehensively guide subsequent perturbation calculations. Next, a quality-aware adaptive pixel-level improvement module is designed to perform quality-aware adaptive fusion of the gradient maps to dynamically adjust their relative contributions, and calculates a pixel-level step matrix to concentrate the perturbation on key pedestrian regions in the image. Finally, the perturbation is calculated based on the fused gradient map and the step matrix, iteratively updating the adversarial image.
[0034] refer to Figure 2 As shown, the specific steps may include: Step S0: Initialize the adversarial image by adding random noise to the original image to generate an initial adversarial image; Step S1: Construct attack targets for image modalities at the sample level. and attack targets targeting text modality By utilizing all image and text data under the same pedestrian identity, the semantics of the adversarial image are simultaneously removed from the original image semantics and the paired text semantics. Step S2: Construct an attack target against the image modality prototype at the data distribution level. and attack targets targeting text modal prototypes This causes the adversarial image to move away from the image modality prototype and closer to the text modality prototype; Step S3, will As the primary target of attack ,Will With weighted , The sum of them serves as a secondary attack target. ; Step S4, which will lead the attack on the target and auxiliary attack targets Taking the partial derivatives with respect to the adversarial image, we obtain the dominant gradient map. and auxiliary gradient map And L1 normalize the two gradient maps; Step S5: Calculate the dominant gradient map according to the gradient map quality factor calculation method. quality factor and auxiliary gradient map quality factor ; Step S6, using the dominant gradient map quality factor and auxiliary gradient map quality factor The quotient of the two The fusion weights are used to obtain the fusion gradient map. ; Step S7, Calculate the fused gradient map mean amplitude At each pixel position gradient magnitude and The quotient is the scaling factor, which is applied to the basic perturbation step size. Scaling yields a pixel-level step matrix. ; Step S8, based on the step size matrix and fusion gradient map The perturbation is calculated, and a cropping function is used to ensure that the perturbation does not exceed the boundary. The adversarial image is then updated iteratively.
[0035] The steps in this exemplary embodiment will now be described in more detail with reference to the accompanying drawings and embodiments.
[0036] Step S0: First, initialize the adversarial image. (Adversarial image) To add perturbation to the image, by using the original input image... I Random noise is added as a perturbation to obtain the initial adversarial image, and its formula is expressed as:
[0037] in, Represents the original image. Represents the initial adversarial image. This represents a random perturbation obtained by sampling from a uniform distribution. It is the disturbance boundary, which limits the maximum range of the disturbance.
[0038] Step S1 proposes a multi-granularity gradient collaboration module. In this module, to integrate multi-granularity relationships to generate comprehensive gradients to guide perturbation calculations, attacks are launched at two levels: sample and data distribution, targeting image semantics and modality respectively. First, at the sample level, two sample-level attack targets are constructed. and Using all image-text data under the same pedestrian identity as a guide, the semantics of the adversarial image are altered. The aim is to make the semantics of the adversarial image deviate from both the semantics of the original image and the semantics of its paired text. The formula is expressed as:
[0039] in , and These represent multiple adversarial image features, original image features, and text features for the same pedestrian identity. Representing the cosine distance, the specific calculation process is as follows:
[0040] in, , , These represent an adversarial image feature, an original image feature, and a text feature, respectively, for the same pedestrian identity. and These represent the number of all image and text data under the same pedestrian identity. The attack targets at the two sample levels mentioned above attack cross-modal feature alignment relationships and image-modal feature alignment relationships, respectively, so that adversarial image semantics can simultaneously deviate from the original semantics in both image and text modalities.
[0041] Step S2, using only the sample-level attack targets, ensures that the adversarial image deviates from the original semantics in both the image and text modalities, but neglects the full utilization of the overall modal distribution features. Therefore, in addition to the two attack targets at the sample level, two more attack targets are constructed at the data distribution level. and This method calculates modal prototypes using all image and text data across multiple pedestrian identities to adjust the distribution of adversarial images. On one hand, it aims to distance adversarial images from image modal prototypes to disrupt the overall semantics within the image modality; on the other hand, it aims to bring adversarial images closer to text modal prototypes, thus more closely approximating common text semantics, making the semantics ambiguous and difficult to distinguish in cross-modal retrieval. The formula is expressed as:
[0042] in, and They are respectively The average image features and average text features for each pedestrian identity represent the image modality prototype and the text modality prototype, respectively. The pedestrian count hyperparameter is a predefined value, with a default value of 50. The attack targets at these two data distribution levels shift the distribution of the adversarial image away from the image modality prototype and closer to the text modality prototype, further enhancing the attack's effectiveness.
[0043] Step S3: In order to integrate the multiple attack targets obtained in steps S1 and S2 to guide the calculation of perturbation, the attack targets are divided into two categories according to their importance: one category is the dominant attack target. Another type is auxiliary attack targets. As an attack method targeting cross-modal pedestrian search, the core objective of the attack is to reduce the cosine similarity between adversarial image features and their paired text features. Because... By distancing the adversarial image from its paired text, the cosine similarity between the adversarial image features and their paired text features is reduced, directly reflecting the core objective of the attack. The primary attack target is classified separately and not mixed with other attack targets. The other three attack targets target intramodal relationships and data distribution relationships in the image, indirectly reducing the cosine similarity between adversarial image features and their paired text features; therefore, these three attack targets are classified as secondary attack targets. The classification formula for multi-granularity attack targets is:
[0044] in, It is the distribution weight coefficient, used to adjust the contribution of the attack target in the auxiliary attack target at the data distribution level. Its default value is 1.0.
[0045] Step S4, the dominant attack target obtained in step S3 and auxiliary attack targets They respectively backpropagated to calculate the dominant gradient map for the adversarial image. and auxiliary gradient map Since the scales of the two gradient maps may differ, L1 normalization is performed on both types of gradient maps to balance their scales and prevent perturbations from being dominated by one of them. The formulas for gradient calculation and normalization are as follows:
[0046] Step S5: Propose a quality-aware adaptive pixel-level improvement module. The dominant gradient map obtained in step S4... and auxiliary gradient map Quality-aware adaptive fusion is performed to obtain the fusion gradient map. When fusion gradient maps, since the quality of the two gradient maps for each sample is generally different, the relative contributions of the two gradient maps to the fused gradient map should also be different. Therefore, a quality factor should be calculated for the gradient map and the gradient map should be fused adaptively based on this factor.
[0047] The magnitude of the gradient, i.e., its absolute value, represents the importance of that pixel location. When the overall magnitude of the gradient map is large, it is likely to capture more important pixels. Therefore, this method considers the average magnitude of the gradient map as its quality factor. Specifically, for a gradient map... First, calculate its absolute value, then sum the three channels to convert it into a single-channel gradient map. Next, single-channel gradient plot Normalization is performed by subtracting the minimum value and then dividing by the maximum value. Finally, the mean value of the normalized gradient plot is calculated to obtain the quality factor. The formula for calculating the quality factor of a gradient plot is:
[0048] in, This represents the average of the gradient plot. This represents finding the minimum value of the gradient graph. This represents finding the maximum value of the gradient graph.
[0049] Step S6: Based on the gradient map quality factor calculation method in step S5, process the dominant gradient map obtained in step S4. and auxiliary gradient map Calculate the dominant gradient quality factor separately and auxiliary gradient quality factor To measure the quality of the gradient maps, a fusion gradient is calculated based on a quality factor, adaptively adjusting the relative contributions of the two gradient maps during fusion. and The quotient is used as an auxiliary gradient map The fusion weights are used to obtain the fusion gradient map. The calculation formula is as follows:
[0050] When the quality of the auxiliary gradient map is worse than that of the dominant gradient map, i.e. When the value is less than 1, the relative contribution of the auxiliary gradient map is reduced. Conversely, when the quality of the auxiliary gradient map is better, If the value is greater than 1, the relative contribution of the auxiliary gradient map is increased. During gradient fusion, the relative contribution of the two gradient maps is adaptively adjusted based on their relative quality, achieving quality-aware adaptive gradient fusion.
[0051] Step S7, based on the fusion gradient map obtained in step S6 To calculate the perturbation, and to concentrate the perturbation on the main pedestrian region of the image, different perturbation step sizes need to be calculated for different regions instead of using a uniform perturbation step size for the entire image. Therefore, in addition to the base perturbation step size, an appropriate step scaling factor is calculated for each pixel location based on its importance, achieving adaptive step size improvement. Specifically, based on each pixel location... Gradient magnitude and mean magnitude of fused gradient map The quotient is used as a scaling factor for the perturbation step size, which affects the basic perturbation step size. Scaling is performed to obtain a pixel-level step matrix. The formula for calculating the step size matrix is:
[0052] in, This is the preset base perturbation step size, with a default value of [value to be filled in]. . This is the average magnitude of the fused gradient map, representing the average level of gradient magnitude in the fused gradient map. Pixels with gradient magnitudes exceeding the average level are important pixels, potentially corresponding to the main subject region of an image, and their perturbation step size should be increased relative to the base perturbation step size; while pixels with gradient magnitudes below the average level are unimportant pixels, potentially corresponding to the background region of the image, and their perturbation step size should be decreased relative to the base perturbation step size. The scaling factor is the ratio of gradient magnitude to... The quotient when the gradient magnitude is greater than A value greater than 1 is equivalent to amplifying the basic perturbation step size; when the gradient magnitude is less than 1... A time value less than 1 is equivalent to reducing the basic disturbance step size.
[0053] Step S8, finally, based on the step size matrix obtained in step S7. The fusion gradient map obtained in step S6 To calculate the perturbation and iteratively update the adversarial image. Step matrix. The strength of the perturbation is determined, and the gradient is fused. This determines the direction of the perturbation. The formulas for perturbation calculation and adversarial image update calculation are:
[0054] in The function outputs only 0, 1, and -1, meaning it only takes the fused gradient map. The direction. It is a clipping function, ensuring Ensure that the infinity norm of the perturbation does not exceed the perturbation boundary. This makes it impossible for human observers to distinguish the subtle differences between the adversarial image and the original image. Repeat steps S1 to S8 for a total of... Next, the final adversarial image is obtained. The number of iterations is a predefined number, with a default value of 10. (Superscript) Representing the Adversarial images in the next iteration.
[0055] This invention is the first to specifically study adversarial attacks against cross-modal pedestrian search technology in natural language, and proposes an adaptive adversarial attack method for cross-modal pedestrian search in open environments. Compared to previous adversarial attacks against traditional single-modal pedestrian search technology that only attack feature alignment relationships within the image modality, this method proposes a multi-granularity gradient collaboration module that simultaneously attacks feature alignment relationships across modalities and within the image modality. Furthermore, it conducts comprehensive attacks on semantics and modality at both the sample level and the data distribution level, enabling gradients of different granularities to collaborate and jointly guide subsequent perturbation calculations. Compared to previous adversarial attacks against cross-modal image-text retrieval technology that only use coarse-grained attacks with uniform perturbation intensity, this method proposes a quality-aware adaptive pixel-level improvement module that utilizes fused gradient maps to calculate pixel-level step matrices, resulting in more targeted fine-grained perturbations. This method achieves significant performance improvements compared to traditional single-modal pedestrian search attack methods and cross-modal image-text retrieval attack methods, reaching the current international leading level.
[0056] The above figures are merely illustrative of the processes included in the method according to exemplary embodiments of the present invention and are not intended to be limiting. It is readily understood that the processes shown in the above figures do not indicate or limit the temporal order of these processes. Furthermore, it is readily understood that these processes may, for example, be executed synchronously or asynchronously in multiple modules.
[0057] Other embodiments of the invention will readily occur to those skilled in the art upon consideration of the specification and practice of the invention herein. This application is intended to cover any variations, uses, or adaptations of the invention that follow the general principles of the invention and include common knowledge or customary techniques in the art not disclosed herein. The specification and embodiments are to be considered exemplary only, and the true scope and spirit of the invention are indicated by the claims.
[0058] It should be understood that the present invention is not limited to the precise structure described above and shown in the accompanying drawings, and various modifications and changes can be made without departing from its scope. The scope of the invention is defined only by the appended claims.
Claims
1. An adaptive adversarial attack method for open environment cross-modal pedestrian search, characterized in that, The method includes: Initialize the adversarial image by adding random noise to the original image; Constructing attack targets for image modalities at the sample level and attack targets for text modalities ; Construct attack targets against image modality prototypes at the data distribution level. and attack targets targeting text modal prototypes ; Will As the primary target of attack ,Will With weighted , The sum of them serves as a secondary attack target. ; Will dominate the attack target and auxiliary attack targets Taking the partial derivatives with respect to the adversarial image, we obtain the dominant gradient map. and auxiliary gradient map And L1 normalize the two gradient maps; Calculate the dominant gradient graph based on the gradient graph quality factor calculation method. quality factor and auxiliary gradient map quality factor The quotient of the two is The fusion weights are used to obtain the fusion gradient map. ; Calculate the fusion gradient map mean amplitude At each pixel position gradient magnitude and The quotient is the scaling factor, which is applied to the basic perturbation step size. Scaling yields a pixel-level step matrix. ; Based on the step size matrix and fusion gradient map The perturbation is calculated, and a cropping function is used to ensure that the perturbation does not exceed the boundary. The adversarial image is then updated iteratively.
2. The adaptive adversarial attack method for cross-modal pedestrian search in open environments according to claim 1, characterized in that, The process of adding random noise to the original image to generate the initial adversarial image specifically involves: in, Represents the original image. Represents the initial adversarial image. This represents a random perturbation obtained by sampling from a uniform distribution. It is the disturbance boundary, which limits the maximum range of the disturbance.
3. The adaptive adversarial attack method for cross-modal pedestrian search in open environments according to claim 2, characterized in that, The attack target targeting the image modality and attack targets targeting text modality The construction method is as follows: in, , and These represent multiple adversarial image features, original image features, and text features for the same pedestrian identity; Represents the cosine distance; , , These represent an adversarial image feature, an original image feature, and a text feature, respectively, for the same pedestrian identity. and These represent the number of all image and text data under the same pedestrian identity.
4. The adaptive adversarial attack method for cross-modal pedestrian search in open environments according to claim 3, characterized in that, The attack target targeting the image modality prototype and attack targets targeting text modal prototypes The construction method is as follows: in, and They are respectively The average image features and average text features for each pedestrian identity represent the image modality prototype and the text modality prototype, respectively. This is a predefined hyperparameter for the number of pedestrians.
5. The adaptive adversarial attack method for cross-modal pedestrian search in open environments according to claim 4, characterized in that, The method for calculating the gradient map quality factor is as follows: For a gradient graph First, calculate its absolute value, then sum the three channels to convert it into a single-channel gradient map. Next, single-channel gradient plot Normalization is performed by subtracting the minimum value and then dividing by the maximum value. Finally, the mean value of the normalized gradient plot is calculated to obtain the quality factor. .
6. The adaptive adversarial attack method for cross-modal pedestrian search in open environments according to claim 5, characterized in that, The pixel-level step size matrix The calculation method is as follows: in, It is the preset basic perturbation step size. It is the mean of the amplitude of the fused gradient map.
7. The adaptive adversarial attack method for cross-modal pedestrian search in open environments according to claim 6, characterized in that, The perturbation calculation and adversarial image update calculation formulas are as follows: in, The function outputs only 0, 1, and -1, meaning it only takes the fused gradient map. The direction; It is a clipping function, ensuring Ensure that the infinity norm of the perturbation does not exceed the perturbation boundary. ; Indicates the first The next adversarial image.
8. A storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by the processor, it implements the adaptive adversarial attack method for cross-modal pedestrian search in open environments as described in any one of claims 1 to 7.
9. A computer program product, comprising a computer program, characterized in that, When the computer program is executed by the processor, it implements the adaptive adversarial attack method for cross-modal pedestrian search in open environments as described in any one of claims 1 to 7.
10. An electronic device, characterized in that, include: processor; as well as Memory for storing the executable instructions of the processor; The processor is configured to execute the adaptive adversarial attack method for cross-modal pedestrian search in open environments as described in any one of claims 1 to 7 by executing the executable instructions.