A multi-depth camera cooperative camouflage target detection method, system and device

Abstract

The present application relates to the technical field of computer vision, and provides a kind of multi-depth camera coordinated camouflage target detection method, comprising the following steps: S1: each depth camera is collected, and the color image and depth map and time stamp of current field of view are obtained;S2: the color image and depth map to be measured are processed, feature and depth gradient graph are extracted, and camouflage target information is output, the camouflage target information includes local 3D candidate position and camouflage suspicious degree score;S3: each depth camera executes centerless data exchange according to the output camouflage target information;S4: combine the feature results of multiple depth cameras, determine and output camouflage target detection results.The detection failure problem caused by "apparent feature height similarity and spatial geometric projection contradiction coexist" when facing camouflage target is solved in traditional multi-view geometric verification.

Description

Technical Field

[0001] This invention relates to the field of computer vision technology, and more particularly to a method, system, and apparatus for detecting camouflaged targets using multi-depth camera collaboration. Specifically, it relates to a technology for 3D target detection based on multi-view depth cameras and collaborative camouflaged target detection using multiple depth cameras. Background Technology

[0002] Detection of camouflaged targets (COD) aims to discover targets that visually blend into their surroundings. In collaborative reconnaissance scenarios involving multiple mobile platforms using unmanned equipment, existing single-unit or collaborative detection solutions face the following insurmountable underlying technical bottlenecks: First, most existing depth-based camouflage detection methods treat depth maps as semantic feature complements to RGB images, performing network-level stitching or attention fusion. However, the essence of camouflage is "deceiving appearance features (texture / color) but not physical space (volume / boundaries)." The high-dimensional RGB features of camouflaged target regions typically exhibit low variance (because they mimic similar appearance texture features of the background), while their corresponding local depth estimates are more prone to anomalous gradients and 3D structural variations (because they possess a 3D structure independent of the background). Existing techniques fail to utilize this local mutual exclusion between "appearance feature similarity" and "3D structural difference" for masking guidance, resulting in prior depth information being overwhelmed by redundant appearance features when facing highly realistic camouflaged targets.

[0003] Secondly, in multi-view 3D target detection, cross-view feature matching or geometric consistency verification (such as depth projection error) typically assumes that the appearance features of the same object under different viewpoints should have a certain degree of distinguishability, and that depth projection should converge in space. However, the appearance features of camouflaged targets under viewpoints A and B are also prone to high similarity, and camouflage is often flexibly fitted to the background, leading to physical contradictions in its 3D spatial coordinates during cross-view projection, such as projection errors from multiple viewpoints and multiple cameras. Existing multi-view fusion algorithms, when faced with this unique characteristic of camouflage where "the more similar the features, the more severe the geometric contradictions," lack a flaw index model to quantify the degree of contradiction, and are prone to misjudging camouflaged areas as continuous background.

[0004] Existing multi-machine collaborative sensing typically employs indiscriminate full feature broadcasting or fixed-threshold filtering strategies. However, these approaches suffer from "semantic asymmetry" in camouflage detection tasks: for regular targets, confirmation is sufficient based on single-view 2D detection confidence, requiring only the broadcast of lightweight categories and coordinates; but for camouflaged targets, single-view confidence is low, and without sharing high-dimensional local feature vectors, other views lose semantic anchors for cross-validation. Current technologies lack a mechanism for dynamically evaluating single-machine view confidence based on "camouflage suspicion" to determine differentiated feature exchange, leading to camouflage cues being ignored in complex environments due to limited exchange content.

[0005] Therefore, this patent urgently needs a method that can combine physical and apparent contradictions, establish a cross-view camouflage flaw model, and adapt to a multi-moving depth camera collaborative camouflage target detection method. Summary of the Invention

[0006] To address the aforementioned problems, the present invention aims to provide a method, system, and apparatus for detecting camouflaged targets using multi-depth camera collaboration. This solves the problem of detection failure caused by the coexistence of "high similarity in appearance features and contradictions in spatial geometric projection" when facing camouflaged targets using traditional multi-view geometric verification.

[0007] The above-mentioned objective of this invention is achieved through the following technical solutions: A method for detecting camouflaged targets using multi-depth camera collaboration includes the following steps: S1: Acquire data from each depth camera to obtain color images, depth maps, and timestamps of the current field of view; S2: Process the color image and depth map to be tested, extract features and depth gradient map, and output camouflage target information, which includes local 3D candidate positions and camouflage suspicion score. S3: Each depth camera performs decentralized data exchange based on the output camouflage target information; S4: Combine the feature results from multiple depth cameras to determine and output the detection results of camouflaged targets.

[0008] Furthermore, step S2 specifically includes: S21: Input the color image into a pre-trained camouflage target detection feature extraction network to obtain RGB region feature maps and preliminary camouflage target candidate boxes output by the network, and at the same time calculate the local depth gradient map of the depth map; S22: Using the preliminary camouflage target candidate box as the spatial constraint region, perform cross-modal coupling processing on the features of the corresponding region in the region feature map and the local depth gradient map to generate a spatial prior mask; S23: The RGB features of the color image are weighted using the spatial prior mask to suppress feature responses consistent with the background depth and enhance texture features at the depth boundary. S24: Based on the enhanced feature map, perform refined extraction of the target region and 3D spatial regression, and combine the preliminary disguised target candidate box and depth flaw features to generate the initial 3D candidate position and disguise suspicion score.

[0009] Furthermore, step S3 specifically includes: S31: For regions corresponding to initial 3D candidate locations whose camouflage suspicion score is higher than the initial preset threshold, broadcast their local 3D spatial coordinates, local region feature vectors, and timestamps; for regular target regions whose camouflage suspicion score is lower than the initial preset threshold, only broadcast their category label, local 3D spatial coordinates, and timestamps.

[0010] Furthermore, step S4 specifically includes: S41: Receive local 3D spatial coordinates and local region feature vectors from other depth cameras, and based on the relative pose relationship between cameras, transform the 3D spatial coordinates of other viewpoints to the local coordinate system of the current local camera for coordinate system alignment. S42: For the same 3D spatial region covered by at least two different viewpoints, use the 3D spatial coordinates transformed to the local coordinate system to determine the spatial distance to confirm the overlapping observation, and use the local region feature vector to perform cross-viewpoint feature soft alignment to extract the overlapping mask. S43: Calculate the cross-view vulnerability index and pseudo-device confidence based on the overlapping mask; S44: Based on the reliability of the fake device, a threshold is used to determine whether it is a disguised target. If it is higher than the threshold, it is confirmed to be a disguised target; if it is lower than the threshold, it is determined to be a non-disguised target and is removed.

[0011] Furthermore, the camouflage target detection feature extraction network used in step S21 is pre-trained on a publicly available camouflage target dataset to extract semantic features from the left and right color images, and is spliced ​​in the mid-to-high-level feature stage of the network to output the region feature map and the preliminary camouflage target candidate box. Preferably, the feature extraction network for camouflage target detection can adopt common visual models such as ResNet50 or ViT-B / 16, which are commonly used feature extraction backbone networks; In step S22, using the preliminary camouflage target candidate box as the spatial constraint region, cross-modal coupling processing is performed to generate a spatial prior mask, specifically including: Determine the spatial downsampling factor 's' of the region feature map relative to the original color image. Within the local region corresponding to the local depth gradient map where the initial camouflage target candidate box is mapped, divide non-overlapping spatial dimensionality reduction windows according to the downsampling factor 's'. ; For each of the aforementioned spatial dimensionality reduction windows Calculate the local dispersion index of all pixel gradient values ​​within the window, and generate a depth disorder matrix that is perfectly aligned with the spatial size of the RGB region feature map, where the depth disorder at coordinate point p is... The calculation formula is: , Display window The depth gradient value of the i-th pixel. Display window The mean of the depth gradient values ​​of all pixels within the range. This indicates the total number of pixels within the window; With the depth disorder matrix and the RGB region feature map constrained by the pixel coordinate system of the initial camouflage target candidate box, for each coordinate point p, the apparent semantic local smoothness of the RGB region feature corresponding to that coordinate point is extracted. , wherein Defined as the negative exponent of the variance of the eigenvectors in the neighborhood of coordinate point p. ; The local mutual exclusion index between the apparent semantic smoothness and the three-dimensional structural abrupt change region is calculated using the following formula. ; The global and local mutual exclusion index within the coverage area of ​​the initial camouflage target candidate box Perform Softmax normalization and output the spatial prior mask with values ​​ranging from 0 to 1. In step S24, calculating and outputting the disguise suspicion score specifically includes: The ratio of the number of pixels falling within the spatial prior mask to the total number of pixels in the two-dimensional projection region of the initial 3D candidate position on the color image is calculated, and this ratio is used as a measure of physical anomalies. ; The confidence score of the two-dimensional object detection classification corresponding to the initial 3D candidate position is obtained as a two-dimensional appearance metric. ; Based on the physical anomaly measurement Constructing dynamic gain coefficients ,in The preset physical gain weight and ; The two-dimensional appearance measurement Multiply by the dynamic gain coefficient to generate the spoofing suspicion score. The calculation formula is as follows: .

[0012] Preferably, S3 performs centerless data exchange for each depth camera based on the output camouflage target information. For regions corresponding to initial 3D candidate positions with camouflage suspicion scores higher than the initial preset threshold, it broadcasts their 3D spatial coordinates and local region feature vectors. For regular target regions with camouflage suspicion scores lower than the initial preset threshold, it only broadcasts their category labels and 3D spatial coordinates.

[0013] Further, step S41 receives the local 3D spatial coordinates and local region feature vectors of other depth cameras, and based on the relative pose relationship between the cameras, transforms the 3D spatial coordinates of other viewpoints to the local coordinate system of the current local camera for coordinate system alignment, specifically including: Based on multi-view scene feature association, the structure-of-motion method is used to estimate the relative pose relationships between depth cameras online. ,in This represents the transformation from the local coordinate system of viewpoint B to the coordinate system of viewpoint A. Let be a rotation matrix. Let be the translation vector; where It can be obtained through the absolute poses of viewpoints A and B in the global coordinate system; When camera A receives local 3D spatial coordinates broadcast from another viewpoint B... At that time, using the relative pose relationship Transform it to the local coordinate system of camera A to obtain the transformed 3D spatial coordinates. : ; In step S42, cross-view feature soft alignment is performed using the local region feature vector to extract the overlapping mask, specifically including: The dot product attention calculation is performed on the local region feature vectors of the current viewpoint and other viewpoints in the same 3D spatial region to obtain the cross-viewpoint feature similarity matrix. Using the feature similarity matrix as weights, the projected pixel coordinates under the current viewpoint are... The corresponding projection point set is weighted and filtered to remove outlier projection points with similarity below the alignment threshold. The retained set of projection points is rasterized to generate the overlapping mask.

[0014] Furthermore, in step S43, the vulnerability index across viewpoints is calculated using the following formula:

[0015] in The vulnerability index, and These are the local region feature vectors of the corresponding pixels in the overlapping mask for the current viewpoint A and the received viewpoint B. The feature cosine similarity function, is the preset bursting magnification factor, and N is the total number of overlapping pixels; The The cross-view depth projection inconsistency value of the i-th pixel is obtained through the following steps: From aligned 3D space homogeneous coordinates In the process, the Z-axis depth component in the current viewpoint A coordinate system is extracted as the predicted depth value. ; In the actual depth map of the current viewpoint A, the projected pixel coordinates are extracted, and the true depth value at that pixel coordinate in the actual depth map of the current viewpoint A is obtained. ; The absolute difference between the predicted depth value and the actual depth value is calculated and used as the cross-view depth projection discrepancy value. .

[0016] Further, in step S43, calculating the spurious device confidence specifically includes: Extract the flaw index sequence within the historical sliding time window, including the current frame, and calculate the first-order time difference of the flaw index between adjacent frames to eliminate the constant depth bias introduced by the static calibration error of the relative pose of multiple cameras and obtain the dynamic change of flaws. The dynamic changes of the flaw within the historical sliding time window are monotonically accumulated in the same direction using a time decay factor to filter out the bidirectional differential disturbances caused by random oscillations in depth estimation and generate a flaw kinetic energy index. The vulnerability kinetic energy index is normalized to the probability space through a nonlinear mapping function to generate the pseudo-device confidence. Based on the reliability of the fake device, a threshold is used to determine whether it is a disguised target. If the value is higher than the threshold, it is confirmed as a disguised target; if the value is lower than the threshold, it is determined as a non-disguised target and is removed.

[0017] The present invention also provides a multi-depth camera cooperative camouflage target detection system for performing the multi-depth camera cooperative camouflage target detection method as described above, comprising: The joint depth vulnerability discovery module is used to acquire and process color images and output camouflage target information, including 3D location and camouflage suspicion score; The feature communication exchange module is used by each depth camera to perform decentralized data exchange based on the output camouflage target information; The multi-camera camouflage target determination module combines feature results from multiple depth cameras to output camouflage target detection results.

[0018] The present invention also provides a multi-depth camera collaborative camouflage target detection device, characterized in that it includes: at least two sets of movable mechanical devices equipped with depth cameras, a processor, and a memory; The depth camera is used to acquire color images of the current field of view; The memory contains the authorized computer program; When the processor executes the computer program, it performs the steps of the method described above.

[0019] The present invention also provides a computer-readable storage medium, characterized in that the computer-readable storage medium stores computer code, and when the computer code is executed, the method described above is performed.

[0020] Compared with the prior art, the beneficial effects of the present invention are: This invention discloses a method and system device for detecting and recognizing camouflaged targets based on the collaboration of multiple depth cameras. The method combines the texture deception features of local image regions and the three-dimensional spatial abrupt change features based on the depth gradient of local regions to obtain the initial 3D candidate positions and camouflage suspicion scores from a single camera viewpoint. Through centerless multi-camera feature communication based on the suspicion score, the method fuses the depth geometric alignment information of multiple cameras across viewpoints with high-dimensional local region feature vectors, and then extracts the flaw trend to calculate the camouflage device confidence. Finally, it achieves improved robustness and accuracy of camouflaged target detection through the collaboration of multiple depth cameras. Attached Figure Description

[0021] Figure 1 This is an overall flowchart of the multi-depth camera collaborative camouflage target detection method of the present invention; Figure 2 This is a schematic diagram of the output space prior mask provided by the present invention according to an exemplary embodiment; Figure 3 This is a schematic diagram of a camouflaged target detection method using multiple depth cameras in accordance with an exemplary embodiment of the present invention; Figure 4 This is a schematic flowchart of a multi-camera collaborative camouflage target detection system provided by the present invention according to an exemplary embodiment. Detailed Implementation

[0022] To make the objectives, technical solutions, and advantages of the embodiments of this application clearer, the technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0023] Those skilled in the art will understand that, unless specifically stated otherwise, the singular forms “a,” “an,” “the,” and “the” used herein may also include the plural forms. It should be further understood that the term “comprising” as used in this specification means the presence of the stated features, integers, steps, operations, elements, and / or components, but does not exclude the presence or addition of one or more other features, integers, steps, operations, elements, components, and / or groups thereof.

[0024] First Embodiment like Figure 1 As shown, this embodiment provides a multi-depth camera collaborative method for detecting camouflaged targets, including the following steps: S1: Acquire data from each depth camera to obtain color images, depth maps, and timestamps of the current field of view.

[0025] In the camouflaged target detection method based on multi-depth camera collaboration, step S1 is the foundation for data acquisition in the entire technical solution. Its purpose is to provide accurate and synchronized raw visual input for subsequent single-view feature mining and multi-view data fusion. Specifically, each depth camera uses a global shutter type RGB-D sensor to ensure that color images and depth maps can be acquired simultaneously in high-speed motion or dynamic scenes, avoiding spatiotemporal misalignment caused by rolling shutters. At the same time, the system assigns a high-precision timestamp to each frame of acquired data (e.g., multi-device clock synchronization based on the IEEE 1588 precise time protocol). This timestamp is not only used to mark the data acquisition sequence, but also serves as a time reference for multi-camera data exchange in step S3 and cross-view feature alignment in step S4. Through timestamp matching, the system can filter out observation frames from different views at the same time, eliminating outdated data caused by communication delays or frame rate jitter, thereby ensuring the effectiveness of multi-view geometric constraints. Furthermore, this step involves preprocessing the acquired raw depth map, including distortion correction based on depth camera intrinsic parameters and removal of invalid points (such as pixels exceeding the measurement range or with excessively low reflectivity), to improve the quality and reliability of the depth data. In summary, S1, through rigorous hardware configuration, clock synchronization, and data preprocessing mechanisms, constructs a data foundation with spatial and temporal consistency for the entire camouflage target detection process, directly supporting the accuracy of local depth gradient map calculation in S2 and the precision of cross-view coordinate system alignment and depth projection discrepancy calculation in S4.

[0026] S2: Process the color image and depth map to be tested, extract features and depth gradient map, and output camouflage target information, which includes local 3D candidate positions and camouflage suspicion score.

[0027] In this embodiment, step S2 specifically includes: S21: Input the color image into a pre-trained camouflage target detection feature extraction network to obtain RGB region feature maps and preliminary camouflage target candidate boxes output by the network, and at the same time calculate the local depth gradient map of the depth map; like Figure 2 As shown, the feature extraction network used in step S21 pre-trained is a camouflage target detection feature extraction network pre-trained on a public camouflage target dataset. It is used to extract semantic features of the left and right color images and to concatenate them in the mid-to-high-level feature stages of the network to output the region feature map and the preliminary camouflage target candidate box. As a preferred example, the feature extraction network for camouflage target detection can adopt common visual models such as ResNet50 or ViT-B / 16, which are commonly used feature extraction backbone networks. S22: Using the preliminary camouflage target candidate box as the spatial constraint region, perform cross-modal coupling processing on the features of the corresponding region in the region feature map and the local depth gradient map to generate a spatial prior mask; In step S22, using the preliminary camouflage target candidate box as the spatial constraint region, cross-modal coupling processing is performed to generate a spatial prior mask, specifically including: Determine the spatial downsampling factor 's' of the region feature map relative to the original color image. Within the local region corresponding to the local depth gradient map where the initial camouflage target candidate box is mapped, divide non-overlapping spatial dimensionality reduction windows according to the downsampling factor 's'. ; For each of the aforementioned spatial dimensionality reduction windows Calculate the local dispersion index of all pixel gradient values ​​within the window, and generate a depth disorder matrix that is perfectly aligned with the spatial size of the RGB region feature map, where the depth disorder at coordinate point p is... The calculation formula is: , Display window The depth gradient value of the i-th pixel. Display window The mean of the depth gradient values ​​of all pixels within the range. Indicates the total number of pixels within the window; With the depth disorder matrix and the RGB region feature map constrained by the pixel coordinate system of the initial camouflage target candidate box, for each coordinate point p, the apparent semantic local smoothness of the RGB region feature corresponding to that coordinate point is extracted. , wherein Defined as the negative exponent of the variance of the eigenvectors in the neighborhood of coordinate point p. ; The local mutual exclusion index between the apparent semantic smoothness and the three-dimensional structural abrupt change region is calculated using the following formula. ; The global and local mutual exclusion index within the coverage area of ​​the initial camouflage target candidate box Perform Softmax normalization and output the spatial prior mask with values ​​ranging from 0 to 1. S23: The RGB features of the color image are weighted using the spatial prior mask to suppress feature responses consistent with the background depth and enhance texture features at depth boundaries; S24: Based on the enhanced feature map, perform refined extraction of the target region and 3D spatial regression, and combine the preliminary disguised target candidate box and depth flaw features to generate the initial 3D candidate position and disguise suspicion score.

[0028] In step S24, calculating and outputting the disguise suspicion score specifically includes: The ratio of the number of pixels falling within the spatial prior mask to the total number of pixels in the two-dimensional projection region of the initial 3D candidate position on the color image is calculated, and this ratio is used as a measure of physical anomalies. ; The confidence score of the two-dimensional object detection classification corresponding to the initial 3D candidate position is obtained as a two-dimensional appearance metric. ; Based on the physical anomaly measurement Constructing dynamic gain coefficients ,in The preset physical gain weight and ; The two-dimensional appearance measurement Multiply by the dynamic gain coefficient to generate the spoofing suspicion score. The calculation formula is as follows: .

[0029] In step S2, the core of this invention lies in fully exploring the inherent physical contradictions between the apparent features and three-dimensional structure of the camouflaged target from the perspective of a single depth camera, thereby generating preliminary camouflaged target information and providing reliable local observation basis for subsequent multi-camera collaborative judgment. Specifically, this step is not a simple end-to-end feature stitching of RGB images and depth maps, but rather a "coarse-to-fine" feature processing and fusion path is designed.

[0030] First, in S21, the system uses a feature extraction network (such as ResNet50 or ViT) pre-trained on a public camouflage dataset to semantically encode the color image, obtaining RGB region feature maps with high-level semantic information. Simultaneously, it outputs preliminary candidate boxes for camouflage targets, initially identified by the network, serving as spatial constraints for subsequent feature focusing. At the same time, it performs pixel-by-pixel gradient calculations on the depth map to obtain local depth gradient maps, characterizing abrupt changes in the 3D structure of the scene. This parallel processing strategy lays the data foundation for subsequently utilizing depth information to capture the "structural flaws" of camouflaged targets.

[0031] Subsequently, in S22, the system uses the initial candidate bounding boxes as spatial constraints to couple the RGB region feature maps with the local depth gradient maps across modalities. Its key innovation lies in constructing a local mutual exclusion index by calculating the dispersion (i.e., depth disorder) of the local depth gradient map and the apparent semantic smoothness of the RGB feature maps, and then generating a spatial prior mask via Softmax normalization. This mask can effectively locate regions with "smooth RGB apparent texture (similar to the background) but drastic changes in local depth gradients (existing independent 3D structures)," thereby accurately activating suspicious spatial locations of camouflaged targets while suppressing redundant feature responses consistent with the background depth.

[0032] Next, in S23, the generated spatial prior mask is used to weight and enhance the original RGB features, so that in subsequent processing, the network pays more attention to texture details with structural anomalies near the depth boundary, thereby improving the ability to perceive the edges and contours of the camouflaged target.

[0033] Finally, in S24, a refined 3D spatial regression is performed based on the enhanced feature map to output initial 3D candidate locations and calculate a comprehensive camouflage likelihood score. This score is ingeniously designed: it uses the "overlap ratio between the initial 3D candidate location and the spatial prior mask within the 2D projection region" as a physical anomaly metric, reflecting the degree of structural anomaly of the target in 3D space; simultaneously, it uses the classification confidence of the 2D target detection as an appearance metric. Finally, amplifying the appearance metric through a dynamic gain coefficient allows candidate regions with "general appearance confidence but significant physical anomalies" to obtain higher camouflage likelihood scores, effectively improving the sensitivity and recall rate for highly realistic camouflaged targets in a single viewpoint. This score directly drives the differentiated communication strategy in subsequent step S3, ensuring that suspicious camouflage clues can be fully cross-validated across multiple cameras.

[0034] S3: Each depth camera performs decentralized data exchange based on the output camouflage target information.

[0035] In this embodiment, step S3 specifically includes: S31: For regions corresponding to initial 3D candidate locations whose camouflage suspicion score is higher than the initial preset threshold, broadcast their local 3D spatial coordinates, local region feature vectors, and timestamps; for regular target regions whose camouflage suspicion score is lower than the initial preset threshold, only broadcast their category label, local 3D spatial coordinates, and timestamps.

[0036] S3 performs centerless data exchange for each depth camera based on the output camouflage target information. For regions corresponding to initial 3D candidate positions with camouflage suspicion scores higher than the initial preset threshold, it broadcasts their 3D spatial coordinates and local region feature vectors. For regular target regions with camouflage suspicion scores lower than the initial preset threshold, it only broadcasts their category labels and 3D spatial coordinates.

[0037] Step S3 and its sub-step S31 construct the differentiated decentralized communication mechanism of this invention, which is a crucial link in the entire multi-camera collaborative detection architecture. The initial design of this mechanism is to solve the "semantic asymmetry" problem unique to camouflage detection tasks: regular targets can be reliably identified with the two-dimensional detection confidence of a single viewpoint, and only lightweight category and coordinate information needs to be broadcast to meet the basic requirements of multi-camera collaborative perception; however, because the appearance features of camouflage targets are highly similar to the background in a single viewpoint, their two-dimensional detection confidence is often low. If only category labels and coarse coordinates are relied upon, other views will completely lose the semantic anchor points for cross-view feature matching and geometric consistency verification, causing camouflage clues to be discarded in the communication stage.

[0038] Based on the above considerations, S31 formulated a refined data broadcasting strategy. For the camouflage suspicion score calculated in step S2, the system set an initial preset threshold as the judgment criterion. When the camouflage suspicion score of an initial 3D candidate location is higher than this threshold, it means that although the two-dimensional appearance confidence of the area may be low, its three-dimensional structure is abnormally significant, possessing a high suspicion of camouflage. The system determines it as a "suspected camouflage target," and the camera broadcasts complete information including local 3D spatial coordinates, local region feature vectors, and timestamps. Among these, the local region feature vectors are the core data for cross-view feature soft alignment and overlapping mask extraction in subsequent step S42, directly determining the feasibility of multi-view collaborative verification. Conversely, when the camouflage suspicion score is lower than this threshold, the system classifies the target as a "regular target," broadcasting only its category label, local 3D spatial coordinates, and timestamp, thereby significantly saving communication bandwidth and computing resources while ensuring basic spatial location sharing.

[0039] Through this dynamic modulation communication strategy based on camouflage suspicion scores, S31 achieves two beneficial effects: Firstly, it ensures that the most critical camouflaged and suspicious areas requiring multi-view cross-verification can obtain sufficient high-dimensional feature sharing, providing necessary data support for the cross-view vulnerability index calculation and fake device reliability determination in the subsequent S4 step; secondly, it avoids the communication congestion and computational waste caused by indiscriminately broadcasting high-dimensional feature vectors across the entire scene, enabling the system to maintain efficient and stable collaborative perception capabilities even in complex and large-scale scenarios. This strategy fully reflects the invention's profound understanding of the characteristics of camouflage detection tasks, namely, fine-grained broadcasting in a few suspicious areas and lightweight notification in a large number of regular areas, providing an engineering-feasible communication solution for real-time collaborative detection of camouflaged targets by multiple mobile platforms in resource-constrained environments.

[0040] S4: Combine the feature results from multiple depth cameras to determine and output the detection results of camouflaged targets.

[0041] In this embodiment, step S4 specifically includes: S41: Receive local 3D spatial coordinates and local region feature vectors from other depth cameras, and based on the relative pose relationship between cameras, transform the 3D spatial coordinates of other viewpoints to the local coordinate system of the current local camera for coordinate system alignment. like Figure 3 As shown, S41 receives the local 3D spatial coordinates and local region feature vectors from other depth cameras. Based on the relative pose relationship between the cameras, it transforms the 3D spatial coordinates from other viewpoints to the local coordinate system of the current local camera for coordinate system alignment. Specifically, this includes: Based on multi-view scene feature association, the structure-of-motion method is used to estimate the relative pose relationships between depth cameras online. ,in This represents the transformation from the local coordinate system of viewpoint B to the coordinate system of viewpoint A. For rotation matrix, Let be the translation vector; where It can be obtained through the absolute poses of viewpoints A and B in the global coordinate system; When camera A receives local 3D spatial coordinates broadcast from another viewpoint B... At that time, using the relative pose relationship Transform it to the local coordinate system of camera A to obtain the transformed 3D spatial coordinates. : ; As an optional embodiment, the structure-of-motion (SfM) method is used to estimate the absolute pose of each depth camera relative to the SfM global coordinate system online, including: and ; The absolute poses of viewpoints A and B in the SfM global coordinate system are respectively, wherein, since depth vision is used to provide a real physical scale, the absolute poses are in metric units. The relative pose relationship between the local coordinate system of viewpoint B and the local coordinate system of viewpoint A is calculated by matrix inversion and multiplication. ,in: When camera A receives local 3D spatial coordinates broadcast from another viewpoint B... At that time, using the relative pose relationship Transform it to the local coordinate system of camera A to obtain the transformed 3D spatial coordinates. : .

[0042] S42: For the same 3D spatial region covered by at least two different viewpoints, use the 3D spatial coordinates transformed to the local coordinate system to determine the spatial distance to confirm the overlapping observation, and use the local region feature vector to perform cross-viewpoint feature soft alignment to extract the overlapping mask. In step S42, cross-view feature soft alignment is performed using the local region feature vector to extract the overlapping mask, specifically including: The dot product attention calculation is performed on the local region feature vectors of the current viewpoint and other viewpoints in the same 3D spatial region to obtain the cross-viewpoint feature similarity matrix. Using the feature similarity matrix as weights, the projected pixel coordinates under the current viewpoint are... The corresponding projection point set is weighted and filtered to remove outlier projection points with similarity below the alignment threshold. The retained set of projection points is rasterized to generate the overlapping mask.

[0043] S43: Calculate the cross-view vulnerability index and pseudo-device confidence based on the overlapping mask; In step S43, the vulnerability index across viewpoints is calculated using the following formula:

[0044] in The vulnerability index, and These are the local region feature vectors of the corresponding pixels in the overlapping mask for the current viewpoint A and the received viewpoint B. The feature cosine similarity function, is the preset bursting magnification factor, and N is the total number of overlapping pixels; The The cross-view depth projection inconsistency value of the i-th pixel is obtained through the following steps: From aligned 3D space homogeneous coordinates In the process, the Z-axis depth component in the current viewpoint A coordinate system is extracted as the predicted depth value. ; In the actual depth map of the current viewpoint A, the projected pixel coordinates are extracted, and the true depth value at that pixel coordinate in the actual depth map of the current viewpoint A is obtained. ; The absolute difference between the predicted depth value and the actual depth value is calculated and used as the cross-view depth projection discrepancy value. .

[0045] In step S43, calculating the reliability of the pseudo-device specifically includes: Extract the flaw index sequence within the historical sliding time window, including the current frame, and calculate the first-order time difference of the flaw index between adjacent frames to eliminate the constant depth bias introduced by the static calibration error of the relative pose of multiple cameras and obtain the dynamic change of flaws. The dynamic changes of the flaw within the historical sliding time window are monotonically accumulated in the same direction using a time decay factor to filter out the bidirectional differential disturbances caused by random oscillations in depth estimation and generate a flaw kinetic energy index. The vulnerability kinetic energy index is normalized to the probability space through a nonlinear mapping function to generate the pseudo-device confidence. Based on the reliability of the fake device, a threshold is used to determine whether it is a disguised target. If the value is higher than the threshold, it is confirmed as a disguised target; if the value is lower than the threshold, it is determined as a non-disguised target and is removed.

[0046] S44: Based on the reliability of the fake device, a threshold is used to determine whether it is a disguised target. If it is higher than the threshold, it is confirmed to be a disguised target; if it is lower than the threshold, it is determined to be a non-disguised target and is removed.

[0047] Step S4 is the core link of multi-view collaborative verification and final judgment in this invention. Its design purpose is to make full use of the observation redundancy of multiple depth cameras from different angles of the same scene. Through strict geometric alignment, feature matching and contradiction quantification, the "suspected" disguised target in the single view in step S2 is transformed into a highly reliable "confirmed" detection result, thereby completely solving the problem of high false detection caused by appearance similarity under single view.

[0048] In S41, the system first performs coordinate system alignment across cameras. After the local camera receives local 3D spatial coordinates and feature vectors broadcast from other viewpoints, it uses pre-estimated online relative pose relationships between cameras to accurately transform the 3D point coordinates from other viewpoints to the local coordinate system of the local camera. This alignment process relies on the real physical scale provided by depth vision, ensuring the accuracy of cross-view spatial mapping and laying a spatial benchmark for subsequent pixel-level geometric consistency verification.

[0049] In S42, the system performs overlapping observation confirmation and feature soft alignment for the same 3D spatial region covered by at least two viewpoints. First, spatial distance is determined using the transformed 3D coordinates to filter out spatially adjacent observation point pairs, confirming that they indeed correspond to the same physical entity. Then, a dot product attention mechanism based on local region feature vectors is introduced to calculate a cross-view feature similarity matrix. This matrix is ​​used to weight and filter the projected point set, eliminating low-similarity outliers caused by occlusion or viewpoint differences, ultimately generating a high-quality overlapping mask. This mask accurately identifies pixel regions that can be used for cross-view comparison.

[0050] In S43, the system calculates the flaw index and pseudo-device confidence based on overlapping masks, which is the most innovative core step in this process. The calculation of the flaw index cleverly integrates the cross-view depth projection contradiction value and feature similarity: when the local features of two views are highly similar, but the predicted depth of the corresponding 3D point in another view deviates significantly from the actual depth, the flaw index is dramatically amplified through an exponential amplification mechanism. This design accurately characterizes the unique attribute of camouflaged targets: "the more similar the apparent features, the more contradictory the spatial geometry," fundamentally breaking through the limitation of traditional geometric verification methods failing in camouflaged scenarios. Building on this, the system further introduces temporal information, extracting the flaw index sequence within the historical sliding window. By calculating the change between adjacent frames, it eliminates the constant bias caused by static calibration errors and uses a time decay factor to accumulate changes in the same direction, generating a "kinetic index" reflecting the evolution trend of the flaw. Finally, it normalizes this index into pseudo-device confidence in the probability space. This temporal processing mechanism effectively filters out noise generated by random oscillations in depth estimation, making the judgment results more robust.

[0051] Finally, in S44, the system performs a thresholding judgment based on the false device confidence level: candidate regions above a preset threshold are confirmed as disguised targets and output, while those below the threshold are judged as non-disguised targets and eliminated. Through the above four interconnected processing steps, S4 upgrades the initial suspected clues from a single perspective to a definitive conclusion verified by multi-perspective collaborative verification, achieving a significant improvement in detection accuracy and robustness.

[0052] Second Embodiment like Figure 4As shown, this embodiment provides a multi-depth camera cooperative camouflage target detection system 100 for performing the multi-depth camera cooperative camouflage target detection method as described in the first embodiment, comprising: The joint depth vulnerability detection module 110 is used to acquire and process color images and output camouflage target information, including 3D location and camouflage suspicion score; The feature communication exchange module 120 is used for each depth camera to perform decentralized data exchange based on the output camouflage target information; The multi-camera camouflage target determination module 130 is used to combine the feature results of multiple depth cameras and output the camouflage target detection results.

[0053] Third Embodiment This embodiment provides a multi-depth camera collaborative camouflage target detection device, including: at least two sets of movable machine devices equipped with depth cameras, a processor, and a memory; The depth camera is used to acquire color images of the current field of view; The memory contains the authorized computer program; When the processor executes the computer program, it implements the steps of the method as described in the first embodiment.

[0054] Optionally, the memory may include volatile memory, such as random-access memory (RAM), such as static random-access memory (SRAM), double-data-rate synchronous dynamic random-access memory (DDR SDRAM), etc.; the memory may also include non-volatile memory, such as flash memory. The memory is used to store computer programs (such as application programs, functional modules, etc. that implement the above methods), computer instructions, etc.

[0055] A processor is used to execute a computer program stored in memory to implement the various steps of the methods described in the above embodiments. For details, please refer to the relevant descriptions in the preceding method embodiments. The processor and memory can be independent structures or integrated structures. When the processor and memory are independent structures, the memory and processor can be coupled together via a bus.

[0056] The aforementioned computer programs, computer instructions, etc., can be stored in partitions within one or more memory locations. Furthermore, the aforementioned computer programs, computer instructions, data, etc., can be accessed by a processor.

[0057] Those skilled in the art will understand that embodiments of this application can be provided as methods, systems, or computer program products. Therefore, this application can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, this application can take the form of a computer program product embodied on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0058] This application is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of this application. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart... Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0059] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0060] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0061] The foregoing has described some specific embodiments of this application. It should be understood that this application is not limited to the specific embodiments described above, and those skilled in the art can make various modifications or variations within the scope of the claims, which do not affect the substantive content of this application. The above-described preferred features can be used in any combination without conflict.

Claims

1. A method for detecting camouflaged targets using multi-depth camera collaboration, characterized in that, Includes the following steps: S1: Acquire data from each depth camera to obtain color images, depth maps, and timestamps of the current field of view; S2: Process the color image and depth map to be tested, extract features and depth gradient map, and output camouflage target information, which includes local 3D candidate positions and camouflage suspicion score. S3: Each depth camera performs decentralized data exchange based on the output camouflage target information; S4: Combine the feature results from multiple depth cameras to determine and output the detection results of camouflaged targets.

2. The multi-depth camera cooperative camouflage target detection method according to claim 1, characterized in that, Step S2 is as follows: S21: Input the color image into a pre-trained camouflage target detection feature extraction network to obtain RGB region feature maps and preliminary camouflage target candidate boxes output by the network, and at the same time calculate the local depth gradient map of the depth map; S22: Using the preliminary camouflage target candidate box as the spatial constraint region, perform cross-modal coupling processing on the features of the corresponding region in the region feature map and the local depth gradient map to generate a spatial prior mask; S23: The RGB features of the color image are weighted using the spatial prior mask to suppress feature responses consistent with the background depth and enhance texture features at depth boundaries; S24: Based on the enhanced feature map, perform refined extraction of the target region and 3D spatial regression, and combine the preliminary disguised target candidate box and depth flaw features to generate the initial 3D candidate position and disguise suspicion score.

3. The multi-depth camera collaborative camouflage target detection method according to claim 1, characterized in that, Step S3 is as follows: S31: For regions corresponding to initial 3D candidate locations whose camouflage suspicion score is higher than the initial preset threshold, broadcast their local 3D spatial coordinates, local region feature vectors, and timestamps; for regular target regions whose camouflage suspicion score is lower than the initial preset threshold, only broadcast their category label, local 3D spatial coordinates, and timestamps.

4. The multi-depth camera collaborative camouflage target detection method according to claim 1, characterized in that, Step S4 is as follows: S41: Receive local 3D spatial coordinates and local region feature vectors from other depth cameras, and based on the relative pose relationship between cameras, transform the 3D spatial coordinates of other viewpoints to the local coordinate system of the current local camera for coordinate system alignment. S42: For the same 3D spatial region covered by at least two different viewpoints, use the 3D spatial coordinates transformed to the local coordinate system to determine the spatial distance to confirm the overlapping observation, and use the local region feature vector to perform cross-viewpoint feature soft alignment to extract the overlapping mask. S43: Calculate the cross-view vulnerability index and pseudo-device confidence based on the overlapping mask; S44: Based on the reliability of the fake device, a threshold is used to determine whether it is a disguised target. If it is higher than the threshold, it is confirmed to be a disguised target; if it is lower than the threshold, it is determined to be a non-disguised target and is removed.

5. The multi-depth camera collaborative camouflage target detection method according to claim 2, characterized in that, The camouflage target detection feature extraction network used in step S21 is pre-trained on a public camouflage target dataset to extract semantic features from the left and right color images, and is spliced ​​in the middle and high-level feature stages of the network to output the region feature map and the preliminary camouflage target candidate box. In step S22, using the preliminary camouflage target candidate box as the spatial constraint region, cross-modal coupling processing is performed to generate a spatial prior mask, specifically including: Determine the spatial downsampling factor 's' of the region feature map relative to the original color image. Within the local region corresponding to the local depth gradient map where the initial camouflage target candidate box is mapped, divide non-overlapping spatial dimensionality reduction windows according to the downsampling factor 's'. ; For each of the aforementioned spatial dimensionality reduction windows Calculate the local dispersion index of all pixel gradient values ​​within the window, and generate a depth disorder matrix that is perfectly aligned with the spatial size of the RGB region feature map, where the depth disorder at coordinate point p is... The calculation formula is: ， Display window The depth gradient value of the i-th pixel. Display window The mean of the depth gradient values ​​of all pixels within the range. Indicates the total number of pixels within the window; With the depth disorder matrix and the RGB region feature map constrained by the pixel coordinate system of the initial camouflage target candidate box, for each coordinate point p, the apparent semantic local smoothness of the RGB region feature corresponding to that coordinate point is extracted. , wherein Defined as the negative exponent of the variance of the eigenvectors in the neighborhood of coordinate point p. ; The local mutual exclusion index between the apparent semantic smoothness and the three-dimensional structural abrupt change region is calculated using the following formula. ; The global and local mutual exclusion index within the coverage area of ​​the initial camouflage target candidate box Perform Softmax normalization and output the spatial prior mask with values ​​ranging from 0 to 1. In step S24, calculating and outputting the disguise suspicion score specifically includes: The ratio of the number of pixels falling within the spatial prior mask to the total number of pixels in the two-dimensional projection region of the initial 3D candidate position on the color image is calculated, and this ratio is used as a measure of physical anomalies. ; The confidence score of the two-dimensional object detection classification corresponding to the initial 3D candidate position is obtained as a two-dimensional appearance metric. ; Based on the physical anomaly measurement Constructing dynamic gain coefficients ,in The preset physical gain weight and ; The two-dimensional appearance measurement Multiply by the dynamic gain coefficient to generate the spoofing suspicion score. The calculation formula is as follows: 。 6. The multi-depth camera collaborative camouflage target detection method according to claim 1, characterized in that, Step S41, based on the pre-established relative pose relationship between cameras, transforms the 3D spatial coordinates of other viewpoints to the local coordinate system of the current local camera, specifically including: Based on multi-view scene feature association, the structure-of-motion method is used to estimate the relative pose relationships between depth cameras online. ,in This represents the transformation from the local coordinate system of viewpoint B to the coordinate system of viewpoint A. For rotation matrix, Let be the translation vector; where It can be obtained through the absolute poses of viewpoints A and B in the global coordinate system; When camera A receives local 3D spatial coordinates broadcast from another viewpoint B... At that time, using the relative pose relationship Transform it to the local coordinate system of camera A to obtain the transformed 3D spatial coordinates. : ； In step S42, cross-view feature soft alignment is performed using the local region feature vector to extract the overlapping mask, specifically including: The dot product attention calculation is performed on the local region feature vectors of the current viewpoint and other viewpoints in the same 3D spatial region to obtain the cross-viewpoint feature similarity matrix. Using the feature similarity matrix as weights, the projected pixel coordinates under the current viewpoint are... The corresponding projection point set is weighted and filtered to remove outlier projection points with similarity below the alignment threshold. The retained set of projection points is rasterized to generate the overlapping mask.

7. The multi-depth camera cooperative camouflage target detection method according to claim 6, characterized in that, In step S43, the vulnerability index across viewpoints is calculated using the following formula: in The vulnerability index, and These are the local region feature vectors of the corresponding pixels in the overlapping mask for the current viewpoint A and the received viewpoint B. The feature cosine similarity function, is the preset bursting magnification factor, and N is the total number of overlapping pixels; The The cross-view depth projection inconsistency value of the i-th pixel is obtained through the following steps: From aligned 3D space homogeneous coordinates In the process, the Z-axis depth component in the current viewpoint A coordinate system is extracted as the predicted depth value. ; In the actual depth map of the current viewpoint A, the projected pixel coordinates are extracted, and the true depth value at that pixel coordinate in the actual depth map of the current viewpoint A is obtained. ; The absolute difference between the predicted depth value and the actual depth value is calculated and used as the cross-view depth projection discrepancy value. .

8. The multi-depth camera cooperative camouflage target detection method according to claim 1 or 7, characterized in that, In step S43, calculating the reliability of the pseudo-device specifically includes: Extract the flaw index sequence within the historical sliding time window, including the current frame, and calculate the first-order time difference of the flaw index between adjacent frames to eliminate the constant depth bias introduced by the static calibration error of the relative pose of multiple cameras and obtain the dynamic change of flaws. The dynamic changes of the flaw within the historical sliding time window are monotonically accumulated in the same direction using a time decay factor to filter out the bidirectional differential disturbances caused by random oscillations in depth estimation and generate a flaw kinetic energy index. The vulnerability kinetic energy index is normalized to the probability space through a nonlinear mapping function to generate the pseudo-device confidence. Based on the reliability of the fake device, a threshold is used to determine whether it is a disguised target. If the value is higher than the threshold, it is confirmed as a disguised target; if the value is lower than the threshold, it is determined as a non-disguised target and is removed.

9. A multi-depth camera cooperative camouflage target detection system for performing the multi-depth camera cooperative camouflage target detection method as described in any one of claims 1-7, characterized in that, include: The joint depth vulnerability discovery module is used to acquire and process color images and output camouflage target information, including 3D location and camouflage suspicion score; The feature communication exchange module is used by each depth camera to perform decentralized data exchange based on the output camouflage target information; The multi-camera camouflage target determination module combines feature results from multiple depth cameras to output camouflage target detection results.

10. A multi-depth camera collaborative camouflage target detection device, characterized in that, include: At least two sets of mobile mechanical devices equipped with depth cameras, processors, and memory; The depth camera is used to acquire color images of the current field of view; The memory contains the authorized computer program; When the processor executes the computer program, it implements the steps of the method as described in any one of claims 1 to 9.

11. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores computer code, and when the computer code is executed, the method as described in any one of claims 1 to 9 is performed.

Images