Cross-camera retrieval method and system for obscured targets in border inspection

Abstract

This invention relates to the field of image recognition technology, specifically to a method and system for cross-camera retrieval of occluded targets in border inspection. The method involves acquiring target images, extracting spatial feature maps and global descriptors, generating confidence maps and uncertainty maps based on the contribution strength of each spatial location to identity determination, and applying soft gating to the spatial feature maps based on the uncertainty maps to perform residual correction on low-confidence regions using neighborhood context. During the training phase, cross-camera neighbors are retrieved for samples of the same target from different cameras. The feature maps are divided into multiple regions for soft alignment, and complementary cues are injected only into low-confidence regions according to uncertainty strength, jointly training the feature extraction network. During the retrieval phase, descriptors for the query and the target in the image library are extracted, and cross-camera nearest neighbor structure correction is performed on the image library descriptors. Similarity and scene cues are fused to output ranking results. By explicitly compensating across cameras, missing cues caused by occlusion are accurately supplemented, improving retrieval accuracy.

Description

Technical Field

[0001] This invention relates to the field of image recognition technology, specifically to a method and system for cross-camera retrieval of occluded targets in border inspection. Background Technology

[0002] The statements in this section are merely background information related to the present invention and do not necessarily constitute prior art.

[0003] Multiple surveillance cameras are typically deployed at border checkpoints, duty lanes, and vehicle inspection areas to enable retrospective inspection and trajectory tracking with the support of image recognition technology. Due to the high density of people, the large amount of luggage and goods, and the complexity of the facilities, targets are often partially obscured by other objects, resulting in only a localized area being retained in the image.

[0004] For target retrieval under occlusion conditions, existing technologies generally locate visible regions through human pose estimation or part analysis, followed by local feature alignment. However, the external models relied upon are prone to errors in low-resolution or crowded scenes. Alternatively, attention mechanisms or feature erasure can be used to enhance visible features within a single image, mitigating occlusion interference to some extent. Another approach attempts to utilize complementary cross-camera information of the same target from different cameras: for example, some schemes perform feature fusion or knowledge distillation on multiple images of the same target during the training phase, enabling single-image features to acquire multi-view information.

[0005] However, existing cross-camera complementation methods typically perform simple averaging or fusion of global features from multiple images, ignoring the reliability differences of local regions under different viewpoints. This leads to severely occluded areas being treated equally with unoccluded areas, diluting the discriminative information for reliable regions. Simultaneously, existing cross-camera complementation methods often align images from different cameras at fixed spatial positions. However, the pose, scale, and viewpoint differences between cross-camera images are significant, and fixed-position alignment can easily mismatch different semantic regions, introducing background or occlusion noise. Therefore, existing cross-camera information utilization methods struggle to accurately and effectively supplement truly missing local regions in the current image, limiting further improvements in retrieval accuracy under occlusion conditions. Summary of the Invention

[0006] This invention provides a method and system for cross-camera retrieval of obscured targets in border inspection, which can accurately determine the reliability of local areas and precisely inject complementary clues from cross-camera neighbors into low-confidence areas according to semantic correspondence, thus solving the problem of cross-camera retrieval of obscured targets in border inspection scenarios.

[0007] To achieve the above objectives, the present invention adopts the following technical solution:

[0008] The first aspect of the present invention provides a method for cross-camera retrieval of obscured targets in border inspection, comprising the following steps:

[0009] Acquire target images during border inspection and extract their spatial feature maps and global descriptors;

[0010] Based on the strength of each spatial location's contribution to target identity determination in the spatial feature map, a local evidence credibility map is generated, and a local uncertainty map is obtained using the complementary quantities of the credibility map. Based on the local uncertainty map, context correction features are extracted from the spatial feature map and continuously fused with the original spatial feature map to obtain a recalibrated feature map.

[0011] During the training phase, cross-camera neighbors are retrieved for training samples from different camera sources under the same target identifier. The recalibrated feature maps of the current sample and its neighbors are divided into multiple local region blocks. The similarity matrix between each region block is calculated and soft alignment is performed. Based on the uncertainty intensity of the region block corresponding to the current sample, complementary cues from cross-camera neighbors are injected into the low-confidence region blocks to obtain the compensation feature map. A joint training objective is constructed based on the global target descriptor, the recalibrated feature map, and the compensation feature map to train the feature extraction network.

[0012] During the retrieval phase, the trained feature extraction network is used to extract descriptors of the query target and the image library target. Cross-camera nearest neighbor structure correction is performed on the image library descriptors, and candidate ranking results are output by combining direct similarity, corrected similarity and scene category clues.

[0013] Furthermore, a local evidence credibility map is generated, specifically by: calculating the category discrimination response using the identity discrimination weight and the feature vectors of each spatial location in the spatial feature map, and then spatially normalizing the category discrimination response to obtain the credibility map; the credibility map is used to represent the effective contribution of local regions to the target identity retrieval.

[0014] Furthermore, the context correction features are obtained by sequentially performing depthwise separable convolution, pointwise convolution, and normalization on the original spatial features; continuous fusion is performed between the original spatial features and the context correction features based on the local uncertainty map, and the replacement intensity is limited by learnable correction coefficients.

[0015] Furthermore, the recalibrated feature maps of the current sample and its neighboring samples are divided into multiple local region blocks. The similarity matrix between each region block is calculated and soft alignment is performed. Specifically, local region blocks are obtained by adaptive grid pooling, strip pooling, or multi-scale pooling. The similarity matrix is ​​calculated after normalizing the current sample region block and the cross-shot neighbor region blocks. Then, the similarity matrix is ​​normalized and weighted in the dimension of the neighbor region blocks to obtain the complementary region block that corresponds to the semantics of the current region block.

[0016] Furthermore, cross-camera neighbor retrieval is limited to candidate sets of images with the same target identifier, different camera numbers, and not their own images; when there are no effective cross-camera neighbors or the quality of candidate neighbors is lower than a preset threshold, the local region block of the current sample remains unchanged, and the sample participates in training only through identity discrimination, retrieval metrics, and local recalibration related losses.

[0017] Furthermore, based on the uncertainty intensity of the region block corresponding to the current sample, complementary cues from cross-camera neighbors are injected into the low-confidence region block. Specifically, the higher the uncertainty intensity, the greater the proportion of injected complementary cues; the lower the uncertainty intensity, the greater the proportion of the original observation features of the current sample that are retained.

[0018] Furthermore, the joint training objectives include identity classification loss, triple ranking loss, local compensation feature identity loss, and descriptor consistency constraint. The descriptor consistency constraint is used to limit the offset between the main retrieval path descriptor and the training auxiliary path descriptor in the identity semantic space, so that the occlusion robustness obtained in the training stage is retained in the main retrieval path in the actual retrieval stage.

[0019] Furthermore, cross-camera nearest neighbor structure correction is performed on the image library descriptors. Specifically, for each image library descriptor, nearest neighbors are retrieved in image library samples from different cameras or different locations. Nearby weights are calculated based on temperature scaling similarity. The nearest neighbor descriptors are weighted, summed, and normalized to obtain the corrected image library descriptor. The structure correction process does not use test identity labels.

[0020] Furthermore, scene category clues also include camera location connectivity and acquisition time interval; the fusion score is obtained by weighting direct similarity, corrected similarity, location connectivity, time interval, and scene type matching relationship; a validity flag and reliability coefficient are set for each metadata item, and when the corresponding metadata is missing, the weight of the item is reset to zero and the weights of the remaining valid clues are renormalized.

[0021] A second aspect of the present invention provides a system for implementing the above-described method, comprising:

[0022] The video image access and feature extraction unit is configured to: acquire target images during border inspection and extract their spatial feature maps and global descriptors;

[0023] The credibility assessment and recalibration unit is configured to generate a local evidence credibility map and a local uncertainty map based on the strength of the contribution of each spatial location in the spatial feature map to the target identity discrimination, and to continuously fuse the original spatial features and context correction features based on the local uncertainty map to obtain a recalibrated feature map.

[0024] The cross-camera compensation training unit is configured to: during the training phase, retrieve cross-camera neighbors for training samples from different camera sources under the same target identifier, perform soft alignment of region blocks and injection of complementary cues, and construct a joint training target training feature extraction network;

[0025] The retrieval output unit is configured to: extract descriptors of the query target and the image library target during the retrieval phase, perform cross-camera nearest neighbor structure correction of the image library, and output candidate ranking results by fusing similarity and scene category clues.

[0026] Compared with existing technologies, one or more of the above technical solutions have the following beneficial effects:

[0027] 1. By generating a local evidence credibility map through identity discrimination response, it distinguishes between high credibility regions that make stable contributions to target identity discrimination and low credibility regions affected by occlusion. It does not require external pose estimation or human body parsing model, avoids error propagation, and continuous weights are more suitable than binary masks for the transition state of partial visibility and partial occlusion in real occlusion scenarios.

[0028] 2. By using cross-camera neighbor retrieval, local region block division, and similarity soft alignment, missing clues in the current image are located from different camera images of the same target according to semantic correspondence. Only low-confidence regions are supplemented, avoiding clue dilution caused by global averaging and semantic misalignment caused by fixed position alignment. It can effectively compensate for information loss caused by occlusion while retaining existing reliable identity clues, and improve the accuracy of cross-camera target retrieval in severely occluded scenarios.

[0029] 3. By limiting the offset between the main retrieval path and the training auxiliary path in the identity semantic space through descriptor consistency constraints, the occlusion robustness obtained through cross-camera compensation during the training phase can be transferred to the main retrieval path in the actual retrieval phase. Moreover, the cross-camera compensation unit is no longer loaded after training is completed, maintaining the lightweight nature of the deployment phase. Attached Figure Description

[0030] The accompanying drawings, which form part of this invention, are used to provide a further understanding of the invention. The illustrative embodiments of the invention and their descriptions are used to explain the invention and do not constitute an improper limitation of the invention.

[0031] Figure 1 This is a flowchart illustrating the method for cross-camera retrieval of obscured targets in border inspection provided by the present invention.

[0032] Figure 2 A schematic diagram illustrating the process of duty video access, personnel detection and cropping, and metadata binding provided by this invention;

[0033] Figure 3 A schematic diagram of the local evidence credibility assessment and local evidence recalibration process provided by the present invention;

[0034] Figure 4 A schematic diagram of the cross-camera local evidence compensation process during the training phase provided by the present invention.

[0035] Figure 5 This is a schematic diagram of the library structure correction and multi-clue fusion sorting process provided by the present invention;

[0036] Figure 6 This is a schematic diagram of the system module deployment structure provided by the present invention. Detailed Implementation

[0037] The present invention will be further described below with reference to the accompanying drawings and embodiments.

[0038] It should be noted that the following detailed descriptions are exemplary and intended to provide further illustration of the invention. Unless otherwise specified, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this invention pertains.

[0039] "Retrieval" typically involves two steps: first, image recognition, which involves extracting distinctive appearance features from a single query image; and second, feature association and matching, which involves comparing the features of the query image with the features of a large number of target images in a historical image library, sorting the results by similarity, and outputting candidate results for subsequent verification by auditors.

[0040] In cross-camera retrieval scenarios, ideal feature association should be able to comprehensively utilize complementary appearance information of the same target captured by different cameras to compensate for the lack of discriminative clues in a single query image due to occlusion. However, as introduced in the background section, existing methods of utilizing cross-camera information typically involve averaging, fusion, or knowledge distillation at the global feature level, or aligning images from different cameras at fixed spatial locations. The former cannot distinguish whether each local region is reliable, resulting in noise in occluded areas being treated the same as effective information in unoccluded areas, diluting the discriminative power of the features; the latter, due to significant differences in pose, scale, and viewpoint between cross-camera images, can easily lead to incorrect mapping of local regions with different semantics (e.g., aligning the upper garment area of ​​the current image to the background or occluder in a neighboring image), thus introducing interference information. Therefore, existing methods of utilizing cross-camera information struggle to accurately and effectively supplement the truly missing local regions in the current image, limiting further improvements in retrieval accuracy under occlusion conditions.

[0041] This solution proposes a cross-camera target retrieval method and system for occluded targets in border inspection, expanding target retrieval under occlusion conditions from "single-image matching" to "cross-camera collaboration." It generates a local evidence credibility map through identity discrimination responses, distinguishing between high-credibility regions that consistently contribute to identity discrimination and low-credibility regions affected by occlusion. During the training phase, cross-camera neighbors are retrieved from samples of the same target from different cameras. The feature map is divided into local region blocks, and semantic soft alignment is performed based on region block similarity. Complementary cues from neighbors are injected only into the low-credibility regions of the current sample. During deployment, only a lightweight main retrieval path is retained, and ranking quality is further improved through graph library structure correction.

[0042] Once trained, this solution improves retrieval accuracy under occlusion conditions by explicitly injecting complementary cues from cross-camera neighbors, based on semantic correspondences, only into low-confidence local regions of the current image. This helps users better perform target tracing and trajectory correlation tasks. Users of this solution can be authorized police officers performing border inspection duties, provided they obtain permission from relevant authorities and complete data security approval before accessing business video data and using the retrieval functions provided by this solution.

[0043] A method for cross-camera retrieval of targets obscured by border inspection includes the following steps:

[0044] Acquire target images during border inspection and extract their spatial feature maps and global descriptors;

[0045] Based on the strength of each spatial location's contribution to target identity determination in the spatial feature map, a local evidence credibility map is generated, and a local uncertainty map is obtained using the complementary quantities of the credibility map. Based on the local uncertainty map, context correction features are extracted from the spatial feature map and continuously fused with the original spatial feature map to obtain a recalibrated feature map.

[0046] During the training phase, cross-camera neighbors are retrieved for training samples from different camera sources under the same target identifier. The recalibrated feature maps of the current sample and its neighbors are divided into multiple local region blocks. The similarity matrix between each region block is calculated and soft alignment is performed. Based on the uncertainty intensity of the region block corresponding to the current sample, complementary cues from cross-camera neighbors are injected into the low-confidence region blocks to obtain the compensation feature map. A joint training objective is constructed based on the global target descriptor, the recalibrated feature map, and the compensation feature map to train the feature extraction network.

[0047] During the retrieval phase, the trained feature extraction network is used to extract descriptors of the query target and the image library target. Cross-camera nearest neighbor structure correction is performed on the image library descriptors, and candidate ranking results are output by combining direct similarity, corrected similarity and scene category clues.

[0048] This solution generates a local evidence credibility map through identity discrimination response, distinguishing between high-credibility regions that consistently contribute to identity discrimination and low-credibility regions affected by occlusion. Then, it accurately locates missing clues in the current image from cross-camera neighbors based on semantic correspondence, supplementing only low-credibility regions. Through on-demand and semantically explicit compensation, it effectively compensates for information loss caused by occlusion while preserving existing reliable identity clues, significantly improving the accuracy of cross-camera target retrieval in severely occluded scenarios.

[0049] The following example illustrates the retrieval of obscured individuals across cameras during border inspection duties. In practical applications, inputs can come from fixed video surveillance, mobile duty terminals, vehicle-mounted video, law enforcement recorders, offline historical video, or real-time captured images; the output results are used to assist users in video playback, linking personnel trajectories, and verifying identity clues.

[0050] The algorithm in this scheme can first be trained, validated, and its parameters determined on a legally publicly available personnel re-identification dataset.

[0051] As a non-restrictive approach, publicly available datasets such as Occluded-DukeMTMC, P-DukeMTMC-reID, Occluded-REID, Market-1501, and MSMT17 can be used. These datasets cover training scales of approximately 100 to 1041 training identities and 1000 to 32621 training images, capable of validating trainability under conditions ranging from small-scale occlusion data to large-scale complex camera data. Publicly available data should be processed in accordance with dataset licensing agreements, research ethics requirements, and the scope of data use.

[0052] When implementing border inspection operations, if scenario adaptation or incremental training is required, written authorization from the relevant authorities should be obtained, the processing purpose and data scope should be clearly defined, and security approval should be completed before constructing an internal training set from historical videos or an authorized capture library. Internal training samples may only store cropped images of personnel, anonymous person_ids, camera numbers, trajectory numbers, capture time, and quality markers; direct identity information such as names and ID numbers should not be used as network input. Data processing, training, and model export can all be completed within the internal security domain, with tiered access, encrypted storage, operation auditing, version tracking, and expiration deletion mechanisms in place.

[0053] For any two training samples i and j, when their anonymized person identifiers satisfy The camera number meets the requirements. and Cross-camera compensation can be defined as candidate pairings of the same person across different cameras. Pairings in public datasets are obtained directly from their identity labels and camera labels; pairings in internal videos can be confirmed by continuity of the same shot trajectory, spatiotemporal connection of adjacent cameras, association of authorized business records, manual double-checking, or a combination of the above methods. Only samples that meet the preset confirmation conditions will enter cross-camera compensation training; samples that cannot be confirmed or have conflicts will only be used for unpaired identity classification training or will be directly eliminated. In the pilot phase, hundreds to thousands of anonymous person identifiers and approximately ten thousand to one hundred thousand cropped images can be used. Each identifier participating in cross-camera compensation preferably has at least two camera sources and at least one qualified image from each source.

[0054] like Figure 1 As shown, the method for cross-camera retrieval of occluded targets provided in this embodiment includes the following steps:

[0055] Step 1: Access duty video / captured images.

[0056] Images or video frames are acquired from video sources such as port waiting areas, inspection channels, vehicle inspection areas, border roads, checkpoints, patrol vehicles, and law enforcement recorders. Time synchronization, device identification resolution, and sample quality screening are then performed. For each video stream, the system obtains a bounding box of personnel through a personnel detection and tracking component. The bounding box undergoes boundary correction, scaling, size standardization, and quality checks. Each target is then associated with its camera number, location, acquisition time, scene category, and video segment identifier.

[0057] Step 2: Personnel detection, trimming, and metadata binding.

[0058] The system inputs the normalized images of people into a feature extraction network to obtain a spatial feature map F and a global person descriptor v. The spatial feature map preserves local region information, while the global descriptor is used for rapid similarity calculation between the query image and images in the image library. The feature extraction network can employ a convolutional neural network, a visual Transformer, or a combination of both, as long as it can output spatial features and a global descriptor.

[0059] Step 3: Feature extraction and local credibility assessment.

[0060] During the training phase, the system generates a local evidence credibility map based on the identity discrimination weights and the response strength of feature vectors at each location in the spatial feature map. and order This serves as a local uncertainty map. High confidence indicates that the region contributes consistently to personnel identification, while low confidence indicates that the region may be affected by occlusion, background, detection box offset, or low resolution. This confidence map uses continuous weights and is not a hard binary labeling of occluded regions.

[0061] Step 4: Local evidence recalibration.

[0062] The system sequentially performs 3×3 depthwise separable convolution, 1×1 pointwise convolution, and normalization on the original spatial features through a local context extraction unit to generate context correction features H. Then, it continuously fuses the original spatial features F and the context correction features H according to the uncertainty graph U, and performs residual interpolation using bounded correction coefficients obtained by mapping trainable parameters through the Sigmoid function to obtain the recalibrated features F'. High-confidence regions primarily retain the original identity features, while low-confidence regions receive more neighborhood context correction information.

[0063] Step 5: Training phase, cross-camera local evidence compensation.

[0064] For training samples confirmed to belong to the same person from different cameras, the system employs P×K grouping sampling. Within the same person group in the current training batch, a preset number of Top-K cross-camera neighbors are retrieved based on the global descriptor, without performing full pairwise matching on the entire training set. Subsequently, the recalibrated features of the current sample and neighboring samples are compressed into a fixed number of local region blocks using adaptive grid pooling. The region block similarity matrix is ​​calculated and soft alignment is performed. The system only supplements low-confidence regions according to the uncertainty intensity of the current region block, limiting the computational load during the training phase from three dimensions: candidate range, number of neighbors, and number of region blocks.

[0065] Retrieval model training. Original spatial feature map. The main path descriptor is generated from the main search header. Then it enters the main identity classifier; the feature map is recalibrated. It is used only as input for cross-shot local region block alignment and compensation, without a separate classification head; compensation feature map Compensation descriptors are generated through pooling, normalization, and independent auxiliary heads. The data is then fed into the auxiliary identity classifier. During training, the shared feature extraction network is jointly optimized using the main path identity classification loss, the compensated auxiliary path identity classification loss, and the main / auxiliary descriptor consistency loss. The compensated auxiliary path is only used during the training phase; cross-camera compensation units and auxiliary heads are not loaded during the deployment phase.

[0066] Step 6: Duty search phase, image library structure correction.

[0067] During deployment, the system extracts global descriptors from images of queried personnel and images of personnel in the image library. For each descriptor in the image library, the system searches for nearest neighbors only in image library samples from different cameras or different locations, and then weights and aggregates the nearest neighbor descriptors to obtain the corrected image library descriptor. The correction process does not use test identity tags and can be pre-calculated offline or updated periodically.

[0068] Step 7: Multi-clue fusion and candidate output.

[0069] The system calculates the direct appearance similarity between the query descriptor and the original image library descriptor, and the structural correction similarity between the query descriptor and the corrected image library descriptor. It then combines these with camera location connectivity, acquisition time intervals, and scene category clues to form a comprehensive score. The system records validity indicators and reliability coefficients for each metadata item: when metadata is missing, the corresponding item is not used and the remaining clue weights are renormalized; when metadata is unreliable, the corresponding weight is reduced or the tolerance range is expanded, enabling the system to automatically degenerate into a retrieval mode primarily based on visual similarity.

[0070] Step 8: User review and business closure.

[0071] The system outputs candidate personnel, associated frequency bands, and descriptions of similar areas to assist in verification and judgment. This embodiment outputs a ranked list of candidates, along with corresponding video clips, capture times, camera locations, similar area prompts, and verification instructions.

[0072] The following section describes this solution in detail with reference to the accompanying drawings.

[0073] like Figure 2 The diagram shows the process of accessing duty videos, personnel detection and cropping, and metadata binding (corresponding to steps 1 and 2).

[0074] The system first connects to border inspection video sources. For fixed cameras, the system records the camera number, location name, area, viewing angle, and scene category; for mobile video sources, the system records the device number, acquisition time, and task number. Scene categories may include waiting halls, inspection lanes, vehicle inspection areas, border roads, checkpoints, patrol routes, and camp perimeters.

[0075] For each video stream, the system uses a person detection algorithm to obtain person bounding boxes and can combine this with a multi-object tracking algorithm to generate short-term trajectories. Person detection and multi-object tracking are general input preprocessing components for forming cropped samples, removing duplicate frames, and binding trajectory metadata; their outputs are only bounding boxes or trajectories and do not participate in the calculation of local confidence maps, soft alignment of region blocks, or compensation weights. The phrase "not dependent on external pose estimation or human parsing models" in this paper means that local evidence confidence assessment and cross-shot compensation do not use human keypoints, human parsing maps, or additional visibility annotations as input, and does not preclude the retrieval system from using general object detection components. Multi-object tracking is an optional component; if the upstream platform has provided cropped images of people and metadata such as camera and time, this system can directly receive them without performing detection or tracking. To reduce the impact of bounding box truncation, the system can appropriately expand the person bounding boxes; when the expanded bounding boxes exceed the image boundaries, boundary correction is performed, and quality screening can be performed based on the bounding box area, aspect ratio, blurriness, and similarity to duplicate frames.

[0076] In video playback scenarios, the system can select several clear or representative frames from the same trajectory segment and store them in the database, avoiding redundancy caused by writing a large number of similar frames into the database. In real-time deployment or temporary search scenarios, the system can directly use user-provided captured images or video clips as query input.

[0077] The accessed video data is parsed and associated with the camera number, location name, and channel information. Combined with the generated short-time trajectory, non-person targets and repetitive segments are filtered out. The data is then cropped according to the detection box to remove samples that are too small, blurry, or severely distorted, resulting in normalized image data.

[0078] The normalized images of people are adjusted to a predetermined resolution, such as 256×128 pixels or other sizes suitable for people retrieval networks. The system performs pixel normalization on the images, and during the training phase, it can also use enhancement methods such as horizontal flipping, random erasing, brightness perturbation, and slight cropping to simulate occlusion, lighting changes, and image jitter in port and border scenes.

[0079] The video sources in border inspection scenarios are diverse (fixed surveillance cameras, mobile terminals, law enforcement recorders), and the image quality varies greatly. Through detection, cropping, quality filtering, and metadata binding, the raw videos are transformed into structured samples in a unified format, providing clean and standardized input for subsequent feature extraction. Simultaneously, binding metadata such as camera ID, location, and time provides a data foundation for metadata fusion and ranking during the retrieval phase.

[0080] The feature extraction network can employ ResNet-50, lightweight convolutional networks, visual Transformers, or networks specifically optimized for video structuring platforms. Let the input image be I, and the feature extraction network output a spatial feature map. Where C is the number of channels, and H and W are the spatial dimensions. The global personnel descriptor v is obtained after pooling, normalizing, and mapping F.

[0081] While saving personnel descriptors, the system also saves metadata such as camera ID (cam_id), location (point_id), acquisition time (time), scene category (scene), video clip (clip_id), and trajectory (track_id) to form a retrieval sample. This sample is then written to the query database / image library for subsequent cross-camera retrieval. This metadata does not replace the visual retrieval results, but it can be used during the candidate ranking stage to constrain candidates that clearly do not conform to spatial and temporal logic.

[0082] This approach maps target personnel images into spatial feature maps (preserving local spatial information) and global descriptors (for fast retrieval) using a feature extraction network. The specific network structure is not limited to ensure the universality and replaceability of the approach.

[0083] like Figure 3 The diagram illustrates the process of local evidence credibility assessment and local evidence recalibration (corresponding to steps 3 and 4). The cropped personnel image undergoes backbone feature extraction to obtain a spatial feature map and a global personnel descriptor. Local regions and identity clues are preserved through spatial local feature segmentation, and local evidence credibility is assessed. Based on the identity discrimination response, the contribution strength of each spatial location is calculated, and a credibility map is output. and uncertainty diagram The uncertainty graph represents the intensity of the impact of occlusion, pollution in Beijing, or detection errors, while the confidence graph represents the stable contribution of local areas to personnel identity retrieval. The confidence graph, together with the original identity features and the extracted and corrected context features, is used for local evidence recalibration to obtain the recalibrated personnel feature map. .

[0084] Specifically, during the training phase, let M be the number of training identity categories, and the weight matrix of the main identity classifier be... Where C is the number of spatial feature channels; the identity label of the i-th training sample is , Indicates the corresponding number in W The weight vector of the class.

[0085] Let spatial feature map The channel feature vector at spatial location (u,v) is Then this position corresponds to the real identity category. The discriminant response is:

[0086] ;

[0087] in , , This is the classification weight vector corresponding to the real identity category, where "T" represents the vector transpose.

[0088] make Represent all H×W locations of the spatial feature map, and define the spatial minimum and maximum values ​​of the response map:

[0089] ;

[0090] The local evidence credibility map is then calculated as follows:

[0091] ;

[0092] In the previous formula, the minimum and maximum response values ​​represent the minimum and maximum values ​​at H×W spatial locations, respectively; ε>0 is a numerical stability constant to prevent the denominator from being zero; confidence value .

[0093] The local uncertainty diagram is defined as:

[0094] ;

[0095] The higher the value, the more stable the support for identity verification at the corresponding position; The higher the value, the weaker the identity support for that location under the current representation. A low response is not necessarily equivalent to physical occlusion, therefore continuous soft correction and continuous soft compensation are used instead of binary deletion.

[0096] It should be noted that the confidence map is not a manually labeled area of ​​occlusion, nor does it require the system to accurately determine the physical occlusion boundaries. Its purpose is to provide continuous weights for subsequent local recalibration and cross-shot compensation, enabling the system to retain reliable identity clues while making appropriate corrections to low-confidence areas.

[0097] This scheme utilizes the identity discrimination response to determine the contribution of each local region to identity discrimination. Regions with high contribution are designated as "high confidence," containing stable identity clues; regions with low contribution are designated as "low confidence," potentially due to occlusion, background contamination, or detection errors. The confidence map uses continuous weights rather than binary masks, providing a more refined description of the blurred boundaries and irregular shapes of occlusion. This approach eliminates the need for external pose estimation or human parsing models, avoiding error propagation; furthermore, continuous weights are better suited than hard binary masks for the transitional state of "partially visible, partially occluded" in real-world occlusion scenarios; and the confidence map provides accurate "navigation signals" for subsequent recalibration and cross-camera compensation.

[0098] Local evidence recalibration is used to mitigate the contamination of feature maps by occlusions, background, and bounding box truncation. This applies to the original spatial features. The local context extraction unit first uses a 3×3 depthwise convolution to extract the neighborhood context, then uses a 1×1 pointwise convolution to complete the channel information fusion, and finally uses batch normalization to stabilize the feature distribution to obtain the context correction features.

[0099] ;

[0100] in and It has the same C×H×W dimensions. This branch does not require additional pose estimation, human body resolution, or pixel-level occlusion annotation.

[0101] Will After broadcasting along the channel dimension, the context interpolation features are obtained first:

[0102] ;

[0103] Then, the recalibration feature is obtained by using bounded residual interpolation:

[0104] ;

[0105] Where ⊙ represents multiplication at each spatial location and broadcasting along the channel dimension; , Since σ is a trainable scalar and σ(·) is the Sigmoid function, therefore ∈(0,1). Used to limit the replacement strength of context correction on the original features.

[0106] Let 'a' be initialized to 0, then α is initialized to 0.5, allowing the module to start training from a conservative correction state. The current implementation uses a single scalar α acting on the entire feature map; without changing the residual soft correction mechanism, it can also be extended to bounded coefficients set by channel or region.

[0107] In border inspection videos, occlusion areas are often not regular rectangles; for example, a suitcase may obscure the lower body, a vehicle frame may obscure the side of the body, or a crowd may obscure the shoulders. Continuous soft gating is more suitable than hard threshold replacement for such irregular occlusions, reducing feature abrupt changes caused by local misjudgments.

[0108] Before obtaining supplementary information from cross-shot neighbors, a "conservative correction" can be performed on the low-confidence region using the neighborhood context within the current image. This involves using the contextual features of the surrounding visible areas to perform residual soft correction on the low-confidence region, rather than directly deleting or hard masking it. More importantly, the recalibrated feature map carries feature information consistent with the current image context in its low-confidence region, providing a better initial state for subsequent cross-shot soft alignment. If cross-shot compensation is performed directly without recalibration, the features of the low-confidence region (containing a large amount of occlusion or background information) are prone to matching incorrect neighbor regions when similarity calculation is performed with neighboring region blocks, leading to alignment deviations.

[0109] Therefore, by recalibrating with local evidence, using residual soft correction instead of hard deletion or replacement, the potential useful information in the original features is preserved; the correction coefficients are controlled by a learnable bounded function to avoid loss of identity clues due to overcorrection; the recalibrated feature map is used as input for cross-shot compensation, improving the accuracy of subsequent soft alignment.

[0110] A characteristic of border inspection scenarios is that the same target (in this example, a person) may be recorded from multiple camera locations. Clothing, backpacks, or body cues that are obscured in one camera may be clearer in another. This solution leverages this characteristic by constructing a cross-camera local evidence compensation mechanism during the training phase. The specific process is as follows: Figure 4 As shown (corresponding to step 5). Figure 4The "uncertainty-guided compensation" in the text uses a continuous injection coefficient. A binary switch without a fixed threshold is not used.

[0111] The training batch uses P×K sampling, ensuring that the same identity contains K image samples in one batch, prioritizing coverage of different cameras. For the current sample i, a cross-camera candidate set is defined:

[0112] ;

[0113] in and Here, i and j are the camera numbers for samples i and j, respectively; the system selects the Top-K candidates from the candidate set based on the inner product similarity of the normalized global descriptors. n There are 10 neighbors. If the candidate set is empty, the compensation path degenerates into an identity mapping.

[0114] Recalibrate the features of the current sample and its neighbors. , The mesh is divided into a fixed number of local regions using Gh×Gw adaptive mesh pooling.

[0115] ;

[0116] L2 normalization is performed on each region block to obtain a normalized region block matrix. and For neighbor j, the region block similarity, row-normalized soft alignment weight, and soft alignment complementary region block are as follows:

[0117] ;

[0118] ;

[0119] ;

[0120] in >0 represents the soft alignment temperature. Softmax normalization is performed on the neighboring region block dimension, so each current region block can obtain a complementary representation from all region blocks of the neighboring images based on semantic similarity, without relying on fixed grid coordinates.

[0121] For the nth current region block, first calculate the neighbor weights based on its similarity to each cross-shot neighbor soft-aligned region block, and then obtain the complementary region blocks aggregated across neighbors:

[0122] ;

[0123] ;

[0124] ;

[0125] The spatial extent covered by the uncertainty map in the nth region block is then determined. The average value is taken from the innermost part, and the actual injection ratio is determined by the bounded learnable coefficients:

[0126] ;

[0127] Finally, continuous blending is performed between the current region block and the cross-shot complementary region block:

[0128] ;

[0129] All merged region blocks are rearranged and upsampled to obtain the compensated feature map. .

[0130] Specific example of fusion: In the initial stage of training If the average uncertainty of a certain region / block Then the injection coefficient The fusion result retains 85% of the current region block features and injects 15% of cross-shot complementary features; at this point, the average confidence level of the region is approximately 0.7, belonging to a relatively confident region, and it is not divided according to the hard threshold of "low confidence / high confidence". ,but It retains 60% of the current features and injects 40% of complementary features. After training... It learns from data, but is always constrained to between 0 and 1 by the Sigmoid function, so the injection ratio is continuous and bounded.

[0131] Therefore, the phrase "injecting more complementary cues into low-confidence regions" in this paper refers to continuous gating, rather than setting a fixed threshold before implementing binary switching. High-confidence regions, due to... Smaller injections result in very weak or near-zero injections, leading to low-confidence regions. Larger size results in stronger injection; this design is consistent with Figure 4 This corresponds to the "uncertainty-guided compensation" step in the text.

[0132] When there are no valid cross-shot neighbors for the same person in a training batch, or when the quality of the neighboring images is lower than a preset threshold, the system keeps the current region block unchanged and the compensation path automatically degenerates into an identity mapping to ensure the stability of the training process.

[0133] To control the training overhead of cross-camera local evidence compensation, candidate retrieval is limited to the same group of people within the current P×K training batch. For example, P=8, K=8, and the batch size is 64; for each sample, no more than 3 neighbors are selected from the same person, different cameras, and not themselves as candidates. The recalibrated features are obtained by 8×4 adaptive grid pooling to obtain 32 local tokens. Cross-camera matching only calculates the 32×32 token similarity matrix between the selected neighbors, without performing full pairwise matching on the original high-resolution feature map or the complete training library. Thus, the computational cost of local matching is constrained by the batch size, the upper limit of neighbors, and the fixed number of tokens, and does not grow linearly with the total size of the historical image library.

[0134] In engineering implementation, the similarity matrix can be computed in parallel using batch matrix multiplication. Furthermore, it can be combined with mixed-precision training, global descriptor caching, quality threshold pre-screening, block computation, and identity degradation when there are no effective neighbors to further reduce memory and computational overhead. The cross-shot compensation unit is an offline training auxiliary path, and its training overhead can be higher than the baseline network. In an implementation with the same hardware and input configuration, the training time per round after adding the complete compensation path is approximately twice that of the baseline. However, this path is not loaded during model deployment, and the number of parameters in the main feature extractor and the forward computation cost per image remain consistent with the baseline.

[0135] In this solution, steps 3 and 4 both handle occlusion within a single image. However, when key identity clues are completely missing from the current viewpoint (e.g., the upper body is completely obscured by the vehicle's frame), there is insufficient information available within a single image. This solution leverages the continuous coverage of multiple cameras in border inspection scenarios to find and inject missing clues from other camera images of the same target based on local semantic correspondences.

[0136] In the specific steps of step 5:

[0137] By dividing the feature map into multiple local regions (through grid pooling, strip pooling, etc.), matching and injection are performed at the local level rather than the global level, avoiding the dilution of clues caused by global averaging.

[0138] Similarity soft alignment is used to calculate the similarity matrix between the current sample block and each block of its neighbors. Normalized weighting is then applied along the neighbor block dimension to find complementary blocks that semantically correspond to the current block. This solves the semantic misalignment problem caused by fixed-position alignment under changes in pose and scale; for example, the "upper body region" in the current image will be correctly aligned to the "upper body region" in a neighboring image, rather than at a fixed coordinate position.

[0139] By supplementing only low-confidence regions, high-confidence regions retain the original observations from the current camera. This avoids diluting reliable clues and prevents injection operations from interfering with already stable identity features.

[0140] The above design can accurately obtain discrimination clues for missing images in the current image from cross-camera neighbors, realizing "cross-camera explicit compensation" instead of simply performing global feature fusion or knowledge distillation; local-level soft alignment avoids semantic misalignment; on-demand injection avoids interference with reliable regions; this mechanism is only used during the training phase, does not need to be loaded during deployment, and does not increase the online computing burden.

[0141] This scheme sets up a main retrieval path and a compensation auxiliary path during the training phase. The main retrieval path consists of the original spatial feature map. Descriptors are generated through global pooling, BNNeck, and the main classifier. and category probability The compensation auxiliary path is determined by the compensation feature map. Descriptors are generated through independent global pooling, BNNeck, and auxiliary classifiers. and category probability Recalibration feature map For intermediate representations aligned across shots, no separate category header is set.

[0142] Both primary and secondary identity classifications use labeled smoothed cross-entropy. Let the number of training identities be M and the smoothing coefficient be η, then the target distribution of the i-th sample is:

[0143] ;

[0144] Therefore, the identity classification loss of the main path and the identity classification loss of the compensating auxiliary path are respectively:

[0145] ;

[0146] To constrain the identity semantic structure of the main and secondary paths, the descriptors are first L2 normalized:

[0147] ;

[0148] For a complete group Gb (containing S samples) with the same identity, construct the main path prototype and the auxiliary path prototype respectively:

[0149] ;

[0150] Let the number of complete groups be Bg, and the group prototype consistency loss, sample-to-auxiliary prototype consistency loss, and final descriptor consistency loss be respectively:

[0151] ;

[0152] ;

[0153] ;

[0154] in , The weights are non-negative; for the remaining samples that are insufficient to form a complete group, sample-level cosine consistency constraints can be used.

[0155] The overall training objective is:

[0156] ;

[0157] in and These represent the weights of the auxiliary identity classification loss and the descriptor consistency loss, respectively. The objective is to maintain stable identity discrimination capability in the main path, ensure that the compensating features do not deviate from identity semantics, and transfer the robustness gained from cross-camera complementarity during training to the main retrieval path retained only during deployment.

[0158] After the model training is completed, the cross-camera local evidence compensation training unit is not loaded during the actual duty retrieval phase. Only the personnel detection, normalization, main feature extraction, image library structure correction, and candidate ranking modules are retained. This utilizes the complementary information from the training phase while avoiding excessive computation during the deployment phase.

[0159] Figure 5 This is a schematic diagram of the image library structure correction and multi-clue fusion sorting process (corresponding to steps 6 to 8). Among them, "auxiliary clue s3" is a cross-camera indication clue in the basic implementation, and in the border inspection engineering embodiment, it can also be extended to location, time and scene soft constraints; all extended clues participate in the weighting through validity flags and reliability coefficients, and do not replace direct appearance similarity and corrected similarity.

[0160] During the on-duty retrieval phase, descriptors are obtained by extracting and L2 normalizing the query images and library samples. and Direct appearance similarity is defined as:

[0161] ;

[0162] To leverage the cross-camera structure within the gallery, for each gallery descriptor Only when satisfied Retrieving Top-Kg Nearest Neighbors from Image Library Samples .

[0163] Temperature scaling weights are:

[0164] ;

[0165] The corrected image library descriptor is:

[0166] ;

[0167] Structural correction similarity is defined as:

[0168] ;

[0169] in This is for temperature parameters. This process uses only the gallery descriptor and camera ID; it does not use query identity tags or gallery test identity tags.

[0170] Step 5's cross-camera compensation enhances the quality of single-image features during the training phase, but the ranking phase still relies on independent query-image library comparisons. This step introduces an image library structure correction mechanism: utilizing existing cross-camera nearest neighbor relationships between image library samples, the structure of each image library descriptor is corrected. Images of a target under different cameras should be close to each other in the feature space. This structural information can be used without relying on test identity labels to obtain corrected image library descriptors. The similarity between the query descriptor and the corrected image library descriptor reflects whether "the query target is consistent with the cross-camera neighbor structure of a target in the image library," thereby reducing false matches. The above image library structure correction does not require the identity label of the query target and can be pre-computed offline or updated periodically; it complements Step 5's cross-camera compensation, with compensation enhancing feature quality and correction optimizing the ranking structure; the correction process only operates on the image library side and does not increase the computational burden on the query side.

[0171] The system can also generate auxiliary clues s3 based on business metadata. The basic auxiliary clue in the paper's implementation is a cross-camera indication function:

[0172]

[0173] in This is an indicator function. In the border inspection engineering implementation, location, time, and scene soft constraints can be further combined without replacing visual similarity.

[0174] For example, if the query camera and the gallery camera are on a reachable route and the time interval is within the time range of pedestrian walking, vehicle passage, or inspection process, the candidate weight is increased; if the spatial relationship or time sequence of the two locations is obviously inconsistent with the duty logic, the candidate weight is decreased.

[0175] When only three clues are used, the final score can be expressed as:

[0176] The fixed-weighted fusion score and normalized weights of the three clues are as follows:

[0177] ;

[0178] ;

[0179] in Fixed fusion parameters are preset by the system or determined through a validation set. After Softmax normalization, all parameters are non-negative and sum to 1; the paper's implementation uses fixed parameters instead of learning parameters during the testing phase.

[0180] When further integrating multiple business metadata, a validity flag is set for the k-th metadata item. and reliability coefficient ,in Indicates whether the field exists. Indicates the credibility level of the field. This represents the normalized score for the soft constraint.

[0181] The overall score, which includes tolerance for missing values, can be calculated using:

[0182] When further integrating multiple business metadata, the comprehensive score, including tolerance for missing values, can be adopted as follows:

[0183] ;

[0184] in and Non-negative weights, visual cues and Always retain ε, which is a numerical stability constant. When a piece of metadata is missing, let... When its reliability is low, reduce The denominator automatically renormalizes the remaining valid clues.

[0185] When a piece of metadata is missing, The remaining valid items are then renormalized; when metadata exists but its reliability is low, it is reduced... This results in only a weak ranking effect.

[0186] For mobile cameras, law enforcement recorders, or vehicle-mounted videos, location information can be represented using task areas, coarse-grained geographic grids, vehicle trajectory intervals, or the nearest known location, with a lower reliability setting based on positioning errors. For older devices without unified time synchronization, the platform's receiving time, video segment start time, device clock offset estimation, or event sequence correction can be prioritized for data acquisition time, and the allowable time window can be extended to include the estimation error. If the scene category cannot be determined, it can be marked as unknown and excluded from scoring. If conflicts occur between metadata, the system does not hard-reject a single metadata entry, but instead reduces the weight of metadata, retains visual candidates, and marks "incomplete metadata" or "low time / location reliability" in the review prompt, allowing users to review the data in conjunction with the original video.

[0187] The candidate ranking results not only provide similarity scores, but also suggest candidate images, video clips, camera locations, timestamps, trajectory numbers, and key matching areas. Users can manually review the candidate rankings and evidence clips; the system does not directly make a final identity determination or decision.

[0188] In port inspection scenarios, users can input a snapshot of the inspection channel, and the system will return candidate results from the image library of the waiting area, channel entrance, baggage area, vehicle area, etc., and display possible passing segments in order of location and time.

[0189] In border patrol scenarios, users can input low-resolution or obscured images of people from patrol videos. The system will then search for similar individuals from fixed checkpoints, road cameras, and historical patrol videos, and provide candidate trajectories across cameras.

[0190] In video playback scenarios, the system can first detect personnel, extract trajectories, and store features in a database for a historical video segment, and then return candidate segments based on subsequent query requests. For the returned results, the system retains information such as similar regions, original video frames, screenshot time, and model version for easy review and auditing.

[0191] Considering that candidate ranking based solely on visual similarity may lead to illogical mismatches in practical applications due to visual ambiguity (such as two locations being far apart or having inconsistent timelines), this solution integrates direct appearance similarity, corrected image library similarity, and business metadata (location connectivity, time intervals, scene categories) to form a multi-clue comprehensive score. Simultaneously, validity flags and reliability coefficients are set for the metadata; when metadata is missing or unreliable, it automatically reduces weight or degrades to pure visual retrieval, ensuring the system's robustness and usability. Multi-clue fusion reduces illogical mismatches; the metadata fault tolerance mechanism ensures system usability even with imperfect data; and the output of candidate rankings and associated video clips assists user verification rather than directly replacing manual judgment, complying with border inspection requirements.

[0192] System implementation, including:

[0193] The video image access module is used to access fixed monitoring, mobile duty terminals, offline video and captured images, and record the camera number, location, acquisition time and scene category;

[0194] The personnel target normalization module is used to detect, crop, scale up, correct out-of-bounds, remove duplicates, and screen quality targets in video frames;

[0195] The feature extraction module is used to input the normalized target image into the feature extraction network to generate spatial feature maps and global descriptors;

[0196] The local evidence credibility assessment module is used to generate a local evidence credibility map and a local uncertainty map based on the strength of the contribution of each spatial location in the spatial feature map to the identification of the target.

[0197] The local evidence recalibration module is used to extract context correction features from the spatial feature map based on the local uncertainty map, and continuously fuse them with the original spatial feature map to obtain a recalibrated feature map.

[0198] The metadata quality assessment module is used to generate validity indicators and reliability coefficients for camera, location, time, and scene fields;

[0199] The image library structure correction module is used to perform cross-camera nearest neighbor structure correction on the image library descriptor to obtain the corrected image library descriptor.

[0200] The candidate ranking and verification output module is used to integrate direct appearance similarity, corrected similarity, and metadata clues to output candidate ranking results and verification prompts.

[0201] like Figure 6 As shown, the system can distinguish between the offline training side and the on-duty deployment side. In the actual deployment structure, the system mainly includes a video image access module, a personnel target normalization module, a feature extraction module, a local evidence credibility assessment module, a local evidence recalibration module, a metadata quality assessment module, a library structure correction module, and a candidate ranking and verification output module. The cross-camera local evidence compensation training module and the retrieval model training module are mainly used in the model training phase and can be deployed on a training server or in an offline model update environment.

[0202] The video image access module is used to access fixed monitoring, mobile duty terminals, offline video, and captured images; the personnel target normalization module is used for detection, cropping, deduplication, and quality screening; the feature extraction module is used to generate spatial feature maps and global personnel descriptors; the local evidence credibility assessment module is used to generate credibility maps and uncertainty maps; the local evidence recalibration module is used to correct low-credibility areas; the cross-camera local evidence compensation training module is used to utilize cross-camera complementary clues during the training phase; the metadata quality assessment module is used to generate validity indicators and reliability coefficients for camera, location, time, and scene fields; the image library structure correction module is used for descriptor-level nearest neighbor correction; and the candidate sorting and verification output module is used to fuse valid clues, output candidate lists, and generate metadata anomaly prompts.

[0203] The system can be deployed on border inspection internal servers, GPU workstations, edge computing nodes, or video structuring platforms. For scenarios with high real-time requirements, personnel detection and feature extraction can be deployed on the edge side, while image library retrieval and candidate ranking can be deployed on the central side. For offline video backtracking scenarios, batch frame extraction, offline library construction, and asynchronous retrieval methods can be adopted.

[0204] The personnel detection component in this solution can be replaced with any detection model capable of outputting personnel bounding boxes; the feature extraction network can be replaced with any network capable of outputting spatial features and global descriptors; the local region block partitioning method can adopt grid pooling, horizontal strip pooling, semantic region pooling, or multi-scale pooling.

[0205] The camera, location, time, and scene clues in this solution can be configured according to actual business needs. When the fixed camera topology and unified timing information are complete, the location connectivity, travel direction, and time interval can be used. When the location accuracy of mobile devices is insufficient, coarse-grained regions or trajectory intervals can be used, and reliability can be reduced. When the device time is not synchronized, it can be corrected by using the platform's received time or clock offset estimation. When any metadata is missing or cannot be corrected, the corresponding clue can be closed and the remaining weights can be renormalized, using only direct appearance similarity and image library structure-corrected similarity to complete the retrieval.

[0206] This solution can also be extended to personnel retrieval under vehicle obstruction, personnel retrieval with strong obstruction due to carried items, cross-day / night personnel retrieval, low-resolution personnel retrieval, and multimodal personnel clue verification. As long as these extensions still employ the core principles of local evidence credibility assessment, low-confidence region correction, cross-camera complementary training, or image database structure correction, they are all equivalent implementations of this solution.

[0207] Those skilled in the art will understand that embodiments of this solution can be embodied as a method, system, or computer program product. Therefore, this solution can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, this solution can also take the form of a computer program product implemented on one or more computer-readable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0208] This solution is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of this solution. It should be understood that each block in the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to create a machine such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, can implement one or more blocks in the flowchart and / or one or more blocks in the block diagram, specifying the function.

[0209] These computer program instructions may also be stored in a computer-readable storage medium capable of directing a computer or other programmable data processing device to operate in a particular manner, such that the instructions stored in the computer-readable storage medium generate an article of manufacture containing instruction means. These instruction means are used to implement the functions specified in one or more flowcharts and / or one or more blocks of a block diagram.

[0210] Furthermore, these computer program instructions can be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to generate a computer-implemented processing procedure. Thus, the instructions that execute on the computer or other programmable equipment will provide steps for implementing one or more processes of the flowchart and / or one or more blocks of the block diagram that specify the functionality.

[0211] The above description is merely a preferred embodiment of the present invention and is not intended to limit the invention. Various modifications and variations can be made to the present invention by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the scope of protection of the present invention.

Claims

1. A method for cross-camera retrieval of obscured targets in border inspection, characterized in that, Includes the following steps: Acquire target images during border inspection and extract their spatial feature maps and global descriptors; Based on the strength of each spatial location's contribution to target identity determination in the spatial feature map, a local evidence credibility map is generated, and a local uncertainty map is obtained using the complementary quantities of the credibility map. Based on the local uncertainty map, context correction features are extracted from the spatial feature map and continuously fused with the original spatial feature map to obtain a recalibrated feature map. During the training phase, cross-camera neighbors are retrieved for training samples from different camera sources under the same target identifier; the recalibrated feature maps of the current sample and neighbor samples are divided into multiple local region blocks, the similarity matrix between each region block is calculated and soft alignment is performed, and complementary cues from cross-camera neighbors are injected into low-confidence region blocks according to the uncertainty intensity of the region block corresponding to the current sample to obtain the compensation feature map. A joint training objective is constructed based on the global target descriptor, recalibrated feature map, and compensated feature map, and the feature extraction network is trained accordingly. During the retrieval phase, the trained feature extraction network is used to extract descriptors of the query target and the image library target. Cross-camera nearest neighbor structure correction is performed on the image library descriptors, and candidate ranking results are output by combining direct similarity, corrected similarity and scene category clues.

2. The method for cross-camera retrieval of obscured targets in border inspection as described in claim 1, characterized in that, The local evidence credibility map is generated by: calculating the category discrimination response using the identity discrimination weight and the feature vectors of each spatial location in the spatial feature map, and then spatially normalizing the category discrimination response to obtain the credibility map; the credibility map is used to represent the effective contribution of the local region to the target identity retrieval.

3. The method for cross-camera retrieval of obscured targets in border inspection as described in claim 1, characterized in that, Contextual correction features are obtained by sequentially performing depthwise separable convolution, pointwise convolution, and normalization on the original spatial features; continuous fusion is performed between the original spatial features and contextual correction features based on the local uncertainty map, and the replacement intensity is limited by learnable correction coefficients.

4. The method for cross-camera retrieval of obscured targets as described in claim 1, characterized in that, The recalibrated feature maps of the current sample and its neighboring samples are divided into multiple local region blocks. The similarity matrix between each region block is calculated and soft alignment is performed. Specifically, local region blocks are obtained by adaptive grid pooling, strip pooling, or multi-scale pooling. The similarity matrix is ​​calculated after normalizing the current sample region block and the cross-shot neighbor region blocks. Then, the similarity matrix is ​​normalized and weighted in the dimension of the neighbor region blocks to obtain the complementary region block that corresponds to the semantics of the current region block.

5. The method for cross-camera retrieval of obscured targets as described in claim 1, characterized in that, Cross-camera neighbor retrieval is limited to candidate sets of images with the same target identifier, different camera numbers, and not their own images. When there are no valid cross-camera neighbors or the quality of candidate neighbors is lower than a preset threshold, the local region block of the current sample remains unchanged, and the sample participates in training only through identity discrimination, retrieval metrics, and local recalibration related losses.

6. The method for cross-camera retrieval of obscured targets as described in claim 1, characterized in that, Based on the uncertainty intensity of the region block corresponding to the current sample, complementary cues from cross-lens neighbors are injected into the low-confidence region block. Specifically, the higher the uncertainty intensity, the greater the proportion of injected complementary cues; the lower the uncertainty intensity, the greater the proportion of the original observation features of the current sample that are retained.

7. The method for cross-camera retrieval of obscured targets in border inspection as described in claim 1, characterized in that, The joint training objectives include identity classification loss, triple ranking loss, local compensation feature identity loss, and descriptor consistency constraint. The descriptor consistency constraint is used to limit the offset between the main retrieval path descriptor and the training auxiliary path descriptor in the identity semantic space, so that the occlusion robustness obtained in the training phase is retained in the main retrieval path in the actual retrieval phase.

8. The method for cross-camera retrieval of obscured targets in border inspection as described in claim 1, characterized in that, Cross-camera nearest neighbor structure correction is performed on the image library descriptors. Specifically, for each image library descriptor, nearest neighbors are retrieved in image library samples from different cameras or different locations. Nearby weights are calculated based on temperature scaling similarity. The nearest neighbor descriptors are weighted, summed, and normalized to obtain the corrected image library descriptor. The structure correction process does not use test identity labels.

9. The method for cross-camera retrieval of obscured targets in border inspection as described in claim 1, characterized in that, Scene category clues also include camera location connectivity and acquisition time interval; the fusion score is obtained by weighting direct similarity, corrected similarity, location connectivity, time interval and scene type matching relationship; a validity flag and reliability coefficient are set for each metadata item, and when the corresponding metadata is missing, the weight of the item is reset to zero and the weights of the remaining valid clues are renormalized.

10. A cross-camera retrieval system for obscured targets at border inspection sites, used to implement the retrieval method as described in any one of claims 1-9, characterized in that, include: The video image access and feature extraction unit is configured to: acquire target images during border inspection and extract their spatial feature maps and global descriptors; The credibility assessment and recalibration unit is configured to generate a local evidence credibility map and a local uncertainty map based on the contribution strength of each spatial location in the spatial feature map to the target identity discrimination, and to continuously fuse the original spatial features and context correction features based on the local uncertainty map to obtain a recalibrated feature map. The cross-camera compensation training unit is configured to: during the training phase, retrieve cross-camera neighbors for training samples from different camera sources under the same target identifier, perform soft alignment of region blocks and injection of complementary cues, and construct a joint training objective to train the feature extraction network; The retrieval output unit is configured to: extract descriptors of the query target and the image library target during the retrieval phase, perform cross-camera nearest neighbor structure correction of the image library, and output candidate ranking results by fusing similarity and scene category clues.

Images