Target retrieval method and system

Abstract

The invention discloses a target retrieval method and system. The method comprises the following steps: S1, data access and preprocessing: acquiring multi-source data, performing cleaning, formatting and time-space standardization, and performing target detection and cutting on an image / video to generate a structured target object; s2, feature extraction: extracting deep semantic feature vectors and auxiliary understanding information, which have discriminability and adapt to complex scenes, from the target image; s3, constructing a data index, including constructing a spatio-temporal semantic hypergraph index of multiple types of nodes and hyperedges based on deep features and spatio-temporal information of the targets to express a complex relationship between the targets; and S4, data query: receiving multi-modal query information of a user, performing candidate region screening, feature matching and relation reasoning by utilizing a hypergraph index, and finally outputting a high-confidence target retrieval result. According to the method, the robustness of complex scenes and target changes and multi-dimensional query of depth are improved.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of retrieval, in particular to a target retrieval method and system. BACKGROUND

[0002] With the rapid development of information technology, various types of data are growing explosively, especially in the fields of video monitoring, geographic information systems, social networks, and Internet of Things, a large amount of structured and unstructured data containing target information are generated. How to quickly and accurately retrieve the target of interest from these massive data and further analyze the potential association between targets has become a research hotspot and technical bottleneck in the current information processing field.

[0003] Existing target retrieval technologies, such as content-based image retrieval (CBIR), keyword-based text retrieval, or specific feature-based matching algorithms, often face challenges when dealing with large-scale, high-dimensional, and complex scene data. The main technical defects are as follows:

[0004] Low retrieval efficiency: When the data volume is huge, traditional linear scanning or global feature matching methods cannot meet the real-time requirements. Although index structures are introduced, the performance of existing data structure indexes (such as some variants of R-tree and K-D tree) will sharply decline when facing high-dimensional feature data, suffering from "dimension disaster".

[0005] Insufficient retrieval accuracy: Targets may exhibit significant changes in appearance, posture, and lighting in different scenarios, leading to poor robustness of matching based on underlying features. At the same time, the semantic analysis and understanding between user query intent and machine understanding also limit the retrieval accuracy.

[0006] Insufficient utilization of spatial information: Many target data naturally have spatial attributes (such as GPS coordinates and relative positions in images), but existing retrieval methods do not fully utilize spatial information, making it difficult to support complex spatial relationship queries and efficient retrieval based on spatial proximity.

[0007] In terms of spatial data indexing, although various spatial index structures have been developed, such as R-tree, R* tree, quadtree, grid index, etc., they have improved the query efficiency of spatial objects to some extent. However, these methods also have the following technical defects:

[0008] Static and dynamic adaptability: Traditional spatial indexes are designed for static data, and for dynamic and frequently updated data scenarios, the maintenance cost of the index is high, and the performance may decrease.

[0009] Multi-level and multi-scale problem: targets may have different importance and relevance at different scales and levels. Single-level spatial index is difficult to effectively express and utilize this multi-level feature, resulting in insufficient flexibility in granularity of retrieval and analysis.

[0010] Difficulty in combining with semantic information: traditional spatial index mainly focuses on the geometric position of the target, and the combination with the semantic characteristics (such as category, attribute) of the target is not close enough, making it difficult to achieve efficient retrieval based on semantic and spatial double constraints.

[0011] Therefore, it is urgent to propose a new technical solution to realize the rapid positioning and screening of massive target data, and ultimately realize efficient and accurate target retrieval to meet the growing demand for intelligent information processing. SUMMARY

[0012] To solve the technical problems existing in the prior art, the present application provides a target retrieval method, comprising the following steps:

[0013] Step S1: data access and preprocessing, including obtaining multi-source data, cleaning, formatting, spatio-temporal standardization, and target detection and cropping of images / videos to generate structured target objects;

[0014] Step S2: feature extraction, including extracting deep semantic feature vectors with discriminative and adaptive complex scene from target images and auxiliary understanding information;

[0015] Step S3: constructing data index, including constructing multi-type node and hyperedge spatio-temporal semantic hypergraph index based on deep features and spatio-temporal information of targets, to express the complex relationship between targets;

[0016] Step S4: data query, including receiving user multi-modal query information, using hypergraph index for candidate region screening, feature matching and relationship reasoning, and finally outputting high confidence target retrieval results.

[0017] Further, the feature extraction step S2 specifically includes:

[0018] S21: based on D 2 CFAN model constructs a feature extraction model;

[0019] S22: optimization training based on the feature extraction model and setting the loss function;

[0020] S23: pre-processing and normalization of the trained feature extraction model in the inference stage;

[0021] S24: extract features and forward propagation and internal dynamic regulation;

[0022] S25: Obtain the output feature vector and perform quality assessment and directional transmission.

[0023] Further, step S21: based on D 2 The CFAN model constructs a feature extraction model, which includes:

[0024] S211: Basic feature extraction and multi-level semantic representation construction are used to extract different depths of pathways and obtain feature maps that can represent different levels of abstract semantic information;

[0025] S212: A dynamic hierarchical decoupling attention submodule is constructed, and attention is applied to each semantic subgroup to dynamically decouple the feature channels of each level of feature map into multiple semantic subgroups;

[0026] S213: A instance-level and class-level collaborative attention submodule is constructed to allow the model to focus on the instance uniqueness and class commonality of the target simultaneously before extracting the final features;

[0027] S214: A context-aware fusion attention submodule is constructed to fuse features from different processing stages;

[0028] S215: Output feature vector, the features obtained by the context-aware fusion attention submodule are passed through one or more fully connected layers to obtain the final feature vector.

[0029] Further, step S22: based on the feature extraction model, the optimization training and the setting of the loss function specifically include:

[0030] S221: Knowledge transfer initialization and domain adaptability fine-tuning;

[0031] S222: Optimize the main identity recognition loss and the DHDA level weighted semantic decoupling orthogonality regularization, and the calculation formula is:

[0032]

[0033] Where: s ij ·s il represents the dot product of vectors s ij and s il , ||s ij ||2 represents the L2 norm of vector s ij , ∈0 is a normal number, and ω i is the decoupling importance weight coefficient set for the i-th semantic level;

[0034] S223: D 2 CFAN model parameter iterative optimization process.

[0035] Further, step S3, constructing a data index includes:

[0036] S31: constructing vertices and hyper-edges of the spatio-temporal semantic hypergraph index;

[0037] S32: initial construction of the spatio-temporal semantic hypergraph index, continuous entry of data objects and dynamic evolution of the index structure, generation and association of relationship hyper-edges.

[0038] Further, S31: constructing vertices and hyper-edges of the spatio-temporal semantic hypergraph index comprises:

[0039] S311: constructing vertices of the spatio-temporal semantic hypergraph index comprises: macro spatio-temporal block vertices, basic spatio-temporal unit vertices, target instance vertices;

[0040] S312: constructing hyper-edges of the spatio-temporal semantic hypergraph index comprises: spatio-temporal co-occurrence hyper-edges, semantic similarity hyper-edges, trajectory / event association hyper-edges, region attribute association hyper-edges;

[0041] S313: determining symbols and attributes of vertices and hyper-edges.

[0042] Further, step S32 comprises:

[0043] S321: initial construction of the index;

[0044] S322: insertion of target instance vertices and dynamic evolution of the basic structure;

[0045] S323: dynamic generation and association of hyper-edges.

[0046] Further, step S4, data query comprises:

[0047] S41: query intention analysis and multi-modal input processing;

[0048] S42: STSGHI hierarchical preliminary screening and candidate region positioning;

[0049] S43: fine-grained target matching and initial candidate V Object set generation;

[0050] S44: association expansion and reasoning retrieval based on STSGHI hyper-edges;

[0051] S45: deduplication, fusion and confidence evaluation on direct matching and expansion results.

[0052] Further, step S1: data access and preprocessing step comprises:

[0053] S11: data source interface configuration and connection establishment;

[0054] S12: raw data acquisition and preliminary analysis;

[0055] S13: data cleaning and quality assessment;

[0056] S14: spatio-temporal information extraction and standardization;

[0057] S15: preliminary target detection;

[0058] S16: data packaging and output.

[0059] The application also provides a target retrieval, comprising a data access and preprocessing module, a feature extraction module, an index construction module, and a target retrieval module, wherein:

[0060] The data access and preprocessing module is used to obtain multi-source data, perform cleaning, formatting, spatio-temporal standardization, and target detection and cropping on images / videos, and generate structured target objects;

[0061] The feature extraction module is used to extract deep semantic feature vectors with discriminative ability and adaptability to complex scenes and auxiliary understanding information from target images;

[0062] The index construction module is used to construct a spatio-temporal semantic hypergraph index of multi-type nodes and hyperedges based on the deep features and spatio-temporal information of targets, to express the complex relationships between targets;

[0063] The target retrieval module is used to receive multi-modal query information from a user, perform candidate region screening, feature matching, and relationship reasoning using the hypergraph index, and finally output a target retrieval result with high confidence.

[0064] The modules in the method and system disclosed in the application can be deployed on a target server, or each module can be independently deployed on a different target server. In particular, to provide more powerful computing processing capability, the modules can be deployed on a cluster target server as needed.

[0065] Therefore, the technical effects achieved by the technical approach adopted by the application are:

[0066] 1. Significantly improving retrieval efficiency and accuracy: by using the D 2 CFAN model to extract high-discriminative features, in combination with the hierarchical index and hypergraph association of the STSGHI, the target can be quickly located from massive data, similar targets can be effectively distinguished, and the precision and recall of retrieval can be improved.

[0067] 2. Enhancing robustness to complex scenes and target changes: by using the D 2 CFAN model to effectively cope with challenges such as complex security scenes, such as changes in light, occlusion, and pose diversity, through its dynamic hierarchical decoupling, context perception, and instance and class collaborative mechanism.

[0068] 3. Realize the deep fusion and utilization of space, time, semantic information: STSGHI structure organically integrates the space-time attributes and deep semantic characteristics of the target through its multi-level vertex and multi-type hyperedge, and supports more flexible and in-depth multidimensional query.

[0069] 4. Support efficient discovery and reasoning of complex association relationship: The hyperedge mechanism of STSGHI can explicitly express the complex association between targets such as space-time co-occurrence, semantic similarity, and trajectory event, so that the system can not only find the target, but also reveal the potential connection between the targets, and support more advanced analysis tasks.

[0070] 5. Improve the intelligent level and practical application value of the system: The present application provides a complete solution from feature extraction to index construction to intelligent retrieval, which can effectively process and analyze large-scale, multi-source urban security target data, and provide strong technical support for urban security, video monitoring, geographic information system and other fields.

[0071] In order to have a more comprehensive understanding of the present application, the specific embodiments of the present application will be described in detail below in conjunction with the accompanying drawings. BRIEF DESCRIPTION OF DRAWINGS

[0072] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings needed in the embodiment description will be briefly introduced. Obviously, the drawings in the following description are only some embodiments of the present application, and other drawings can be obtained by those skilled in the art without creating any creative labor on the basis of these drawings.

[0073] Figure 1 The flowchart of a target retrieval method according to an embodiment of the present application.

[0074] Figure 2 The vertex structure diagram of an embodiment of the present application.

[0075] Figure 3 The structure diagram of a target retrieval system according to an embodiment of the present application. DETAILED DESCRIPTION

[0076] Please refer to Figure 1 The technical scheme of the present application provides a target retrieval method, which includes the following steps:

[0077] Step S1: Data access and preprocessing step includes: acquiring multi-source data, cleaning, formatting, space-time standardization, and target detection and cropping of image / video, and generating structured target object.

[0078] The step S1 accesses raw data from diversified data sources (such as video monitoring, supervision records, card hole data, mobile device positioning data, etc.), performs data cleaning, format conversion, space-time information extraction and standardization (including timestamp unification, geographic coding and coordinate system unification), and performs preliminary target detection on image / video data, crops target image blocks, and finally encapsulates into a unified structured object output.

[0079] The step S2 includes extracting deep semantic feature vectors with discriminative and adaptive complex scene and auxiliary understanding information from the target image.

[0080] The step S2 processes the preprocessed target image block through a dynamic hierarchical decoupling and context-aware fusion attention network (CFAN) model. The model extracts deep, discriminative and adaptive complex scene semantic feature vectors through its core components, including dynamic hierarchical decoupling attention (DHDA), instance-level and category-level collaborative attention (ICCA), and context-aware fusion attention (CAFA). 2 CFAN: Dynamic Hierarchical Decoupling and Context-Aware Fusion Attention Network) model. The model extracts deep, discriminative and adaptive complex scene semantic feature vectors through its core components, including dynamic hierarchical decoupling attention (DHDA), instance-level and category-level collaborative attention (ICCA), and context-aware fusion attention (CAFA).

[0081] The model training uses a composite loss function, including a main identity recognition loss and customized auxiliary losses and regularization terms for DHDA and ICCA characteristics. During inference, the input image block is normalized and preprocessed, and the DHDA, ICCA and CAFA are used to extract deep, discriminative and adaptive complex scene semantic feature vectors. 2 CFAN forwards and uses internal dynamic regulation mechanisms to output the final feature vector and optional auxiliary understanding information.

[0082] The step S3 includes constructing a space-time semantic hypergraph index based on the deep features and space-time information of the target, including multiple types of nodes and hyperedges, to express the complex relationships between targets.

[0083] The step S3 processes the preprocessed target image block through a dynamic hierarchical decoupling and context-aware fusion attention network (CFAN) model. The model extracts deep, discriminative and adaptive complex scene semantic feature vectors through its core components, including dynamic hierarchical decoupling attention (DHDA), instance-level and category-level collaborative attention (ICCA), and context-aware fusion attention (CAFA). 2CFAN feature vector and target data with accurate spatio-temporal information, construct a spatio-temporal semantic hypergraph index (STSGHI: Spatio-Temporal-Semantic Hypergraph Index). The index structure includes multiple types of vertices (macroscopic spatio-temporal block VMBlock, adaptive semantic region VARegion, basic spatio-temporal unit VCell, target instance V Object Etc.) and multiple types of hyperedges (spatio-temporal co-occurrence HESTC, semantic similarity HESemSim, track / event association HETrack / Event, region attribute association HERegionAttr, etc.), to express the complex spatio-temporal co-occurrence, semantic similarity and behavior association between targets.

[0084] Step S4, data query, including: receiving user multi-modal query information, using hypergraph index for candidate region screening, feature matching and relationship reasoning, and finally outputting high confidence target retrieval result.

[0085] This step S4 receives user multi-modal query request (image, text, spatio-temporal range, target ID or composite query), performs query intent analysis and feature extraction. The hierarchical structure (VMBlock, VARegion) and region semantic image (SemSumV) of STSGHI are used for preliminary screening and candidate region positioning. In VCell in the candidate region, based on D 2 CFAN feature vector, core semantic similarity matching is performed, and initial candidate V Object Set is generated combining other attributes of the target. Further, the initial results are associated, expanded and reasoned by using various hyperedges (HESTC, HESemSim, HETrack / Event, HERegionAttr) in STSGHI, to find accompanying targets, cross-spatio-temporal similar targets, target trajectories, event participants and targets meeting specific region attributes. Finally, the direct matching and expanded results are de-duplicated, fused, and the comprehensive confidence is calculated based on the original detection confidence, the matching degree with the query and the association path score, and the retrieval result is output.

[0086] The technical solutions of the present application will be described below in conjunction with various embodiments.

[0087] Step S1: data access and preprocessing step includes: acquiring multi-source data, cleaning, formatting, spatio-temporal standardization, and performing target detection and cropping on images / videos to generate structured target objects.

[0088] For example, in the scenario of urban security, from a variety of data sources including fixed monitoring cameras throughout the city, videos / images taken by mobile law enforcement terminals, databases storing historical monitoring reports, vehicle checkpoint capture systems, public Wi-Fi hotspot records, and even publicly published information related to public safety on social media, accessing these raw data requires a series of preprocessing operations, including data cleaning, format conversion, spatio-temporal information extraction and standardization, and preliminary target (such as pedestrians, vehicles, and specific objects) detection, to provide standardized input for subsequent feature extraction and index construction.

[0089] As a preferred embodiment, step S1 includes:

[0090] S11: Data source interface configuration and connection establishment, specifically including:

[0091] S111: Configure the corresponding access adapter according to the data source type. For example, configure RTSP / RTMP / GB28181 clients for real-time urban monitoring video streams (such as intersection cameras and internal monitoring of shopping malls); configure JDBC / ODBC connectors for monitoring databases or vehicle management databases; configure paths and access permissions for file systems storing historical monitoring video files or image libraries; configure authentication and request parameters for public security event API (such as weather warnings and traffic control information) interfaces.

[0092] S112: Initialize and establish stable connections with each data source to achieve continuous (such as real-time video stream access) or batch (such as daily monitoring data import) data acquisition.

[0093] S12: Raw data acquisition and preliminary analysis, specifically including:

[0094] S121: For video / image data (such as monitoring screens and checkpoint capture images), it can be divided into two types:

[0095] Real-time video stream: decode and capture image frames from the video stream at a preset frame rate (for example, 5-10 frames per second for key area monitoring, 1-3 frames per second for ordinary areas, or dynamically adjusted according to motion detection results in the screen).

[0096] Historical video / image file: batch read image files or frame extraction from historical monitoring videos.

[0097] S122: For structured / semi-structured data (such as monitoring texts, sensor readings, and checkpoint records), the processing steps include:

[0098] a) Obtain data records through API calls or database queries, for example, obtain monitoring records containing incident time, location description, and target personnel / vehicle feature description.

[0099] b) Parsing JSON (such as vehicle information uploaded by the fisheye system), XML, CSV, etc. formats, extracting key fields (such as alarm time, case location text description, event type, license plate number, vehicle color, sensor ID, people flow statistical value).

[0100] S13: Data cleaning and quality assessment, as a preferred embodiment, includes the following steps:

[0101] S131: Image quality assessment: quality detection is performed on the acquired monitoring image frames or fisheye pictures, and images that are too blurred (such as caused by camera shaking or low night illumination), overexposed / underexposed, severely occluded (such as pedestrians being occluded by large objects more than 70%), or have too low resolution (such as unable to identify faces or license plates) are removed.

[0102] S132: Data record verification: checking the integrity (such as whether the case location is missing), type correctness (such as whether the time format is standard YYYY-MM-DD HH:MM:SS, and whether the coordinates are within the city range) and consistency (such as whether the vehicle entry and exit fisheye times are logically correct) of structured data fields such as supervision records, fisheye data, etc.

[0103] S133: Missing value processing: for missing values of key fields, according to security business rules, marking, filling (such as marking 'unknown' for unknown vehicle color, or inferring from historical records of the same vehicle) or deleting the record without affecting core analysis.

[0104] S134: Redundant data removal: identifying and removing duplicate supervision records (such as the same case being reported multiple times but with consistent content) or almost identical consecutive monitoring image frames.

[0105] S14: Spatio-temporal information extraction and standardization, specifically including:

[0106] S141: Time information processing: extracting time stamps from file names (such as monitoring video file names often contain time), image EXIF metadata, or case occurrence time fields in supervision records. All time stamps are uniformly converted to standard time formats (such as UTC or Beijing time, ISO 8601).

[0107] S142: Spatial information processing, specifically including:

[0108] S1421: If the data comes with GPS coordinates (such as the location of the event reported by the mobile police terminal), directly extract.

[0109] S1422: If it is an address description (such as "XX Road XX No. Shop entrance" in the regulatory record), call the geocoding service (such as Gaode API, Baidu API, or the address library matching service inside the regulatory department) to convert it into latitude and longitude coordinates.

[0110] S1423: If it is a fixed camera or other device, obtain its geographic location from the camera installation location recorded in the device account or GIS system.

[0111] S1424: Convert all geographic coordinates to a standard coordinate system (such as WGS84 or National 2000 Coordinate System).

[0112] S15: Preliminary target detection, preliminary target detection is mainly for image or video data, such as surveillance pictures, etc., which specifically includes:

[0113] S151: Load a pre-trained general target detection model (for example, YOLOv7 optimized for urban scenes, Faster R-CNN in Detectron2, which have been trained on COCO, Cityscapes, and specific security datasets).

[0114] S152: Perform target detection on image frames that pass quality assessment, obtaining the bounding box, preliminary class label (such as "person", "car", "bicycle"), and confidence score of potential security-related targets (such as pedestrians, faces, motor vehicles, non-motor vehicles, backpacks, dangerous goods signs, etc.) in the image.

[0115] S153: Filter out low-confidence detection results based on a pre-set confidence threshold (for example, 0.5 for regular target detection, and appropriately lower for key targets such as faces to improve recall rate, and then further refine).

[0116] S154: According to the detected bounding box, crop the target area image block (Patch) from the original image, for example, crop individual pedestrian images, vehicle images.

[0117] S16: Data packaging and output, specifically including:

[0118] S161: Package the processed data into a unified structured object / message, which in the security scenario includes: a globally unique target ID (which can be generated based on timestamp, camera ID, detection box location), target image block (if any), target preliminary class (such as "pedestrian"), accurate time and space stamp (snap time, latitude and longitude), original data source information (such as camera number, regulatory ID), preliminary detection confidence, etc.

[0119] S162: The encapsulated data object, especially the target image block contained therein and the related metadata (such as the accurate spatiotemporal stamp, the preliminary category label, etc.), is sent to the D 2 CFAN for deeper semantic feature analysis.

[0120] Step S2, the feature extraction step includes: extracting a deep, discriminative semantic feature vector and auxiliary understanding information from the target image which is adaptive to complex scenes.

[0121] The present application improves the D 2 CFAN model, which can extract a deep, discriminative semantic feature vector that is more adaptive to complex security scenes (such as light changes, occlusions, small targets, and similar appearances).

[0122] As a preferred embodiment, step S2 includes:

[0123] S21: Based on the D 2 CFAN model, a feature extraction model is constructed, specifically including:

[0124] S211: Basic feature extraction and multi-level semantic representation construction, as the "perception front end" and "original data supplier" of the model, to extract features at different depths to obtain feature maps that can represent semantic information at different levels of abstraction.

[0125] To provide the D 2 CFAN model with rich and hierarchical initial feature input, the present application draws on the deep convolutional network architecture, but does not simply reuse a specific grid structure, but uses it as the front end of feature extraction, i.e., the basic feature extraction path. The network structure and parameter selection of the present scheme depend on maximizing the performance of the subsequent innovative sub-modules (DHDA, ICCA, CAFA).

[0126] From different depths (i.e., different calculation stages) of the basic feature extraction path, feature maps that can represent semantic information at different levels of abstraction are obtained. These feature maps form the basis of the feature pyramid, defined as F_low, F_mid, and F_high, which capture visual information from low-level texture details to mid-level component structures to high-level abstract concepts. This multi-level representation is a prerequisite for the subsequent dynamic hierarchical decoupling attention sub-module for fine processing.

[0127] S212: Constructing a dynamic hierarchical decoupling attention sub-module DHDA to dynamically decouple the feature channels of each level of feature map into multiple semantic subgroups and apply attention to each semantic subgroup to achieve dynamic semantic decoupling and preliminary attention focusing on different levels of features.

[0128] The implementation of multiple semantic subgroups includes dynamic subspace projection, subspace attention, and selective reorganization, in which:

[0129] Dynamic subspace projection: for each layer of feature map F_i (i ∈ {low, mid, high}), it is projected into K different feature subspaces {S_i1, S_i2,..., S_iK} through a small learnable convolutional network (called "subspace projector", SP_i).

[0130] The present application considers that different levels of feature maps contain different levels of abstract semantic information, and different projection methods are needed to decouple them, so F_i and SP_i are in a one-to-one correspondence. SP_i can learn how to dynamically extract and group features according to the input features using its internal K processing channels, rather than fixed channel grouping. Each processing channel of SP_i is responsible for extracting and outputting a specific type of semantic information from the input F_i, and the output is one of the K feature subspaces S_i2.

[0131] For example, in the security scenario, when analyzing a photo of a target person taken by a surveillance camera, assuming that the F_mid level feature map of the photo is being processed, after receiving F_mid, the corresponding subspace projector SP_mid will start working simultaneously with its internal K different "analysis engines": analysis engine 1 may be specially trained to extract and output information about "clothing main contour and wrinkles" from F_mid, forming feature subspace S_mid1; analysis engine 2 may focus on identifying and outputting "the shape and approximate type of carrying objects (such as backpacks, handbags)", forming feature subspace S_mid2; and so on, to get the Kth feature subspace S_midK. This SP_i is automatically determined by learning how to most effectively decouple the input F_i into these K meaningful and different emphasis feature subspaces, rather than relying on a pre-set fixed grouping method. These K subspaces {S_i1, S_i2,..., S_iK} together constitute a more detailed and multi-dimensional decomposition of the original F_i information.

[0132] Subspace attention: in the above dynamic subspace projection step, the feature map F_i has been decomposed into K feature subspaces S_ij that focus on different semantic aspects (such as S_mid1 focusing on "clothing main contour and wrinkles" and S_mid2 focusing on "carrying object shape" for F_mid).

[0133] For each such semantic subspace S_ij, the system will independently apply a lightweight attention module. This attention module is a combination of spatial attention and channel attention. Its role is to perform a "deep focus" on the current semantic subspace S_ij, to learn and enhance its most discriminative feature representation.

[0134] Continuing with the above security example, for the semantic subspace S_mid1 that focuses on "clothing main silhouette and wrinkles", the attention module applied on it will carefully examine these silhouette and wrinkle information, and possibly pay special attention to an "abnormally prominent shoulder contour" or an "unusual wrinkle at a specific location", as these details can be crucial for individual recognition. Similarly, for the semantic subspace S_mid2 that focuses on "carry-on shape and general type", its corresponding attention module will concentrate on analyzing this part of information, and possibly highlight a "unique square patch on the backpack" or a "special handle design on the tote bag". By individually applying such attention to each S_ij, the system is able to dig out the most core and attention-worthy details from each semantic side. After this step, each original subspace S_ij is refined and strengthened, forming a corresponding weighted subspace feature AS_ij. Thus, we now have a set of "refined" semantic features {AS_i1, AS_i2,..., AS_iK}, where each AS_ij is more capable of highlighting the key information of its corresponding semantic aspect than the original S_ij.

[0135] Selective recombination: After the "refinement" by subspace attention, we have a set of high-quality, weighted subspace features {AS_i1, AS_i2,..., AS_iK} that each focus on a specific semantic aspect (e.g., refined "clothing silhouette" feature AS_mid1, refined "carry-on" feature AS_mid2, etc.).

[0136] This step needs to intelligently integrate these information from different semantic sides to form a comprehensive and highlighted final representation F'_i for the current level feature map F_i. This task is accomplished by a "Gated Aggregation Unit" (GAU_i). GAU_i will learn a weight vector W_i = [w_i1,..., w_iK], where each weight w_ij corresponds to a weighted subspace feature AS_ij. This weight vector W_i acts like a "commander", deciding which semantic side should have more "speak" in the final combined feature F'_i = Σ(w_ij * AS_ij), (j = 1, 2,..., K), according to the overall situation of the current input image and the importance of each "refined" semantic feature AS_ij.

[0137] Assume that in the current target person photo, the facial details are very blurred due to light reasons, but the clothing and carrying objects are relatively clear and characteristic. In this case, GAU mid will learn to give those attention-enhanced "clothing silhouette" feature AS mid1 and "carrying object" feature AS mid2 higher weights w mid1 and w mid2. If a semantic side (such as the posture feature AS mid3) in the photo becomes unreliable or has less information due to occlusion or other reasons, its corresponding weight AS mid3 may be lower. In this way, GAU i enables the network to dynamically and selectively aggregate those decoupled and attention-enhanced semantic information according to the current specific input information, ensuring that the final output feature F'_i can maximize the use of the most valuable clues, thereby providing the strongest support for subsequent recognition tasks. If all decoupled semantic information is valuable, they may all get relatively balanced weights.

[0138] As can be seen, unlike traditional multi-head self-attention, DHDA dynamically and learnable decouples semantic information in the feature channel dimension, independently applies attention to the decoupled sub-features, and then intelligently aggregates them. This enables the model to more finely capture and distinguish information at different levels (e.g., the clothing texture of pedestrians, the shape of carrying objects, and the overall posture).

[0139] S213: Constructing an instance-level and class-level collaborative attention sub-module ICCA, which is a key step for the model to improve robustness by enhancing the discriminability of features from both instance and class dimensions. This step is used to make the model focus on the instance uniqueness and class commonality of the target at the same time before extracting the final feature, and to make them mutually enhance each other. The specific implementation method includes the following instance branch, class branch and collaborative enhancement.

[0140] Instance branch: First, the features F'_i, i = high (e.g., the target person features that have been analyzed in detail by the DHDA) obtained from the previous stage are fed into an "instance branch". The core task of this branch is to learn and identify those subtle and unique features that distinguish the current specific target ("this one" target person) from all other different individuals (other pedestrians, other vehicles, etc.). Just like an experienced detective searching for a specific target person "Zhang San" in a crowd of people, he will try his best to search for those personal marks that only belong to "Zhang San" in his image F'_i. For example, this branch may locate "a not obvious mole above Zhang San's left eyebrow" or "his habit of turning his right ankle inward when walking" through contrast learning (learning to distinguish different individuals with different IDs in the model training stage) or a discriminative head specially trained to identify specific instances. After processing by this branch, an "instance attention map" M_inst is generated, which is like a highlight map that clearly marks which areas in the original features F'_i are crucial for confirming "this is Zhang San, not Li Si or Wang Wu".

[0141] Category branch: At the same time, the features F'_i are also input into another parallel "category branch". Unlike the instance branch, which focuses on "who is he", the goal of the category branch is to confirm "what is he", i.e., to identify the general category to which the target belongs and its robust features. In the security scenario, this means judging whether the target in front is "a pedestrian", "a car", or "a backpack", etc. For example, when analyzing the image F'_i of the target person "Zhang San", this branch will focus on those common structural features that all "pedestrians" should have, such as "a roughly human-shaped outline with a head, torso, and limbs". This is usually achieved through a category classification head. This branch will generate a "category attention map" M_cat, which will highlight the areas in F'_i that are most critical for determining that "this is a pedestrian" as the basic category attribute. This step is very important because it helps the system first distinguish the relevant target category (such as pedestrians) from the complex scene, thereby excluding trees, buildings or other non-target objects in the background, and avoiding confusing them with specific instances.

[0142] Synergistic enhancement: Based on the above instance branch and category branch, there are two important "intelligence": one is M_inst from the instance branch, which points out the unique features of "Zhang San"; the other is M_cat from the category branch, which confirms that "Zhang San" is first a "pedestrian". The next step of "synergistic enhancement" is to intelligently combine these two pieces of intelligence to form a more comprehensive and accurate judgment.

[0143] The system will perform element-level operations on the instance attention map M inst and the category attention map M cat (such as multiplying the attention values of the corresponding positions of the two maps, or adding them and then normalizing them through an activation function), thereby generating a "collaborative attention map" M collab = f(M inst, M cat). This M collab combines two judgments, and the highlighted area meets both "pedestrian" and "has unique features of Zhang San". It is equivalent to target locking, which is a 'pedestrian' (category confirmation), and according to its'mole on the left eyebrow' and 'varus of the right ankle' (instance details), it can be highly determined that he is 'Zhang San'! Finally, the system applies this collaborative attention map M collab back to the original features F'_i (by element-wise multiplication: F''_i = F'_i * M_collab). The final features F''_i obtained in this way are effectively "guided", and they will pay more attention to key information areas that meet both category commonality and individual uniqueness.

[0144] ICCA can clearly separate and collaborate the attention of instance-level and category-level information, so that the model can grasp both the details that make the individual unique (such as specific marks, minor actions) and the category commonality (such as human shape, vehicle type) when performing Re-ID (such as finding people, finding cars) tasks, and resist the interference brought by light changes, partial occlusions or changes in perspective, thereby significantly improving the accuracy and robustness of identification. For example, even if "Zhang San" changes a coat (some visual instance features change), as long as his core unique biological features (such as moles, gait) and the basic form as a "pedestrian" are still identifiable, the system still has a high probability of accurately identifying him through this collaborative mechanism.

[0145] As can be seen, ICCA learns both the uniqueness of the instance (distinguishing between Zhang San and Li Si) and the commonality of the category (both are pedestrians) when extracting features, and through the attention mechanism, the two are collaboratively enhanced, making the extracted features both discriminative and robust.

[0146] S214: Construct a context-aware fusion attention sub-module CAFA, which is responsible for intelligently fusing multi-source, multi-level features from the front, and dynamically adjusting according to the real-time scene context, for fusing features from different processing stages.

[0147] Specifically, the features of F'_low, F'_mid processed by DHDA and F''_high processed by ICCA are fused, while considering the context information of the input image block, which specifically includes:

[0148] Contextual Feature Extraction Step S2141: In security surveillance, the quality of a picture taken by the same camera at different times or under different weather conditions can vary greatly. The first step of CAFA is like a seasoned surveillance room officer who, before analyzing a specific target, would first assess the overall "shooting environment" or "context" of the current picture. It extracts a compact "context descriptor" V_ctx from the original input image patch (or from the features obtained at an early stage of the base feature extraction pipeline) through a small convolutional network. This V_ctx is like a short environmental report that encodes key contextual information such as the overall lighting conditions of the image (e.g., "daylight with sufficient light", "night with insufficient light and strong light source interference"), image clarity (e.g., "clear picture", "picture blurred due to rain and snow"), and background complexity (e.g., "simple and empty background", "crowded and chaotic background") and so on.

[0149] Multi-source Feature Preparation Step S2142: After the previous DHDA and ICCA sub-modules, the system has extracted multiple levels and types of feature information for the current target of interest (such as a suspicious pedestrian). These features may include: F'_low from DHDA, focusing on low-level textures and details (e.g., fine grid patterns on clothing); F'_mid, focusing on mid-level components and structures (e.g., "the pedestrian is wearing a dark baseball cap", "left hand holding a rectangular briefcase"); and F'_high, high-level semantic features that have been strengthened by ICCA to enhance instance uniqueness and class commonality. To facilitate subsequent unified processing and fusion, these feature maps from different stages, possibly with different spatial dimensions (F'_low, F'_mid, and F'_high), are converted into fixed-length feature vectors through adaptive pooling operations, referred to as the feature vector set {V_low, V_mid, V_high_collab}.

[0150] Context-guided Gated Fusion Step S2143: This is the core of CAFA, which intelligently decides how to combine the various feature vectors {V_low, V_mid, V_high_collab} prepared in step b according to the current "shooting environment" (V_ctx) perceived in step a to form the most reliable and comprehensive final judgment of the target, including the following steps S21431-S21433:

[0151] S21431: Concatenate the context descriptor V_ctx with each of the feature vectors V_j to be fused. That is, the system will inform each of the feature vectors V_j to be fused (i.e. any one of {V_low, V_mid, V_high_collab}) of this "environment report" V_ctx by concatenating V_ctx with each V_j. In this way, each feature is "aware" of the conditions under which it was extracted when its importance is evaluated.

[0152] S21432: The system will calculate a "trustworthiness" or "importance" gating weight g_j = sigmoid(MLP(concat(V_j, V_ctx))) for each "environment-aware" feature V_j by a small multi-layer perceptron (MLP). The size of this weight g_j will be significantly influenced by the current environment V_ctx. For example, in a security scenario: if V_ctx indicates that it is "night, extremely poor light, and the camera has stains resulting in a very blurred picture", then the gating weight g_low for the low-level texture feature V_low (e.g. clothing grid pattern) that relies on high definition may become very small, indicating that "under such poor conditions, these detailed information is almost not reliable, and its dependence should be greatly reduced". On the contrary, for some relatively more anti-interference high-level semantic features V_high_collab (such as "overall human profile and general motion trend"), their reliability may still be relatively high even in harsh environments, so their gating weight g_high_collab may be assigned a relatively large value. If V_ctx indicates that it is "day, sunny, and picture extremely clear", then g_low may become very large because the detailed features are very reliable and have high discriminability at this time.

[0153] S21433: The system will weight and sum all these feature vectors V_j according to their respective "trustworthiness" g_j under the current environment to obtain the final fused feature V_final = Σ(g_j * V_j). This V_final is the most optimized and adaptive comprehensive feature description of the target after fully considering the actual observation conditions of the security monitoring site.

[0154] As can be seen, the fusion weight of the present application is no longer based on the feature itself, but is explicitly influenced by the global context information of the input image. This mechanism enables the model to dynamically adjust the degree of dependence on different sources of information according to the actual situation (such as light, weather, image quality), like an experienced security personnel, so as to maintain a high level of analysis ability and judgment accuracy in the complex and variable urban security environment.

[0155] S215: Output feature vector, which is used to pass the features obtained by the context-aware fusion attention sub-module through one or more fully connected layers to obtain the final feature vector as the output backend of the model, and convert the complex fusion features into standardized feature vectors that can be used for downstream tasks, specifically including:

[0156] S2151: Output standardized feature vector. After intelligent fusion by CAFA, a feature V_final that integrates multiple aspects of information and takes into account the context of the scene is obtained. This feature V_final contains very rich information, but it may also have a high dimension or contain some components that are not the most core for the final recognition task. Therefore, the system will pass it through one or more fully connected layers (FC layers). The role of this step is similar to the final "editing and condensing" of a detailed but possibly slightly cumbersome "case investigation report" (V_final), removing redundant information, extracting the most essential and representative core content, and adjusting it to a dimension suitable for subsequent efficient comparison and storage.

[0157] S2152: Output standardized feature vector, output auxiliary understanding information. After the above refinement and dimension adjustment, the system will output a final, fixed-length feature vector (for example, it can be 512-dimensional, 1024-dimensional, or 2048-dimensional, etc. according to specific application requirements). This feature vector can be regarded as a highly condensed, digital "identity signature" or "feature file" of the analyzed target (such as a pedestrian or a vehicle) in the current monitoring picture. In the security system, this "identity signature" is crucial because the subsequent target retrieval module 4 can use it to quickly and accurately find other target records with similar "identity signatures" in a large historical database, thereby realizing key functions such as image search and vehicle search.

[0158] S2153: In order to enhance the transparency and interpretability of the system, in addition to outputting the core feature vector, the system can also selectively output some intermediate results generated during the processing of the various innovative sub-modules (such as DHDA, ICCA, and CAFA). These intermediate results may include: the attention maps assigned to each subspace in DHDA (revealing which semantic sub-features are being focused on), the instance attention and class attention maps generated in ICCA (showing which areas embody individual uniqueness and class commonality), and the gating weights calculated for different source features in CAFA (reflecting the degree of trust that the system has in different information in a particular scenario).

[0159] S22: Optimize training based on the feature extraction model and set the loss function, specifically including:

[0160] S221: Knowledge transfer initialization and field adaptability fine-tuning, including:

[0161] Layered pre-training weight loading: D 2 The basic feature extraction path of the CFAN model (see step S211 for details) is initialized using mature deep convolutional network weights pre-trained on large general image datasets to introduce general visual prior knowledge and accelerate subsequent training convergence.

[0162] End-to-end fine-tuning for security scenarios: the entire D 2 CFAN model, including DHDA, ICCA and CAFA core innovative modules, will be fine-tuned end-to-end on large-scale labeled datasets for specific security tasks (such as pedestrian re-identification, vehicle re-identification).

[0163] S222: Optimize the main identity recognition loss, and the DHDA level weighted semantic decoupling orthogonality regularization.

[0164] To optimize the D 2 CFAN model performance, and give full play to the unique advantages of each core component (DHDA, ICCA), this step includes the main recognition loss and innovative auxiliary loss and regularization terms for DHDA semantic decoupling and ICCA collaborative mechanism. These customized loss terms together constitute the composite loss function L_total.

[0165] Main identity recognition loss (L_ReID): advanced metric learning loss functions such as Triplet Loss, Circle Loss or ArcFace Loss are used to optimize the D 2 CFAN model final output feature vector V_final (S2.15.b). The core goal of this loss function is to maximize the distance between different identity (ID) target features in the learned feature space (inter-class separability), while minimizing the distance between features of the same identity target in different observation samples (intra-class compactness). The selection of this loss function is D 2 The final feature output of CFAN directly gives strong identity discrimination ability.

[0166] DHDA level weighted orthogonality regularization (L_DHDA_ortho_weighted): To promote the dynamic subspace projectors SP_i in DHDA to learn feature subspaces {S_i1, S_i2,..., S_iK} that are truly independent (orthogonal) and complementary in semantics, and considering that different semantic levels (low, mid, high) may have different requirements for the degree of decoupling, this scheme designs a level-weighted orthogonality regularization term. For each layer of feature maps F_i (where i ∈ {low, mid, high}), the K subspace feature maps S_ij obtained by SP_i projection are first converted into corresponding feature vectors s ij = GAP(S_ij) by global average pooling (GAP) operation. This regularization term aims to minimize the cosine similarity between different subspace feature vectors within the same layer (i.e., maximize their orthogonality), and allows different importance weights ω i to be assigned to decoupling of different levels. The regularization formula is:

[0167]

[0168] where: s ij · s il represents the dot product of vectors s ij and s il . ||s ij ||2 represents the L2 norm (i.e., the length) of vector s ij . ∈0 is a very small normal number (e.g., 1 × 10 -8 ) to prevent the denominator from being zero and ensure numerical stability. ω i is the decoupling importance weight coefficient set for the i-th semantic level (low, mid, high). For example, in some tasks, clear decoupling of mid-level semantics may be more critical than low-level, in which case ω mid can be assigned a relatively high value. These weights ω i can be set as hyperparameters according to experience, or optimized through experiments. This regularization term penalizes linear correlation between subspaces through squared cosine similarity, encouraging each subspace to capture unique semantic information in F_i, thereby improving the decoupling quality of DHDA and the richness of feature expression. The introduction of level weights ω i is a specific design of this scheme for the multi-level characteristics of D 2 CFAN, allowing model training to focus more on optimizing the decoupling effect of key semantic levels.

[0169] ICCA instance-category joint confidence maximization loss (L_ICCA_jcm): The core goal of ICCA is to synergize instance uniqueness and category commonality. To more directly reinforce this "synergistic" effect and ensure the model has high confidence correct judgments at both levels, this scheme designs an "instance-category joint confidence maximization loss". This loss not only depends on the respective discriminative losses of the instance branch and the category branch (e.g., standard cross-entropy losses L_inst and L_cat), but also introduces a term that directly measures and optimizes the joint prediction confidence. Assuming that for the input feature F'_i, the prediction probability of the corresponding correct instance ID output by the instance branch is and the prediction probability of the corresponding correct macro-category output by the category branch is The "joint prediction confidence score" S joint is defined as:

[0170]

[0171] This score will be higher only when the model makes high-confidence correct predictions for both the instance ID and the macro-category it belongs to. Based on this score, the joint confidence maximization loss L_ICCA_jcm is designed as follows:

[0172]

[0173] where E[·] represents the expectation over the training data distribution, and ∈1 is a small normal number to ensure numerical stability. Therefore, the total loss L_ICCA of ICCA is designed as:

[0174] L_ICCA = λ inst L_inst + λ cat L_cat + λ jcm L_ICCA_jcm

[0175] where λ inst , λ cat , λ jcm are the weight coefficients of each loss. The introduction of L_ICCA_jcm is an enhancement of standard multi-task learning. Instead of optimizing each task independently, it explicitly encourages the model to learn the inherent consistency and synergistic relationship between instance features and category features. By maximizing the joint confidence, the model is guided to a state where its accurate identification of instances is based on the correct understanding of the categories they belong to, and vice versa, so that the "synergistic attention" of ICCA is well deserved.

[0176] Total loss function (L_total):

[0177] is finally used to optimize the entire D2 The total loss function of the CFAN model is the weighted sum of the above losses:

[0178] L_total = w reid L_ReID + w ortho L_DHDA_ortho_weighted + w icca L_ICCA

[0179] where w reid , w ortho , w icca are the weight hyperparameters of each loss, which reflect the relative importance of different optimization objectives in the overall training process, and can be determined according to task requirements, model convergence, or through automated hyperparameter search methods (such as Bayesian optimization).

[0180] S223: D 2 Parameter iterative optimization process of the CFAN model.

[0181] In each training iteration, a mini-batch of samples is extracted from the training dataset and fed into the D 2 CFAN model for forward propagation, and the outputs including the final feature vector are calculated. Then, the total loss of the current batch of data is calculated according to the composite loss function L_total defined in S222 above. The gradient of this total loss with respect to all learnable parameters of the model is calculated (i.e., backpropagation). Finally, using optimization algorithms such as AdamW, the model parameters are updated based on the calculated gradient. This iterative process (forward propagation, loss calculation, backpropagation, parameter update) will be repeated for multiple training epochs until the model's performance on the validation set reaches the pre-set convergence criteria or the pre-determined number of training epochs is completed. During training, data augmentation techniques and learning rate scheduling strategies are usually used to improve training effectiveness and model generalization ability.

[0182] S23: Preprocessing and normalization of the trained feature extraction model in the inference stage.

[0183] After training and deployment, the input target image block is normalized and preprocessed to ensure consistency with the training stage and activate the best performance of the model.

[0184] a. Target image block receiving and size alignment: The target image block is received from the data access and preprocessing module 1, and is adjusted to the standard input size (e.g., HxW pixels) required by the CFAN model's basic feature extraction path through scaling, cropping or padding (maintaining the aspect ratio). 2 CFAN model basic feature extraction path requires standard input size (e.g., HxW pixels).

[0185] b. Pixel value normalization: the resized image patch is subjected to pixel value normalization, i.e. pixel value normalization processing.

[0186] S24: Feature extraction, forward propagation and internal dynamic regulation, including:

[0187] S241: Data input and layer-by-layer feature forward calculation: the preprocessed image patch tensor is input to the D 2 CFAN model, the data flow through the core components defined in S2.1, including the basic feature extraction path, DHDA, ICCA, CAFA and the final feature vector generation layer.

[0188] S242: DHDA subspace attention dynamic gain adjustment: in the subspace attention processing step of DHDA, the intensity or concentration of the attention weight map (such as spatial attention map M_s,ij and channel attention map M_c,ij) calculated for each semantic subspace AS_ij is quantified as a "subspace activation quality score" q_ij. For example, q_ij can be the ratio of the maximum activation value to the average activation value in M_s,ij. In the subsequent selective recombination step, the weight w_ij learned by the gating aggregation unit GAU_i can be dynamically modulated by this quality score:

[0189] w'_ij=w_ij·sigmoid(β·(q_ij-q_threshold))

[0190] where β is a scaling factor and q_threshold is an activation threshold. This mechanism allows subspaces with more focused attention and stronger signals to obtain higher weights in the final feature fusion.

[0191] S243: Adaptive sharpening of context-aware gating in CAFA: in the context-guided gating fusion step (S2143) of CAFA, the gating weight g_j generated for each to-be-fused feature vector V_j is g_j = sigmoid(MLP(concat(V_j, V_ctx))), the logical value calculated inside (i.e. the output of MLP, denoted as z_j = MLP(concat(V_j, V_ctx))) can be adjusted by a learnable or context V_ctx determined temperature parameter τ j (or a global temperature parameter τ) before passing through the sigmoid activation function. The adjusted gating weight g'_j can be represented as: g'_j = sigmoid(z_j / τ j ). When the temperature parameter τ jWhen smaller (e.g. close to 0 but greater than 0), the sigmoid function curve becomes steeper, making the output gating weight g'_j more close to 0 or 1, which means the model is more "decisive" or "determined" in its decision on whether to adopt the feature source V_j. Conversely, when g'_j is larger, the sigmoid curve is more gentle, and the output weight tends to be more intermediate, indicating a more "soft" or "undetermined" decision. This temperature parameter τ j The temperature parameter τ itself can also be designed as the output of a small network h(·) with the current context descriptor V_ctx as input, i.e. τ j = h(V_ctx). This adaptive sharpening mechanism enables the model to dynamically adjust its "decision decisiveness" in fusing different feature sources according to the overall certainty of the scene (e.g. the image quality, sufficiency of information, etc. perceived by V_ctx), being more confident in choosing when information is sufficient, and more conservative in fusing when information is insufficient.

[0192] S25: Obtain the output feature vector and perform quality assessment and directional transmission

[0193] D 2 The CFAN model outputs a fixed-length feature vector V_final for each input target image block (S2152), which is normalized (e.g. L2 normalization) to adapt to subsequent tasks. The core output feature vector V_final, together with the acquired metadata (target ID, spatiotemporal stamp, preliminary category, etc.), are encapsulated as a structured target feature data record. All records are sent to the spatiotemporal semantic hypergraph index construction module.

[0194] Step S3, constructing a data index includes: based on the deep features of the target and the spatiotemporal information, constructing a spatiotemporal semantic hypergraph index of multiple types of nodes and hyperedges to express the complex relationships between targets, specifically including:

[0195] S31: Constructing the vertices and hyperedges of the spatiotemporal semantic hypergraph index;

[0196] S311: Constructing the vertices of the spatiotemporal semantic hypergraph index, please refer to Figure 2 These vertices are divided into the following categories, which together form a basic, hierarchical spatiotemporal coverage (i.e. these vertices are distinguished by "size" and "functionality", and they are layered one on top of the other, managing target information in the city from coarse to fine):

[0197] A: Macro spatiotemporal block vertex (V MBlock ): represents a top-level spatiotemporal unit that performs coarse-grained division on the entire monitored city or region. For example, the entire city map is divided into large blocks according to time and region, such as "Chaoyang District on April 25th morning this large region". A V MBlockThis represents a "macro-spatiotemporal large area". This is the top layer of the index network, like the entrance to a large container, which can serve as the entry point and initial filtering layer for large-scale spatial and temporal range queries.

[0198] B: Adaptive semantic region vertex VARegion: When a V MBlock When the data density or semantic complexity within V reaches a certain threshold, this V MBlock Instead of simple uniform subdivision, it is divided into one or more VARegions. The subdivision method and boundaries of each VARegion are adaptive, and can be determined based on the spatiotemporal distribution characteristics of its internal data (such as clustering along streets) and D. 2 The semantic distribution of CFAN feature vectors (e.g., forming several semantic clusters) is dynamically determined. For example, in the large area of ​​"Chaoyang District on the morning of April 25th," some places may be particularly busy or have particularly complex situations, such as "Sanlitun Commercial Street during the evening rush hour." VARegion represents such a "key focus area" "intelligently" divided based on actual conditions (how many people, what kind of people / vehicles). Its boundary may not be a perfect square, but rather "outlined" according to the actual situation. VARegion is V MBlock This large area is further divided into smaller, more specific regions. A V MBlock There can be several different VARegions. Defining VARegion elements enables refined and intelligent management of complex regions, making the index structure more closely match the actual data distribution.

[0199] C: Vertex of the basic spacetime unit (V) Cell A VARegion can be further subdivided into the smallest granular, standardized spatiotemporal unit vertices. For example, within the VARegion of "Sanlitun Commercial Street during the evening rush hour," it can be further subdivided into very, very small standard grids, such as "the 10×10 meter area in front of a certain store during the minute from 7 PM to 7:01 PM." Cell This represents a "miniature spacetime grid." This element is the basic container that holds references to specific target instances, namely V. Cell It is the smallest cell within a VARegion, used to precisely "locate" the target.

[0200] D: Target instance vertex V Object: represents a specific, complete information object instance O coming from the feature extraction module 2. VObject is the most atomized vertex in STSGHI, and is the final landing point of all queries and analyses. Specifically, this is the "person" or "vehicle" that is really being looked for, such as, in the above "10x10 meter ground at the entrance of the store, 7 to 7:01", the camera has captured a "pedestrian A wearing a red hat and a white T-shirt", and the specific snapshot of this pedestrian A is a V Object . Object This is the most basic and specific information point in this information network, and it "lives" in a VCell small cell.

[0201] A target instance V Object appears in a micro-time-space small cell VCell → this VCell belongs to a key attention area VARegion → this VARegion is located in a macro-time-space large block VMBlock. This hierarchical relationship from large to small can facilitate fast spatial and temporal positioning.

[0202] S312: Constructing the hyperedges (Hyperedges: HE) of the spatio-temporal semantic supergraph index, including spatio-temporal co-occurrence hyperedges, semantic similarity hyperedges, trajectory / event association hyperedges, and region attribute association hyperedges.

[0203] Hyperedges HE are the core of STSGHI for expressing complex and multi-element relationships. A hyperedge can connect two or any number of vertices (which can be any combination of VMBlock, VARegion, VCell, V Object In a popular way, the role of hyperedge HE is to connect those "information nodes" (vertices) in the city in various meaningful ways to inform users that there is a certain "story" or "association" between them. Hyperedges mainly connect vertices, and they do not generally have "relationships" between them, but indirectly produce connections through their common vertices, or in analysis, multiple types of hyperedges are considered to understand the complete situation of a target. For example, pedestrian A (V Object _A) may be connected by a HESTC (he and B appear together) and a HETrack (his own trajectory). These two hyperedges are indirectly associated because they are connected to A. In the city security scenario, it may be required to "find someone who has appeared with A (check HESTC) and behaves like a certain pattern (check HE SemSim )". This combines the use of different types of hyperedge information. The following details the various types of hyperedges defined in the structure of STSGHI:

[0204] a. Spatio-temporal co-occurrence hyperedge (HE STC): Connects a set of target instance vertices V that coexist within a specific spatiotemporal neighborhood. Object Or fine-grained unit vertices (V) containing these instances Cell For example, if a camera captures pedestrians A, B, and C at the same intersection within one minute, then there is a traffic violation (HE). STC Take the three Vs: A, B, and C. Object Connecting them indicates that they "appeared at the same time and in the same place." This definition of the hyperedge element helps in the rapid discovery and querying of spatiotemporally associated targets.

[0205] b. Semantic similarity hyperedge (HE) SemSim ): Connect a set of its D 2 CFAN eigenvectors FVec O Target instance vertices (V) exhibiting high semantic similarity Object Even if they are not adjacent in space and time. That is, those that may not appear in the same place or at the same time, but have the same "appearance" (D) 2 Target instances V with very similar CFAN features or behavioral patterns Object Connect them. For example, if the system detects that Zhang San in the east of the city is wearing a distinctive fluorescent green jacket, and Li Si in the west is wearing the exact same jacket, then there is a HE (Hexagram / Hypertext Mark). SemSim Put Zhang San and Li Si as the two V's Object When put together, this indicates that their clothing features are highly similar. This effectively supports pure semantic similarity retrieval across time and space, uncovering potential connections.

[0206] c. Track / Event-Associated Hyperedges (HEEvent / Track): Connect a series of time-dependent V-shaped edges that form a complete motion trajectory. Object (Representing snapshots of the same target at different times), or a group of V objects identified by target retrieval module 4 as participating in the same complex event (such as group gathering or relay relay). Object That is, if a series of V Object HETrack is used to connect the continuous activity trajectories of the same person or vehicle at different points in time. If the target retrieval module 4 finds that several people are involved in a case (e.g., A, B, and C committing a crime together), HEEvent is used to link the V objects of A, B, and C. Object And connect all the vertices of potentially related locations. For example, if camera 1 captures Zhang San's image, then camera 2 captures Zhang San's image 5 minutes later, and then camera 3 captures Zhang San's image 10 minutes later, these three Zhang San's VObjects can be connected by a single VObject. Object Connecting these elements forms his movement trajectory. This hyperedge element supports organizing and querying dynamic target trajectories, expressing higher-order event relationships.

[0207] d: Region attribute associated hyperedge HERegionAttr: Connects a group of VARegions or V regions that share some common region-level attribute or semantic label. MBlock Vertices are regions that share a common "property" (such as multiple VARegions or V vertices). MBlock Connect them. For example, if the system analyzes and finds that three VARegions in a city are all "high-incidence areas of theft", then there will be a HERegionAttr connecting the vertices of these three regions. It supports queries and analysis based on macro-level regional attributes.

[0208] S313: Determine the symbols and attributes of vertices and hyperedges.

[0209] a. General Vertex Attributes (Applicable to VMBlock, VARegion, VCell): ①IDV: Unique identifier of the vertex, i.e., unique number; ②TypeV: Vertex type (MBlock, ARegion, Cell, Object); ③ExtV = (GeoBoundV, TimeBoundV): Geographic boundaries and time range covered by the vertex; ④ChildVertices V :(For non-V Object (Vertex) points to the set of its next-level child vertices; ⑤ParentVertices V :(For non-V MBlock A set of vertices pointing to their parent vertices at the next higher level (Note: In a hypergraph structure, a lower-level vertex may participate in the formation of multiple higher-level regions, and therefore may have multiple parents); ⑥Stat V :Statistical summary of the vertices, including the total number of internal targets (Count) O (V), Main category distribution Dist C at(V), such as Dist C at(V 三里屯傍晚 = {"Pedestrian": 4500, "Car": 800, "Bicycle": 500, "Bus": 50, "Van": 10, ...}

[0210] Count O (V 三里屯傍晚 ) equals the sum of the objectives in the above categories; ⑦SemSum V : Represents the semantic profile of this vertex. Where AllObjects(V) represents the region vertex V (regardless of V) Cell V ARegion Or V MBlock The final recursion includes all original target instances V. Objectthe set of all objects in the region V, |AllObjects(V)| represents the total number of target instances in the set (note that this formula is valid when |AllObjects(V)| > 0; if there is no target in the region, SemSum V is a zero vector).

[0211] b. Target instance vertex V Object attributes (i.e. the attributes of object O): ① IDvo, the unique number of the specific target instance in our index system; ② Locvo, the precise geographic location (longitude, latitude, and possibly height) where the target instance was captured; ③ Tvo, the precise time point when the target instance was captured; ④ FVecvo, the key feature vector describing the “appearance” and “connotation” of the target instance, which is extracted by the CFAN module. 2

[0212] c. Hyperedge HE attributes: ① IDHE: the unique identifier of the hyperedge; TypeHE: the type of the hyperedge (STC, SemSim, Track / Event, RegionAttr); ② Vertices HE = {V a ,V b ,...,V n}: the set of vertices connected by the hyperedge; ③ WeightHE: the association strength represented by the hyperedge, i.e. how strong the association is of this “relationship line”. For example, two people just passing by and a gang closely cooperating, the “association strength” is obviously different; ④ TimestampHE: the time when the hyperedge association relationship is formed or effective, i.e. when this “relationship line” is established or in which time period it is effective; ⑤ SemSum HE : the semantic summary of the hyperedge itself, i.e. the “semantic portrait” of this “relationship line”. For example, if it is a HE SemSim connecting multiple “similarly dressed” targets, its SemSum HE may be a general description of such similar dress. If it is a HE Event , its SemSum HE may be a feature describing the core semantics of the event (such as “group fight”, “suspicious transaction”).

[0213] S32: Initial construction of the spatio-temporal semantic hypergraph index, continuous entry of data objects and dynamic evolution of the index structure, generation and association of relationship hyperedges.

[0214] S321: Initial construction of the index;

[0215] ​a. Preliminary division of macro spatio-temporal blocks: The system first logically constructs a top-level "macro spatio-temporal block vertex VMBlock" set covering the entire target spatio-temporal range according to the set city geographical scope or key monitoring area, combined with the pre-defined time management granularity (e.g., in days or hours).

[0216] b. Preparation of storage structure: Simultaneously, the system initializes the data structure for storing various vertices and hyperedges in the STSGHI.

[0217] S322: Insertion of target instance vertex (V Object ) and dynamic evolution of the underlying structure;

[0218] When the feature extraction module 2 finishes processing a new target object and passes its attribute information (including ID O , Loc O , T O , FVec O , Q O , Cat O ) to the index module 3, the following operations will be performed:

[0219] a. Target instance vertex creation: A corresponding "target instance vertex (V Object )" is generated for the incoming new target object, and its various attributes are recorded in full.

[0220] b. Target positioning in the index hierarchy: According to the spatio-temporal attributes (Loc O , T O ) of the V Object , navigate top-down in the hierarchical structure of the STSGH (from VMBlock to VARegion), and finally accurately place it in a leaf-level "basic spatio-temporal unit vertex VCell". If the target position corresponds to a non-existent VCell, dynamically create a VCell vertex structure containing only this V Object .

[0221] c. Adaptive region structure evolution: When the data density (e.g., the number of V Object or VCell) within a "macro spatio-temporal block vertex (VMBlock)" or "adaptive semantic region vertex (VARegion)" exceeds a pre-set threshold (this threshold is a positive integer greater than 0), or the semantic complexity SemSumV of the data within it reaches a certain value, the adaptive adjustment mechanism of the region vertex will be triggered. This mechanism will comprehensively analyze the spatio-temporal distribution pattern of the existing data in this region (such as street distribution, regional aggregation) and D 2The semantic aggregation property revealed by CFAN feature vectors is used to re-partition the region, generating a new set of finer-grained "adaptive semantic region vertices VARegion" or "basic spatio-temporal cell vertices VCell". This process ensures that the index structure can dynamically adapt to the actual distribution of data, maintaining query efficiency.

[0222] d. Update of related vertex summary information: Whenever a new V Object is successfully inserted, or the region hierarchy changes due to adaptive adjustment, the system will update the statistical summary information StatV and semantic image SemSumV of all affected parent region vertices (VCell, VARegion, VMBlock) from bottom to top, to ensure the timeliness and accuracy of these summary data.

[0223] S323: Dynamic generation and association of hyper-edges HE.

[0224] The generation of hyper-edges is the core mechanism for STSGHI to express complex relationships. Its generation process can be based on predefined rules or driven by other analysis modules:

[0225] a. Spatio-temporal co-occurrence hyper-edge HESTC generation: When multiple "target instance vertices V Object are detected to appear simultaneously within a preset short time window ΔT and small spatial neighborhood ΔD, a HESTC type hyper-edge will be automatically created or updated to connect these target vertices with spatio-temporal co-occurrence relationship.

[0226] b. Semantic similarity hyper-edge HESemSim generation: When a new V Object is inserted, its D 2 CFAN feature vector FVecVO is used to efficiently estimate the similarity of other V Object in the index library. For those V Object with feature similarity higher than the preset threshold, even if they have no direct spatio-temporal association, the system will create or add them to the corresponding HESemSim type hyper-edge.

[0227] c. Generation of trajectory / event association hyper-edge HETrack / Event:

[0228] Trajectory integration: Selecting continuous spatio-temporal observation point sequences (each observation point corresponds to a V Object ) belonging to the same moving entity, creating a HETrack type hyper-edge for ordered connection, and explicitly expressing the complete motion trajectory of the target in the index.

[0229] Event Association: When the target retrieval module 4 identifies a specific complex security event (e.g., multiple individuals acting in coordination, the transfer process of a specific item, etc.), all "target instance vertices V Object " related to this event and possibly involving key "region vertices" (VCell or VRegion) can be associated through one or more hyper-edges of the HEEvent type. The attribute SemSumHE of this hyper-edge stores description information about the event type, time, participant roles, etc.

[0230] d. Region Attribute Association Hyper-Edge HERegionAttr Generation: Through analysis or external annotation, it is identified that multiple different "region vertices" (e.g., multiple VARegion) have a certain specific macro attribute or label in common (e.g., all are designated as "high-risk areas", or their "region semantic image SemSumV" presents similar specific patterns), then a hyper-edge of the HERegionAttr type can be created to connect these region vertices, to support macro analysis based on the common attributes of regions.

[0231] Step S4, data query includes: receiving user multi-modal query information, using hypergraph index for candidate region screening, feature matching and relationship reasoning, finally outputting high-confidence target retrieval results, specifically including:

[0232] S41: Query intent analysis and multi-modal input processing;

[0233] This step is responsible for receiving and understanding the user's diversified query requirements and converting them into internal representations that the system can process.

[0234] S411: Receive user query request: the system designs flexible query interfaces to adapt to various retrieval scenarios in urban security practice.

[0235] ① Image query: the user provides a picture containing the target (such as target person, target vehicle).

[0236] ② Text description query: the user inputs a natural language description of the target, such as "looking for a black SUV without license plate that passed the intersection of Zhongshan East Road and Jiefang Road between 10:00 and 11:00 last night, with obvious scratches on the roof".

[0237] ③ Spatio-temporal range query: the user specifies a specific time period and geographic area, such as "retrieve all pedestrians carrying large backpacks near the No. 1 gate of X Park between 9:00 and 9:30 on May 24, 2025".

[0238] ④ Target ID query: the user provides a known target instance vertex ID (IDvo) to query the detailed information of the target, associated targets or historical trajectories.

[0239] 5. Compound query: Any combination of the above query modes, e.g. "Provide a picture of the target person, and limit its last appearance time to within 24 hours, and its activity region to Y district".

[0240] S412: Query pre-processing and feature extraction:

[0241] Image query processing: If the query input is an image, call "D 2 CFAN-based feature extraction module 2" to pre-process and forward-propagate the query image as in S23 and S24, and extract its high-discriminative D 2 CFAN feature vector FVec_query.

[0242] Text description processing: If the query input is a text, extract key entities (e.g. "black SUV", "backpack"), attributes (e.g. "unlicensed", "large"), times (e.g. "10pm last night"), locations (e.g. "Zhongshan East Road"), and behaviors (e.g. "passing by") through named entity recognition. Further structure these information through semantic analysis, and attempt to map some semantic information (e.g. appearance description) to a semantic vector space compatible with the D 2 CFAN feature space, or convert it to constraints on target attributes.

[0243] Temporal and spatial information standardization: Convert all temporal and spatial descriptions in the query to the standard format and coordinate system defined in S14.

[0244] S413: Generate unified query object: Encapsulate all parsed and extracted information into a unified "query object". This object contains: D 2 CFAN feature vector FVec_query of the query image, structured semantic attribute constraints (e.g. target category, color, carried objects, etc.), standardized temporal and spatial range constraints, and other possible query parameters (e.g. expected number of returned results, sorting preference, etc.).

[0245] S42: STSGHI hierarchical pre-filtering and candidate region localization;

[0246] This step utilizes the hierarchical structure and region summary information of STSGHI to quickly narrow down the search range and improve the efficiency of subsequent matching.

[0247] S421: Based on the temporal and spatial constraints in the query, quickly filter out VMBlock and VARegion vertices in STSGHI:

[0248] The system first utilizes the explicit "temporal range" and "geographical boundary" information in the query object.

[0249] By querying the ExtV attributes (i.e. GeoBoundV, TimeBoundV) of the macro-spatiotemporal block vertex (VMBlock) and the adaptive semantic region vertex (VARegion) in the STSGHI, the VMBlock and VARegion that have intersection with the query spatiotemporal range can be efficiently filtered out. This step takes advantage of the hierarchical characteristics of the STSGHI, starting from the top layer for filtering, avoiding full-scan of massive bottom-layer data.

[0250] S422: Intelligent pruning of candidate regions based on regional semantic portrait (SemSumV):

[0251] For the candidate VMBlock and VARegion after spatiotemporal filtering, if the query object contains explicit semantic information (for example, "find suspicious personnel gathering area" extracted from the text description, or the main category of the target in the image query is "vehicle"), the semantic portrait SemSumV of these region vertices can be further utilized for intelligent pruning.

[0252] The similarity or matching degree between the query semantics and the region SemSumV is calculated. For example, if the query is "suspicious gathering", regions with "multiple targets appearing in short time with high density" or containing specific risk semantic labels in SemSumV are preferred. If the query is "vehicle", regions with high proportion of vehicle targets in SemSumV or containing semantics such as "parking lot" and "main intersection" are preferred.

[0253] By introducing semantic pre-judgment at the region level, irrelevant regions can be excluded earlier, further focusing on high-potential regions, significantly improving the accuracy and efficiency of retrieval.

[0254] S43: Fine-grained target matching and initial candidate V Object set generation;

[0255] In the pre-screened candidate regions, more detailed target matching is performed.

[0256] S431: Locating V Object : The system further navigates to the basic spatiotemporal unit vertices (VCell) contained in the candidate VARegion determined in S42. Then all the reference of target instance vertices (V Object ) stored in these VCell are traversed.

[0257] S432: Core semantic similarity matching based on D 2 CFAN feature vector (FVecVO):

[0258] This is the core matching step. If the query object contains a query feature vector FVec_query (from an image query or text semantic conversion), the semantic similarity between FVec_query and the feature vector FVecVO of each V Object being traversed is computed (using cosine similarity formula).

[0259] A dynamic or preset similarity threshold is set. Only V Object with similarity higher than the threshold are considered as potential matches. 2 The high discriminative feature of CFAN model ensures that this deep semantic based matching can effectively deal with the challenges of complex security scenarios such as illumination, pose, partial occlusion, etc.

[0260] S433: Precise filtering combined with other attributes of the target (such as preliminary category, precise spatio-temporal stamp):

[0261] For V Object that passed the feature similarity matching, further verification is performed using other precise constraints in the query object. For example, checking whether the preliminary category label of V Object is consistent with the query requirement, whether its precise spatio-temporal stamp Tvo falls strictly within the query's time range, etc.

[0262] After this step, an "initial matching V Object set" is obtained, which are V Object that are directly related to the query request in both feature and attribute.

[0263] S44: Association expansion and reasoning retrieval based on STSGHI hyper-edges;

[0264] This step is the most innovative part of this module, which goes beyond simple direct matching and discovers deeper and more dimensional association relationships between targets by utilizing various hyper-edges defined in STSGHI.

[0265] S441: Activation of various hyper-edges associated with the initial matching V Object For each V Object in the "initial matching V Object set" produced by S43, the system searches for all hyper-edge instances in STSGHI that are directly connected to this V Object . These hyper-edges include: spatio-temporal co-occurrence hyper-edge (HESTC), semantic similarity hyper-edge (HESemSim), track / event association hyper-edge (HETrack / Event), and region attribute association hyper-edge (HERegionAttr) that may be indirectly associated through the VCell / VARegion they are in.

[0266] S442: Spatio-Temporal Co-occurrence Expansion (Discovering co-located targets with HESTC):

[0267] If the user's query intent involves finding other targets that co-occur with a certain target at the same time and place (e.g., "Find all people who entered the building with this target person"), the system will focus on analyzing the HESTCs connected to the initial matching V Object .

[0268] By traversing other V Object vertices connected by these HESTCs, we can add these targets with spatio-temporal co-occurrence relationships to the result set. The WeightHE attribute of the hyperedge can be used to evaluate the strength or confidence of the co-occurrence relationship.

[0269] S443: Semantic Similarity Expansion (Discovering similar targets across time and space with HESemSim):

[0270] If the user wishes to find other target instances that are highly similar to the query target in appearance, behavior patterns, etc., but may occur at different times, different places (e.g., "Find all vehicles in the city that are similar to this target vehicle in features"), the system will utilize HESemSim.

[0271] HESemSim directly connects D 2 CFAN feature highly similar V Object . Through these hyperedges, we can quickly recall those targets that are homologous or highly related in visual or deep semantic aspects to the query target, even if they are spatio-temporally separated.

[0272] S444: Trajectory / Event Association Expansion and Reasoning (Utilize HETrack to track target trajectories; utilize HEEvent to mine event participants):

[0273] Trajectory tracking: If the initial matching V Object belongs to a certain HETrack (representing the motion trajectory of the same target), the system can trace forward or backward along the HETrack to find all V Object snapshots of the target at other time points, thus presenting its activity trajectory in its entirety.

[0274] Event participant mining: If the initial matching V Object participates in a certain HEEvent (representing a complex security event), the system can find other V Object (participants, vehicles, objects) and related key area vertices that co-participate in this event through the HEEvent. The SemSumHE attribute of the hyperedge can provide a semantic description of the event.

[0275] S445: Region attribute association extension (use HERegionAttr to search with region common attributes):

[0276] If the query involves macro attributes of regions (e.g., "find all loitering pedestrians in VARegion that are marked as 'high theft area' at night"), the system can first find all VARegion vertices that have the attribute of "high theft area" by HERegionAttr, and then limit the search range to these regions, and then combine other conditions to match the target. This realizes the linkage query from the macro characteristics of the region to the specific micro target.

[0277] S45: De-duplication, fusion and confidence evaluation of direct matching and extended results

[0278] The direct matching V Object from S43 is combined with the V Object from each type of hyper-edge extension from S44, and the repeated V Object are removed.

[0279] For each final candidate V Object , a comprehensive confidence is calculated, which is the original detection confidence, the matching degree with the query, and the association strength (WeightHE) of the associated hyper-edge, and the final confidence score formula is designed as follows:

[0280] Score_final(V)=Conf_detect(V)*MaxPathScore(V,Q)

[0281] Conf_detect(V) represents the confidence score of V Object when it is initially detected in the data access and preprocessing module 1; MaxPathScore(V,Q) (V Object The maximum path association score of V relative to the query Q) indicates the strength of the highest scoring path from the user query Q to the target V (note: before de-duplication, the resulting V Object set will have duplicate target objects, which are obtained from the objects initially related to the query starting point along different paths to the query endpoint):

[0282] MaxPathScore(V,Q)=maxPathj∈AllPaths(Q→V)[ScorePathj(Q→V)]

[0283] AllPaths(Q→V): the set of all associated paths from query Q to V; ScorePathj(Q→V) (single path Pathj score):

[0284] ScorePathj(Q→V) = CalcSimilarity(FVecS,...,FVecV) * WeightHE

[0285] where CalcSimilarity(FVecS,...,FVecV) represents the similarity score of the current path (S is the first V found that directly matches Q Object ) from S all the way to the end of the query V, and WeightHE represents the associated strength value of the hyperedge on which the current path depends. 2 CFAN feature vector similarity score; WeightHE represents the associated strength value of the hyperedge on which the current path depends.

[0286] Please refer to Figure 3 Based on the above embodiments, the application further provides a target retrieval, comprising: a data access and preprocessing module 1, a feature extraction module 2, an index construction module 3, and a target retrieval module 4.

[0287] The data access and preprocessing module 1 is used to obtain multi-source data, perform cleaning, formatting, and spatiotemporal standardization, and perform target detection and cropping on images / videos to generate structured target objects.

[0288] The feature extraction module 2 is used to extract deep semantic feature vectors and auxiliary understanding information from target images, which have discriminative and adaptive complex scene characteristics.

[0289] The index construction module 3 is used to construct a spatiotemporal semantic hypergraph index of multiple types of nodes and hyperedges based on the deep features and spatiotemporal information of the target, to express the complex relationships between targets.

[0290] The target retrieval module 4 is used to receive user multi-modal query information, perform candidate region screening, feature matching, and relationship reasoning using the hypergraph index, and finally output a high-confidence target retrieval result.

[0291] The application embodiments further provide a storage medium, which stores a computer program. When the computer program is run by a processor, the computer program performs the target retrieval method as described above.

[0292] The application embodiments further provide an electronic device, which comprises a processor, a storage medium, and a bus. The storage medium stores machine-readable instructions executable by the processor. When the electronic device is running, the processor communicates with the storage medium through the bus. The processor executes the machine-readable instructions to perform the target retrieval method as described above.

Claims

1. A target search method characterized by comprising: The method comprises the following steps: Step S1: data access and preprocessing, including obtaining multi-source data, cleaning, formatting, spatio-temporal standardization, and target detection and cropping of images / videos to generate structured target objects; Step S2: feature extraction, including extracting deep semantic feature vectors with discriminative and adaptive complex scene from target images and auxiliary understanding information; Step S3: constructing a data index, including constructing a spatio-temporal semantic hypergraph index of multiple types of nodes and hyperedges based on the deep features and spatio-temporal information of the target, to express the complex relationship between targets; Step S4: data query, including receiving user multi-modal query information, using the hypergraph index to perform candidate region screening, feature matching and relationship reasoning, and finally outputting a high-confidence target retrieval result.

2. The target search method of claim 1, wherein the step of S2: the feature extraction step specifically comprises: S21: Based on D 2 The CFAN model constructs a feature extraction model; S22: optimizing training based on the feature extraction model and setting a loss function; S23: pre-processing and normalization of the trained feature extraction model in the inference stage; S24: extracting features and forwarding and internally dynamically regulating; S25: obtaining an output feature vector and performing quality evaluation and directional transmission.

3. The target search method according to Claim 2, wherein Step S21: based on D 2 The CFAN model constructs a feature extraction model, which includes: S211: basic feature extraction and multi-level semantic representation construction are used to extract different depths of the path and obtain feature maps that can represent different levels of abstract semantic information; S212: a dynamic decoupling attention sub-module is constructed, and attention is applied to each semantic subgroup to dynamically decouple the feature channels of each level of feature map into multiple semantic subgroups; S213: an instance-level and class-level collaborative attention sub-module is constructed to allow the model to focus on the instance uniqueness and class commonality of the target before extracting the final features; S214: a context-aware fusion attention sub-module is constructed to fuse features from different processing stages; S215: outputting a feature vector, which is used to pass the features obtained by the context-aware fusion attention sub-module through one or more fully connected layers to obtain the final feature vector.

4. The target search method of claim 1, wherein, Step S22: optimizing training based on the feature extraction model and setting a loss function specifically comprises: S221: knowledge transfer initialization and field adaptability fine tuning; S222: optimizing the main identity recognition loss and the DHDA level weighted semantic decoupling orthogonality regularization, and the calculation formula is: wherein: s ij · s il represents the dot product of vectors s ij and s il ||s ij ||2represents the L2 norm of vector s ij , and ∈0is a constant, ω i is a decoupling importance weight coefficient set for the i-th semantic hierarchy. S223: D 2 CFAN model parameter iterative optimization process.

5. The target retrieval method of claim 1, wherein, Step S3, constructing a data index comprises: S31: constructing the vertices and hyperedges of the spatio-temporal semantic hypergraph index; S32: initial construction of the spatio-temporal semantic hypergraph index, continuous entry of data objects and dynamic evolution of the index structure, generation and association of relationship hyperedges.

6. The target retrieval method of claim 5, wherein, S31: constructing the vertices and hyperedges of the spatio-temporal semantic hypergraph index comprises: S311: constructing the vertices of the spatio-temporal semantic hypergraph index includes macroscopic spatio-temporal block vertices, basic spatio-temporal unit vertices, and target instance vertices; S312: constructing the hyperedges of the spatio-temporal semantic hypergraph index includes spatio-temporal co-occurrence hyperedges, semantic similarity hyperedges, trajectory / event association hyperedges, and region attribute association hyperedges; S313: determining the symbols and attributes of the vertices and hyperedges.

7. The target search method according to Claim 5, wherein Step S32 comprises: S321: initial construction of the index; S322: insertion of target instance vertex and dynamic evolution of underlying structure; S323: dynamic generation of hyperedge and association.

8. The target search method of claim 1, wherein, Step S4, data query includes: S41: query intention analysis and multi-modal input processing; S42: STSGHI hierarchical screening and candidate region positioning; S43: Fine-grained target matching with initial candidate V Object set generation; S44: association expansion and reasoning retrieval based on STSGHI hyperedge; S45: deduplication, fusion and confidence evaluation of direct matching and expansion results.

9. The target retrieval method of claim 1, wherein the step of S1: data access and preprocessing steps include: S11: data source interface configuration and connection establishment; S12: raw data acquisition and preliminary analysis; S13: data cleaning and quality evaluation; S14: spatiotemporal information extraction and standardization; S15: preliminary target detection; S16: data packaging and output.

10. A target search, characterized by, Comprise: data access and preprocessing module, feature extraction module, index construction module, target retrieval module, wherein: the data access and preprocessing module is used to obtain multi-source data, clean, format, spatiotemporal standardization, and perform target detection and cropping on images / videos to generate structured target objects; the feature extraction module is used to extract deep semantic feature vectors and auxiliary understanding information from target images, which have discriminative and adaptive complex scene; the index construction module is used to construct a multi-type node and hyperedge spatiotemporal semantic hypergraph index based on the deep features and spatiotemporal information of the target, to express the complex relationship between targets; the target retrieval module is used to receive user multi-modal query information, use the hypergraph index to perform candidate region screening, feature matching and relationship reasoning, and finally output high-confidence target retrieval results.

Images