Cross-view target association method, electronic device and storage medium

Abstract

The application discloses a cross-view target association method, an electronic device and a storage medium. The method comprises the following steps: in response to a source camera detecting a source target to be associated, acquiring association information of targets detected by the rest of the cameras; performing time-space constraint filtering on the association information according to a preset camera network topology to determine effective candidate targets and corresponding time-space consistency scores; determining multi-dimensional matching scores of each effective candidate target, wherein the multi-dimensional matching scores at least include visual similarity scores, motion state matching scores and home scene context matching scores; performing fusion processing on the time-space consistency scores, the visual similarity scores, the motion state matching scores and the home scene context matching scores to obtain comprehensive matching scores of each effective candidate target; and associating the source target with the effective candidate target that successfully matches according to the comprehensive matching scores. The cross-view target association can be realized in a stable, interpretable and high-accuracy manner in a resource-limited home monitoring scene.

Description

Technical Field

[0001] This invention relates to the field of computer technology, and in particular to a cross-view target association method, electronic device, and storage medium. Background Technology

[0002] Existing multi-camera target tracking methods in home surveillance systems typically assign a new local identifier to a target when it leaves the field of view of one camera and enters the field of view of another non-overlapping camera. This results in the same physical entity being identified as different in different cameras, making it impossible to form a continuous trajectory across cameras. To address this issue, existing solutions employ end-to-end deep learning models or global optimization methods for cross-camera target re-identification. However, these methods do not utilize prior information unique to the home environment (such as the activity patterns of family members, differences in movement speed among different roles, and strong correlations between objects and rooms). Furthermore, they often employ black-box probabilistic fusion or relative ranking strategies, lacking an explicit and interpretable scoring system, making it difficult to achieve lightweight, real-time, and interpretable target association on home edge devices. Summary of the Invention

[0003] This invention provides a cross-view target association method, electronic device, and storage medium to solve the technical problem that the identity identification of the same target is interrupted and a continuous cross-camera trajectory cannot be formed when switching between different cameras in a home monitoring system where the fields of view of multiple cameras do not overlap.

[0004] According to one aspect of the present invention, a cross-view target association method is provided, applied to a home monitoring system containing multiple cameras, the method comprising: In response to the source camera detecting the source target to be associated, obtain the association information of the targets detected by the other cameras; Based on the preset camera network topology, the associated information is filtered for spatiotemporal constraints to determine valid candidate targets and their corresponding spatiotemporal consistency scores. Determine the multidimensional matching score for each valid candidate target. The multidimensional matching score includes at least visual similarity score, motion state matching score, and home scene context matching score. The spatiotemporal consistency score, visual similarity score, motion state matching score, and family scene context matching score are fused to obtain the comprehensive matching score of each effective candidate target; Based on the comprehensive matching score, the source target is associated with the successfully matched valid candidate targets.

[0005] According to another aspect of the present invention, a cross-view target association device is provided, applied to a home monitoring system including multiple cameras, the device comprising: The information acquisition module is used to acquire the association information of the targets detected by the other cameras in response to the source camera detecting the source target to be associated. The spatiotemporal filtering module is used to perform spatiotemporal constraint filtering on associated information based on a preset camera network topology, and to determine valid candidate targets and their corresponding spatiotemporal consistency scores. The multidimensional scoring module is used to determine the multidimensional matching score of each valid candidate target. The multidimensional matching score includes at least visual similarity score, motion state matching score and home scene context matching score. The scoring fusion module is used to fuse spatiotemporal consistency scores, visual similarity scores, motion state matching scores, and home scene context matching scores to obtain a comprehensive matching score for each valid candidate target. The target association module is used to establish associations between the source target and the successfully matched valid candidate targets based on the comprehensive matching score.

[0006] According to another aspect of the present invention, an electronic device is provided, the electronic device comprising: At least one processor; and A memory communicatively connected to the at least one processor; wherein, The memory stores a computer program that can be executed by the at least one processor, the computer program being executed by the at least one processor to enable the at least one processor to perform the cross-view target association method according to any embodiment of the present invention.

[0007] According to another aspect of the present invention, a computer-readable storage medium is provided, the computer-readable storage medium storing computer instructions for causing a processor to execute and implement the cross-view target association method according to any embodiment of the present invention.

[0008] According to another aspect of the present invention, a computer program product is provided, the computer program product comprising a computer program that, when executed by a processor, implements the cross-view target association method according to any embodiment of the present invention.

[0009] The technical solution of this invention, in response to a source camera detecting a target to be associated, acquires the association information of targets detected by other cameras; performs spatiotemporal constraint filtering on the association information according to a preset camera network topology to determine valid candidate targets and their corresponding spatiotemporal consistency scores; determines a multidimensional matching score for each valid candidate target, which includes at least a visual similarity score, a motion state matching score, and a home scene context matching score; fuses the spatiotemporal consistency score, visual similarity score, motion state matching score, and home scene context matching score to obtain a comprehensive matching score for each valid candidate target; and establishes an association between the source target and the successfully matched valid candidate targets based on the comprehensive matching score. This technical solution effectively filters invalid candidate targets through spatiotemporal constraint filtering to reduce computational overhead, and combines visual similarity, motion state matching, and home scene context to construct a comprehensive scoring framework for association determination, solving the problem of target identity interruption in scenarios where multiple camera views do not overlap. It achieves stable, interpretable, and highly accurate cross-view target association in home monitoring scenarios with limited computational resources.

[0010] It should be understood that the description in this section is not intended to identify key or essential features of the embodiments of the present invention, nor is it intended to limit the scope of the invention. Other features of the invention will become readily apparent from the following description. Attached Figure Description

[0011] To more clearly illustrate the technical solutions in the embodiments of the present invention, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0012] Figure 1 A flowchart illustrating a cross-view target association method provided in an embodiment of the present invention; Figure 2 A flowchart of another cross-view target association method provided in an embodiment of the present invention; Figure 3 This is a schematic diagram of the structure of a cross-view target association device provided in an embodiment of the present invention; Figure 4 This is a schematic diagram of the structure of an electronic device that implements the cross-view target association method of the present invention. Detailed Implementation

[0013] To enable those skilled in the art to better understand the present invention, the technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings of the embodiments of the present invention. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort should fall within the scope of protection of the present invention.

[0014] It should be noted that the terms "first," "second," etc., in the specification, claims, and accompanying drawings of this invention are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments of the invention described herein can be implemented in orders other than those illustrated or described herein. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover non-exclusive inclusion; for example, a process, method, system, product, or apparatus that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus.

[0015] Figure 1 This is a flowchart illustrating a cross-view target association method provided in an embodiment of the present invention. This embodiment is applicable to scenarios where multiple cameras in a home surveillance system have non-overlapping fields of view, and involves identity association and continuous trajectory tracking of moving targets across cameras. This method can be executed by a cross-view target association device, which can be implemented in hardware and / or software. This cross-view target association device can be configured in an electronic device, which may include, but is not limited to, home edge devices, edge servers, cloud computing platforms, etc. Figure 1 As shown in the figure, this embodiment provides a cross-view target association method, which is applied to a home monitoring system containing multiple cameras. The method specifically includes the following steps: S110. In response to the source camera detecting a source target to be associated, obtain the association information of the targets detected by the other cameras.

[0016] In this context, the source camera refers to a specific camera device deployed in a home monitoring system that detects a moving target to be tracked at the current moment and triggers a cross-view association process. The source target refers to a moving object detected within the field of view of the source camera that needs to be identified and matched with targets from other cameras, such as a family member. The association information refers to a multi-dimensional set of attribute data associated with the target detected by other cameras, which may include, but is not limited to, the detection time, visual feature vector (extracted from the original image features), motion trajectory information, motion direction (velocity vector), the type of room where the target is located, and the types of objects around the target, to support subsequent cross-view fusion matching and determination.

[0017] In this embodiment of the invention, during the continuous operation of the home monitoring system, when a camera (i.e., the source camera) detects and identifies a moving object in the video frame and determines it as the source target to be associated, the system immediately triggers a cross-view matching process. Specifically, the system uses the detection time of the source target as a benchmark to obtain all active target records detected by all other cameras (excluding the source camera itself) within a preset time window around that time. The system summarizes and encapsulates the multi-dimensional attribute data of each candidate target, such as the detection time, visual feature vector, historical trajectory point sequence, velocity vector, room type, and surrounding item category, according to a unified data structure to form structured association information, providing a complete data foundation for subsequent cross-view matching based on spatiotemporal constraint filtering and multi-dimensional fusion scoring.

[0018] S120. Based on the preset camera network topology, perform spatiotemporal constraint screening on the associated information to determine the effective candidate targets and their corresponding spatiotemporal consistency scores.

[0019] The preset camera network topology refers to pre-stored configuration data describing the spatial connectivity and cross-camera travel time range between cameras in a home, used to guide the logical inference of target cross-regional transfer. Spatiotemporal constraint filtering refers to the process of eliminating invalid candidates from the globally detected targets that are physically unreachable or temporally impossible to transfer across cameras, based on two dimensions: the legality of the target's transfer path in physical space (i.e., spatial topology constraints) and the reasonableness of the time interval between its appearances among cameras (i.e., time window constraints). Valid candidate targets are those that, after spatiotemporal constraint filtering, meet the basic prerequisites for cross-camera transfer in both physical topology and temporal order, and have the potential to match the source target. The spatiotemporal consistency score is a score used to quantify the reasonableness of the matching between valid candidate targets and the source target in terms of spatial connectivity and temporal transfer, with a value range of [0,1]. A higher value indicates a stronger spatiotemporal matching.

[0020] In this embodiment of the invention, the system can evaluate associated information from two dimensions—spatial connectivity legitimacy and temporal transition rationality—based on a preset camera network topology. For example, according to the physical connectivity defined by the preset camera network topology, only targets detected by cameras that have direct or indirect reachable paths to the source camera in the spatial topology can be retained, while targets in spatially inaccessible cameras are excluded from the candidate range, thereby filtering out valid candidate targets. Next, the system can quantify the temporal rationality of each valid candidate target based on the temporal transition distribution pattern defined by the preset camera network topology, obtaining a spatiotemporal consistency score that characterizes the degree of reasonable matching between each valid candidate target and the source target in the spatiotemporal dimension.

[0021] S130. Determine the multidimensional matching score for each valid candidate target. The multidimensional matching score shall include at least the visual similarity score, the motion state matching score, and the home scene context matching score.

[0022] The visual similarity score is a quantitative value calculated by comparing the external appearance features (such as color, texture, shape, and other deep semantic information) of the source and candidate targets, reflecting their degree of consistency in appearance. The motion state matching score is a quantitative value calculated by comparing the continuity of motion direction and the spatiotemporal continuity of historical trajectories of the source and candidate targets, reflecting whether their motion inertia remains consistent before and after crossing the camera. The home scene context matching score (technical feature) is a quantitative value calculated using prior knowledge specific to the home monitoring system's environment (such as statistical associations between objects and rooms, family members' daily routines, movement patterns of different roles, and co-occurrence social relationships among members), used to assist in determining the target's identity from an environmental semantic perspective.

[0023] In this embodiment of the invention, for each selected valid candidate target, its matching score with the source target can be determined from multiple dimensions. For example, the multidimensional matching score may include at least: ① Visual similarity scoring can extract the visual feature vectors of the source target and the effective candidate target respectively, calculate the similarity between the two and map them to the preset scoring interval [0,1] to obtain the visual similarity score, which is used to quantify the degree of consistency between the two in terms of appearance.

[0024] ② Motion state matching score: The motion direction and historical trajectory data of the source target and the effective candidate target can be extracted separately. Then, the sub-scores of the two targets in terms of motion direction continuity and spatiotemporal continuity of historical trajectory are determined. The two sub-scores are then fused to obtain the motion state matching score, which is used to quantify whether the motion inertia of the two targets remains consistent before and after crossing the camera.

[0025] ③ The family scene context matching score can extract family-specific scene information such as the room where the effective candidate target is located, the time of appearance, the target's movement speed, the people coexisting in the scene, and the surrounding household items, to construct a family scene-specific context matching basis. Then, the extracted scene information is evaluated for conformity with the inherent multi-dimensional prior rules of the family. For example, it may include, but is not limited to: matching the time of appearance with the time period-activity pattern of the area, matching the role type reflected by the target's movement speed with the role-space adaptability of the area, matching the relationship between the people coexisting in the scene with the family's social co-occurrence norms, and matching the category of surrounding household items with the functional affiliation of the area with item-space associations. Finally, the prior conformity evaluation results corresponding to various scene information are combined in a weighted fusion manner to obtain the family scene context matching score, which is used to quantify the overall degree of conformity between the effective candidate target and the source target's identity background in terms of environmental semantics and family life logic. It is important to understand that the family scene context matching score is a unique dimension in this embodiment that distinguishes it from general cross-camera association schemes. Its introduction enables the system to utilize family-specific prior knowledge such as family daily routines, differences in members' movement patterns, and social co-occurrence relationships, which can effectively compensate for the insufficient discriminative power of pure visual and motion features in complex scenes.

[0026] S140. The spatiotemporal consistency score, visual similarity score, motion state matching score, and home scene context matching score are fused to obtain the comprehensive matching score of each effective candidate target.

[0027] Among them, the comprehensive matching score can refer to the quantitative value obtained after fusion processing, which is used to comprehensively measure the overall credibility of the source target and the effective candidate target belonging to the same physical entity. The value can be located in the range [0,1], and the higher the value, the higher the matching credibility.

[0028] In this embodiment of the invention, the system can summarize the multi-dimensional matching scores corresponding to each valid candidate target, such as at least spatiotemporal consistency score, visual similarity score, motion state matching score, and home scene context matching score; then, the matching scores of the above dimensions can be fused, such as weighted fusion, Bayesian fusion, etc., to obtain the comprehensive matching score corresponding to each valid candidate target, which is used to quantify the matching credibility of the source target and each valid candidate target belonging to the same physical entity.

[0029] S150. Based on the comprehensive matching score, establish a connection between the source target and the successfully matched valid candidate targets.

[0030] In this embodiment of the invention, the system can summarize the comprehensive matching scores corresponding to each valid candidate target and compare the highest score with a pre-configured matching threshold. If the highest score is greater than or equal to the matching threshold, the valid candidate target corresponding to the highest score is determined as a successfully matched target and an association is established between it and the source target. If the highest score is less than the matching threshold, the system determines that there are currently no valid candidate targets that match the source target and ends the current association process.

[0031] The technical solution of this invention pre-filters a large number of invalid candidate targets through spatiotemporal constraint screening, significantly reducing the computational overhead of subsequent multi-dimensional feature matching, enabling the overall solution to run in real time on home edge devices. By decoupling cross-view target association into multiple independent computable dimensions including visual similarity, motion state matching, and home scene context, the outputs of each dimension are uniformly classified into the same comparable interval and fused, providing traceable interpretable basis for association decisions. By introducing home scene context into the matching process, candidate targets that are similar in appearance but have unreasonable scenes can be effectively suppressed, significantly improving the accuracy of target association. Stable, interpretable, and highly accurate cross-view target association is achieved in home monitoring scenarios with limited computing resources.

[0032] Furthermore, based on the above embodiments, the preset camera network topology includes the spatial adjacency relationships between cameras and the target transfer time window between adjacent cameras. The associated information is then subjected to spatiotemporal constraint filtering based on the preset camera network topology to determine valid candidate targets and their corresponding spatiotemporal consistency scores, specifically including the following steps: Based on spatial adjacency, candidate targets corresponding to cameras adjacent to the source camera are determined from the associated information; Determine the detection time difference between each candidate target and the source target, and identify candidate targets whose detection time difference is within the target transfer time window as valid candidate targets; Based on the relative position of the detection time difference within the target transfer time window, the spatiotemporal consistency score corresponding to each valid candidate target is determined.

[0033] Spatial adjacency can refer to the pre-defined physical pathways such as doors and corridors in a home, indicating that a target can move directly from one camera's area to another without crossing the field of view of other cameras. The target transfer time window can be the time interval between the minimum and maximum transfer times required for a target to disappear from the source camera and reappear in the adjacent camera, calculated in advance based on the actual physical distance between the two cameras and the normal human movement speed, with additional tolerance margin. Detection time difference can be the time difference between the timestamp when a candidate target is captured by its own camera and the timestamp when the source target is captured by the source camera.

[0034] In this embodiment of the invention, the process of determining valid candidate targets and their spatiotemporal consistency scores by performing spatiotemporal constraint filtering on associated information based on a preset camera network topology specifically includes: S1. The system can traverse the associated information based on the spatial adjacency relationship between cameras recorded in the preset camera network topology, filter out neighboring cameras that have a direct adjacency relationship with the source camera, and determine the target captured by the neighboring camera as the candidate target.

[0035] S2. For each candidate target obtained in the previous round of screening, the system calculates the difference between the detection time of the current candidate target and the detection time of the source target, obtaining the detection time difference. Next, the system queries the target transfer time window recorded in the preset camera network topology between adjacent cameras and determines whether the above detection time difference falls within this target transfer time window. If so, the current candidate target is marked as a valid candidate target; otherwise, it is directly eliminated. It is important to understand that if the detection time difference is less than the minimum transfer time (min_win) of the window, it indicates that the target is moving too fast and is physically impossible to reach; if the time difference is greater than the maximum transfer time (max_win) of the window, it indicates that the target is moving too slowly or has already left, which is also unreasonable. Only candidate targets whose detection time difference falls exactly within the target transfer time window are considered valid candidate targets.

[0036] S3. For all the selected valid candidate targets, the system can perform quantitative calculations by combining the relative positional relationship of the detection time difference within the target transfer time window, and output a spatiotemporal consistency score with continuous values ​​to characterize the credibility of the matching between the valid candidate target and the source target in the spatiotemporal dimension.

[0037] Specifically, based on the relative position of the detection time difference within the target transfer time window, the spatiotemporal consistency score corresponding to each valid candidate target is determined, which includes the following steps: Determine the window midpoint and window half-width of the target transfer time window; Determine the distance between the detection time difference and the point in the window, and use the ratio of the distance to the half-width of the window as the median value; Determine the difference between the median value and the preset maximum score, and determine the maximum value between the difference and the preset minimum score as the spatiotemporal consistency score.

[0038] In this embodiment of the invention, the process of determining the spatiotemporal consistency score specifically includes: the system first calculates the midpoint of the target transfer time window [min_win, max_win], i.e., mid = (min_win + max_win) / 2, and the half-width of the window, i.e., half = (max_win - min_win) / 2, where the midpoint represents the ideal time consumption of the transfer path in the time dimension as "most typical and most reliable". Then, the system calculates the absolute distance of the detection time difference time_diff from the midpoint mid, and divides this deviation distance by the half-width half to obtain a normalized deviation ratio. Finally, the system subtracts this deviation ratio from the preset maximum score value (i.e., 1), and takes the larger of the result and the preset minimum score value (i.e., 0) as the final spatiotemporal consistency score P1, i.e., P1 = max(0, 1 - |time_diff - mid| / half). The core idea of ​​this scoring logic is: the closer the detection time difference is to the midpoint of the window, the closer the score is to the full score of 1; the closer it is to the window boundary, the more smoothly the score decays to 0; once it exceeds the window boundary, the score is directly locked to 0.

[0039] This embodiment, through prior spatial adjacency constraints and strict time window thresholds, can quickly eliminate a large number of spatially inaccessible or temporally unreasonable interfering candidates, thereby effectively reducing the subsequent computational load and strongly ensuring low-latency response of home edge devices in real-time scenarios. Simultaneously, by upgrading the filtering method for boundary candidates from "hard truncation" to "continuous scoring," it avoids false deletions caused by differences in individual target walking speeds or minor system clock errors. This allows valid targets located at the window edge to still enter the subsequent weighted fusion stage with lower scores, which are then compensated for by other discrimination dimensions such as appearance or home scene. Thus, without sacrificing computational efficiency, it significantly improves the recall rate and overall robustness of cross-camera associations.

[0040] In one embodiment, for spatiotemporal consistency scoring, in addition to the midpoint-priority continuous scoring described above, this embodiment of the invention also provides two alternative methods, which can be selected according to the actual application scenario: ① Gaussian decay scoring, which uses the mean of the Gaussian distribution at the midpoint of the time window and the standard deviation of the half-width as the standard deviation, so that the time difference of the candidate targets decays according to the Gaussian function, which is suitable for scenarios that require "high credibility near the midpoint and rapid decline far from the midpoint"; ② Hard threshold binary scoring, which directly scores the candidate targets within the time window as 1 point and those outside the window as 0 points, which is suitable for fast hard decision scenarios that only care whether the candidate targets fall within the time window and do not need to distinguish the boundary credibility.

[0041] Furthermore, based on the above embodiments of the invention, the process of determining the visual similarity score includes: Extract the first visual feature vector of the source target and the second visual feature vector of the effective candidate targets respectively; Determine the similarity between the second visual feature vector and the first visual feature vector; The similarity scores are normalized to obtain a visual similarity score.

[0042] In this embodiment of the invention, the process of determining the visual similarity score between a valid candidate target and a source target specifically includes: S1. For the source target that triggers the matching process, a target image patch within the source camera's view can be cropped. Then, a pre-trained feature extraction network is used to extract the first visual feature vector corresponding to the image patch. Similarly, the second visual feature vector corresponding to the valid candidate target is also extracted. The visual feature vector extraction process may include: cropping the target image region from the target detection box detected by the camera, scaling the image region to a preset standard size (e.g., 224×224 pixels), and then inputting it into a pre-trained visual feature extraction network (e.g., a lightweight ReID network) for forward computation, outputting a fixed-dimensional feature vector. Then, L2 norm normalization is performed on the feature vector to obtain a visual feature vector of unit length, eliminating the influence of scale differences on subsequent comparisons.

[0043] S2. Calculate the similarity between the first visual feature vector and the second visual feature vector of each valid candidate target, where the similarity can be cosine similarity. Then, map the similarity to a preset scoring interval [0,1] to obtain the visual similarity score corresponding to each valid candidate target, which is used to quantify the degree of consistency between each valid candidate target and the source target at the appearance level.

[0044] This embodiment extracts visual feature vectors from the source target and valid candidate targets respectively, transforms visual appearance information into a computable numerical representation, and further determines the similarity between the two feature vectors, providing an objective quantitative basis for cross-camera appearance matching. On this basis, the similarity is normalized to unify the score to the [0,1] interval, making the score of the visual dimension comparable to other scoring dimensions on a numerical scale, laying the foundation for subsequent multi-dimensional explicit weighted fusion. At the same time, the normalized score can be output independently, making it easy for users to trace the matching basis and achieve interpretable target association.

[0045] In one embodiment, the present invention uses feature cosine similarity as the default calculation method for visual similarity scoring. Based on this, two alternative solutions are provided to adapt to deployment scenarios with different computing power and accuracy requirements: Alternative method one is CLIP Embedding cosine similarity, which uses the multimodal pre-trained model CLIP to replace the dedicated ReID network for extracting image features. This model, trained through image-text alignment, possesses stronger semantic understanding and cross-scene generalization capabilities, exhibiting higher robustness under complex conditions such as drastic changes in lighting, significant shifts in viewpoint, or partial occlusion of the target. However, the trade-off is a larger model parameter count and longer inference time, making it suitable for scenarios with sufficient computing power and extremely high accuracy requirements. Alternative method two is color histogram similarity, which calculates the color distribution histograms of two target image patches across the RGB channels and uses the Bhattacharyya distance to measure the similarity of their distributions. This method does not require neural network inference, only involves pixel statistical operations, resulting in extremely low computational overhead, making it suitable for edge devices with extremely limited computing power. However, it only utilizes color information and ignores texture and shape features, resulting in insufficient differentiation between different targets with similar clothing colors. The above alternative methods do not change the overall framework of this invention, which uses visual appearance as an independent scoring dimension to participate in weighted fusion. When implementing them, they can be flexibly selected based on the computing resources and accuracy requirements of the actual deployment environment.

[0046] Furthermore, based on the above embodiments of the invention, the process of determining the motion state matching score includes: The first direction of motion and the first historical trajectory of the source target, and the second direction of motion and the second historical trajectory of the effective candidate targets are obtained respectively. Determine the degree of consistency between the first and second motion directions to obtain a direction consistency sub-score; Determine the degree of continuity in time and space between the end of the first historical trajectory and the beginning of the second historical trajectory to obtain a trajectory continuity sub-score; The motion state matching score is obtained by weighted fusion of the orientation consistency sub-score and the trajectory continuity sub-score.

[0047] The first motion direction can refer to the velocity vector of the source target in the last frame or several frames before leaving the field of view of the source camera, representing its motion orientation upon departure. The first historical trajectory can refer to the set of a series of continuous position points and their corresponding timestamps recorded during the tracking of the source target in the source camera, representing its complete motion path before departure. The second motion direction can refer to the velocity vector of the effective candidate target in the first frame or several frames after entering the field of view of the adjacent camera, representing its motion orientation upon entry. The second historical trajectory can refer to the set of a series of continuous position points and their corresponding timestamps recorded during the tracking of the effective candidate target in the adjacent camera, representing its complete motion path after entry. The direction consistency sub-score can refer to a score used to quantify the consistency between the first and second motion directions, and can be used to assess the convergence of their motion orientations. The trajectory continuity sub-score can refer to a score calculated based on the time interval between the end point of the first historical trajectory and the starting point of the second historical trajectory, as well as their spatial distance, used to quantify the smoothness of the spatiotemporal connection between the two trajectories.

[0048] In this embodiment of the invention, the process of determining the motion state matching score between a valid candidate target and a source target specifically includes: S1. The system can extract the corresponding first motion direction and first historical trajectory from the tracking data of the source target, and similarly obtain the second motion direction and second historical trajectory of the valid candidate targets. The extraction of motion direction and historical trajectory does not rely on additional hardware devices, but is calculated entirely based on the position coordinates and timestamp information in the existing detection results of each camera. The computational complexity is low, making it suitable for real-time operation on edge devices.

[0049] S2. Perform a matching operation on the first and second motion direction vectors to quantify the degree of overlap between the two target directions. Output a direction consistency sub-score D_con with a value range of [0,1]. The score approaches 1 when the two motion directions are completely consistent and approaches 0 when the motion directions are completely opposite. In one embodiment, the cosine similarity between the first and second motion directions can be calculated, and the maximum value between the cosine similarity and 0 is determined as the direction consistency sub-score corresponding to the valid candidate target. That is: if the cosine similarity is positive (indicating that the angle between the two directions is less than 90 degrees, i.e., the motion directions are roughly the same), then the positive number is directly retained as the direction consistency sub-score; if the dot product is negative or zero (indicating that the angle between the two directions is greater than or equal to 90 degrees, i.e., the motion directions are perpendicular, opposite, or completely opposite), then the direction consistency sub-score is forcibly set to 0.

[0050] S3. The system extracts the latest endpoint from the first historical trajectory and the earliest start point from the second historical trajectory. Then, in the time dimension, it calculates the difference between the start and end times. If this time difference is less than 0 seconds (i.e., the candidate target appears earlier than the source target disappears) or greater than a preset time threshold (i.e., the time interval is too long), the trajectory continuity sub-score is directly determined to be 0. In the spatial dimension, it calculates the Euclidean distance S_gap between the endpoint and the start point. If this distance S_gap is greater than a preset spatial threshold Th (e.g., 5 meters), the trajectory continuity sub-score is also directly determined to be 0. If neither the time difference nor the spatial distance exceeds the above rejection threshold, a linear decay calculation is performed based on the proportion of spatial distance within the preset spatial threshold to obtain the trajectory continuity sub-score T_con, i.e., T_con = 1-min(S_gap / Th, 1), meaning the closer the spatial distance, the closer the score is to 1, and the farther the spatial distance, the closer the score is to 0.

[0051] S4. The system performs a weighted sum of the directional consistency sub-score D_con and the trajectory continuity sub-score T_con according to preset weights. The final weighted sum is the motion state matching score, which falls within the interval [0,1]. It can be directly fused with other dimension scores using a unified weighted method. The directional consistency sub-score can be assigned a weight of 0.6, and the trajectory continuity sub-score can be assigned a weight of 0.4. The sum of their weights is 1.

[0052] This embodiment independently evaluates motion state through two factors: directional consistency and trajectory continuity. Directional consistency utilizes the similarity between motion directions to quickly eliminate targets moving in opposite directions, effectively breaking the identity ambiguity caused by similar appearances but different motion directions. Trajectory continuity checks the time interval and spatial distance between the end of the source target trajectory and the beginning of the candidate target trajectory, eliminating unreasonable associations such as time inversion, excessively long intervals, or spatial jumps from a physical feasibility perspective. The complementary cooperation of the two factors makes the motion state matching score not only have the advantage of lightweight calculation, but also provide robust auxiliary judgment criteria in scenarios where cross-camera fields of view do not overlap and target appearances differ greatly, thereby significantly improving the overall accuracy of cross-target association.

[0053] Furthermore, based on the above embodiments of the invention, the family scene context matching score includes a neighborhood item context sub-score and a family scene spatiotemporal context sub-score.

[0054] The process of obtaining the context sub-rating of neighboring items includes: Obtain the categories of items detected within the monitored area where valid candidate targets are located, as well as the type of family room to which the monitored area belongs; By querying a pre-defined item-room statistics table, the first degree of matching between item categories and family room types is determined; By querying a pre-set item-camera statistics table, a second degree of matching between item categories and monitored areas is determined; We perform a weighted fusion of the first and second matching scores to obtain the context sub-score of the neighboring items.

[0055] The "neighborhood item context sub-score" refers to a quantitative indicator used to measure the degree of matching between the detected item category and the spatial affiliation of the monitored area where the candidate target is located. It reflects the auxiliary support of item distribution patterns for target identification and can be used to help distinguish between different human targets with highly similar appearances. "Item category" refers to the result of classifying detected objects within the monitored area according to semantic categories, such as specific category labels like cups, backpacks, toys, and washing machines. "Family room type" refers to the category labels after dividing the monitored spatial areas of each camera according to family functions, such as kitchen, living room, bedroom, study, and bathroom. The "item-room statistics table" refers to a pre-built and continuously dynamically updated statistical data table used to record the historical frequency and matching probability of various items appearing in different family rooms. The "first matching degree" refers to the matching result obtained from the item-room statistics table, representing the statistical association strength between a specific item category and a specific family room type; a higher value indicates that the item appears more frequently in that type of room. The "item-camera statistics table" refers to a pre-built and continuously dynamically updated statistical data table used to record the historical frequency and matching probability of various items appearing in the monitored areas corresponding to each camera location. The second matching degree can refer to the matching results obtained from the item-camera statistics table, representing the statistical correlation strength between a specific item category and a specific monitoring area. The higher the value, the more frequently the item appears in the field of view of the camera.

[0056] In this embodiment of the invention, the home scene context matching score can be composed of two parts: a neighborhood item context sub-score and a home scene spatiotemporal context sub-score. The determination process of the neighborhood item context sub-score specifically includes: S1. The system first performs target detection and object detection on the current video frame containing the valid candidate targets, identifying various static or semi-static items (such as cups, books, toys, home appliances, etc.) existing in the monitored area, and outputting the category label for each detected item. At the same time, based on the camera's pre-configuration information, the system reads the type of room in the home to which the camera belongs (such as kitchen, living room, bedroom, etc.) and the camera's unique identifier, which serve as the index key for subsequent statistical queries.

[0057] S2. The system uses the item category and room type obtained in step S1 as the joint query key to access the preset item-room statistics table. This statistics table records the cumulative frequency of each item category observed in each room type throughout history. The value returned by the query is the first matching degree, ranging from 0 to 1. The closer the value is to 1, the stronger the binding relationship between the item and the room type (e.g., the frequency of a cup in the kitchen is 0.9), and the closer the value is to 0, the weaker the relationship or the item has never been observed (e.g., the frequency of a washing machine in the study is close to 0).

[0058] S3. The system uses the item category obtained in step S1 and the camera identifier of the current monitoring area as the joint query key to access the preset item-camera statistics table. This statistics table is similar in structure to the item-room statistics table but with finer granularity. It records the cumulative frequency of a specific item category being observed within the field of view of a specific single camera, rather than summarizing it to the room type level. The value returned by the query is the second matching degree, which also ranges from 0 to 1. This statistics table can capture the positional differences between different cameras within the same room type. For example, in the same kitchen, the distribution of items observed by the camera near the countertop may be different from that observed by the camera near the dining table.

[0059] In one embodiment, if the system is in the cold start phase and there is no valid data in the item-room statistics table and the item-camera statistics table, the first matching degree and the second matching degree are reverted to the default value of 0.5 (medium confidence level), thereby ensuring that the system remains neutral in its contextual evaluation of any candidate target when there is no prior knowledge.

[0060] If the category of the item is known (e.g., cups, washing machines, toys, etc.), the system uses common knowledge in daily life to assign differentiated values ​​to the probability of the item appearing in different types of family rooms. For example, a cup is assigned a high-frequency value of 0.8 in the kitchen, a medium-low frequency value of 0.3 in the living room, and a low-frequency value of 0.1 in the bedroom. These values ​​are derived from the general consensus of people on family life scenarios.

[0061] Once the system enters normal, uninterrupted operation, it triggers an online update process for the statistics table whenever a valid observation event occurs (i.e., the system confirms that a certain item category has actually appeared at a certain location or has not appeared at a certain location). The update uses an exponential moving average algorithm: New statistical value = 0.9 × Old statistical value + 0.1 × Current observation value. Here, the old statistical value is the current stored value of the item; the current observation value is a binary fact, set to 1 if the item actually appeared and 0 if it did not; 0.9 is the historical retention coefficient, controlling the proportion of old information retained; and 0.1 is the observation weighting coefficient, controlling the correction force of the new observation on the statistical value. This update is performed in real time after each observation, without requiring system shutdown or manual intervention.

[0062] S4. The system calculates a weighted sum of the first and second matching degrees according to preset weights. The final weighted sum is the neighborhood item context sub-score, which falls within the interval [0,1] and can be directly weighted and fused with other dimension scores. The first matching degree can be assigned a weight of 0.6, and the second matching degree can be assigned a weight of 0.4, with the sum of the two weights being 1.

[0063] This embodiment creatively introduces the unique priori understanding of "strong correlation between objects and spatial location" in a home setting into a cross-camera target association system. Object-room statistics utilize general patterns at the functional area level (e.g., washing machines are often located in bathrooms or balconies), while object-camera statistics further capture the differences in object distribution across different locations within the same room. The combination of these two approaches allows the system to assess the confidence level of a target's presence in the current area even when targets have similar appearances (such as the same clothing or similar body shape) and are difficult to distinguish visually. For example, detecting a target carrying a washing machine in the bathroom has a higher matching confidence level than detecting a similar target in the study, effectively overcoming visual ambiguity.

[0064] Furthermore, based on the above embodiments of the invention, the process of obtaining the spatiotemporal context sub-score of the home scene includes: Obtain the detection time of valid candidate targets; By querying a preset time period-room statistics table, the time pattern sub-score between the detection time and the family room type is determined; The corresponding candidate target role type is determined based on the movement speed of the valid candidate targets; By querying the preset role-position statistics table, the role movement sub-score between the candidate target role type and the monitoring area is determined; Obtain the role types of other targets that appear simultaneously with the valid candidate target, and determine the co-occurrence relationship sub-score between the role types of the candidate target and the role types of other targets by querying a preset co-occurrence statistics table; We fused the time pattern sub-score, the role movement sub-score, and the co-occurrence relationship sub-score with weights to obtain the spatiotemporal context sub-score for the family scene.

[0065] The family scene spatiotemporal context sub-score refers to a score calculated based on three dimensions: the unique time and routine patterns of the family environment, the movement patterns of family members, and the co-occurrence relationships among family members. This score characterizes the reasonableness of a candidate target's appearance within the current family spatiotemporal context. The time-room statistics table is a pre-stored statistical table recording the historical probability of target activity in various family rooms (such as kitchen, living room, bedroom, and study) during different time periods (e.g., breakfast time, lunch time, evening time, late night time). The time pattern sub-score is a score calculated based on the matching degree between the time period of detection and the room type of the current monitoring area, reflecting whether the appearance of a target in that room during that time period conforms to the family's daily routine. The candidate target role type refers to the identity category label categorized based on the movement speed values ​​of valid candidate targets, such as including but not limited to children, adults, and the elderly, used to characterize the target's mobility characteristics. The role-location statistics table is a pre-stored statistical table recording the prior probability values ​​of various role types (e.g., children, adults, and the elderly) appearing in different monitoring areas or room types. Role movement sub-scores can be scores calculated based on the matching degree between candidate target role types and the current monitoring area, reflecting whether the appearance of this type of role in this location conforms to the family's daily behavioral patterns. Co-occurrence statistics tables can be pre-stored statistical tables used to record the prior probability values ​​of different role types appearing simultaneously in the same spatiotemporal range. Co-occurrence relationship sub-scores can be scores calculated based on the association relationship between candidate target role types and other simultaneously appearing target role types, reflecting whether the co-occurrence of this combination conforms to the family's inherent social structure.

[0066] In this embodiment of the invention, the family scene context matching score further includes a family scene spatiotemporal context sub-score, the determination process of which specifically includes: S1. The system can obtain the detection time of valid candidate targets, such as reading the detection timestamp of the target entering the current camera from the detection record of the target.

[0067] S2. The system can determine the time period category based on the detection time, including but not limited to breakfast time, lunch time, dinner time, and late night time. Then, using the "time period category" and "candidate target room type" as a joint index, it queries a pre-defined time period-room statistics table. This statistics table stores the historical probability values ​​of target activity occurring in various types of family rooms under each time period category. The obtained probability value is used as a time pattern sub-score; a higher probability indicates that the occurrence of the target activity in that room during that time period is more consistent with the family's daily routine.

[0068] S3. The system can extract the movement speed of valid candidate targets and compare it with a preset speed range threshold to determine the corresponding candidate target role type. For example, when the speed value is greater than 2 m / s, the target role type is determined to be a child, representing running behavior; when the speed value is between 0.8 m / s and 1.5 m / s, it is determined to be an adult, representing normal walking behavior; when the speed value is less than 0.5 m / s, it is determined to be an elderly person, representing slow walking behavior. If the speed value falls outside the above range, the system can classify it as unknown or other types.

[0069] S4. The system uses the candidate target role type determined in step S3 and the room type of the current monitoring area as a joint index to query the preset role-location statistics table. This statistics table stores the prior probability value of each role type appearing in each room type. The obtained probability value is the role movement sub-score. The higher the score, the more consistent the appearance of this type of role in that location is with the family's daily behavior pattern.

[0070] S5. Based on the detection time, the system searches for other detected moving targets within the same frame of the camera containing the valid candidate target. If any exist, the system determines the role type of these other targets according to the method in step S3. Then, using the role type of the candidate target and the role type of other targets as a binary index, the system queries a preset co-occurrence statistics table to obtain the prior probability value of both role types appearing simultaneously in the household. This probability value is the co-occurrence relation sub-score. If no other targets are present in the frame, the co-occurrence relation sub-score can be a default average value, such as 0.5.

[0071] S6. The system weights and sums the time pattern sub-score, the role movement sub-score, and the co-occurrence relationship sub-score according to preset weights. The final weighted sum is the spatiotemporal context sub-score of the family scene, which falls within the interval [0,1]. The time pattern sub-score can be assigned a weight of 0.4, the role movement sub-score can be assigned a weight of 0.3, and the co-occurrence relationship sub-score can be assigned a weight of 0.3. The sum of the weights of the three is 1.

[0072] This embodiment explicitly models three types of prior knowledge inherent in family scenarios—time-based routines, differences in member movement patterns, and social co-occurrence structures—as independently computable scoring factors. This enables the cross-camera target association system to possess scene semantic understanding capabilities that surpass purely visual appearance discrimination. Specifically, the time-based routine sub-score utilizes the periodic constraints of daily family routines to exclude illogical associations (such as an adult target appearing in the living room late at night). The role-based movement sub-score uses significant differences in movement speed among members of different age groups to constrain identity determination (such as distinguishing between a child running and an elderly person walking slowly when they appear similar). The co-occurrence relationship sub-score utilizes the stable social accompaniment structure among family members to provide additional identity verification signals (such as the co-occurrence of mother and infant). These three sub-scores are independent yet complementary, collectively forming a time-individual-social triple semantic verification network. This allows for high-confidence auxiliary discrimination criteria even in challenging scenarios where appearance features become ineffective due to changes in perspective, lighting differences, or similar clothing, significantly improving the overall accuracy and robustness of cross-camera target association in home surveillance environments.

[0073] Furthermore, the independent scores of the five dimensions—spatiotemporal consistency score, visual similarity score, motion state matching score, neighborhood item context sub-score, and home scene spatiotemporal context sub-score—can be explicitly weighted and fused together, with the sum of the preset fusion weights corresponding to each dimension strictly normalized to 1.0. The visual similarity score is given the highest weight as the core matching criterion for cross-camera target recognition, because visual appearance is the most direct and stable signal for determining the same physical entity. The motion state matching score is given the second highest weight as an auxiliary verification method, which uses the consistency of the target's motion direction and the continuity of its trajectory in spatiotemporal connection when switching between cameras to provide strong evidence for visual similarity matching. The spatiotemporal consistency score is given a moderate weight to quickly filter effective candidate targets in the initial stage of the matching process by using the spatial adjacency relationship and time window in the preset camera network topology, which greatly reduces the processing overhead of subsequent high-computational-dimensional tasks without losing boundary candidates. The neighborhood item context sub-score is given a low weight to provide auxiliary discrimination signals in subtle ways by using the statistical correlation between items and rooms in the home scene. The home scene spatiotemporal context sub-score is given a moderate weight as a unique dimension that distinguishes this invention from general multi-target tracking technology. It makes full use of three types of prior knowledge in the home scene: time period-room routine, role-position movement pattern, and co-occurrence relationship between roles to enhance the semantic depth of discrimination. The aforementioned weight allocation system is driven by visual similarity scoring, uses motion state matching scoring for spatial constraint verification, and uses home scene spatiotemporal context sub-scoring for scene semantic enhancement. The five dimensions each have their own focus and complement each other, together forming a comprehensive scoring framework that is well-defined, highly interpretable, and suitable for lightweight deployment of home edge devices. The final comprehensive matching score is used to compare with a preset matching threshold to determine whether a cross-camera association should be established between the source target and the valid candidate target.

[0074] In one embodiment, this invention also provides two alternative fusion strategies to adapt to the needs of different application scenarios: Alternative method one is adaptive weight fusion, which uses learnable parameters instead of fixed preset weights. It collects labeled data from home scenarios (i.e., known correct and incorrect matching sample pairs) as supervision signals and uses a backpropagation algorithm to iteratively optimize the weight parameters, allowing the weights of each dimension to adaptively adjust to the optimal configuration during training. This method is suitable for home deployment environments that have accumulated a certain amount of labeled data and pursue higher accuracy. Alternative method two is Bayesian fusion, which treats the scores of each dimension as independent evidence information. It uses Bayes' theorem to convert the likelihood probability of each dimension into a posterior probability, expressing the credibility of "the source target and the candidate target are the same entity" in probabilistic form. This method is suitable for scenarios with a strong need for probabilistic interpretation of decision results (such as needing to output confidence scores for secondary verification or human-machine collaborative judgment). Neither alternative implementation changes the five-dimensional scoring framework of this invention; they only provide flexible selection space for the fusion mechanism.

[0075] Furthermore, based on the above embodiments of the invention, according to the comprehensive matching score, an association is established between the source target and the successfully matched valid candidate targets, including: Determine the highest score among all comprehensive matching scores; Determine whether the highest score is greater than or equal to a preset matching threshold; If so, the valid candidate target corresponding to the highest score is determined as the successfully matched target, and a correlation is established between the source target and the successfully matched target; If not, then it is determined that there is no valid candidate target that matches the source target.

[0076] In this embodiment of the invention, the process of determining a successfully matched target based on a comprehensive matching score and associating it with the source target specifically includes: the system can aggregate the comprehensive matching scores corresponding to each valid candidate target and determine the highest score among them. Then, the highest score is compared with a preset matching threshold (e.g., 0.65). If the highest score is greater than or equal to the preset matching threshold, the valid candidate target corresponding to the highest score is determined as a successfully matched target, and an association is established with the source target; if the highest score is less than the preset matching threshold, the system directly determines that there is currently no valid candidate target belonging to the same physical entity as the source target.

[0077] This includes establishing the association between the source target and the successfully matched target, including: Determine whether a globally recognized identity already exists for the successfully matched target; If a target with a global identity already exists, the source target's local identity will be associated with the global identity. If the successfully matched target does not have a global identity, a new global identity is created, and the local identities of both the source target and the successfully matched target are associated with the new global identity.

[0078] The global identity identifier can refer to a unique and unchanging identification code assigned to a target individual in the entire home monitoring system, which is used to connect all the trajectory segments of the individual appearing in any camera and at any time period.

[0079] Specifically, after completing multi-dimensional score fusion and threshold determination to identify a successfully matched target, the system can use the camera ID and its corresponding local identifier of the successfully matched target as joint query conditions to retrieve the system's global identity mapping table. This mapping table records the global identity identifier corresponding to each physical entity that has been confirmed by the system, as well as all local identifiers that the entity has appeared in various cameras. Based on the joint query conditions, the system searches the mapping table to see if there is a record whose "camera ID" and "local identifier" completely match the current information of the successfully matched target.

[0080] When the query results show that a successfully matched target has already been assigned a global identity, the system does not create a new identity number. Instead, it directly appends the source target's local identifier (including the source camera number and its local temporary ID) as a new record to the mapping entry corresponding to the global identity. At the same time, the system appends the source target's trajectory point data (position coordinates, detection time, velocity vector, etc.) in the current camera to the cross-camera complete trajectory list corresponding to the global identity, completing the continuation and stitching of the trajectory.

[0081] When the query results show that the successfully matched target has never been assigned any global identity, the system generates a brand new global identity according to a preset numbering format (e.g., "GLOBAL-" followed by six incrementing digits). The system then creates a new identity record in the global mapping table with this new global identity as the primary key. It then writes both the local identifier of the successfully matched target (the candidate target's camera number and its local ID) and the local identifier of the source target (the source camera number and its local ID) into this record. Simultaneously, the system merges the trajectory point data of the source target and the successfully matched target in both cameras and stores them together in the cross-camera trajectory list corresponding to the new global identity, forming a complete transfer path from the source camera to the adjacent camera.

[0082] This embodiment employs a two-level decision-making mechanism—comparing the highest score and determining a preset threshold—to ensure that association is only executed when the comprehensive matching score reaches a reliable confidence level. This effectively avoids erroneous bindings caused by low-confidence matching, significantly improving the accuracy and robustness of cross-camera association. Simultaneously, a global identity query and branching strategy is used in the association establishment phase: if a successfully matched target already has a global identity, the source target's local identity is directly appended to maintain identity uniqueness and trajectory continuity; if not, a new global identity is created at once and simultaneously bound to the source and target's local identities, ensuring a complete identity mapping closure from the initial moment of matching confirmation. This mechanism seamlessly transforms multi-dimensional fusion scoring results into stable cross-camera identity management with extremely low storage and query overhead. It eliminates the problem of creating multiple global IDs for the same target and ensures smooth connection of target trajectories during cross-view transfers. This creates a logically rigorous and data-consistent closed loop for the entire target association system from decision-making to execution, significantly improving the user experience and system reliability of continuous multi-target tracking in home surveillance scenarios.

[0083] Figure 2 A flowchart illustrating another cross-view target association method provided in an embodiment of the present invention. (See flowchart for example.) Figure 2 As shown, the entire process begins with the source camera detecting the target to be associated. The system responds to this detection event by acquiring the association information of targets detected by other cameras. First, candidate targets corresponding to adjacent cameras are filtered according to the spatial adjacency relationship in the preset camera network topology. Candidate targets whose detection time difference is within the target transfer time window are identified as valid candidate targets. Simultaneously, the spatiotemporal consistency score of each valid candidate target is calculated based on the relative position of the detection time difference within the window. Then, multi-dimensional matching scores are calculated in parallel for each selected valid candidate target. These include extracting visual feature vectors and calculating normalized similarity to obtain a visual similarity score; obtaining motion direction and historical trajectory and fusing direction consistency sub-scores and trajectory continuity sub-scores to obtain a motion state matching score; and obtaining context sub-scores for neighboring items and the time frame of the home scene. The spatial context sub-score is used to construct the family scene context matching score. Then, the spatiotemporal consistency score, visual similarity score, motion state matching score, and family scene context matching score are fused according to preset weights to obtain the comprehensive matching score of each valid candidate target. Finally, the highest score among all comprehensive matching scores is determined, and it is judged whether the highest score is greater than or equal to the preset matching threshold. If the condition is met, the valid candidate target corresponding to the highest score is determined as the successfully matched target and an association is established. Specifically, it is checked whether the successfully matched target already has a global identity. If it already exists, the local identity of the source target is associated with the global identity. If it does not exist, a new global identity is created and the local identities of the source target and the successfully matched target are associated with the new identity. If the threshold condition is not met, it is determined that there is no matching target.

[0084] This embodiment uses a preset camera network topology for spatiotemporal constraint screening, eliminating a large number of physically unrelated candidate targets in the early stages of matching calculations. This significantly reduces the computational overhead of subsequent multi-dimensional scoring, enabling the system to run in real-time on home edge devices. Simultaneously, it integrates multi-dimensional information such as visual similarity, motion consistency, and home scene context with explicit weights, compensating for the insufficient discriminative power of single appearance features under cross-view lighting changes and viewpoint differences. The home scene context matching scoring fully utilizes the strong correlation between objects and rooms in the home environment, the temporal regularity of family members' activities, differences in role movement patterns, and co-occurrence relationships among members, effectively improving the distinguishability between appearance-similar targets. The highest score combined with a preset threshold judgment mechanism ensures that identity binding is only performed when the matching confidence is sufficiently high. The query and branching processing of global identity identifiers ensures that the same physical entity has a unique global identity throughout the system and that its trajectory is continuous and complete. Ultimately, this achieves a cross-camera target association scheme with high accuracy, low computational cost, interpretability, and unified identity management capabilities.

[0085] Figure 3 This is a schematic diagram of a cross-view target association device provided in an embodiment of the present invention. Figure 3 As shown, this device is used in a home monitoring system that includes multiple cameras, specifically including: The information acquisition module 21 is used to acquire the association information of the targets detected by the other cameras in response to the source camera detecting the source target to be associated. The spatiotemporal filtering module 22 is used to perform spatiotemporal constraint filtering on associated information based on a preset camera network topology, and to determine valid candidate targets and their corresponding spatiotemporal consistency scores. The multidimensional scoring module 23 is used to determine the multidimensional matching score of each valid candidate target. The multidimensional matching score includes at least visual similarity score, motion state matching score and home scene context matching score. The scoring fusion module 24 is used to fuse the spatiotemporal consistency score, visual similarity score, motion state matching score and family scene context matching score to obtain the comprehensive matching score of each effective candidate target; The target association module 25 is used to establish an association between the source target and the successfully matched valid candidate targets based on the comprehensive matching score.

[0086] Furthermore, based on the above embodiments of the invention, the preset camera network topology includes the spatial adjacency relationship between each camera and the target transfer time window between adjacent cameras. The spatiotemporal filtering module 22 is specifically used for: Based on spatial adjacency, candidate targets corresponding to cameras adjacent to the source camera are determined from the associated information; Determine the detection time difference between each candidate target and the source target, and identify candidate targets whose detection time difference is within the target transfer time window as valid candidate targets; Based on the relative position of the detection time difference within the target transfer time window, the spatiotemporal consistency score corresponding to each valid candidate target is determined.

[0087] Furthermore, based on the above embodiments of the invention, when the spatiotemporal screening module 22 determines the spatiotemporal consistency score according to the relative position of the detection time difference within the target transfer time window, it is specifically used for: Determine the window midpoint and window half-width of the target transfer time window; Determine the distance between the detection time difference and the point in the window, and use the ratio of the distance to the half-width of the window as the median value; Determine the difference between the median value and the preset maximum score, and determine the maximum value between the difference and the preset minimum score as the spatiotemporal consistency score.

[0088] Furthermore, based on the above embodiments of the invention, the multidimensional scoring module 23 includes a visual scoring unit, which is specifically used for: Extract the first visual feature vector of the source target and the second visual feature vector of the effective candidate targets respectively; Determine the similarity between the second visual feature vector and the first visual feature vector; The similarity scores are normalized to obtain a visual similarity score.

[0089] Furthermore, based on the above embodiments of the invention, the multidimensional scoring module 23 includes a motion scoring unit, which is specifically used for: The first direction of motion and the first historical trajectory of the source target, and the second direction of motion and the second historical trajectory of the effective candidate targets are obtained respectively. Determine the degree of consistency between the first and second motion directions to obtain a direction consistency sub-score; Determine the degree of continuity in time and space between the end of the first historical trajectory and the beginning of the second historical trajectory to obtain a trajectory continuity sub-score; The motion state matching score is obtained by weighted fusion of the orientation consistency sub-score and the trajectory continuity sub-score.

[0090] Furthermore, based on the above embodiments of the invention, the family scene context matching score includes a neighborhood item context sub-score and a family scene spatiotemporal context sub-score; the multi-dimensional scoring module 23 includes a family scene scoring unit; When using the home scene scoring unit to obtain contextual sub-scorings for neighboring items, it is specifically used for: Obtain the categories of items detected within the monitored area where valid candidate targets are located, as well as the type of family room to which the monitored area belongs; By querying a pre-defined item-room statistics table, the first degree of matching between item categories and family room types is determined; By querying a pre-set item-camera statistics table, a second degree of matching between item categories and monitored areas is determined; We perform a weighted fusion of the first and second matching scores to obtain the context sub-score of the neighboring items.

[0091] When using the family scene scoring unit to obtain the spatiotemporal context sub-score of the family scene, it is specifically used for: Obtain the detection time of valid candidate targets; By querying a preset time period-room statistics table, the time pattern sub-score between the detection time and the family room type is determined; The corresponding candidate target role type is determined based on the movement speed of the valid candidate targets; By querying the preset role-position statistics table, the role movement sub-score between the candidate target role type and the monitoring area is determined; Obtain the role types of other targets that appear simultaneously with the valid candidate target, and determine the co-occurrence relationship sub-score between the role types of the candidate target and the role types of other targets by querying a preset co-occurrence statistics table; We fused the time pattern sub-score, the role movement sub-score, and the co-occurrence relationship sub-score with weights to obtain the spatiotemporal context sub-score for the family scene.

[0092] Furthermore, based on the above embodiments of the invention, the target association module 25 is specifically used for: Determine the highest score among all comprehensive matching scores; Determine whether the highest score is greater than or equal to a preset matching threshold; If so, the valid candidate target corresponding to the highest score is determined as the successfully matched target, and a correlation is established between the source target and the successfully matched target; If not, then it is determined that there is no valid candidate target that matches the source target.

[0093] Furthermore, based on the above embodiments of the invention, the target association module 25, when establishing the association relationship between the source target and the successfully matched target, is specifically used for: Determine whether a globally recognized identity already exists for the successfully matched target; If a target with a global identity already exists, the source target's local identity will be associated with the global identity. If the successfully matched target does not have a global identity, a new global identity is created, and the local identities of both the source target and the successfully matched target are associated with the new global identity.

[0094] The cross-view target association device provided in the embodiments of the present invention can execute the cross-view target association method provided in any embodiment of the present invention, and has the corresponding functional modules and beneficial effects of the method execution.

[0095] Figure 4 A schematic diagram of an electronic device 40 that can be used to implement embodiments of the present invention is shown. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device can also represent various forms of mobile devices, such as personal digital processors, cellular phones, smartphones, wearable devices (e.g., helmets, glasses, watches, etc.), and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely illustrative and are not intended to limit the implementation of the invention described and / or claimed herein.

[0096] like Figure 4 As shown, the electronic device 40 includes at least one processor 41 and a memory, such as a read-only memory (ROM) 42 or a random access memory (RAM) 43, communicatively connected to the at least one processor 41. The memory stores computer programs executable by the at least one processor. The processor 41 can perform various appropriate actions and processes based on the computer program stored in the ROM 42 or loaded from storage unit 48 into the RAM 43. The RAM 43 may also store various programs and data required for the operation of the electronic device 40. The processor 41, ROM 42, and RAM 43 are interconnected via a bus 44. An input / output (I / O) interface 45 is also connected to the bus 44.

[0097] Multiple components in electronic device 40 are connected to I / O interface 45, including: input unit 46, such as keyboard, mouse, etc.; output unit 47, such as various types of monitors, speakers, etc.; storage unit 48, such as disk, optical disk, etc.; and communication unit 49, such as network card, modem, wireless transceiver, etc. Communication unit 49 allows electronic device 40 to exchange information / data with other devices through computer networks such as the Internet and / or various telecommunications networks.

[0098] Processor 41 can be a variety of general-purpose and / or special-purpose processing components with processing and computing capabilities. Some examples of processor 41 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various special-purpose artificial intelligence (AI) computing chips, various processors running machine learning model algorithms, a digital signal processor (DSP), and any suitable processor, controller, microcontroller, etc. Processor 41 performs the various methods and processes described above, such as cross-view target association methods.

[0099] In some embodiments, the cross-view target association method may be implemented as a computer program tangibly contained in a computer-readable storage medium, such as storage unit 48. In some embodiments, part or all of the computer program may be loaded and / or mounted on electronic device 40 via ROM 42 and / or communication unit 49. When the computer program is loaded into RAM 43 and executed by processor 41, one or more steps of the cross-view target association method described above may be performed. Alternatively, in other embodiments, processor 41 may be configured to perform the cross-view target association method by any other suitable means (e.g., by means of firmware).

[0100] Various embodiments of the systems and techniques described above herein can be implemented in digital electronic circuit systems, integrated circuit systems, field-programmable gate arrays (FPGAs), application-specific integrated circuits (ASICs), application-specific standard products (ASSPs), systems-on-a-chip (SoCs), payload-programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various embodiments may include implementations in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which may be a dedicated or general-purpose programmable processor, capable of receiving data and instructions from a storage system, at least one input device, and at least one output device, and transmitting data and instructions to the storage system, the at least one input device, and the at least one output device.

[0101] In some embodiments, the cross-view target association method can be implemented as a computer program, which is implicitly included in a computer program product. When executed by a processor, the computer program implements the cross-view target association method of the present invention. The computer program product can be understood as a software product that primarily implements its solution through a computer program. The computer program used to implement the method of the present invention can be written in any combination of one or more programming languages. These computer programs can be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing device, such that when executed by the processor, the computer program causes the functions / operations specified in the flowcharts and / or block diagrams to be implemented. The computer program can be executed entirely on a machine, partially on a machine, as a standalone software package partially on a machine and partially on a remote machine, or entirely on a remote machine or server.

[0102] In the context of this invention, a computer-readable storage medium can be a tangible medium that may contain or store a computer program for use by or in conjunction with an instruction execution system, apparatus, or device. A computer-readable storage medium may include, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatus, or devices, or any suitable combination thereof. Alternatively, a computer-readable storage medium may be a machine-readable signal medium. More specific examples of machine-readable storage media include electrical connections based on one or more wires, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fibers, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination thereof.

[0103] To provide interaction with a user, the systems and techniques described herein can be implemented on an electronic device having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user; and a keyboard and pointing device (e.g., a mouse or trackball) through which the user provides input to the electronic device. Other types of devices can also be used to provide interaction with the user; for example, feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including sound input, voice input, or tactile input).

[0104] The systems and technologies described herein can be implemented in computing systems that include backend components (e.g., as data servers), or middleware components (e.g., application servers), or frontend components (e.g., user computers with graphical user interfaces or web browsers through which users can interact with implementations of the systems and technologies described herein), or any combination of such backend, middleware, or frontend components. The components of the system can be interconnected via digital data communication of any form or medium (e.g., communication networks). Examples of communication networks include local area networks (LANs), wide area networks (WANs), blockchain networks, and the Internet.

[0105] A computing system can include clients and servers. Clients and servers are generally located far apart and typically interact through communication networks. The client-server relationship is created by computer programs running on the respective computers and having a client-server relationship with each other. The server can be a cloud server, also known as a cloud computing server or cloud host, which is a hosting product within the cloud computing service system to address the shortcomings of traditional physical hosts and VPS services, such as high management difficulty and weak business scalability.

[0106] It should be understood that the various forms of processes shown above can be used, with steps reordered, added, or deleted. For example, the steps described in this invention can be executed in parallel, sequentially, or in different orders, as long as the desired result of the technical solution of this invention can be achieved, and this is not limited herein.

[0107] The specific embodiments described above do not constitute a limitation on the scope of protection of this invention. Those skilled in the art should understand that various modifications, combinations, sub-combinations, and substitutions can be made according to design requirements and other factors. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of this invention should be included within the scope of protection of this invention.

Claims

1. A cross-perspective target association method, characterized in that, Applied to a home monitoring system containing multiple cameras, the method includes: In response to the source camera detecting the source target to be associated, obtain the association information of the targets detected by the other cameras; Based on the preset camera network topology, the associated information is subjected to spatiotemporal constraint filtering to determine valid candidate targets and their corresponding spatiotemporal consistency scores. A multidimensional matching score is determined for each of the effective candidate targets, wherein the multidimensional matching score includes at least a visual similarity score, a motion state matching score, and a home scene context matching score; The spatiotemporal consistency score, the visual similarity score, the motion state matching score, and the home scene context matching score are fused to obtain a comprehensive matching score for each of the effective candidate targets; Based on the comprehensive matching score, the source target is associated with the successfully matched valid candidate targets.

2. The method according to claim 1, characterized in that, The preset camera network topology includes the spatial adjacency relationships between cameras and the target transfer time window between adjacent cameras. The step of performing spatiotemporal constraint filtering on the associated information based on the preset camera network topology to determine valid candidate targets and their corresponding spatiotemporal consistency scores includes: Based on the spatial adjacency relationship, candidate targets corresponding to cameras adjacent to the source camera are determined from the association information; Determine the detection time difference between each candidate target and the source target, and identify the candidate targets whose detection time difference is within the target transfer time window as the valid candidate targets; Based on the relative position of the detection time difference within the target transfer time window, the spatiotemporal consistency score corresponding to each of the valid candidate targets is determined.

3. The method according to claim 2, characterized in that, The step of determining the spatiotemporal consistency score corresponding to each of the valid candidate targets based on the relative position of the detection time difference within the target transfer time window includes: Determine the window midpoint and window half-width of the target transfer time window; Determine the distance between the detection time difference and the point in the window, and use the ratio of the distance to the half-width of the window as an intermediate value; The difference between the intermediate value and the preset maximum score is determined, and the maximum value between the difference and the preset minimum score is determined as the spatiotemporal consistency score.

4. The method according to claim 1, characterized in that, The process of determining the visual similarity score includes: Extract the first visual feature vector of the source target and the second visual feature vector of the effective candidate target respectively; Determine the similarity between the second visual feature vector and the first visual feature vector; The similarity is normalized to obtain the visual similarity score.

5. The method according to claim 1, characterized in that, The process of determining the motion state matching score includes: The first direction of motion and the first historical trajectory of the source target, and the second direction of motion and the second historical trajectory of the effective candidate target are obtained respectively. Determine the degree of consistency between the first motion direction and the second motion direction to obtain a direction consistency sub-score; Determine the degree of continuity in time and space between the end of the first historical trajectory and the beginning of the second historical trajectory to obtain a trajectory continuity sub-score; The motion state matching score is obtained by weighted fusion of the direction consistency sub-score and the trajectory continuity sub-score.

6. The method according to claim 1, characterized in that, The family scene context matching score includes a neighborhood item context sub-score and a family scene spatiotemporal context sub-score; The process of obtaining the neighborhood item context sub-rating includes: Obtain the categories of items detected within the monitoring area where the valid candidate target is located, as well as the type of family room to which the monitoring area belongs; By querying a preset item-room statistics table, the first degree of matching between the item category and the family room type is determined; By querying a preset item-camera statistics table, a second matching degree between the item category and the monitoring area is determined; The first matching degree and the second matching degree are weighted and fused to obtain the neighborhood item context sub-score; The process of obtaining the spatiotemporal context sub-score for the family scene includes: Obtain the detection time of the valid candidate target; By querying a preset time period-room statistics table, a time pattern sub-score is determined between the detection time and the family room type; The corresponding candidate target role type is determined based on the movement speed of the valid candidate targets; By querying a preset role-position statistics table, the role movement sub-score between the candidate target role type and the monitoring area is determined; Obtain the role types of other targets that appear simultaneously with the valid candidate target, and determine the co-occurrence relationship sub-score between the role types of the candidate target and the role types of the other targets by querying a preset co-occurrence statistics table; The time pattern sub-score, the role movement sub-score, and the co-occurrence relationship sub-score are weighted and fused to obtain the spatiotemporal context sub-score of the family scene.

7. The method according to claim 1, characterized in that, The step of establishing an association between the source target and the successfully matched valid candidate targets based on the comprehensive matching score includes: Determine the highest score among all the comprehensive matching scores; Determine whether the highest score is greater than or equal to a preset matching threshold; If so, the valid candidate target corresponding to the highest score is determined as the successfully matched target, and an association is established between the source target and the successfully matched target; If not, it is determined that there is no valid candidate target that matches the source target.

8. The method according to claim 7, characterized in that, Establishing the association between the source target and the successfully matched target includes: Determine whether the successfully matched target already has a global identity identifier; If the successfully matched target already has a global identity identifier, then the local identifier of the source target is associated with the global identity identifier; If the successfully matched target does not have a global identity, a new global identity is created, and the local identities of both the source target and the successfully matched target are associated with the new global identity.

9. An electronic device, characterized in that, The electronic device includes: At least one processor; and A memory communicatively connected to the at least one processor; wherein, The memory stores a computer program that can be executed by the at least one processor to enable the at least one processor to perform the cross-view target association method according to any one of claims 1-8.

10. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores computer instructions that cause a processor to execute the cross-view target association method of any one of claims 1-8.

Images