Trajectory generation method, electronic device, and storage medium

Abstract

The application provides a trajectory generation method, an electronic device and a storage medium. The trajectory generation method comprises the following steps: performing encoding processing on video data collected by a plurality of cameras and Internet of Things data of an environment to be detected, to obtain a space-time auxiliary vector of the environment to be detected; determining visual features of a to-be-detected object recognized by the plurality of cameras based on the video data; determining a joint feature vector of the to-be-detected object based on the space-time auxiliary vector and the visual features; determining an entry strategy of the joint feature vector based on the joint feature vector and state parameters of the plurality of cameras; determining an index parameter of the to-be-detected object based on the joint feature vector and topological information of the plurality of cameras; and fusing identity association results and real-time position information of the to-be-detected object based on a coordinate system conversion rule of the plurality of cameras, to obtain trajectory data of the to-be-detected object moving in the environment to be detected. The application can improve the accuracy of cross-region object tracking.

Description

Technical Field

[0001] This application relates to the field of industrial internet technology, specifically to the field of trajectory tracking technology, and particularly to a trajectory generation method, electronic device, and storage medium. Background Technology

[0002] With the deepening of smart city construction and the continuous upgrading of public safety and security needs, the deployment scale of video surveillance networks has also increased. A large number of edge cameras operating independently have formed isolated visual data silos. In security monitoring scenarios, continuous tracking and identity association of targets captured by multiple cameras deployed across regions have become mainstream business requirements.

[0003] Currently, cross-regional target tracking is usually carried out through manual screening, which requires security personnel to repeatedly compare video data collected by multiple camera devices. This method is inefficient, has slow response times, and high labor costs, and cannot meet the real-time response requirements of large-scale monitoring networks. Summary of the Invention

[0004] In view of the above, it is necessary to propose a trajectory generation method, electronic device, and storage medium to solve the technical problem of low accuracy in locating and tracking cross-regional objects. The related equipment includes a trajectory generation device and an electronic device.

[0005] This application provides a trajectory generation method, the method comprising: encoding video data collected by multiple camera devices in an environment to be detected and IoT data of the environment to be detected to obtain a spatiotemporal auxiliary vector of the environment to be detected; determining visual features of the object to be detected by the multiple camera devices based on the video data; determining a joint feature vector of the object to be detected based on the spatiotemporal auxiliary vector and the visual features; determining an input strategy for the joint feature vector based on the joint feature vector and the state parameters of the multiple camera devices, the input strategy indicating that the joint feature vector is written into a feature vector library; determining an index parameter of the object to be detected based on the joint feature vector and the topological information of the multiple camera devices, the index parameter being used to determine the identity association result of the object to be detected collected by the multiple camera devices from the feature vector library; fusing the identity association result and the real-time position information of the object to be detected collected by the multiple camera devices based on the coordinate system transformation rules corresponding to the multiple camera devices to obtain trajectory data of the object to be detected moving in the environment to be detected; and outputting the trajectory data.

[0006] In some embodiments, encoding video data collected by multiple camera devices in the environment to be detected and IoT data of the environment to be detected to obtain a spatiotemporal auxiliary vector of the environment to be detected includes: synchronously acquiring the video data from the multiple camera devices based on a preset communication protocol; performing structured processing on the IoT data to obtain structured IoT data; the structured IoT data includes the three-dimensional coordinates of the multiple camera devices in the environment to be detected, the timestamps corresponding to the video frames of the video data, the light intensity of the environment to be detected, the access control status of the environment to be detected, and the latitude and longitude coordinates of the object to be detected; and encoding the video data and the structured IoT data using a preset encoding model to obtain the spatiotemporal auxiliary vector.

[0007] In some embodiments, determining the visual features of the object to be detected by the plurality of cameras based on the video data includes: decoding the video data to obtain an original image in the video data; determining a target detection box corresponding to the object to be detected in the original image based on the original image using a preset target detection model; determining the image in the target detection box as the target image; and determining the visual features of the object to be detected based on the target image using a preset feature extraction model.

[0008] In some embodiments, determining the joint feature vector of the object to be detected based on the spatiotemporal auxiliary vector and the visual features includes: normalizing the visual features to obtain normalized visual features; reducing the dimensionality of the normalized visual features to obtain cross-scene visual features; concatenating the cross-scene visual features and the spatiotemporal auxiliary vector to obtain a first fusion feature; normalizing the first fusion feature to obtain a normalized fusion feature; encoding the identifiers of the plurality of camera devices, the timestamps for acquiring the visual features, the three-dimensional coordinates of the camera devices in a preset coordinate system, and the identifier of the object to be detected to obtain metadata encoding; and generating key-value pairs based on the metadata encoding and the normalized fusion feature to obtain a joint feature vector.

[0009] In some embodiments, determining the database entry strategy for the joint feature vector based on the joint feature vector and the state parameters of the plurality of camera devices includes: acquiring the state parameters of the plurality of camera devices, the state parameters including the frame rate at which the plurality of camera devices acquire video data, the cache queue length corresponding to the joint feature vector to be entered into the database, and the processor utilization rate; determining the database entry frequency of the joint feature vector based on the frame rate and the number of joint feature vectors; updating the database entry frequency based on the processor utilization rate to obtain an updated database entry frequency; determining the write batch of the joint feature vector based on the updated database entry frequency and the cache queue length to obtain the database entry strategy for the joint feature vector; the database entry strategy indicates the strategy for writing joint feature vector data generated from video data of different camera devices and different frame rates into the fragments corresponding to the feature vector database.

[0010] In some embodiments, determining the index parameters of the object to be detected based on the joint feature vector and the topological information of the plurality of camera devices includes: using the three-dimensional coordinates of the camera device corresponding to the joint feature vector and the timestamp of obtaining the joint feature vector as index fields to obtain the spatiotemporal index of the object to be detected; normalizing the joint feature vector to obtain the visual index of the object to be detected; and determining the spatiotemporal index and the visual index as the index parameters of the object to be detected.

[0011] In some embodiments, determining the identity association result of the object to be detected collected by the plurality of cameras from the feature vector library based on the index parameter includes: determining the preliminary association result of the object to be detected collected by the plurality of cameras from the feature vector library based on the spatiotemporal index; determining the distance and viewing angle between any two cameras based on the topological information of the plurality of cameras; determining the dynamic threshold between any two cameras based on the distance and the viewing angle; determining the similarity between the preliminary association result and the visual index; and determining the preliminary association result as the identity association result if the similarity is greater than or equal to the dynamic threshold.

[0012] In some embodiments, the step of fusing the identity association results and the real-time location information of the object to be detected collected by the multiple camera devices based on the coordinate system transformation rules corresponding to the multiple camera devices to obtain the trajectory data of the object to be detected moving in the environment to be detected includes: determining the transformation rules of the coordinate systems corresponding to any two camera devices based on a preset homography matrix; concatenating the identity association results into the historical movement trajectory of the object to be detected based on the transformation rules; wherein, the homography matrix is ​​used to indicate the coordinate transformation relationship of different camera devices; and concatenating the real-time location information of the object to be detected with the historical movement trajectory to obtain the trajectory data of the object to be detected moving in the environment to be detected.

[0013] This application also provides an electronic device, which includes: a memory for storing at least one instruction; and a processor for executing the instructions stored in the memory to implement the trajectory generation method.

[0014] This application also provides a computer-readable storage medium storing at least one instruction, which is executed by a processor in an electronic device to implement the trajectory generation method described above.

[0015] As can be seen from the above technical solutions, the embodiments of this application adopt a distributed parallel processing architecture, dynamically creating independent parallel processing task units for the video stream of each camera device, distributing high-load computing to multiple edge nodes for parallel execution, and realizing distributed deployment and storage-computation separation of computing resources. Through a dual normalization mechanism combining batch normalization and layer normalization, feature distribution bias caused by cross-camera device heterogeneity is eliminated; simultaneously, through a multimodal joint feature vector construction mechanism, standardized visual features and spatiotemporal auxiliary vectors are deeply integrated, improving the accuracy of target identity association across camera devices. A spatiotemporal hierarchical hybrid index system is constructed. First, a first-level spatiotemporal structured index quickly converges the retrieval scope and filters invalid spatiotemporal data. Then, a second-level customized visual index performs accurate similarity matching of high-dimensional vectors, balancing retrieval efficiency and matching accuracy. By using a pre-calibrated homography matrix, a standardized transformation rule was constructed between the image coordinate system of all camera devices and the global world coordinate system. This achieved global coordinate unification of all target location information and transformed discrete cross-camera device identity association results into standardized spatiotemporal location points with temporal attributes in the global coordinate system. This enabled continuous trajectory tracking of the detected object across regions within the monitoring area, improving the accuracy of locating and tracking cross-regional objects. Attached Figure Description

[0016] Figure 1 This is an application scenario diagram of a trajectory generation method provided in an embodiment of this application.

[0017] Figure 2 This is a flowchart of a trajectory generation method provided in an embodiment of this application.

[0018] Figure 3 This is a flowchart of a method for determining a spatiotemporal auxiliary vector provided in an embodiment of this application.

[0019] Figure 4 This is a flowchart of a method for determining visual features provided in an embodiment of this application.

[0020] Figure 5 This is a flowchart of a method for determining joint feature vectors provided in an embodiment of this application.

[0021] Figure 6 This is a flowchart of a method for determining an inbound strategy provided in an embodiment of this application.

[0022] Figure 7 This is a flowchart of a method for determining index parameters provided in an embodiment of this application.

[0023] Figure 8 This is a flowchart of a method for determining identity association results provided in an embodiment of this application.

[0024] Figure 9 This is a flowchart of a method for determining trajectory data provided in an embodiment of this application.

[0025] Figure 10 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application. Detailed Implementation

[0026] To better understand the purpose, features, and advantages of this application, a detailed description of the application is provided below with reference to the accompanying drawings and specific embodiments. It should be noted that, unless otherwise specified, the embodiments and features described in the embodiments of this application can be combined with each other. Numerous specific details are set forth in the following description to provide a thorough understanding of this application; the described embodiments are only a part of the embodiments of this application, and not all of them.

[0027] Furthermore, the terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of technical features indicated. Thus, a feature defined as "first" or "second" may explicitly or implicitly include one or more of the stated features. In the description of this application, "a plurality of" means two or more, unless otherwise explicitly specified.

[0028] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this application belongs. The terminology used herein is for the purpose of describing particular embodiments only and is not intended to be limiting of the application. The term "and / or" as used herein includes any and all combinations of one or more of the associated listed items.

[0029] This application provides a trajectory generation method that can be applied to one or more electronic devices. An electronic device is a device that can automatically perform numerical calculations and / or information processing according to pre-set or stored instructions. Its hardware includes, but is not limited to, microprocessors, application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), digital signal processors (DSPs), embedded devices, etc.

[0030] Electronic devices can be any electronic product that allows human-computer interaction with a customer, such as personal computers, tablets, smartphones, personal digital assistants (PDAs), game consoles, interactive network television (IPTV), smart wearable devices, etc.

[0031] Electronic devices may also include network devices and / or client devices. The network devices include, but are not limited to, a single network server, a server group consisting of multiple network servers, or a cloud based on cloud computing consisting of a large number of hosts or network servers.

[0032] The networks in which electronic devices are located include, but are not limited to, the Internet, wide area networks, metropolitan area networks, local area networks, and virtual private networks (VPNs).

[0033] like Figure 1 As shown, the trajectory generation method provided in this application can be applied to an electronic device 100, which is communicatively connected to multiple camera devices 200 in the environment to be detected. The electronic device 100 is used to encode video data collected by the multiple camera devices 200 in the environment to be detected and IoT data of the environment to be detected to obtain a spatiotemporal auxiliary vector of the environment to be detected.

[0034] Electronic device 100 determines the visual features of the object to be detected by the plurality of cameras based on video data; determines the joint feature vector of the object to be detected based on the spatiotemporal auxiliary vector and the visual features; determines the storage strategy of the joint feature vector based on the joint feature vector and the state parameters of the plurality of cameras, the storage strategy instructing the joint feature vector to be written into a feature vector library; and determines the index parameter of the object to be detected based on the joint feature vector and the topological information of the plurality of cameras, the index parameter being used to determine the identity association result of the object to be detected collected by the plurality of cameras from the feature vector library.

[0035] The electronic device 100 also integrates the identity association results and the real-time location information of the object to be detected collected by the multiple camera devices based on the coordinate system transformation rules corresponding to the multiple camera devices to obtain the trajectory data of the object to be detected moving in the detection environment; and outputs the trajectory data.

[0036] like Figure 2 The diagram shown is a flowchart of a trajectory generation method according to an embodiment of this application. The order of the steps in the flowchart can be changed, and some steps can be omitted, depending on different requirements. The trajectory generation method provided in this embodiment includes the following steps.

[0037] S20, the video data collected by multiple camera devices in the environment to be detected and the IoT data of the environment to be detected are encoded and processed to obtain the spatiotemporal auxiliary vector of the environment to be detected.

[0038] In one embodiment of this application, real-time access to video data and IoT data is achieved through a standard communication protocol. The IoT data is subjected to structured cleaning and feature extraction to obtain core spatiotemporal and environmental feature fields. The core feature fields include the three-dimensional spatial coordinates of the camera device, the millisecond-level timestamp of video acquisition, the ambient light intensity, the access control trigger status, and the latitude and longitude projection coordinates of the object to be detected. Redundant and invalid fields and outlier data are removed.

[0039] Load the pre-configured multilayer perceptron coding model, and lock the model's structural parameters, including: 16-dimensional input layer, 32-dimensional hidden layer, and 64-dimensional output layer. Use ReLU as the activation function and Adam as the optimizer to complete the initialization of the model inference environment.

[0040] For example, the encoding model can be a multilayer perceptron model or a fully connected neural network model. This application does not limit the specific type of encoding model.

[0041] The training data used to train the coding model can be historical data stored in advance within any sampling time period. This historical data can include the spatial three-dimensional coordinates of the camera device, the millisecond-level timestamp of the video captured by the camera device, the ambient light intensity within the sampling time period, the access control trigger status within the sampling time period, and the latitude and longitude projection coordinates of different active objects in the environment to be detected within the sampling time period.

[0042] For example, the initial parameters of the encoding and decoding models can be randomly generated using a preset program to initialize the encoding and decoding models. The decoding model can be a multilayer perceptron model or a fully connected neural network model; this application does not limit the specific type of the encoding model.

[0043] During the training of the encoding model, the latitude and longitude projection coordinates of different active objects in the training data can be used as label data. The training data is input into the encoding model in batches to obtain the predicted encoding vector output by the encoding model. The predicted encoding vector is then input into the decoding model to obtain the predicted latitude and longitude coordinates of different active objects output by the decoding model. The loss values ​​for the encoding and decoding models are determined based on the difference between the predicted latitude and longitude coordinates and the label data in the training data; Based on this loss value, the parameters of the encoding and decoding models are continuously updated using the backpropagation algorithm until the loss value meets a preset termination condition (e.g., the loss value is less than or equal to a preset termination threshold). At this point, updating the parameters of the encoding and decoding models stops, resulting in an encoding model and decoding model trained to a convergent state. The converged encoding model can then be used to generate spatiotemporal auxiliary vectors based on IoT data from the environment to be detected.

[0044] The preprocessed spatiotemporal and environmental feature fields are input into the initialized multilayer perceptron coding model. The input layer completes feature mapping, the hidden layer completes nonlinear feature transformation, and the output layer completes dimensional constraints and standardized output to generate fixed-dimensional feature coding results. The dimensionality and numerical range of the encoded feature vectors are checked to remove invalid vectors with abnormal dimensions or numerical overflow. Finally, the spatiotemporal auxiliary vector that meets the preset standard is output.

[0045] In a preferred embodiment, to meet the grid-based security monitoring requirements of the environment to be monitored, an independent encoding processing subunit is configured for each camera device. This subunit generates a unique spatiotemporal auxiliary vector for the video data of a single camera device and the IoT data of the corresponding deployment point, enabling a one-to-one binding between the spatiotemporal auxiliary vector and the camera device, thus providing a foundation for subsequent regional retrieval. Simultaneously, for low-light scenarios such as nighttime and backlighting, a light intensity weight coefficient is added to the core feature fields to improve the constraint effectiveness of the spatiotemporal auxiliary vector in low-light scenarios.

[0046] Output the spatiotemporal auxiliary vector after encoding and verification; at the same time, output the structured metadata of the camera device, including the unique ID of the camera device, spatial three-dimensional coordinates, and acquisition timestamp, which will be used for the subsequent metadata embedding of the joint feature vector.

[0047] In one embodiment of this application, the specific method for determining the spatiotemporal auxiliary vector can be found in [link to relevant documentation]. Figure 3 The corresponding detailed explanation.

[0048] S21, based on the video data, determine the visual features of the object to be detected by the plurality of camera devices; based on the spatiotemporal auxiliary vector and the visual features, determine the joint feature vector of the object to be detected.

[0049] In one embodiment of this application, real-time decapsulation and decoding processing is performed on the input compressed video data to restore it to the original image frame in RGB format. The original image frame is then subjected to 640×640 normalized scaling, color gamut calibration, and data enhancement to eliminate the resolution and lighting heterogeneity issues of different camera devices and output standardized image frame data.

[0050] Standardized image frame data is input into a pre-trained multi-object tracking model. Multi-scale basic features are extracted through a backbone feature extraction layer, and the receptive field of the features is enhanced by spatial pyramid pooling. Then, a path aggregation network is used to achieve bidirectional fusion of high- and low-dimensional features. Finally, the category, confidence score, and detection box coordinates of the object to be detected are output through a decoupled head structure. At the same time, through a tracking association layer, the appearance features of the object to be detected are fused with the motion information predicted by Kalman filter to perform inter-frame association, assign a unique tracking ID to each object to be detected, and output a cropped image of the object to be detected with a unique tracking ID.

[0051] The cropped image of the object to be detected is input into a pre-trained cross-camera device feature normalization model. The backbone network extracts the original visual features, capturing the core visual information of the object's appearance, texture, and contour. Feature optimization is performed using a triplet loss function to minimize the feature distance between similar objects across camera devices and maximize the feature distance between dissimilar objects. A dual normalization mechanism of batch normalization and layer normalization is introduced to align the features of the same object collected from different camera devices, eliminating heterogeneous biases in viewpoint, illumination, and resolution across camera devices. Finally, dimensionality reduction is performed through a fully connected layer to generate a standardized cross-scene visual feature vector.

[0052] The generated standardized cross-scene visual feature vectors are concatenated with spatiotemporal auxiliary vectors dimension by dimension to generate basic joint feature vectors, thus achieving the initial fusion of visual features and spatiotemporal environmental features.

[0053] Normalization is performed on the basic joint feature vectors, and the vector magnitude is strictly constrained to 1 to eliminate the impact of feature numerical scale differences on subsequent similarity retrieval and ensure the consistency of feature vector measurement.

[0054] A pre-defined metadata encoding rule for the camera device is constructed. The unique ID of the camera device, timestamp, spatial three-dimensional coordinates, and tracking ID of the object to be detected are standardized and digitally encoded to generate metadata encoding fields. These fields are then embedded into the header extension field of the basic joint feature vector through a key-value pair embedding mechanism, thereby achieving a strong binding between metadata and feature vector.

[0055] The vector with embedded metadata is input into the pre-trained verification network to perform feature integrity and encoding validity checks on the vector, eliminating invalid vectors with encoding errors and missing features, and finally generating a cross-camera device-specific joint feature vector with embedded metadata fields.

[0056] In a preferred embodiment, to address the variability of pedestrian features, during the training phase of the feature normalization model, publicly available pedestrian re-identification datasets and pedestrian samples collected from different camera devices in the actual detection environment are incorporated. This covers scenarios involving changes in pedestrian posture, occlusion, and clothing changes, improving the scene adaptability of the visual feature vectors. Simultaneously, in the metadata embedding stage, unique category identifier codes are assigned to different categories of detected objects (e.g., pedestrians, vehicles, non-motorized vehicles) to enable rapid category filtering during the retrieval process. The output is a cross-camera device-specific joint feature vector with metadata embedding fields.

[0057] In one embodiment of this application, the method for determining visual aid features can be found in [link to relevant documentation]. Figure 4 For detailed explanations, please refer to the section on methods for determining joint feature vectors. Figure 5 The corresponding detailed explanation.

[0058] S22, based on the joint feature vector and the state parameters of the multiple camera devices, determine the database entry strategy for the joint feature vector, wherein the database entry strategy indicates that the joint feature vector is written into the feature vector database.

[0059] In one embodiment of this application, a real-time status monitoring module is configured for each distributed processing node corresponding to the camera device to continuously collect and report the camera device's real-time frame rate, single-frame vector generation amount, cache queue length, node CPU / memory usage core status parameters, providing data support for the dynamic adjustment of the data entry strategy.

[0060] A preset data entry threshold is set, which is tied to the frame rate of the camera device. For example, 15fps corresponds to a baseline batch of 64 vectors and a baseline data entry frequency of 200ms; 20fps corresponds to a baseline batch of 128 vectors and a baseline data entry frequency of 150ms; 25 / 30fps corresponds to a baseline batch of 256 vectors and a baseline data entry frequency of 50 / 100ms. The system determines the initial data entry parameters by matching the corresponding data entry threshold based on the current main frame rate of the camera device.

[0061] For example, if the frame rate fluctuation of a single camera device reaches ±5fps, or the length of the vector cache queue to be loaded into the database at the corresponding node exceeds 3 times the baseline batch, a linear adjustment of ±30% is performed based on the baseline batch. With the addition of a node load feedback mechanism, if the node's CPU / memory utilization exceeds 70%, the batch size is reduced by 20%-30% based on the corrected batch size, while the loading interval is extended. If the node's CPU / memory utilization is below 30%, the loading batch size is increased and the loading interval is shortened, and finally, a dynamic loading strategy adapted to the current state is determined.

[0062] For all data insertion tasks on distributed processing nodes, asynchronous submissions are performed through the distributed vector database's insertion interface according to a defined dynamic data insertion strategy. A batch-based, shard-based writing approach can be used to write joint feature vectors generated by different camera devices and at different frame rates to different shards of the vector database, thereby achieving distributed load balancing for data insertion.

[0063] It can collect vector entry latency and entry success rate status data in real time. If the end-to-end entry latency exceeds the preset threshold or the entry success rate is lower than the preset standard, it will automatically trigger a second adjustment of the entry strategy, forming a closed-loop optimization of the entry strategy.

[0064] In some embodiments, for camera devices in key monitoring areas of the environment to be detected, a high-priority data entry channel can be set up to reserve dedicated data entry resources and sharding for the joint feature vectors of high-priority camera devices, ensuring that the data entry latency of vectors for targets in key areas is stably controlled within 10ms; at the same time, for low-frame-rate camera devices in non-key areas, a batch aggregation data entry mechanism can be set up to reduce the frequency of database write I / O and improve the utilization of storage resources.

[0065] The system outputs a structured joint feature vector dataset, which is written to and fully stored in a feature vector library in a distributed shard manner, forming a feature vector library of hundreds of millions of objects to be detected. The database entry strategy dynamically adjusts parameters and database entry status feedback data for load balancing scheduling of distributed nodes.

[0066] For a specific method of determining the inbound strategy in one embodiment of this application, please refer to [link to relevant documentation]. Figure 6 The corresponding detailed explanation.

[0067] S23, based on the joint feature vector and the topological information of the multiple camera devices, determine the index parameter of the object to be detected. The index parameter is used to determine the identity association result of the object to be detected collected by the multiple camera devices from the feature vector library.

[0068] In one embodiment of this application, the basic parameters of all camera devices in the environment to be detected are entered, the spatial straight-line distance and viewing angle between any two camera devices are pre-calculated, and a global camera device topology database is constructed to provide basic support for index parameter configuration and retrieval threshold calculation.

[0069] Two-dimensional binning rules for "spatial binning and time windows" can be preset. Specifically, spatial binning is divided into grids according to the latitude and longitude intervals of the camera device in the environment to be detected, and time windows are set at the minute level, forming several independent sub-buckets with unique spatiotemporal identifiers; B+ tree structured index parameters are determined with the spatial latitude and longitude intervals of the sub-buckets and the start and end times of the time windows as the joint primary key, and the unique mapping rules between the sub-buckets and the joint feature vectors are determined, thus completing the locking of the first-level spatiotemporal index parameters.

[0070] For each spatiotemporal sub-bucket, the secondary HNSW index parameters are configured independently, specifically including: determining the number of index levels based on the vector size within a single spatiotemporal sub-bucket; locking the number of levels to 16 when the vector size of a single bucket is ≥10 million, to 12 when it is between 1 million and 10 million, and to 8 when it is below 1 million; locking the level decay factor mL to 1 / 32, the neighborhood exploration depth during the construction phase to 200, the maximum neighborhood out-degree M of a single node to 32, and the top level to 16; and locking the L2 Euclidean distance as the similarity metric to complete the locking of the secondary visual index parameters.

[0071] The formula for calculating the preset dynamic threshold satisfies the following relationship: ; Where D is the straight-line distance between the query camera device and the camera device to be matched. This represents the maximum straight-line distance between any two cameras in the monitored environment. θ is the angle between the viewing angle of the query camera device and the camera device to be matched. This represents the maximum value of the angle between any two cameras in the monitored environment. and The preset weighted threshold is used to adjust the degree to which the spatial straight-line distance and the angle of view between the two camera devices affect the dynamic threshold; for example... The value can be 0.0001. The value can be 0.001; the frame rate difference correction value is used to indicate the degree of influence of the difference between the frame rates of the two camera devices on the dynamic threshold. For example, the frame rate difference correction value can be set to 0.02 and is triggered only when the frame rate difference between the two camera devices is greater than or equal to 10 fps. It is used to characterize that when the frame rate difference between the two camera devices is greater than or equal to 10 fps, the similarity of the images captured by the two camera devices is reduced. Simultaneously, the boundary constraint rule for the dynamic threshold can be locked as 0.7≤T≤0.9, thus completing the determination of the retrieval threshold parameter.

[0072] In one embodiment of this application, when a cross-camera device identity association retrieval is triggered, the spatiotemporal metadata of the joint feature vector of the query target is extracted. Through the B+ tree index corresponding to the first-level spatiotemporal index parameter, sub-buckets with mismatched spatiotemporal ranges are quickly filtered, and the retrieval range is narrowed to 5%-10% of the full data of the feature vector library, thus completing the rapid convergence of the retrieval range.

[0073] Extract the secondary HNSW index parameters corresponding to the target sub-bucket after convergence, call the HNSW index of the corresponding sub-bucket, and perform an approximate nearest neighbor search; at the same time, based on the camera device topology information, calculate the real-time matching threshold through the dynamic threshold calculation formula, filter out the joint feature vectors whose similarity meets the threshold requirements, and output the matching vector set.

[0074] The matching vector set is deduplicated and its spatiotemporal consistency is verified. Mismatched vectors that do not conform to the spatiotemporal movement pattern of the object to be detected are removed. The matching results of the same object to be detected collected by different camera devices are determined, and the cross-camera device identity association results of the object to be detected are finally output.

[0075] In a preferred embodiment, based on the regional division of camera devices in the environment to be detected, topological association weights are set for camera devices on key paths such as park entrances and exits and main roads. When calculating dynamic thresholds, the upper limit of the matching threshold for camera devices on adjacent paths is increased, and the lower limit of the matching threshold for camera devices in non-adjacent areas is decreased, further reducing the cross-regional false matching rate. At the same time, for high-speed moving objects to be detected, a time window sliding matching mechanism is superimposed during index retrieval to improve the identity association recall rate of fast-moving targets.

[0076] Output the full index parameters of the spatiotemporal hierarchical hybrid index, including the first-level spatiotemporal index parameters, the second-level visual HNSW index parameters, and the dynamic matching threshold parameters; the cross-camera device accurate identity association results of the object to be detected, including the joint feature vector of the matching target, the information of the camera device to which it belongs, spatiotemporal information, and feature similarity data.

[0077] For a specific method of determining the index parameter in one embodiment of this application, please refer to [link to relevant documentation]. Figure 7For detailed instructions on how to determine the identity association result, please refer to [link to relevant documentation]. Figure 8 The corresponding detailed explanation.

[0078] S24. Based on the coordinate system transformation rules corresponding to the multiple camera devices, the identity association results and the real-time location information of the object to be detected collected by the multiple camera devices are fused to obtain the trajectory data of the object to be detected moving in the environment to be detected.

[0079] In one embodiment of this application, based on the global world coordinate system of the electronic map of the environment to be detected, a corresponding homography matrix is ​​calibrated for each camera device. The homography matrix is ​​used to describe the projection mapping relationship between the image coordinate system of the camera device and the global world coordinate system. A standardized coordinate system transformation rule from image coordinates to global world coordinates is constructed for each camera device, so as to achieve global uniformity of the coordinate system of all camera devices.

[0080] The homography matrix can be calibrated using the direct linear transformation (DLT) method, the Zhang Zhengyou calibration method, or a deep learning-based calibration method. This application does not limit the calibration method of the homography matrix.

[0081] For the real-time location information of the object to be detected collected by each camera device in the identity association results, the coordinates of the center point of the detection box of the object to be detected in the image coordinate system are extracted. The coordinate transformation is performed by the homography matrix of the corresponding camera device to convert the position information in the image coordinate system into the standardized position coordinates in the global world coordinate system. At the same time, the corresponding collection timestamp is bound to generate the spatiotemporal location point data of the object to be detected.

[0082] Based on the order of the collected timestamps, the spatiotemporal location data corresponding to different camera devices are sorted in a temporal sequence to complete the global temporal alignment of discrete location points and form a temporal location point sequence of the object to be detected.

[0083] Based on the temporal location point sequence, combined with the topological information of the camera device and the motion characteristics of the object to be detected, discrete spatiotemporal location points are stitched together into a continuous movement trajectory. At the same time, through preset anomaly point removal rules, mismatched location points in the trajectory that do not conform to the motion logic of the object to be detected are identified and removed, including spatiotemporal jump points and coordinate anomalies, thereby improving the continuity and accuracy of the trajectory.

[0084] The stitched trajectory is smoothed and optimized using the Kalman filter algorithm to correct noise errors in the position coordinates, fill in the blind spot trajectory segments between adjacent camera devices, and finally generate continuous trajectory data of the object to be detected moving in the environment to be detected.

[0085] In a preferred embodiment, for the scenario of simultaneous multi-target tracking, a unique global identity is assigned to the trajectory of each object to be detected, so as to realize parallel processing and independent management of multi-target trajectories; at the same time, for the scenario where the object to be detected enters the blind area of the field of view of the camera device, combined with historical motion features and topology information of the camera device, trajectory prediction is performed through Kalman filtering to complete the blind area trajectory segment and ensure the continuity of the trajectory.

[0086] Output continuous moving trajectory data of the object to be detected in the global coordinate system of the environment to be detected, which includes the full-time spatiotemporal position sequence of the trajectory, associated camera device information, collection timestamp, and trajectory feature data.

[0087] In an embodiment of the present application, for the specific method of determining trajectory data, please refer to Figure 9 the corresponding detailed description.

[0088] S25, output the trajectory data.

[0089] In an embodiment of the present application, the trajectory data is standardized and encapsulated according to preset format rules, and extended fields such as the category of the object to be detected, global identity, associated alarm information, and information of the collecting camera device are supplemented to generate a structured trajectory data set that conforms to the security monitoring data standard.

[0090] The encapsulated trajectory data is pushed to the security data large screen and management terminal in the environment to be detected, so as to realize the visual display of the real-time position, continuous moving trajectory and associated camera device information of the object to be detected, and support the functions of historical trajectory tracing and simultaneous multi-target tracking at the same time.

[0091] The trajectory data is pushed to third-party business systems such as the security management platform, access control system, and alarm system in the environment to be detected through a standardized interface, so as to realize the linkage response of the business systems; based on the trajectory data, the preset alarm rule engine triggers real-time alarms for abnormal behaviors of the object to be detected, such as entering a dangerous area, moving in violation of regulations, and gathering, and synchronously pushes the alarm information and corresponding trajectory data to the security management terminal; the trajectory data is written into the Redis cache database in real time for high-frequency real-time query and historical trajectory tracing; at the same time, the full amount of trajectory data is persistently stored in the time-series database to meet the requirements of data retention and retrospective audit; the output status, push success rate, and visual rendering delay of the trajectory data are monitored in real time to ensure that the end-to-end delay of the large screen visualization is ≤100ms, the push success rate of the business system reaches 100%, and the end-to-end business process closed loop is completed.

[0092] In a preferred embodiment, a dedicated output channel for the trajectory of key targets is set up for emergency response scenarios. The trajectory data of blacklisted targets and alarm targets are pushed with high priority and real-time pop-up alarms are displayed. At the same time, surrounding camera devices are linked to perform real-time tracking to improve the efficiency of emergency response. Meanwhile, the standardized format export of trajectory data is supported to meet the business needs of event backtracking and evidence preservation.

[0093] It outputs standardized and encapsulated structured trajectory datasets for visualization, business system linkage, data storage and backtracking; it can also provide real-time alarm information and linkage control commands based on abnormal behaviors triggered by trajectory data.

[0094] As can be seen from the above technical solutions, the embodiments of this application adopt a distributed parallel processing architecture, dynamically creating independent parallel processing task units for the video stream of each camera device, distributing high-load computing to multiple edge nodes for parallel execution, and realizing distributed deployment and storage-computation separation of computing resources. Through a dual normalization mechanism combining batch normalization and layer normalization, feature distribution bias caused by cross-camera device heterogeneity is eliminated; simultaneously, through a multimodal joint feature vector construction mechanism, standardized visual features and spatiotemporal auxiliary vectors are deeply integrated, improving the accuracy of target identity association across camera devices. A spatiotemporal hierarchical hybrid index system is constructed. First, a first-level spatiotemporal structured index quickly converges the retrieval scope and filters invalid spatiotemporal data. Then, a second-level customized visual index performs accurate similarity matching of high-dimensional vectors, balancing retrieval efficiency and matching accuracy. By using a pre-calibrated homography matrix, a standardized transformation rule was constructed between the image coordinate system of all camera devices and the global world coordinate system. This achieved global coordinate unification of all target location information and transformed discrete cross-camera device identity association results into standardized spatiotemporal location points with temporal attributes in the global coordinate system. This enabled continuous trajectory tracking of the detected object across regions within the monitoring area, improving the accuracy of locating and tracking cross-regional objects.

[0095] like Figure 3 The diagram shown is a flowchart of a method for determining a spatiotemporal auxiliary vector according to an embodiment of this application. The order of steps in this flowchart can be changed, and some steps can be omitted, depending on different requirements. The method for determining a spatiotemporal auxiliary vector provided in this embodiment includes the following steps.

[0096] S30, synchronously acquire the video data from the multiple camera devices based on a preset communication protocol.

[0097] In one embodiment of this application, based on the device type and access configuration parameters of the plurality of camera devices, a corresponding preset communication protocol is automatically matched to establish a two-way trusted communication connection with each camera device; for camera devices with incompatible proprietary protocols, protocol standardization and adaptation are completed through a protocol conversion gateway to achieve seamless access for all camera devices.

[0098] Clock calibration is performed with the global clock server of the environment under test via the NTP network time protocol, a unified global time reference is assigned to each camera device, and the deviation between the local clock of each camera device and the global clock is controlled within ±1ms, eliminating clock offset between multiple camera devices.

[0099] Based on the established communication connection, and according to the preset bitstream configuration parameters, real-time video streams in H.264 / H.265 compression format are synchronously pulled from the multiple camera devices. A unique device identifier ID is assigned to each video stream, and the global synchronization timestamp and frame sequence number of the corresponding camera device are bound to each frame of the video stream, thereby achieving frame-level timing alignment of multiple video streams.

[0100] The synchronously fetched multi-channel video streams are written into a circular buffer queue with the corresponding device identifier. A buffer window of preset length is set. Out-of-order video frames are reordered, and missing keyframes are filled in. At the same time, the encoding format and frame rate of the video stream are standardized and verified. Abnormal frame data that does not conform to the preset standard is removed, and video stream data with stable timing and standard format is output.

[0101] S31, perform structured processing on the IoT data to obtain structured IoT data; the structured IoT data includes the three-dimensional coordinates of the multiple camera devices in the environment to be detected, the timestamps corresponding to the video frames of the video data, the light intensity of the environment to be detected, the access control status of the environment to be detected, and the latitude and longitude coordinates of the object to be detected.

[0102] In one embodiment of this application, raw IoT data from various IoT devices are accessed via MQTT, Modbus, and HTTP protocols respectively. Protocol parsing and format restoration are performed on the raw data in different formats to extract the original values ​​of each data field and the corresponding acquisition device identifier and local acquisition timestamp, thereby completing the unified aggregation of multi-source heterogeneous data.

[0103] Based on preset data cleaning rules, outlier removal and missing value completion are performed on the parsed raw data: outlier data that exceeds the preset reasonable value range and redundant data collected repeatedly are removed; missing data with continuous time sequence is completed by linear interpolation; and missing discrete state data is filled by the previous valid state value to ensure the integrity and continuity of the data.

[0104] According to preset field mapping rules, the cleaned raw data is mapped to fixed standardized fields. Numerical normalization processing is performed on each field, specifically including: the three-dimensional coordinates of the multiple cameras in the environment to be detected are uniformly mapped to longitude, latitude, and altitude in the WGS84 geodetic coordinate system, normalized to 64-bit double-precision floating-point data, with precision controlled within 0.01 meters; the timestamps corresponding to the video frames of the video data are uniformly mapped to 13-bit millisecond-level Unix global synchronization timestamps, perfectly aligned with the video frame timestamps, with time deviation controlled within ±1ms; the light intensity of the environment to be detected is uniformly mapped to numerical data measured in lux, normalized to 32-bit floating-point data, with a numerical range of... The range is limited to 0~200000 lux; the access control status of the environment to be detected is uniformly mapped to a standardized Boolean status value, where 1 represents the access control being triggered to open and 0 represents the access control being closed, and is simultaneously bound to the deployment location coordinates and status change timestamp of the corresponding access control device; the latitude and longitude coordinates of the object to be detected are uniformly mapped to two-dimensional longitude and latitude data in the WGS84 geodetic coordinate system, standardized to 64-bit double-precision floating-point data, and strongly bound to the timestamp of the video frame where the object to be detected is located; the standardized field data are strongly bound to the corresponding video frame data according to the global synchronization timestamp and the camera device identification ID, ensuring that the video data corresponding to the same timestamp and the same camera device is completely spatiotemporally matched with the structured IoT data.

[0105] Perform integrity checks on the structured IoT data that has been spatiotemporally bound to ensure that all required fields are complete, the numerical ranges conform to preset specifications, and the spatiotemporal reference alignment errors are within the allowable range. Remove invalid data that fails the check, and finally output standardized structured IoT data.

[0106] S32, the video data and the structured IoT data are encoded using a preset encoding model to obtain the spatiotemporal auxiliary vector.

[0107] In one embodiment of this application, five core standardized fields of structured IoT data are extracted, all fields are converted into continuous numerical features, and dimension-by-dimensional splicing is performed according to a preset feature order to generate a fixed-length model input feature vector, ensuring that the dimension of the input features is completely matched with the dimension of the input layer of the encoding model; at the same time, the temporal context information of the corresponding video frame is used as an auxiliary feature and integrated into the input feature vector to strengthen the temporal correlation of the features. Load the system's pre-set multilayer perceptron coding model and lock the model's structure and inference parameters: the input layer dimension is 16-dimensional, perfectly matching the dimension of the input feature vector; the hidden layer dimension is 32-dimensional, using the ReLU non-linear activation function; the output layer dimension is 64-dimensional, using the linear activation function; the model optimizer is the Adam optimizer, with a preset learning rate of 0.001; complete the initialization of the model's inference environment and the loading of weights to ensure that the model is in a reasoning-ready state.

[0108] The constructed input feature vector is input into the initialized multilayer perceptron coding model. Feature mapping is completed through the input layer, nonlinear feature transformation and high-dimensional feature extraction are completed through the hidden layer, and deep correlations between spatiotemporal, environmental and state features are explored. Dimension constraints and standardized output are completed through the output layer to generate the initial feature encoding result.

[0109] The generated initial feature encoding results are normalized, and the vector magnitude is strictly constrained to 1 to eliminate the impact of feature numerical scale differences on subsequent feature fusion and similarity retrieval, ensure the consistency of vector space measurement, and generate standardized feature vectors.

[0110] The standardized feature vectors are subjected to dimension verification, numerical range verification, and integrity verification to remove invalid vectors with abnormal dimensions, numerical overflow, or missing features. For vectors that pass the verification, the corresponding camera device's device identifier ID and the video frame's global synchronization timestamp are bound to them to finally generate the spatiotemporal auxiliary vector.

[0111] like Figure 4 The diagram shown is a flowchart of a method for determining visual features according to an embodiment of this application. The order of the steps in this flowchart can be changed, and some steps can be omitted, depending on different requirements. The method for determining visual features provided in this embodiment includes the following steps.

[0112] S40, decode the video data to obtain the original image in the video data.

[0113] In one embodiment of this application, a decoding frame rate adaptive adjustment mechanism is set up to meet the real-time processing needs of low-computing-power nodes at the edge. When the node's computing power load exceeds a preset threshold, non-critical frames are automatically skipped for decoding, reducing node computing power consumption while ensuring the continuity of target detection. For video streams in low-light scenarios such as nighttime and backlight, image enhancement preprocessing is automatically performed after decoding, including contrast adjustment, noise suppression, and backlight compensation, to improve the image restoration quality in low-light scenarios. For parallel decoding scenarios of multiple video streams, a multi-threaded parallel decoding architecture is adopted, allocating an independent decoding thread to each video stream to ensure that the single-frame decoding latency is stably controlled within 5ms when decoding thousands of video streams in parallel.

[0114] Output the original RGB format image after decoding, verification and standardization. Each original image is bound to a unique camera device ID, global synchronization timestamp and frame number.

[0115] S41, Based on the original image, using a preset target detection model, determine the target detection box corresponding to the object to be detected in the original image.

[0116] In one embodiment of this application, for scenarios involving frequent target occlusion and dense arrangement in security monitoring, an occlusion perception optimization mechanism is introduced during the model inference stage. Feature enhancement is performed on partially occluded targets to improve the detection recall rate of occluded targets. For key monitoring areas of the environment to be detected, region of interest filtering rules are set to perform target detection inference only on preset key monitoring areas, ignoring invalid detection results in non-monitored areas, reducing the computational power consumption of model inference and improving detection efficiency. For low-computing-power deployment scenarios at the edge, model quantization and pruning lightweight processing are performed on the target detection model. While ensuring that the detection accuracy loss does not exceed 2%, the model inference latency is reduced by more than 50%, adapting to the real-time processing requirements of the edge.

[0117] Output the standardized target detection box corresponding to the object to be detected in the original image. Each detection box is bound to a unique detection box identifier ID, target category, classification confidence score, absolute pixel coordinates of the original image, corresponding camera device identifier ID, and global synchronization timestamp.

[0118] S42, the image in the target detection box is determined as the target image.

[0119] In one embodiment of this application, to address the issue of target edge truncation caused by target detection box positioning deviation, an edge expansion and cropping mechanism is set up. Based on the original detection box coordinates, the pixel boundary is expanded to the surrounding area by a preset proportion before cropping, ensuring that the edge features of the object to be detected are completely preserved and avoiding feature loss. For target images with severe background interference, instance segmentation mask processing is introduced. The main body mask of the object to be detected is extracted through a lightweight segmentation model, and the pixel values ​​of the background area are set to zero, further reducing the impact of background interference on subsequent feature extraction. For batch cropping scenarios, a multi-threaded parallel cropping architecture is adopted to achieve synchronous cropping processing of multiple targets and multiple frames, ensuring that the cropping processing latency of a single target image is controlled within 1ms.

[0120] Output the target image after cropping, standardization, and quality verification. The fixed size is 256×128. Bind a unique corresponding detection box ID, target category, camera device ID, and global synchronization timestamp.

[0121] S43, Based on the target image, determine the visual features of the object to be detected using a preset feature extraction model.

[0122] In one embodiment of this application, to address the variability issues of pedestrians being detected due to pose changes, occlusion, and clothing changes, during the model training phase, publicly available pedestrian re-identification datasets and pedestrian samples collected from different camera devices in the actual detection environment are incorporated to perform scenario-based incremental fine-tuning of the model, significantly improving the scenario adaptability and robustness of pedestrian visual features. For edge deployment scenarios with low computing power, the feature extraction model is lightweighted and replaced with a lightweight backbone network, reducing model computing power consumption and inference latency while ensuring feature discrimination. To meet the adaptation requirements of multiple target categories, dedicated feature extraction branches are trained for pedestrians, vehicles, and non-motorized vehicles, and customized optimizations are performed on the feature characteristics of different target categories to further improve the discrimination of visual features of each target category.

[0123] The output is a fixed-dimensional visual feature vector after deep extraction, cross-scene normalization, and L2 standardization. It is bound to the corresponding unique target identifier, camera device ID, and global synchronization timestamp. It can be directly fused with spatiotemporal auxiliary vectors to generate a joint feature vector for subsequent cross-camera device target identity association retrieval.

[0124] like Figure 5 The diagram shown is a flowchart of a method for determining a joint feature vector according to an embodiment of this application. The order of steps in this flowchart can be changed, and some steps can be omitted, depending on different requirements. The method for determining a joint feature vector provided in this embodiment includes the following steps.

[0125] S50, the visual features are normalized to obtain normalized visual features.

[0126] In one embodiment of this application, the original visual feature vector of the object to be detected, output by a preset feature extraction model, is used as the sole input. A dual normalization processing mechanism combining batch normalization and layer normalization is adopted. First, batch normalization is used to align the numerical distribution of the same target features collected by different camera devices from the batch dimension, eliminating the feature distribution deviation of different samples and different camera devices during batch processing. Then, layer normalization is used to constrain the numerical fluctuation range of the feature vector from the single sample dimension, and perform dimension-by-dimensional standardization alignment processing on the original visual feature vector to eliminate the feature distribution offset caused by heterogeneity across camera device viewpoints, illumination, resolution, and frame rate. Finally, the output dimension is consistent with the input original visual features and the feature distribution meets the preset standardization requirements.

[0127] S51, the normalized visual features are subjected to dimensionality reduction processing to obtain cross-scene visual features.

[0128] In one embodiment of this application, normalized visual features are used as input, and a fully connected layer dimensionality reduction network with Triplet Loss constraints, which has been pre-trained for cross-camera device re-identification scenes, is loaded. The normalized visual features are input into the fully connected layer network. During the dimensionality reduction process, the convergence of triplet loss constraints during the pre-training stage is used to further enhance the aggregation degree of features of the same type of target across camera devices and the distinguishability of features of different types of targets. Under the premise of completely preserving the core identity representation information of the target, the precise compression of feature dimensions and cross-scene feature alignment are completed, eliminating the subsequent computational loss caused by redundant feature dimensions. Finally, a fixed-dimensional cross-scene visual feature with strong cross-scene robustness and high identity distinguishability is output.

[0129] S52, the cross-scene visual features and the spatiotemporal auxiliary vector are concatenated to obtain the first fused feature.

[0130] In one embodiment of this application, fixed-dimensional cross-scene visual features and fixed-dimensional spatiotemporal auxiliary vectors generated in the preceding steps and synchronized with the visual features are used as dual inputs. A dimension-by-dimensional alignment and splicing processing method is adopted. According to the preset feature splicing order, the cross-scene visual features and spatiotemporal auxiliary vectors are spliced ​​one by one from the beginning to the end, so that the target identity representation information contained in the visual features and the target spatiotemporal location and environmental constraint information contained in the spatiotemporal auxiliary vectors are initially fused. During the splicing process, the dimensional integrity of the two feature vectors is strictly maintained, and the numerical distribution of the single feature is not changed. The final output dimension is the first fused feature, which is the sum of the dimensions of the two input features.

[0131] S53, normalize the first fused feature to obtain the normalized fused feature.

[0132] In one embodiment of this application, the first fused feature is used as input, and the L2 norm normalization method is adopted to perform full-dimensional normalization calculation on the first fused feature. The magnitude of the fused feature vector is strictly constrained to 1, eliminating the numerical scale difference between the two different feature domains of cross-scene visual features and spatiotemporal auxiliary vectors, ensuring the metric consistency of the fused feature vector in the high-dimensional vector space, and avoiding matching deviation caused by the difference in feature numerical scale during subsequent similarity retrieval. Finally, the normalized fused feature with a magnitude of 1 and a dimension completely consistent with the first fused feature is output.

[0133] S54, the identifiers of the multiple camera devices, the timestamps for acquiring the visual features, the three-dimensional coordinates of the camera devices in the preset coordinate system, and the identifier of the object to be detected are encoded to obtain metadata encoding.

[0134] In one embodiment of this application, the inputs are: a unique device identifier of the camera device spatiotemporally bound to the visual features; a globally synchronized millisecond-level timestamp of the video frame corresponding to the visual features; the three-dimensional coordinates of the camera device in the WGS84 preset geodetic coordinate system (longitude, latitude, and altitude); and a unique tracking identifier of the object to be detected. According to preset metadata standardization encoding rules, each input field is encoded with a fixed length of binary digitization. A preset encoding bit width and a fixed encoding position are assigned to each field to generate continuous, fixed-length standardized binary encoding fields. At the same time, the integrity and validity of the encoded fields are checked, and invalid codes with encoding errors or missing fields are eliminated. Finally, metadata encoding strongly bound to the corresponding visual features is output.

[0135] S55. Based on the metadata encoding and the normalized fusion features, key-value pairs are generated to obtain a joint feature vector.

[0136] In one embodiment of this application, normalized fusion features and metadata encoding spatiotemporally bound to the fusion features are used as inputs. A preset key-value pair embedding mechanism is adopted, using the metadata encoding as the key and the normalized fusion features as the value, to construct a one-to-one key-value pair structure. The metadata encoding is standardized and embedded into the header extension field of the normalized fusion features, so that the metadata encoding and the normalized fusion features form an inseparable unified structure. Then, a preset lightweight convolutional neural network is used to perform feature integrity and encoding validity verification on the structure, and invalid structures are eliminated. Finally, a cross-camera device-specific joint feature vector with metadata embedding field is generated, which can be directly used for vector database storage and similarity retrieval.

[0137] like Figure 6 The diagram shown is a flowchart of a method for determining an inbound strategy according to an embodiment of this application. The order of steps in this flowchart can be changed, and some steps can be omitted, depending on different requirements. The method for determining an inbound strategy provided in this embodiment includes the following steps.

[0138] S60, obtain the status parameters of the plurality of camera devices, the status parameters including the frame rate of the video data acquired by the plurality of camera devices, the cache queue length corresponding to the joint feature vector to be entered into the database, and the processor utilization rate.

[0139] In one embodiment of this application, a real-time status monitoring module with a preset sampling period of 10ms is configured for each distributed parallel processing node corresponding to a camera device. The module synchronously collects the real-time operating status parameters of multiple camera devices connected to the node, including the real-time processing frame rate of the video stream of each camera device, the real-time length of the cache queue corresponding to the joint feature vector to be added to the database within the node, and the real-time CPU and memory utilization of the node. At the same time, the collected status parameters are processed by moving average smoothing and outlier filtering to remove invalid sampling data with sudden fluctuations. Finally, standardized status parameters matching the real-time operating status of the node are output, providing basic input data for the calculation of subsequent database entry frequency and writing batches.

[0140] S61, based on the frame rate and the number of joint feature vectors, determine the entry frequency of the joint feature vectors into the database.

[0141] In one embodiment of this application, the system first calls the preset data frame rate of each camera device and the number of joint feature vectors generated by a single frame of video data as input. Specifically, the preset data entry benchmark threshold mapping rule is called, which is bound to the frame rate. The mapping rule is that 15fps corresponds to a benchmark data entry frequency of 200ms, 20fps corresponds to a benchmark data entry frequency of 150ms, 25fps corresponds to a benchmark data entry frequency of 100ms, and 30fps corresponds to a benchmark data entry frequency of 50ms. Then, the corresponding benchmark data entry frequency is matched according to the main frame rate of the multi-channel video streams processed by the current node. At the same time, the benchmark data entry frequency is linearly corrected by the number of joint feature vectors generated by a single frame of video data. When the number of vectors generated by a single frame exceeds the preset number threshold, the data entry frequency is shortened to reduce the amount of data entered in a single time. Finally, the initial data entry frequency of the joint feature vectors that is adapted to the input characteristics of the current video stream is determined.

[0142] S62, update the inbound frequency based on the processor utilization rate to obtain the updated inbound frequency.

[0143] In one embodiment of this application, the initial inbound frequency and the real-time utilization rate of the node processor are taken as inputs. The system calls the preset processor utilization rate graded load threshold rules. When the real-time utilization rate of the processor is higher than the high load threshold of 70%, the inbound interval is extended by 20%-30% based on the initial inbound frequency to reduce the inbound frequency and alleviate the computing power pressure on the node. When the real-time utilization rate of the processor is lower than the low load threshold of 30%, the inbound interval is shortened by 10%-20% based on the initial inbound frequency to increase the inbound frequency and improve the inbound efficiency. When the real-time utilization rate of the processor is in the normal load range of 30%-70%, the initial inbound frequency is kept unchanged. At the same time, preset upper and lower limit boundary constraints are set on the corrected inbound frequency to avoid the inbound frequency from exceeding the reasonable operating range. Finally, the updated inbound frequency adapted to the real-time computing power load of the node is obtained.

[0144] S63, based on the updated inbound frequency and the cache queue length, determine the write batch of the joint feature vector to obtain the inbound strategy of the joint feature vector; the inbound strategy indicates the strategy of writing the joint feature vector data generated by video data from different camera devices and different frame rates into the corresponding fragment of the feature vector library.

[0145] In one embodiment of this application, the updated inbound frequency and the length of the cache queue corresponding to the joint feature vector to be inbound are used as inputs. First, the inbound time window is determined based on the updated inbound frequency. Then, combined with the real-time length of the cache queue, a linear adjustment is made based on the preset baseline batch threshold. When the cache queue length exceeds 3 times the baseline batch size, the single write batch size is increased accordingly. When the cache queue length is less than 0.5 times the baseline batch size, the single write batch size is decreased accordingly. At the same time, for the joint feature vectors generated by video data from different camera devices and different frame rates, the corresponding independent shards in the feature vector library are allocated. The execution rules for asynchronous writing in batches and shards are determined. Finally, a complete inbound strategy is formed, which includes a dynamic inbound time window, adaptive write batches, and shard mapping rules. The inbound strategy is used to guide the distributed processing nodes to write the joint feature vectors into the corresponding shards of the feature vector library efficiently and stably.

[0146] like Figure 7 The diagram shown is a flowchart of a method for determining index parameters according to an embodiment of this application. The order of steps in this flowchart can be changed, and some steps can be omitted, depending on different requirements. The method for determining index parameters provided in this embodiment includes the following steps.

[0147] S70, the three-dimensional coordinates of the camera device corresponding to the joint feature vector and the timestamp of obtaining the joint feature vector are used as index fields to obtain the spatiotemporal index of the object to be detected.

[0148] In one embodiment of this application, the core inputs are the three-dimensional coordinates of the corresponding camera device in the preset WGS84 geodetic coordinate system, the global synchronization millisecond-level timestamp corresponding to the joint feature vector whose header has been embedded and stored in the database, and the system's preset monitoring area spatial gridding bucketing rules and minute-level time window bucketing rules. First, the extracted three-dimensional coordinates of the camera device and the acquisition timestamp are determined as unique index key fields. Then, according to the spatial bucketing and time window rules, each joint feature vector is uniquely assigned to the corresponding spatiotemporal sub-bucket. A B+ tree structured index is constructed using the spatial latitude and longitude interval of the spatiotemporal sub-bucket and the start and end times of the time window as the joint primary key. At the same time, a global index directory covering all buckets is generated, and a unique mapping relationship is established between the index key fields and the storage address of the corresponding joint feature vector and the spatiotemporal sub-bucket to which it belongs. Finally, a spatiotemporal index of the object to be detected with the ability to quickly filter in a spatiotemporal range is obtained.

[0149] S71, normalize the joint feature vector to obtain the visual index of the object to be detected.

[0150] In one embodiment of this application, the fused feature subject, which has undergone L2 normalization in the joint feature vector, is taken as input. First, a second L2 normalization compliance check is performed on the fused feature. Features with vector magnitude deviation exceeding ±0.001 are re-normalized to ensure the consistency of measurement of all feature vectors. Then, for the effective feature vectors in each spatiotemporal sub-bucket, the Hierarchical Navigation Small World (HNSW) algorithm, which is customized and optimized for security cross-camera device scenarios, is adopted. Using L2 Euclidean distance as the similarity metric, the index level, neighborhood exploration depth, and maximum neighborhood out-degree hyperparameters that are adapted to the size of the bucket vector are locked. The construction of a multi-layer directed graph structure and the hierarchical insertion of nodes are performed to build an independent high-dimensional vector retrieval index for each spatiotemporal sub-bucket. Finally, a visual index of the object to be detected that can achieve millisecond-level approximate nearest neighbor search is obtained.

[0151] S72, the spatiotemporal index and the visual index are determined as the index parameters of the object to be detected.

[0152] In one embodiment of this application, using spatiotemporal index and visual index as input, a one-to-one strong binding mapping relationship is first established between each spatiotemporal sub-bucket in the spatiotemporal index and the independent visual index corresponding to that bucket. The storage address, structural parameters, and vector size metadata of the visual index are embedded into the extended fields of the corresponding sub-bucket of the spatiotemporal index, thus constructing a two-level collaborative retrieval mechanism of "first-level spatiotemporal index coarse filtering and second-level visual index fine matching". Then, a full verification is performed on the mapping consistency, data integrity, and retrieval link connectivity of the two-level indexes, and invalid index nodes with mapping failures are eliminated. Finally, the complete dataset containing all structural parameters of the spatiotemporal index and all configuration parameters of the visual index that has passed the verification is determined as the index parameters of the object to be detected. The index parameters are used to retrieve the joint feature vectors of the same object to be detected that match the feature vectors collected across camera devices from the feature vector library, and output the identity association result.

[0153] like Figure 8 The diagram shown is a flowchart of a method for determining identity association results according to an embodiment of this application. The order of steps in this flowchart can be changed, and some steps can be omitted, depending on different requirements. The method for determining identity association results provided in this embodiment includes the following steps.

[0154] S80, based on the spatiotemporal index, determine the preliminary association results of the object to be detected collected by the multiple camera devices from the feature vector library.

[0155] In one embodiment of this application, the spatiotemporal metadata embedded in the header of the joint feature vector of the target object that triggers cross-camera device identity association retrieval, the pre-built global spatiotemporal index, and the feature vector library storing the full set of joint feature vectors are taken as input. First, the three-dimensional coordinates of the camera device and the acquisition timestamp corresponding to the query joint feature vector are extracted to determine the effective spatial range and time window of the target retrieval. Then, the matching spatiotemporal sub-buckets are quickly located through the global B+ tree index directory of the spatiotemporal index. More than 90% of invalid vector data with mismatched spatiotemporal ranges are filtered out, and the retrieval range is converged to the set of joint feature vectors in the target spatiotemporal sub-bucket. After performing basic validity checks on the vectors in the set, the preliminary association results of the target object that are collected by the multiple camera devices and match the spatiotemporal attributes of the query target are finally output.

[0156] S81, based on the topology information of the plurality of camera devices, determine the distance and viewing angle between any two camera devices.

[0157] In one embodiment of this application, the system takes as input a topology information database of multiple camera devices in the environment to be detected, which is pre-constructed during the system initialization phase. The topology information database contains the core basic parameters of all camera devices in the preset WGS84 geodetic coordinate system, including their three-dimensional coordinates, installation orientation, lens focal length, and field of view. First, the basic parameters of all camera devices to be matched corresponding to the target camera device and the preliminary association results are extracted. Then, the spatial straight-line distance between two camera devices is calculated using the geodetic coordinate system spatial distance calculation formula. Finally, based on the installation orientation and field of view parameters of the two camera devices, the angle between their optical axes is calculated. Finally, the values ​​of the spatial straight-line distance and angle between any two camera devices are output.

[0158] S82, based on the distance and the angle of view, determine the dynamic threshold between any two camera devices.

[0159] In one embodiment of this application, the calculation formula for the preset dynamic threshold satisfies the following relationship: ; Where D is the straight-line distance between the query camera device and the camera device to be matched. This represents the maximum straight-line distance between any two cameras in the monitored environment. θ is the angle between the viewing angle of the query camera device and the camera device to be matched. This represents the maximum value of the angle between any two cameras in the monitored environment. and The preset weighted threshold is used to adjust the degree to which the spatial straight-line distance and the angle of view between the two camera devices affect the dynamic threshold; for example... The value can be 0.0001. The value can be 0.001; the frame rate difference correction value is used to indicate the degree of influence of the difference between the frame rates of the two camera devices on the dynamic threshold. For example, the frame rate difference correction value can be set to 0.02 and is triggered only when the frame rate difference between the two camera devices is greater than or equal to 10 fps. It is used to characterize that when the frame rate difference between the two camera devices is greater than or equal to 10 fps, the similarity of the images captured by the two camera devices is reduced. Simultaneously, the boundary constraint rule for the dynamic threshold can be locked as 0.7≤T≤0.9, thus completing the determination of the retrieval threshold parameter. Specifically, when the frame rate difference between the two camera devices is ≥10fps, the frame rate difference correction value is 0.02; otherwise, it is 0. At the same time, a preset boundary constraint is applied to the calculated threshold T, strictly limiting the value range of T to the interval 0.7≤T≤0.9. Finally, a dynamic matching threshold that adapts to the spatial topology and frame rate characteristics of the two camera devices is output.

[0160] S83, determine the similarity between the preliminary association result and the visual index.

[0161] In one embodiment of this application, the joint feature vector corresponding to the preliminary association result of the object to be detected, the pre-constructed visual index, and the joint feature vector of the query target are used as inputs. First, the visual index strongly bound to the spatiotemporal sub-bucket to which the preliminary association result belongs is called. The normalized fusion feature subject in the joint feature vector of the query target is used as the retrieval vector. Approximate nearest neighbor search is performed through the HNSW multi-layer directed graph structure of the visual index. The similarity value between each joint feature vector in the preliminary association result and the query vector is calculated using L2 Euclidean distance as the similarity metric. The similarity value is linearly negatively correlated with the L2 Euclidean distance. Finally, the similarity between each vector in the preliminary association result and the query target is output.

[0162] S84, if the similarity is greater than or equal to the dynamic threshold, the preliminary association result is determined as the identity association result.

[0163] In one embodiment of this application, using similarity values, dynamic thresholds, and spatiotemporal metadata corresponding to preliminary association results as input, the similarity of each preliminary association result is first compared with the dynamic threshold between the corresponding camera devices one by one. Matching results with similarity greater than or equal to the dynamic threshold are filtered out. Then, spatiotemporal consistency verification is performed on the filtered matching results to eliminate erroneous matching results that do not conform to the spatiotemporal movement pattern of the object to be detected. The matching results that pass the verification are bound with the unique identifier of the corresponding camera device, the collection timestamp, and the target location information. Finally, the accurate identity association result of the object to be detected across multiple camera devices is determined.

[0164] like Figure 9 The diagram shown is a flowchart of a method for determining trajectory data according to an embodiment of this application. The order of steps in this flowchart can be changed, and some steps can be omitted, depending on different requirements. The method for determining trajectory data provided in this embodiment includes the following steps.

[0165] S90, based on a preset homography matrix, determine the transformation rules of the coordinate systems corresponding to any two camera devices, and based on the transformation rules, stitch the identity association results into the historical movement trajectory of the object to be detected; wherein, the homography matrix is ​​used to indicate the coordinate transformation relationship of different camera devices.

[0166] In one embodiment of this application, the system initialization phase uses a pre-calibrated homography matrix for each camera device based on a global world coordinate system of the environment to be detected, the accurate identity association results of the object to be detected across multiple camera devices output from the previous steps, and the intrinsic and extrinsic parameter calibration data of each camera device as input. First, based on the pre-calibrated homography matrix corresponding to each camera device, which is used to characterize the projection mapping relationship between the image coordinate system of the corresponding camera device and the global world coordinate system, the coordinate mapping relationship between the image coordinate systems of any two camera devices and the standardized transformation rules for transforming the image coordinate system of each camera device to the global world coordinate system are derived and determined. Then, the objects to be detected collected by each camera device in the identity association results are extracted. The corresponding detection frame center point coordinates, global synchronous acquisition timestamps, and unique identifiers of the respective camera devices are used to convert all target position coordinates scattered in different camera device image coordinate systems into standardized spatiotemporal position points in the global world coordinate system through corresponding standardized transformation rules. Then, all spatiotemporal position points are sorted and globally aligned according to the order of global synchronous acquisition timestamps. Combining the topological information of the camera devices in the environment to be detected and the target motion characteristics, the continuity of the time-aligned position points is checked, and spatiotemporal jump anomalies that do not conform to the motion logic of the object to be detected are eliminated. Finally, the discrete standardized spatiotemporal position points are stitched together to form the continuous historical movement trajectory of the object to be detected in the global world coordinate system of the environment to be detected.

[0167] S91, the real-time location information of the object to be detected is spliced ​​with the historical movement trajectory to obtain the trajectory data of the object to be detected moving in the environment to be detected.

[0168] In one embodiment of this application, the following are inputs: the continuous historical movement trajectory of the object to be detected, the real-time detection frame position information of the object to be detected collected in real time by multiple camera devices, the pre-calibrated coordinate system transformation rules, and the preset Kalman filter trajectory smoothing optimization parameters. First, the real-time position information of the object to be detected is subjected to standardization preprocessing. Through the pre-calibrated homography matrix and coordinate system transformation rules, the real-time position information is converted from the image coordinate system of the corresponding camera device to a standardized real-time spatiotemporal position point in the global world coordinate system that is of the same origin as the historical movement trajectory. The corresponding global synchronization millisecond-level timestamp is bound to it. Then, according to the chronological order of the timestamps, the standardized real-time spatiotemporal position point is seamlessly spliced ​​and fused with the end time sequence node of the historical movement trajectory. At the same time, the preset Kalman filter algorithm is used to perform smoothing optimization processing on the spliced ​​full trajectory sequence to correct the acquisition noise error of the position coordinates and fill in the trajectory segments in the blind spots of the field of view between adjacent camera devices. Finally, the spatiotemporal consistency and continuity verification of the fused trajectory is performed, and the complete continuous movement trajectory data of the object to be detected in the global coordinate system of the detection environment, covering the historical movement process and the real-time position, is output.

[0169] Please see Figure 10 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application. The electronic device 100 includes a memory 12 and a processor 13. The memory 12 is used to store computer-readable instructions, and the processor 13 executes the computer-readable instructions stored in the memory to implement a trajectory generation method as described in any of the above embodiments.

[0170] In one embodiment of this application, the electronic device 100 further includes a bus and a computer program stored in the memory 12 and executable on the processor 13, such as a trajectory generation program.

[0171] Figure 10 Only an electronic device 100 with memory 12 and processor 13 is shown; those skilled in the art will understand that... Figure 10 The structure shown does not constitute a limitation on the electronic device 100, and may include fewer or more components than shown, or combine certain components, or have different component arrangements.

[0172] Combination Figure 2 The memory 12 in the electronic device 100 stores a plurality of computer-readable instructions to implement a trajectory generation method. The processor 13 can execute the plurality of instructions to: encode video data collected by a plurality of cameras in the environment to be detected and IoT data of the environment to be detected to obtain a spatiotemporal auxiliary vector of the environment to be detected; determine the visual features of the object to be detected by the plurality of cameras based on the video data; determine the joint feature vector of the object to be detected based on the spatiotemporal auxiliary vector and the visual features; and determine the trajectory generation method based on the joint feature vector and the state parameters of the plurality of cameras. A strategy for storing the joint feature vectors is employed, instructing the joint feature vectors to be written into a feature vector library. Based on the joint feature vectors and the topological information of the multiple camera devices, an index parameter for the object to be detected is determined. This index parameter is used to determine the identity association results of the object to be detected collected by the multiple camera devices from the feature vector library. Based on the coordinate system transformation rules corresponding to the multiple camera devices, the identity association results and the real-time position information of the object to be detected collected by the multiple camera devices are fused to obtain the trajectory data of the object to be detected moving in the detection environment. The trajectory data is then output.

[0173] Specifically, the processor 13's implementation method for the above instructions can be found in [reference needed]. Figure 2 The descriptions of the relevant steps in the corresponding embodiments are not repeated here.

[0174] Those skilled in the art will understand that the schematic diagram is merely an example of the electronic device 100 and does not constitute a limitation on the electronic device 100. The electronic device 100 may be a bus-type structure or a star-type structure. The electronic device 100 may also include more or fewer other hardware or software than shown in the diagram, or different component arrangements. For example, the electronic device 100 may also include input / output devices, network access devices, etc.

[0175] It should be noted that electronic device 100 is only an example. Other existing or future electronic products that are suitable for this application should also be included within the scope of protection of this application and are incorporated herein by reference.

[0176] The memory 12 includes at least one type of readable storage medium, which can be non-volatile or volatile. The readable storage medium includes flash memory, portable hard drives, multimedia cards, card-type memory (e.g., SD or DX memory), magnetic storage, magnetic disks, optical disks, etc. In some embodiments, the memory 12 can be an internal storage unit of the electronic device 100, such as the portable hard drive of the electronic device 100. In other embodiments, the memory 12 can also be an external storage device of the electronic device 100, such as a plug-in portable hard drive, smart media card (SMC), secure digital (SD) card, flash card, etc., equipped on the electronic device 100. The memory 12 can be used not only to store application software and various types of data installed on the electronic device 100, such as the code of a trajectory generation program, but also to temporarily store data that has been output or will be output.

[0177] In some embodiments, the processor 13 may be composed of integrated circuits, such as a single packaged integrated circuit or multiple integrated circuits packaged with the same or different functions, including combinations of one or more central processing units (CPUs), microprocessors, digital processing chips, graphics processors, and various control chips. The processor 13 is the control unit of the electronic device 100, connecting to various components of the electronic device 100 via various interfaces and lines. It executes programs or modules stored in the memory 12 (e.g., executing a trajectory generation program) and calls data stored in the memory 12 to perform various functions of the electronic device 100 and process data.

[0178] The processor 13 executes the operating system of the electronic device 100 and various installed applications. The processor 13 executes the applications to implement the steps in each of the above-described embodiments of the trajectory generation method, for example... Figure 2 The steps are shown.

[0179] For example, the computer program may be divided into one or more modules / units, which are stored in the memory 12 and executed by the processor 13 to complete this application. The one or more modules / units may be a series of computer-readable instruction segments capable of performing a specific function, which describe the execution process of the computer program in the electronic device 100.

[0180] The integrated unit implemented as a software functional module described above can be stored in a computer-readable storage medium. This software functional module, stored in a storage medium, includes several instructions to cause a computer device (which may be a personal computer, computer equipment, or network device, etc.) or processor to execute portions of the trajectory generation method described in the various embodiments of this application.

[0181] If the modules / units integrated in the electronic device 100 are implemented as software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, all or part of the processes in the methods of the above embodiments can also be implemented by a computer program instructing related hardware devices. The computer program can be stored in a computer-readable storage medium, and when executed by a processor, it can implement the steps of the various method embodiments described above.

[0182] The computer program includes computer program code, which may be in the form of source code, object code, executable file, or some intermediate form. The computer-readable medium may include: any entity or device capable of carrying the computer program code, recording media, USB flash drive, portable hard drive, magnetic disk, optical disk, computer memory, read-only memory (ROM), random access memory, and other memory.

[0183] Furthermore, the computer-readable storage medium may primarily include a stored program area and a stored data area, wherein the stored program area may store the operating system, an application program required for at least one function, etc.; and the stored data area may store data created based on the use of blockchain nodes, etc.

[0184] The bus can be a Peripheral Component Interconnect (PCI) bus or an Extended Industry Standard Architecture (EISA) bus, etc. This bus can be divided into address bus, data bus, control bus, etc. For ease of representation, in... Figure 10 The symbol is represented by only one arrow, but this does not indicate that there is only one bus or one type of bus. The bus is configured to enable communication between the memory 12 and at least one processor 13, etc.

[0185] This application also provides a computer-readable storage medium (not shown), which stores computer-readable instructions that are executed by a processor in an electronic device to implement a trajectory generation method described in any of the above embodiments.

[0186] In the several embodiments provided in this application, it should be understood that the disclosed systems, apparatuses, and methods can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative; for instance, the division of modules is only a logical functional division, and other division methods may be used in actual implementation.

[0187] The modules described as separate components may or may not be physically separate. The components shown as modules may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs.

[0188] Furthermore, the functional modules in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or in the form of hardware plus software functional modules.

[0189] Furthermore, it is clear that the word "comprising" does not exclude other units or steps, and the singular does not exclude the plural. Multiple units or devices described in the specification may also be implemented by a single unit or device through software or hardware. Terms such as "first," "second," etc., are used to indicate names and do not indicate any specific order.

[0190] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of this application and are not intended to limit it. Although this application has been described in detail with reference to preferred embodiments, those skilled in the art should understand that modifications or equivalent substitutions can be made to the technical solutions of this application without departing from the spirit and scope of the technical solutions of this application.

Claims

1. A trajectory generation method, characterized in that, The method includes: Encoding video data collected by multiple camera devices in the environment to be tested, as well as IoT data of the environment to be tested, to obtain a spatiotemporal auxiliary vector of the environment to be tested includes: synchronously acquiring the video data from the multiple camera devices based on a preset communication protocol; performing structured processing on the IoT data to obtain structured IoT data; the structured IoT data includes the three-dimensional coordinates of the multiple camera devices in the environment to be tested, the timestamps corresponding to the video frames of the video data, the light intensity of the environment to be tested, the access control status of the environment to be tested, and the latitude and longitude coordinates of the object to be tested; and encoding the video data and the structured IoT data using a preset encoding model to obtain the spatiotemporal auxiliary vector. Based on the video data, the visual features of the object to be detected by the plurality of camera devices are determined; based on the spatiotemporal auxiliary vector and the visual features, the joint feature vector of the object to be detected is determined. Based on the joint feature vector and the state parameters of the multiple camera devices, a database entry strategy for the joint feature vector is determined, including: acquiring the state parameters of the multiple camera devices, the state parameters including the frame rate at which the multiple camera devices acquire video data, the cache queue length corresponding to the joint feature vector to be entered into the database, and the processor utilization rate; determining the database entry frequency of the joint feature vector based on the frame rate and the number of joint feature vectors; updating the database entry frequency based on the processor utilization rate to obtain an updated database entry frequency; determining the write batch of the joint feature vector based on the updated database entry frequency and the cache queue length to obtain a database entry strategy for the joint feature vector, the database entry strategy indicating that the joint feature vector is written into a feature vector database; the database entry strategy indicating a strategy for writing joint feature vector data generated from video data of different camera devices and different frame rates into the corresponding fragments of the feature vector database; Based on the joint feature vector and the topological information of the multiple camera devices, the index parameters of the object to be detected are determined. The index parameters are used to determine the identity association results of the object to be detected collected by the multiple camera devices from the feature vector library. Based on the coordinate system transformation rules corresponding to the multiple camera devices, the identity association results and the real-time location information of the object to be detected collected by the multiple camera devices are fused to obtain the trajectory data of the object to be detected moving in the environment to be detected. Output the trajectory data.

2. The trajectory generation method as described in claim 1, characterized in that, The step of determining the visual features of the object to be detected by the plurality of camera devices based on the video data includes: The video data is decoded to obtain the original image from the video data; Based on the original image, a preset target detection model is used to determine the target detection box corresponding to the target object in the original image; The image within the target detection box is identified as the target image; Based on the target image, the visual features of the object to be detected are determined using a preset feature extraction model.

3. The trajectory generation method as described in claim 1, characterized in that, The step of determining the joint feature vector of the object to be detected based on the spatiotemporal auxiliary vector and the visual features includes: The visual features are normalized to obtain normalized visual features; The normalized visual features are subjected to dimensionality reduction processing to obtain cross-scene visual features; The cross-scene visual features and the spatiotemporal auxiliary vectors are concatenated to obtain the first fusion feature; The first fused feature is normalized to obtain the normalized fused feature; The identifiers of the multiple camera devices, the timestamps for acquiring the visual features, the three-dimensional coordinates of the camera devices in a preset coordinate system, and the identifier of the object to be detected are encoded to obtain metadata encoding; Based on the metadata encoding and the normalized fusion features, key-value pairs are generated to obtain a joint feature vector.

4. The trajectory generation method as described in claim 1, characterized in that, The determination of the index parameters of the object to be detected based on the joint feature vector and the topological information of the multiple camera devices includes: The spatiotemporal index of the object to be detected is obtained by using the three-dimensional coordinates of the camera device corresponding to the joint feature vector and the timestamp of acquiring the joint feature vector as index fields. The joint feature vector is normalized to obtain the visual index of the object to be detected; The spatiotemporal index and the visual index are determined as the index parameters of the object to be detected.

5. The trajectory generation method as described in claim 4, characterized in that, The step of determining the identity association results of the object to be detected collected by the multiple camera devices from the feature vector library includes: Based on the spatiotemporal index, preliminary association results of the objects to be detected collected by the multiple camera devices are determined from the feature vector library; Based on the topology information of the multiple camera devices, determine the distance and viewing angle between any two camera devices; Based on the distance and the angle of view, determine the dynamic threshold between any two camera devices; Determine the similarity between the preliminary association results and the visual index; If the similarity is greater than or equal to the dynamic threshold, the preliminary association result is determined as the identity association result.

6. The trajectory generation method as described in claim 1, characterized in that, The method, based on the coordinate system transformation rules corresponding to the multiple camera devices, fuses the identity association results and the real-time location information of the object to be detected collected by the multiple camera devices to obtain the trajectory data of the object to be detected moving in the environment to be detected, including: Based on a preset homography matrix, the transformation rules for the coordinate systems corresponding to any two camera devices are determined, and the identity association results are spliced ​​together to form the historical movement trajectory of the object to be detected based on the transformation rules; wherein, the homography matrix is ​​used to indicate the coordinate transformation relationship between different camera devices; By splicing the real-time location information of the object to be detected with the historical movement trajectory, the trajectory data of the object to be detected moving in the environment to be detected is obtained.

7. An electronic device, characterized in that, The electronic device includes a processor and a memory, the processor being configured to implement the trajectory generation method as described in any one of claims 1 to 6 when executing a computer program stored in the memory.

8. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores computer-readable instructions that are executed by a processor in an electronic device to implement the trajectory generation method as described in any one of claims 1 to 6.

Images