Data processing method, storage medium and electronic device

Abstract

The application discloses a data processing method, a storage medium and an electronic device. The method comprises the following steps: acquiring multi-modal data; performing feature extraction processing on the multi-modal data to obtain first comparison features of a to-be-compared sequence, wherein the to-be-compared sequence is a target object tracking sequence acquired by an image acquisition device; selecting a plurality of clusters based on the first comparison features to obtain second comparison features of each cluster in the plurality of clusters, wherein each cluster in the plurality of clusters comprises at least one sequence belonging to the same target object; performing first similarity comparison processing on the first comparison features and the second comparison features to obtain a first comparison result; and merging the to-be-compared sequence to a target cluster according to the first comparison result. The application solves the technical problems that the existing target object re-identification scheme cannot realize accurate target object association of cross-border heads and real-time merging of target object data.

Description

Technical Field

[0001] This application relates to the field of data processing, and more specifically, to a data processing method, a storage medium, and an electronic device. Background Technology

[0002] Intelligent video surveillance is a key application scenario for artificial intelligence. It utilizes video technology to digitize target objects (including living and non-living objects) in important locations (shopping malls, communities, stations, etc.), and is one of the main ways to achieve precise and intelligent management. How to accurately detect, identify, track, and locate target objects from a large number of deployed cameras is a crucial aspect of intelligent applications. In the video stream processing of intelligent applications, the core of target object digitization is to achieve cross-camera target object association, i.e., target object re-identification. This involves associating the same target object discretely seen in different shots, enabling correlation analysis of the same object in the video stream, or full-field, all-around trajectory tracking and positioning across shots and regions.

[0003] However, target object re-identification in video stream processing scenarios is a non-contact data acquisition technology under natural conditions. Target objects acquired under non-cooperative conditions present numerous challenges, including diverse poses, changing lighting, partial occlusion, different viewing angles, varying distances, and image blur. Therefore, achieving accurate target object association across multiple cameras is a highly challenging research topic and an inevitable technical hurdle in implementation. Furthermore, video surveillance systems deployed in areas such as shopping malls, industrial parks, and train stations typically contain hundreds of cameras, constantly reporting video streams. Target object merging systems often face enormous computational pressure when merging this continuously reported data. Therefore, reducing the computational complexity of target object data merging and improving the timeliness of data processing are also key issues that need to be addressed in the practical application of intelligent monitoring systems.

[0004] There is currently no effective solution to the above problems. Summary of the Invention

[0005] This application provides a data processing method, storage medium, and electronic device to at least solve the technical problem that existing target object re-identification schemes cannot achieve accurate cross-border target object association and real-time merging of target object data.

[0006] According to one aspect of the embodiments of this application, a data processing method is provided, comprising: acquiring multimodal data, wherein the multimodal data is determined by image information acquired by an image acquisition device and attribute information of the image acquisition device itself; performing feature extraction processing on the multimodal data to obtain a first alignment feature of a sequence to be compared, wherein the sequence to be compared is a target object tracking sequence acquired by the image acquisition device; selecting multiple clusters based on the first alignment feature to obtain a second alignment feature of each of the multiple clusters, wherein each of the multiple clusters includes at least one sequence belonging to the same target object; performing a first similarity comparison processing on the first alignment feature and the second alignment feature to obtain a first alignment result; and merging the sequence to be compared into a target cluster according to the first alignment result.

[0007] According to another aspect of the embodiments of this application, a data processing apparatus is also provided, comprising: an acquisition module for acquiring multimodal data, wherein the multimodal data is determined by image information acquired by an image acquisition device and attribute information of the image acquisition device itself; an extraction module for performing feature extraction processing on the multimodal data to obtain a first alignment feature of a sequence to be compared, wherein the sequence to be compared is a target object tracking sequence acquired by the image acquisition device; a selection module for selecting multiple clusters based on the first alignment feature to obtain a second alignment feature of each of the multiple clusters, wherein each of the multiple clusters includes at least one sequence belonging to the same target object; an alignment module for performing a first similarity comparison processing on the first alignment feature and the second alignment feature to obtain a first alignment result; and a merging module for merging the sequence to be compared into a target cluster according to the first alignment result.

[0008] According to another aspect of the embodiments of this application, a non-volatile storage medium is also provided, the non-volatile storage medium including a stored program, wherein, when the program is running, it controls the device where the non-volatile storage medium is located to execute any of the above-described data processing methods.

[0009] According to another aspect of the embodiments of this application, an electronic device is also provided, including: a processor; and a memory connected to the processor, configured to provide the processor with instructions to process the following processing steps: acquiring multimodal data, wherein the multimodal data is determined by image information acquired by an image acquisition device and attribute information of the image acquisition device itself; performing feature extraction processing on the multimodal data to obtain a first alignment feature of a sequence to be compared, wherein the sequence to be compared is a target object tracking sequence acquired by the image acquisition device; selecting multiple clusters based on the first alignment feature to obtain a second alignment feature of each of the multiple clusters, wherein each of the multiple clusters includes at least one sequence belonging to the same target object; performing a first similarity comparison processing on the first alignment feature and the second alignment feature to obtain a first alignment result; and merging the sequence to be compared into a target cluster according to the first alignment result.

[0010] In this embodiment, multimodal data is acquired, wherein the multimodal data is determined by image information acquired by an image acquisition device and attribute information of the image acquisition device itself; feature extraction processing is performed on the multimodal data to obtain a first alignment feature of the sequence to be compared, wherein the sequence to be compared is a target object tracking sequence acquired by the image acquisition device; multiple clusters are selected based on the first alignment feature to obtain a second alignment feature of each of the multiple clusters, wherein each of the multiple clusters includes at least one sequence belonging to the same target object; a first similarity comparison processing is performed on the first alignment feature and the second alignment feature to obtain a first alignment result; and the sequence to be compared is merged into the target cluster according to the first alignment result.

[0011] It is noteworthy that the embodiments of this application provide a complete processing procedure for monitoring data from camera acquisition to merging into the database (e.g., monitoring animals in a zoo). The XGBoost comprehensive decision-making scheme, utilizing the extreme gradient booster, achieves comprehensive utilization of multimodal information. The ReID streaming merging scheme fully leverages the streaming reporting characteristics of target object data in the monitoring system. By caching a clustering pool, newly entered sequences to be compared only require similarity calculation at the cluster level during clustering. Real-time merging of target object data is achieved through incremental clustering, exhibiting unique advantages such as low algorithm complexity, low computational overhead, and high operating efficiency. This enables the digitization and structuring of target object identity and trajectory data throughout the entire field and lifecycle. It can better meet the technical requirements of comprehensive data utilization and fast system operation faced by intelligent monitoring systems during implementation, promoting the better implementation of target object re-identification technology and ultimately achieving better business results.

[0012] Therefore, the embodiments of this application achieve the goal of accurately associating cross-border head objects and merging target object data in real time, thereby realizing the technical effect of providing more efficient and higher-precision intelligent monitoring applications, facilitating the optimization, upgrading and continuous iteration of intelligent monitoring systems, and solving the technical problem that existing target object re-identification schemes cannot achieve accurate cross-border head object association and real-time merging of target object data. Attached Figure Description

[0013] The accompanying drawings, which are included to provide a further understanding of this application and form part of this application, illustrate exemplary embodiments and are used to explain this application, but do not constitute an undue limitation of this application. In the drawings:

[0014] Figure 1 This is a hardware structure block diagram of a computer terminal (or mobile device) for implementing a data processing method according to an embodiment of this application;

[0015] Figure 2 This is a flowchart of a data processing method according to an embodiment of this application;

[0016] Figure 3 This is a schematic block diagram of an optional streaming merge system based on XGBoost synthesis decision according to an embodiment of this application;

[0017] Figure 4 This is a schematic diagram illustrating the principle of an optional streaming merge according to an embodiment of this application;

[0018] Figure 5 This is a schematic diagram of the structure of a data processing apparatus according to an embodiment of this application;

[0019] Figure 6 This is a structural block diagram of another computer terminal according to an embodiment of this application. Detailed Implementation

[0020] To enable those skilled in the art to better understand the present application, the technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present application, and not all embodiments. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative effort should fall within the scope of protection of the present application.

[0021] It should be noted that the terms "first," "second," etc., in the specification, claims, and accompanying drawings of this application are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments of this application described herein can be implemented in orders other than those illustrated or described herein. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover non-exclusive inclusion; for example, a process, method, system, product, or apparatus that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus.

[0022] First, some nouns or terms that appear in the description of the embodiments of this application shall be interpreted as follows:

[0023] Person reidentification (Person ReID): A technique for associating the identity of a target object across different shots.

[0024] Extreme Gradient Boosting (XGBoost): A machine learning method based on gradient boosting decision trees.

[0025] Multimodal data: Structured / unstructured data that has significant modal differences in data source, acquisition method, data format, and physical meaning.

[0026] Target object merging: Aggregate target object data reported by multiple cameras according to the identity of the target object, so that the same target object is merged into the same cluster, while different target objects are assigned to different clusters.

[0027] Streaming merge: A (near) real-time merging process that can merge target object data reported by cameras in real time in a streaming manner.

[0028] Current academic research on this issue mainly focuses on single data sources such as target object images. In practical applications, this can easily lead to serious erroneous associations, affecting the effectiveness of the application. The fundamental reason is that target object images in a monitoring environment have many limitations mentioned above, and relying solely on target object data is insufficient to overcome these shortcomings. In reality, in addition to target object data, monitoring systems typically collect other data simultaneously, such as target object attribute data, camera location data, and target object acquisition time. This data plays a crucial supporting role in accurate target object association. For example, using the time and spatial information reported by the target object can largely eliminate the possibility of erroneous merging of target objects from multiple cameras simultaneously. Therefore, the key to cross-camera target object association lies in how to organically organize and utilize multimodal data from these various sources.

[0029] Video surveillance systems deployed in areas such as shopping malls, industrial parks, and train stations typically contain hundreds of cameras, constantly reporting monitoring data streams. When merging this continuous stream of data, target object merging systems often face immense computational pressure. To avoid impacting system operation, a common practice is to first store the reported data in a database and then perform merging using a (near) offline method. The drawback of this approach is the significant delay in data merging, limiting applications requiring immediate response, such as identifying high-risk personnel and objects at train stations, locking stolen goods in shopping malls, and intelligent customer marketing at vending machines. Reducing the computational complexity of target object merging and improving the timeliness of data processing are key issues that need to be addressed in the practical application of intelligent monitoring systems.

[0030] To address the two major pain points mentioned above, this application proposes targeted solutions. First, regarding the comprehensive utilization of multimodal data, this application proposes an information fusion scheme based on the XGBoost decision-maker from machine learning. This scheme autonomously selects and fuses valuable information to execute comprehensive decisions in a data-driven manner, overcoming the limitations of a single data source while avoiding the cumbersome and non-scalable nature of manually designed decision rules. Second, to improve the real-time performance of the merging system, this application proposes a streaming target object merging algorithm. This algorithm achieves efficient correlation across long-term, cross-camera scenarios through real-time merging and dynamic aggregation. Using this application, the technical requirements of comprehensive data utilization and fast system operation faced by intelligent monitoring systems during implementation can be effectively addressed, promoting the better implementation of target object re-identification technology and achieving better business results.

[0031] Example 1

[0032] According to an embodiment of this application, an embodiment of a data processing method is provided. It should be noted that the steps shown in the flowchart in the accompanying drawings can be executed in a computer system such as a set of computer-executable instructions. Furthermore, although a logical order is shown in the flowchart, in some cases, the steps shown or described may be executed in a different order than that shown here.

[0033] The method embodiment provided in Embodiment 1 of this application can be executed in a mobile terminal, computer terminal or similar computing device. Figure 1 A hardware structure block diagram of a computer terminal (or mobile device) for implementing a data processing method is shown. Figure 1 As shown, the computer terminal 10 (or mobile device 10) may include one or more processors 102 (shown as 102a, 102b, ..., 102n in the figure) (processor 102 may include, but is not limited to, a microprocessor MCU or a programmable logic device FPGA, etc.), a memory 104 for storing data, and a transmission module 106 for communication functions. In addition, it may also include: a display, an input / output interface (I / O interface), a universal serial bus (USB) port (which may be included as one of the ports of a BUS bus), a network interface, a power supply, and / or a camera. Those skilled in the art will understand that... Figure 1 The structure shown is for illustrative purposes only and does not limit the structure of the aforementioned electronic device. For example, computer terminal 10 may also include... Figure 1 The more or fewer components shown, or having the same Figure 1 The different configurations shown.

[0034] It should be noted that the aforementioned one or more processors 102 and / or other data processing circuits are generally referred to herein as "data processing circuits". These data processing circuits may be embodied, in whole or in part, in software, hardware, firmware, or any other combination thereof. Furthermore, the data processing circuits may be a single, independent processing module, or may be integrated, in whole or in part, into any other element within the computer terminal 10 (or mobile device). As involved in the embodiments of this application, the data processing circuits serve as a processor control mechanism (e.g., selection of a variable resistor termination path connected to an interface).

[0035] The memory 104 can be used to store software programs and modules of application software, such as program instructions / data storage devices corresponding to the data processing method in this embodiment. The processor 102 executes various functional applications and data processing by running the software programs and modules stored in the memory 104, thereby implementing the aforementioned data processing method. The memory 104 may include high-speed random access memory, and may also include non-volatile memory, such as one or more magnetic storage devices, flash memory, or other non-volatile solid-state memory. In some instances, the memory 104 may further include memory remotely located relative to the processor 102, and these remote memories can be connected to the computer terminal 10 via a network. Examples of such networks include, but are not limited to, the Internet, corporate intranets, local area networks, mobile communication networks, and combinations thereof.

[0036] The transmission device 106 is used to receive or send data via a network. Specific examples of the network described above may include a wireless network provided by the communication provider of the computer terminal 10. In one example, the transmission device 106 includes a Network Interface Controller (NIC), which can connect to other network devices via a base station to communicate with the Internet. In another example, the transmission device 106 may be a Radio Frequency (RF) module, used for wireless communication with the Internet.

[0037] The display can be, for example, a touchscreen liquid crystal display (LCD) that allows the user to interact with the user interface of the computer terminal 10 (or mobile device).

[0038] In recent years, with the development of deep learning technology, person re-identification (ReID) technology has also made significant progress and is gradually moving from academic research to industrial applications. Every year, numerous related papers emerge at top academic conferences (CVPR, ECCV, AAAI, etc.), and domestic internet companies are actively deploying in this field, vying for technological dominance and market share. In academia, current research mainly focuses on single-point technological breakthroughs, improving the discriminative power of target object features through algorithmic innovation. Examples include target object recognition under occlusion, cross-scene feature transfer, and unsupervised model training. The common thread is starting with target object images and improving the comparison capability of target object features through deeper algorithmic innovation, model design, and optimized training. However, considering the inherent challenges of occlusion, blurring, pose changes, and perspective differences in surveillance videos, innovation and optimization at the model level are gradually facing technical bottlenecks. While improvements can be achieved in some aspects, significant technical obstacles and cost limitations exist in practical implementation.

[0039] In the early stages of system construction, the utilization of multi-source information was mainly based on human experience, through the design of corresponding rules and strategies. For example, the comparison pool was first screened according to time and space, and then the corresponding comparison rules were designed for the target objects for aggregation. This approach required a lot of human intervention. It was manageable when there were few data sources in the early stages of the project. However, as the system continued to improve and the data sources became more diverse and refined, this model based on human rules became increasingly unsustainable.

[0040] From a systems theory perspective, innovation in the model represents depth-level exploration, while the comprehensive utilization of multi-source data in this application represents breadth-level expansion. The two complement each other, and the improvement of the target object characteristics will also have a positive impact on the final effect of this application.

[0041] In industry, target object re-identification technology is typically offered as a feature extraction API or SDK, but at the application level, it remains limited to a single data source—the target object image—and does not address the rich multi-source, multi-modal data inherent in video surveillance systems. Compared to the single solutions in existing technologies, the data processing method provided in this application is a system-level solution that comprehensively utilizes various data related to the target object, main areas of the object, attributes, time, and space involved in the monitoring system, resulting in more thorough and comprehensive information utilization.

[0042] Under the aforementioned operating environment, this application provides the following: Figure 2 This illustrates a data processing method. Figure 2 This is a flowchart of a data processing method according to an embodiment of this application, such as... Figure 2 As shown, the above data processing method includes:

[0043] Step S202: Obtain multimodal data, wherein the multimodal data is determined by the image information acquired by the image acquisition device and the attribute information of the image acquisition device itself.

[0044] Step S204: Perform feature extraction processing on the above multimodal data to obtain the first comparison feature of the sequence to be compared, wherein the above comparison sequence is a target object tracking sequence obtained by the above image acquisition device;

[0045] Step S206: Select multiple clusters based on the first alignment features mentioned above, and obtain the second alignment features of each cluster in the multiple clusters, wherein each cluster in the multiple clusters includes at least one sequence belonging to the same target object;

[0046] Step S208: Perform a first similarity comparison process on the first comparison feature and the second comparison feature to obtain a first comparison result;

[0047] Step S210: Merge the above-mentioned sequences to be compared into the target cluster according to the first comparison result.

[0048] It should be noted that the data processing method provided in this application can be applied to, but is not limited to, intelligent video surveillance scenarios, such as intelligent security, smart supermarkets, zoo monitoring, and digital government applications. Intelligent video surveillance plays an irreplaceable and important role in these digital applications. Addressing the challenges of cross-camera and cross-regional target object re-identification in video surveillance systems, including diverse data types, significant modal differences, and high computational complexity, the data processing method provided in this application innovates on application models and basic algorithms. It offers a streaming merging method and system based on XGBoost comprehensive decision-making. This streaming merging system based on XGBoost comprehensive decision-making is essentially a video surveillance system that can be used to implement or realize the streaming merging method based on XGBoost comprehensive decision-making. To a certain extent, it effectively solves the two major technical challenges of fully utilizing data and efficiently operating the system in video surveillance systems, providing a new solution for more efficient and higher-precision intelligent monitoring applications.

[0049] Optionally, in terms of multimodal data utilization, this application proposes a comprehensive decision-making method based on XGBoost, which automatically achieves optimal utilization of multi-source information through machine learning, and has outstanding advantages such as low system complexity, low R&D cost, and good scalability.

[0050] Optionally, in the field of data merging of target objects across shots, this application proposes a streaming merging algorithm for monitoring data streams. By fully utilizing the streaming characteristics and spatiotemporal correlation of monitoring data, it achieves real-time merging of target object data through incremental clustering, which has unique advantages such as low algorithm complexity, low computational overhead, and high operating efficiency.

[0051] It should be noted that the data processing solution provided in this application embodiment originates from the actual business of offline security on the Internet. For example, in scenarios such as new retail digital supermarkets and smart parks, tasks such as intelligent customer flow analysis and intelligent item management are required, which necessitates automated target object merging and trajectory association technologies. Unlike existing solutions that primarily rely on human experience and multi-source information, the comprehensive decision-making scheme based on the Extreme Gradient Booster (XGBoost) proposed in this application can not only adapt to any amount and type of multi-source, multi-modal data, but also achieve higher decision accuracy with richer data sources. This greatly facilitates the optimization, upgrading, and continuous iteration of intelligent monitoring systems.

[0052] It should be noted that in the cross-camera merging process, the commonly used solution in existing technologies relies on data clustering techniques. This requires first collecting the data to be clustered, then using general clustering algorithms (hierarchical clustering, DBSCAN, HDBSCAN, etc.) to aggregate the data, and the output clusters are the merging results. However, since clustering algorithms are typically very time-consuming and cannot be performed in real-time, they are insufficient for applications requiring real-time merging. Therefore, in this embodiment, an incremental clustering algorithm for streaming data is designed to address the characteristics of the data flow in the monitoring system. This algorithm can remember historical clustering results, and only requires a small amount of computation to complete the merging of new reported data, greatly simplifying the computational complexity and providing a technical advantage for real-time merging.

[0053] The following uses the data processing method provided in this application embodiment applied to a zoo area monitoring scenario as an example to illustrate an optional data processing method embodiment. It should be noted that this data processing method can display the data processing process through a visual graphical user interface (interactive interface, monitoring management interface) provided by the monitoring system (monitoring platform, video monitoring application software, etc.). During the automatic data processing through machine learning, it interacts with the user in real time, making it more flexible, convenient, and user-friendly. For example, for multimodal data acquired by an image acquisition device, the multimodal data can be displayed on the monitoring management interface of the monitoring system. The target object tracking sequence acquired by the image acquisition device is the comparison sequence. In response to the user's execution command, the multimodal data is sampled. The extraction of this basic feature involves obtaining the first alignment feature of the sequence to be compared, and displaying the sequence to be compared, the extracted first alignment feature, and multiple clusters to be selected on the monitoring and management interface. In response to the user's selection command, multiple clusters are selected based on the first alignment feature, and the second alignment feature of each of the multiple clusters is obtained and displayed on the monitoring and management interface. In response to the user's comparison command, a first similarity comparison process is then performed between the first alignment feature and the second alignment feature. Finally, based on the obtained first comparison result, the sequence to be compared is merged into the target cluster, and the first comparison interface and the merging result of the sequence to be compared in the target cluster are displayed on the monitoring and management interface, thereby realizing the process of performing streaming merging on the sequence to be compared.

[0054] In one optional embodiment, the multimodal data described above includes some or all of the following data:

[0055] The target object feature data, target object attribute data, target object quality data, target object front and back identification data, target object tracking sequence data of the aforementioned image acquisition device, target object sequence acquisition time data, identification data of the aforementioned image acquisition device, and location data of the aforementioned image acquisition device.

[0056] The system-level intelligent monitoring solution provided by the embodiments of this application, and the data processing method provided by the embodiments of this application, are based on atomic capabilities such as target object feature extraction and target object attribute extraction. For multimodal data acquired by image acquisition devices, firstly, the basic features of the sample are extracted from the multimodal data to obtain the first alignment features of the sequence to be compared, and multiple clusters are selected based on the first alignment features to obtain the second alignment features of each of the multiple clusters. Then, the first alignment features and the second alignment features are compared for a first similarity. Finally, streaming merging is performed on the sequence to be compared based on the obtained first alignment results.

[0057] As an optional embodiment, the aforementioned image acquisition device is a camera device, such as a surveillance camera or an AI camera. The image information acquired by the aforementioned image acquisition device and the attribute information of the aforementioned image acquisition device itself are used to determine multimodal data. The image information acquired by the aforementioned image acquisition device includes some or all of the following data: target object feature data, target object attribute data, target object quality data, and target object front and back identification data. The attribute information of the aforementioned image acquisition device itself includes some or all of the following data: target object tracking sequence data of the aforementioned image acquisition device, target object sequence acquisition time data, identification data of the aforementioned image acquisition device, and location data of the aforementioned image acquisition device.

[0058] As an optional embodiment, the streaming merge system based on XGBoost synthesis decision provided in this application embodiment can be used to implement or realize the streaming merge method based on XGBoost synthesis decision provided in this application embodiment, such as... Figure 3 As shown, the system includes: a feature extraction module, a TopK candidate pool construction module, an XGBoost feature calculation module, an XGBoost decision maker, and a conflict resolution module. The following sections provide a detailed description of each module, corresponding to the method implementation process:

[0059] In this embodiment, the feature extraction module is a multi-source data acquisition module. It mainly extracts various structured data and metadata such as device parameters and acquisition time from unstructured surveillance videos through various network models, i.e., multimodal data. The multimodal data includes some or all of the following data: target object feature vector, target object attribute array, target object quality score, target object front and back identification, target object tracking sequence ID of the lens, target object sequence acquisition time, reported lens ID, and geographical location, etc. In this embodiment, the feature extraction module generally extracts the multimodal data in real time on AI cameras or edge devices and then reports it to the merging server.

[0060] In an optional embodiment, the second comparison feature is obtained by selecting the plurality of clusters based on the first comparison feature, including:

[0061] Step S302: Perform a second similarity comparison process on the first comparison features and the currently cached clusters in the clustering pool to obtain the second comparison result;

[0062] Step S304: Sort the second comparison results according to the similarity from high to low to obtain the sorting results;

[0063] Step S306: Select the above multiple clusters from the above currently cached clusters based on the above sorting results to obtain the above second comparison features.

[0064] Optionally, before determining the second alignment feature of each cluster among the above multiple clusters, multiple clusters need to be selected based on the determined first alignment feature. That is, before performing the comprehensive decision based on XGBoost, a preliminary screening is performed on the candidate pool of each sequence track, that is, the most likely TopK clusters are selected from the existing cluster pool, and then the XGBoost comprehensive similarity is calculated only for these few selected multiple clusters.

[0065] In this embodiment of the application, a second comparison result is obtained by performing a second similarity comparison process on the first comparison features and the currently cached clusters in the clustering pool; for example, using... Figure 3 The TopK candidate pool construction module shown sorts the second comparison results according to the similarity from high to low to obtain the sorting results; then, it selects multiple clusters from the current cached clusters based on the sorting results to obtain the second comparison features.

[0066] As an optional embodiment, refer to, Figure 3 The TopK candidate pool construction module shown below provides a detailed explanation of the implementation process of the above method steps. On the merging server, for the basic multimodal data reported by AI cameras or edge devices, before performing the comprehensive decision based on XGBoost, a preliminary screening is performed on the candidate pool of each sequence track to be compared (e.g., a target object tracking sequence within a shot). That is, the most likely TopK clusters are selected from the existing clustering pool, and then the XGBoost comprehensive similarity is calculated only for these few clusters.

[0067] It should be noted that since XGBoost calculation involves the construction of relatively complex alignment features, full calculation would incur significant computational overhead. Therefore, in this embodiment, limiting the alignment range to the most likely TopK can significantly reduce the computational load without having a noticeable impact on the final result.

[0068] It should be further noted that, in the embodiments of this application, the selection of TopK can be determined based on the characteristics of the target object. For example, the topK with the highest similarity can be selected from the cluster pool as candidates for further comprehensive comparison by calculating the cosine similarity of the characteristics of the target object.

[0069] In an optional embodiment, performing the first similarity comparison processing on the first comparison feature and the second comparison feature to obtain the first comparison result includes:

[0070] Step S402: The first comparison feature and the second comparison feature are fused to obtain the fused feature;

[0071] Step S404: Obtain the first comparison result based on the above-mentioned fusion features.

[0072] Optionally, the aforementioned fusion features include: similarity features in multiple dimensions; the aforementioned first alignment result includes: multiple predicted probabilities, which represent the probability that the aforementioned sequence to be aligned belongs to each of the aforementioned multiple clusters.

[0073] As an optional embodiment, refer to, Figure 3 The XGBoost feature calculation module shown below provides a detailed explanation of the implementation process of the above method steps. The XGBoost decision maker in this application is used to determine whether two sequences to be compared (e.g., track2track), or sequences to be compared and a cluster (track2session), or two clusters (session2session) are of the same person. Its input is the fused feature of the two features to be compared after certain preprocessing—feature engineering. Here, track is a target object tracking sequence within the shot, and session is a cluster composed of tracks of the same person in the clustering pool.

[0074] It should be noted that, in the embodiments of this application, the above feature engineering includes: 1) the average similarity of target object features within the track, 2) the maximum similarity of target object features within the track, 3) the average similarity of main region features of objects within the track, 4) the maximum similarity of main region features of objects within the track, 5) the product of the average quality scores of tracked target objects, 6) the product of the quality scores corresponding to the maximum similarity of tracked target objects, 7) the product of the average quality scores of main regions of tracked objects, 8) the product of the quality scores corresponding to the maximum similarity of main regions of tracked objects, 9) the product of the 0%, 50%, and 100% percentiles of the average similarity of tracked target objects within the session and the corresponding quality scores, 10) the product of the 0%, 50%, and 100% percentiles of the average similarity of main regions of tracked objects within the session and the corresponding quality scores, 11) the minimum time interval between the same camera, and 12) the minimum time interval across cameras, etc. The above feature quantities constitute the corresponding dimensions of the XGBoost comprehensive features and serve as the feature inputs for its comprehensive decision.

[0075] It should be further explained that in the feature design of the above-mentioned "XGBoost feature calculation module", in addition to the feature engineering proposed in this application, any other meaningful feature construction can be directly added, such as new information sources, new feature combinations, etc. The result is only to increase the dimension of the input features of XGBoost, without requiring any adjustment to the entire system. This is also the flexibility of this application.

[0076] In an optional embodiment, merging the sequences to be compared into the target cluster according to the first comparison result includes:

[0077] Step S502: When some or all of the predicted probabilities among the above multiple predicted probabilities are greater than or equal to the first preset threshold, the cluster with the highest predicted probability among the above multiple predicted probabilities is determined as the target cluster.

[0078] Step S504: Merge the above-mentioned sequences to be compared into the above-mentioned target cluster.

[0079] As an optional embodiment, refer to, Figure 3The XGBoost decision exemplified here provides a detailed explanation of the implementation process of the above method steps. The XGBoost decision exemplifies the similarity between tracks and sessions, making a comprehensive prediction about whether they belong to the same person and outputting the predicted probability of them being the same person. Before deployment to a business system, an XGBoost model needs to be trained based on labeled data. This involves collecting and labeling a sufficient number of target object tracks, constructing feature engineering using the feature calculation module, and then training the XGBoost decision exempl.

[0080] Since, in this embodiment of the application, the XGBoost decision maker gives the probability that each sequence to be compared reported by the lens belongs to each cluster in the clustering pool, when some or all of the predicted probabilities are greater than or equal to the first preset threshold, the cluster with the highest predicted probability among the multiple predicted probabilities is determined as the target cluster; the sequences to be compared are merged into the target cluster, and then each sequence to be compared can be merged into its most similar target cluster according to the magnitude of the probability.

[0081] In an optional embodiment, merging the sequences to be compared into the target cluster according to the first comparison result includes:

[0082] Step S602: When all of the above predicted probabilities are less than the above first preset threshold, the above target cluster is created for the above sequence to be compared.

[0083] Step S604: Merge the above-mentioned sequences to be compared into the above-mentioned target cluster.

[0084] In the above optional embodiments, if multiple predicted probabilities obtained by the XGBoost decision maker are all less than a pre-set first preset threshold, it indicates that the sequence to be compared does not belong to any cluster. In this case, a new target cluster is created for the sequence to be compared, and the sequence to be compared is merged into the target cluster.

[0085] In an optional embodiment, the above data processing method further includes:

[0086] Step S606: When it is determined that multiple sequences to be compared obtained at the same time all belong to the above target cluster, the sequence to be compared with the highest predicted probability is determined from the multiple sequences to be compared.

[0087] Step S608: Merge the pair of sequences with the highest predicted probability into the target cluster, and re-determine new clusters for the remaining pair of sequences other than the pair of sequences with the highest predicted probability, until all the pair of sequences are merged into the corresponding clusters.

[0088] As an optional embodiment, refer to, Figure 3 The conflict resolution module shown and as Figure 4 The schematic diagram shown illustrates the principle of streaming merging, and the implementation process of the above method steps is explained in detail. Since each surveillance camera continuously reports data streams carrying sequences to be compared to the merging server, for multiple sequences arriving at the same time T(0), the XGBoost integrated decision model is first used to calculate the most similar target cluster for each sequence in the clustering pool. At this point, multiple sequences may simultaneously be most similar to a certain cluster. However, common sense dictates that sequences arriving at the same time cannot belong to the same target object, thus a conflict arises, requiring conflict resolution.

[0089] In this embodiment of the application, when it is determined that multiple sequences to be compared obtained at the same time all belong to the target cluster, that is, for multiple sequences to be compared that are in conflict, the sequence to be compared with the highest prediction probability can be determined from the multiple sequences to be compared. For example, the target cluster can be assigned to the sequence to be compared with the highest prediction probability based on the XGBoost prediction value of the multiple sequences to be compared and the target cluster. The other sequences to be compared, except for the sequence to be compared with the highest prediction probability, are re-assigned to new clusters. That is, the other sequences to be compared search for their respective second similar target clusters. If the second similar target clusters are not shared, the remaining sequence to be compared is assigned to the second similar target cluster. Otherwise, conflict elimination is performed again according to similar logic until there are no more conflicts and the multiple sequences to be compared are all merged into the corresponding clusters.

[0090] It should be further noted that this application's solution not only simplifies the design complexity of the target object merging system but also significantly reduces deployment costs. The XGBoost decision tree classifier is a highly efficient decision tree classifier that supports parallel acceleration and boasts high execution efficiency; the streaming merge algorithm is an incremental clustering algorithm that requires only a small amount of computation per iteration, greatly reducing the computational burden on the clustering module. It can achieve a computational load of over 50 QPS on a single machine, satisfying the real-time merging of over 100 cameras. For a small to medium-sized monitoring scenario, only a single ordinary server is needed to meet the requirements.

[0091] In an optional embodiment, the above data processing method further includes:

[0092] Step S702: In the clusters currently cached in the clustering pool, perform the first similarity comparison process between the target cluster and the adjacent clusters to obtain the third comparison result;

[0093] Step S704: When it is determined based on the third comparison result that the similarity between the target cluster and the adjacent cluster is greater than the second preset threshold, the target cluster and the adjacent cluster are aggregated.

[0094] As an optional embodiment, it is still combined with, as Figure 4 The schematic diagram shown illustrates the principle of streaming merge, and provides a detailed explanation of the implementation process of the above method steps. For example... Figure 4 The dynamic aggregation shown is to perform aggregation on similar clusters in the clustering pool, that is, to merge multiple clusters belonging to the same target object currently cached in the clustering pool into one cluster.

[0095] For example, in a monitoring system, due to the variable pose of the target object, differences in shooting angle, changes in lighting, and object occlusion, the target object features of the same target object may vary greatly. In the initial stage of merging, they are split into different clusters. However, as the data stream continues to arrive, the target object features of the same target object under different poses are collected. Through the accumulation and transmission of samples, two clusters that were initially far apart may become adjacent. Therefore, dynamic aggregation is to calculate the XGBoost similarity (session2session) between the target cluster and the adjacent clusters for the updated target cluster after multiple new comparison sequences arrive. If it is determined that the similarity between the target cluster and the adjacent cluster is greater than a second preset threshold, it means that the target cluster and the adjacent cluster belong to the same target object. The target cluster and the adjacent cluster can be aggregated. Thus, through the embodiments of this application, as the data stream continues to arrive, the cluster structure of the clustering pool can be dynamically adjusted in a timely manner to reduce the split rate of target object merging.

[0096] In an optional embodiment, the above data processing method further includes:

[0097] Step S802: Record the target duration of the target cluster, wherein the target duration is the duration during which the target cluster has not been updated.

[0098] Step S804: When the target duration exceeds the third preset threshold, remove the target cluster from the clustering pool and perform behavior analysis on the target object based on the target cluster.

[0099] Optionally, the process of removing the target cluster from the clustering pool, known as cluster popping, is the final step in the ReID merging system, ensuring that terminated clusters are promptly removed from the merging system. For target clusters that have not been updated for a long time, such as when no new sequences to be compared are merged into the pool for a sufficiently long period (e.g., 1 hour), it indicates that the target object has left the pool. These "dormant" clusters can be promptly removed from the clustering pool, which reduces memory usage and minimizes unnecessary computations during the construction of the TopK candidate pool.

[0100] In this embodiment of the application, the target cluster popped from the stack is the final output of the merging system. It contains the complete behavioral trajectory of the corresponding target object in the entire field. After removing the target cluster, the application layer behavior analysis can be performed based on the trajectory data.

[0101] The data processing method described in this application provides a complete process for monitoring data from camera acquisition to data merging and storage. The XGBoost integrated decision module enables the comprehensive utilization of multimodal information, and the ReID streaming merging module realizes the real-time merging of target object data across the network, thereby achieving the digitization and structuring of personnel identity and trajectory data throughout the entire field and lifecycle.

[0102] As another optional embodiment, in terms of utilizing spatiotemporal relationships, in this application embodiment, in addition to utilizing the time interval within / between cameras, if spatial map data of the cameras is provided, the movement speed of the target object can also be incorporated. On the one hand, movement speed is added as a feature in the feature construction of XGBoost to increase the amount of information in the input features; on the other hand, movement speed is added as a judgment item in the conflict elimination module to further eliminate unreasonable merging.

[0103] As can be seen from the above embodiments of this application, the comprehensive decision-making scheme based on XGboost organically combines multiple information (target object, attributes, time, space, etc.) in the monitoring system, rather than relying solely on the traditional processing method of identifying only a single feature of the target object. For example, when re-identifying animals in a zoo, identification is based solely on the animal's facial features. This application addresses the pain points of data defects and insufficient information encountered in target object re-identification from a system perspective, improving decision-making accuracy and tracking performance. This application's solution utilizes a data-driven machine learning model to automatically achieve deep mining and organic utilization of multimodal data, overcoming the inherent defects of traditional manual rules and strategies, such as high complexity and poor combination effects. Furthermore, this application's solution also has the technical advantages of high flexibility and easy expansion. For example, when adding new data sources, such as map data, Wi-Fi data, infrared data, etc., they can be directly added to the XGboost feature extraction module as new features without any system-level adjustments, enabling rapid system iteration and upgrades.

[0104] The ReID streaming merge algorithm proposed in this application fully utilizes the streaming reporting characteristics of target object data in the monitoring system. By caching a clustering pool, newly entered sequences to be compared only need to have their similarity calculated on a per-cluster session basis during clustering. In contrast, traditional clustering algorithms require calculation on a per-sequence basis. The processing method in this application not only reduces the computational complexity from O(N) to O(N) 2 The computational complexity is reduced to O(N). Simultaneously, the clusters in the clustering pool dynamically adjust the session hierarchy as new tracks arrive, achieving incremental clustering with minimal computation.

[0105] Unlike traditional clustering algorithms that typically start from scratch and cannot utilize existing clustering results, resulting in a large amount of redundant computation, the streaming merge algorithm in this application is actually an incremental clustering algorithm specifically designed for monitoring systems. It can greatly reduce the computational complexity of clustering and transform the traditional offline target object merging into a real-time cross-regional trajectory tracking system, meeting the needs of intelligent applications requiring rapid response, such as rapid identification of at-risk personnel and intelligent shopping guides for supermarket customers.

[0106] It should be noted that, for the sake of simplicity, the foregoing method embodiments are all described as a series of actions. However, those skilled in the art should understand that this application is not limited to the described order of actions, as some steps may be performed in other orders or simultaneously according to this application. Furthermore, those skilled in the art should also understand that the embodiments described in the specification are preferred embodiments, and the actions and modules involved are not necessarily essential to this application.

[0107] Through the above description of the embodiments, those skilled in the art can clearly understand that the methods according to the above embodiments can be implemented by means of software plus necessary general-purpose hardware platforms. Of course, they can also be implemented by hardware, but in many cases the former is a better implementation method. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk) and includes several instructions to cause a terminal device (which may be a mobile phone, computer, server, or network device, etc.) to execute the methods described in the various embodiments of this application.

[0108] Example 2

[0109] According to an embodiment of this application, an apparatus embodiment for implementing the above-described data processing method is also provided. Figure 5 This is a schematic diagram of the structure of a data processing apparatus according to an embodiment of this application, such as... Figure 5 As shown, the data processing device includes: an acquisition module 500, an extraction module 502, a selection module 504, a comparison module 506, and a merging module 508, wherein:

[0110] The acquisition module 500 is used to acquire multimodal data, wherein the multimodal data is determined by image information acquired by the image acquisition device and the attribute information of the image acquisition device itself; the extraction module 502 is used to perform feature extraction processing on the multimodal data to obtain a first alignment feature of the sequence to be compared, wherein the sequence to be compared is a target object tracking sequence acquired by the image acquisition device; the selection module 504 is used to select multiple clusters based on the first alignment feature to obtain a second alignment feature of each of the multiple clusters, wherein each of the multiple clusters includes at least one sequence belonging to the same target object; the comparison module 506 is used to perform a first similarity comparison processing on the first alignment feature and the second alignment feature to obtain a first comparison result; and the merging module 508 is used to merge the sequence to be compared into the target cluster according to the first comparison result.

[0111] It should be noted that the acquisition module 500, extraction module 502, selection module 504, comparison module 506, and merging module 508 mentioned above correspond to steps S202 to S210 in Embodiment 1. The five modules and their corresponding steps implement the same examples and application scenarios, but are not limited to the content disclosed in Embodiment 1. It should be noted that the above modules, as part of the device, can run in the computer terminal 10 provided in Embodiment 1.

[0112] It should be noted that the preferred implementation of this embodiment can be found in the relevant description in Embodiment 1, and will not be repeated here.

[0113] Example 3

[0114] According to an embodiment of this application, an embodiment of an electronic device is also provided. This electronic device can be any computing device in a group of computing devices. The electronic device includes: a processor and a memory, wherein:

[0115] A processor; and a memory, connected to the processor, for providing the processor with instructions to perform the following processing steps: acquiring multimodal data, wherein the multimodal data is determined by image information acquired by an image acquisition device and attribute information of the image acquisition device itself; performing feature extraction processing on the multimodal data to obtain a first alignment feature of a sequence to be compared, wherein the sequence to be compared is a target object tracking sequence acquired by the image acquisition device; selecting multiple clusters based on the first alignment feature to obtain a second alignment feature of each of the multiple clusters, wherein each of the multiple clusters includes at least one sequence belonging to the same target object; performing a first similarity comparison processing on the first alignment feature and the second alignment feature to obtain a first alignment result; and merging the sequence to be compared into a target cluster according to the first alignment result.

[0116] In this embodiment, multimodal data is acquired, wherein the multimodal data is determined by image information acquired by an image acquisition device and attribute information of the image acquisition device itself; feature extraction processing is performed on the multimodal data to obtain a first alignment feature of the sequence to be compared, wherein the sequence to be compared is a target object tracking sequence acquired by the image acquisition device; multiple clusters are selected based on the first alignment feature to obtain a second alignment feature of each of the multiple clusters, wherein each of the multiple clusters includes at least one sequence belonging to the same target object; a first similarity comparison processing is performed on the first alignment feature and the second alignment feature to obtain a first alignment result; and the sequence to be compared is merged into the target cluster according to the first alignment result.

[0117] It is noteworthy that the embodiments of this application provide a complete processing flow of monitoring data from camera acquisition to merging into the database. The XGBoost comprehensive decision-making scheme, utilizing the extreme gradient booster, achieves comprehensive utilization of multimodal information. The ReID streaming merging scheme fully leverages the streaming reporting characteristics of target object data in the monitoring system. By caching a clustering pool, newly entered sequences to be compared only require similarity calculation at the cluster level during clustering. Real-time merging of target object data is achieved through incremental clustering, exhibiting unique advantages such as low algorithm complexity, low computational overhead, and high operating efficiency. This enables the digitization and structuring of personnel identity and trajectory data throughout the entire process, effectively meeting the technical requirements of comprehensive data utilization and fast system operation faced by intelligent monitoring systems during implementation. This promotes the better implementation of target object re-identification technology, leading to better business results.

[0118] Therefore, the embodiments of this application achieve the goal of accurately associating cross-border head objects and real-time merging of monitoring and reporting data, thereby realizing the technical effect of providing more efficient and higher-precision intelligent monitoring applications, facilitating the optimization, upgrading and continuous iteration of intelligent monitoring systems, and solving the technical problem that existing target object re-identification schemes cannot achieve accurate cross-border head object association and real-time merging of target object data.

[0119] It should be noted that the preferred implementation of this embodiment can be found in the relevant description in Embodiment 1, and will not be repeated here.

[0120] Example 4

[0121] According to an embodiment of this application, an embodiment of a computer terminal is also provided. This computer terminal can be any one of a group of computer terminal devices. Optionally, in this embodiment, the aforementioned computer terminal can also be replaced with a mobile terminal or other terminal device.

[0122] Optionally, in this embodiment, the computer terminal may be located in at least one of a plurality of network devices in a computer network.

[0123] In this embodiment, the computer terminal can execute the program code for the following steps in the data processing method: acquiring multimodal data, wherein the multimodal data is determined by image information acquired by the image acquisition device and the attribute information of the image acquisition device itself; performing feature extraction processing on the multimodal data to obtain a first alignment feature of the sequence to be compared, wherein the sequence to be compared is a target object tracking sequence acquired by the image acquisition device; selecting multiple clusters based on the first alignment feature to obtain a second alignment feature of each of the multiple clusters, wherein each of the multiple clusters includes at least one sequence belonging to the same target object; performing a first similarity comparison processing on the first alignment feature and the second alignment feature to obtain a first alignment result; and merging the sequence to be compared into the target cluster according to the first alignment result.

[0124] Optionally, Figure 6 This is a structural block diagram of another computer terminal according to an embodiment of this application, such as... Figure 6 As shown, the computer terminal may include: one or more (only one is shown in the figure) processors 602, memory 604, and peripheral interfaces 606.

[0125] The memory can be used to store software programs and modules, such as the program instructions / modules corresponding to the data processing method and apparatus in this application embodiment. The processor executes various functional applications and data processing by running the software programs and modules stored in the memory, thereby implementing the aforementioned data processing method. The memory may include high-speed random access memory, and may also include non-volatile memory, such as one or more magnetic storage devices, flash memory, or other non-volatile solid-state memory. In some instances, the memory may further include memory remotely located relative to the processor, and these remote memories can be connected to a computer terminal via a network. Examples of such networks include, but are not limited to, the Internet, corporate intranets, local area networks, mobile communication networks, and combinations thereof.

[0126] The processor can invoke information and application programs stored in the memory via a transmission device to perform the following steps: acquiring multimodal data, wherein the multimodal data is determined by image information acquired by an image acquisition device and attribute information of the image acquisition device itself; performing feature extraction processing on the multimodal data to obtain a first alignment feature of the sequence to be compared, wherein the sequence to be compared is a target object tracking sequence acquired by the image acquisition device; selecting multiple clusters based on the first alignment feature to obtain a second alignment feature for each of the multiple clusters, wherein each of the multiple clusters includes at least one sequence belonging to the same target object; performing a first similarity comparison processing on the first alignment feature and the second alignment feature to obtain a first alignment result; and merging the sequence to be compared into the target cluster according to the first alignment result.

[0127] Optionally, the processor described above can also execute program code for any of the method steps provided in Embodiment 1 of this application, as described in the relevant description in Embodiment 1, which will not be repeated here.

[0128] This application provides a data processing scheme. It involves acquiring multimodal data, wherein the multimodal data is determined by image information acquired by an image acquisition device and the attribute information of the image acquisition device itself; performing feature extraction processing on the multimodal data to obtain a first alignment feature of the sequence to be compared, wherein the sequence to be compared is a target object tracking sequence acquired by the image acquisition device; selecting multiple clusters based on the first alignment feature to obtain a second alignment feature for each of the multiple clusters, wherein each of the multiple clusters includes at least one sequence belonging to the same target object; performing a first similarity comparison processing on the first alignment feature and the second alignment feature to obtain a first alignment result; and merging the sequence to be compared into the target cluster according to the first alignment result.

[0129] It is noteworthy that the embodiments of this application provide a complete processing flow of monitoring data from camera acquisition to merging into the database. The XGBoost comprehensive decision-making scheme, utilizing the extreme gradient booster, achieves comprehensive utilization of multimodal information. The ReID streaming merging scheme fully leverages the streaming reporting characteristics of target object data in the monitoring system. By caching a clustering pool, newly entered sequences to be compared only require similarity calculation at the cluster level during clustering. Real-time merging of target object data is achieved through incremental clustering, exhibiting unique advantages such as low algorithm complexity, low computational overhead, and high operating efficiency. This enables the digitization and structuring of personnel identity and trajectory data throughout the entire process, effectively meeting the technical requirements of comprehensive data utilization and fast system operation faced by intelligent monitoring systems during implementation. This promotes the better implementation of target object re-identification technology, leading to better business results.

[0130] Therefore, the embodiments of this application achieve the goal of accurately associating cross-border head objects and real-time merging of monitoring and reporting data, thereby realizing the technical effect of providing more efficient and higher-precision intelligent monitoring applications, facilitating the optimization, upgrading and continuous iteration of intelligent monitoring systems, and solving the technical problem that existing target object re-identification schemes cannot achieve accurate cross-border head object association and real-time merging of target object data.

[0131] Those skilled in the art will understand that Figure 6 The structure shown is for illustrative purposes only. The computer terminal can also be a smartphone (such as an Android phone, an iOS phone, etc.), a tablet computer, a mobile internet device (MID), a PAD, and other terminal devices. Figure 6 This does not limit the structure of the aforementioned electronic devices. For example, a computer terminal may also include components that are more... Figure 6 The more or fewer components shown (such as network interfaces, display devices, etc.), or having the same Figure 6 The different configurations shown.

[0132] Those skilled in the art will understand that all or part of the steps in the various methods of the above embodiments can be implemented by a program instructing the hardware related to the terminal device. The program can be stored in a computer-readable non-volatile storage medium, which may include: flash drive, read-only memory (ROM), random access memory (RAM), magnetic disk or optical disk, etc.

[0133] Example 5

[0134] According to an embodiment of this application, an embodiment of a non-volatile storage medium is also provided. Optionally, in this embodiment, the non-volatile storage medium includes a stored program, wherein, when the program runs, it controls the device where the non-volatile storage medium is located to execute the data processing method described above.

[0135] Optionally, in this embodiment, the non-volatile storage medium may be located in any computer terminal in a group of computer terminals in a computer network, or in any mobile terminal in a group of mobile terminals.

[0136] Optionally, in this embodiment, the non-volatile storage medium is configured to store program code for performing the following steps: acquiring multimodal data, wherein the multimodal data is determined by image information acquired by the image acquisition device and attribute information of the image acquisition device itself; performing feature extraction processing on the multimodal data to obtain a first alignment feature of the sequence to be compared, wherein the sequence to be compared is a target object tracking sequence acquired by the image acquisition device; selecting multiple clusters based on the first alignment feature to obtain a second alignment feature of each of the multiple clusters, wherein each of the multiple clusters includes at least one sequence belonging to the same target object; performing a first similarity comparison processing on the first alignment feature and the second alignment feature to obtain a first alignment result; and merging the sequence to be compared into the target cluster according to the first alignment result.

[0137] Optionally, in this embodiment, the non-volatile storage medium is configured to store program code for executing any of the method steps provided in Embodiment 1 of this application. Please refer to the relevant description in Embodiment 1, which will not be repeated here.

[0138] The sequence numbers of the embodiments in this application are for descriptive purposes only and do not represent the superiority or inferiority of the embodiments.

[0139] In the above embodiments of this application, the descriptions of each embodiment have different focuses. For parts not described in detail in a certain embodiment, please refer to the relevant descriptions of other embodiments.

[0140] In the several embodiments provided in this application, it should be understood that the disclosed technical content can be implemented in other ways. The device embodiments described above are merely illustrative; for example, the division of units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the displayed or discussed mutual coupling, direct coupling, or communication connection may be through some interfaces; the indirect coupling or communication connection between units or modules may be electrical or other forms.

[0141] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0142] Furthermore, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit.

[0143] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as a USB flash drive, read-only memory (ROM), random access memory (RAM), portable hard drive, magnetic disk, or optical disk.

[0144] The above description is only a preferred embodiment of this application. It should be noted that for those skilled in the art, several improvements and modifications can be made without departing from the principle of this application, and these improvements and modifications should also be considered within the scope of protection of this application.

Claims

1. A data processing method, characterized in that, include: Acquire multimodal data, wherein the multimodal data is determined by image information acquired by the image acquisition device and attribute information of the image acquisition device itself; The multimodal data is subjected to feature extraction processing to obtain the first alignment feature of the alignment sequence to be compared, wherein the alignment sequence is a target object tracking sequence obtained by the image acquisition device; Based on the first alignment feature, multiple clusters are selected to obtain a second alignment feature for each of the multiple clusters, wherein each of the multiple clusters includes at least one sequence belonging to the same target object; A first similarity comparison process is performed on the first comparison feature and the second comparison feature to obtain a first comparison result. The first comparison result is obtained based on a fusion feature, which is obtained by fusing the first comparison feature and the second comparison feature. The fusion feature includes: multiple dimensions of similarity features. The first comparison result includes: multiple predicted probabilities, which are used to represent the probability that the sequence to be compared belongs to each of the multiple clusters. The sequences to be compared are merged into the target cluster according to the first comparison result.

2. The data processing method according to claim 1, characterized in that, Based on the first comparison feature, the multiple clusters are selected to obtain the second comparison feature, which includes: The first comparison feature is compared with the currently cached cluster in the clustering pool to obtain the second comparison result; The second comparison results are sorted according to the similarity from high to low to obtain the sorted results; The sorting results are used to select the multiple clusters from the current cached clusters to obtain the second comparison feature.

3. The data processing method according to claim 1, characterized in that, Merging the sequences to be compared into the target cluster according to the first comparison result includes: When some or all of the multiple predicted probabilities are greater than or equal to a first preset threshold, the cluster with the highest predicted probability among the multiple predicted probabilities is determined as the target cluster; The sequences to be compared are merged into the target cluster.

4. The data processing method according to claim 3, characterized in that, Merging the sequences to be compared into the target cluster according to the first comparison result includes: When all of the predicted probabilities are less than the first preset threshold, the target cluster is created for the sequence to be compared. The sequences to be compared are merged into the target cluster.

5. The data processing method according to claim 1, characterized in that, The data processing method further includes: When it is determined that multiple sequences to be compared obtained at the same time all belong to the target cluster, the sequence to be compared with the highest predicted probability is determined from the multiple sequences to be compared. The sequence to be compared with the highest predicted probability is merged into the target cluster, and new clusters are re-determined for the remaining sequences to be compared, except for the sequence to be compared with the highest predicted probability, until all the sequences to be compared are merged into the corresponding clusters.

6. The data processing method according to claim 1, characterized in that, The data processing method further includes: In the clusters currently cached in the clustering pool, the target cluster is compared with neighboring clusters using the first similarity comparison process to obtain the third comparison result. When the similarity between the target cluster and the adjacent cluster is determined to be greater than the second preset threshold based on the third comparison result, the target cluster and the adjacent cluster are aggregated.

7. The data processing method according to claim 1, characterized in that, The data processing method further includes: Record the target duration of the target cluster, wherein the target duration is the duration during which the target cluster has not been updated; When the target duration exceeds a third preset threshold, the target cluster is removed from the clustering pool and the target object is subjected to behavioral analysis based on the target cluster.

8. A non-volatile storage medium, characterized in that, The non-volatile storage medium includes a stored program, wherein, when the program is executed, it controls the device containing the non-volatile storage medium to perform the data processing method according to any one of claims 1 to 7.

9. An electronic device, characterized in that, include: processor; as well as A memory, connected to the processor, for providing the processor with instructions to perform the following processing steps: Acquire multimodal data, wherein the multimodal data is determined by image information acquired by the image acquisition device and attribute information of the image acquisition device itself; The multimodal data is subjected to feature extraction processing to obtain the first alignment feature of the alignment sequence to be compared, wherein the alignment sequence is a target object tracking sequence obtained by the image acquisition device; Based on the first alignment feature, multiple clusters are selected to obtain a second alignment feature for each of the multiple clusters, wherein each of the multiple clusters includes at least one sequence belonging to the same target object; A first similarity comparison process is performed on the first comparison feature and the second comparison feature to obtain a first comparison result. The first comparison result is obtained based on a fusion feature, which is obtained by fusing the first comparison feature and the second comparison feature. The fusion feature includes: multiple dimensions of similarity features. The first comparison result includes: multiple predicted probabilities, which are used to represent the probability that the sequence to be compared belongs to each of the multiple clusters. The sequences to be compared are merged into the target cluster according to the first comparison result.

Images