Target fusion file gathering method and system based on multi-dimensional analysis
Through the target fusion archive aggregation method based on multi-dimensional analysis, combined with the Infomap community discovery algorithm and face recognition algorithm, cross-domain archive data is structured cleaning, purity management and fusion processing, solving the problem of cross-domain archive data aggregation and fusion , achieving efficient improvement of archive purity and retrieval accuracy.
Patent Information
- Application Number
- CN202411515340.8
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-10-29
- Publication Date
- 2025-02-18
AI Technical Summary
The existing technology is difficult to effectively solve the problem of aggregation and integration of cross-domain archive data, especially in cross-city and time-space-accessible judgments and cross-domain archive scenarios with low accuracy in human data aggregation and merging of cross-domain archive scenarios.
Using the target fusion archive aggregation method based on multi-dimensional analysis, the archive data of the lower platform is unifiedly accessed through the archive data access module, and combined with the Infomap community discovery algorithm and face recognition algorithm, the archive data is structured and cleaned and purified. Governance and integration processing to achieve rapid collision and merger of cross-domain archives.
It effectively improves the purity of archives, solves the problem of "one person and multiple files" in cross-domain archive fusion, improves the accuracy and recall rate of archives, and reduces the diffusion rate of archives.
Smart Images

Figure CN119474012A_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the field of data processing technology, and in particular to a target fusion and archiving method and system based on multidimensional analysis. Background Art
[0002] At present, various places are actively carrying out the construction of video image target aggregation systems, forming a large amount of aggregated data. The use of the view library platform requires the fusion of the target face aggregated data uploaded by various places, and various places also have an urgent need to aggregate and fuse the aggregated data uploaded by the next-level platform. The current archive fusion solution uses the spatiotemporal information (capture time, capture device coordinates) of the archive trajectory (i.e., the captured image data) to detect spatiotemporal contradictions, which strongly relies on the full trajectory information of the archive. However, for the cascade aggregation of cross-domain archive data (i.e., archive data aggregated from different regions to the upper platform), it is often impossible to aggregate all trajectory data upward, and it is difficult to judge the spatiotemporal reachability after crossing cities. Therefore, it is difficult for existing technologies to solve the problem of cross-domain archive governance. In addition, the current face and body association archive solution has a low accuracy rate for the aggregation and merging of human body data for cross-domain archive scenarios, making it difficult to merge cross-domain archives. Summary of the invention
[0003] In view of the deficiencies in the prior art, the present invention aims to provide a target fusion and archiving method and system based on multi-dimensional analysis.
[0004] In order to achieve the above object, the present invention adopts the following technical solution:
[0005] A target fusion and aggregation method based on multidimensional analysis includes the following steps:
[0006] S1, the archive data access module uniformly accesses the archive data of each subordinate platform; each archive data contains cover image data, trajectory image data and the time and space information of each trajectory image, and the cover image data and trajectory image data are both facial images of the person to whom the corresponding archive belongs;
[0007] S2. File structure cleaning:
[0008] The archive structured cleaning module uses an incremental mode to perform structured cleaning on archives. The newly added archive data is first verified by the structured compliance operator. The archive data that fails the verification is directly written into the problem archive table. After the verified archive data is internally connected with the cover image data, the cover image of the archive data is verified by the image quality operator. The archive data with substandard cover image quality is screened out and written into the problem archive table. The archive data that passes the cover image quality detection is transferred to step S3 for purity management.
[0009] S3. Archive purity management:
[0010] The archive purity management module uses the features of the archive cover image and the trajectory images of the sampled archives and the spatiotemporal information of the trajectory images, combined with the Infomap community discovery algorithm and the face recognition algorithm, to analyze the cohesion of the archive data, screen out the archive data with multiple people in one file and write them into the problem archive table, and the remaining normal archives enter the archive fusion processing of step S4; multiple people in one file means that there are multiple different target persons in one target archive;
[0011] S4. Archives Integration:
[0012] The archive fusion module is based on adaptive file collision logic. It collides anonymous files with real-name files, and anonymous files with anonymous files, to dig out files with multiple files for one person. These files are written into merged files and the source of the files is retained for file tracing. Multiple files for one person means that the file subjects of multiple files are the same target person.
[0013] Furthermore, in step S1, the archive data access module performs hierarchical decoupling of access through the Kafka cluster, and the archive data of the lower-level platform is pushed to the Kafka cluster. The data extraction service developed based on Flink CDC pushes the archive data in the Kafka cluster to the upper-level platform in real time and incrementally, forming the original archive data on the upper-level platform.
[0014] Furthermore, in step S2, the contents verified by the structured compliance operator include whether the certificate number is abnormal, image availability and clock accuracy.
[0015] Furthermore, in step S2, the archive structured cleaning module supports structured cleaning of the stock archive data, and performs tidal scheduling on the structured cleaning tasks of the stock archive data through the scheduling engine.
[0016] Furthermore, in step S3, the Spark parallel computing engine is used to encapsulate the Infomap community discovery algorithm and the face recognition algorithm into Spark UDF operators, and the parallel computing capability of Spark is used to perform the task of archive purity management.
[0017] Furthermore, the specific process of step S3 is:
[0018] After analyzing the cover image and trajectory data of the archives after structured cleaning, the archive purity management module samples and extracts some historical trajectory data of the archives, calls the face recognition algorithm to calculate the similarity between different trajectory images in the archives and between the trajectory image and the cover image, and inputs the image ID and similarity of the trajectory image and the cover image into the Infomap community discovery algorithm. The image ID is used as the node and the similarity is used as the edge. The groups are divided according to the archive ID. The data with different archive IDs are different groups. By analyzing the flow of the archive cover image data and trajectory image data in the community network, the abnormal trajectory data that does not belong to the archive is identified. Archives with abnormal trajectory data are judged as archive data with multiple people in one file, and they are written into the problem archive table. Archives without abnormal trajectory data are normal archives and enter the subsequent archive fusion processing.
[0019] Furthermore, in step S4, the archive fusion algorithm based on the adaptive inter-gear collision logic is encapsulated as a SparkUDF operator, and the archive fusion calculation is performed using the Spark parallel computing capability.
[0020] Furthermore, the specific process of step S4 is:
[0021] The archive fusion module calls the UDF operator encapsulated by the 1:N face comparison algorithm on the archive data that has undergone purity management to identify the top N archives whose cover images are most similar to the cover image of the target archive. The cover image of the target archive is subjected to adaptive inter-archive collision with the cover images of the identified Top N archives and part of the trajectory image to obtain the overlap, and the obtained overlaps are sorted. The archive corresponding to the highest overlap that exceeds the preset overlap threshold is taken out, and it is considered that this archive has a merge relationship with the target archive. For archives that have a merge relationship, a merged archive list is output and fed back to the lower-level platform to help the lower-level platform reduce the clustering diffusion rate.
[0022] Furthermore, the above method also includes: providing multimodal retrieval, relationship analysis, and label analysis capabilities through a multimodal target archiving application module; the multimodal target archiving application module provides archive retrieval functions based on a multimodal retrieval method, and supports the selection of time intervals and location information while inputting images, and integrates features with structured information for retrieval through a vector search engine; the multimodal target archiving application module also uses the parallel computing engine capabilities to analyze and mine the correct archives and trajectory data after governance, discover possible companion, peer, and group relationships between archives, and save the relationship data to form an archive label library, thereby providing relationship analysis and label analysis capabilities.
[0023] The present invention also provides a target fusion archive system based on multidimensional analysis for implementing the above method, comprising an archive data access module, an archive structured cleaning module, an archive purity management module and an archive fusion module;
[0024] The archive data access module is used to aggregate archive data from lower-level platforms;
[0025] The archive structured cleaning module is used to verify the archive data collected by the archive data access module in terms of whether the certificate number is abnormal, image availability, image quality and clock accuracy. The archive data that passes the verification will enter the subsequent purity management, and the archive data that fails the verification will be written into the problem archive table and fed back to the corresponding lower-level platform;
[0026] The archive purity management module is used to perform purity management on the archive data that has undergone structured cleaning. It uses the cover image features of the archive, the trajectory image data of a sampled portion of the archive, and the spatiotemporal information of the trajectory image data, combined with the Infomap community discovery algorithm and face recognition algorithm, to analyze the cohesion of the archive data, identify archives with multiple people in one file and write them into the problem archive table, and continue with the subsequent archive fusion for normal archives.
[0027] The file fusion module is used for the file fusion algorithm based on the adaptive file collision logic, which collides anonymous files with real-name files and anonymous files with anonymous files, digs out files with multiple files for one person, writes the files with multiple files for one person into the merged file and retains the file source.
[0028] The beneficial effects of the present invention are:
[0029] 1. The file purity management of the present invention is based on the Infomap community discovery algorithm and face recognition algorithm, which analyzes the image features, time, space, etc. of the cover image of the file and the sampled trajectory image data, and mines out files that are non-compliant and have problems with multiple people sharing one file, effectively improving the purity of the file.
[0030] 2. The cross-domain archive fusion of the present invention is implemented based on adaptive inter-file collision logic, which quickly collides the accessed archives, discovers multiple archives belonging to the same person and merges them, solving the problem of "one person with multiple archives" generated in the cross-domain cascade aggregation of archives.
[0031] 3. The present invention provides a multimodal archive search module, which is based on a vector search engine and incorporates structured retrieval information while performing image search, thereby improving the accuracy and recall rate of archive retrieval. BRIEF DESCRIPTION OF THE DRAWINGS
[0032] Figure 1 This is a flow chart of the implementation of the method of embodiment 1 of the present invention;
[0033] Figure 2 This is a structured cleaning flow chart of the method in Example 1 of the present invention;
[0034] Figure 3 This is a flow chart of file fusion in the method of Example 1 of the present invention. DETAILED DESCRIPTION
[0035] The present invention will be further described below in conjunction with the accompanying drawings. It should be noted that this embodiment is based on the technical solution and provides a detailed implementation method and specific operation process, but the protection scope of the present invention is not limited to this embodiment.
[0036] Example 1
[0037] This embodiment provides a target fusion and file aggregation method based on multi-dimensional analysis, which is applied to cross-domain file data management. It splits and merges specific target files through multi-dimensional analysis such as target portrait feature vector similarity calculation, spatiotemporal verification, and relationship analysis. Figure 1 As shown, the specific implementation process is as follows:
[0038] S1. The archive data access module provides unified access to multi-source archive data.
[0039] Specifically, the cluster data access module performs unified access to the standardized cluster data of multiple lower-level platforms in accordance with the requirements of the "Technical Requirements for Video Image Target Clustering Services".
[0040] In this embodiment, the archive data access module performs hierarchical decoupling of access through the Kafka cluster, and the archive data of the lower-level platform is pushed to the Kafka cluster. The data extraction service developed based on Flink CDC pushes the archive data in the Kafka cluster to the upper-level platform in real time and incrementally, forming the original archive data on the upper-level platform; each archive data contains cover image data, trajectory image data and the spatiotemporal information of each trajectory image, and the cover image data and trajectory image data are both facial images of the person to whom the corresponding archive belongs.
[0041] S2. Structural cleaning of archives.
[0042] In order to more effectively carry out subsequent tasks such as file purity management and reduce invalid file data, the file structured cleaning module first performs structured cleaning on the accessed file data. Figure 2 shown.
[0043] In this embodiment, the archive structured cleaning module adopts an incremental mode to perform structured cleaning on the archives. The newly added archive data every day is first verified by the structured compliance operator in terms of whether the certificate number is abnormal, image availability, clock accuracy, etc. The archive data that fails the verification is directly written into the problem archive table. After the verified archive data is internally connected with the cover image data, the cover image of the archive data is verified by the image quality operator. The archive data with non-compliant cover image quality is screened out and written into the problem archive table. The archive data that passes the cover image quality verification enters the subsequent purity management.
[0044] In addition to the incremental mode, the archive structured cleaning module also supports structured cleaning of existing archive data. However, due to the large amount of historical archive data, in order to ensure the resources for daily incremental archive cleaning tasks, the archive structured cleaning module uses the scheduling engine to perform tidal scheduling on the structured cleaning tasks of existing archive data to effectively utilize computing resources.
[0045] S3. Archive purity management.
[0046] Due to the influence of factors such as camera capture angle, occlusion, and lighting, there may be mis-aggregation problems, resulting in the presence of multiple people in one file in the archive data aggregated to the upper-level platform, that is, there are multiple different target persons in one target file, affecting the application effect of the aggregation file. The method in this embodiment solves the above problem through archive purity management.
[0047] The archive purity management module uses the features of the archive cover image, the trajectory images of the sampled part of the archive, and the spatiotemporal information of the trajectory images, combined with the Infomap community discovery algorithm and the face recognition algorithm, to analyze the cohesion of the archive data. Specifically, in this embodiment, the Spark parallel computing engine is used to encapsulate the Infomap community discovery algorithm and the face recognition algorithm as Spark UDF operators, and the parallel computing capability of Spark is used to realize the analysis of the cohesion of the archive data.
[0048] Infomap is mainly used for community discovery. In the process of community discovery, it can identify abnormal points that do not conform to the overall structure. In the method of this embodiment, the purity of archive data is managed based on the Infomap community discovery algorithm, and abnormal trajectory points in the archive data are mined.
[0049] Specifically, after parsing the cover image and trajectory data of the archives after structured cleaning, the archive purity management module samples and extracts part of the historical trajectory data of the archives, calls the face recognition algorithm to calculate the similarity between different trajectory images in the archives and between the trajectory image and the cover image, and inputs the image ID (a field that identifies the uniqueness of the trajectory image or cover image) and the similarity of the trajectory image and the cover image into the Infomap community discovery algorithm. The image ID is used as the node and the similarity is used as the edge. The groups are divided according to the archive ID (a field that identifies the uniqueness of the archive). Data with different archive IDs are different groups. By analyzing the flow of the archive cover image data and trajectory image data in the community network, abnormal trajectory data that does not belong to the archive is identified. Archives with abnormal trajectory data are judged as archive data with multiple people in one file, and they are written into the problem archive table. Archives without abnormal trajectory data are normal archives and enter the subsequent archive fusion processing.
[0050] In this embodiment, the file purity management is implemented based on Spark development and can be submitted to the Yarn cluster for execution, so as to quickly discover problematic files with multiple people sharing one file and improve the accuracy of the files.
[0051] S4. Archive integration.
[0052] In the method of this embodiment, the archive fusion module collides anonymous files with real-name files and anonymous files with anonymous files based on the adaptive file collision logic, quickly digs out files with multiple files for one person, that is, the file subjects of multiple files are the same target person, and writes the files with multiple files for one person into the merged file and retains the file source, which is convenient for file tracing. Through file fusion, the file diffusion rate is reduced. In this embodiment, the file fusion algorithm based on the adaptive file collision logic is encapsulated as a Spark UDF operator, and the parallel computing capability of Spark is used to perform file fusion calculations.
[0053] In the past, the idea of realizing file fusion was often to find out whether there was a merger based on the collision of the file cover images. However, the reliability of the mining results of this method was not high. The reason was that this method was strictly limited by the quality of the file cover images, and the representation ability of the deep features was limited by the training data. There was a problem of insufficient representation of the real data. It was difficult to judge whether two cover images belonged to the same category based on their deep features. Therefore, the method of this embodiment adopted adaptive file collision to realize file fusion, such as Figure 3 As shown, the specific process is:
[0054] First, the archive fusion module calls the UDF operator encapsulated by the 1:N face comparison algorithm on the archive data that has undergone purity management to identify the top N archives whose cover images are most similar to the cover image of the target archive. The cover image of the target archive is subjected to adaptive inter-file collision with the cover images of the identified Top N archives and part of the trajectory image to obtain the overlap, and the obtained overlaps are sorted. The archive corresponding to the highest overlap that exceeds the preset overlap threshold is taken out, and it is considered that this archive has a merge relationship with the target archive.
[0055] For archives with merged relationships, a merged archive list is output and fed back to the lower-level platform to help the lower-level platform reduce the cluster diffusion rate.
[0056] In this embodiment, after the upper-level platform accesses the aggregated archive data of multiple lower-level platforms (such as multiple cities), the aggregated archive data volume of the upper-level platform can reach tens of millions, and the archive track volume exceeds 100 million. In addition, archive management needs to rely on face recognition algorithms and graph analysis algorithms. Therefore, it is necessary to build an AI parallel computing engine layer, encapsulate the algorithm as the engine algorithm, and use the parallel computing capabilities of the engine to accelerate the execution of archive purity management and archive fusion to meet archive management needs. The AI parallel computing engine layer is composed of graph computing algorithms such as the offline batch processing framework Spark, the Yarn resource scheduling engine, the face recognition algorithm, and the Infomap community discovery algorithm.
[0057] The Yarn resource scheduling engine is a resource manager in the Hadoop ecosystem, which is responsible for resource allocation and management in the Yarn cluster. In the Yarn cluster, resource isolation is mainly achieved by allocating independent containers to each application, and each container has fixed memory, CPU and other resources. In this embodiment, archive purity management and archive fusion are both implemented based on Spark. By encapsulating the face recognition algorithm and the Infomap community discovery algorithm as UDF operators, the face recognition algorithm and the Infomap community discovery algorithm can be called through Spark, and the task is submitted to the Yarn cluster, and the Yarn cluster allocates corresponding resources to ensure efficient and reliable execution of the task.
[0058] In this embodiment, the multimodal target aggregation application module provides the capabilities of multimodal retrieval, relationship analysis, and label analysis.
[0059] Traditional archive retrieval is mostly based on image search, but this method is limited by the representation ability of single-point image features, and the accuracy of the search results is not high. In the method of this embodiment, the multimodal target archiving application module provides archive retrieval functions based on multimodal retrieval methods. When inputting images, it supports the selection of time intervals and location information, etc. Through the vector search engine, the features and structured information are integrated for retrieval, thereby improving the accuracy of the retrieval.
[0060] The multimodal target aggregation application module also uses the parallel computing engine capabilities to analyze and mine the correct archives and trajectory data after governance, discover possible companion, peer, group and other relationships between archives, and save the relationship data to form an archive label library, thereby providing relationship analysis and label analysis capabilities.
[0061] Example 2
[0062] This embodiment provides a target fusion archive system based on multidimensional analysis of the method described in Embodiment 1, including an archive data access module, an archive structured cleaning module, an archive purity management module and an archive fusion module;
[0063] The archive data access module is used to aggregate archive data from lower-level platforms;
[0064] The archive structured cleaning module is used to verify the archive data gathered by the archive data access module in terms of whether the certificate number is abnormal, image availability, image quality, clock accuracy, etc. The archive data that passes the verification will enter the subsequent purity management, and the archive data that fails the verification will be written into the problem archive table and fed back to the corresponding lower-level platform.
[0065] The archive purity management module is used to perform purity management on the archive data that has undergone structured cleaning. It uses the cover image features of the archive, the trajectory image data of a sampled portion of the archive, and the spatiotemporal information of the trajectory image data, combined with the Infomap community discovery algorithm and face recognition algorithm, to analyze the cohesion of the archive data, identify archives with multiple people in one file and write them into the problem archive table, and continue with the subsequent archive fusion for normal archives.
[0066] The archive fusion module is used for the archive fusion algorithm based on adaptive inter-file collision logic. It collides anonymous files with real-name files, and anonymous files with anonymous files, to mine out archives with multiple files for one person, and writes these archives into a merged file while retaining the source of the archives, to facilitate archive tracing. Through archive fusion, the diffusion rate of archives in the system can be reduced.
[0067] For those skilled in the art, various corresponding changes and modifications can be made according to the above technical solutions and concepts, and all of these changes and modifications should be included in the protection scope of the claims of the present invention.
Claims
1. A target fusion and aggregation method based on multidimensional analysis, characterized in that: The steps include: S1, the archive data access module uniformly accesses the archive data of each subordinate platform; each archive data contains cover image data, trajectory image data and the time and space information of each trajectory image, and the cover image data and trajectory image data are both facial images of the person to whom the corresponding archive belongs; S2. File structure cleaning: The archive structured cleaning module uses an incremental mode to perform structured cleaning on archives. The newly added archive data is first verified by the structured compliance operator. The archive data that fails the verification is directly written into the problem archive table. After the verified archive data is internally connected with the cover image data, the cover image of the archive data is verified by the image quality operator. The archive data with substandard cover image quality is screened out and written into the problem archive table. The archive data that passes the cover image quality detection is transferred to step S3 for purity management. S3. Archive purity management: The archive purity management module uses the features of the archive cover image and the trajectory images of the sampled archives and the spatiotemporal information of the trajectory images, combined with the Infomap community discovery algorithm and the face recognition algorithm, to analyze the cohesion of the archive data, screen out the archive data with multiple people in one file and write them into the problem archive table, and the remaining normal archives enter the archive fusion processing of step S4; multiple people in one file means that there are multiple different target persons in one target archive; S4. Archives Integration: The archive fusion module is based on adaptive file collision logic. It collides anonymous files with real-name files, and anonymous files with anonymous files, to dig out files with multiple files for one person. These files are written into merged files and the source of the files is retained for file tracing. Multiple files for one person means that the file subjects of multiple files are the same target person.
2. The method according to claim 1, characterized in that In step S1, the archive data access module performs hierarchical decoupling of access through the Kafka cluster, and the archive data of the lower-level platform is pushed to the Kafka cluster. The data extraction service developed based on Flink CDC pushes the archive data in the Kafka cluster to the upper-level platform in real time and incrementally, forming the original archive data on the upper-level platform.
3. The method according to claim 1, characterized in that: In step S2, the contents verified by the structured compliance operator include whether the certificate number is abnormal, image availability and clock accuracy.
4. The method according to claim 1, characterized in that: In step S2, the archive structured cleaning module supports structured cleaning of the existing archive data, and performs tidal scheduling of the structured cleaning tasks of the existing archive data through the scheduling engine.
5. The method according to claim 1, characterized in that In step S3, the Spark parallel computing engine is used to encapsulate the Infomap community discovery algorithm and face recognition algorithm into Spark UDF operators, and the parallel computing capability of Spark is used to perform the task of archive purity management.
6. The method according to claim 1 or 5, characterized in that: The specific process of step S3 is: After analyzing the cover image and trajectory data of the archives after structured cleaning, the archive purity management module samples and extracts some historical trajectory data of the archives, calls the face recognition algorithm to calculate the similarity between different trajectory images in the archives and between the trajectory image and the cover image, and inputs the image ID and similarity of the trajectory image and the cover image into the Infomap community discovery algorithm. The image ID is used as the node and the similarity is used as the edge. The groups are divided according to the archive ID. The data with different archive IDs are different groups. By analyzing the flow of the archive cover image data and trajectory image data in the community network, the abnormal trajectory data that does not belong to the archive is identified. Archives with abnormal trajectory data are judged as archive data with multiple people in one file, and they are written into the problem archive table. Archives without abnormal trajectory data are normal archives and enter the subsequent archive fusion processing.
7. The method according to claim 1, characterized in that In step S4, the file fusion algorithm based on the adaptive file collision logic is encapsulated as a Spark UDF operator, and the file fusion calculation is performed using the Spark parallel computing capability.
8. The method according to claim 1, characterized in that The specific process of step S4 is: The archive fusion module calls the UDF operator encapsulated by the 1:N face comparison algorithm on the archive data that has undergone purity management to identify the top N archives whose cover images are most similar to the cover image of the target archive. The cover image of the target archive is subjected to adaptive inter-archive collision with the cover images of the identified Top N archives and part of the trajectory image to obtain the overlap, and the obtained overlaps are sorted. The archive corresponding to the highest overlap that exceeds the preset overlap threshold is taken out, and it is considered that this archive has a merge relationship with the target archive. For archives that have a merge relationship, a merged archive list is output and fed back to the lower-level platform to help the lower-level platform reduce the clustering diffusion rate.
9. The method according to claim 1, characterized in that: Also includes: The multimodal target archive application module provides the capabilities of multimodal retrieval, relationship analysis, and label analysis. The multimodal target archive application module provides archive retrieval based on the multimodal retrieval method. When inputting images, it supports the selection of time intervals and location information, and integrates features with structured information for retrieval through a vector search engine. The multimodal target aggregation application module also uses the parallel computing engine capabilities to analyze and mine the correct archives and trajectory data after governance, discover possible companion, peer and group relationships between archives, and save the relationship data to form an archive label library, thereby providing relationship analysis and label analysis capabilities.
10. A target fusion and archiving system based on multidimensional analysis for implementing any of the methods described in claims 1-9, characterized in that: It includes archive data access module, archive structure cleaning module, archive purity management module and archive fusion module; The archive data access module is used to aggregate archive data from lower-level platforms; The archive structured cleaning module is used to verify the archive data collected by the archive data access module in terms of whether the certificate number is abnormal, image availability, image quality and clock accuracy. The archive data that passes the verification will enter the subsequent purity management, and the archive data that fails the verification will be written into the problem archive table and fed back to the corresponding lower-level platform; The archive purity management module is used to manage the purity of archive data that has been cleaned by structured cleaning. It uses the cover image features of the archive, the trajectory image data of a sampled part of the archive, and the spatiotemporal information of the trajectory image data, combined with the Infomap community discovery algorithm and the face recognition algorithm, to analyze the cohesion of the archive data, identify archives with multiple people in one file and write them into the problem archive table, and continue to perform subsequent archive fusion for normal archives; The file fusion module is used for the file fusion algorithm based on the adaptive file collision logic, which collides anonymous files with real-name files and anonymous files with anonymous files, digs out files with multiple files for one person, writes the files with multiple files for one person into the merged file and retains the file source.
Citation Information
Cited By
One-person, one-file and one-attribution cleaning and attribution method for historical repeated archives of crowds in super-large area
CN120047294A