Data deduplication processing method and system based on face recognition equipment, PC terminal and medium

Through real-time analysis and local comparison, automatic deduplication processing of facial recognition data is solved, data redundancy problem in public places with large traffic is achieved, efficient deduplication and data integrity are achieved, and it is suitable for multi-equipment systems, reducing costs and Redundant data storage.

CN119003494AInactive Publication Date: 2024-11-22JASEN DEV CO LTD ZHUHAI
View PDF 4 Cites 0 Cited by

Patent Information

Application Number
CN202410778511.X
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2024-06-17
Publication Date
2024-11-22
Estimated Expiration
Not applicable · inactive patent

AI Technical Summary

Technical Problem

The facial recognition data processing methods in the prior art cannot be effectively deduplicated, resulting in an increase in redundant data in the system and cannot be widely used in public places with large traffic, resulting in high communication, labor and server costs.

Method used

Through real-time analysis and monitoring of video content, use local personnel database for comparison, automatically deduplicate known personnel data, upload unknown personnel data to the server to establish archives, and update the equipment library simultaneously to avoid duplicate data generation, and use efficient data storage and Synchronization mechanism ensures data integrity and traceability.

Benefits of technology

Realize real-time, accuracy, and efficient deduplication of data, save resources and costs, is suitable for multi-equipment systems, supports personnel management and trajectory analysis, reduces duplicate data storage and transmission, and improves the scalability and flexibility of the system. .

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN119003494A_ABST
    Figure CN119003494A_ABST
Patent Text Reader

Abstract

The invention provides a data deduplication processing method and system based on face recognition equipment, a PC terminal and a medium. The method comprises the following steps: performing real-time analysis on monitoring video content; comparing an analysis and identification result with a local personnel library of the equipment; if the comparison is successful, local duplicate removal processing is automatically carried out; if the comparison fails, the captured photo is collected and uploaded to a server, a personnel file is newly created for the personnel, and the personnel file is synchronously updated to other equipment personnel libraries in the network; when other equipment identifies the person again, a data deduplication method in the equipment is used, so that repeated data is avoided; all data are summarized and uploaded to a server, and the server stores a data link with time, position and personnel information for analyzing, tracking and generating personnel action tracks. The method has the characteristics of real-time performance, accuracy, high de-duplication efficiency, expandability and flexibility, resource and cost saving and the like, and has wide application prospects in the fields of video monitoring, personnel management and the like.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention relates to the field of monitoring and identification technology, and in particular to a data deduplication processing method, system, PC terminal and computer-readable storage medium based on a face recognition device. Background Art

[0002] The facial recognition data processing methods in the prior art include: obtaining the recognition result data of the front-end facial recognition equipment developed by major manufacturers but having no practical operational practicality after saving; only the internally maintained personnel database can realize the automatic deduplication function, and the stranger function cannot be maintained manually; the analysis results only save the data without classification, aggregation or other operations; even if the personnel in the personnel database are successfully matched, the remaining strangers still have a large amount of duplicate data, which will lead to more and more useless redundant data in the system.

[0003] Currently, there is a need for a data processing system based on a method for deduplication of data from a front-end face recognition device to efficiently process and summarize the recognition data.

[0004] Through the above analysis, the problems and defects of the prior art are as follows:

[0005] The existing communication equipment cannot be widely used in other public information platforms that need to handle large traffic flows, resulting in high communication costs, labor costs, and server storage costs. Summary of the invention

[0006] In order to solve the problems existing in the prior art, the purpose of the present invention is to provide a data deduplication processing method, system, PC terminal and medium based on face recognition equipment. The method has the characteristics of real-time, accuracy, high deduplication efficiency, scalability and flexibility, data integrity and traceability, resource and cost saving, etc., which makes the method have broad application prospects in video surveillance, personnel management and other fields.

[0007] The present invention achieves the above-mentioned purpose through the following technical solutions:

[0008] A data deduplication processing method based on a face recognition device, the method comprising the following steps:

[0009] Conduct real-time analysis of surveillance video content to obtain analysis and identification results;

[0010] Using the analysis and recognition result to compare with a local personnel database of the device, the local personnel database contains known or registered personnel information;

[0011] If the match is successful, that is, the identified person is already in the local personnel database, local deduplication processing will be automatically performed and no new data record will be generated for the person;

[0012] If the comparison fails, the captured photos will be collected and uploaded to the server, a new personnel file will be created for the person on the server, and the files will be updated synchronously to the personnel databases of other devices in the network;

[0013] When other devices in the network recognize the person again, they use the data deduplication method inside the device, that is, compare it with the updated personnel database to avoid generating duplicate data;

[0014] All data will be aggregated and uploaded to the server, which will save a data chain with time, location and personnel information. These data chains will be used to analyze, track and generate personnel movement trajectories.

[0015] According to a data deduplication processing method based on a face recognition device provided by the present invention, a face image is captured by using the internal video recording of a front-end face recognition device for analysis and recognition, and the extracted face features are compared with the local personnel database of the device;

[0016] Calculate the similarity between the recognized face and the faces in the database, and determine whether the recognized face belongs to a registered person in the local database based on the preset matching threshold; if the similarity exceeds the threshold, it is considered a successful match, that is, the recognized person is a registered person; if the similarity is lower than the threshold, it is considered a failed match, that is, the recognized person is not a registered person;

[0017] Before uploading the recognition results to the server, the local data is first deduplicated, and then the deduplicated data records are uploaded to the server and saved.

[0018] According to a data deduplication processing method based on a face recognition device provided by the present invention, when performing deduplication processing, for the successfully matched person, it is checked whether he already exists in the recognized records. If the same recognition record exists, deduplication processing is performed. If the time interval is less than a certain threshold and the location information is similar, these records are considered to be duplicates, and these duplicate records are removed, and only the latest or most complete records are retained; alternatively, it is determined whether it is a repeated recognition of the same location based on the location information.

[0019] According to a data deduplication processing method based on a face recognition device provided by the present invention, when performing deduplication processing, if the person is an unregistered or unknown person, the face photo taken for the first time is used as the identification standard and saved as a new person. After successful saving, the person information is sent to all devices and the person database is updated; if the same face photo has been uploaded before, it will not be uploaded again.

[0020] According to a data deduplication method based on a face recognition device provided by the present invention, the deduplication processed data records are packaged, including the latest records of registered persons, the first records of strangers, etc., and the packaged data records are uploaded to a server. After the server receives the uploaded data, it saves it to a designated storage location.

[0021] According to a data deduplication processing method based on a face recognition device provided by the present invention, after the front-end face recognition device recognizes a new person, the person database on the server is updated in real time to ensure that all devices in the network can quickly obtain the latest person information, including:

[0022] Immediately package the captured photos and related information and upload them to the server. After receiving the uploaded data, the server verifies it and uses efficient data storage technology to ensure fast data writing and reading. Establish real-time indexes on the personnel database to speed up the query. Optimize query statements and data structures to reduce query time. The personnel database on the server adds a timestamp to each record. When the device needs to synchronize data, it only requests new or updated records after the timestamp. A version number is assigned to each personnel record. When the device is synchronizing, it only requests data with a version number greater than the local record.

[0023] Before synchronization, the device calculates the hash value of the local personnel database, compares it with the hash value on the server, and only downloads the difference part, and only transmits the difference part of the file instead of the entire file.

[0024] After receiving the data uploaded by the device, the server asynchronously processes the data storage and synchronization tasks to avoid blocking the main thread.

[0025] According to a data deduplication processing method based on a face recognition device provided by the present invention, if the time interval is less than a certain threshold and the location information is similar, then these records are considered to be duplicates, including:

[0026] Set a time threshold, which is used to cover the time interval of a person passing through the identification area normally and can avoid repeated identification due to accidental factors;

[0027] A position threshold is set, which is used to reflect the displacement distance of a person's normal activity range in a short period of time;

[0028] In the system implementation, for the successfully matched persons, the timestamp of their last identification is checked. If the difference between the current time and the timestamp of the last identification is less than the set time threshold, the difference between the newly recorded location information and the previously recorded location information is further checked to see if it is less than the set location threshold. If both conditions are met, it is considered a repeated identification and no new data records are generated.

[0029] It can be seen that compared with the prior art, the present invention has the following beneficial effects:

[0030] 1. Real-time and accuracy: The present invention can analyze the monitoring video content in real time and quickly obtain the analysis and recognition results, thereby ensuring the real-time nature of the data. At the same time, by comparing with the local personnel database, it can accurately identify known or registered personnel, avoiding the generation of duplicate data.

[0031] 2. High deduplication efficiency: Once the identified personnel are already in the local personnel database, the present invention will automatically perform local deduplication processing and no new data records will be generated, which greatly improves the efficiency of data processing. At the same time, the mechanism of creating new personnel files on the server and updating the personnel database of other devices also ensures that the devices in the network can respond quickly, reducing the transmission and storage of duplicate data in the network.

[0032] 3. Scalability and flexibility: This method is applicable to multiple devices in the network, and the personnel files are updated synchronously through the server, ensuring the scalability and flexibility of the entire system. When other devices in the network recognize the person again, they can directly use the updated personnel database for comparison, further improving the recognition efficiency.

[0033] 4. Data integrity and traceability: All data of the present invention will be summarized and uploaded to the server, and the server will save the data chain with time, location and personnel information, which makes the data complete and traceable, facilitating subsequent analysis, tracking and generation of personnel movement trajectories.

[0034] 5. Saving resources and costs: The present invention reduces the generation and storage of duplicate data through deduplication processing, thereby saving storage space and computing resources. At the same time, the method can automatically complete the identification and deduplication process, reducing the investment of human costs.

[0035] A data deduplication processing system based on a face recognition device, characterized in that the system is used to implement the above-mentioned data deduplication processing method based on a face recognition device, comprising:

[0036] The front-end face recognition device module is provided by the front-end face recognition device manufacturer. It compares the actual surveillance video with the personnel information database and performs data deduplication.

[0037] The personnel information synchronization management module is used to use an encrypted channel to receive personnel information after data encryption and protection, and the central server sends it to synchronize all subordinate devices and ensure the success of synchronization;

[0038] The communication record storage module is used to save the communication record information after receiving the information.

[0039] Therefore, the data deduplication processing system and method based on the front-end face recognition device provided by the present invention solves the problem that there is no need to worry about not being able to track strangers in public places with large traffic. The system of the present invention supports online viewing, editing, and cloud storage of various personnel trace data chains, and can be widely used in personnel information management, smart campuses, smart buildings, and smart management information platforms.

[0040] The present invention also provides a PC terminal, which includes a memory and a processor, wherein the memory stores a computer program, and when the computer program is executed by the processor, the processor executes the above-mentioned data deduplication processing method based on the face recognition device.

[0041] The present invention also provides a computer-readable storage medium, which stores a computer program. When the computer program is executed by a processor, the processor executes the above-mentioned data deduplication processing method based on a face recognition device.

[0042] It can be seen that the present invention also provides a PC end and storage medium of a data deduplication processing method based on a face recognition device, which includes: one or more memories and one or more processors. The memory is used to store program codes and intermediate data generated during program operation, storage of model output results, and storage of models and model parameters; the processor is used for processor resources occupied by code operation and multiple processor resources occupied when training models.

[0043] The present invention is further described in detail below in conjunction with the accompanying drawings and specific embodiments. BRIEF DESCRIPTION OF THE DRAWINGS

[0044] Figure 1 It is a flow chart of an embodiment of a data deduplication processing method based on a face recognition device of the present invention.

[0045] Figure 2 It is a schematic diagram of an embodiment of a data deduplication processing system based on a face recognition device of the present invention. DETAILED DESCRIPTION

[0046] In order to make the purpose, technical solution and advantages of the present invention clearer, the technical solution of the present invention will be clearly and completely described below in conjunction with the drawings of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0047] Reference to "embodiments" herein means that a particular feature, structure, or characteristic described in conjunction with the embodiments may be included in at least one embodiment of the present application. The appearance of the phrase in various locations in the specification does not necessarily refer to the same embodiment, nor is it an independent or alternative embodiment that is mutually exclusive with other embodiments. It is explicitly and implicitly understood by those skilled in the art that the embodiments described herein may be combined with other embodiments.

[0048] See also Figure 1 The present invention provides a data deduplication processing method based on a face recognition device, the method comprising the following steps:

[0049] Step S1, performing real-time analysis on the monitoring video content to obtain analysis and recognition results;

[0050] Step S2, using the analysis and recognition result to compare with a local personnel database of the device, where the local personnel database contains known or registered personnel information;

[0051] Step S3: If the comparison is successful, that is, the identified person is already in the local person database, local deduplication processing is automatically performed and no new data record is generated for the person;

[0052] Step S4: if the comparison fails, the captured photos are collected and uploaded to the server, a new personnel file is created for the person on the server, and the files are updated synchronously to the personnel databases of other devices in the network;

[0053] Step S5, when other devices in the network recognize the person again, the data deduplication method inside the device is used, that is, the data is compared with the updated personnel database to avoid generating duplicate data;

[0054] Step S6, all data are aggregated and uploaded to the server, and the server will save the data chain with the time, location and personnel information, and use these data chains to analyze, track and generate the movement trajectory of the personnel.

[0055] In the above step S1, the facial image is captured by using the internal video recording of the front-end facial recognition device for analysis and recognition, and the extracted facial features are compared with the local personnel database of the device; the similarity between the recognized face and the faces in the database is calculated, and it is determined whether the recognized face belongs to a registered person in the local personnel database based on a preset matching threshold; if the similarity exceeds the threshold, it is considered to be a successful match, that is, the recognized person is a registered person; if the similarity is lower than the threshold, it is considered to be a failed match, that is, the recognized person is not a registered person.

[0056] Before uploading the recognition results to the server, the local data is first deduplicated, and then the deduplicated data records are uploaded to the server and saved.

[0057] Among them, this embodiment selects the most advanced face recognition algorithm currently, such as a method based on deep learning, especially using a convolutional neural network (CNN) such as VGGNet, ResNet or Inception network structure. These network structures have been trained on a large number of data sets and can capture complex features in face images. Traditional feature extraction methods such as PCA and LDA may perform poorly under large-scale data, while CNN-based methods can extract richer features through a deep network structure, thereby improving recognition accuracy and speed.

[0058] In this embodiment, a large number of face image data sets are also used to train the model, including images under various lighting, angles and occlusion conditions. Data enhancement techniques such as random cropping, rotation, flipping and noise addition are used to increase the diversity of training data and the robustness of the model. The performance of the model is optimized by adjusting parameters such as network structure, optimizer settings and regularization strategy. Under the premise of ensuring recognition accuracy, the computational complexity of the algorithm is optimized to improve real-time performance.

[0059] Before the image is transmitted to the face recognition algorithm, image preprocessing is also performed to improve the image quality, which may include noise reduction processing (such as filters or wavelet transforms), normalization processing (such as histogram equalization, scale transformation, and rotation correction), etc.

[0060] In this embodiment, a real-time calibration mechanism is also introduced to regularly check the position and parameter settings of the front face recognition device to ensure that it is always in the best state. The front face recognition device is monitored in real time using monitoring software to ensure that it can work normally under various conditions and produce high-quality face images.

[0061] Through the above steps, more advanced face recognition algorithms can be introduced to improve recognition accuracy and speed, and through front-end face recognition equipment calibration and configuration optimization, high-quality face images can be obtained under different lighting, angles and occlusion conditions.

[0062] In this embodiment, when performing deduplication processing, for the successfully matched persons, it is checked whether they already exist in the identified records. If the same identification records exist, deduplication processing is performed. If the time interval is less than a certain threshold and the location information is similar, these records are considered to be duplicates, and these duplicate records are removed, and only the latest or most complete records are retained; alternatively, it is determined based on the location information whether it is a repeated identification of the same location.

[0063] In this embodiment, when performing deduplication processing, if the person is an unregistered or unknown person, the facial photo taken for the first time is used as the identification standard and saved as a new person. After successful saving, the person information is sent to all devices and the person database is updated; if the same facial photo has been uploaded before, it will not be uploaded again.

[0064] In this embodiment, the deduplicated data records are packaged, including the latest records of registered persons, the first records of strangers, etc., and the packaged data records are uploaded to the server. After receiving the uploaded data, the server saves it to a designated storage location.

[0065] When the front-end face recognition device recognizes a new person, it updates the person database on the server in real time to ensure that all devices in the network can quickly obtain the latest person information, including:

[0066] Immediately package the captured photos and related information and upload them to the server. After receiving the uploaded data, the server verifies it and uses efficient data storage technology to ensure fast data writing and reading. Establish real-time indexes on the personnel database to speed up the query. Optimize query statements and data structures to reduce query time. The personnel database on the server adds a timestamp to each record. When the device needs to synchronize data, it only requests new or updated records after the timestamp. A version number is assigned to each personnel record. When the device is synchronizing, it only requests data with a version number greater than the local record.

[0067] Before synchronization, the device calculates the hash value of the local personnel database, compares it with the hash value on the server, and only downloads the difference part, and only transmits the difference part of the file instead of the entire file.

[0068] After receiving the data uploaded by the device, the server asynchronously processes the data storage and synchronization tasks to avoid blocking the main thread.

[0069] In this embodiment, in order to enhance data security and privacy protection, encrypted transmission data is also performed, and access control and data backup strategies are implemented on the server. For example, the SSL / TLS protocol uses public key encryption and private key decryption to encrypt and decrypt data to ensure that the data is not stolen or tampered with during transmission. If data needs to be transmitted over an unsecured public network, a VPN can be used. VPN creates an encrypted channel to transmit data through the tunnel to ensure data security.

[0070] When implementing access control and data backup strategies on the server, it is an important measure to protect the security of the database by setting database user permissions reasonably. Different permissions should be set according to the user's responsibilities and needs to prevent users from operating the database beyond their authority. Users should only be assigned the minimum permissions they need to complete their work. Full backup or incremental backup can be selected to ensure the security and integrity of the data. Use a multi-point backup strategy to store backup data in different locations, such as local hard disks, cloud storage, etc., to avoid data loss caused by single point failures. The backup data may contain sensitive information. In order to protect data security, the backup data can be encrypted. Use encryption algorithms to encrypt backup data to ensure that the backup data is not stolen during transmission and storage. Regular verification of the integrity and recoverability of backup data is an important step to ensure the effectiveness of backups. Administrators can restore backup data regularly to ensure that the backup data can be restored normally and avoid situations where the backup data is damaged and cannot be restored.

[0071] In this embodiment, when the time interval is less than a certain threshold and the location information is similar, these records are considered to be duplicates, which specifically includes:

[0072] A time threshold is set to cover the time interval for a person to normally pass through the identification area and to avoid repeated identification due to accidental factors (such as a person's short stay); for example, if a person passes through the identification area once a minute on average, the time threshold can be set to 30 seconds or 1 minute.

[0073] Set a position threshold, which is used to reflect the displacement distance of a person's normal activity range in a short period of time; when a person is identified, calculate the distance between his current position and the position when he was last identified, which can be achieved by comparing the position information (such as longitude and latitude coordinates) of the two identifications. For example, if a person moves a maximum of 10 meters per minute within the normal activity range, the spatial threshold can be set to 5 meters or 10 meters.

[0074] In the system implementation, for the successfully matched persons, the timestamp of their last identification is checked. If the difference between the current time and the timestamp of the last identification is less than the set time threshold, the difference between the newly recorded location information and the previously recorded location information is further checked to see if it is less than the set location threshold. If both conditions are met, it is considered a repeated identification and no new data records are generated.

[0075] In practical applications, the time threshold and location threshold need to be adjusted according to the actual situation to achieve the best deduplication effect.

[0076] This embodiment can effectively avoid generating too many records of repeated identification of the same person in a short period of time by adjusting the time threshold for deduplication and introducing the space threshold, which can not only improve the accuracy and efficiency of the system, but also reduce the burden of data storage and processing. In practical applications, it is necessary to set reasonable thresholds according to specific scenarios and needs, and adjust and optimize according to actual conditions.

[0077] In addition, in order to improve the efficiency of comparison and deduplication, efficient data storage structures (such as hash tables, B+ trees, etc.) can be used to store personnel identification records. At the same time, indexing technology can be used to speed up query operations and improve the response speed of the system.

[0078] In this embodiment, when comparing data in the above step S2, if the comparison with the local personnel database is successful, the data of the person will be used.

[0079] In this embodiment, if the data comparison in the above step S2 is unsuccessful, the facial photo taken for the first time is referenced as the identification standard and saved as a new person.

[0080] In this embodiment, in the above step S4, after successful saving, the personnel information is sent to all devices to update the personnel database.

[0081] Of course, the data deduplication processing method provided by the present invention can also be implemented by other steps by ordinary technicians in the industry. Figure 1 The data deduplication processing method provided by the present invention is only a specific embodiment.

[0082] In summary, the present invention can analyze the content of the surveillance video in real time and quickly obtain the analysis and recognition results, thereby ensuring the real-time nature of the data. At the same time, by comparing with the local personnel database, known or registered personnel can be accurately identified, avoiding the generation of duplicate data. Once the identified personnel are already in the local personnel database, the present invention will automatically perform local deduplication processing, and no new data records will be generated, which greatly improves the efficiency of data processing. At the same time, the mechanism of creating new personnel files on the server and updating the personnel database of other devices also ensures that the devices in the network can respond quickly, reducing the transmission and storage of duplicate data in the network. The method is applicable to multiple devices in the network, and the personnel files are synchronously updated by the server, ensuring the scalability and flexibility of the entire system. When other devices in the network recognize the person again, the updated personnel database can be directly used for comparison, further improving the recognition efficiency. All data of the present invention will be summarized and uploaded to the server, and the server will save the data chain with time and location personnel information, which makes the data complete and traceable, and facilitates subsequent analysis, tracking and generation of personnel action trajectories. The present invention reduces the generation and storage of duplicate data through deduplication processing, thereby saving storage space and computing resources. At the same time, this method can automatically complete the identification and deduplication process, reducing the investment in manpower costs.

[0083] like Figure 2 As shown, this embodiment also provides a data deduplication processing system based on a face recognition device, which is used to implement the above-mentioned data deduplication processing method based on a face recognition device, including:

[0084] The front-end face recognition device module 1 is provided by the front-end face recognition device manufacturer, which compares the actual surveillance video with the personnel information database and performs data deduplication;

[0085] Personnel information synchronization management module 2 is used to use an encrypted channel to receive the encrypted personnel information, and the central server sends it to synchronize all subordinate devices and ensure the synchronization is successful;

[0086] The communication record storage module 3 is used to store the communication record information after receiving the information.

[0087] The data deduplication processing system based on the front-end face recognition device provided by the present invention is designed and developed to solve the problem of tracking the movement trajectory of strangers in various public places with large traffic. Based on the local application of the data deduplication processing system of the front-end face recognition device, it is automatically started when the PC system starts. In the business system, the front-end face recognition device deployed in public places compares the video analysis and recognition results with the local personnel database of the device; if the comparison with the local personnel database is successful, the data of the person is used; if the comparison is unsuccessful, the face photo taken for the first time is referenced as the recognition standard and saved as a new person; after successful saving, the person information is sent to all devices and the personnel database is updated; after the operation is completed, the person's access record is sent to the server for storage.

[0088] After data collection, a data chain with time and location personnel information will be saved, and the movement trajectory of personnel can be regenerated based on the data chain. The processing service of camera recognition personnel data is applied in online viewing, editing, storage, and deduplication of camera recognition personnel data in personnel information management, smart campus, smart building, and smart management information platform.

[0089] Therefore, the data deduplication processing system and method based on the front-end face recognition device provided by the present invention solves the problem that there is no need to worry about not being able to track strangers in public places with large traffic. The system of the present invention supports online viewing, editing, and cloud storage of various personnel trace data chains, and can be widely used in personnel information management, smart campuses, smart buildings, and smart management information platforms.

[0090] In one embodiment, a PC terminal or electronic device is provided, which may be a server. The computer device includes a processor, a memory and a network interface connected via a system bus. The processor of the electronic device is used to provide computing and control capabilities. The memory of the electronic device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system, a computer program and a database. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The database of the electronic device is used to store data. The network interface of the electronic device is used to communicate with an external terminal via a network connection. When the computer program is executed by the processor, a data deduplication processing method based on a face recognition device is implemented.

[0091] Those skilled in the art will understand that the electronic device structure shown in this embodiment is only a partial structure related to the scheme of the present application, and does not constitute a limitation on the electronic device to which the scheme of the present application is applied. The specific electronic device may include more or fewer components than those shown in this embodiment, or combine certain components, or have a different arrangement of components.

[0092] In one embodiment, a computer-readable storage medium is provided, on which a computer program is stored. When the computer program is executed by a processor, the steps in the above-mentioned method embodiments are implemented.

[0093] Those of ordinary skill in the art can understand that all or part of the processes in the above-mentioned embodiment methods can be completed by instructing the relevant hardware through a computer program, and the computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to memory, storage, database or other media used in the embodiments provided in this application can include at least one of non-volatile and volatile memory. Non-volatile memory may include read-only memory (ROM), magnetic tape, floppy disk, flash memory or optical memory, etc. Volatile memory may include random access memory (RAM) or external cache memory. As an illustration and not limitation, RAM can be in various forms, such as static random access memory (SRAM) or dynamic random access memory (DRAM).

[0094] In addition, the logic instructions in the above-mentioned memory can be implemented in the form of software functional units and can be stored in a computer-readable storage medium when sold or used as an independent product. Based on such an understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art or the part of the technical solution, can be embodied in the form of a software product, which is stored in a storage medium and includes several instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present invention. The aforementioned storage medium includes: various media that can store program codes, such as a USB flash drive, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk.

[0095] It can be seen that this embodiment also provides a PC end and storage medium of a data deduplication processing method based on a face recognition device, which includes: one or more memories, and one or more processors. The memory is used to store program code and intermediate data generated during program operation, storage of model output results, and storage of models and model parameters; the processor is used for processor resources occupied by code operation and multiple processor resources occupied when training the model.

[0096] It should be noted that the embodiments of the present invention can be deployed on a server with a network. A person skilled in the art can understand that the above-mentioned devices and methods can be implemented using computer executable instructions and / or included in a processor control code. The above is only a specific embodiment of the present invention, but the protection scope of the present invention is not limited thereto. Any modification, equivalent replacement and improvement made by any person skilled in the art within the technical scope disclosed by the present invention and within the spirit and principle of the present invention shall be covered within the protection scope of the present invention.

[0097] The above-mentioned embodiments are only preferred embodiments of the present invention and cannot be used to limit the scope of protection of the present invention. Any non-substantial changes and substitutions made by technicians in this field on the basis of the present invention shall fall within the scope of protection required by the present invention.

Claims

1. A data deduplication processing method based on a face recognition device, characterized in that: The method comprises the following steps: Conduct real-time analysis of surveillance video content to obtain analysis and identification results; Using the analysis and recognition result to compare with a local personnel database of the device, the local personnel database contains known or registered personnel information; If the match is successful, that is, the identified person is already in the local personnel database, local deduplication processing will be automatically performed and no new data record will be generated for the person; If the comparison fails, the captured photos will be collected and uploaded to the server, a new personnel file will be created for the person on the server, and the files will be updated synchronously to the personnel databases of other devices in the network; When other devices in the network recognize the person again, they use the data deduplication method inside the device, that is, compare it with the updated personnel database to avoid generating duplicate data; All data will be aggregated and uploaded to the server, which will save a data chain with time, location and personnel information. These data chains will be used to analyze, track and generate personnel movement trajectories.

2. The method according to claim 1, characterized in that: Use the internal video recording of the front-end face recognition device to capture the face image for analysis and recognition, and compare the extracted face features with the local personnel database of the device; Calculate the similarity between the recognized face and the faces in the database, and determine whether the recognized face belongs to a registered person in the local database based on the preset matching threshold; if the similarity exceeds the threshold, it is considered a successful match, that is, the recognized person is a registered person; if the similarity is lower than the threshold, it is considered a failed match, that is, the recognized person is not a registered person; Before uploading the recognition results to the server, the local data is first deduplicated, and then the deduplicated data records are uploaded to the server and saved.

3. The method according to claim 1, characterized in that: When performing deduplication processing, for those who have been matched successfully, check whether they already exist in the identified records. If the same identification records exist, deduplication processing is performed. If the time interval is less than a certain threshold and the location information is similar, these records are considered to be duplicates. These duplicate records are removed and only the latest or most complete records are retained; alternatively, determine whether it is a duplicate identification of the same location based on the location information.

4. The method according to claim 3, characterized in that: When performing deduplication processing, if the person is unregistered or unknown, the facial photo taken for the first time is used as the identification standard and saved as a new person. After successful saving, the person information is sent to all devices and the person database is updated. If the same facial photo has been uploaded before, it will not be uploaded again.

5. The method according to claim 1, characterized in that: The deduplicated data records are packaged, including the latest records of registered persons, the first records of strangers, etc., and the packaged data records are uploaded to the server. After the server receives the uploaded data, it saves it to the specified storage location.

6. The method according to claim 1, characterized in that: When the front-end face recognition device recognizes a new person, it updates the person database on the server in real time to ensure that all devices in the network can quickly obtain the latest person information, including: Immediately package the captured photos and related information and upload them to the server. After receiving the uploaded data, the server verifies it and uses efficient data storage technology to ensure fast data writing and reading. Establish real-time indexes on the personnel database to speed up the query. Optimize query statements and data structures to reduce query time. The personnel database on the server adds a timestamp to each record. When the device needs to synchronize data, it only requests new or updated records after the timestamp. A version number is assigned to each personnel record. When the device is synchronizing, it only requests data with a version number greater than the local record. Before synchronization, the device calculates the hash value of the local personnel database, compares it with the hash value on the server, and only downloads the difference part, and only transmits the difference part of the file instead of the entire file. After receiving the data uploaded by the device, the server asynchronously processes the data storage and synchronization tasks to avoid blocking the main thread.

7. The method according to claim 3, characterized in that If the time interval is less than a certain threshold and the location information is similar, then these records are considered to be duplicates, including: Set a time threshold, which is used to cover the time interval of a person passing through the identification area normally and can avoid repeated identification due to accidental factors; A position threshold is set, which is used to reflect the displacement distance of a person's normal activity range in a short period of time; In the system implementation, for the successfully matched persons, the timestamp of their last identification is checked. If the difference between the current time and the timestamp of the last identification is less than the set time threshold, the difference between the newly recorded location information and the previously recorded location information is further checked to see if it is less than the set location threshold. If both conditions are met, it is considered a repeated identification and no new data records are generated.

8. A data deduplication processing system based on face recognition equipment, characterized in that: The system is used to implement the data deduplication processing method based on the face recognition device as claimed in any one of claims 1 to 7, comprising: The front-end face recognition device module is provided by the front-end face recognition device manufacturer. It compares the actual surveillance video with the personnel information database and performs data deduplication. The personnel information synchronization management module is used to use an encrypted channel to receive the personnel information after data encryption and protection, and the central server sends it to synchronize all subordinate devices and ensure the success of synchronization; The communication record storage module is used to save the communication record information after receiving the information.

9. A PC terminal, characterized in that: The PC end includes a memory and a processor, the memory stores a computer program, and when the computer program is executed by the processor, the processor executes the data deduplication processing method based on the face recognition device as described in any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that: The medium stores a computer program, and when the computer program is executed by a processor, the processor executes the data deduplication processing method based on a face recognition device as described in any one of claims 1 to 7.

Citation Information

Patent Citations

  • Returned customer automatic identification method and system

    CN107368770A

  • Information processing method, apparatus, electronic device, and computer-readable storage medium

    CN109345316A

  • Identity archiving method and device, electronic equipment and storage medium

    CN113887367A

  • Data synchronization method and device, electronic equipment and computer readable medium

    CN114860750A