Monitoring system and monitoring method

Abstract

To provide a monitoring system and a monitoring method for protecting privacy of a monitoring object.SOLUTION: The method includes capturing an image, obtaining a face image of a to-be-monitored object from the image, performing an anonymization operation on the image to obtain an anonymized image, and outputting the anonymized image, performing a first anonymization operation on the face image to generate an anonymization feature, and generating a verification result according to the anonymization feature, performing a second anonymization operation on the face image to generate an anonymization label, and querying an image database according to the anonymization label to obtain a historical anonymized image corresponding to the anonymization label.SELECTED DRAWING: Figure 4

Description

[Technical Field]

[0001] The present invention relates to a monitoring system and a monitoring method. [Background technology]

[0002] With the widespread use of monitors and advances in image recognition technology, existing surveillance systems can almost completely track the location of monitored targets and store and query their associated image data. However, this technology significantly violates people's privacy. Furthermore, if the stored image data is leaked, the identity of the person contained in the image data may also be leaked, affecting the person's personal safety. Therefore, how to protect people's privacy while storing surveillance image data is one of the key issues in this technical field. Summary of the Invention [Problem to be solved by the invention]

[0003] The present invention provides a monitoring system and a monitoring method that can protect the privacy of a monitored subject. [Means for solving the problem]

[0004] The monitoring system of the present invention includes an image capture device and a processing device. The image capture device captures an image. The processing device is communicatively connected to the image capture device and configured to acquire a facial image of a monitoring target from the image, perform an anonymization operation on the image to acquire an anonymized image, output the anonymized image, perform a first anonymization operation on the facial image to generate an anonymization feature, and generate a verification result according to the anonymization feature.

[0005] In one embodiment of the present invention, the processing device is further configured to generate a verification result by determining whether the anonymized features match features pre-stored in the feature database.

[0006] In one embodiment of the present invention, the processing device is further configured to perform a second anonymization operation on the facial image to generate anonymization labels, and establish a mapping relationship between the anonymization labels and the anonymized images to build or update the image database.

[0007] In one embodiment of the present invention, the second anonymization operation is different from the first anonymization operation, and the processing device performs the first anonymization operation based on a differential privacy algorithm and the second anonymization operation based on a homomorphic encryption algorithm.

[0008] In one embodiment of the present invention, the anonymization process includes using a machine learning model to cover the objects in the image to generate an anonymized image.

[0009] In one embodiment of the present invention, the first anonymization operation on the facial image includes performing the first anonymization operation on the facial image using a machine learning model that supports privacy preserving techniques.

[0010] In one embodiment of the present invention, the machine learning model includes a plurality of neurons divided into a plurality of layers, converts a face image into feature quantities of a plurality of neurons in a first layer of the plurality of layers, adds the feature values ​​of each converted neuron to noise generated using privacy parameters for input to the next layer, obtains an anonymized image after multi-layer processing, and converts the anonymized image into anonymized features.

[0011] In one embodiment of the present invention, the processing device is further configured to verify the identity of the monitored subject to which the anonymized feature belongs by a trained machine learning model to generate a verification result, where the machine learning model is trained using the anonymized features and identities of a plurality of pre-enrolled users.

[0012] In one embodiment of the present invention, the processing device is further configured to perform a second anonymization operation on the face image to generate an anonymization label, and query the image database according to the anonymization label to obtain a historical anonymized image corresponding to the anonymization label.

[0013] In one embodiment of the present invention, the processing device is further configured to determine whether the verification result is successful, and in response to the verification result being successful, query the image database according to the anonymization label to obtain historical anonymized images corresponding to the anonymization label.

[0014] The monitoring method of the present invention includes capturing an image, obtaining a face image of a monitored subject from the image, performing an anonymization process on the image to obtain an anonymized image, outputting the anonymized image, performing a first anonymization operation on the face image to generate an anonymization feature, and generating a verification result according to the anonymization feature.

[0015] In one embodiment of the present invention, generating a verification result according to the anonymization feature includes determining whether the anonymization feature matches a feature pre-stored in a feature database and generating the verification result.

[0016] In one embodiment of the present invention, the monitoring method further includes performing a second anonymization operation on the face image to generate anonymization labels, and establishing a mapping relationship between the anonymization labels and the anonymized images to build or update the image database.

[0017] In one embodiment of the present invention, the second anonymization operation is different from the first anonymization operation, and the first anonymization operation is performed based on a differential privacy algorithm and the second anonymization operation is performed based on a homomorphic encryption algorithm.

[0018] In one embodiment of the present invention, performing an anonymization process on the image to obtain the anonymized image includes using a machine learning model to cover the monitored object in the image to generate the anonymized image.

[0019] In one embodiment of the present invention, the first anonymization operation on the facial image includes performing the first anonymization operation on the facial image using a machine learning model that supports privacy preserving techniques.

[0020] In one embodiment of the present invention, the machine learning model includes a plurality of neurons divided into a plurality of layers, converts a face image into feature quantities of a plurality of neurons in a first layer of the plurality of layers, adds the feature values ​​of each converted neuron to noise generated using privacy parameters for input to the next layer, obtains an anonymized image after multi-layer processing, and converts the anonymized image into anonymized features.

[0021] In one embodiment, generating a verification result according to the anonymized features includes verifying the identity of the monitored subject to which the anonymized features belong with a trained machine learning model to generate the verification result, where the machine learning model is trained using the anonymized features and identities of a plurality of pre-enrolled users.

[0022] In one embodiment of the present invention, the monitoring method further includes: performing a second anonymization operation on the face image to generate an anonymization label; and querying an image database according to the anonymization label to obtain a historical anonymized image corresponding to the anonymization label.

[0023] The surveillance system of the present invention includes an image capture device and a processing device. The image capture device captures images. The processing device is communicatively connected to the image capture device and configured to: acquire a facial image of a monitored subject from the images; perform an anonymization operation on the facial image to generate anonymization labels; perform the anonymization process on the image to obtain an anonymized image; establish a mapping relationship between the anonymization labels and the anonymized image to build or update an image database; and output the anonymized image stored in the image database in response to receiving a query command that matches the anonymization label. [Effects of the Invention]

[0024] Based on the above, the monitoring system of the present invention can use machine learning to perform anonymization processing on images to protect the privacy of people in the images. For a monitored target in an image, the monitoring system can perform an anonymization operation on the monitored target's facial image to generate anonymized features for verifying the person's identity or anonymized labels for establishing an image database. The monitoring system can compare the anonymized features with features pre-stored in the feature database to determine the monitored target's identity. Meanwhile, the monitoring system can use the anonymized labels to build or update an image database that stores anonymized images. When a user attempts to find the location of a specific target, the monitoring system can query the image database to complete tracking of the specific target without violating the individual's privacy. [Brief explanation of the drawings]

[0025] [Figure 1] 1 is a schematic diagram of a monitoring system according to one embodiment of the present invention; [Figure 2] 1 is a schematic diagram of an identity authentication process according to an embodiment of the present invention; [Figure 3] FIG. 2 is a schematic diagram of an image data query process according to an embodiment of the present invention. [Figure 4] 2 is a flowchart of a monitoring method according to an embodiment of the present invention. [Figure 5] 4 is another flowchart of a monitoring method according to an embodiment of the present invention. [Figure 6] 1 is a schematic diagram of a face recognition system according to an embodiment of the present invention; [Figure 7] 1 is a schematic diagram of a face recognition method according to an embodiment of the present invention; [Figure 8] 1 is a flowchart of a face recognition method according to an embodiment of the present invention. [Figure 9] 1 is a block diagram of a face recognition system according to one embodiment of the present invention. DETAILED DESCRIPTION OF THE INVENTION

[0026] 1 is a schematic diagram of a surveillance system 10 according to one embodiment of the present invention. The surveillance system 10 may include a processing unit 100 and an image capture device 200. In one embodiment, the processing unit 100 and the image capture device 200 are implemented by different hardware devices, and the processing unit 100 and the image capture device 200 can communicate with each other. In one embodiment, the processing unit 100 and the image capture device 200 may be implemented by the same hardware device. For example, the processing unit 100 may be an image signal processor (ISP) of the image capture device 200.

[0027] The image capture device 200 may include a charge coupled device (CCD), a complementary metal-oxide semiconductor (CMOS), or other type of photosensitive device to sense light intensity and generate an image of the camera scene. The image capture device 200 may include a communication device supporting a communication protocol such as wireless fidelity (Wi-Fi), radio frequency identification (RFID), Bluetooth, infrared, near-field communication (NFC), or device-to-device (D2D), a network connection device supporting an application programming interface (API), or an internet connection, for communicating or networking with an external device or the processing device 100.

[0028] The processing device 100 may be, for example, a server, workstation, or other electronic device. The processing device 100 may include a communication device, a storage device, and a processor. The communication device may support a communication protocol, such as wireless fidelity, radio frequency identification, Bluetooth, infrared, near field communication, or device-to-device communication, an application programming interface, or an Internet connection, for communicating or networking with the image capture device 200 or external devices. The storage device may be, for example, any type of fixed or removable random access memory (RAM), read-only memory (ROM), flash memory, hard disk, or similar element, or a combination of these elements, for storing a computer program executable by the processor. The processor may be, for example, a central processing unit (CPU), other programmable general-purpose or special-purpose microprocessor, microcontroller, digital signal processor (DSP), programmable controller, application specific integrated circuit (ASIC), programmable logic device (PLD), other similar device, or a combination of these devices. In one embodiment, the processor is capable of reading a computer program from a storage device and executing a monitoring method according to an embodiment of the present invention.

[0029] The image capture device 200 can capture an image. The processing device 100 can perform an anonymization process on the image using a machine learning model 110 (e.g., a deep learning (DL) model) pre-stored in the processing device 100. Specifically, the processing device 100 can input the image to the machine learning model 110. The machine learning model 110 has an object detection function and can identify a monitoring target 25 in the input image. The machine learning model 110 can cover the monitoring target 25 in the image to generate an anonymized image 20. The processing device 100 can output the anonymized image 20, for example, through a display for user reference. Because the monitoring target 25 is concealed in the anonymized image 20, even if the contour of the monitoring target 25 is shown in the anonymized image 20, a person viewing the anonymized image 20 still cannot identify the identity of the monitoring target 25. Therefore, the anonymized image 20 can protect the privacy of the monitoring target 25.

[0030] In one embodiment, the machine learning model 110 may include a DL model such as a deep neural network (DNN).

[0031] The machine learning model 110 can capture a facial image 30 of the monitoring target 25 from an input image. The processing device 100 can perform an anonymization operation on the facial image 30 to generate one or more anonymized features 31. Specifically, the processing device 100 can input the facial image 30 into the machine learning model to perform an anonymization operation on the facial image 30. Here, the machine learning model can support privacy preservation techniques such as differential privacy, homomorphic encryption, shuffling, pixelation, etc. The machine learning model can include multiple neurons divided into multiple layers, and the transformed feature value of each neuron can be added to noise generated using a privacy parameter for input to the next layer, obtaining an anonymized image after multi-layer processing, and converting the anonymized image into anonymized features 31.

[0032] In one embodiment, the processing device 100 can, for example, use an artificial intelligence model to determine whether the anonymized feature 31 matches a pre-stored feature in the feature space 60 of the feature database 120 and generate a verification result. The processing device 100 can, for example, perform the anonymization operation based on a differential privacy algorithm to reduce the time required to generate the anonymized feature 31, or the processing device 100 can perform the anonymization operation based on another encryption algorithm (for example, a homomorphic encryption algorithm). If the anonymized feature 31 matches a pre-stored feature (e.g., the similarity between the anonymized feature 31 and the pre-stored feature is greater than a threshold), it means that the identity of the monitored subject 25 is a specific person corresponding to the pre-stored feature. Therefore, the processing device 100 can generate a successful verification result. If the anonymized feature 31 does not match any of the pre-stored features (e.g., the similarity between the anonymized feature 31 and the pre-stored feature is equal to or less than a threshold), it means that the identity of the monitored subject 25 is unknown. Therefore, the processing device 100 may generate a failed verification result. After generating the verification result, the processing device 100 may output the verification result for user reference.

[0033] In one embodiment, the processing device 100 may verify the identity of the monitored subject to which the anonymized feature 31 belongs, for example, by inputting the anonymized feature 31 into a trained machine learning model to generate a verification result. The machine learning model may be trained using the anonymized features and identities of multiple pre-enrolled users (e.g., historical anonymized features and corresponding historical identities in the feature space 60).

[0034] To establish the feature space 60 in the feature database 120, the processing device 100 can acquire multiple historical images of multiple people (e.g., via the image capture device 100). The processing device 100 performs an anonymization operation on the historical images according to the machine learning model 110 to generate multiple historical anonymized features 50. The processing device 100 can establish the feature space 60 according to the historical anonymized features 50. The feature space 60 can include one or more historical anonymized features corresponding to the identity of a particular person. The feature space 60 can be obtained, for example, from an embedded space or loss function such as AdaFace or ArcFace, which includes optimizing a geodesic distance margin through correspondence between angles or radians in a normalized hypersphere. The feature database 120 can be stored, for example, on the processing device 100 or an external cloud server (e.g., cloud server 300 as shown in FIG. 3).

[0035] Meanwhile, the processing device 100 can perform an anonymization operation on the face image 30 to generate an anonymization label 32. Here, the anonymization operation for generating the anonymization label 32 and the anonymization operation for generating the anonymization feature 31 may be the same or different. That is, the anonymization label 32 and the anonymization feature 31 may be the same or different. In one embodiment, the processing device 100 can perform the anonymization operation for generating the anonymization label 32 based on, for example, a homomorphic encryption algorithm to generate a more easily identifiable anonymization label 32, or the processing device 100 can perform the anonymization operation based on another encryption algorithm (e.g., a differential privacy algorithm). In one embodiment, the processing device 100 can perform the anonymization operation based on a homomorphic encryption algorithm based on post-quantum-secure de-identification.

[0036] In one embodiment, after generating the anonymization labels 32, the processing device 100 may use the anonymization labels 32 to build or update the image database 130. Specifically, the processing device 100 may establish a mapping relationship between the anonymization labels 32 and the anonymized images 20, thereby building or updating the image database 130. Here, the image database 130 may store the anonymization labels 32, the anonymized images 20, and the mapping relationship between the two. The image database 130 may be stored, for example, in the processing device 100 or an external cloud server (e.g., the cloud server 300 shown in FIG. 3 ).

[0037] In one embodiment, after generating the anonymization label 32, the processing device 100 can use the anonymization label 32 to query related image data of the monitored target 25. Specifically, the image database 130 can pre-store historical anonymization labels and historical anonymized images having a mapping relationship. The processing device 100 can query the image database 130 to determine whether a historical anonymization label matching the anonymization label 32 is stored. For example, the processor 100 can perform a fuzzy search on the image database 130 according to the anonymization label 32 to determine whether the image database 130 stores a historical anonymization label matching the anonymization label 32. If the anonymization label 32 matches the historical anonymization label in the image database 130 (e.g., the similarity between the anonymization label 32 and the historical anonymization label is greater than a threshold), the processing device 100 can output the historical anonymized image corresponding to the historical anonymization label for user reference. If the anonymization label 32 does not match any of the historical anonymization labels in the image database 130 , it means that the image database 130 does not store image data associated with the subject 25 .

[0038] FIG. 2 is a schematic diagram of an identity verification process according to an embodiment of the present invention. The monitoring system 100 may execute a registration process to establish a feature space 60. Specifically, the processing device 100 may be communicatively connected to an external terminal device. A data provider may send a historical image for registration, including an image of a specific target (e.g., a blacklisted person or a shopping mall member), to the processing device 100 via the terminal device. In step S201, the processing device 100 may execute the registration process. The processing device 100 may perform an anonymization operation (e.g., an anonymization operation based on a differential privacy algorithm) on the historical image to obtain one or more historical anonymization features. In step S202, the processing device 100 may establish a feature space 60 including one or more pre-stored features according to the one or more historical anonymization features.

[0039] After the establishment of the feature space 60 is completed, the processing device 100 can perform identity verification according to the feature space 60. Specifically, the processing device 100 can acquire an image including the monitored subject 25 through the image capture device 200. In step S203, the processing device 100 can use the machine learning model 110 to capture a facial image 30 of the monitored subject 25 from the image, and perform an anonymization operation on the facial image 30 to generate anonymized features 31. In step S204, the processing device 100 can compare the similarity of the anonymized features 31 with pre-stored features in the feature space 60 to verify the identity of the monitored subject 25 and generate a verification result.

[0040] 3 is a schematic diagram of an image data query process according to an embodiment of the present invention. To build or update the image database 130 on the cloud server 300, in step S301, a data provider can upload historical anonymized labels and historical anonymized images with mapping relationships to the image database 130 on the cloud server 300.

[0041] A data user (or image capture device 200) can send a query command including an image to the processing device 100. The processing device 100 can capture a facial image of a monitored subject from the image, and perform an anonymization process on the facial image to obtain anonymized features and anonymized labels. In step S302, the processing device 100 can access the feature database 120 of the cloud server 300 to determine whether the feature space stored in the feature database 120 includes pre-stored features that match the anonymized features.

[0042] If the feature space contains pre-stored features that match the anonymization features, it means that the verification of the identity of the monitored subject has been successful. Therefore, the processing device 100 can further query whether the image database 130 stores a historical anonymization label that matches the anonymization label. In response to an anonymization label that matches the historical anonymization label in the image database 130, the processing device 100 can retrieve a historical anonymized image corresponding to the historical anonymization label from the image database 130. In step S303, the processing device 100 can output a historical anonymized image corresponding to the monitored subject for reference by the data user. Based on the above, the monitoring system 100 of the present invention can first reduce the time or computing resources spent verifying the identity of the monitored subject through the anonymization features. After the identity of the monitored subject is verified, the monitoring system 100 can spend more time or computing resources querying the anonymized image associated with the monitored subject through the anonymization label.

[0043] FIG. 4 is a flowchart of a monitoring method according to an embodiment of the present invention, which can be implemented by the monitoring system 10 shown in FIG. 1. In step S401, an image is captured. In step S402, a face image of a monitoring target is obtained from the image. In step S403, an anonymization process is performed on the image to obtain and output an anonymized image. In step S404, a first anonymization operation is performed on the face image to generate anonymization features. In step S405, a verification result is generated according to the anonymization features. In step S406, the verification result is output.

[0044] 5 is a flowchart of another monitoring method according to an embodiment of the present invention, which can be implemented by the monitoring system 10 shown in FIG. 1. In step S501, an image is captured. In step S502, a face image of a monitored subject is obtained from the image, and an anonymization process is performed on the face image to generate an anonymization label. In step S503, the anonymization process is performed on the image to obtain an anonymized image. In step S504, a mapping relationship between the anonymization label and the anonymized image is established, and an image database is constructed or updated. In step S505, in response to receiving a query command that matches the anonymization label, the anonymized image stored in the image database is output.

[0045] In summary, the surveillance system of the present invention employs advanced technology that can observe and track suspicious activity in a targeted manner while protecting individual privacy. The surveillance system uses a decentralized artificial intelligence model and carefully designed differential privacy and homomorphic encryption technologies to track specific individuals without violating their privacy. An advanced multi-modal deep neural network model based on post-quantum-safe anonymization can achieve image data anonymization while ensuring high efficiency in human image processing tasks and high accuracy in identification tasks. The surveillance system can be seamlessly integrated with existing surveillance infrastructure, providing a powerful solution to the challenges of mass surveillance while maintaining individual privacy.

[0046] The surveillance system of the present invention may have the following advantages: The surveillance system can be seamlessly integrated with existing surveillance systems through an application programming interface. The surveillance system has high compatibility, simultaneously supporting cloud computing platforms and edge computing platforms, thereby providing flexibility and scalability. The surveillance system can implement strong privacy protection and secure image search using differential privacy and homomorphic encryption algorithms. The surveillance system can identify and track specific objects with very high accuracy.

[0047] Ensuring privacy during data collection is essential in industries such as finance, healthcare, cryptocurrency, and electronic signature platforms. A facial recognition system according to an embodiment of the present invention is specifically designed and built for cloud and edge computing, with artificial intelligence (AI) recognition models stored therein to achieve high computing efficiency. Furthermore, embodiments of the present invention provide private and secure identity verification, with image processing completed only on the local device, and sensitive personal facial photos are not uploaded to the cloud to avoid data leakage.

[0048] 6 is a schematic diagram of a face recognition system according to one embodiment of the present invention. Referring to FIG. 6, face recognition system 60 of this embodiment includes image capture device 62 and processing device 64. Here, the functions of image capture device 62 and processing device 64 are the same as or similar to the functions of image capture device 200 and processing device 100 in the above-described embodiment.

[0049] Image capture device 62 is a device or equipment located at the local end that includes, for example, a CCD, CMOS device, or other type of light-sensitive device that can sense light intensity and generate an image of the camera scene. Image capture device 62 also includes a communication device that supports communication protocols such as Wi-Fi, RFID, Bluetooth, infrared, NFC, D2D, etc., or a network connection device that supports Internet connectivity, for communication with external devices or network connections. In some embodiments, image capture device 62 also includes an image signal processor (ISP) that can be used to process the captured image.

[0050] The processing device 64 may be, for example, a server, workstation, or other electronic device at the remote end. The processing device 64 includes a communication device, a storage device, and a processor. For example, the communication device may be used for communication with the image capture device 62 or for network connection, and may support communication protocols such as Wi-Fi, RFID, Bluetooth, infrared, NFC, D2D, or Internet connection. The storage device may be, for example, any type of fixed or removable RAM, ROM, flash memory, hard disk, or similar element, or a combination of these elements, used to store a computer program executed by the processor. The processor may be, for example, a CPU, other programmable general-purpose or special-purpose microprocessor, microcontroller, DSP, programmable controller, ASIC, PLD, or other similar device, or a combination of these elements, but the present invention is not limited thereto. In this embodiment, the processor may read the computer program from the storage device and execute the facial recognition method according to an embodiment of the present invention. In some embodiments, the processor of the processing device 64 is provided with an API, which has a trained machine learning model embedded therein and can be used to verify a user's identity.

[0051] In step S602, the image capture device 62 captures an image of the camera scene and performs facial recognition to obtain a facial image 662. Here, the image capture device 62 may, for example, run a facial recognition algorithm on the captured image to capture the facial image 662.

[0052] In step S604, the image capture device 62 (or the processing device 64) anonymizes the face image 662 using a machine learning model that supports a privacy preservation technique to obtain anonymized image data 664, and converts the anonymized image data 664 into a plurality of image data for output to the processing device 64. The privacy preservation techniques include, but are not limited to, differential privacy, homomorphic encryption, shuffling, or pixelation.

[0053] In step S606, the processing unit 64 trains a machine learning model using the anonymized features 666 and identities of multiple pre-registered users. The machine learning model may be, for example, a convolutional neural network (CNN), a DNN, a recurrent neural network (RNN) including an input layer, at least one hidden layer, and an output layer, or other models with a learning function, but the present invention is not limited thereto.

[0054] In step S608, the processing device 64 verifies the identity of the user to which the anonymized feature belongs using the trained machine learning model and outputs the verification result 668.

[0055] In some embodiments, for example, the facial recognition system 60 employs a loosely coupled DNN as the machine learning model, and keeps only a small portion of the neural layers on the local device / appliance and the rest on a cloud or third-party server to achieve an optimal trade-off between computing resources, privacy loss, and model quality.

[0056] Based on the framework of the face authentication system described above, the face recognition system of this embodiment is divided into an enrollment phase and a recognition phase. Figure 7 is a schematic diagram of a face recognition method according to an embodiment of the present invention. Referring to Figures 6 and 7 simultaneously, the face recognition method of this embodiment is applied to the face recognition system 60 of Figure 6.

[0057] Step S710 is the enrollment phase, and in step S712 of step S710, the image capture device 62 inputs a plurality of captured facial images 720 into a machine learning model to generate a plurality of anonymized image data 722. The machine learning model includes a plurality of neurons divided into a plurality of layers, converts a facial image into feature values ​​of a plurality of neurons in a first layer of the plurality of layers, adds the converted feature values ​​of each neuron to noise generated using privacy parameters for input to the next layer, and obtains anonymized image data after multi-layer processing.

[0058] [Number 1] In detail, the machine learning model of this embodiment is a privacy-preserving algorithm for feature region calculation, i.e. A neural network model that preserves privacy via JPEG2026001000000002.jpg1643, where: JPEG2026001000000003.jpg812 is the specific data in the neural network, JPEG2026001000000004.jpg96 is the privacy parameter The noise is calculated using a noise distribution or substitution algorithm using JPEG2026001000000005.jpg64. JPEG2026001000000006.jpg812 is variable and can be adjusted by neural layers depending on computing resources, privacy loss, and model quality.

[0059] In step S714, the image capture device 62 (or processing device 64) further performs data processing on the anonymized image data to convert the anonymized image data into a plurality of anonymized features to establish an anonymized feature space 724. Here, the feature space 724 can be obtained, for example, by an embedding space or loss function, including optimizing a margin of geodesic distance through correspondence between angles or radians on a hypersphere.

[0060] Meanwhile, step S720 is the recognition phase, and in step S722 of step S720, the image capture device 62 inputs the currently captured face image 740 into the trained machine learning model to generate anonymized image data 742. In step S724, the image capture device 62 (or the processing device 64) performs data processing on the anonymized image data 742 to convert the anonymized image data 742 into a plurality of anonymized features to output an anonymized feature vector 744. In this embodiment, the anonymized feature vector 744 is generated by converting the feature values ​​X1 to X2 of the 512 features. 512 However, the present invention is not limited thereto.

[0061] Step S730 is also a recognition phase, in which the processing unit 64 verifies the identity of the user to which the anonymized features belong by a trained machine learning model, where the machine learning model is trained, for example, using the anonymized features and identities of multiple pre-registered users. In some embodiments, the processing unit 64 calculates similarities 760 (similarities S1-S2) between the anonymized features and a feature space established using the anonymized features of each pre-registered user. N (N is a positive integer) and calculate the similarity S1 to S N According to the magnitude of , the identity of the user to whom the anonymized feature belongs is verified.

[0062] However, in other embodiments, the processing unit 64 may use different activation functions, such as an S (sigmoid) function and a hyperbolic tangent (tanh) function, in the hidden layer of the machine learning model to calculate the output of the neurons, and may also use different transformation functions, such as using a normalized exponential (softmax) function in the output layer to calculate the predicted results, or using gradient descent (GD), backpropagation (BP), or other methods to update the weights of each neuron in the hidden layer; the present invention does not limit the manner in which the machine learning model is used to verify the identity of a user.

[0063] 8 is a flowchart of a face recognition method according to an embodiment of the present invention. Referring to both FIG. 6 and FIG. 8, the face recognition method of this embodiment can also be applied to the face recognition system 60 of FIG.

[0064] In step S802, the facial recognition system 60 captures a facial image of the user to be recognized by the image capture device 62.

[0065] In step S804, the image capture device 62 anonymizes the facial image to obtain anonymized image data. Here, the image capture device 62 anonymizes the facial image using, for example, a machine learning model that supports privacy-preserving techniques. Examples of privacy-preserving techniques include, but are not limited to, differential privacy, homomorphic encryption, shuffling, pixelation, etc.

[0066] In step S806, the image capture device 62 converts the anonymized image data into a plurality of anonymized features for output.

[0067] In step S808, the processing device 64 verifies the identity of the user to which the anonymized feature belongs using a trained machine learning model. The machine learning model is trained, for example, using the anonymized features and identities of multiple pre-registered users. Here, the processing device 64, for example, calculates, using the machine learning model, a similarity between the anonymized feature and a feature space established using the anonymized features of each pre-registered user, and verifies the identity of the user to which the anonymized feature belongs according to the calculated similarity.

[0068] In this embodiment, the increased speed of edge computing and cloud computing enables facial recognition to be performed efficiently, eliminating the need for account passwords or other hardware keys. Furthermore, the user's facial image is not uploaded to the cloud in its original form, allowing for secure identity verification without leaking personal information.

[0069] The design of the face recognition system is flexible, allowing it to be easily integrated and connected with existing systems and customized according to specific needs. Enterprises in different industries can quickly and easily integrate the face recognition system of the present embodiment into their existing devices or systems according to their own hardware device specifications and software requirements.

[0070] In some embodiments, the facial recognition system may be integrated into a single device for implementation, for example, the facial recognition system may be integrated into an electronic device such as a notebook computer or desktop computer, and may verify a user's identity while protecting the user's facial image from theft.

[0071] 9 is a block diagram of a face recognition system according to one embodiment of the present invention. Referring to FIG. 9, face recognition system 90 of this embodiment includes image capture device 92 and processing device 94. Here, the functions of image capture device 92 and processing device 94 are the same as or similar to the functions of image capture device 62 (or image capture device 200) and processing device 64 (or processing device 100) in the above embodiment, and therefore the details thereof will not be repeated here.

[0072] Unlike the previous embodiment, in this embodiment, the facial recognition system 90 may be a computer-based system, i.e., the image capture device 92 and the processing device 94 may be integrated into the same device.

[0073] The image capture device 92 includes an ISP supporting a neural network with an embedded AI-driven machine learning model, and is capable of anonymizing the captured facial image to obtain anonymized image data and converting the anonymized image data into a plurality of anonymized features.

[0074] The processing device 94 is connected via an interface device such as a USB (Universal Serial Bus) or a system bus, and the processor of the processing device 94 is provided with an API and a trained machine learning model is embedded therein. Here, the machine learning model is, for example, trained using anonymized features and identities of multiple pre-registered users and can be used to verify the identity of the user to which the anonymized features belong. Here, the processing device 94, for example, calculates, using the machine learning model, a similarity between the anonymized features and a feature space established using the anonymized features of each pre-registered user, and verifies the identity of the user to which the anonymized features belong according to the calculated similarity.

[0075] In summary, the face recognition system and method of the present invention have the following features.

[0076] The facial recognition system has a DNN processing scheme for privacy-preserving facial recognition and can be easily integrated with existing multi-factor identity verification systems.

[0077] The facial recognition system is an offloading system that can perform DNN training and recognition tasks in a private manner by designing privacy-preserving algorithms to trigger operations.

[0078] The face recognition system employs a DNN separation method optimized to prevent the first layer from being unloaded, which is the best choice considering the trade-off between computing resources, privacy loss, and model quality.

[0079] Any image data sent from the terminal device is anonymized and made private. At the same time, the false acceptance rate (FAR) is 10 ー6 In this case, the prediction / verification accuracy of the facial recognition system can be maintained at 99% or higher. [Industrial Applicability]

[0080] The monitoring system and monitoring method of the present invention can be applied to an image recognition system. [Explanation of symbols]

[0081] 10: Surveillance Systems 100, 64, 94: Processing unit 110: Machine Learning Models 120: Feature database 130: Image Database 20: Anonymized image 200, 62, 92: Image capture device 25: Monitored 30: Facial image 300: Cloud Server 31: Anonymization feature 32: Anonymization Label 50: History anonymization feature 60: Feature Space 600, 90: Facial recognition system 662, 720, 740: Facial images 664, 722, 742: Anonymized image data 666: Anonymization feature 668: Verification results 724: Anonymized Image Space 744: Anonymized feature vector 760: Similarity S201, S202, S203, S204, S301, S302, S303, S401, S402, S403, S404, S405, S406, S501, S502, S503, S504, S505, S602, S604, S606, S608, S710, S712, S714, S720, S722, S724, S730, S802, S804, S806, S808: Step

Claims

1. an image capture device for capturing an image; communicatively coupled to the image capture device; acquiring a face image of a monitoring target from the image; performing an anonymization process on the image to obtain an anonymized image, and outputting the anonymized image; performing a first anonymization operation on the facial image to generate anonymized features; generating a verification result according to the anonymization characteristics; a processing unit configured to execute including, a surveillance system.

2. The processor further comprises: generating the verification result by determining whether the anonymized features match features pre-stored in a feature database; The monitoring system of claim 1 configured to perform the following:

3. The processing device further comprises: performing a second anonymization operation on the face image to generate anonymization labels, and establishing a mapping relationship between the anonymization labels and the anonymized images to construct or update an image database; The monitoring system of claim 1 configured to perform the following:

4. the second anonymization operation is different from the first anonymization operation, and the processing device performs the first anonymization operation based on a differential privacy algorithm and performs the second anonymization operation based on a homomorphic encryption algorithm. The monitoring system of claim 3 .

5. The anonymization process for the image includes: using a machine learning model to cover the object in the image to generate the anonymized image. The monitoring system of claim 1 .

6. The first anonymization operation on the face image is performing the first anonymization operation on the facial image using a machine learning model that supports privacy preserving techniques. The monitoring system of claim 1 .

7. The machine learning model includes a plurality of neurons divided into a plurality of layers, and converts the face image into feature quantities of the plurality of neurons in a first layer of the plurality of layers, adds the feature values ​​of the converted neurons to noise generated using privacy parameters for input to a next layer, obtains an anonymized image after multi-layer processing, and converts the anonymized image into the anonymized features. The monitoring system of claim 6.

8. The processing device further comprises: verifying the identity of the monitored subject to which the anonymized features belong using a machine learning model trained using the anonymized features and identities of a plurality of pre-enrolled users to generate the verification result; The monitoring system of claim 1 configured to perform the following:

9. The processing device further comprises: performing a second anonymization operation on the facial image to generate an anonymization label; querying an image database according to the anonymization labels to obtain historical anonymized images corresponding to the anonymization labels; The monitoring system of claim 1 configured to perform the following:

10. The processing device further comprises: determining whether the verification was successful; and In response to the verification being successful, querying the image database according to the anonymization label to obtain the historical anonymized image corresponding to the anonymization label; 10. The monitoring system of claim 9, configured to perform the following:

11. Capturing an image; acquiring a face image of a monitoring target from the image; performing an anonymization process on the image to obtain an anonymized image, and outputting the anonymized image; performing a first anonymization operation on the facial image to generate anonymized features; generating a verification result according to the anonymization characteristics; A monitoring method, including:

12. generating the verification result according to the anonymization characteristics, determining whether the anonymized features match features pre-stored in a feature database and generating the verification result; The monitoring method of claim 11.

13. performing a second anonymization operation on the facial image to generate an anonymization label; establishing a mapping relationship between the anonymization labels and the anonymized images to build or update an image database; The monitoring method of claim 11 further comprising:

14. the second anonymization operation is different from the first anonymization operation, the first anonymization operation is performed based on a differential privacy algorithm, and the second anonymization operation is performed based on a homomorphic encryption algorithm. The monitoring method of claim 13.

15. The step of performing the anonymization process on the image to obtain the anonymized image includes: using a machine learning model to cover the object in the image to generate the anonymized image. The monitoring method of claim 11.

16. The first anonymization operation on the face image is performing the first anonymization operation on the facial image using a machine learning model that supports privacy preserving techniques. The monitoring method of claim 11.

17. The machine learning model includes a plurality of neurons divided into a plurality of layers, and converts the face image into feature quantities of the plurality of neurons in a first layer of the plurality of layers, adds the feature values ​​of the converted neurons to noise generated using privacy parameters for input to a next layer, obtains an anonymized image after multi-layer processing, and converts the anonymized image into the anonymized features.

17. The monitoring method of claim 16.

18. generating the verification result according to the anonymization characteristics, verifying the identity of the monitored subject to which the anonymized features belong using a machine learning model trained using the anonymized features and identities of a plurality of pre-enrolled users to generate the verification result. The monitoring method of claim 11.

19. performing a second anonymization operation on the facial image to generate an anonymization label; querying an image database according to the anonymization labels to obtain historical anonymized images corresponding to the anonymization labels; The monitoring method of claim 11 further comprising:

20. an image capture device for capturing an image; communicatively coupled to the image capture device; obtaining a face image of a target from the image, and performing an anonymization process on the image to generate an anonymization label; performing an anonymization process on the image to obtain an anonymized image; establishing a mapping relationship between the anonymization labels and the anonymized images to build or update an image database; outputting the anonymized image stored in the image database in response to receiving a query command that matches the anonymization label; a processing unit configured to execute including, a surveillance system.

Images