Method and apparatus for advanced convolution on encrypted data

Abstract

Methods and apparatus for performing advanced convolutions on encrypted data are provided. One apparatus includes a processor programmed to: define an input matrix and a kernel matrix based on the encrypted data; identify an algebraic structure of an encryption method applied to the encrypted data; determine primitive roots of unity in the algebraic structure in response to the size of the input matrix and the size of the kernel matrix; transform the input matrix and the kernel matrix into transformed input matrices and transformed kernel matrices using the primitive roots of unity; compute element-wise multiplications of the transformed input matrix and the transformed kernel matrix; apply an inverse discrete Fourier transform; and output a convolution of the input matrix and the kernel matrix based on the encrypted data.

Description

Technical Field

[0001] This disclosure relates to the processing of encrypted data, including data such as image data or sensor data. Background Technology

[0002] Secure multi-party computation (MPC) is a field in cryptography that provides a way for many parties to jointly compute functions on private inputs, such as encrypted data. In MPC, each party receives a "share" of the inputs on which they want to compute functions. It provides a way to keep inputs private and exempt from participation in MPC. Furthermore, many parties use MPC to jointly compute several functions of interest.

[0003] On the other hand, deep learning (DL) can be considered a subfield of machine learning, which has challenges in image recognition. The incorporation of MPC (Multi-Process Computation) into the deep learning setting has become more intuitive for protecting data privacy, but it also allows for computational efficiency through new techniques in deep learning. However, achieving DL methods that work as efficiently on encrypted data as they do on plaintext data can be difficult.

[0004] The SPDZ protocol can be used as an MPC protocol. SPDZ can utilize standard secret-sharing methods to share function inputs privately. Secret-sharing algorithms can be used in cryptographic methods. SPDZ enables the application of deep learning computations to encrypted data of any type. Summary of the Invention

[0005] According to one embodiment, a method for processing encrypted data includes defining an input matrix and a kernel matrix based on the encrypted data, identifying an algebraic structure of an encryption method applied to the encrypted data, determining a primitive root of unity in the algebraic structure in response to the size of the input matrix and the size of the kernel matrix, transforming the input matrix and the kernel matrix into transformed input matrices and transformed kernel matrices using the primitive root of unity, calculating element-wise multiplications of the transformed input matrix and the transformed kernel matrix, applying an inverse discrete Fourier transform, and outputting a convolution of the input matrix and the kernel matrix based on the encrypted data.

[0006] According to a second embodiment, a computer-implemented method for processing encrypted data includes defining an input matrix and a kernel matrix based on the encrypted data, identifying the algebraic structure of the encryption method applied to the encrypted data, determining the primitive root of unity in the algebraic structure in response to the size of the input matrix and the size of the kernel matrix, transforming the input matrix and the kernel matrix into transformed input matrices and transformed kernel matrices using the primitive root of unity, calculating element-wise multiplication of the transformed input matrix and the transformed kernel matrix and the inverse discrete Fourier transform of the multiplication, and outputting a convolution of the input matrix and the kernel matrix based on the encrypted data.

[0007] According to a third embodiment, an apparatus includes a processor programmed to: define an input matrix and a kernel matrix based on encrypted data; identify an algebraic structure of an encryption method applied to the encrypted data; determine a primitive root of unity in the algebraic structure in response to the size of the input matrix and the size of the kernel matrix; transform the input matrix and the kernel matrix into transformed input matrices and transformed kernel matrices using the primitive root of unity; compute element-wise multiplication of the transformed input matrix and the transformed kernel matrix; apply an inverse discrete Fourier transform; and output a convolution of the input matrix and the kernel matrix based on the encrypted data. Attached Figure Description

[0008] Figure 1 An exemplary data annotation system 100 for implementing a system for annotating data is described.

[0009] Figure 2 An exemplary flowchart is depicted that utilizes DFT to apply convolution to encrypted data.

[0010] Figure 3 The illustration shows a computing system for controlling a robot that is at least partially autonomous.

[0011] Figure 4 This is an example of a computer system 40 used to control manufacturing machines.

[0012] Figure 5 The diagram illustrates a computing system that controls an automated personal assistant.

[0013] Figure 6 The diagram illustrates the computing system of the control system for the access control system.

[0014] Figure 7 The diagram illustrates the computing system of the control system of the control and monitoring system.

[0015] Figure 8 The diagram illustrates the computational system of the control system that controls the imaging system. Detailed Implementation

[0016] This document describes embodiments of the present disclosure. However, it will be understood that the disclosed embodiments are merely examples, and other embodiments may take various forms and alternative forms. The figures are not necessarily to scale; some features may be enlarged or minimized to show details of specific components. Therefore, the specific structural and functional details disclosed herein should not be construed as limiting, but only as a representative basis for teaching those skilled in the art to employ the embodiments in different ways. As will be understood by those skilled in the art, various features illustrated and described with reference to any of the figures may be combined with features illustrated in one or more other figures to produce embodiments not explicitly illustrated or described. The combinations of illustrated features provide representative embodiments for typical applications. However, various combinations and modifications of features consistent with the teachings of this disclosure may be desired for a particular application or implementation.

[0017] The system and method described below propose a faster method for convolution computation on encrypted data by leveraging the Discrete Fourier Transform (DFT). Advantages include the formal definition of the DFT within a general algebraic ring structure, and how it is integrated into the algebraic structure of the SPDZ encryption protocol. It also analyzes the complexity of the proposed method and provides its correctness. (Regarding kernel matrix size...) k take k This method can also achieve speeds up to [number missing]. k Therefore, DFT can compute faster convolutions of encrypted data in the SPDZ setting. This system and method can utilize different versions of DFT in different settings.

[0018] One drawback of deep learning (DL) in cryptographic primitives is the significant slowdown in computation because operations are performed on encrypted data in a more complex manner. Specifically, DL uses convolutional layers to extract features from the input data. In image recognition, the input image is represented as a matrix with its pixels, and some kind of filtering is applied using a small matrix. In many applications, convolutional computation on encrypted data can become cumbersome. In this application, the system can take the DFT and combine the DFT, machine learning, and encryption scheme to run faster convolutional computations on encrypted data. In an embodiment, encryption can be performed via a specific secret-sharing protocol called SPDZ. Performing convolutions on secret-shared data can accelerate processing.

[0019] Figure 1An exemplary data annotation system 100 implementing the system is depicted. Data system 100 may include at least one computing system 102. Computing system 102 may include at least one processor 104 operatively connected to memory unit 108. Processor 104 may include one or more integrated circuits implementing the functionality of a central processing unit (CPU) 106. CPU 106 may be a commercially available processing unit implementing an instruction set, such as one of the x86, ARM, Power, or MIPS instruction set families. During operation, CPU 106 may execute stored program instructions retrieved from memory unit 108. The stored program instructions may include software controlling the operation of CPU 106 to perform the operations described herein. In some examples, processor 104 may be a system-on-a-chip (SoC) that integrates the functionality of CPU 106, memory unit 108, network interface, and input / output interface into a single integrated device. Computing system 102 may implement an operating system for managing various aspects of operation.

[0020] Memory cell 108 may include volatile and non-volatile memory for storing instructions and data. Non-volatile memory may include solid-state memory, such as NAND flash memory, magnetic and optical storage media, or any other suitable data storage device that retains data when the computing system 102 is disabled or loses power. Volatile memory may include static and dynamic random access memory (RAM) for storing program instructions and data. For example, memory cell 108 may store a machine learning model 110 or algorithm, a training dataset 112 for the machine learning model 110, and the original source dataset 115.

[0021] The computing system 102 may include a network interface device 122 configured to provide communication with external systems and devices. For example, the network interface device 122 may include wired and / or wireless Ethernet interfaces as defined by the Institute of Electrical and Electronics Engineers (IEEE) 802.11 family of standards. The network interface device 122 may include a cellular communication interface for communicating with cellular networks (e.g., 3G, 4G, 5G). The network interface device 122 may be further configured to provide a communication interface to an external network 124 or the cloud.

[0022] External network 124 may be referred to as the World Wide Web or the Internet. External network 124 can establish standard communication protocols between computing devices. External network 124 can allow information and data to be easily exchanged between computing devices and the network. One or more servers 130 can communicate with external network 124.

[0023] The computing system 102 may include an input / output (I / O) interface 120, which may be configured to provide digital and / or analog inputs and outputs. The I / O interface 120 may include an additional serial interface (e.g., a Universal Serial Bus (USB) interface) for communicating with external devices.

[0024] The computing system 102 may include a human-machine interface (HMI) device 118, which may include any device that enables the system 100 to receive control input. Examples of input devices may include human interface inputs such as a keyboard, mouse, touchscreen, voice input device, and other similar devices. The computing system 102 may include a display device 132. The computing system 102 may include hardware and software for outputting graphical and textual information to the display device 132. The display device 132 may include an electronic display screen, projector, printer, or other suitable device for displaying information to a user or operator. The computing system 102 may be further configured to allow interaction with remote HMIs and remote display devices via a network interface device 122.

[0025] System 100 can be implemented using one or more computing systems. While this example depicts a single computing system 102 implementing all the described features, it is intended that various features and functions can be decoupled and implemented by multiple computing units communicating with each other. The specific system architecture chosen can depend on a variety of factors.

[0026] System 100 can implement a machine learning algorithm 110 configured to analyze a raw source dataset 115. The raw source dataset 115 may include raw or unprocessed sensor data, which may represent the input dataset for the machine learning system. The raw source dataset 115 may include video, video clips, images, text-based information, and raw or partially processed sensor data (e.g., radar images of objects). The raw source dataset 115 may also include encrypted data based on video, video clips, images, text-based information, sensor data, or other types of data. In some examples, the machine learning algorithm 110 may be a neural network algorithm designed to perform a predetermined function. For example, a neural network algorithm may be configured in an automotive application to identify pedestrians in video images.

[0027] Computer system 100 may store a training dataset 112 for machine learning algorithm 110. Training dataset 112 may represent a collection of previously constructed data used to train machine learning algorithm 110. Machine learning algorithm 110 can use training dataset 112 to learn weighting factors associated with a neural network algorithm. Training dataset 112 may include a set of source data having corresponding outputs or results that machine learning algorithm 110 attempts to replicate via a learning process. In this example, training dataset 112 may include source videos with and without pedestrians, along with corresponding presence and location information. The source videos may include various contexts in which pedestrians are identified.

[0028] Machine learning algorithm 110 can operate in a learning mode using training dataset 112 as input. Machine learning algorithm 110 can be executed in multiple iterations using data from training dataset 112. With each iteration, machine learning algorithm 110 can update its internal weighting factors based on the achieved results. For example, machine learning algorithm 110 can compare its output (e.g., annotations) with those included in training dataset 112. Since training dataset 112 includes expected results, machine learning algorithm 110 can determine when performance is acceptable. After machine learning algorithm 110 achieves a predetermined performance level (e.g., consistent with 100% of the output associated with training dataset 112), machine learning algorithm 110 can be executed using data not in training dataset 112. The trained machine learning algorithm 110 can be applied to new datasets to generate annotated data.

[0029] Machine learning algorithm 110 can be configured to identify specific features in raw source data 115. Raw source data 115 can include multiple instances or input datasets for which it expects annotation results. For example, machine learning algorithm 110 can be configured to identify the presence of pedestrians in video images and annotate their presence. Machine learning algorithm 110 can be programmed to process raw source data 115 to identify the presence of specific features. Machine learning algorithm 110 can be configured to identify features in raw source data 115 as predetermined features. Raw source data 115 can be derived from various sources. For example, raw source data 115 can be actual input data collected by a machine learning system. Raw source data 115 can be machine-generated for testing a system. As an example, raw source data 115 can include raw video images from a camera. Raw source data 115 can also include plaintext data (e.g., unencrypted data) or encrypted data.

[0030] In this example, machine learning algorithm 110 can process the raw source data 115 and generate outputs. Machine learning algorithm 110 can generate confidence levels or factors for each generated output. For example, a confidence value exceeding a predetermined high confidence threshold can indicate that machine learning algorithm 110 is confident that the identified feature corresponds to a specific feature. A confidence value below a low confidence threshold can indicate that machine learning algorithm 110 has some uncertainty regarding the existence of a specific feature.

[0031] Figure 2 An exemplary flowchart illustrating the application of DFT to convolution in encrypted data is provided. Convolution can be a mathematical operation where two functions produce a third function, describing how the shape of one function is modified by the other. Convolution can refer both to the resulting function and the process of computing it, as well as the integral of the product of two functions after one function has been inverted and shifted. It can be applied to 2D matrices of integer size, such as images represented in 2D. M take N The input matrix I (i.e., the number of rows multiplied by the number of columns) and the matrix having an integer size a take b On the kernel matrix K. Therefore, I It can be regarded as a pair take A function that maps to real numbers. Similarly, K can be viewed as a pair of... take A function mapped to real numbers. Circular convolution is applicable when one of two aperiodic functions is convolved with the period and sum of the periods of the other function in a normal manner. Circular convolution is a mathematical representation of normal convolution in mathematical constructs. Circular convolution must output the same result as normal convolution. To achieve this, the system can construct circular convolutions. It is equal to ordinary convolution. C mod R, where R is a ring. At step 201, the system can identify the algebraic ring structure (R) of the encryption mechanism.

[0032] for and The ordinary convolution function on the input matrix and the kernel matrix can be defined as:

[0033]

[0034] Given C yes I and K When the system calculates a convolution using such a formula, the number of multiplications will be... Given I and KThe fact that the data is encrypted, and the high cost of multiplications required for communication between servers on the encrypted data, will be shown in the following formula (1):

[0035]

[0036] The following section describes this disclosure, which shows how to avoid additional encrypted multiplication by transforming encrypted data in another form.

[0037] The following Table 1 lists all the symbols used throughout the description of this disclosure:

[0038] Table 1: Symbol Table

[0039] symbol meaning The input image matrix and a function for mapping the indices of the input matrix to their values. The kernel matrix and the function used to map kernel matrix indices to their values. The convolution function of sum , corresponding DFT transformation Number of rows and columns of the unfilled input matrix Number of rows and columns of the unfilled kernel matrix The closest larger factor from (m-1) to M The closest larger factor from (m-1) to N SPDZ prime modulus In the ring R, the primitive unit root (,)

[0040] Given encrypted data that includes its own algebraic structure, a convolution using the DFT on the encrypted data can be unique. The system can define the encrypted input matrix in a way that allows the application of the DFT. For example, since the system can define... I and K Therefore, to protect the privacy of the input image, the system performs matrix... I sum matrix K Each entry is encrypted. Then, the convolution function to be applied to the input image and the given kernel can mean that the input matrix can be viewed as acting on a function representing the input image. I Furthermore, the kernel matrix can be viewed as a function. K Therefore, by using two variables, namely... and Define function I and K Where x is the row number and y is the column number in the corresponding matrix. The system can use the SPDZ library to apply encryption to hide the data being processed. The system can use... another The library will use functions I and K These encrypted versions are "transformed" into a structure in which ordinary convolution operations (and the corresponding circular convolutions) still produce the same result without this transformation. This mathematical transformation is called the DFT. The DFT allows functions to be written as linear combinations of characters over groups. In the SPDZ algebraic structure as a ring R, the system can operate on characters defined from groups to the ring—such as those relative to complex numbers. Then, the system may need to define (in the SPDZ ring R)... P (corresponding) Q The primitive unit root is denoted as α (corresponding to β) for use with DFT.

[0041] At step 203, the system calculates the value in R. PPrimal root of unity. Given an integer. P The system can allow α to be a ring P Primal unit roots. The system may require... For all It is reversible.

[0042] At step 203, the system calculates the value in R. Q Primal root of unity. Given an integer. Q The system can allow β to be in the ring. Q Primordial unit root. The system can assume For all It is reversible. The system can convert the encrypted input matrix... I and kernel matrix K The size is taken accordingly M take N and a take b And fill them with DFT transformation. P take Q Size.

[0043] As defined in Table 1, P It can be reached M The closest larger prime number, M It is the input matrix I The number of rows in the input matrix. I This can include encrypted data, such as image data obtained from a camera or image capture device, or sensor data obtained from a sensor. The SPDZ protocol software can apply computations to the input matrix. I encryption.

[0044] This method ensures that equality in the encrypted structure (ring) implies equality in the related values. Therefore, when the system transforms the encrypted data, the elements of a typical convolution can be... ab The dot product over the terms. Each term can be derived from... Define. If the modulus is greater than Equal moduli mean equal values. If the modulus is low, the system may not be certain whether they are equal. In the SPDZ library, a fixed-point representation called sfix is ​​used to represent encrypted data. On the other hand, the DFT transformation works on the integer representation of the encrypted sfix value (called sint). Therefore, the upper bound B should be an upper bound on the elements after the transformation from sfix to sint. In the SPDZ library, which uses 20 bits to represent the integer part of the sfix value, it can utilize 20 bits to represent the fractional value. Therefore, each element of the encryption matrix will be multiplied by 2. 20 To obtain the integer representation of the encrypted value. Therefore, if I andK All values ​​are Within the range, the modulus should be at least 1. In this application, for a given modulus m, the ring is... m is a prime number.

[0045] The system should take as little as possible and This minimizes the computational overhead of encrypting data. However, the system should first determine the... P and Q The corresponding primitive root of unity exists. For example, the system can take... The factors. The following content further explains how to identify them. P Value and Q value.

[0046] SPDZ ring can be , where m is a prime number, therefore Therefore, when calculating primitive roots, the set It has an order of m–1. According to Lagrange's theorem, the order of any element in a group can always be a factor of the order of the group, i.e., m–1. Therefore, in order to have P The original unit root We must make P As a factor of m–1, the system can select... generator g Then the system can calculate α as:

[0047]

[0048] The system should ensure that α is not equal to 1. Accordingly, α is the modulus m of the invertible ring, since m is a prime number. Therefore, when m is a prime number, the ring... The calculation of the primitive root of unity above can be done in a simple way.

[0049] At step 207, the system can determine and The system can determine the encryption of data. and This does not require any expensive computations such as data exchange between servers. Therefore, as opposed to ordinary convolution computations that may require data exchange between servers during the convolution process of encrypted data, the disclosed embodiments determine... and It may not require exchanging data between servers; instead, it can be performed very quickly on a single server, just as if it were operating on unencrypted data. Therefore, this step may involve operations on publicly available data, which, in terms of computational power, does not cost as much as operating on encrypted data.

[0050] The DFT transform of the input image matrix can be:

[0051] For all take , .

[0052] Only the function / matrix of I is encrypted, while the values ​​of α and β are publicly available.

[0053] The DFT transformation of the kernel matrix can be performed using a similar formula:

[0054] For all take , .

[0055] Only the function / matrix of K is encrypted; the values ​​of α and β are publicly available.

[0056] The convolution function can be determined using the DFT transformation of the kernel matrix and the input image matrix.

[0057] At step 209, the system can calculate (For example, in) I and K (C-cap on the DFT transform of the convolution function). Can be with Same, among which It is an element-wise product. The system can utilize the controller or processor as follows: Calculate C:

[0058] For all take , .

[0059] For the system to function correctly, it is essential to ensure that DFT convolutions utilize circular convolutions to compute the correct output. Similar to regular convolutions, DFT convolutions should also ensure that the DFT circular convolutions over encrypted data are well-defined. It can be defined as:

[0060]

[0061] That is, for all take , Therefore, the system can freely compute... and Calculate under the following circumstances In addition, the system can utilize PQ Ring multiplication calculation At step 210, the system can... Calculate freely The expensive number of multiplications can be derived from... Reduce to PQ It is a rough division by ab (For example, the number of rows and columns of an unfilled kernel matrix). In terms of computational power, such operations may involve publicly available data that does not utilize as many resources as operations on encrypted data.

[0062] At step 211, the system can output C. This output can be used in deep learning systems that utilize neural networks. For example, the output can help the neural network classify image data.

[0063] Therefore, the complexity estimate can be reduced based on the server's processor processing capacity. The online phase of the SPDZ algorithm for multiplication can be optimized. send and . square Can send and Such calculations can include four subtractions. and calculate and This can include two subtractions, each of which implies four subtractions. calculate This can include 3 additions and 3 multiplications. calculate This could include 3 additions and 3 multiplications. Therefore, it might require a total of 8 subtractions, 6 additions, and 6 multiplications. The system can count subtractions and additions as a single category.

[0064] Assuming the multiplication of two shared parts is equivalent to 14 additions and 6 multiplications—a rough estimate of what happens during the online phase—we can apply... The cost of the convolution formula. To calculate... S For the DFT, we can calculate the following table:

[0065] .

[0066] When applying DFT to encrypted data, the expensive number of multiplications requiring server communication becomes...

[0067] .

[0068] for The ratio of equations (2) and (1), which correspond to the expensive encrypted multiplication between DFT convolution and ordinary convolution, can be as small as the following equation.

[0069] .

[0070] Figure 3 An embodiment is shown in which a computing system 340 is used to control a robot that is at least partially autonomous, such as a vehicle 300 that is at least partially autonomous. The computing system 340 may be similar to... Figure 1 The computing system described herein. Sensor 330 may include one or more video sensors and / or one or more radar sensors and / or one or more ultrasonic sensors and / or one or more LiDAR sensors and / or one or more location sensors (such as GPS). Some or all of these sensors are preferred but not necessarily integrated into vehicle 300.

[0071] Alternatively or additionally, sensor 330 may include an information system for determining the state of the actuator system. Sensor 330 may collect sensor data or other information to be used by computing system 340. An example of such an information system is a weather information system that determines the current or future state of weather in the environment. For example, by using an input signal x, a classifier can, for example, detect objects near a robot that is at least partially autonomous. The output signal y may include information characterizing where the object is located near the robot that is at least partially autonomous. A control command A can then be determined based on this information, for example, to avoid a collision with the detected object.

[0072] The actuator 310, which can be integrated into the vehicle 300, can be provided by the vehicle 300's brakes, propulsion system, engine, drivetrain, or steering mechanism. An actuator control command can be determined to control the actuator (or multiple actuators) 310 so that the vehicle 300 avoids collisions with the detected objects. The detected objects can also be classified according to what a classifier considers most likely to be (e.g., pedestrians or trees), and the actuator control command A can be determined based on this classification.

[0073] In another embodiment, the at least partially autonomous robot may be provided by another mobile robot (not shown), which may move, for example, by flying, swimming, diving, or walking. This mobile robot may, in particular, be a lawnmower that is at least partially autonomous, or a cleaning robot that is at least partially autonomous. In all the embodiments above, actuator command control A may be determined such that the propulsion unit and / or steering device and / or brakes of the mobile robot are controlled, enabling the mobile robot to avoid collisions with the identified object.

[0074] In another embodiment, at least partially autonomous robotic operation may be provided by a gardening robot (not shown) that uses sensor 330, preferably an optical sensor, to determine the state of plants in the environment. Actuator 310 may be a nozzle for spraying chemicals. Depending on the identified species and / or identified state of the plant, actuator control command A may be determined to cause actuator 310 to spray the plant with an appropriate amount of appropriate chemical.

[0075] In another embodiment, at least partially autonomous robots may be provided by household appliances (not shown), such as washing machines, stoves, ovens, microwave ovens, or dishwashers. Sensors 330, such as optical sensors, can detect the state of objects to be processed by the household appliance. For example, in the case of a washing machine, sensor 330 can detect the state of the clothes inside the washing machine. The actuator control signal A can then be determined based on the detected clothing material.

[0076] Figure 4 The illustration shows an embodiment in which a computer system 440 controls a manufacturing machine 411 (e.g., a stamping tool, cutting tool, or gun drill) of a manufacturing system 400 (e.g., as part of a production line). The control system 440 controls an actuator 410, which in turn controls the manufacturing machine 411.

[0077] Sensor 430 may be provided by an optical sensor that captures, for example, the attributes of the manufactured product 412. Sensor 430 can collect data on such attributes of the manufactured product and the operation of actuator 410. A classifier can determine the state of the manufactured product 412 based on these captured attributes. Then, the actuator 410 controlling the manufacturing machine 411 can be controlled depending on the determined state of the manufactured product 412 for subsequent manufacturing steps of the manufactured product 412. Alternatively, it can be envisioned that the actuator 410 is controlled depending on the determined state of the manufactured product 412 during subsequent manufacturing of the manufactured product 412.

[0078] Figure 5 The illustration shows an embodiment in which computer system 540 controls automated personal assistant 550. Sensor 530 may be an optical sensor, such as a video image of a gesture from user 549. Alternatively, sensor 530 may also be an audio sensor, such as a sensor for receiving voice commands from user 549.

[0079] The control system 540 then determines an actuator control command A for controlling the automated personal assistant 550. The actuator control command A is determined based on a sensor signal S from sensor 530. The sensor signal S is transmitted to the control system 540. For example, a classifier can be configured to implement a gesture recognition algorithm to identify gestures made by the user 549. The control system 540 can then determine the actuator control command A for transmission to the automated personal assistant 550. It then transmits the actuator control command A to the automated personal assistant 550.

[0080] For example, actuator control command A can be determined based on the identified user gesture recognized by a classifier. It can then include information that causes the automated personal assistant 550 to retrieve information from a database and output the retrieved information in a form suitable for reception by the user 549.

[0081] In another embodiment, instead of the automated personal assistant 550, the control system 540 may control a household appliance (not shown) controlled according to an identified user gesture. The household appliance may be a washing machine, stove, oven, microwave oven, or dishwasher.

[0082] Figure 6 The illustration shows an embodiment of a computing system controlling an access control system 600. The access control system can be designed to physically control access. For example, it may include a door 601. A sensor 630 is configured to detect and determine the scene in relation to whether access should be authorized. The sensor 630 may collect image or video data related to the scene. For example, it may be an optical sensor used to provide image or video data for detecting faces. A classifier can be configured to interpret the image or video data, for example, by matching the identity with known people stored in a database, thereby determining the identity of the person. Then, depending on the classifier's interpretation, for example, based on the determined identity, an actuator control signal A can be determined. The actuator 610 may be a lock that authorizes or disauthorizes access based on the actuator control signal A. Non-physical, logical access control is also possible.

[0083] Figure 7 The illustration shows an embodiment of the computing system 740 controlling and monitoring the system 700. This embodiment is largely similar to... Figure 5The embodiments shown are equivalent. Therefore, only the different aspects will be described in detail. Sensor 730 is configured to detect a scene under supervision. Sensor 730 may collect image or video data related to the scene. The computing system does not necessarily control actuator 710, but rather controls display 710a. For example, a machine learning system may determine the classification of the scene, such as whether the scene detected by optical sensor 730 is suspicious. Actuator control signal A transmitted to display 710a may then be configured, for example, to cause display 710a to adjust the displayed content based on the determined classification, for example, to highlight objects deemed suspicious by the machine learning system.

[0084] Figure 8 The illustration shows an embodiment of a computing system 840 for controlling an imaging system 800, such as an MRI apparatus, an X-ray imaging apparatus, or an ultrasound imaging apparatus. The sensor 830 may be, for example, an imaging sensor. The machine learning system can then determine a classification of all or part of the sensed image. The computing system 840 can collect data associated with the image from the X-ray or ultrasound imaging. An actuator control signal A can then be selected based on this classification to control the display 810a. For example, the machine learning system may interpret a region of the sensed image as potentially anomalous. In this case, the actuator control signal A can be determined to cause the display 810a to display the image and highlight the region of potential anomalousness.

[0085] The processes, methods, or algorithms disclosed herein may be deliverable to / implemented by a processing device, controller, or computer, which may include any existing programmable electronic control unit or dedicated electronic control unit. Similarly, processes, methods, or algorithms may be stored in a variety of forms as data and instructions executable by a controller or computer, including but not limited to information permanently stored on non-writable storage media such as ROM devices and information reproducibly stored on writable storage media such as floppy disks, magnetic tapes, CDs, RAM devices, and other magnetic and optical media. Processes, methods, or algorithms may also be implemented in a software executable object. Alternatively, processes, methods, or algorithms may be embodied, wholly or partially, using suitable hardware components such as application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), state machines, controllers, or other hardware components or devices, or a combination of hardware, software, and firmware components.

[0086] While exemplary embodiments have been described above, they are not intended to describe all possible forms covered by the claims. The terms used in this specification are descriptive and not limiting, and it should be understood that various changes may be made without departing from the spirit and scope of this disclosure. As previously stated, features of various embodiments may be combined to form other embodiments of the invention that may not be explicitly described or illustrated. While various embodiments may have been described as providing advantages over or preferred implementations of other embodiments or prior art in one or more desired features, those skilled in the art will recognize that one or more features or characteristics may be compromised depending on the specific application and implementation to achieve the desired overall system properties. These properties may include, but are not limited to, cost, strength, durability, lifecycle cost, merchantability, appearance, packaging, size, suitability, weight, manufacturability, ease of assembly, etc. Accordingly, any embodiment described as less desirable in one or more features than other embodiments or prior art is not outside the scope of this disclosure and may be desirable for a particular application.

Claims

1. A method for processing encrypted data, comprising: The input matrix and kernel matrix are defined based on encrypted data; An algebraic structure that identifies the encryption method used to encrypt data; The primitive root of unity in the algebraic structure is determined in response to the size of the input matrix and the size of the kernel matrix; The original root of unity is used to transform the input matrix and kernel matrix into transformed input matrices and transformed kernel matrices; Calculate the element-wise multiplication of the transformed input matrix and the transformed kernel matrix; Apply the inverse discrete Fourier transform to the calculation; as well as Convolutions that output input and kernel matrices based on encrypted data.

2. The method of claim 1, wherein the encrypted data includes sensor data.

3. The method of claim 1, wherein the encrypted data includes image data.

4. The method according to claim 1, wherein the method includes encrypting the input matrix and implementing the kernel matrix using the SPDZ protocol.

5. The method of claim 1, wherein the input matrix and the kernel matrix are of different sizes.

6. The method of claim 5, wherein the kernel matrix is ​​smaller than the input matrix.

7. The method of claim 1, wherein the primitive root of unity is a publicly available value.

8. The method according to claim 1, wherein, The output of the inverse discrete Fourier transform is the original bit structure of the encrypted data.

9. The method of claim 1, wherein the convolution utilizes the same encryption protocol as the encrypted data.

10. A computer-implemented method for processing encrypted data, comprising: The input matrix and kernel matrix are defined based on encrypted data; An algebraic structure that identifies the encryption method used to encrypt data; The primitive root of unity in the algebraic structure is determined in response to the size of the input matrix and the size of the kernel matrix; The original root of unity is used to transform the input matrix and kernel matrix into transformed input matrices and transformed kernel matrices; Calculate the element-wise multiplication of the transformed input matrix and the transformed kernel matrix, and the inverse discrete Fourier transform of the multiplication; as well as Convolutions that output input and kernel matrices based on encrypted data.

11. The computer-implemented method of claim 10, wherein the input matrix is ​​determined and the kernel matrix is ​​implemented using the SPDZ protocol.

12. The computer-implemented method of claim 10, wherein the input matrix comprises a first size having a first row and a first column, and the kernel matrix comprises a second size having a second row and a second column, wherein the convolution is output with a third size comprising a third row greater than the difference between the first row and the second row and a third column greater than the difference between the first column and the second column.

13. The computer-implemented method of claim 10, wherein the encrypted data includes sensor data.

14. The computer-implemented method of claim 10, wherein the encrypted data includes image data.

15. An apparatus comprising: The processor is programmed to: The input matrix and kernel matrix are defined based on encrypted data; An algebraic structure that identifies the encryption method used to encrypt data; The primitive root of unity in the algebraic structure is determined in response to the size of the input matrix and the size of the kernel matrix; The original root of unity is used to transform the input matrix and kernel matrix into transformed input matrices and transformed kernel matrices; Calculate the element-wise multiplication of the transformed input matrix and the transformed kernel matrix; Apply the inverse discrete Fourier transform to the calculation; as well as Convolutions that output input and kernel matrices based on encrypted data.

16. The apparatus of claim 15, wherein the encrypted data includes sensor data.

17. The apparatus of claim 15, wherein the encrypted data includes image data.

18. The apparatus of claim 15, wherein the controller is further configured to determine the convolution input matrix and convolution kernel matrix over the encrypted data at a single server.

19. The apparatus of claim 15, wherein the controller is configured to determine the input matrix and the kernel matrix using the SPDZ protocol.

20. The apparatus of claim 15, wherein the input matrix has a first size having a first row and a first column, and the kernel matrix has a second size having a second row and a second column, wherein the convolution is output with a third size, the third size including a third row greater than the difference between the first row and the second row and a third column greater than the difference between the first column and the second column.

Images