Apparatus and method for providing deep learning service for protecting original data

Abstract

The present invention relates to an apparatus and a method for providing a deep learning service. The apparatus for providing a deep learning service, according to one embodiment, comprises: a transmission / reception device for receiving a deep learning execution request from a data owner or an artificial intelligence (AI) service provider and a deep learning execution result from the AI service provider, and transmitting, to the data owner and the AI service provider performing deep learning, whether authority for deep learning execution is held and hyperparameters; and a processor for determining, in response to the deep learning execution request, whether the data owner holds authority, and providing, on the basis of whether the authority is held, the deep learning service by requesting the data owner and the AI service provider to perform deep learning.

Description

Device and method for providing deep learning services for original data protection

[0001] The present invention relates to a device and method for providing a deep learning service for protecting original data.

[0002] The present invention is derived from a project (24AB1600, development of intelligent technology for key industries and autonomous human-mobile-space autonomous collaboration intelligence technology) carried out as part of the Ulsan Metropolitan City-ETRI (2nd) joint cooperation project.

[0003] Deep learning can be defined as a set of machine learning algorithms that attempt a high level of abstraction through a combination of various non-linear transformation techniques. In a broad sense, deep learning is a field of machine learning that teaches computers human ways of thinking.

[0004] Due to Fourth Industrial Revolution technologies such as artificial intelligence (AI) and big data, the era of the data economy is opening up, in which the utilization of data acts as a catalyst for the development of all industries and the creation of new value.

[0005] As the AI ​​industry has flourished, the demand for data has increased significantly, and simultaneously, the importance of data sovereignty has also risen. However, data owners do not wish to expose their data externally, while AI service providers and launchers require various types of data to perform deep learning. Consequently, there is a need for corresponding supply and demand.

[0006] Therefore, there is a growing need for reliable deep learning techniques that support data confidentiality from the perspective of a third party, rather than the data owner or the AI ​​service provider.

[0007] Considering these problems, the present invention aims to provide a deep learning service provision apparatus and method in which learning of neural networks separated from data providers and AI service providers is performed using a divisible neural network.

[0008] A deep learning service providing device according to one embodiment of the present disclosure for achieving the above-described technical problem includes: a transceiver that receives a deep learning execution request from a data owner or an AI (Artificial Intelligence) service provider and a deep learning execution result from the AI ​​service provider, and transmits whether the data owner has authority for the deep learning execution and hyperparameters to the data owner and the AI ​​service provider; and a processor that determines whether the data owner has authority in response to the deep learning execution request, and provides a deep learning service by requesting the data owner and the AI ​​service provider to perform deep learning based on whether the data owner has authority.

[0009] The neural network that is the subject of the deep learning execution includes a separation front network and a separation back network, and the processor can transmit parameters related to the neural network to the data owner based on whether the authority is held, and receive the median value and the correct answer value of the lower network separation back network from the data owner and transmit them to the AI ​​service provider.

[0010] The processor can receive a gradient based on the output of a separation overall network learned using the intermediate value and the correct answer value from the AI ​​service provider and transmit it to the data owner.

[0011] The processor can change the deep learning execution state based on parameters related to the late-separation network updated based on the gradient, the deep learning execution epoch, the hash value of the early-separation network, and the hash value of the late-separation network.

[0012] The processor may, in response to the interruption of the deep learning service, obtain the timeout point at which the deep learning service was interrupted, and transmit a request to the data owner or the AI ​​service provider to restart the deep learning service based on the timeout point, the number of epochs executed up to the timeout point, and the batch number executed up to the timeout point.

[0013] The above processor can determine whether the authority is possessed based on a certificate including the certificate owner's certificate issuance date, certificate issuance status, and certificate password.

[0014] A method for providing a deep learning service according to one embodiment includes the steps of: receiving a request for deep learning execution from a data owner or an AI service provider performing deep learning; determining whether the data owner possesses authority in response to the request for deep learning execution; transmitting whether the data owner possesses authority for the deep learning execution and hyperparameters to the data owner and the AI ​​service provider; receiving a result of deep learning execution from the AI ​​service provider; and providing a deep learning service by requesting the data owner and the AI ​​service provider to perform deep learning based on whether the data owner possesses authority.

[0015] The neural network that is the subject of the deep learning performance includes a separation front network and a separation back network, and the step of providing the deep learning service may include the step of transmitting parameters related to the separation back network to the data owner based on whether the authority is held, and the step of receiving the intermediate value and the correct answer value of the separation back network from the data owner and transmitting them to the AI ​​service provider.

[0016] The step of providing the deep learning service may further include the step of receiving a gradient based on the output of a separation overall network learned using the intermediate value and the correct answer value from the AI ​​service provider and transmitting it to the data owner.

[0017] The step of providing the deep learning service may further include the step of changing the deep learning execution state based on parameters related to the late-separation network updated based on the gradient, the deep learning execution epoch, the hash value of the early-separation network, and the hash value of the late-separation network.

[0018] The step of providing the deep learning service may include the step of obtaining a timeout point at which the deep learning service was interrupted in response to the interruption of the deep learning service, and the step of transmitting a request to re-execute the deep learning service to the data owner or the AI ​​service provider based on the timeout point, the number of epochs executed up to the timeout point, and the batch number executed up to the timeout point.

[0019] The step of determining whether the above authority is possessed may include a step of determining whether the above authority is possessed based on a certificate including the certificate owner, the certificate issuance date, the certificate issuance status, and the certificate password.

[0020] According to the deep learning service provision device and method of the present invention, deep learning is performed while the data owner and the AI ​​service provider are physically separated, and furthermore, a deep learning service with guaranteed reliability and confidentiality can be provided by using NFTs and private blockchains.

[0021] FIG. 1 shows a schematic block diagram of a deep learning service providing device according to one embodiment.

[0022] Figure 2 shows a service provision process using the deep learning service provision device illustrated in Figure 1.

[0023] Figure 3 shows an example of a sequence diagram of a deep learning process.

[0024] Figure 4 shows an example of a sequence diagram for resuming deep learning when a problem occurs.

[0025] Figure 5 shows an example of a sequence diagram of a monitoring process.

[0026] Figure 6 shows an example of an authorization management contract.

[0027] Figure 7 shows an example of a deep learning contract.

[0028] Figure 8 shows an example of a data NFT management contract.

[0029] Figure 9 shows an example of a model NFT management contract.

[0030] Figure 10 shows an example of a service process management contract.

[0031] Figure 11 is a diagram illustrating the authorization authentication operation.

[0032] Figure 12 is a diagram illustrating a data confidentiality guarantee service.

[0033] Figure 13 shows a flowchart of the operation of the deep learning service provider device illustrated in Figure 1.

[0034] A deep learning service providing device according to one embodiment includes a processor; The deep learning service provider includes a memory for storing instructions, wherein the instructions, when executed by the processor, cause the deep learning service provider to receive a deep learning service request from a data owner who owns data for deep learning or an AI (Artificial Intelligence) service provider who performs deep learning of a separated neural network together with the data owner, and in response to the deep learning service request, determine whether the data owner and the AI ​​service provider possess authority, and perform deep learning of the separated neural network together with the data owner and the AI ​​service provider based on a blockchain, wherein in performing the deep learning, the data owner performs deep learning of the separated second network constituting the separated neural network, and the AI ​​service provider performs deep learning of the separated first network constituting the separated neural network, and the deep learning service provider transmits parameters associated with the separated second network or the separated first network between the data owner and the AI ​​service provider based on the blockchain.

[0035] When the above instruction is executed by the processor, the deep learning service provider device may transmit parameters related to the separated latter network to the data owner based on whether the authority is held, and receive the median value and correct answer value of the separated latter network from the data owner and transmit them to the AI ​​service provider.

[0036] When the above instruction is executed by the processor, the deep learning service provider device may receive a gradient based on the output of a separated overall network learned using the intermediate value and the correct answer value from the AI ​​service provider and transmit it to the data owner.

[0037] The above instruction, when executed by the processor, can cause the deep learning service provider to change the deep learning execution state based on parameters related to the late-separation network updated based on the gradient, the deep learning execution epoch, the hash value of the early-separation network, and the hash value of the late-separation network.

[0038] When the above instruction is executed by the processor, the deep learning service providing device may obtain a timeout point in time when the deep learning service was interrupted in response to the interruption of the deep learning service, and transmit a request to the data owner or the AI ​​service provider to re-execute the deep learning service based on the timeout point, the number of epochs executed up to the timeout point, and the batch number executed up to the timeout point.

[0039] The above instruction, when executed by the processor, can cause the deep learning service provider to determine whether the authority is held based on a certificate including the certificate owner, the certificate issuance date, the certificate issuance status, and the certificate password.

[0040] Specific structural or functional descriptions of the embodiments are disclosed for illustrative purposes only and may be modified and implemented in various forms. Accordingly, actual implementations are not limited to the specific embodiments disclosed, and the scope of this specification includes modifications, equivalents, or substitutions included in the technical concept described by the embodiments.

[0041] Terms such as "first" or "second" may be used to describe various components, but these terms should be interpreted solely for the purpose of distinguishing one component from another. For example, the first component may be named the second component, and similarly, the second component may be named the first component.

[0042] When it is stated that a component is "connected" to another component, it should be understood that it may be directly connected to or joined to that other component, or that there may be other components in between.

[0043] Singular expressions include plural expressions unless the context clearly indicates otherwise. In this document, phrases such as “A or B,” “at least one of A and B,” “at least one of A or B,” “A, B or C,” “at least one of A, B and C,” and “at least one of A, B, or C” may each include any one of the items listed together with the corresponding phrase, or all possible combinations thereof. In this specification, terms such as “comprising” or “having” are intended to designate the existence of the described feature, number, step, action, component, part, or combination thereof, and should be understood as not precluding the existence or addition of one or more other features, numbers, steps, actions, components, parts, or combinations thereof.

[0044] Unless otherwise defined, all terms used herein, including technical or scientific terms, have the same meaning as generally understood by those skilled in the art. Terms such as those defined in commonly used dictionaries should be interpreted as having a meaning consistent with their meaning in the context of the relevant technology, and should not be interpreted in an ideal or overly formal sense unless explicitly defined in this specification.

[0045] As used herein, the term "module" may include a unit implemented in hardware, software, or firmware, and may be used interchangeably with terms such as logic, logic block, component, or circuit. A module may be a component formed integrally, or a minimum unit of said component or a part thereof that performs one or more functions. For example, according to one embodiment, a module may be implemented in the form of an application-specific integrated circuit (ASIC).

[0046] As used in this document, the term "part" refers to a software or hardware component, such as an FPGA or ASIC, that performs certain roles. However, "part" is not limited to software or hardware. "Part" may be configured to reside in an addressable storage medium or configured to operate one or more processors. For example, "part" may include components such as software components, object-oriented software components, class components, and task components, as well as processes, functions, attributes, procedures, subroutines, segments of program code, drivers, firmware, microcode, circuits, data, databases, data structures, tables, arrays, and variables. The functions provided within the components and "parts" may be combined into a smaller number of components and "parts" or further separated into additional components and "parts." Furthermore, components and "parts" may be implemented to operate one or more CPUs within a device or secure multimedia card. Additionally, '~part' may include one or more processors.

[0047] Hereinafter, embodiments will be described in detail with reference to the attached drawings. In the description with reference to the attached drawings, identical components are given the same reference numeral regardless of the drawing number, and redundant descriptions thereof will be omitted.

[0048]

[0049] FIG. 1 shows a schematic block diagram of a deep learning service providing device according to one embodiment.

[0050] Referring to FIG. 1, a deep learning service providing device (10) can provide deep learning services. Deep learning can be defined as a set of machine learning algorithms that attempt high levels of abstraction through a combination of various non-linear transformation techniques. The deep learning service providing device (10) can provide deep learning services to the data owner by receiving data from the data owner and having deep learning performed through an AI (Artificial Intelligence) service provider.

[0051] The deep learning service providing device (10) can provide deep learning services without exposing the data held by the data owner to the outside. The deep learning service providing device (10) can reliably secure ownership of the neural network model (or neural network) purchased by the AI ​​service launcher and provide detailed information about the neural network model. The deep learning service providing device (10) can transmit information about the data used in the AI ​​service to the marketplace and information about the neural network model applied to the AI ​​service.

[0052] The deep learning service providing device (10) can enable an AI service provider to perform deep learning on a neural network using data owned by the data owner as input, and can transmit the results of the deep learning to the data provider. At this time, the data owner and the AI ​​service provider may exist physically separated from each other as different entities.

[0053] A neural network can refer to a model in which artificial neurons (nodes) forming a network through synaptic connections change the strength of these connections through learning to possess problem-solving capabilities.

[0054] Neurons in a neural network may include a combination of weights or biases. A neural network may include one or more neurons or one or more layers composed of nodes. A neural network can infer a result to be predicted from an arbitrary input by changing the weights of the neurons through learning.

[0055] Neural networks can include deep neural networks. Neural networks include CNN (Convolutional Neural Network), RNN (Recurrent Neural Network), perceptron, multilayer perceptron, FF (Feed Forward), RBF (Radial Basis Network), DFF (Deep Feed Forward), LSTM (Long Short Term Memory), GRU (Gated Recurrent Unit), AE (Auto Encoder), VAE (Variational Auto) Encoder), DAE (Denoising Auto Encoder), SAE (Sparse Auto Encoder), MC (Markov Chain), HN (Hopfield Network), BM (Boltzmann Machine), RBM (Restricted Boltzmann Machine), DBN (Depp Belief Network), DCN (Deep Convolutional Network), DN (Deconvolutional Network), DCIGN (Deep Convolutional Inverse Graphics Network), GAN (Generative Adversarial Network), LSM (Liquid State Machine), ELM (Extreme Learning Machine), ESN (Echo It may include State Network, DRN (Deep Residual Network), DNC (Differentiable Neural Computer), NTM (Neural Turning Machine), CN (Capsule Network), KN (Kohonen Network), AN (Attention Network), SplitNN (Split Neural Network), and NoPeekNN (NoPeek Neural Network).

[0056] The deep learning service providing device (10) can be implemented in a PC (personal computer), a data server, or a portable device.

[0057] Portable devices can be implemented as laptop computers, mobile phones, smartphones, tablet PCs, mobile internet devices (MID), personal digital assistants (PDA), enterprise digital assistants (EDA), digital still cameras, digital video cameras, portable multimedia players (PMP), personal navigation devices (PND), handheld game consoles, e-books, or smart devices. Smart devices can be implemented as smart watches, smart bands, or smart rings.

[0058] The deep learning service providing device (10) includes a transceiver (100) and a processor (200). The language learning support device (10) may further include memory (300).

[0059] The transceiver (100) can receive a request for deep learning execution from a data owner or an AI (Artificial Intelligence) service provider performing deep learning, and the results of deep learning execution from the AI ​​service provider. The transceiver (100) can transmit whether it has authority for deep learning execution and hyperparameters to the data owner and the AI ​​service provider performing deep learning.

[0060] The transceiver (100) may include a transmission and reception interface. The transceiver (100) may output a received deep learning execution request and a deep learning execution result to a processor (200).

[0061] The processor (200) can process data stored in memory (300). The processor (200) can execute computer-readable code (e.g., software) stored in memory (300) and instructions triggered by the processor (200).

[0062] The "processor (200)" may be a data processing device implemented in hardware having a circuit having a physical structure for executing desired operations. For example, the desired operations may include code or instructions included in a program.

[0063] For example, a data processing device implemented in hardware may include a microprocessor, a central processing unit, a processor core, a multi-core processor, a multiprocessor, an Application-Specific Integrated Circuit (ASIC), and a Field Programmable Gate Array (FPGA).

[0064] The processor (200) can perform deep learning as an AI service provider in response to a deep learning execution request from the data owner. The neural network that is the subject of the deep learning execution may include a separation-first network and a separation-second network.

[0065] The processor (200) can determine whether the data owner has authority in response to a deep learning execution request. The processor (200) can determine whether the authority is based on a certificate including the certificate owner, the certificate issuance date, the certificate issuance status, a list of launched services, the certificate issuance status and / or the certificate password.

[0066] The processor (200) can provide deep learning services by requesting deep learning to be performed by the data owner and the AI ​​service provider based on whether they have authority. The processor (200) can transmit parameters related to the neural network to the data owner based on whether they have authority. The processor (200) can receive the intermediate value and the correct answer value of the separated network from the data owner and transmit them to the AI ​​service provider.

[0067] The processor (200) can receive a gradient based on the output of a learned network using intermediate values ​​and correct answers from an AI service provider and transmit it to the data owner.

[0068] The processor (200) can change the deep learning execution state based on parameters related to the late-separation network updated based on the gradient, the deep learning execution epoch, the hash value of the network before the separation, and the hash value of the late-separation network.

[0069] The processor (200) can obtain the timeout point at which the deep learning service was interrupted in response to the interruption of the deep learning service. The processor (200) can send a request to re-execute the deep learning service to the data owner or AI service provider based on the timeout point, the number of epochs executed up to the timeout point, and the number of batches executed up to the timeout point.

[0070] The memory (300) can store data for an operation or the result of an operation. The memory (300) can store instructions (or programs) executable by the processor (200). For example, the instructions may include instructions for executing the operation of the processor and / or the operation of each component of the processor.

[0071] The memory (300) can be implemented as a volatile memory device or a non-volatile memory device.

[0072] Volatile memory devices can be implemented as DRAM (dynamic random access memory), SRAM (static random access memory), T-RAM (thyristor RAM), Z-RAM (zero capacitor RAM), or TTRAM (Twin Transistor RAM).

[0073] Non-volatile memory devices can be implemented as EEPROM (Electrically Erasable Programmable Read-Only Memory), flash memory, MRAM (Magnetic RAM), Spin-Transfer Torque (STT)-MRAM, Conductive Bridging RAM (CBRAM), FeRAM (Ferroelectric RAM), PRAM (Phase change RAM), Resistive RAM (RRAM), Nanotube RRAM, Polymer RAM (PoRAM), Nano Floating Gate Memory (NFGM), holographic memory, Molecular Electronic Memory Device, or Insulator Resistance Change Memory.

[0074]

[0075] Figure 2 shows a service provision process using the deep learning service provision device illustrated in Figure 1.

[0076] Referring to FIG. 2, a processor (e.g., the processor (200) of FIG. 1) can provide a deep learning service with guaranteed confidentiality. The processor (200) can issue a Non-Fungible Token (NFT) for data information and information on the final neural network model to ensure reliable transactions. The processor (200) can perform monitoring to transparently provide the deep learning execution process and the deep learning service provision process.

[0077] An NFT can refer to a cryptocurrency where it is impossible to replace one token with another. An NFT is a unit of data stored on a blockchain, representing a unique and non-exchangeable token. NFTs can be used to represent photos, videos, audio, and other types of digital files. Because they serve as a virtual certificate of authenticity, they are non-fungible and copies are not recognized; while anyone can obtain a copy of a digital item, NFTs are tracked on the blockchain, which can serve as proof of copyright and ownership to the owner.

[0078] The processor (200) can enable the training of neural networks separated from the data provider and the AI ​​service provider using a splittable neural network. The processor (200) can enable deep learning to be performed with the data owner and the AI ​​service provider physically separated.

[0079] The processor (200) can issue NFTs. The processor (200) can issue and manage NFTs based on information regarding data and neural network models. The processor (200) can determine ownership and reliable authenticity through the NFTs.

[0080] The processor (200) can monitor the entire deep learning process, including the data used in deep learning and the neural network generated as a result of performing deep learning. The processor (200) can monitor the entire deep learning service process, including the data used in the deep learning service, the neural network model, the service launch date, and the launcher information.

[0081] The processor (200) can provide deep learning services using a private blockchain (210). The processor (200) can perform deep learning services and management of services that ensure confidentiality based on the private blockchain.

[0082] The processor (200) can perform deep learning using a separated neural network (230). The processor (200) can issue and sell NFTs (250). The processor (200) can monitor the overall process of performing deep learning services (270).

[0083] The processor (200) can perform deep learning without external exposure of the data owner's data. The processor (200) can issue an NFT for the processed data or received data.

[0084] The processor (200) can enable an AI service provider to perform deep learning using various data. The processor (200) can issue an NFT for the final neural network model and deliver it to the AI ​​service provider.

[0085] The processor (200) can ensure that the AI ​​service launcher secures ownership of the neural network model purchased. The processor (200) can provide monitoring of the data and neural network models being used to the marketplace for data and neural network models.

[0086]

[0087] Figure 3 shows an example of a sequence diagram of a deep learning process.

[0088] Referring to FIG. 3, a processor (e.g., processor (200) of FIG. 1) may provide deep learning services in response to a deep learning execution request from a data owner (310). The processor (200) may provide deep learning services using an AI service provider (330) in response to a request from the data owner (310). The processor (200) may provide deep learning services using a deep learning contract (350) and an NFT contract (370). The deep learning contract (350) and the NFT contract (370) may be included in the processor (200).

[0089] The deep learning contract (350) can provide deep learning services using a divisible neural network. For example, the deep learning contract (350) can provide deep learning services based on NoPeekNN.

[0090] The deep learning contract (350) can receive from the data owner a certificate to verify whether it has access to the data's NFT unique value and blockchain to execute deep learning and whether it has the authority.

[0091] The deep learning contract (350) can use a certificate to verify whether a user (e.g., an AI service launcher, a data owner (310), or an AI service provider (330)) has authority. The deep learning contract (350) can provide the user with the result of the verification regarding whether authority is held.

[0092] The deep learning contract (350) can notify the data owner (310) and the AI ​​service provider (330) of the start of deep learning. The deep learning contract (350) can receive a request for authorization verification from the AI ​​service provider (330). The deep learning contract (350) can receive deep learning configuration values ​​from the AI ​​service provider (330). The deep learning contract (350) can receive from the AI ​​service provider (330) a decoupled network that is part of the neural network, hyperparameters of the neural network, and a certificate for authorization verification.

[0093] The deep learning contract (350) can check whether the user has authority using the attribute value stored in the certificate. The deep learning contract (350) can transmit whether the user has authority to the user. The deep learning contract (350) can transmit whether the user has authority to the AI ​​service provider (330).

[0094] The deep learning contract (350) can change the deep learning execution state to start after verifying whether it holds authority. The deep learning contract (350) can perform deep learning using a separable neural network and a blockchain. The deep learning contract (350) can perform learning by repeating for a predetermined number of epochs.

[0095] The deep learning contract (350) can transmit the decoupled network and hyperparameters, which are part of the neural network, to the data owner (310) based on the blockchain. The data owner can perform learning using locally stored input data based on the received decoupled network. The data owner (310) can transmit the intermediate value, which is the output result of the decoupled network, and the target data, which is the correct answer value corresponding to the input data, to the deep learning contract (350).

[0096] The deep learning contract (350) can provide median values ​​and target data to the AI ​​service provider (330) via the blockchain. The AI ​​service provider (330) can train the separation overall network using the median values ​​and target data, calculate the error, and extract the gradient. The AI ​​service provider (330) can transmit the gradient to the deep learning contract (350).

[0097] The deep learning contract (350) can transmit the gradient to the data owner (310) via the blockchain. The data owner (310) can update the network after the separation based on the gradient. The data owner (310) can transmit the updated network after the separation to the deep learning contract (350).

[0098] The deep learning contract (350) can receive a deep learning epoch, a hash value of the network before the separation, and a hash value of the network after the separation from the AI ​​service provider (330). The deep learning contract (350) can change the deep learning execution state to a terminated state based on parameters related to the network after the separation, the deep learning execution epoch, the hash value of the network before the separation, and the hash value of the network after the separation.

[0099] The NFT contract (370) can issue an NFT for the final neural network model in response to changing the deep learning execution state to terminate.

[0100]

[0101] Figure 4 shows an example of a sequence diagram for resuming deep learning when a problem occurs.

[0102] Referring to FIG. 4, a processor (e.g., processor (200) of FIG. 1) can reliably resume the process without interrupting the deep learning service if a problem occurs during the deep learning service provision process. The processor (200) may include a deep learning contract (450).

[0103] The example in FIG. 4 illustrates a case where an unexpected problem occurs at the AI ​​service provider (430) during deep learning, resulting in the failure to deliver the gradient. If the data owner (410) fails to receive the gradient, it may trigger a timeout and send a timeout notification to the deep learning contract (450).

[0104] The deep learning contract (450) can store the timeout point, epoch number, and batch number, and change the deep learning execution state to a 'suspended' state.

[0105] The deep learning contract (450) may receive a request to re-run deep learning after the problem of the AI ​​service provider (430) has been resolved. The deep learning contract (450) may respond to requests from the data owner (410) and the AI ​​service provider (430) by providing the timeout point, the number of epochs to be re-runned, and the number of batches.

[0106] The data owner (410) may pass a response to the re-execution to the deep learning contract (450). The deep learning contract (450) may change the deep learning execution state to a 're-execution' state based on the response to the re-execution. In response to the state change, the AI ​​service provider (430) may recognize that it is its turn to perform an action and pass the gradient at that point in time to the deep learning contract (450) to resume deep learning.

[0107]

[0108] Figure 5 shows an example of a sequence diagram of a monitoring process.

[0109] Referring to FIG. 5, a processor (e.g., the processor (200) of FIG. 1) can monitor the process of providing a deep learning service. Through the monitoring service, the processor (200) can transparently and reliably provide various users with the process of deep learning being performed and the process of the service being provided.

[0110] The processor (200) may include a monitoring service module (530), a data confidentiality guarantee service module (540), a service process management contract (550), an NFT management contract (560), and a deep learning contract (570).

[0111] When the service launcher (520) purchases the necessary data and neural network model NFTs through the marketplace and is ready to launch the service to be provided using them, it may request service registration with the data confidentiality guarantee service module (540).

[0112] The data confidentiality guarantee service module (540) may request an inquiry into information regarding the NFT received from the NFT management contract (560). The NFT management contract (560) may determine whether the data is valid and whether it is a valid neural network model NFT. The NFT management contract (560) may transmit the determination result to the data confidentiality guarantee service module (540).

[0113] The data confidentiality guarantee service module (540) can search for metadata and information regarding the NFT based on the result of the judgment. Based on the result of the judgment of whether the NFT is valid, the data confidentiality guarantee service module (540) transmits information related to the requested service to the service process management contract (550) if the NFT is valid, and the service process management contract (550) can store the transmitted information. The service process management contract (550) can transmit whether to store the information to the data confidentiality guarantee service module (540).

[0114] The data confidentiality guarantee service module (540) can transmit the results of service registration to be launched to the service launcher (520). The monitoring service module (530) can provide monitoring results by querying whether there is a newly registered service through the service process management contract (550).

[0115] The service launcher (520) can launch a service and send a request to change the service launch status to the data confidentiality guarantee service module (540). The data confidentiality guarantee service module (540) can respond to the request and change the service launch status to 'in progress'.

[0116] The service launcher (520) can receive monitoring results from the monitoring service module (530) while providing the service. The monitoring service module (530) can provide transparent and reliable monitoring functions by providing information on data issued as NFTs, neural network models, and registered services through the service process management contract (550), NFT management contract (560), and deep learning contract (570).

[0117] The monitoring service module (530) can provide the data owner (510) with information about the process of using the data for deep learning and the type of service used. The monitoring service module (530) can provide the AI ​​service provider with information about the process of using the generated neural network model.

[0118]

[0119] Hereinafter, with reference to FIGS. 6 to 10, the authorization management contract, deep learning contract, data NFT management contract, model NFT management contract, and service process management contract will be described.

[0120]

[0121] Figure 6 shows an example of an authorization management contract.

[0122] Referring to FIG. 6, a processor (e.g., processor (200) of FIG. 1) may include an authorization management contract. The authorization management contract may include a key (610) containing an authentication list and a value (630) containing a full list of user unique IDs (Identification) and a list of user information structures. The list of user information structures may include a list of high-trust passwords for certificate issuance, certificate issuance dates, certificate issuance status, a list of launched services, a list of unique values ​​for owned data NFTs, and a list of owned model (or neural network model) NFTs.

[0123] A user of a deep learning service provider (e.g., the deep learning service provider (10) of FIG. 1) can obtain a certificate to access a private blockchain.

[0124] An authorization management contract can store information regarding certificate issuance to manage permissions. The authorization management contract can store passwords for users to obtain or reissue certificates. Passwords can be stored encrypted with high-trust security.

[0125]

[0126] Figure 7 shows an example of a deep learning contract.

[0127] Referring to FIG. 7, a processor (e.g., processor (200) of FIG. 1) may include a deep learning contract.

[0128] A deep learning contract may include a key (710) containing a data NFT unique value and a neural network model name, and a value (730) containing information related to the deep learning process.

[0129] A value (730) containing information related to the deep learning execution process may include a user unique ID, deep learning execution status, description, a list of hash values ​​of the entire network model of the split separation, hash values ​​of the latter network model of the split separation, a list of training hyperparameter structures, and a list of training parameter structures.

[0130] A deep learning contract can store the entire process of deep learning while ensuring data confidentiality. A deep learning contract can enable deep learning to be performed by providing data owners and AI service providers with stored parameters.

[0131] A deep learning contract can mediate between a data owner and an AI service provider to resume the process if the deep learning process is interrupted due to a problem occurring during the deep learning execution.

[0132]

[0133] Figure 8 shows an example of a data NFT management contract.

[0134] Referring to FIG. 8, the processor (e.g., the processor (200) of FIG. 1) may include a data NFT management contract.

[0135] The data NFT management contract stores information regarding the issuance of data NFTs and can store a list of all issued data NFTs. The data NFT management contract can store NFT metadata as a list of structures.

[0136] A data NFT management contract may include a data NFT (810) and a value (830) containing NFT-related information. The value (830) containing NFT-related information may include a list of NFT unique values, a list of NFT metadata structures, and a list of NFT issuance dates. The list of NFT metadata structures may include a data set name, a data set hash value, a description, a count, a type, and an owner.

[0137]

[0138] Figure 9 shows an example of a model NFT management contract.

[0139] Referring to FIG. 9, the processor (e.g., the processor (200) of FIG. 1) may include a model NFT management contract.

[0140] The ModelNFT Management Contract can store information regarding the issuance of NFTs for deep learning completed models. The ModelNFT Management Contract stores a list of all issued Model NFTs and stores the metadata of each NFT as a list of structures.

[0141] A model NFT management contract may include a key (910) containing a model NFT and a value (930) containing information related to the model NFT. The value (930) containing information related to the model NFT may include a list of NFT unique values, a list of NFT metadata structures, and a list of NFT issuance dates. The list of NFT metadata structures may include a deep learning history key, a used data NFT unique value, a parameter structure, a hyperparameter structure, an owner, and a model hash value.

[0142]

[0143] Figure 10 shows an example of a service process management contract.

[0144] Referring to FIG. 10, a processor (e.g., processor (200) of FIG. 1) may include a service process management contract.

[0145] The Service Process Management Contract can manage services conducted by purchasing NFTs of issued data and models. The Service Process Management Contract stores a list of all services currently in operation and stores information about the services in the form of a structure.

[0146] A service process management contract may include a key (1010) containing a list of services and a value (1030) containing service-related information. The value (1030) containing service-related information may include a list of all services currently in service and a list of service structures. The list of service structures may include a company ID, service IP (Intellectual Property), service ID, service name, service status, service launch date, unique usage data NFT value, and unique usage model NFT value.

[0147]

[0148] Figure 11 is a diagram illustrating the authorization authentication operation.

[0149] Referring to FIG. 11, a processor (e.g., the processor (200) of FIG. 1) can determine whether a user (e.g., a data owner, an AI service provider, and / or a service launcher) (1110) holds authority. The processor (200) can manage authority through an authority management contract (1170) and a data confidentiality guarantee service module (1130).

[0150] To access a private blockchain, a user may obtain a certificate to prove that they are an authorized user. The user (1110) may request the issuance of a network configuration file to communicate with the CA server (1150) and transmit a unique user ID to the data confidentiality guarantee service module (1130).

[0151] The data confidentiality guarantee service module (1130) can issue a network configuration file to the user (1110). The data confidentiality guarantee service module (1130) transmits the history of issuing the network configuration file to the authorization management contract (117), and the authorization management contract (1170) can store it along with the user's unique ID.

[0152] A user (1110) may request the registration of a unique user ID to the data confidentiality guarantee service module (1130) in order to obtain a certificate. The data confidentiality guarantee service module (1130) may generate a password that satisfies a predetermined security level and request the CA server (1150) to register the requested unique user ID and password. The CA server (1150) may send a response regarding whether the unique user ID has been registered to the data confidentiality guarantee service module (1130).

[0153] The data confidentiality guarantee service module (1130) can transmit the generated password to the user (1110). The user (1110) can send a request for certificate issuance to the CA server (1150) using the user's unique ID and password. The CA server (1150) can issue a certificate that allows access to the private blockchain in response to the request for certificate issuance.

[0154]

[0155] Figure 12 is a diagram illustrating a data confidentiality guarantee service.

[0156] Referring to FIG. 12, a processor (e.g., the processor (200) of FIG. 1) may issue data owned and the hash value of the final neural network model on which deep learning has been performed as an NFT to ensure reliable and transparent transactions.

[0157] The processor (200) may include a data confidentiality guarantee service module (1210), an NFT contract (1230), and an NFT management contract (1250). The NFT management contract (1250) may include a data NFT management contract and a model NFT management contract.

[0158] Users (e.g., data owners, AI service providers and / or launchers) (1220) can receive NFT-related services through the data confidentiality guarantee service module (1210).

[0159] The NFT contract (1230) can issue an NFT for data sets to be used for deep learning to the data owner. The data confidentiality guarantee service module (1210) can provide the source and usage history of the data by issuing the data owner, through the NFT contract (1230), an NFT containing basic information such as the name and description of the data set and the hash value of the data set.

[0160] The data confidentiality guarantee service module (1210) can enable the AI ​​service provider to verify the data NFT of the necessary data set in the marketplace and to perform deep learning using the NFT.

[0161] The data confidentiality guarantee service module (1210) can issue the hash value of the final neural network model and parameter information as an NFT after performing learning by changing the neural network model and hyperparameters used in deep learning in various ways.

[0162] The data confidentiality guarantee service module (1210) can sell NFTs to the service launcher (1260) through the data and model marketplace (1240). The service launcher (1260) can launch a new service by purchasing NFTs through the data and model marketplace (1240), and the data confidentiality guarantee service module (1210) can manage the deep learning service process based on the blockchain.

[0163] The data confidentiality guarantee service module (1210) can receive a request from the data owner for the issuance of an NFT for information including the hash value of the data set to be deep learned. The AI ​​service provider can issue an NFT for the hash value of the final model after deep learning is completed.

[0164] The data confidentiality guarantee service module (1210) can issue an NFT through an NFT contract (1230) and transmit the unique NFT value to the user (1220). The NFT management contract (1230) can store the NFT issuance history.

[0165] The data and model marketplace (1240) can sell data and model NFTs through the data confidentiality guarantee service module (1210).

[0166] The data confidentiality guarantee service module (1210) can provide NFT management services through the NFT management contract (1250). The service launcher (1260) launches the service by purchasing data and model NFTs through the data and model marketplace (1240), and the process of the launched service can be managed based on the blockchain by the data confidentiality guarantee service module (1210).

[0167]

[0168] Figure 13 shows a flowchart of the operation of the deep learning service provider device illustrated in Figure 1.

[0169] Referring to FIG. 13, a transceiver (e.g., transceiver (100) of FIG. 1) may receive a request to execute deep learning from a data owner or an AI service provider performing deep learning (1310).

[0170] A processor (e.g., the processor (200) of FIG. 1) can determine whether the data owner has authority in response to a deep learning execution request (1330). The processor (200) can determine whether the authority has authority based on a certificate including the certificate owner, the certificate issuance date, the certificate issuance status, and the certificate password.

[0171] The processor (200) can perform deep learning as an AI service provider in response to a deep learning execution request from the data owner. The neural network that is the subject of the deep learning execution may include a separation-first network and a separation-second network.

[0172] The transceiver (100) can transmit whether it has the authority to execute deep learning and hyperparameters to the data owner and the AI ​​service provider performing deep learning (1350).

[0173] The transceiver (100) can receive deep learning results from an AI service provider (1370).

[0174] The processor (200) can provide deep learning services by requesting deep learning to the data owner and the AI ​​service provider based on whether they have the authority (1390).

[0175] The processor (200) can provide deep learning services by requesting deep learning to the data owner and the AI ​​service provider based on whether they have authority. The processor (200) can transmit parameters related to the separated network to the data owner based on whether they have authority. The processor (200) can receive the median value and the correct answer value of the separated network from the data owner and transmit them to the AI ​​service provider.

[0176] The processor (200) can receive a gradient based on the output of a learned network using intermediate values ​​and correct answers from an AI service provider and transmit it to the data owner.

[0177] The processor (200) can change the deep learning execution state based on parameters related to the late-separation network updated based on the gradient, the deep learning execution epoch, the hash value of the network before the separation, and the hash value of the late-separation network.

[0178] The processor (200) can obtain the timeout point at which the deep learning service was interrupted in response to the interruption of the deep learning service. The processor (200) can send a request to re-execute the deep learning service to the data owner or AI service provider based on the timeout point, the number of epochs executed up to the timeout point, and the number of batches executed up to the timeout point.

[0179]

[0180] The embodiments described above may be implemented as hardware components, software components, and / or combinations of hardware and software components. For example, the devices, methods, and components described in the embodiments may be implemented using a general-purpose computer or a special-purpose computer, such as, for example, a processor, a controller, an arithmetic logic unit (ALU), a digital signal processor, a microcomputer, a field programmable gate array (FPGA), a programmable logic unit (PLU), a microprocessor, or any other device capable of executing and responding to instructions. The processing unit may execute an operating system (OS) and software applications executed on said operating system. Additionally, the processing unit may access, store, manipulate, process, and generate data in response to the execution of the software. For ease of understanding, the processing unit may be described as being used as a single unit, but those skilled in the art will understand that the processing unit may include multiple processing elements and / or multiple types of processing elements. For example, the processing unit may include multiple processors or one processor and one controller. In addition, other processing configurations, such as parallel processors, are also possible.

[0181] Software may include computer programs, code, instructions, or a combination of one or more of these, and may configure a processing unit to operate as desired or command the processing unit independently or collectively. Software and / or data may be permanently or temporarily embodied in any type of machine, component, physical device, virtual equipment, computer storage medium or device, or transmitted signal wave so as to be interpreted by the processing unit or to provide instructions or data to the processing unit. Software may be distributed over networked computer systems and may be stored or executed in a distributed manner. Software and data may be stored on computer-readable recording media.

[0182] The method according to the embodiment may be implemented in the form of program instructions that can be executed through various computer means and recorded on a computer-readable medium. The computer-readable medium may store program instructions, data files, data structures, etc., either individually or in combination, and the program instructions recorded on the medium may be those specifically designed and configured for the embodiment or those known and available to those skilled in the art of computer software. Examples of computer-readable recording media include magnetic media such as hard disks, floppy disks, and magnetic tapes; optical recording media such as CD-ROMs and DVDs; magneto-optical media such as floptical disks; and hardware devices specifically configured to store and execute program instructions, such as ROM, RAM, and flash memory. Examples of program instructions include machine code, such as that generated by a compiler, as well as high-level language code that can be executed by a computer using an interpreter, etc.

[0183] The hardware device described above may be configured to operate as one or more software modules to perform the operation of the embodiment, and vice versa.

[0184] Although the embodiments have been described above with reference to the limited drawings, those skilled in the art can apply various technical modifications and variations based thereon. For example, suitable results may be achieved even if the described techniques are performed in a different order than described, and / or if the components of the described system, structure, device, circuit, etc. are combined or assembled in a form different from described, or replaced or substituted by other components or equivalents.

[0185] Therefore, other implementations, other embodiments, and equivalents to the claims also fall within the scope of the claims set forth below.

Claims

1. In a deep learning service providing device, processor; and Memory that stores instructions Includes, The above instruction, when executed by the processor, causes the deep learning service providing device, Receiving a deep learning service request from a data owner who owns data for deep learning or from an AI (Artificial Intelligence) service provider that performs deep learning of a detached neural network together with the data owner, and In response to the above deep learning service request, determine whether the data owner and the AI ​​service provider possess authority, and Based on blockchain, deep learning of the separable neural network is performed together with the data owner and the AI ​​service provider, and In performing the above deep learning, The above data owner performs deep learning of the separated latter network constituting the above separated neural network, and The above AI service provider performs deep learning of the separated overall network constituting the above separated neural network, and The deep learning service providing device described above transmits parameters associated with the separated latter network or the separated earlier network between the data owner and the AI ​​service provider based on the blockchain. Deep learning service providing device.

2. In Paragraph 1, The above instruction, when executed by the processor, causes the deep learning service providing device, Based on whether the above authority is held, parameters related to the above-detached latter network are transmitted to the above-detached data owner, and Receiving the median value and the correct answer value of the separated latter network from the data owner and transmitting them to the AI ​​service provider, Deep learning service providing device.

3. In Paragraph 2, The above instruction, when executed by the processor, causes the deep learning service providing device, Receiving a gradient based on the output of a separation overall network learned using the median value and the correct answer value from the AI ​​service provider and transmitting it to the data owner. Deep learning service providing device.

4. In Paragraph 3, The above instruction, when executed by the processor, causes the deep learning service providing device, Parameters related to the late-separation network updated based on the above gradient, deep learning execution epoch, hash value of the early-separation network, and changing the deep learning execution state based on the hash value of the late-separation network, Deep learning service providing device.

5. In Paragraph 1, The above instruction, when executed by the processor, causes the deep learning service providing device, In response to the interruption of the deep learning service, the timeout point at which the deep learning service was interrupted is obtained, and Sending a request to re-execute the deep learning service to the data owner or the AI ​​service provider based on the above timeout point, the number of epochs executed up to the above timeout point, and the number of batches executed up to the above timeout point. Deep learning service providing device.

6. In Paragraph 1, The above instruction, when executed by the processor, causes the deep learning service providing device, Determining whether the above authority is possessed based on a certificate including the certificate owner, certificate issuance date, certificate issuance status, and certificate password, Deep learning service providing device.

7. In a method of providing a deep learning service of a deep learning service providing device, An operation in which a deep learning service providing device receives a deep learning service request from a data owner who owns data for deep learning, or from an AI service provider who performs deep learning of a detached neural network together with said data owner; A deep learning service providing device performs an operation of determining whether the data owner and the AI ​​service provider possess authority in response to the deep learning service request; and An operation in which a deep learning service providing device performs deep learning of the separated neural network together with the data owner and the AI ​​service provider based on a blockchain. Includes, In the operation of performing the deep learning described above, The above data owner performs deep learning of the separated latter network constituting the above separated neural network, and The above AI service provider performs deep learning of the separated overall network constituting the above separated neural network, and The deep learning service providing device described above transmits parameters associated with the separated latter network or the separated earlier network between the data owner and the AI ​​service provider based on the blockchain. Method for providing deep learning services.

8. In Paragraph 7, The operation of performing the above deep learning is, The operation of transmitting parameters related to the separated latter network to the data owner based on whether the above authority is held; and The operation of receiving the median value and the correct answer value of the separated latter network from the data owner and transmitting them to the AI ​​service provider. A method for providing a deep learning service including 9. In Paragraph 8, The operation of performing the above deep learning is, The operation of receiving a gradient based on the output of a separation overall network learned using the median value and the correct answer value from the AI ​​service provider and transmitting it to the data owner. A method for providing a deep learning service that further includes 10. In Paragraph 9, The operation of performing the above deep learning is Parameters related to the late-separation network updated based on the above gradient, deep learning execution epochs, the hash value of the early-separation network, and an operation to change the deep learning execution state based on the hash value of the late-separation network. A method for providing a deep learning service that further includes 11. In Paragraph 7, The operation of performing the above deep learning is, An operation to acquire a timeout point in time when the deep learning service was interrupted in response to the interruption of the deep learning service; and An operation of transmitting a request to re-execute the deep learning service to the data owner or the AI ​​service provider based on the above timeout point, the number of epochs executed up to the above timeout point, and the number of batches executed up to the above timeout point. A method for providing a deep learning service including 12. In Paragraph 7, The operation for determining whether the above authority is possessed is, An operation to determine whether the above authority is possessed based on a certificate including the certificate owner, certificate issuance date, certificate issuance status, and certificate password. A method for providing a deep learning service including

Images