Information processing apparatus, controlling method thereof, and program therefor, and information processing system

JP2024044786A5Pending Publication Date: 2025-09-18CANON KK

View PDF 0 Cites 0 Cited by

Patent Information

Application Number: JP2022150530
Authority / Receiving Office: JP · JP
Patent Type: Applications
Current Assignee / Owner
Filing Date: 2022-09-21
Publication Date: 2025-09-18

AI Technical Summary

Technical Problem

Existing methods for decoding encrypted inference network models are slow due to the time-consuming decryption process, which affects the performance of inference devices.

Method used

The information processing device separates the inference network model into first and second data, encrypts the first data, and transmits both files to the inference device, allowing for faster decoding by enabling parallel processing of the less complex second data decryption.

Benefits of technology

This approach significantly reduces the time required for decoding the inference network model, enabling quicker inference processing on the receiving device.

✦ Generated by Eureka AI based on patent content.

Smart Images

Figure 00000000_0000_ABST

Patent Text Reader

Abstract

To improve a processing speed of an apparatus for receiving an inference network model when decoding the inference network model.SOLUTION: An information processing apparatus for transmitting data indicating inference network model to an external device has a separation unit for separating the data indicating the inference network model into first data including information for use in representing at least a layer structure of the inference network model and second data indicating coefficient data of each layer, a generation unit for generating a first file including the first data and a second file including the second data, and a transmission unit for transmitting the generated first and second files to the information processing apparatus. The generation unit generates the first file using the data obtained by subjecting the first data to encoding processing.SELECTED DRAWING: Figure 3

Need to check novelty before this filing date? Find Prior Art

Description

[Technical field]

[0001] The present invention relates to a concealment technique for communication of data relating to a neural network. [Background technology]

[0002] Deep learning technology using neural networks is applied in a wide range of technical fields, and in particular, image recognition and classification is said to exceed human recognition capabilities. Among them, convolutional neural networks (CNNs) are widely used because they realize high-precision deep learning processing by recursively applying convolution operations to images.

[0003] In recent years, interest in security of data, including images, has been growing due to the popularity of SNS (Social Networking Services). For example, data communication from edge devices to cloud servers is essential for data sharing services such as SNS. Moreover, such data communication is generally performed with the data kept confidential to protect privacy.

[0004] As a countermeasure against such security issues, the trained dictionary data is made confidential in order to protect the neural network.

[0005] For example, according to Patent Document 1, it is possible to generate an encrypted training model by encrypting an inference network model using a first public key, which is a public key for fully homomorphic encryption. [Prior art documents] [Patent documents]

[0006] [Patent Document 1] JP 2019-168590 A Summary of the Invention [Problem to be solved by the invention]

[0007] However, when using the technology of Patent Document 1, when decrypting an encrypted inference network model and executing inference processing, the decryption processing may take a long time depending on the performance of the inference device.

[0008] In view of the above problems, the present invention aims to provide a technique that enables an inference device to perform decoding faster than ever before, even when using the same inference network model. [Means for solving the problem]

[0009] In order to solve this problem, for example, an information processing device of the present invention has the following arrangement. An information processing device that transmits data representing an inference network model to an external device, A separation means for separating data representing the inference network model into first data including information for representing at least a layer configuration of the inference network model and second data representing coefficient data of each layer; A generating means for generating a first file including the first data and a second file including the second data; a transmitting means for transmitting the generated first file and the generated second file to an information processing device, The generating means generates the first file from data obtained by encrypting the first data. Effect of the Invention

[0010] According to the present invention, it is possible to improve the processing speed when decoding an inference network model in a device that receives the inference network model. [Brief description of the drawings]

[0011] [Figure 1]FIG. 1 is a block diagram showing the configuration of an information processing apparatus according to an embodiment of the present invention. [Diagram 2] FIG. 1 is a block diagram of an inference device according to an embodiment of the present invention. [Diagram 3] FIG. 1 is a diagram illustrating a system configuration of an information processing and inference device according to an embodiment of the present invention. [Figure 4] 1A and 1B are diagrams illustrating the data structure of an inference network model in an embodiment, FIG. 1C is a diagram illustrating the data structure of an encrypted inference network model, and FIG. 1D is a diagram illustrating the conversion process of the data structure of the inference network model during encryption. [Diagram 5] 4 is a flowchart illustrating a processing method in the information processing device according to the embodiment. [Figure 6] 5 is a flowchart showing a format analysis process in the information processing device according to the embodiment. [Figure 7] 4 is a flowchart illustrating an inference process in the inference device according to the present embodiment. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0012] Hereinafter, the embodiments will be described in detail with reference to the attached drawings. Note that the following embodiments do not limit the invention according to the claims. Although the embodiments describe a number of features, not all of these features are essential to the invention, and the features may be combined in any manner. Furthermore, in the attached drawings, the same reference numbers are used for the same or similar configurations, and duplicated descriptions are omitted.

[0013] [First embodiment] FIG. 1 is a block diagram of an information processing device 100 according to the first embodiment.

[0014] The information processing device 100 includes a CPU 101 , a memory 102 , a display unit 103 , an internal bus 104 , an operation unit 105 , a recording unit 106 , a communication unit 107 , a neural network processing unit 108 , and an encryption processing unit 109 .

[0015] The CPU 101 executes a computer program stored in the recording unit 206 to control all processing blocks that constitute the information processing device.

[0016] The memory 102 is a memory that is mainly used as a work area for the CPU 101 and a temporary buffer area for data.

[0017] The display unit 103 performs drawing processing (drawing of a GUI screen) according to instructions from the CPU 101, and outputs to a display device such as a liquid crystal panel or an organic EL panel.

[0018] The internal bus 104 is a bus for connecting each processing block in the information processing device to each other.

[0019] The operation unit 105 is configured with a keyboard, a mouse, buttons, a touch panel, a remote control, etc., and receives operation instructions from a user. Operation information input from the operation unit 105 is transmitted to the CPU 101, and the CPU 101 executes control of each processing block based on the operation information.

[0020] The recording unit 106 is a processing block that is configured by a recording medium and stores and reads various data in the recording medium based on instructions from the CPU 101. The recording medium is configured by, for example, an EEPROM, an internal flash memory, an internal hard disk, or a removable memory card. In the embodiment, the recording unit 106 stores input data and teacher data, which are learning data for the neural network processing unit 108.

[0021] The communication unit 107 communicates with an external device and is typically a network interface. The communication may be wireless or wired, but in this embodiment, it is assumed to be a wireless LAN interface. The wireless LAN interface is, for example, an interface conforming to the IEEE802.11n / a / g / b standard. The communication unit 107 connects to an external access point via wireless LAN and performs wireless LAN communication with other wireless communication devices via the access point. The communication unit 107 can also be connected to other devices via a serial cable such as a USB. The communication unit 107 communicates with external devices including an inference device, and exchanges information such as images, control data, learning data, teacher data, models, confidentiality models, and decryption models.

[0022] The neural network processing unit 108 performs a learning process of the neural network using the learning data and teacher data stored in the recording unit 106 and a program in which the configuration of a neural network model created in advance is described. The learning data, dictionary data, and model may be received separately from the communication unit 107, or may be those stored in the recording unit 106. The neural network processing unit 108 is composed of a GPU or a DSP (Digital Signal Processor). The type of processing to be performed by the neural network processing unit 108 is not particularly limited, but it is assumed to be, for example, a class classification process of a subject (human, cat, etc.) in an image. The learning process of the neural network may be performed by the CPU 101 without providing the neural network processing unit 108, or may be realized by a joint process of both. The inference network model, which is the result of the learning process performed by the neural network processing unit 108, is also held in the recording unit 106. The neural network processing unit 108 can also perform inference using the inference network model stored in the recording unit 106 and the inference input data.

[0023] The encryption processing unit 109 expands the inference network model recorded in the recording unit 106 into the memory 102, and performs a specific encryption process on part of the data of the inference network model to generate an encrypted file. The public key and private key may be exchanged via the communication unit 216, or may be shared in advance between the information processing device 100 and the inference device 200. The data may be encrypted by either asymmetric encryption or symmetric encryption. Note that the encryption process is a known technology and will not be described here.

[0024] 2 is a block diagram of an inference device 200 according to an embodiment. In the embodiment, an example in which the inference device 200 is applied to a device having an image capturing function, such as a digital camera, will be described.

[0025] 2, the inference device 200 includes a CPU 201, a memory 202, a non-volatile memory 203, an operation unit 204, a neural network processing unit 205, a recording unit 206, an encryption / decryption unit 207, a lens 211, an imaging unit 212, an image processing unit 213, and an encoding processing unit 214. Furthermore, the inference device 200 includes a display unit 215, a communication unit 216, and an internal bus 220.

[0026] The CPU 201 executes a computer program stored in the non-volatile memory 203 to control the operation of each part (each functional block) of the inference apparatus via the internal bus 220 .

[0027] The memory 202 is a rewritable volatile memory. The memory 202 temporarily records a computer program for controlling the operation of each part of the inference device, information such as parameters related to the operation of each part of the inference device, a model received by the communication unit 216, a concealment model, and other information. The memory 202 also temporarily records an image acquired by the imaging unit 212, and images and information processed by the image processing unit 213, the encoding processing unit 214, etc. The memory 202 has a sufficient storage capacity for temporarily recording these.

[0028] The non-volatile memory 203 is an electrically erasable and recordable memory, and may be, for example, an EEPROM, a hard disk, or the like. The non-volatile memory 203 stores computer programs that control the operation of each part of the inference device, and information such as parameters related to the operation of each part of the inference device. The various operations performed by the inference device 200 are realized by such computer programs. Furthermore, the non-volatile memory 203 is used by the neural network processing unit 205, and is capable of holding multiple models and a concealment model.

[0029] The operation unit 204 provides a user interface for operating the inference device.

[0030] The neural network processing unit 205 executes neural network processing using a previously created inference network model. Note that the neural network processing is, for example, one that is configured with a CNN (Convolutional Neural Network) and a fully connected layer, but is not limited to this. The above-mentioned learned coefficient parameters correspond to the weight coefficients and bias values of each edge connecting the nodes of each layer in the fully connected layer, and the weight coefficients and bias values of the kernel in the CNN. The neural network processing unit 205 also has a temporary storage area for temporarily storing the weight coefficients and bias values. The neural network processing unit 205 also has a function of decoding compressed input data and learned coefficient parameters.

[0031] The recording medium control unit 206 controls writing / reading to / from the recording medium under the control of the CPU 201. For example, captured image data is recorded on the recording medium. As the recording medium, for example, a non-volatile memory, a magnetic disk, or the like is used.

[0032] The encryption / decryption unit 207 decrypts data encrypted by a predetermined encryption method for encrypted files in the inference network model deployed in the memory 202. Encryption may be realized by previously exchanging a secret key and a public key via the network processing unit 205.

[0033] The photographing lens (lens unit) 211 is made up of a lens group (not shown) including a zoom lens, a focus lens, etc., a lens control unit (not shown), an aperture (not shown), etc.

[0034] The imaging unit 212 uses an area image sensor such as a CCD (charge-coupled device) or a CMOS (complementary metal-oxide semiconductor) element, and converts an optical image of a subject into an electrical signal. The imaging unit 212 has the ability to capture images at a rate of, for example, 30 frames per second, and can capture moving images and still images.

[0035] The image processing unit 213 performs predetermined image processing on the image data output from the imaging unit 212 or the image data read from the memory 202 .

[0036] The encoding processing unit 214 compresses the size of the image data by performing intra-frame predictive coding (intra-screen predictive coding), inter-frame predictive coding (inter-screen predictive coding), or the like on the image data.

[0037] The display unit 215 generates a displayable image and outputs it to the display screen. The display unit 315 has an OSD (On Screen Display) function, which is a function for displaying a setting screen such as a menu on the display screen. The display unit 216 is composed of a liquid crystal display, an organic EL display, or the like.

[0038] The communication unit 216 is controlled by the CPU 201. The communication unit 216 is configured to generate a modulated signal conforming to a wireless communication standard such as IEEE802.11, output the modulated signal, and receive the modulated signal from an external device. The communication unit 216 may also control a wired LAN or a USB connected by a wired cable. The communication unit 216 is also capable of transmitting and receiving a control signal for a video signal. For example, the communication unit 216 may send a video signal conforming to a communication standard such as HDMI (registered trademark) (High Definition Multimedia Interface) or SDI (Serial Digital Interface). Each functional block is accessible to each other via an internal bus 220.

[0039] FIG. 3 is a functional configuration diagram of a system including an information processing device 100 and an inference device 200 for performing anonymity in this embodiment.

[0040] The information processing device 100 and the inference device 200 are connected to each other so as to be able to communicate with each other via communication units 107 and 207. Note that the communication units 107 and 207 only need to be able to communicate with each other, and each of them is configured using wireless communication such as WiFi (IEEE802.11 standard), or wired communication such as a wired LAN (Local Area Network) or USB (Universal Serial Bus).

[0041] The information processing device 100 separates (divides) the inference network model, which has been learned and recorded in advance in the recording unit 106, into first data and second data (the first data and the second data will be described later with reference to FIG. 4). Then, the information processing device 100 causes the encryption processing unit 109 to perform encryption processing on the first data, and generates a first file by adding an appropriate header. Then, the information processing device 100 causes the CPU 101 to add an appropriate header to the second data, and generates a second file. Note that, when generating the second file from the second data, the encryption processing unit 109 may perform encryption with a lower encryption strength than the first data, which allows for high-speed decryption. The information processing device 100 controls the communication unit 107 to transmit the generated first file and second file to the inference device 200. The inference device 200 controls the communication unit 207 to receive a first file and a second file from the information processing device 100. The inference device controls the encryption / decryption unit 209 to perform a decryption process on the received first file, restore the first data to the original first data, and record the data in the recording unit 206. If the received second file is unencrypted data, the inference device 200 controls the CPU 201 to extract the original second data from the second file and record the data in the recording unit 206. If the second file is encrypted data, the inference device 200 controls the encryption / decryption unit 209 to decrypt the second file to the original second data, and record the data in the recording unit 206. The inference device 200 generates an inference network model from the first data and the second data, and performs inference processing in the neural network processing unit 208.

[0042] FIG. 4 shows an example of data of an inference network model in the embodiment of the present invention.

[0043] FIG. 4(a) is an example of a case where data of an inference network model is composed of one file. For example, an .onnx file of ONNX (Open Neural Network Exchange) is an example of data composed of one file. As shown in FIG. 4(a), the inference network model includes layer configuration information 401 and coefficient data 402. The layer configuration information 401 is information indicating what layers exist in the neural network of the inference network model and how each layer is connected. The layer configuration information 401 also includes a file header. The coefficient data 402 is information such as a weight coefficient and a bias coefficient value, which are parameters of each layer.

[0044] Fig. 4(b) is an example of a case where the data format is composed of two files. For example, the .prototxt file and .caffemodel file of Caffe (Convolutional Architecture for Fast Feature Embedding) are examples of data formats composed of two files. As shown in Fig. 4(b), the inference network model is divided into a file storing layer configuration information 411, which indicates what layers exist in the network and how each layer is connected, and a file storing coefficient data 412, which is information such as weight coefficients and bias coefficient values that are parameters of each layer, and layer name 413, which indicates which layer the coefficient data corresponds to.

[0045] 4(c) shows an example of a case where an inference network model in an embodiment is separated into first data 421 to be encrypted and second data 422 to be unencrypted or to be encrypted with a lower encryption strength than the first encryption. The first data 421 stores layer configuration information and address information indicating which coefficient data stored in the second data 422 corresponds to which layer. The second data 422 also stores coefficient data.

[0046] FIG. 4(d) shows an example of converting the data of the second data 422. When a convolution layer and a nonlinear layer such as Relu are arranged as shown in the data 431 before conversion in FIG. 4(d), the boundary between layers becomes easy to see from the arrangement of the binary data. For this reason, in the embodiment, layers having similar or the same attributes are connected to convert the data as shown in the converted data 432 so that the boundary between layers cannot be easily identified. When rearranging the layer data, information for returning the data of each layer to its original order is included in the first data. In addition, at this time, it is desirable to store the conversion information for returning the data of each layer to its original order at the beginning or near the beginning of the first data. It takes a long time to complete the encryption / decryption process of the first data. However, if the inference device 200 can decode the conversion information to be referred to when returning the data of each layer of the second data to its original data at the initial stage of encryption / decryption of the first data, the inference device 200 can start the process of returning the data of each layer of the second data to its original order at that point in time without waiting for the completion of encryption / decryption of the first data. As a result, the time required for encryption / decryption processing of the entire neural network can be reduced.

[0047] 5 is a flowchart illustrating the processing of the information processing device 100 in the embodiment. This operation is realized by developing a computer program stored in the recording unit 106 in the memory 102 while the power supply of the information processing device 100 is on, and by the CPU 101 reading and executing the computer program in the memory 102.

[0048] In S501, the CPU 101 analyzes the data format of the inference network model (details will be described later with reference to FIG. 6). Then, the CPU 101 advances the process from S501 to S502.

[0049] In S502, CPU 101 loads setting information that has been set in advance via operation unit 105 from recording unit 106 to memory 102, and determines the data to be included in the first data described above in Fig. 3 and to be encrypted according to the setting information. For example, although all of the coefficient data is included in the second data in Fig. 4, some layers may be included in the first data. CPU 101 then advances the process from S502 to S503.

[0050] In S503, CPU 101 confirms with inference device 200 the encryption methods for the first data and the second data via communication unit 107. Upon receiving a response from inference device 200, CPU 101 can obtain the type of encryption that encryption / decryption unit 207 of inference device 200 can decrypt. CPU 201 determines the type of encryption to be used for the first data (for example, encryption with the highest encryption strength). CPU 201 also determines whether to encrypt the second data, and if so, the type (a type requiring less computation than the encryption of the first data). CPU 101 then advances the process from S503 to S504.

[0051] In S504, CPU 101 extracts a portion to be the first data from the inference network model, and controls encryption processing unit 109 to perform the previously determined type of encryption processing. CPU 101 records the encrypted first data as a first file in recording unit 106. CPU 101 then advances the process from S504 to S505.

[0052] In S505, the CPU 101 converts the data of the inference network model that was not included in the first data in S504 into the second data, performs data conversion processing, and records the result in the recording unit 106 as a second file. This data conversion refers to rearrangement processing as described in FIG. 4(d). The first data also includes conversion information related to rearrangement for returning the data before the possibility of the data in FIG. 4(d). In addition, in the conversion processing of the second data, processing such as shifting the bits of each coefficient to the left or right and averaging may be performed, or the bit shift may be used in combination with FIG. 4(d). However, when bit shifting is performed, the conversion information included in the first data also includes information indicating the number of bits to be shifted. The CPU 101 advances the process from S505 to S506.

[0053] In S506, CPU 101 determines whether or not to encrypt the second data (data of the second file recorded in recording unit 103) in S503. If CPU 101 determines to encrypt (YES in S506), the process proceeds from S506 to S507. If CPU 101 determines not to encrypt (NO in S506), the process proceeds from S506 to S508.

[0054] In S507, CPU 101 controls encryption processing unit 109 to perform encryption processing on the second data. Note that the encryption processing in this case is weaker than the encryption used in S504, and requires less computational effort for decryption. CPU 101 re-records (overwrites) the encrypted second data in recording unit 106 as a second file. At this time, CPU 101 includes, in the header of the second file, information specifying the type of encryption, etc., and further information indicating the presence of the conversion information described in S505. After this, CPU 101 advances the process from S507 to S508.

[0055] In S508, CPU 101 controls communication unit 107 to transmit the first file recorded in recording unit 106 to inference device 200. CPU 101 advances the process from S508 to S509.

[0056] In S509, CPU 101 controls communication unit 107 to transmit the second file recorded in recording unit 106 to inference device 200. CPU 101 ends the processing in this flowchart in S509.

[0057] Next, the process of S501 in FIG. 5 will be described in detail with reference to the flowchart in FIG.

[0058] In S601, CPU 101 determines whether the inference network model is in the file format of Fig. 4(a) or Fig. 4(b). If CPU 101 determines that the inference network model is composed of one file as shown in Fig. 4(a) (YES in S601), the process proceeds from S601 to S602. If CPU 101 determines that the inference network model is composed of two or more files as shown in Fig. 4(b) (NO in S602), the process proceeds from S601 to S603.

[0059] In S602, the CPU 101 determines the data format of the inference network model from the file extension or header information, analyzes the layer configuration information, which is graph data, and address information in which the coefficient data is stored, and records the analyzed information in the recording unit 106. The CPU 101 then ends this process.

[0060] In S603, CPU 101 determines the data format of the inference network model from the file extension or from a file among multiple files in which header information is saved, analyzes layer configuration information, which is graph data, and address information in which coefficient data is saved, and records the analysis in recording unit 106. CPU 101 advances the process from S603 to S604.

[0061] In S604, the CPU 101 extracts the data of the layer name 413 described above in Fig. 4B from the coefficient data portion analyzed in S603, and adds it to the first data. The CPU 101 ends the process in this flowchart in S604.

[0062] 7 is a flowchart showing the reception and encryption / decryption process of an inference network model in inference device 200 in the embodiment. This operation is started when, with inference device 100 powered on, a computer program stored in recording unit 206 is loaded into memory 202, and CPU 201 reads out and executes the computer program from memory 202. Note that, in the following description, it is assumed that information processing device 100 is performing the sorting process shown in FIG. 4(d) on the second data.

[0063] In S701, the CPU 201 receives an inquiry request for the encryption method from the information processing device 100 via the communication unit 216. In response to the inquiry request, the CPU 201 returns encryption method information (information indicating the types of encryption methods that the inference device 200 can decrypt) recorded in the memory 202 via the communication unit 216. The CPU 201 advances the process from S701 to S702.

[0064] In S702, the CPU 201 receives the first file from the information processing device 100 via the communication unit 216, and stores it in the recording unit 206. The CPU 201 advances the process from S702 to S703.

[0065] In S703, CPU 201 controls encryption / decryption unit 207 to start encryption / decryption processing for the data portion (first data) of the first file recorded in recording unit 206. Since the first data of the first file has a high encryption strength and therefore requires a long processing time, the processing is started prior to the reception of the second data. Thereafter, the processing in this step is executed in parallel while the processing in S704 to S708 is being executed, but the data necessary for restoring the second data is processed with priority. CPU 201 advances the processing from S703 to S704.

[0066] In S704, the CPU 201 receives the second file via the communication unit 216, and stores it in the recording unit 206. The CPU 201 advances the process from S704 to S705.

[0067] In S705, CPU 201 determines whether the second file is encrypted. This determination is made, for example, by analyzing the file header. If CPU 201 determines that the second data of the second file is encrypted (YES in S705), the process proceeds from S705 to S706. If CPU 201 determines that the second data is not encrypted (NO in S705), the process proceeds from S705 to S710.

[0068] In S706, the CPU 201 controls the encryption / decryption unit 207 to perform encryption / decryption processing on the data portion (second data) of the second file recorded in the recording unit 206. The CPU 201 advances the process from S706 to S710.

[0069] In S710, CPU 201 obtains conversion information related to the conversion performed on the second data from the encryption / decryption process of the first data. If the conversion information has not been obtained in the encryption / decryption process of the first data, CPU 201 waits for the conversion information to be obtained. As described above, in the embodiment, the conversion information for the second data is included in the beginning of the first data, such as at or near the beginning, before the encryption of the first data is started. Therefore, since the conversion information is obtained at an early stage of the encryption / decryption process of the first data, the waiting time of S710 is not required or can be made sufficiently short. When CPU 201 obtains the conversion information, the process proceeds from S710 to S707.

[0070] In S707, the CPU 201 performs a reverse conversion process to return the second data to the data before processing. For example, in order to perform a process to return the data converted in S505 in Fig. 5 to the original data, if a bit shift has been performed, the data is returned to the state before the bit shift, and if the layers have been rearranged, the data is returned to the state before the rearrangement. Also, if the layer name 413 has been moved from the second data to the first data, a process to return to the second data is performed. The CPU 201 advances the process from S707 to S708.

[0071] In S708, CPU 201 determines whether or not both the encryption / decryption process of the first data and the conversion of the second data started in S703 have been completed, and waits for their completion. Upon completing the encryption / decryption of the first data and the process of restoring the second data to its original state, CPU 201 moves data not required for the first data to the second data, and performs a process of restoring (regenerating) the inference network model. When CPU 201 determines that the restoration process has been completed (YES in S708), it advances the process from S708 to S709.

[0072] In S709, the CPU 201 controls the neural network processing unit 205 to execute a desired inference process (for example, an object recognition process) using the reproduced inference network model that has been restored and is recorded in the recording unit 206. The CPU 201 ends the process in this flowchart in S709.

[0073] As described above, in this embodiment, compared to the case where the entire inference network model is encrypted, the inference device 200 is able to start inference processing in a short time after receiving the encrypted inference network model.

[0074] Although each embodiment has been described above in detail, the present invention is not limited to the specific embodiment, and various modifications and changes are possible within the scope of the claims. In addition, it is also possible to combine all or a plurality of the components of the above-described embodiments. (Other Examples) The present invention can also be realized by a process in which a program for implementing one or more of the functions of the above-described embodiments is supplied to a system or device via a network or a storage medium, and one or more processors in a computer of the system or device read and execute the program. The present invention can also be realized by a circuit (e.g., ASIC) that implements one or more of the functions.

[0075] The disclosure of this specification includes the following information processing device, method, program, and information processing system. (Item 1) An information processing device that transmits data representing an inference network model to an external device, A separation means for separating data representing the inference network model into first data including information for representing at least a layer configuration of the inference network model and second data representing coefficient data of each layer; a generating means for generating a first file including the first data and a second file including the second data; a transmitting means for transmitting the generated first file and the generated second file to an information processing device, The generating means generates the first file from data obtained by encrypting the first data. 23. An information processing apparatus comprising: (Item 2) The method further includes a determination means for determining a format of the inference network model, The separating means extracts data in which the first data is stored from an inference network model based on a result of the determination by the determining means, and generates the first file. 2. The information processing device according to item 1, (Item 3) The second data includes coefficient data of each layer used in inference execution; The first data includes address information indicating a location in the second data where coefficient data of each layer is stored. 3. The information processing device according to item 1 or 2, (Item 4) Further comprising a processing means for processing the second data, The first data includes information for restoring the data to the data before processing by the processing means. 4. The information processing device according to any one of items 1 to 3. (Item 5) The processing means is bit shifting means for shifting each coefficient data of the second data by a predetermined number of bits 5. The information processing device according to item 4, (Item 6) The processing means is bit shifting means for shifting each coefficient data of the second data by a predetermined number of bits 5. The information processing device according to item 4, (Item 7) 7. The information processing device according to any one of items 4 to 6, wherein information for restoring the data to the state before processing by the processing means is stored at the beginning of the first data. (Item 8) The generating means generates the second file by not encrypting the second data, or by encrypting the second data with lower encryption strength and with less computational complexity for decryption than the encryption of the first data. 8. The information processing device according to any one of items 1 to 7. (Item 9) The method further includes acquiring information indicating a type of encryption that the external device can decrypt by communicating with the external device, The generating means generates the first file by encrypting the first data with the encryption having the maximum encryption strength among the acquired encryption types. 9. The information processing device according to any one of items 1 to 8. (Item 10) The external device is an inference device that receives the first and second files, performs encryption / decryption to reproduce the inference network model, and executes an inference process. 10. The information processing device according to any one of items 1 to 9. (Item 11) 1. A method for controlling an information processing device that transmits data representing an inference network model to an external device, comprising: a separation step of separating the data representing the inference network model into first data including information for representing at least a layer configuration of the inference network model and second data representing coefficient data of each layer; a file generating step of generating a first file including the first data and a second file including the second data; a transmitting step of transmitting the generated first file and the generated second file to an information processing device; The file generating step generates the first file from data obtained by encrypting the first data. 23. A method for controlling an information processing apparatus comprising: (Item 12) A program that, when read and executed by a computer, causes the computer to execute each step of the method according to item 11. (Item 13) An information processing system including an information processing device that creates an inference network model, and an inference device that receives data representing the inference network model from the information processing device and executes inference processing, The information processing device includes: A separation means for separating data representing the inference network model into first data including information for representing at least a layer configuration of the inference network model and second data representing coefficient data of each layer; A generating means for generating a first file including the first data and a second file including the second data; a transmitting means for transmitting the generated first file and the generated second file to an information processing device, The generating means generates the first file from data obtained by encrypting the first data, The inference device comprises: a receiving means for receiving the first file and the second file from an information processing device; a reproduction means for reproducing the inference network model by decrypting an encrypted data portion of the received first file to generate the first data and extracting a data portion of the second file as the second data; an inference means for performing inference processing using the inference network model reproduced by said reproduction means; An information processing system comprising:

[0076] (Item 14) the information processing device has a processing means for processing the second data, The generating means generates a second file from the second data processed by the processing means, storing information for restoring the second data to the data before processing at the beginning of the first data, and then encrypting the data to generate a first file; The reproduction means of the inference device Upon receiving the first file, starting an encryption / decryption process of a data portion of the first file to obtain the first data; In response to obtaining information for restoring the data to the data before processing by the processing means through the encryption / decryption process of the data portion of the first file, the second data is generated by performing a process for restoring the data portion of the second file to the data before processing based on the obtained information. 14. The information processing system according to item 13,

[0077] The invention is not limited to the above-described embodiments, and various modifications and variations are possible without departing from the spirit and scope of the invention. Accordingly, the following claims are appended to apprise the public of the scope of the invention. [Explanation of symbols]

[0078] 100...information processing device, 101...CPU, 102...memory, 103...display unit, 104...internal bus, 105...operation unit, 106...recording unit, 107...communication unit, 108...neural network processing unit, 109...encryption processing unit, 200...inference device, 201...CPU, 202...memory, 203...non-volatile memory, 204...operation unit, 205...neural network processing unit, 206...recording unit, 207...encryption / decryption unit

Claims

1. An information processing device that transmits data representing an inference network model to an external device, a separating means for separating data representing the inference network model into first data including information for representing at least a layer configuration of the inference network model and second data representing coefficient data of each layer; a generating means for generating a first file including the first data and a second file including the second data; a transmitting means for transmitting the generated first file and the generated second file to an information processing device, The generating means generates the first file from the data after encrypting the first data.

1. An information processing device comprising:

2. Further, a determination means for determining a format of the inference network model is provided, The separating means extracts data in which the first data is stored from the inference network model based on the result of the determination by the determining means, and generates the first file.