Media capture device with power saving and encryption functions for a partitioned neural network

Abstract

A method for power conservation and encryption when analyzing media captured by an information capture device using a partitioned neural network includes replicating, by the information capture device, an artificial neural network (ANN) from a computer server to the information capture device. Both the ANN on the computer server and the replicated ANN on the information capture device include M layers. The method further includes, in response to the captured data being input to be processed, partially processing the captured data by running the first k layers with the replicated ANN, by the information capture device, where only the k layers are selected for execution on the information capture device. The method further includes transmitting, by the information capture device, an output of the kth layer to the computer server, where the computer server partially processes the captured data by running the remainder of the M layers using the ANN and the output of the kth layer.

Description

Technical Field

[0001] The present invention generally relates to computing technology, and more specifically, to media capture devices, and neural networks that facilitate media capture devices that perform power saving and data security protection.

Background Art

[0002] Today, several devices such as telephones, tablet computers, and wearable devices capture or create, or both, media objects such as digital images, audio, and video. As the need to classify large amounts of captured or extracted media, or both, increases, learning models have become a common way to classify captured media objects. For example, learning models such as artificial neural networks (ANNs) or convolutional neural networks (CNNs), or both, are trained with sample data, i.e., sample media objects, and continuously evolve (learn) in the process of classifying new (previously unseen) media objects.

Summary of the Invention

[0003] One or more embodiments of the present invention include a computer-implemented method for power savings and encryption in analyzing media captured by an information capture device using a partitioned neural network. The method includes replicating, by the information capture device, an artificial neural network (ANN) from a computer server to the information capture device, both the ANN on the computer server and the replicated ANN on the information capture device including M layers. The method further includes partially processing the captured data by the information capture device by executing the first k layers using the replicated ANN in response to the captured data being an input to be processed, with only the k layers being selected for execution on the information capture device. The method further includes transmitting, by the information capture device, the output of the k-th layer to the computer server, the computer server partially processing the captured data by executing the remainder of the M layers using the ANN and the output of the k-th layer.

[0004] According to one or more embodiments of the present invention, a system includes a memory and one or more processors coupled to the memory, and the one or more processors execute a method for power saving and encryption when analyzing media captured by an information capture device using a partitioned neural network. The method includes, by an information capture device, replicating an artificial neural network (ANN) from a computer server to the information capture device, and both the ANN on the computer server and the replicated ANN on the information capture device include M layers. The method further includes, in response to the captured data being an input to be processed, partially processing the captured data by the information capture device by executing the first k layers using the replicated ANN, and only k layers are selected for execution on the information capture device. The method further includes transmitting, by the information capture device, the output of the k-th layer to the computer server, and the computer server partially processes the captured data by executing the remaining of the M layers using the ANN and the output of the k-th layer.

[0005] According to one or more embodiments of the present invention, a computer program product includes a computer-readable storage medium having program instructions embodied thereon. The program instructions are executable by one or more processors and cause the one or more processors to perform a method including operations for power saving and encryption in the analysis of media captured by an information capture device using a partitioned neural network. The method includes the information capture device replicating an artificial neural network (ANN) from a computer server to the information capture device, both the ANN on the computer server and the replicated ANN on the information capture device including M layers. The method further includes the information capture device partially processing the captured data by executing the first k layers using the replicated ANN in response to the captured data being an input to be processed, with only the k layers being selected for execution on the information capture device. The method further includes the information capture device transmitting the output of the k layers to the computer server, the computer server partially processing the captured data by executing the remaining of the M layers using the ANN and the output of the k-th layer.

[0006] Other embodiments of the present invention implement the features of the method described above in a computer system and a computer program product.

[0007] Additional technical features and advantages are realized by the techniques of the present invention. Embodiments and aspects of the present invention are described in detail herein and are considered a part of the claimed subject matter. Refer to the detailed description and the drawings for a better understanding.

Brief Description of the Drawings

[0008] The details of the exclusive rights described in this specification are specifically pointed out and clearly claimed in the claims at the end of this specification. The above and other features and advantages of the embodiments of the present invention will be apparent from the following detailed description used in conjunction with the accompanying drawings.

[0009]

Figure 1

Figure 2

Figure 3

Figure 4

Figure 5

Figure 6

DETAILED DESCRIPTION OF THE INVENTION

[0010] Embodiments of the present invention facilitate a media capture device having power saving and encryption capabilities when using a partitioned neural network to process one or more media objects such as images, audio, video, etc. Currently, large amounts of data such as images, audio, video, etc. are created by multiple users using edge devices such as phones, tablets, computers, wearable devices, dash cameras, voice recorders, security cameras, etc. There are technical challenges in processing large amounts of data including such media using deep neural network (DNN) architectures. Here, "large volume" may involve millions of images, audio, video, and it is impracticable, if not impossible, to manually process and classify data of such amounts. Accordingly, embodiments of the present invention provide a practical application for classifying large volumes of media captured by one or more information capture devices. Further, embodiments of the present invention improve the operation of information capture devices by facilitating a media capture device that performs power saving and data security protection. Still further, embodiments of the present invention address the limitations of computing resources in information capture devices by improving the computational efficiency of information capture devices during such media classification tasks.

[0011] FIG. 1 shows a block diagram of a system 100 for processing information captured by one or more information capture devices according to one or more embodiments of the present invention. An information capture device 102 (e.g., a camera, a phone, a security camera, a tablet computer, a voice recorder, etc.) captures information as an analog signal that can be stored in one or more digitized files 103. The analog signal sensed by the information sensing array 112 of the information capture device 102 is digitized by an analog-to-digital (ADC) module 114. The information capture device 102 can further include a processor 116 that can perform one or more digital signal processing operations on the digitized data, such as image processing, audio processing, video processing, etc. The processor 116 can store the digitized data as a digitized file 103. The digitized file 103 can be stored as an electronic file using one or more digital file storage formats. For example, the visual information captured by the information sensing array 112 can be stored using an image file format such as Portable Network Graphics (PNG), Bitmap (BMP), etc. In the case of audio data sensed by the information sensing array 112, the digitized audio can be stored using a file format such as Waveform Audio File Format (WAV), Free Lossless Audio Codec (FLAC), etc. When video is captured, the data can be stored using a digital file format such as Video Object (VOB), Audio Video Interleave (AVI), MPEG-14, etc.

[0012] In one or more embodiments of the present invention, the processor 116 can include one or more processing units, such as a processor core. The processor 116 can be a microprocessor, a multiprocessor, a digital signal processor, a graphics programming unit, a central processing unit, and other such types of processing units, or a combination thereof. The processor 116 can include or be coupled to a memory device 117. The processor 116 can perform one or more operations by executing one or more computer-executable instructions. Such instructions can be stored in the memory device 117. The memory device 117 can store additional information / data that can be used or output by the processor 116.

[0013] The captured data from the digitized file 103 is transferred via the communication network 104 to the computer server 106, where further processing, such as the classification of the digitized file 103, is performed.

[0014] The communication network 104 can be a computer network, such as the Internet, that uses one or more communication protocols, such as Ethernet. In one or more embodiments of the present invention, the digitized file 103 is captured by the user 101 using the information capture device 102.

[0015] Computer server 106 can be a server cluster or distributed server that provides cloud-based processing services for digitized file 103 captured by information capture device 102. In one or more embodiments of the present invention, computer server 106 includes artificial neural network (ANN) 122. ANN 122 can be a convolutional neural network, feedforward network, recurrent neural network, multilayer perceptron, or a combination thereof. ANN 122 can be an independent hardware module in one or more embodiments of the present invention. Alternatively or additionally, ANN 122 can be implemented using processor 127 of computer server 106. ANN 122 includes multiple layers, where the output of one layer is used by a later layer until final output 123 is generated.

[0016] In one or more embodiments of the present invention, processor 126 can include one or more processing units such as processor cores. Processor 126 can be a microprocessor, multiprocessor, digital signal processor, graphics programming unit, central processing unit, and other such types of processing units, or a combination thereof. Processor 126 can include or be coupled to memory device 127. Processor 126 can perform one or more operations by executing one or more computer-executable instructions. Such instructions can be stored on memory device 127. Memory device 127 can store additional information / data that can be used or output by processor 126.

[0017] In one or more embodiments of the present invention, ANN122 is trained using training data 124. The training data 124 can include labels and other hints that can be used to train ANN122 to analyze the captured data from the information capture device 102 and generate an ANN output 123 during the inference stage, and can include predetermined media such as images, audio, video, etc. The ANN output 123 can include classification into one or more categories of the digitized file 103, object detection results of the digitized file 103, and other such image processing / computer vision, as well as audio processing results.

[0018] In a conventional system, the digitized file 103 is encrypted before being sent to the server 106. If the encryption is compromised (i.e., hacked), the captured data from the digitized file 103 may be exposed.

[0019] Embodiments of the present invention combine neural network analysis with encryption by splitting ANN122 and creating a replica of ANN122 on the information capture device 102. In one or more embodiments, the captured data is first processed through one or more layers of ANN122, and then the output of ANN122 is sent to the computer server 106 via the network 104, and the remaining layers of ANN122 are further processed.

[0020] Alternatively, in other embodiments, the analog signals sensed by the information sensing array 112 on the information capture device 102 are first connected through one or more layers of ANN122. The output of one or more layers of ANN122 is sent to the computer server 106 via the network 104, and the remaining layers of ANN122 are further processed.

[0021] In embodiments of the present invention, it is also possible to encrypt the weights of one or more intermediate layers of ANN122 and the output of the intermediate layer of ANN122. In this way, since the captured data is not transferred via network 104, the security of the captured data is improved. Embodiments of the present invention provide improvements to system 100, components of system 100 such as information capture device 102, computer server 106, and one or more methods of using system 100 or components of system 100, or both, for example, to analyze digitized file 103 captured by information capture device 102 in a secure manner.

[0022] FIG. 2 is a block diagram showing improvements to one or more components of system 100 for power savings and encryption of captured data using a partitioned neural network, according to one or more embodiments of the present invention. The depiction shows layer 202 within ANN122. ANN122 includes M layers, where M is any integer. Each layer, except layer #1, uses the output from the previous layer.

[0023] ANN122 is trained using training data 124. Such training includes learning (i.e., configuring, setting) one or more weights associated with each of the layers 202 of ANN122. The weights are automatically learned using one or more training techniques such as supervised learning, unsupervised learning, or any other learning technique for ANN122.

[0024] The information capture device 102 includes an ANN-replica 204 that is a replica of the ANN 122. The ANN-replica 204 is identical to the ANN 122 and includes the same M layers. Further, to make the ANN-replica 204 identical to the ANN 122, the weights learned by the ANN 122 are transmitted to the ANN-replica 204 on the information capture device 102. In one or more embodiments of the present invention, the weights are encrypted by the encryption unit 230 of the computer server 106. The decryption unit 232 of the information capture device 102 decrypts the encrypted weights from the encryption unit 230. The decrypted weights output by the decryption unit 232 are constituted by the ANN-replica 204.

[0025] The information capture device 102 further includes a layer selector 210 that selects which of the M layers from the ANN-replica 204 are to be executed on the information capture device to analyze the digitized file 103 created by the information capture device 102. For example, the layer selector 210 can use the digitized file 103 as an input to select the first k layers (1 ≤ k ≤ M) of the ANN-replica 204 to be executed by the information capture device 102. In one or more embodiments of the present invention, the layer selector 210 determines the value of k based on the power consumed by the information capture device 102 to execute the layers of the ANN-replica 204. In other embodiments, additional or alternative parameters can be used to select the value of k.

[0026] The output of layer #k from the ANN-replica 204, which uses the digitized file 103 as an input to the ANN-replica 204, is transmitted to the computer server 106. In one or more embodiments of the present invention, the output of layer #k is encrypted by the encryption unit 220 of the information capture device 102 before transmission. The decryption unit 222 of the computer server 106 decrypts the output of layer #k. This received output of layer #k is input to layer #(k + 1) of the ANN 122. In one or more embodiments of the present invention, the layer locator 212 of the computer server 106 identifies layer #(k + 1) within the ANN 122 and inputs the received output of layer #k to that layer #(k + 1) within the ANN 122.

[0027] In one or more embodiments of the present invention, the layer selector 210 transmits the identification of the layer whose output is to be transmitted, i.e., layer #k, to the layer locator 212. In one or more embodiments of the present invention, the identification of layer #k is encrypted by the encryption unit 220 before transmission. The decryption unit 222 decrypts the identification of layer #k for use by the layer locator 212.

[0028] Thereafter, layers (k + 1) to M of the ANN 122 are executed to generate the result 123 of the ANN 122. In one or more embodiments of the present invention, the result 123 is transmitted to the information capture device 102 or to any other device (not shown).

[0029] Accordingly, the system 100 facilitates a variable workload split, where a subset of the ANN's layers are executed on the information capture device 102 and the remaining layers are executed on the computer server 106. Further, the data exchanged between the information capture device 102 and the computer server 106 is securely protected, and in that case as well, only intermediate data is exchanged to limit the exposure of the entire digitized file 103 and thus limit the possibility of the digitized file 103 being hacked during such data exchange.

[0030] In one or more embodiments of the present invention, the information capture device 102 transmits the output of each of the layers, i.e., layers 1 to k, which is executed by the ANN-replica 204, along with the identification of layer #k.

[0031] In one or more embodiments of the present invention, the ANN-replica 204 uses the analog signals captured by the information sensing array 112 before the captured data is converted into the digitized file 103. This makes it easier to securely protect the captured data from being further jeopardized. In this case, the ANN 122 is trained using the training data 124 including the analog signals.

[0032] FIG. 3 shows a flowchart of a method 300 for analyzing data captured by power saving and encryption using a partitioned neural network, according to one or more embodiments of the present invention. Method 300 includes, at block 302, training the ANN 122 of computer server 106 using training data 124. The training can include supervised learning, unsupervised learning, or training of any other type of neural network. The training data 124 can include analog signals captured by an information sensing array, such as the information sensing array 112. Alternatively or additionally, the training data 124 can also include media obtained after digitizing such analog signals. The training facilitates configuring the M layers 202 of the ANN 122 to have weights. Here, a "weight" is a parameter within the ANN 122 that transforms the input data provided to any of the M layers 202. Each of the M layers 202 can include a plurality of weights. The ANN 122 is trained to analyze the captured data either in the form of analog signals captured by the information sensing array 112 or in the form of the digitized file 103. For example, such analysis can include detecting and identifying objects within the captured data. Further, the analysis can include classifying the identified objects or the captured data, or both, into one or more categories. In one or more embodiments of the present invention, other types of analysis can also be performed additionally or alternatively.

[0033] Further, in block 304, ANN 122 is replicated onto the information capture device 102. The replication includes configuring an ANN-replica 204 on the information capture device 102. The ANN-replica 204 is configured to have the same number of layers, i.e., M layers. Further, each of the layers of the ANN-replica 204 is configured to have exactly the same weights as the M layers 202 of the ANN 122 of the computer server 106. In one or more embodiments of the present invention, such replication includes encrypting the weights trained using the encryption unit 230 and transmitting the encrypted values to the information capture device 102. The decryption unit 232 decrypts the weight values and then uses this to configure the ANN-replica 204.

[0034] Thereafter, in block 306, the information capture device 102 captures analog signal data using the information sensing array 112. In block 308, the captured data is input into the ANN-replica 204 of the information capture device 102 and processed using only k out of the M layers of the ANN-replica 204. The captured data input into the ANN-replica 204 can be an analog signal captured by the information sensing array 112 or the corresponding digitized file 103.

[0035] In block 310, processing the captured data includes selecting the number of layers, i.e., k, to be executed by the information capture device 102. The layer selector 210 determines the value of k based on one or more factors associated with the information capture device. In one or more embodiments of the present invention, the layer selector 210 monitors the amount of power consumed to execute each of the layers of the ANN-replica 204. Alternatively or additionally, the layer selector 210 has access to power consumption data indicating the amount of power required by the information capture device 102 to execute each of the layers of the ANN-replica 204. In one or more embodiments of the present invention, the layer selector 210 can further include a power consumption budget for the ANN-replica 204. The power consumption budget can be a configurable value.

[0036] The power consumption budget indicates the maximum amount of power that the ANN-replica 204 can consume to analyze the captured data. In one or more embodiments of the present invention, the power consumption budget can be a value that depends on the total amount of power available to the information capture device 102. For example, if the information capture device 102 receives power from a battery or any other such limited power source (not shown), the amount of power available can be determined by the charge level of the power source. As the charge level changes, the power consumption budget can change. For example, when the charge level is at least 75% of the capacity of the power source, the power consumption budget for the ANN-replica 204 can be 100 milliwatts to analyze the captured data. When the charge level drops to 50%, the power consumption budget decreases to 80 milliwatts, and when the charge level drops to 30%, it further decreases to 50 milliwatts, etc. It is understood that the exemplary values above can vary in one or more embodiments of the present invention. In one or more embodiments of the present invention, the relationship between the power consumption budget and the charge level can be configurable.

[0037] Accordingly, based on the determined power consumption budget and the amount of power required for each of the layers within ANN-Replica 204, the layer selector determines that the information capture device 102 can execute k layers without exceeding the power consumption budget. In response, the first k layers of ANN-Replica 204 are executed by the information capture device 102 (at block 308).

[0038] Since ANN-Replica 204 includes an exact replica of the M layers 202 of ANN122, the remaining layers (k + 1) to M of ANN122 can take over the analysis of the captured data. For this purpose, at block 312, the output of layer #k from ANN-Replica 204 is transmitted to the computer server 106 via the network 104. The transmission can further include the identification of layer k, for example, the value of k.

[0039] In one or more embodiments of the present invention, the transmission is encrypted by the encryption unit 220. In one or more embodiments of the present invention, the output of layer #k and the identification of k can be part of a single encrypted transmission. Alternatively, separate encrypted transmissions can be performed for the output of layer #k and the identification of k.

[0040] At block 314, ANN122 analyzes the captured data by executing layers (k + 1) to M using the output of layer #k. Such analysis includes decrypting the information received from the information capture device 102 by the decryption unit 222. Further, the layer locator 212 identifies layers #k and #k + 1 of ANN122 and configures these layers with the information from the information capture device 102 so that ANN122 can execute the remaining M - k layers starting from layer #k + 1.

[0041] In block 316, the processing result of ANN 122 is output. In one or more embodiments of the present invention, the result can be transmitted to the information capture device 102. Alternatively or additionally, the result can also be transmitted to another device such as another computer server, database, or any other device.

[0042] In one or more embodiments of the present invention, FIGS. 1 and 2 show a single information capture device 102, but it should be noted that multiple information capture devices 102 can communicate with the computer server 106. Further, each information capture device 102 may have its own power consumption budget, charge level, and other such variable factors. Accordingly, the number of layers executed on the first information capture device 102 may be different from the number of layers on the second information capture device, for example, k' (k ≠ k'). Correspondingly, for the first information capture device, the computer server 106 executes a different number of layers (M - k) compared to the number of layers (M - k') executed for the second information capture device.

[0043] Furthermore, even for a single information capture device 102, the number of layers k can vary based on the charge level. For example, when the charge level is X%, at time t1, the computer server 106 executes (M - k) layers of ANN 122 for the first captured data captured by the information capture device 102, while when the charge level is Y%, at time t1, the computer server 106 can execute (M - p) layers of ANN 122 for the second captured data captured by the information capture device 102, where p is the number of layers selected by the layer selector 210.

[0044] Embodiments of the present invention encrypt the neural network process by partitioning the neural network and creating a replica of the neural network on an information capture device. The captured data is analyzed in the information capture device using a subset of the layers of the neural network, and the output of such processing is transmitted to a computer server and further processed using the remaining layers of the neural network. The number of layers executed on the information capture device is based on one or more factors in the information capture device, such as power consumption. The captured data can be used in the form of an analog signal or in the form of a digitized file. Further, all transmissions, such as weights for replicating the neural network, outputs of the layers executed on the information capture device, etc., are encrypted. In this way, the captured data is not transferred directly over the network, and as a result, the security of the captured data can be enhanced.

[0045] Referring now to FIG. 4, a computer system 400 according to an embodiment is generally shown. In one or more embodiments of the present invention, the computer system 400 can be used as an information capture device 102 or a computer server 106, or both. As described herein, the computer system 400 can include and / or utilize any number and combination of computing devices and networks that utilize various communication technologies, and can be an electronic, computer framework. The computer system 400 can be easily scalable, extensible, modular, and can have the ability to change to different services or reconfigure some functions independently of others. The computer system 400 can be, for example, a server, a desktop computer, a laptop computer, a tablet computer, or a smartphone. In some examples, the computer system 400 can be a cloud computing node. The computer system 400 may be described in the general context of computer system executable instructions, such as program modules, executed by a computer system. Generally, program modules can include routines, programs, objects, components, logic, data structures, etc., that perform specific tasks or implement specific abstract data types. The computer system 400 can be implemented in a distributed cloud computing environment where tasks are executed by remote processing devices linked through a communication network. In a distributed cloud computing environment, program modules can be located on both local computer system storage media, including memory storage devices, and remote computer system storage media.

[0046] As shown in FIG. 4, computer system 400 has one or more central processing units (CPUs) 401a, 401b, 401c, etc. (collectively or generally referred to as processor 401). The processor 401 can be a single-core processor, a multi-core processor, a computing cluster, or any number of other configurations. The processor 401, also called a processing circuit, is coupled to system memory 403 and various other components via system bus 402. System memory 403 can include read-only memory (ROM) 404 and random access memory (RAM) 405. ROM 404 is coupled to system bus 402 and can include a basic input / output system (BIOS) that controls certain basic functions of computer system 400. RAM is a read / write memory coupled to system bus 402 for use by processor 401. System memory 403 provides a temporary memory space for the operation of the aforementioned instructions during operation. System memory 403 can include random access memory (RAM), read-only memory, flash memory, or any other suitable memory system.

[0047] Computer system 400 includes an input / output (I / O) adapter 406 and a communication adapter 407 coupled to system bus 402. I / O adapter 406 can be a small computer system interface (SCSI) adapter that communicates with a hard disk 408 or any other similar component, or both. In this specification, I / O adapter 406 and hard disk 408 are collectively referred to as mass storage 410.

[0048] Software 411 that runs on computer system 400 can be stored in mass storage 410. Mass storage 410 is an example of a tangible storage medium readable by processor 401, and software 411 is stored as instructions to be executed by processor 401 to operate computer system 400 as described hereinafter with respect to various figures herein. Examples of computer program products and execution of such instructions are described in more detail herein. Communication adapter 407 interconnects system bus 402 with network 412, which can also be an external network, enabling computer system 400 to communicate with other such systems. In one embodiment, a portion of system memory 403 and mass storage 410 collectively store an operating system that can be any suitable operating system, such as the z / OS or AIX operating system from IBM Corporation, and coordinate the functions of the various components shown in FIG. 4.

[0049] Additional input / output devices are shown as being connected to system bus 402 via display adapter 415 and interface adapter 416. In one embodiment, adapters 406, 407, 415, and 416 can be connected to one or more I / O buses that are connected to system bus 402 via an intermediate bus bridge (not shown). Display 419 (e.g., a screen or display monitor) is connected to system bus 402 by a display adapter 415 that can include a graphics controller and a video controller to improve the performance of graphics-intensive applications. Keyboard 421, mouse 422, speaker 423, etc. can be interconnected to system bus 402 via an interface adapter 416 that can include, for example, a super I / O chip that integrates multiple device adapters into a single integrated circuit. Appropriate I / O buses for connecting peripheral devices such as hard disk controllers, network adapters, and graphics adapters typically include a common protocol such as Peripheral Component Interconnect (PCI). Thus, as configured in FIG. 4, computer system 400 includes processing capabilities in the form of processor 401, storage capabilities including system memory 403 and mass storage 410, input means such as keyboard 421 and mouse 422, and output capabilities including speaker 423 and display 419.

[0050] In some embodiments, communication adapter 407 can transmit data using any suitable interface or protocol, such as, in particular, an Internet Small Computer System Interface. Network 412 can be, in particular, a cellular network, a wireless network, a wide area network (WAN), a local area network (LAN), or the Internet. An external computing device can be connected to computer system 400 via network 412. In some examples, the external computing device can be an external web server or a cloud computing node.

[0051] It is understood that the block diagram of FIG. 4 is not intended to show that computer system 400 includes all of the components shown in FIG. 4. Rather, computer system 400 can include any suitable fewer or additional components not shown in FIG. 4 (e.g., additional memory components, embedded controllers, modules, additional network interfaces, etc.). Further, the embodiments described herein with respect to computer system 400 can be implemented with any suitable logic, where the logic referred to herein can include, in various embodiments, any suitable hardware (e.g., in particular, a processor, an embedded controller, or an application specific integrated circuit), software (e.g., in particular, an application), firmware, or any suitable combination of hardware, software, and firmware.

[0052] Although this disclosure includes a detailed description regarding cloud computing, it should be understood that the implementation of the teachings described herein is not limited to a cloud computing environment. Rather, embodiments of the invention can be implemented with any other type of computing environment now known or later developed.

[0053] Cloud computing is a service delivery model that enables convenient on-demand network access to a shared pool of configurable computing resources (such as networks, network bandwidth, servers, processing, memory, storage, applications, virtual machines, and services) that can be rapidly provisioned and released with minimal management effort or interaction with a service provider. This cloud model can include at least five characteristics, at least three service models, and at least four deployment models.

[0054] The characteristics are as follows.

[0055] On-demand self-service: Cloud consumers can, as needed, automatically and unilaterally provision computing capabilities such as server time and network storage without the need for human interaction with the service provider.

[0056] Broad network access: The capabilities are available over the network and accessed through standard mechanisms that promote use by heterogeneous thin or thick client platforms (such as mobile phones, laptops, and PDAs).

[0057] Resource pooling: The provider's computing resources are pooled to serve multiple consumers using a multi-tenant model, and different physical and virtual resources are dynamically assigned and re-assigned according to demand. Consumers can generally be said to be location-independent in that they have no control or knowledge of the exact location of the resources provided, although they may be able to specify a higher level of abstraction location (such as a country, state, or data center).

[0058] Rapid elasticity: The ability to provision, scale out quickly, and release resources rapidly to scale in, either quickly and elastically or in some cases automatically. To the consumer, the capabilities available for provisioning often appear to be unlimited and can be purchased in any quantity at any time.

[0059] Measured services: Cloud systems automatically control and optimize resource use by using metering capabilities at some level of abstraction appropriate to the type of service (e.g., storage, processing, bandwidth, and active user accounts). Resource usage can be monitored, controlled, and reported, providing transparency to both the provider and consumer of the utilized service.

[0060] The service model is as follows.

[0061] Software as a Service (SaaS): The capabilities provided to the consumer are to use the provider's applications that run on the cloud infrastructure. These applications are accessible from various client devices through a client interface such as a web browser (e.g., web-based email). The consumer does not manage or control the underlying cloud infrastructure, including the network, servers, operating systems, storage, or individual application capabilities, with the exception of limited user-specific application configuration settings.

[0062] Platform as a Service (PaaS): The function provided to the consumer is to deploy the applications created or acquired by the consumer, which are created using the programming languages and tools supported by the provider, onto the cloud infrastructure. The consumer does not manage or control the underlying cloud infrastructure including the network, servers, operating systems, or storage, but controls the deployed applications and, in some cases, the environmental configuration that hosts the applications.

[0063] Infrastructure as a Service (IaaS): The function provided to the consumer is to provision processing, storage, network, and other basic computing resources on which the consumer can deploy and run any software that may include operating systems and applications. The consumer does not manage or control the underlying cloud infrastructure, but has limited control over the operating system, storage, control of the deployed applications, and, in some cases, selection of network components (e.g., the host firewall).

[0064] The deployment model is as follows.

[0065] Private cloud: The cloud infrastructure is operated only for a certain organization. It can be managed by that organization or a third party and can exist on-premises or off-premises.

[0066] Community Cloud: The cloud infrastructure is shared by several organizations and supports a specific community with common concerns (e.g., mission, security requirements, policies, and compliance considerations). It can be managed by those organizations or a third party and can exist on-premises or off-premises.

[0067] Public Cloud: The cloud infrastructure is available to the general public or a large industry group and is owned by an organization that sells cloud services.

[0068] Hybrid Cloud: The cloud infrastructure remains a distinct entity but is a hybrid of two or more clouds (private, community, or public) tied together by standardized or proprietary technologies (e.g., cloud bursting for load balancing between clouds) that enable data and application portability.

[0069] The cloud computing environment is service-oriented, focusing on statelessness, low coupling, modularity, and semantic interoperability. The core of cloud computing is an infrastructure that includes a network of interconnected nodes.

[0070] Referring now to FIG. 5, an exemplary cloud computing environment 50 is shown. As illustrated, cloud computing environment 50 includes one or more cloud computing nodes 10 with which local computing devices such as, for example, a personal digital assistant (PDA) or cellular telephone 54A, desktop computer 54B, laptop computer 54C, or automotive computer system 54N, or a combination thereof, can communicate. The nodes 10 can communicate with one another. The nodes 10 can be physically or virtually grouped in one or more networks such as the private cloud, community cloud, public cloud, or hybrid cloud, or a combination thereof, described above (not shown). This allows cloud computing environment 50 to provide Infrastructure as a Service, Platform as a Service or Software as a Service, or a combination thereof, which does not require a cloud consumer to maintain resources on a local computing device. The types of computing devices 54A - N shown in FIG. 5 are intended to be exemplary only, and it should be understood that cloud computing node 10 and cloud computing environment 50 can communicate with any type of computerized device via any type of network or network addressable connection, or both (e.g., using a web browser).

[0071] Referring now to FIG. 6, a set of functional abstractions provided by cloud computing environment 50 (FIG. 5) is shown. It should be understood in advance that the components, layers, and functions shown in FIG. 6 are intended to be exemplary only and embodiments of the invention are not limited thereto. As illustrated, the following layers and corresponding functions are provided.

[0072] The hardware and software layer 60 includes hardware components and software components. Examples of hardware components include mainframe 61, RISC (Reduced Instruction Set Computer) architecture-based server 62, server 63, blade server 64, storage device 65, and network and networking components 66. In some embodiments, the software components include network application server software 67 and database software 68.

[0073] The virtualization layer 70 provides an abstraction layer that can provide the following examples of virtual entities: virtual server 71, virtual storage 72, virtual network 73 including a virtual private network, virtual applications and operating systems 74, and virtual client 75.

[0074] In one example, the management layer 80 can provide the functions described below. Resource provisioning 81 provides for the dynamic procurement of computing resources and other resources utilized to execute tasks within a cloud computing environment. Metering and pricing 82 provides for cost tracking when resources are utilized within a cloud computing environment and for charging or billing for the consumption of these resources. In one example, these resources can include application software licenses. Security provides for authentication of cloud consumers and tasks and for protection of data and other resources. The user portal 83 provides access to the cloud computing environment for consumers and system administrators. Service level management 84 provides for the allocation and management of cloud computing resources such that required service levels are met. Planning and fulfillment of service level agreements (SLAs) 85 provides for the pre - placement and procurement of cloud computing resources whose future requirements are predicted according to the SLA.

[0075] The workload layer 90 provides examples of functions that can utilize a cloud computing environment. Examples of workloads and functions that can be provided from this layer include mapping and navigation 91, software development and life cycle management 92, virtual classroom education delivery 93, data analysis processing 94, transaction processing 95, and media processing and classification 96.

[0076] Various embodiments of the present invention are described herein with reference to the accompanying drawings. Alternative embodiments of the present invention can be devised without departing from the scope of the present invention. Various connections and positional relationships (e.g., on, under, adjacent to, etc.) are shown between elements in the following description and drawings. These connections or positional relationships, or both, can be direct or indirect, unless otherwise specified, and the present invention is not intended to be limited in this regard. Thus, the coupling of entities can refer to either direct or indirect coupling, and the positional relationship between entities can be a direct or indirect positional relationship. Further, the various tasks and process steps described herein can be incorporated into more comprehensive procedures or processes having additional steps or functions not detailed herein.

[0077] One or more of the methods described herein can each be implemented in any one or combination of the following techniques well known in the art: discrete logic circuits having logic gates for implementing logical functions by data signals, application specific integrated circuits (ASICs) having appropriate combinational logic gates, programmable gate arrays (PGAs), field programmable gate arrays (FPGAs), and the like.

[0078] For the sake of brevity, the prior art related to the creation and use of aspects of the present invention may or may not be described in detail herein. In particular, various aspects of computing systems and specific computer programs for implementing the various technical features described herein are well known. Thus, for the sake of brevity, many details of conventional implementations are only briefly described herein, or are completely omitted without providing details of well-known systems or processes or both.

[0079] In some embodiments, various functions or operations can be performed at a given location, or in relation to the operation of one or more devices or systems, or both. In some embodiments, a portion of a given function or operation can be performed at a first device or location, and the remaining portion of the function or operation can be performed at one or more additional devices or locations.

[0080] The terms used herein are for the purpose of describing particular embodiments only and are not intended to be limiting of the invention. As used herein, the singular forms "a", "an", and "the" are intended to include the plural forms as well, unless the context clearly indicates otherwise. Further, the terms "comprise", "comprising", or both as used herein, when used in this specification, specify the presence of the stated features, integers, steps, operations, elements, or components or combinations thereof, but do not preclude the presence or addition of one or more other features, integers, steps, operations, elements, components, or groups or combinations thereof.

[0081] All means or steps-plus-function elements in the following claims corresponding structures, materials, acts, and equivalents are intended to include any structure, material, or act for performing the functions in combination with other claimed elements specifically recited for performing the functions. The present disclosure has been presented for purposes of illustration and description, but is not intended to be exhaustive or limited to the forms disclosed. Many modifications and variations will be apparent to those of ordinary skill in the art without departing from the scope and spirit of the disclosure. Embodiments were chosen and described in order to best explain the principles of the disclosure and the practical application, and to enable others of ordinary skill in the art to understand the disclosure for various embodiments with various modifications as are suited to the particular use contemplated.

[0082] The figures shown in this specification are exemplary. Without departing from the spirit of the present invention, many modifications are possible to the figures or the steps (or operations) described therein. For example, actions can be performed in a different order, or actions can be added, deleted, or modified. Also, the term "coupled" does not mean a direct connection having no intervening elements / connections between elements, but rather describes that there is a signal path between two elements. All of these modifications are considered to be part of this disclosure.

[0083] The following definitions and abbreviations are used to interpret the claims and the specification. As used herein, the terms "comprise", "comprising", "include", "including", "have", "having", "contain", or "containing", or any other variation, are intended to cover non-exclusive inclusion. For example, a composition, mixture, process, method, article, or apparatus that includes a list of elements is not necessarily limited to those elements, but may include other elements not expressly listed, or other elements inherent to such composition, mixture, process, method, article, or apparatus.

[0084] Furthermore, the term "exemplary" is used herein to mean "functioning as an example, instance, or illustration". Any embodiment or design described as "exemplary" herein should not necessarily be construed as preferred or advantageous over other embodiments or designs. The terms "at least one" and "one or more" can be understood to include any integer greater than or equal to 1, i.e., 1, 2, 3, 4, etc. The term "a plurality" can be understood to include any integer greater than or equal to 2, i.e., 2, 3, 4, 5, etc. The term "connected" can include both indirect "connection" and direct "connection".

[0085] The terms "about," "substantially," "approximately," and variations thereof are intended to include the degree of error associated with the measurement of a particular quantity based upon the available equipment at the time of filing. For example, "about" can include a range of ±8% or 5% or 2% of a given value.

[0086] The present invention may be integrated at any possible technical detail level in a system, method, computer program product, or combination thereof. The computer program product can include a computer-readable storage medium (s) having computer-readable program instructions for causing a processor to execute aspects of the present invention.

[0087] A computer-readable storage medium can be a tangible device that holds and stores instructions for use by an instruction execution device. The computer-readable storage medium can be, for example, but is not limited to, an electronic storage device, a magnetic storage device, an optical storage device, an electromagnetic storage device, a semiconductor storage device, or any suitable combination of the foregoing. A non-exhaustive list of more specific examples of computer-readable storage media includes the following: portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), static random access memory (SRAM), portable compact disk read-only memory (CD-ROM), digital versatile disks (DVDs), memory sticks, floppy disks, punch cards, or mechanically encoded devices such as raised structures in grooves in which instructions are recorded, and any suitable combination of the foregoing. As used herein, a computer-readable storage medium is not construed as being a transient signal per se, such as a radio wave, or other freely propagating electromagnetic wave, an electromagnetic wave propagating through a waveguide or other transmission medium (e.g., an optical pulse through an optical fiber cable), or an electrical signal transmitted through a wire.

[0088] The computer-readable program instructions described herein can be downloaded from a computer-readable storage medium to respective computing / processing devices, or to an external computer or external storage device via a network, such as, for example, the Internet, a local area network, a wide area network, or a wireless network, or combinations thereof. The network can include copper transmission cables, optical transmission fibers, wireless transmission, routers, firewalls, switches, gateway computers or edge servers, or combinations thereof. A network adapter card or network interface in each computing / processing device receives the computer-readable program instructions from the network and transfers the computer-readable program instructions for storage in a computer-readable storage medium within each computing / processing device.

[0089] The computer-readable program instructions for carrying out the operations of the present invention may be source code or object code described in any combination of one or more programming languages, including assembler instructions, instruction set architecture (ISA) instructions, machine instructions, machine-dependent instructions, microcode, firmware instructions, state-setting data, configuration data for integrated circuits, or object-oriented programming languages such as Smalltalk, C++, and conventional procedural programming languages such as the "C" programming language or similar programming languages. The computer-readable program instructions may execute entirely on the user's computer, partly on the user's computer, as a stand-alone software package, partly on the user's computer and partly on a remote computer, or entirely on the remote computer or server. In the last scenario, the remote computer may be connected to the user's computer through any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computer (e.g., through the Internet using an Internet service provider). In some embodiments, for example, an electronic circuit including a programmable logic circuit, a field programmable gate array (FPGA), or a programmable logic array (PLA) may execute the computer-readable program instructions by utilizing the state information of the computer-readable program instructions to implement aspects of the present invention and to customize the electronic circuit.

[0090] Aspects of the present invention are described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the invention. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer-readable program instructions.

[0091] These computer-readable program instructions can be provided to a processor of a general purpose computer, special purpose computer, or other programmable data processing apparatus to produce a machine, such that the instructions executed by the processor of the computer or other programmable data processing apparatus create means for implementing the functions / operations specified in one or more blocks of a flowchart, a block diagram, or both. These computer program instructions can also be stored in a computer-readable medium, such that the instructions stored in the computer-readable medium include a product comprising instructions for implementing the aspects of the functions / operations specified in one or more blocks of a flowchart, a block diagram, or both. These computer program instructions can also be loaded onto a computer, other programmable data processing apparatus, or other device to cause a series of operational steps to be performed on the computer, other programmable data processing apparatus, or other device to produce a computer-implemented process, such that the instructions executed on the computer or other programmable apparatus provide a process for implementing the functions / operations specified in one or more blocks of a flowchart, a block diagram, or both.

[0092]

[0093] ​The flowcharts and block diagrams in the drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of the present invention. In this regard, each block in the flowchart may represent a module, segment, or portion of code that includes one or more executable instructions for implementing the specified logical function. In some alternative implementations, the functions noted in the blocks may occur out of the order noted in the figures. For example, two blocks shown in succession may, in fact, be executed substantially concurrently depending on the functionality involved, or these blocks may sometimes be executed in the reverse order. It should also be noted that each block of the block diagrams or flowchart diagrams, or combinations of blocks in the block diagrams or flowchart diagrams or both, can be implemented by a dedicated hardware-based system that performs the specified function or operation, or a combination of dedicated hardware and computer instructions.

[0094] The descriptions of the various embodiments of the present disclosure have been presented for purposes of illustration, but are not intended to be exhaustive or limited to the disclosed embodiments. Many modifications and variations will be apparent to those of ordinary skill in the art without departing from the scope of the described embodiments. The terms used herein were chosen in order to best explain the principles of the embodiments, the practical application, or a technical improvement over technologies found in the marketplace, or to enable those of ordinary skill in the art to understand the embodiments disclosed herein.

Claims

1. A method, executed by computer information processing, for power saving and encryption when analyzing media captured by an information capture device using a partitioned neural network, comprising: the information capture device replicating an artificial neural network (ANN) from a computer server to the information capture device, both the ANN on the computer server and the replicated ANN on the information capture device including M layers; in response to the captured data being an input to be processed, the information capture device selecting the first k layers for execution on the information capture device and transmitting the value of k to the computer server; in response to the captured data being an input to be processed, the information capture device partially processing the captured data by executing the first k layers using the replicated ANN, wherein only the first k layers are selected for execution on the information capture device; the information capture device transmitting the output of the k-th layer to the computer server, and the computer server partially processing the captured data by executing the remaining of the M layers using the ANN and the output of the k-th layer; A method comprising the above steps.

2. The method according to claim 1, further comprising the information capture device receiving the result of the ANN from the computer server.

3. The method according to claim 1, wherein the ANN is trained by the computer server before being replicated onto the information capture device.

4. The method according to claim 1, wherein the captured data includes analog signals captured by an information sensing array.

5. The method according to claim 1, wherein the captured data includes digitized media.

6. Copying the ANN to the information capture device includes copying one or more weights of each of the M layers of the ANN to corresponding M layers of the copied ANN, the method of claim 1.

7. The method of claim 6, wherein the one or more weights are encrypted before being transmitted to the information capture device.

8. The method of claim 1, further comprising encrypting the output of the k-th layer before transmitting the output to the computer server.

9. The method of claim 1, further comprising encrypting the value of k before transmitting to the computer server.

10. A system comprising: a memory; one or more processors coupled to the memory wherein the one or more processors are configured to perform the method according to any one of claims 1 to 9 for power saving and encryption when analyzing media captured by an information capture device using a partitioned neural network.

11. A computer program executable by one or more processors, the one or more processors being caused to perform the method according to any one of claims 1 to 9 for power saving and encryption during analysis of media captured by an information capture device using a partitioned neural network.

12. A computer-readable storage medium storing the computer program of claim 11.

Images