Media capture device with power saving and encryption features for partitioned neural networks

Abstract

A method for conserving power and encryption during analysis of media captured by an information capture device using a partitioned neural network includes copying, by an information capture device, an artificial neural network (ANN) from a computer server to the information capture device. Both the ANN on the computer server and the copied ANN include M layers. The method further includes, in response to captured data to be processed being input, partially processing, by the information capture device, the captured data by using the copied ANN to execute the first k layers, where only the k layers are selected to be executed on the information capture device. The method further includes sending, by the information capture device, an output of the kth layer to the computer server, the computer server partially processing the captured data by using the ANN and the output of the kth layer to execute the remaining of the M layers.

Description

Background Technology

[0001] This invention relates generally to computing technology, and more specifically to media capture devices and neural networks that facilitate power saving and ensure data security in media capture devices.

[0002] Today, various devices such as telephones, tablets, and wearable devices capture and / or create media objects, such as digital images, audio, and video. With the increasing need to classify the vast amounts of captured and / or extracted media, learning models have become a common practice for classifying captured media objects. Learning models (such as, for example, artificial neural networks (ANNs) and / or convolutional neural networks (CNNs)) are trained with sample data; that is, they sample media objects and continuously evolve (learn) during the process of classifying new (previously unseen) media objects. Summary of the Invention

[0003] One or more embodiments of the present invention include a computer-implemented method for saving power and encryption during analysis of media captured by an information capture device using a partitioned neural network. The method includes copying an artificial neural network (ANN) from a computer server to the information capture device, wherein both the ANN on the computer server and the copied ANN on the information capture device comprise M layers. The method further includes, in response to input of captured data to be processed, partially processing the captured data by the information capture device by executing the first k layers using the copied ANN, wherein only the k layers are selected to be executed on the information capture device. The method further includes sending the output of the kth layer to the computer server via the information capture device, the computer server partially processing the captured data by executing the remaining portions of the M layers using the ANN and the output of the kth layer.

[0004] According to one or more embodiments of the present invention, a system includes a memory and one or more processors coupled to the memory, wherein the one or more processors perform a method for saving power and encryption during analysis of media captured by an information capture device using a partitioned neural network. The method includes copying an artificial neural network (ANN) from a computer server to the information capture device, wherein both the ANN on the computer server and the copied ANN on the information capture device comprise M layers. The method further includes, in response to the input of captured data to be processed, partially processing the captured data by the information capture device by executing the first k layers using the copied ANN, wherein only the k layers are selected to be executed on the information capture device. The method further includes sending the output of the kth layer to the computer server via the information capture device, the computer server partially processing the captured data by executing the remaining portions of the M layers using the ANN and the output of the kth layer.

[0005] According to one or more embodiments of the present invention, a computer program product includes a computer-readable storage medium having program instructions embodied therein. The program instructions are executable by one or more processors to cause the one or more processors to perform operations including power saving and encryption during analysis of media captured by an information capture device using a partitioned neural network. The method includes copying an artificial neural network (ANN) from a computer server to the information capture device, wherein both the ANN on the computer server and the copied ANN on the information capture device comprise M layers. The method further includes, in response to the input of captured data to be processed, partially processing the captured data by the information capture device by executing the first k layers using the copied ANN, wherein only the k layers are selected to be executed on the information capture device. The method further includes sending the output of the kth layer to the computer server via the information capture device, wherein the computer server partially processes the captured data by executing the remaining portions of the M layers using the ANN and the output of the kth layer.

[0006] Other embodiments of the present invention implement the features of the above-described method in computer systems and computer program products.

[0007] Additional technical features and advantages are achieved through the technology of this invention. Embodiments and aspects of the invention are described in detail herein and are considered part of the claimed subject matter. For a better understanding, refer to the detailed description and accompanying drawings. Attached Figure Description

[0008] The details of the patent rights described herein are specifically pointed out and explicitly claimed in the claims at the end of the specification. The foregoing and other features and advantages of embodiments of the invention will become clear from the following detailed description taken in conjunction with the accompanying drawings, in which:

[0009] Figure 1 This is a block diagram of a system for saving power and encrypting media captured by an information capture device during analysis using a partitioned neural network, according to one or more embodiments of the present invention.

[0010] Figure 2 This is a block diagram of a system for saving power and encrypting media captured by an information capture device during analysis using a partitioned neural network according to one or more embodiments of the present invention.

[0011] Figure 3 This is a flowchart of a method for saving power and encrypting media captured by an information capture device during analysis using a partitioned neural network, according to one or more embodiments of the present invention.

[0012] Figure 4 A computer system implementing one or more embodiments of the present invention is described;

[0013] Figure 5 A cloud computing environment according to one or more embodiments of the present invention is described; and

[0014] Figure 6 An abstract model layer according to one or more embodiments of the present invention is shown. Detailed Implementation

[0015] Embodiments of the present invention facilitate power-saving and encryption features in media capture devices when using partitioned neural networks to process one or more media objects (such as images, audio, video, etc.). Currently, large amounts of data (such as images, audio, video, etc.) are created by multiple users using edge devices (such as phones, tablets, computers, wearable devices, dashcams, recorders, security cameras, etc.). There are technical challenges in using deep neural network (DNN) architectures to process such large amounts of data, including media. Here, "large amounts" may require millions of images, audio, and video files, and manually processing and classifying such a volume of data is, if not impossible, impractical. Therefore, embodiments of the present invention provide a practical application for classifying large amounts of media captured by one or more information capture devices. Furthermore, embodiments of the present invention improve the operation of information capture devices by promoting power saving and ensuring data security. Additionally, embodiments of the present invention address the limited computing resources on information capture devices by improving the computational efficiency of the information capture devices during such media classification tasks.

[0016] Figure 1 A block diagram of a system 100 for processing information captured by one or more information capturing devices according to one or more embodiments of the present invention is described. Information capturing device 102 (e.g., camera, telephone, security camera, tablet computer, recorder, etc.) captures information as analog signals that can be stored in one or more digital files 103. The analog signals sensed by the information sensing array 112 of information capturing device 102 are digitized by an analog-to-digital converter (ADC) module 114. Information capturing device 102 may further include a processor 116 capable of performing one or more digital signal processing operations (e.g., image processing, audio processing, video processing, etc.) on the digitized data. Processor 116 can save the digitized data as a digital file 103. The digitized file 103 can be saved as an electronic file using one or more digital file storage formats. For example, visual information captured by information sensing array 112 can be stored using image file formats such as Portable Network Graphics (PNG), Bitmap (BMP), etc. In the case of audio data sensed by information sensing array 112, the digitized audio can be stored using file formats such as Waveform Audio File Format (WAV), Free Lossless Audio Codec (FLAC), etc. When video is captured, data can be stored using digital file formats such as Video Object (VOB), Audio Video Interleaving (AVI), MPEG-14, etc.

[0017] In one or more embodiments of the present invention, processor 116 may include one or more processing units, such as processor cores. Processor 116 may be a microprocessor, multiprocessor, digital signal processor, graphics programming unit, central processing unit, and other processing units of this type or combinations thereof. Processor 116 may include or be coupled to memory device 117. Processor 116 may perform one or more operations by executing one or more computer-executable instructions. Such instructions may be stored on memory device 117. Memory device 117 may store additional information / data that may be used or output by processor 116.

[0018] Data captured from digitized document 103 is transmitted via communication network 104 to computer server 106 for further processing, such as the classification of digitized document 103.

[0019] The communication network 104 may be a computer network using one or more communication protocols (such as Ethernet), such as the Internet. In one or more embodiments of the invention, the digitized document 103 is captured by the user 101 using the information capture device 102.

[0020] Computer server 106 may be a server cluster or a distributed server, providing cloud-based processing services for the digitized documents 103 captured by information capture device 102. In one or more embodiments of the invention, computer server 106 includes an artificial neural network (ANN) 122. ANN 122 may be a convolutional neural network, a feedforward network, a recurrent neural network, a multilayer perceptron, or a combination thereof. In one or more embodiments of the invention, ANN 122 may be a stand-alone hardware module. Alternatively or additionally, ANN 122 may be implemented using the processor 127 of computer server 106. ANN 122 includes multiple layers, the output of which is used by subsequent layers until the final output 123 is generated.

[0021] In one or more embodiments of the present invention, processor 126 may include one or more processing units, such as processor cores. Processor 126 may be a microprocessor, multiprocessor, digital signal processor, graphics programming unit, central processing unit, and other processing units of this type, or combinations thereof. Processor 126 may include or be coupled to memory device 127. Processor 126 may perform one or more operations by executing one or more computer-executable instructions. These instructions may be stored on memory device 127. Memory device 127 may store additional information / data that may be used or output by processor 126.

[0022] In one or more embodiments of the present invention, ANN 122 is trained using training data 124. Training data 124 includes predefined media, such as images, audio, video, etc., and may include labels and other cues that can train ANN 122 to analyze captured data from information capture device 102 during the inference phase and generate ANN output 123. ANN output 123 may include classification of digitized document 103 into one or more categories, object detection results of digitized document 103, and other such image processing / computer vision and audio processing results.

[0023] In a conventional system, the digitized file 103 is encrypted before being sent to the server 106. If the encryption is compromised (i.e., hacked), data captured from the digitized file 103 may be exposed.

[0024] Embodiments of the present invention integrate neural network analysis with encryption by segmenting ANN122 and creating a copy of ANN122 on information capture device 102. In one or more embodiments, the captured data is first processed through one or more layers of ANN122, and then the output of ANN122 is sent to computer server 106 via network 104 for further processing together with the remaining layers of ANN122.

[0025] Alternatively, in other embodiments, the analog signals sensed by the information sensing array 112 on the information capturing device 102 are first connected through one or more layers of the ANN 122. The outputs of one or more layers of the ANN 122 are sent via network 104 to computer server 106 for further processing together with the remaining layers of the ANN 122.

[0026] In embodiments of the present invention, the weights of one or more intermediate layers of ANN122 and the outputs of the intermediate layers of ANN122 can also be encrypted. In this way, the captured data is not transmitted through network 104, thereby improving the security of the captured data. Embodiments of the present invention provide improvements to system 100 and its components (such as information capture device 102, computer server 106), as well as improvements to one or more methods (e.g.) using system 100 and / or its components to securely analyze digitized documents 103 captured by information capture device 102.

[0027] Figure 2This is an improved block diagram depicting one or more components of a system 100 for power saving and encryption of captured data using a partitioned neural network according to one or more embodiments of the present invention. The depiction illustrates layer 202 in ANN 122. ANN 122 may include M layers, where M is any integer. Each layer uses the output from previous layers except for layer #1.

[0028] ANN122 is trained using training data 124. Such training involves learning (i.e., configuring, building) one or more weights associated with each of the layers 202 of ANN122. The weights are learned automatically using one or more training techniques, such as supervised learning, unsupervised learning, or any other learning technique for ANN122.

[0029] Information capture device 102 includes an ANN copy 204, which is a copy of ANN 122. ANN copy 204 is identical to ANN 122 and includes the same M layers. Furthermore, to make ANN copy 204 identical to ANN 122, the weights learned by ANN 122 are transferred to ANN copy 204 on information capture device 102. In one or more embodiments of the invention, the weights are encrypted by encryption unit 230 of computer server 106. Decryption unit 232 of information capture device 102 decrypts the encrypted weights from encryption unit 230. The decrypted weights output by decryption unit 232 are configured in ANN copy 204.

[0030] The information capture device 102 also includes a layer selector 210, which selects how many layers out of the M layers from the ANN copy 204 to be executed on the information capture device for analyzing the digitized document 103 created by the information capture device 102. For example, the layer selector 210 may select the first k layers (1 ≤ k ≤ M) of the ANN copy 204 to be executed by the information capture device 102 with the digitized document 103 as input. In one or more embodiments of the invention, the layer selector 210 determines the value of k based on the power consumed by the information capture device 102 in executing the layers of the ANN copy 204. In other embodiments, additional or alternative parameters may be used to select the value of k.

[0031] The output of layer #k from ANN copy 204 (with digitized file 103 as input to copy ANN 204) is sent to computer server 106. In one or more embodiments of the invention, the output of layer #k is encrypted by encryption unit 220 of information capture device 102 before transmission. Decryption unit 222 of computer server 106 decrypts the output of layer #k. The received output of layer #k is input to layer #(k+1) of ANN 122. In one or more embodiments of the invention, layer locator 212 of computer server 104 identifies layer #(k+1) in ANN 122 and inputs the received output of layer #k into layer #(k+1) in ANN 122.

[0032] In one or more embodiments of the present invention, the layer selector 210 sends an identifier of the layer being transmitted (i.e., layer #k) to the layer locator 212. In one or more embodiments of the present invention, the identifier of layer #k is encrypted by the encryption unit 220 before transmission. The decryption unit 222 decrypts the identifier of layer #k for use by the layer locator 212.

[0033] The layers (k+1) through M of ANN122 are then executed to generate result 123 of ANN122. In one or more embodiments of the invention, result 123 is sent to information capturing device 102 or any other device (not shown).

[0034] Therefore, system 100 facilitates variable workload partitioning, where a subset of the ANN layers is executed on information capture device 102, while the remaining layers are executed on computer server 104. Furthermore, the data exchanged between information capture device 102 and computer server 104 is secure, and even subsequently only intermediate data is exchanged to limit the exposure of the entire digitized document 103, and consequently limit the possibility of the digitized document 103 being hacked during such data exchange.

[0035] In one or more embodiments of the present invention, the information capture device 102 sends the output of each layer (i.e., layer 1-k) performed by the ANN copy 204 and the identifier of layer #k.

[0036] In one or more embodiments of the invention, ANN copy 204 uses analog signals captured by information sensing array 112 before converting the captured data into digital file 103. This helps to further protect the captured data from misuse. In this case, ANN 122 is trained using training data 124 that includes analog signals.

[0037] Figure 3A flowchart depicts a method 300 for analyzing captured data using a partitioned neural network in a power-saving and encrypted manner according to one or more embodiments of the present invention. Method 300 includes training an ANN 122 of a computer server 106 using training data 124 at block 302. Training may include supervised learning, unsupervised learning, or any other type of neural network training. Training data 124 may include analog signals captured by an information sensing array (such as information sensing array 112). Alternatively or additionally, training data 124 may include media obtained after digitizing such analog signals. Training facilitates the configuration of weights into M layers 202 of the ANN 122. Here, "weights" are parameters within the ANN 122 whose transformations are provided to the input data of any of the M layers 202. Each of the M layers 202 may include multiple weights. The ANN 122 is trained to analyze the captured data in the form of analog signals captured by the information sensing array 112 or in the form of a digitized document 103. For example, such analysis may include detecting and identifying objects in the captured data. Furthermore, the analysis may include classifying identified objects and / or captured data into one or more categories. In one or more embodiments of the invention, other types of analysis may be performed additionally or alternatively.

[0038] Further, at box 304, ANN 122 is replicated on information capture device 102. Replication includes configuring ANN copy 204 on information capture device 102. ANN copy 204 is configured with the same number of layers, i.e., M. Further, each layer of ANN copy 204 is configured with the exact same weights as the M layers 202 of ANN 122 on computer server 106. In one or more embodiments of the invention, this replication includes encrypting the trained weights using encryption unit 230 and sending the encrypted values ​​to information capture device 102. Decryption unit 232 decrypts the weight values, which are then used to configure ANN copy 204.

[0039] Subsequently, at box 306, the information capture device 102 uses the information sensing array 112 to capture analog signal data. At box 308, the captured data is input to the ANN copy 204 of the information capture device 102 for processing using only k layers out of the M layers of the ANN copy 204. The captured data input to the ANN copy 204 can be an analog signal captured by the information sensing array 112 or a corresponding digitized file 103.

[0040] Processing the captured data includes selecting, at box 310, the number (i.e., k) of layers to be executed by the information capture device 102. Layer selector 210 determines the value of k based on one or more factors associated with the information capture device. In one or more embodiments of the invention, layer selector 210 monitors the power consumption of each layer executing the ANN copy 204. Alternatively or additionally, layer selector 210 accesses power consumption data indicating the power required for each of the layers of the ANN copy 204 to be executed by the information capture device 102. In one or more embodiments of the invention, layer selector 210 may also include a power consumption budget for the ANN copy 204. The power consumption budget may be a configurable value.

[0041] The power consumption budget indicates the maximum amount of power that ANN copy 204 can consume to analyze the captured data. In one or more embodiments of the invention, the power consumption budget can be a value that depends on the total amount of power available to the information capture device 102. For example, if the information capture device 102 is receiving power from a battery or any other such limited power source (not shown), the available power can depend on the charge level of the power source. As the charge level changes, the power consumption budget can change. For example, if the charge level is at least 75% of the power source's capacity, the power consumption budget for ANN copy 204 can be 100 milliwatts to analyze the captured data; when the charge level drops to 50%, the power consumption budget decreases to 80 milliwatts; when the charge level drops to 30%, the power consumption budget further decreases to 50 milliwatts, and so on. It should be understood that the above example values ​​can vary in one or more embodiments of the invention. In one or more embodiments of the invention, the relationship between the power consumption budget and the charge level can be configurable.

[0042] Therefore, based on the determined power consumption budget and the power required for each layer in ANN copy 204, the layer selector determines that the information capture device 102 can execute k layers without exceeding the power consumption budget. In response, the first k layers of ANN copy 204 are executed by the information capture device 102 (at box 308).

[0043] Because ANN replica 204 comprises exact copies of M layers 202 of ANN 122, the remaining layers (k+1) through M of ANN 122 can take over the analysis of the captured data. To this end, at box 312, the output of layer #k from ANN replica 204 is sent via network 104 to computer server 106. The transmission may also include an identifier for layer k, such as the value of k.

[0044] In one or more embodiments of the invention, the transmission is encrypted by encryption unit 220. In one or more embodiments of the invention, the output of layer #k and the identifier of k may be part of a single encrypted transmission. Alternatively, separate encrypted transmissions may be performed for the output of layer #k and the identifier of k.

[0045] At box 314, ANN 122 analyzes the captured data by using the output of layer #k to execute layers (k+1) through M. This analysis includes decrypting the information received from the information capture device 102 by the decryption unit 222. Further, layer locator 212 identifies layers #k and #k+1 of ANN 122 and configures these layers with the information from the information capture device 102, so that ANN 122 can access the remaining M layers starting from layer #k+1.

[0046] In box 316, the processing result of ANN122 is output. In one or more embodiments of the invention, the result may be sent to information capturing device 102. Alternatively, or additionally, the result may be sent to another device, such as another computer server, database, or any other device.

[0047] It should be noted that, although Figure 1 and 2 A single information capture device 102 is depicted, but in one or more embodiments of the invention, multiple information capture devices 102 may communicate with a computer server 106. Furthermore, each information capture device 102 may have its own power consumption budget, charge level, and other such varying factors. Therefore, the number of layers executed on the first information capture device 102 may differ from the number of layers executed on the second information capture device, for example, k' (k≠k'). In response, for the first information capture device, the computer server 106 executes a different number of layers (Mk) compared to the number of layers (M-k') executed for the second information capture device.

[0048] Furthermore, even for a single data capture device 102, the number of layers k can be varied based on the charge level. For example, the computer server 106 can perform an ANN 122 of (Mk) layers for the first capture data captured by the data capture device 102 at time t1 when the charge level is X%; however, when the charge level is Y%, the computer server 106 performs an ANN 122 of (Mp) layers for the second capture data captured by the data capture device 102 at time t1, where p is the number of layers selected by the layer selector 210.

[0049] Various embodiments of the present invention integrate neural network processes with encryption by partitioning the neural network and creating a copy of the neural network on an information capture device. A subset of the neural network layers is used at the information capture device to analyze the captured data, and the output of this processing is sent to a computer server for further processing using the remaining layers of the neural network. The number of layers executed at the information capture device is based on one or more factors at the information capture device, such as power consumption. The captured data can be used in the form of analog signals or digital files. Furthermore, all transmissions (such as those used to replicate the weights of the neural network, the outputs of the layers executed on the information capture device, etc.) are encrypted. In this way, the captured data is not transmitted directly over the network, thereby increasing the security of the captured data.

[0050] Now go to Figure 4 The computer system 400 is generally illustrated according to embodiments. In one or more embodiments of the invention, the computer system 400 may serve as an information capture device 102 and / or a computer server 106. The computer system 400 may be an electronic computer framework that includes and / or employs any number and combination of computing devices and networks utilizing various communication technologies, as described herein. The computer system 400 may be easily scalable, extensible, and modular, with the ability to be changed to different services or reconfigured to some features independently of others. The computer system 400 may be, for example, a server, desktop computer, laptop computer, tablet computer, or smartphone. In some examples, the computer system 400 may be a cloud computing node. The computer system 400 may be described in the general context of computer system executable instructions (such as program modules) executed by the computer system. Generally, program modules may include routines, programs, objects, components, logic, data structures, etc., that perform specific tasks or implement specific abstract data types. The computer system 400 may be practiced in a distributed cloud computing environment, where tasks are performed by remote processing devices linked via a communication network. In a distributed cloud computing environment, program modules can reside in local and remote computer system storage media, including memory storage devices.

[0051] like Figure 4As shown, the computer system 400 has one or more central processing units (CPUs) 401a, 401b, 401c, etc. (collectively or generally referred to as processor 401). Processor 401 can be a single-core processor, a multi-core processor, a computing cluster, or any number of other configurations. Processor 401 (also referred to as processing circuitry) is coupled to system memory 403 and various other components via system bus 402. System memory 403 may include read-only memory (ROM) 404 and random access memory (RAM) 405. ROM 404 is coupled to system bus 402 and may include a basic input / output system (BIOS) that controls certain basic functions of computer system 400. RAM is a read-write memory coupled to system bus 402 for use by processor 401. System memory 403 provides temporary memory space for the operation of instructions during operation. System memory 403 may include random access memory (RAM), read-only memory, flash memory, or any other suitable memory system.

[0052] Computer system 400 includes an input / output (I / O) adapter 406 and a communication adapter 407 coupled to a system bus 402. I / O adapter 406 may be a Small Computer System Interface (SCSI) adapter that communicates with a hard disk 408 and / or any other similar component. I / O adapter 406 and hard disk 408 are collectively referred to herein as mass storage device 410.

[0053] Software 411 executing on computer system 400 may be stored in mass storage device 410. Mass storage device 410 is an example of a tangible storage medium readable by processor 401, wherein software 411 is stored as instructions for execution by processor 401 to operate computer system 400, such as those described below with respect to the various figures. Examples of computer program products and the execution of these instructions are discussed in more detail herein. Communication adapter 407 interconnects system bus 402 with network 412, which may be an external network enabling computer system 400 to communicate with other such systems. In one embodiment, mass storage device 410 and a portion of system memory 403 jointly store an operating system, which can be any suitable operating system, such as z / OS or AIX from IBM, for coordination. Figure 4 The functions of the different components shown.

[0054] Additional input / output devices are shown connected to the system bus 402 via display adapter 415 and interface adapter 416. In one embodiment, adapters 406, 407, 415, and 416 may be connected to one or more I / O buses connected to the system bus 402 via an intermediate bus bridge (not shown). A display 419 (e.g., a screen or display monitor) is connected to the system bus 402 via display adapter 415, which may include a graphics controller and a video controller for improving performance in graphics-intensive applications. Keyboard 421, mouse 422, speakers 423, etc., may be interconnected to the system bus 402 via interface adapter 416, which may include, for example, a super I / O chip integrating multiple device adapters into a single integrated circuit. Suitable I / O buses for connecting peripheral devices such as hard disk controllers, network adapters, and graphics adapters typically include common protocols such as Peripheral Component Interconnect (PCI). Therefore, as Figure 4 The computer system 400 configured therein includes processing capabilities in the form of a processor 401, storage capabilities including system memory 403 and mass storage 410, input devices such as a keyboard 421 and a mouse 422, and output capabilities including a speaker 423 and a display 419.

[0055] In some embodiments, the communication adapter 407 may use any suitable interface or protocol (such as an Internet Small Computer System Interface) to transmit data. The network 412 may be a cellular network, radio network, wide area network (WAN), local area network (LAN), or the Internet. External computing devices may be connected to the computer system 400 via the network 412. In some examples, the external computing device may be an external web server or a cloud computing node.

[0056] It should be understood that Figure 4 The block diagram is not intended to indicate the computer system 400 including Figure 4 All components shown. Conversely, computer system 400 may include... Figure 4 Any suitable fewer or additional components not shown herein (e.g., additional memory components, embedded controllers, modules, additional network interfaces, etc.). Furthermore, the embodiments described herein with respect to computer system 400 can be implemented with any suitable logic, wherein in different embodiments, the logic as mentioned herein may include any suitable hardware (e.g., processor, embedded controller, or application-specific integrated circuit, etc.), software (e.g., applications, etc.), firmware, or any suitable combination of hardware, software, and firmware.

[0057] It should be understood that while this disclosure includes a detailed description of cloud computing, the implementation of the teachings cited herein is not limited to cloud computing environments. Rather, embodiments of the invention can be implemented in conjunction with any other type of computing environment now known or developed hereafter.

[0058] Cloud computing is a service delivery model that enables convenient, on-demand network access to a shared pool of configurable computing resources (e.g., networks, network bandwidth, servers, processing, memory, storage, applications, virtual machines, and services), which can be rapidly provisioned and released with minimal management effort or interaction with the service provider. This cloud model may include at least five features, at least three service models, and at least four deployment models.

[0059] The features are as follows:

[0060] On-demand self-service: Cloud consumers can unilaterally and automatically provide computing power, such as server time and network storage, as needed, without requiring human interaction with the service provider.

[0061] Extensive network access: Capabilities are available through networks and accessed via standard mechanisms that facilitate the use of heterogeneous thin client or thick client platforms (e.g., mobile phones, laptops, and PDAs).

[0062] Resource pooling: A provider's computing resources are pooled to serve multiple consumers using a multi-tenant model, where different physical and virtual resources are dynamically assigned and reassigned as needed. There is a sense of location independence because consumers typically do not have control or knowledge of the exact location of the resources provided, but may be able to specify the location at a higher level of abstraction (e.g., country, state, or data center).

[0063] Rapid flexibility: The ability to provide capacity quickly and flexibly, automatically scaling down and up rapidly in some situations to scale up rapidly. For consumers, the available supply capacity often appears unlimited and can be purchased in any quantity at any time.

[0064] Measuring services: Cloud systems automatically control and optimize resource usage by leveraging metering capabilities at a level of abstraction appropriate to the service type (e.g., storage, processing, bandwidth, and active user accounts). Resource usage can be monitored, controlled, and reported, providing transparency to both service providers and consumers.

[0065] The service model is as follows:

[0066] Software as a Service (SaaS): This provides consumers with the ability to use the provider's applications running on cloud infrastructure. Applications can be accessed from different client devices via thin client interfaces such as web browsers (e.g., web-based email). Consumers do not manage or control the underlying cloud infrastructure, including the network, servers, operating system, storage, or even individual application capabilities, with possible exceptions such as limited user-specific application configuration settings.

[0067] Platform as a Service (PaaS): This provides consumers with the ability to deploy applications created or acquired by the consumer using programming languages ​​and tools supported by the provider onto cloud infrastructure. Consumers do not manage or control the underlying cloud infrastructure, including networks, servers, operating systems, or storage, but they have control over the deployed applications and the configuration of any application hosting environment.

[0068] Infrastructure as a Service (IaaS): The capabilities offered to consumers are processing, storage, networking, and other basic computing resources that enable consumers to deploy and run arbitrary software, which may include operating systems and applications. Consumers do not manage or control the underlying cloud infrastructure, but rather have control over the operating system, storage, deployed applications, and potentially limited control over selected networking components (e.g., host firewalls).

[0069] The deployment model is as follows:

[0070] Private cloud: A cloud infrastructure that operates solely for an organization. It can be managed by the organization or a third party and can exist on-site or off-site.

[0071] Community cloud: A cloud infrastructure shared by several organizations and supporting a specific community with shared concerns (e.g., tasks, security requirements, policies, and compliance considerations). It can be managed by an organization or a third party and can exist on-site or off-site.

[0072] Public cloud: Makes cloud infrastructure available to the public or large industry groups and is owned by an organization that sells cloud services.

[0073] Hybrid cloud: A cloud infrastructure is a combination of two or more clouds (private, community, or public) that remain a single entity but are bound together by standardized or proprietary technologies that enable data and applications to be ported (e.g., cloud bursting for load balancing between clouds).

[0074] Cloud computing environments are service-oriented, focusing on statelessness, loose coupling, modularity, and semantic interoperability. At the heart of cloud computing is the infrastructure comprising a network of interconnected nodes.

[0075] Now for reference Figure 5This describes an illustrative cloud computing environment 50. As shown, the cloud computing environment 50 includes one or more cloud computing nodes 10 to which local computing devices used by cloud consumers can communicate. These local computing devices include, for example, personal digital assistants (PDAs) or cellular phones 54A, desktop computers 54B, laptop computers 54C, and / or automotive computer systems 54N. The nodes 10 can communicate with each other. They can be physically or virtually grouped (not shown) in one or more networks, such as private clouds, community clouds, public clouds, or hybrid clouds, or combinations thereof, as described above. This allows the cloud computing environment 50 to provide infrastructure, platforms, and / or software as services that cloud consumers do not need to maintain on their local computing devices. It should be understood that... Figure 5 The types of computing devices 54A-N shown are intended to be illustrative only, and computing node 10 and cloud computing environment 50 can communicate with any type of computerized device via any type of network and / or network-addressable connection (e.g., using a web browser).

[0076] See now Figure 6 This demonstrates a cloud computing environment of 50 ( Figure 5 This provides a set of functional abstractions. It should be understood beforehand. Figure 6 The components, layers, and functions shown are intended to be illustrative only, and embodiments of the invention are not limited thereto. As described, the following layers and corresponding functions are provided:

[0077] The hardware and software layer 60 includes hardware and software components. Examples of hardware components include a host 61; a server 62 based on a RISC (Reduced Instruction Set Computer) architecture; a server 63; a blade server 64; a storage device 65; and a network and network components 66. In some embodiments, software components include network application server software 67 and database software 68.

[0078] The virtualization layer 70 provides an abstraction layer from which the following examples of virtual entities can be provided: virtual server 71; virtual storage 72; virtual network 73, including virtual private network; virtual application and operating system 74; and virtual client 75.

[0079] In one example, management layer 80 may provide the following functionalities: Resource Provisioning 81 provides dynamic procurement of computing resources and other resources used to perform tasks within the cloud computing environment. Metering and Pricing 82 provides cost tracking as resources are utilized within the cloud computing environment and bills or invoices for the consumption of these resources. In one example, these resources may include application software licenses. Security provides authentication for cloud consumers and tasks, as well as protection for data and other resources. User Portal 83 provides access to the cloud computing environment for consumers and system administrators. Service Level Management 84 provides cloud resource allocation and management to ensure that required service levels are met. Service Level Agreement (SLA) Planning and Fulfillment 85 provides pre-scheduling and procurement of cloud resources based on anticipated future needs according to the SLA.

[0080] The workload layer 90 provides examples of functionalities that can be leveraged in a cloud computing environment. Examples of workloads and functionalities that can be provided from this layer include mapping and navigation 91; software development and lifecycle management 92; virtual classroom education delivery 93; data analytics and processing 94; transaction processing 95; and media processing and classification 96.

[0081] Various embodiments of the invention are described herein with reference to the accompanying drawings. Alternative embodiments of the invention may be devised without departing from its scope. In the following description and drawings, various connections and positional relationships (e.g., above, below, adjacent, etc.) are illustrated between elements. Unless otherwise specified, these connections and / or positional relationships may be direct or indirect, and the invention is limited in this respect by not illustrating the figures. Therefore, the connection of entities may refer to direct or indirect connections, and the positional relationship between entities may be direct or indirect positional relationships. Furthermore, the various tasks and process steps described herein may be incorporated into a more comprehensive procedure or process with additional steps or functions not described in detail herein.

[0082] One or more methods described herein can be implemented using any of the following techniques or combinations thereof, each of which is well known in the art: discrete logic circuits having logic gates for implementing logical functions on data signals, application-specific integrated circuits (ASICs) having appropriately combined logic gates, programmable gate arrays (PGAs), field-programmable gate arrays (FPGAs), etc.

[0083] For the sake of brevity, conventional techniques relating to the manufacture and use of the present invention may or may not be described in detail herein. Specifically, various aspects of the computing systems and specific computer programs used to implement the different technical features described herein are well known. Consequently, for the sake of brevity, many conventional implementation details are only briefly mentioned or omitted entirely herein, without providing well-known system and / or process details.

[0084] In some embodiments, various functions or actions may occur at a given location and / or in conjunction with the operation of one or more devices or systems. In some embodiments, a portion of a given function or action may be performed at a first device or location, and the remainder of the function or action may be performed at one or more additional devices or locations.

[0085] The terminology used herein is for the purpose of describing particular embodiments only and is not intended to be limiting. As used herein, unless the context clearly indicates otherwise, the singular forms “a,” “an,” and “the” are intended to include the plural forms as well. It should also be understood that when the terms “comprises” and / or “comprising” are used in this specification, they specify the presence of the stated features, integrals, steps, operations, elements, and / or components, but do not exclude the presence or addition of one or more other features, integrals, steps, operations, elements, components, and / or combinations thereof.

[0086] All means or steps in the following claims, plus corresponding structures, materials, actions, and equivalents of the functional elements, are intended to include any structure, material, or action for performing the function in conjunction with other claimed elements as specifically claimed. This disclosure has been presented for purposes of illustration and description, but is not intended to be exhaustive or limited to the forms disclosed. Many modifications and variations will be apparent to those skilled in the art without departing from the scope of this disclosure. These embodiments were chosen and described in order to best explain the principles and practical application of this disclosure, and to enable others skilled in the art to understand this disclosure with respect to different embodiments having different modifications suitable for the particular intended use.

[0087] The diagrams depicted herein are illustrative. Many variations may be made to the diagrams or steps (or operations) described herein without departing from the scope of this disclosure. For example, actions may be performed in a different order, or actions may be added, deleted, or modified. Furthermore, the term "coupled" describes a signal path between two elements and does not imply a direct connection between elements without intermediate elements / connections. All such variations are considered part of this disclosure.

[0088] The following definitions and abbreviations are used to interpret the claims and description. As used herein, the terms “comprise,” “comprising,” “includes,” “including,” “has,” “having,” “contains,” or “containing,” or any other variations thereof, are intended to cover a non-exclusive inclusion. For example, a composition, mixture, process, method, article, or apparatus that comprises a list of elements is not necessarily limited to those elements, but may include other elements not expressly listed or inherent to such composition, mixture, process, method, article, or apparatus.

[0089] Furthermore, the term "exemplary" is used herein to mean "used as an example, illustration, or illustration." Any implementation or design described herein as "exemplary" is not necessarily to be construed as superior to or better than other implementations or designs. The terms "at least one" and "one or more" should be understood to include any integer greater than or equal to one, i.e., one, two, three, four, etc. The term "multiple" should be understood to include any integer greater than or equal to two, i.e., two, three, four, five, etc. The term "connection" can include both indirect "connection" and direct "connection."

[0090] The terms “about,” “substantially,” “roughly,” and their variations are intended to include the degree of error associated with a measurement based on a specific quantity of equipment available at the time of application submission. For example, “about” could include a range of ±8%, 5%, or 2% of a given value.

[0091] This invention can be a system, method, and / or computer program product with any possible level of technical detail integration. The computer program product may include a computer-readable storage medium (or media) having computer-readable program instructions thereon for causing a processor to execute aspects of the invention.

[0092] Computer-readable storage media can be a tangible means for retaining and storing instructions for use by an instruction execution device. Computer-readable storage media can be, for example, but not limited to, electronic storage devices, magnetic storage devices, optical storage devices, electromagnetic storage devices, semiconductor storage devices, or any suitable combination of the foregoing. A non-exhaustive list of more specific examples of computer-readable storage media includes: portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), static random access memory (SRAM), portable compact disk read-only memory (CD-ROM), digital universal disk (DVD), memory sticks, floppy disks, mechanical encoding devices such as punch cards or protrusions in slots having instructions recorded thereon, and any suitable combination of the foregoing. As used herein, computer-readable storage media should not be construed as transient signals themselves, such as radio waves or other freely propagating electromagnetic waves, electromagnetic waves propagating through waveguides or other transmission media (e.g., light pulses passing through fiber optic cables), or electrical signals transmitted through wires.

[0093] The computer-readable program instructions described herein can be downloaded from a computer-readable storage medium to a suitable computing / processing device via a network (e.g., the Internet, a local area network, a wide area network, and / or a wireless network), or to an external computer or external storage device. The network may include copper cables, optical fibers, wireless transmissions, routers, firewalls, switches, gateway computers, and / or edge servers. A network adapter card or network interface in each computing / processing device receives the computer-readable program instructions from the network and forwards them to a computer-readable storage medium within the suitable computing / processing device.

[0094] Computer-readable program instructions used to perform the operations of this invention may be assembly instructions, instruction set architecture (ISA) instructions, machine instructions, machine-dependent instructions, microcode, firmware instructions, state setting data, integrated circuit configuration data, or source code or object code written in any combination of one or more programming languages, including object-oriented programming languages ​​(such as Smalltalk, C++, etc.) and procedural programming languages ​​(such as the "C" programming language or similar programming languages). The computer-readable program instructions may be executed entirely on a user's computer, partially on a user's computer, as a standalone software package, partially on a user's computer and partially on a remote computer, or entirely on a remote computer or server. In the latter case, the remote computer may be connected to the user's computer via any type of network (including a local area network (LAN) or a wide area network (WAN)) or may be connected to an external computer (e.g., via the Internet using an Internet service provider). In some embodiments, electronic circuitry including, for example, programmable logic circuitry, field-programmable gate arrays (FPGAs), or programmable logic arrays (PLAs) may execute computer-readable program instructions by utilizing state information from the computer-readable program instructions to personalize the electronic circuitry in order to perform aspects of this invention.

[0095] The present invention will now be described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the invention. It should be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer-readable program instructions.

[0096] These computer-readable program instructions may be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, create means for implementing the functions / actions specified in one or more blocks of a flowchart and / or block diagram. These computer-readable program instructions may also be stored in a computer-readable storage medium that causes a computer, programmable data processing apparatus, and / or other device to operate in a particular manner, such that the computer-readable storage medium storing the instructions includes an article of manufacture containing instructions that implement aspects of the functions / actions specified in one or more blocks of a flowchart and / or block diagram.

[0097] Computer-readable program instructions may also be loaded onto a computer, other programmable data processing apparatus, or other device to cause a series of operational steps to be performed on the computer, other programmable apparatus, or other device to produce computer-implemented processing, such that the instructions executed on the computer, other programmable apparatus, or other device perform the functions / actions specified in one or more boxes of a flowchart and / or block diagram.

[0098] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of the present invention. Each block in a flowchart or block diagram may represent a module, segment, or portion of instructions, including one or more executable instructions for implementing a specified logical function. In some alternative implementations, the functions marked in the blocks may occur in a different order than indicated in the figures. For example, depending on the functions involved, two consecutively shown blocks may actually be executed substantially simultaneously, or these blocks may sometimes be executed in reverse order. It should also be noted that each block in the block diagrams and / or flowcharts, and combinations of blocks in the block diagrams and / or flowcharts, can be implemented using a dedicated hardware-based system that performs the specified function or action or executes a combination of dedicated hardware and computer instructions.

[0099] Various embodiments of the invention have been described for illustrative purposes, but are not intended to be exhaustive or limited to the disclosed embodiments. Many modifications and variations will be apparent to those skilled in the art without departing from the scope of the described embodiments. The terminology used herein has been chosen to best explain the principles of the embodiments, their practical application, or technical improvements over those found in the market, or to enable those skilled in the art to understand the embodiments described herein.

Claims

1. A computer-implemented method for saving power and encrypting media captured by an information capture device using a partitioned neural network, the computer-implemented method comprising: An artificial neural network (ANN) is copied from a computer server to the information capturing device, wherein both the ANN on the computer server and the copied ANN on the information capturing device comprise M layers; In response to the input of captured data to be processed, the first k layers are selected by the information capture device to be executed on the information capture device, and the value of k is sent to the computer server; In response to the input of captured data to be processed, the information capturing device partially processes the captured data by executing the k layers using the replicated ANN, wherein only the k layers are selected to be executed on the information capturing device; as well as The information capture device sends the output of the k-th layer to the computer server, which then partially processes the captured data by using the ANN and the output of the k-th layer to execute the remaining parts of the M layers. The value of k is selected by the information capturing device based at least on the charge level of the power supply of the information capturing device.

2. The computer-implemented method of claim 1 further includes receiving the result of the ANN from the computer server by the information capturing device.

3. The computer-implemented method according to claim 1, wherein, The ANN is trained by the computer server before it is copied to the information capture device.

4. The computer-implemented method according to claim 1, wherein, The captured data includes analog signals captured by the information sensing array.

5. The computer-implemented method according to claim 1, wherein, The captured data includes digital media.

6. The computer-implemented method of claim 1, wherein copying the ANN to the information capturing device comprises copying one or more weights of each of the M layers of the ANN to the corresponding M layers of the copied ANN.

7. The computer-implemented method according to claim 6, wherein, The one or more weights are encrypted before being sent to the information capturing device.

8. The computer-implemented method of claim 1, further comprising encrypting the output before sending the output of the k-th layer to the computer server.

9. The computer-implemented method of claim 1, further comprising encrypting the value of k before sending it to the computer server.

10. A system comprising: Memory; as well as One or more processors coupled to the memory, wherein the one or more processors are configured to perform a method for saving power and encrypting media captured by an information capture device using a partitioned neural network, the method comprising: An artificial neural network (ANN) is copied from a computer server to the information capturing device, wherein both the ANN on the computer server and the copied ANN on the information capturing device comprise M layers; The first k layers are selected to be executed on the information capture device, wherein the value of k is selected based at least on the charge level of the power supply of the information capture device; In response to the input of captured data to be processed, the captured data is partially processed by executing the k layers using the replicated ANN; and The output of the k-th layer is sent to the computer server, which partially processes the captured data by using the ANN and the output of the k-th layer to execute the remaining parts of the M layers.

11. The system according to claim 10, wherein, The ANN is trained by the computer server before it is copied to the information capture device.

12. The system according to claim 10, wherein, The captured data includes analog signals captured by the information sensing array.

13. The system according to claim 10, wherein, The captured data includes digital media.

14. The system according to claim 10, wherein, The method further includes encrypting the output before sending the output of the k-th layer to the computer server.

15. The system according to claim 10, wherein, The method further includes encrypting the value of k and sending it to the computer server.

16. A computer program product comprising a computer-readable storage medium having program instructions embodied therein, the program instructions being executable by one or more processors to cause the one or more processors to perform operations of saving power and encryption during analysis of media captured by an information capture device using a partitioned neural network, the operations including: The information capturing device copies an artificial neural network (ANN) from a computer server to the information capturing device, wherein both the ANN on the computer server and the copied ANN on the information capturing device comprise M layers; In response to the input of captured data to be processed, the first k layers are selected by the information capture device to be executed on the information capture device, and the value of k is sent to the computer server; In response to the input of captured data to be processed, the information capturing device partially processes the captured data by executing the k layers using the replicated ANN, wherein only the k layers are selected to be executed on the information capturing device; as well as The information capture device sends the output of the k-th layer to the computer server, which then partially processes the captured data by using the ANN and the output of the k-th layer to execute the remaining parts of the M layers. The value of k is selected by the information capturing device based at least on the charge level of the power supply of the information capturing device.

17. The computer program product according to claim 16, wherein, The captured data includes analog signals captured by the information sensing array.

18. The computer program product according to claim 16, wherein, The captured data includes digital media.

19. The computer program product according to claim 16, wherein, The operation further includes encrypting the output before sending the output of the k-th layer to the computer server.

Images