Processing system and processing method

Abstract

To control execution of processing in cooperation with an edge device and a cloud according to a request of a user.SOLUTION: In a processing system 100, an edge device 30 has: an inference unit 32 which outputs a processing result of processing target data; and a determination unit 33 which determines to cause a server device 20 to execute processing related to the processing target data when an evaluation value for evaluating which of the edge device 30 and the server device 20 is to process the processing target data satisfies a condition, determines that the evaluation value is a value included in a range to be processed by the edge device 30 when the processing result of the processing target data satisfies a predetermined evaluation, and outputs the processing result of the processing target data processed by the inference unit 32. The evaluation value is an output deadline of the processing result of the processing target data. When the output deadline is short, and is determined to be processed by the edge device 30.SELECTED DRAWING: Figure 4

Description

[Technical Field]

[0001] The present invention relates to a processing system and a processing method. [Background technology]

[0002] The amount of data collected by IoT devices, such as sensors, is enormous, and when aggregating and processing the collected data using cloud computing, a huge amount of communication traffic occurs. For this reason, attention is being paid to edge computing, which processes collected data on edge devices close to users.

[0003] However, the computational load and resources such as memory used in edge devices are poor compared to devices other than edge devices (hereinafter referred to as cloud for simplicity) that are located physically and logically farther from users than edge devices. For this reason, when a process with a large computational load is performed on an edge device, it may take a long time to complete the process, or other processes with a smaller computational load may also take a long time to complete.

[0004] Here, one type of processing that requires a large amount of computation is processing related to machine learning. Non-Patent Document 1 proposes the application of so-called adaptive learning to edge clouds. That is, the method described in Non-Patent Document 1 deploys a trained model trained in the cloud using general-purpose training data to an edge device, and then uses data acquired by the edge device to retrain the model trained in the cloud, thereby realizing operation that takes advantage of the advantages of both the cloud and the edge device. [Prior art documents] [Non-patent literature]

[0005] [Non-Patent Document 1] Okoshi et al., "Proposal and Evaluation of a DNN Model Operation Method Using Cloud-Edge Collaboration," Proceedings of the 80th National Conference of the Information Processing Society of Japan, 2018(1), 3-4, 2018-03-13. Summary of the Invention [Problem to be solved by the invention]

[0006] However, the method described in Non-Patent Document 1 does not consider inference processing. The more complex the object to be processed, i.e., the data to be inferred, and the more difficult the problem to be solved, the greater the amount of computation required. It is assumed that such computationally intensive processing should be performed in the cloud. However, in order to determine which computationally intensive processing should be performed in the cloud, the complexity of the data to be inferred and the difficulty of the problem to be solved must be determined by the edge device.

[0007] In addition to the difficulty of the problem to be solved, there are also other factors, such as the inference accuracy and response required by the user. That is, there are cases where the user requires a quick response even if the inference accuracy is not very high, and cases where the user requires high inference accuracy even if the response is slow. However, Non-Patent Document 1 does not describe a method for an edge device to determine a computationally intensive process to be performed in the cloud while taking into account the inference accuracy and response required by the user.

[0008] The present invention has been made in consideration of the above, and aims to provide a processing system and a processing method that can control the execution of processing in cooperation with an edge device and a cloud in accordance with user requests. [Means for solving the problem]

[0009] In order to solve the above-mentioned problems and achieve the object, the processing system of the present invention is a processing system that uses an edge device and a server device, wherein the edge device has an edge processing unit that processes data to be processed and outputs the processing results of the data to be processed, and a determination unit that, if an evaluation value that evaluates whether the data to be processed should be processed in the edge device or the server device satisfies a condition, determines that the processing related to the data to be processed should be executed by the server device, and, if the processing result of the data to be processed satisfies a predetermined evaluation, determines that the evaluation value is a value that is included in the range that should be processed by the edge device and outputs the processing result of the data to be processed processed by the edge processing unit, and a transmission unit that, if the determination unit determines that the processing related to the data to be processed should be executed by the server device, transmits data to cause the server device to execute the processing related to the data to be processed, wherein the evaluation value is an output deadline for the processing results of the data to be processed, and if the output deadline is short, it is determined that the data will be processed by the edge device.

[0010] Furthermore, the processing method of the present invention is a processing method executed by a processing system that uses an edge device and a server device, and includes: a processing step in which the edge device processes data to be processed and outputs the processing results of the data to be processed; a determination step in which, if an evaluation value used by the edge device to evaluate whether the data to be processed should be processed by the edge device or the server device satisfies a condition, the edge device determines to have the server device execute the processing related to the data to be processed; and, if the processing result of the data to be processed satisfies a predetermined evaluation, determines that the evaluation value is a value that falls within a range that should be processed by the edge device and outputs the processing result of the data to be processed in the processing step; and a transmission step in which, if the edge device determines in the determination step that the server device should execute the processing related to the data to be processed, the edge device transmits data to cause the server device to execute the processing related to the data to be processed; wherein the evaluation value is a deadline for outputting the processing results of the data to be processed, and if the output deadline is short, it is determined that the data will be processed by the edge device. [Effects of the Invention]

[0011] According to the present invention, it is possible to control the execution of processing in cooperation between edge devices and the cloud in accordance with user requests, and to efficiently operate the entire system combining the devices and the cloud. [Brief explanation of the drawings]

[0012] [Figure 1] FIG. 1 is a diagram illustrating an outline of a processing method of a processing system according to a first embodiment. [Figure 2] FIG. 2 is a diagram illustrating an example of DNN1 and DNN2. [Figure 3] FIG. 3 is a diagram illustrating an example of DNN1 and DNN2. [Figure 4] FIG. 4 is a diagram schematically illustrating an example of the configuration of the processing system according to the first embodiment. [Figure 5] FIG. 5 is a sequence diagram showing the flow of processing in the processing system according to the first embodiment. [Figure 6] FIG. 6 is a diagram illustrating an example of the configuration of a learning device that learns a lightweight model and a high-precision model. [Figure 7] FIG. 7 is a diagram showing an example of loss for each case. [Figure 8] FIG. 8 is a flowchart showing the flow of the learning process for the high-precision model. [Figure 9] FIG. 9 is a flowchart showing the flow of the learning process of the lightweight model. [Figure 10] FIG. 10 is a diagram schematically illustrating an example of the configuration of a processing system according to the second embodiment. [Figure 11] FIG. 11 is a sequence diagram showing the flow of processing in the processing system according to the second embodiment. [Figure 12] FIG. 12 is a diagram schematically illustrating another example of the configuration of the processing system according to the second embodiment. [Figure 13] FIG. 13 is a diagram schematically illustrating an example of the configuration of a processing system according to the third embodiment. [Figure 14] FIG. 14 is a diagram schematically illustrating an example of the edge device illustrated in FIG. [Figure 15] FIG. 15 is a diagram schematically illustrating an example of the server device illustrated in FIG. [Figure 16] FIG. 16 is a sequence diagram showing the flow of processing in the processing system according to the third embodiment. [Figure 17] FIG. 17 is a diagram schematically illustrating an example of the configuration of a processing system according to the fourth embodiment. [Figure 18] FIG. 18 is a diagram schematically illustrating an example of the edge device illustrated in FIG. [Figure 19] FIG. 19 is a sequence diagram showing the flow of processing in the processing system according to the third embodiment. [Figure 20] FIG. 20 is a diagram showing an outline of a processing system according to a modification of the first to fourth embodiments. [Figure 21] FIG. 21 is a diagram showing variations in the functions of the DNN, the determination unit, the encoding unit, and the decoding unit, and variations in the content of communication. [Figure 22] FIG. 22 is a diagram illustrating an example of a computer that implements an edge device and a server device by executing a program. DETAILED DESCRIPTION OF THE INVENTION

[0013] Hereinafter, an embodiment of the present invention will be described in detail with reference to the drawings. Note that the present invention is not limited to this embodiment. In addition, in the description of the drawings, the same parts are designated by the same reference numerals.

[0014] [Embodiment 1] [Outline of the first embodiment] An embodiment of the present invention will be described. In a first embodiment of the present invention, a processing system that performs inference processing using a trained high-precision model and a lightweight model will be described. Note that the processing system of the embodiment will be described taking as an example a case where a DNN (Deep Neural Network) is used as the model used in the inference processing. The processing system of the embodiment may use a neural network other than a DNN, or may use low-computation-amount signal processing and high-computation-amount signal processing instead of a trained model.

[0015] FIG. 1 is a diagram illustrating an outline of a processing method of a processing system according to a first embodiment. In the processing system according to the first embodiment, a high-precision model and a lightweight model form a model cascade. In the processing system according to the first embodiment, an evaluation value is used to control whether processing is executed in an edge device that uses a high-speed, low-precision lightweight model (e.g., DNN1) or in a cloud (server device) that uses a low-speed, high-precision high-precision model (e.g., DNN2). For example, a server device is a device located in a location that is physically and logically far from a user. An edge device is an IoT device or various terminal devices located in a location that is physically and logically close to a user, and has fewer resources than a server device.

[0016] DNN1 and DNN2 are models that output inference results based on input data. In the example of Figure 1, DNN1 and DNN2 take an image as input and infer the probability of each class of object appearing in the image. Note that both images shown in Figure 1 are the same image. For example, DNN1 is trained to determine which model, DNN1 or DNN2, will perform inference and provide the greater benefit desired by the user. DNN1 and DNN2 are optimized to obtain the optimal value for the evaluation score.

[0017] The user's requests include, for example, higher accuracy of inference results, reduction in data communication volume, faster calculation processing, and optimization of edge device resources. The evaluation value is a value that evaluates whether the target data should be processed by the edge device or the server device while satisfying the user's requests. The more difficult the processing of the target data is, the more the evaluation value tends to fall within a range where the target data should be evaluated by the server device.

[0018] As shown in FIG. 1, the processing system acquires an evaluation value for the inference of the class classification of an object in an input image by DNN1. If the acquired evaluation value satisfies a predetermined condition, for example, a predetermined value, the processing system adopts the inference result of DNN1. In other words, the inference result of the lightweight model is output as the final estimation result of the model cascade. On the other hand, if the evaluation value does not satisfy the predetermined value, the processing system outputs the inference result obtained by inputting the same image into DNN2 as the final inference result. Satisfying a predetermined value includes, for example, whether or not a condition based on a predetermined threshold is satisfied, or whether or not the result is within a predetermined range.

[0019] In this way, the processing system according to the first embodiment selects an edge device or a server device based on an evaluation value that evaluates whether the data should be processed in an edge device or a server device in accordance with a user's request, and processes the data to be processed. Therefore, the processing system according to the first embodiment can control whether the processing is to be performed in an edge device or in the cloud in accordance with a user's request.

[0020] [Lightweight and high-precision models] Next, we will explain DNN1 and DNN2. Figures 2 and 3 are diagrams explaining examples of DNN1 and DNN2. A DNN has an input layer that receives data, one or more intermediate layers that transform the data input from the input layer in various ways, and an output layer that outputs so-called inference results such as probability or likelihood. The output values ​​output from each layer may be non-reversible if the input data needs to remain anonymous.

[0021] As shown in Figure 2, the processing system may use independent DNN1a and DNN2a. For example, after DNN2a is trained using a known method, DNN1a is trained to determine which model, DNN1a or DNN2a, performs inference, providing the greatest benefit desired by the user. DNN1a is trained to output a value related to the evaluation value. DNN1a outputs an intermediate output value, which is the output value of the intermediate layer of DNN1a, as the value related to the evaluation value. The evaluation value may be a value calculated based on the intermediate output value, or it may be the intermediate output value itself. The intermediate output value may be the result of applying a cost function, such as correlating the intermediate output value with the likelihood, to a predetermined intermediate layer, which can be trained to better meet the user's needs. Alternatively, the output of any intermediate layer of a trained model designed with only the problem to be solved as the cost function may be used. This is because, for example, when using a network (such as a CNN) whose higher-order intermediate layers tend to reflect features that determine the input data, the output values ​​of the higher-order intermediate layers contain useful features that can be used in the problem to be solved. Note that DNN1a and DNN2a may be assigned the same task with different accuracy and performance, or different tasks.

[0022] Also, as shown in Figure 3, the processing system divides DNN3, which has been trained as a single DNN, into DNN1b and DNN2b between the Rth layer and the (R+1)th layer using a predetermined criterion. The processing system may then apply the previous-stage DNN1b to an edge device and the subsequent-stage DNN1b to a server device. In this case, DNN1b outputs an intermediate output value from the Rth hidden layer as an evaluation value. Note that DNN1b may also output an intermediate output value from a layer earlier than the Rth hidden layer as an evaluation value.

[0023] Furthermore, the evaluation value is not limited to the intermediate output value output from DNN1a or DNN1b. For example, the evaluation value may be an inference error output from DNN1a or a value based on the inference error. For example, the evaluation value may be a value indicating the degree of certainty that the result of processing performed by the edge device is correct. The evaluation value may also be determined based on the time required to obtain the processing result of the target data, the deadline for obtaining the processing result of the target data, the resource usage status of the edge device when determining whether the target data should be processed in the edge device or the server device, or whether the target data is data in which an event has occurred compared to other data. The resource usage status of the edge device may be the CPU or memory usage rate or power consumption of the edge device alone, or the difference in the operating volume or resource usage rate between the edge device and other edge devices. An event may be, for example, a change in the target frame compared to the previous frame that is greater than the desired magnitude, or the occurrence of an object requiring detailed estimation. Data indicating the objects and results calculated by the edge device may be transmitted to a server device, and the server device may be designed to calculate only the objects not calculated by the edge device. Specifically, the bounding box coordinate values, the classification results, and their reliability may be sent together, and only those objects that do not satisfy the reliability may be calculated on the server device.

[0024] [Processing System] Next, the configuration of the processing system will be described. Fig. 4 is a diagram schematically illustrating an example of the configuration of the processing system according to the first embodiment.

[0025] The processing system 100 according to the embodiment includes a server device 20 and an edge device 30. The server device 20 and the edge device 30 are connected via a network N. The network N is, for example, the Internet. In this case, the server device 20 may be a server provided in a cloud environment. The edge device 30 may be an IoT device or various terminal devices.

[0026] The server device 20 and the edge device 30 are realized by loading a predetermined program into a computer or the like including a ROM (Read Only Memory), a RAM (Random Access Memory), a CPU (Central Processing Unit), etc., and executing the predetermined program by the CPU. Also, so-called accelerators such as a GPU, a VPU (Vision Processing Unit), an FPGA (Field Programmable Gate Array), an ASIC (Application Specific Integrated Circuit), or a dedicated AI (Artificial Intelligence) chip may also be used. The server device 20 and the edge device 30 each have a NIC (Network Interface Card) or the like, and can communicate with other devices via telecommunications lines such as a LAN (Local Area Network) or the Internet.

[0027] 4, the server device 20 stores DNN2, which is a trained high-precision model. The DNN2 includes information such as model parameters. The server device 20 also includes an inference unit 22.

[0028] The inference unit 22 inputs inference data (data to be processed) into the DNN2 and obtains an inference result (processing result). The inference unit 22 accepts the input of inference data and outputs the inference result. The inference data is assumed to be data with unknown labels. For example, the inference data is an image. Furthermore, when returning the inference result to the user, the inference result obtained by the inference unit 22 may be transmitted to an edge device and returned from the edge device to the user.

[0029] Here, the server device 20 and the edge device 30 constitute a model cascade. Therefore, the inference unit 22 does not always perform inference on the inference data. When it is determined that the server device 20 should execute inference processing related to the inference data, the inference unit 22 performs inference using DNN2.

[0030] The edge device 30 stores DNN1, which is a trained lightweight model. DNN1 includes information such as model parameters. DNN1 is trained to determine whether inference is performed using DNN1 or DNN2, which model provides the greatest benefit desired by the user. Pre-trained parameters are set in DNN1 so that a model cascade including DNN1 and DNN2 is optimized, including determining whether the greatest benefit desired by the user is achieved. The edge device 30 also includes an inference unit 32 (edge ​​processing unit), a determination unit 33, and a communication unit 34 (transmission unit).

[0031] The inference unit 32 obtains an inference result by inputting inference data (data to be processed) into the DNN 1. The inference unit 32 accepts input of the inference data, processes the inference data, and outputs an inference result (a processing result of the data to be processed).

[0032] The determination unit 33 determines whether an evaluation value, which is designed to reflect the user's request and determines whether the edge device 30 or the server device 20 should process the inference data, satisfies a predetermined value.

[0033] If the evaluation value satisfies a predetermined value, the determination unit 33 determines that the inference result of the inference data satisfies the predetermined evaluation, determines that the evaluation value is a value within a range to be processed by the edge device 30, and outputs the inference result inferred by the inference unit 32. If the evaluation value does not satisfy the predetermined value, the determination unit 33 determines that the evaluation value is a value within a range to be evaluated by the server device 20, and determines to have the server device 20 execute processing related to the inference data (inference processing). As described above, the evaluation value is an intermediate output value, an inference error, a confidence level, etc. The determination unit 33 may also narrow down the processing data to be transmitted. For example, the determination unit 33 narrows down the processing data to data of nodes necessary for DNN2 processing. Here, an example of a narrowing down criterion when the inference data is an image will be described. If an event has occurred in a partial area of ​​the image, the determination unit 33 may narrow down the processing data to that partial area or to an area necessary for making an inference related to the event. Furthermore, when determining whether to process each region of an image on an edge device or on a server device, the determination unit 33 may narrow down the region to one that is to be processed on a server device. Although narrowing down from a spatial perspective has been exemplified, the determination unit 33 may also narrow down from a temporal perspective.

[0034] The communication unit 34 communicates with other devices (for example, the server device 20) via the network N. When the determination unit 33 determines that the server device 20 should execute the inference processing related to the inference data, the communication unit 34 transmits processing data to the server device 20 to cause the server device 20 to execute the inference processing. When the evaluation value is an intermediate output value, the communication unit 34 transmits the intermediate output value to the server device 20.

[0035] [Processing system procedure] Fig. 5 is a sequence diagram showing the flow of processing in the processing system according to embodiment 1. As shown in Fig. 5, first, in the edge device 30, upon receiving input of inference data (step S1), the inference unit 32 inputs the inference data to the DNN 1 (step S2).

[0036] The determination unit 33 acquires the intermediate output value of the DNN1 (steps S3 and S4), and acquires an evaluation value (step S5). The determination unit 33 determines whether the evaluation value satisfies a predetermined value (step S6).

[0037] If the evaluation value satisfies the predetermined value (step S6: Yes), the determination unit 33 inputs the intermediate output value to the next intermediate layer of the DNN1 (step S7). The inference unit 32 acquires the inference result of the DNN1 (step S8) and outputs the acquired inference result of the DNN1 (step S9).

[0038] On the other hand, if the evaluation value does not satisfy the predetermined value (step S6: No), the determination unit 33 transmits processing data for causing the server device 20 to execute the inference process to the server device 20 via the communication unit 34 (steps S10 and S11). For example, the processing data is the inference data and the confidence level of the DNN1. Alternatively, the processing data is an intermediate output value.

[0039] In the server device 20, the inference unit 22 inputs the processing data to the DNN2 (step S11) and acquires the inference result of the DNN2 (steps S12 and S13). The inference result of the DNN2 is transmitted to the edge device 30 (steps S14 and S15) and output from the edge device 30 (step S16). Note that in this embodiment, a configuration is assumed in which the inference result is returned to the user, and the edge device 30 is configured to output the final inference result. However, if the final inference result is to be used on the server device 20 side, the inference result of the DNN2 may be output from the server device 20 or may be stored in the server device 20 as is. If the inference result of the DNN1 is to be used on the server device 20 side, the edge device 30 may transmit the inference result to the server device 20.

[0040] [Effects of the First Embodiment] As described above, according to the first embodiment, the edge device or the server device is selected based on an evaluation value that evaluates whether the data to be processed should be processed in the edge device or the server device in response to a user request, and the data to be processed is then processed. Therefore, the processing system according to the first embodiment can control whether the processing should be performed in the edge device or the cloud in response to a user request.

[0041] In this embodiment 1, the case where the number of edge devices 30 and the number of server devices 20 are both singular has been described, but there may be multiple edge devices 30 or server devices 20, or there may be multiple edge devices 30 and server devices 20.

[0042] [Application example] This embodiment 1 is applied to the demand for high accuracy of inference results, and an example will be described in which a certainty factor is used as an evaluation value. First, learning of a lightweight model and a high accuracy model to achieve high accuracy of inference results will be described.

[0043] Fig. 6 is a diagram showing an example of the configuration of a learning device that learns lightweight models and high-precision models. As shown in Fig. 6, the learning device 10 receives input of learning data and outputs trained high-precision model information and trained lightweight model information. The learning device 10 also includes a high-precision model learning unit 11 and a lightweight model learning unit 12.

[0044] The high-precision model learning unit 11 has an estimation unit 111, a loss calculation unit 112, and an update unit 113. The high-precision model learning unit 11 also stores high-precision model information 114. The high-precision model information 114 is information such as parameters for constructing a high-precision model. The learning data is assumed to be data with known labels. For example, the learning data is a combination of an image and a label (correct class).

[0045] The estimation unit 111 inputs learning data into a high-accuracy model constructed based on the high-accuracy model information 114, and acquires an estimation result. The estimation unit 111 accepts input of learning data and outputs an estimation result.

[0046] The loss calculation unit 112 calculates a loss based on the estimation result acquired by the estimation unit 111. The loss calculation unit 112 receives input of the estimation result and the label, and outputs the loss. For example, the loss calculation unit 112 calculates a loss such that the smaller the confidence level for the label in the estimation result acquired by the estimation unit 111, the larger the loss. For example, the confidence level is a degree of likelihood that the estimation result is correct. For example, the confidence level may be the probability output by the multi-class classification model described above. Specifically, the loss calculation unit 112 can calculate the softmax cross entropy, which will be described later, as the loss.

[0047] The update unit 113 updates the parameters of the high-precision model so as to optimize the loss. For example, if the high-precision model is a neural network, the update unit 113 updates the parameters of the high-precision model by backpropagation or the like. Specifically, the update unit 113 updates the high-precision model information 114. The update unit 113 receives an input of the loss calculated by the loss calculation unit 112 and outputs information on the updated model.

[0048] The lightweight model learning unit 12 includes an estimation unit 121, a loss calculation unit 122, and an update unit 123. The lightweight model learning unit 12 also stores lightweight model information 124. The lightweight model information 124 is information such as parameters for constructing a lightweight model.

[0049] The estimation unit 121 inputs learning data into a lightweight model constructed based on the lightweight model information 124, and acquires an estimation result. The estimation unit 121 accepts input of learning data and outputs an estimation result.

[0050] Here, the high-precision model learning unit 11 learns the high-precision model based on the output of the high-precision model, while the lightweight model learning unit 12 learns the lightweight model based on the outputs of both the high-precision model and the lightweight model.

[0051] The loss calculation unit 122 calculates the loss based on the estimation result obtained by the estimation unit. The loss calculation unit 122 receives input of the estimation result by the high-precision model, the estimation result by the lightweight model, and the label, and outputs the loss. The estimation result by the high-precision model may be an estimation result obtained by inputting further learning data into the high-precision model after learning by the high-precision model learning unit 11. More specifically, the lightweight model learning unit 12 receives input of whether the estimation result by the high-precision model was correct. For example, if the class with the highest probability output by the high-precision model matches the label, the estimation result is correct.

[0052] The loss calculation unit 122 calculates the loss with the aim of maximizing the estimation accuracy of a single lightweight model, as well as maximizing the profit when a model cascade is configured. Here, the profit increases as the estimation accuracy increases and as the calculation cost decreases.

[0053] For example, a high-precision model is characterized by high estimation accuracy but high calculation costs. On the other hand, a light-weight model is characterized by low estimation accuracy but low calculation costs. Therefore, the loss calculation unit 122 calculates the loss as shown in equation (1), where w is a weight and is a parameter that is set in advance.

[0054]

number

[0055] where L classifier is the softmax entropy in a multi-class classification model. classifieris an example of the first term that increases as the confidence in the correct answer in the estimation result by the lightweight model decreases. classifier is expressed as in equation (2). Here, N is the number of samples, k is the number of classes, y is the label representing the correct class, q is the probability output by the lightweight model, i is the number identifying the sample, and j is the number identifying the class. The label y i,j is 1 if the jth class is correct for the ith sample, and 0 if it is incorrect.

[0056]

number

[0057] Also, L cascade is the term for maximizing profit when constructing a model cascade. L cascade represents the loss when the estimation results of the high-precision model and the lightweight model are used for each sample based on the confidence of the lightweight model. Here, the loss includes the penalty for inappropriate confidence and the cost of using the high-precision model. Furthermore, the loss is divided into four patterns depending on whether the estimation result of the high-precision model is correct or not and whether the estimation result of the lightweight model is correct or not. Details will be explained later, but if the estimation of the high-precision model is incorrect and the confidence of the lightweight model is low, the penalty will be large. On the other hand, if the estimation of the lightweight model is correct and the confidence of the lightweight model is high, the penalty will be small. L cascade is expressed as in equation (3).

[0058]

number

[0059] 1 fast is an indicator function that returns 0 if the estimation result of the lightweight model is correct, and 1 if the estimation result of the lightweight model is incorrect. accis an indicator function that returns 0 if the estimation result of the high-precision model is correct, and 1 if the estimation result of the high-precision model is incorrect. acc is the cost required to perform estimation using a high-precision model, and is a parameter that is set in advance.

[0060] max j q i,j is the maximum probability output by the lightweight model, and is an example of a confidence level. If the estimation result is correct, the higher the confidence level, the higher the estimation accuracy. On the other hand, if the estimation result is incorrect, the higher the confidence level, the lower the estimation accuracy.

[0061] (3) max j q i,j 1 fast is an example of the second term that increases as the confidence level of the estimation result by the lightweight model increases when the estimation result by the lightweight model is incorrect. j q i,j )1 acc is an example of the third term that increases as the confidence level of the estimation result by the lightweight model decreases when the estimation result by the high-precision model is incorrect. j q i,j )COST acc is an example of the fourth term that increases as the confidence level of the estimation result by the lightweight model decreases. In this case, minimizing the loss by the update unit 123 corresponds to optimizing the loss.

[0062] The update unit 123 updates the parameters of the lightweight model so as to optimize the loss. That is, the update unit 123 updates the parameters of the lightweight model so as to optimize a model cascade including the lightweight model and the high-precision model, based on the estimation result from the lightweight model and the estimation result obtained by inputting learning data into a high-precision model, which is a model that outputs an estimation result based on input data and has a slower processing speed and higher estimation accuracy than the lightweight model. The update unit 123 accepts input of the loss calculated by the loss calculation unit 122, and outputs information about the updated model.

[0063] FIG. 7 shows an example of loss for each case. The vertical axis is L cascade The horizontal axis is the value of max j q i,j Also, the COST acc =0.5. max j q i,j is the confidence level of the estimation result by the lightweight model, and is simply referred to as the confidence level here.

[0064] The "□" in Figure 7 indicates the L confidence when the estimation results of both the lightweight model and the high-precision model are correct. cascade In this case, the greater the confidence, the greater the value of L cascade This is because if the estimation result using the lightweight model is correct, the greater the confidence level, the more likely the lightweight model is to be adopted.

[0065] In Figure 7, the "◇" indicates the L of the confidence when the estimation result of the lightweight model is correct and the estimation result of the high-precision model is incorrect. cascade In this case, the greater the confidence, the greater the value of L cascade The value of L is smaller than that of "□". cascade This is because if the estimation result by the high-precision model is incorrect and the estimation result by the lightweight model is correct, the tendency for the lightweight model to be adopted increases as the confidence level increases.

[0066] The black squares in Figure 7 represent the L confidence when the estimation result of the lightweight model is incorrect and the estimation result of the high-precision model is correct. cascade In this case, the greater the confidence, the greater the value of L cascade This is because even if the estimation result of the lightweight model is incorrect, the smaller the confidence level, the less likely the estimation result is to be adopted.

[0067] In Figure 7, the "◆" indicates the L confidence when the estimation results of both the lightweight model and the high-precision model are incorrect. cascadeIn this case, the greater the confidence, the greater the value of L cascade However, the value of L is smaller than in the case of "□". cascade The value of is large because the loss is always large due to incorrect estimation results from both models, and in such situations the lite model should be able to provide an accurate estimate.

[0068] [Learning process] Fig. 8 is a flowchart showing the flow of the learning process for the high-accuracy model. As shown in Fig. 8, first, the estimation unit 111 estimates the class of the learning data using the high-accuracy model (step S101).

[0069] Next, the loss calculation unit 112 calculates the loss based on the estimation result of the high-precision model (step S102). Then, the update unit 113 updates the parameters of the high-precision model so as to optimize the loss (step S103). Note that the learning device 10 may repeat the processes from step S101 to step S103 until a termination condition is satisfied. The termination condition may be that the processes have been repeated a predetermined number of times, or that the update width of the parameters has converged.

[0070] 9 is a flowchart showing the flow of the learning process of the lightweight model. As shown in Fig. 9, first, the estimation unit 121 estimates the class of the learning data using the lightweight model (step S201).

[0071] Next, the loss calculation unit 122 calculates the loss based on the estimation result of the lightweight model, the estimation result of the high-precision model, and the cost of estimation using the high-precision model (step S202). Then, the update unit 123 updates the parameters of the lightweight model so as to optimize the loss (step S203). Note that the learning device 10 may repeat the processes from step S201 to step S203 until a termination condition is satisfied.

[0072] In this way, the estimation unit 121 inputs training data into a lightweight model that outputs an estimation result based on the input data, and obtains a first estimation result. Furthermore, the update unit 123 updates the parameters of the lightweight model based on the first estimation result and a second estimation result obtained by inputting training data into a high-precision model that outputs an estimation result based on the input data and that has a slower processing speed and higher estimation accuracy than the lightweight model, so that the model cascade including the lightweight model and the high-precision model is optimized. In this way, the learning device 10 can improve the performance of a model cascade composed of a lightweight model and a high-precision model by enabling the lightweight model to perform estimation suitable for the model cascade. As a result, the learning device 10 not only improves the accuracy of the model cascade, but also reduces computational costs and overhead of computational resources. Furthermore, in the first embodiment, because the loss function is modified, there is no need to modify the model architecture, and there are no limitations on the model or optimization method that can be applied.

[0073] The update unit 123 updates the parameters of the lightweight model so as to minimize a loss calculated based on a loss function including: a first term that increases as the confidence level of the first estimation result decreases; a second term that increases as the confidence level of the first estimation result increases when the first estimation result is incorrect; a third term that increases as the confidence level of the first estimation result decreases when the second estimation result is incorrect; and a fourth term that increases as the confidence level of the first estimation result decreases. As a result, in the first embodiment, in a model cascade composed of a lightweight model and a high-precision model, it is possible to improve the estimation accuracy of the model cascade while taking into account the cost of adopting the estimation result of the high-precision model.

[0074] In the processing system 100, when inference is performed using a high-precision model and a lightweight model that have been trained by the learning device 10, the edge device 30 inputs inference data to the lightweight model (DNN1), obtains a confidence level, and adopts the estimation result of the lightweight model using the lightweight model if the confidence level is equal to or greater than a threshold. Furthermore, if the confidence level is less than the threshold, the edge device 30 transmits processing data to the server device 20. The processing system then adopts the estimation result of the high-precision model obtained by inputting the inference data to the high-precision model (DNN2) of the server device 20.

[0075] In the first embodiment, an example in which DNN is learned has been described, but a machine learning mechanism other than DNN may also be used.

[0076] [Embodiment 2] Next, a second embodiment will be described. In the second embodiment, the edge device encodes the data to be processed and then transmits it to the server device.

[0077] 10 is a diagram schematically illustrating an example of the configuration of a processing system according to embodiment 2. A processing system 200 according to embodiment 2 has a server device 220 instead of the server device 20 shown in FIG. 4, and has an edge device 230 instead of the edge device 30.

[0078] Compared to the edge device 30, the edge device 230 has an encoding unit 235. The encoding unit 235 encodes the data that the communication unit 34 transmits to the server device 220. For example, the encoding unit 235 compresses the data to be transmitted to reduce the amount of communication. Note that when the data to be transmitted to the server device 220 is the output value of an intermediate layer of the DNN1, even if the data is intercepted, the interceptor cannot interpret the meaning of the transmitted data, thereby ensuring security.

[0079] The intermediate output value is selected from among the multiple intermediate output values ​​of the DNN1 output in the process of outputting the inference result of the inference data, and is a value that is easier to encode than other intermediate output values. Easier to encode means that the entropy is smaller or the sparsity is higher than other intermediate output values. For example, the intermediate output value is an intermediate output value of an intermediate layer of a trained DNN1 that has been trained to reduce the entropy of the output value of the desired intermediate layer. The intermediate output value is an intermediate output value of an intermediate layer of a trained DNN1 that has been trained to increase the sparsity of the output value of the desired intermediate layer.

[0080] The server device 220 differs from the server device 20 in that it includes a decoding unit 223. The decoding unit 223 decodes the processing data coded by the coding unit 235 and outputs the decoded data to the inference unit 22.

[0081] Here, if DNN1 and DNN2 are models in which DNN3 (see Figure 3) that was trained as a single DNN is divided into DNN1b and DNN2b using a predetermined criterion, it is desirable to construct an efficient encoding unit 235 with little distortion in the inference results.

[0082] For example, once the data in the training set has been learned, the maximum value and frequency of occurrence of zero for each node in the intermediate layer to be transferred become apparent, and the encoding unit 235 is designed to perform encoding processing corresponding to this. The encoding processing may be processing that reduces the dimension of the representation space to be encoded by minimizing the influence of nodes with a high frequency of occurrence of zero, or processing that determines the range of the value of each node and selects a method or determines the granularity of quantization that reflects that tendency.

[0083] Alternatively, the encoding unit 235 may perform encoding based on a vector quantization technique. In this case, instead of quantizing the node values ​​individually, the encoding unit 235 regards the values ​​of all the nodes as vectors, clusters them in a vector space, and encodes them.

[0084] In addition, by finding a layer with low entropy and dividing the DNN3 at that layer, the encoding unit 235 can obtain intermediate output values ​​with low entropy.

[0085] Furthermore, the encoding unit 235 and the decoding unit 223 may employ an encoding / decoding method based on known rules, or may employ a learning-based method such as an AE (AutoEncoder) or a VAE (Variational AutoEncoder).

[0086] The encoding unit 235 may switch the encoding method for the processing data from among a plurality of encoding methods according to the intermediate output value and the destination DNN2. The decoding unit 223 decodes the data using a method according to the encoding method executed by the encoding unit 235.

[0087] [Processing system procedure] 11 is a sequence diagram showing the flow of processing in the processing system according to Embodiment 2. Steps S21 to S29 shown in FIG.

[0088] If the evaluation value does not satisfy the predetermined value (step S26: No), the encoding unit 235 encodes processing data for causing the server device 220 to execute the inference process (step S30) and transmits the encoded data to the server device 220 via the communication unit 34 (steps S31, S32). In the server device 220, the decoding unit 223 decodes the encoded data (step S33) and outputs the decoded processing data to the inference unit 22 (step S34). Steps S35 to S40 correspond to steps S11 to S16 shown in FIG. 5.

[0089] [Effects of the second embodiment] In this way, in the second embodiment, the edge device 230 encodes the processing data before transmitting it to the server device 220, thereby enabling the transmission of processing data with security maintained, the transmission of processing data in a data format with less distortion to the inference results, or the efficient transmission of processing data.

[0090] In the second embodiment, the edge device 230 has the encoding unit 236, and the server device 220 has the decoding unit 223. However, the present invention is not limited to this. Fig. 12 is a diagram schematically illustrating another example of the configuration of the processing system according to the second embodiment. As shown in Fig. 12, the encoding unit 235 may be provided in the network device 240A on the edge device 230A side between the edge device 230A and the server device 220A, and the decoding unit 223 may be provided in the network device 250A on the server device 220A side.

[0091] In the second embodiment, there may be a plurality of edge devices 230 or a plurality of server devices 220, or there may be a plurality of both edge devices 230 and server devices 220.

[0092] [Embodiment 3] Next, a third embodiment will be described. FIG. 13 is a diagram schematically illustrating an example of the configuration of a processing system according to the third embodiment. As shown in FIG. 13, the processing system 300 according to the third embodiment has a configuration in which a plurality of edge devices 330-1 and 330-2 are connected to one server device 320 via a network N. The number of edge devices is an example, and may be three or more. When the edge devices 330-1 and 330-2 are collectively referred to, they will be referred to as edge device 330.

[0093] Fig. 14 is a diagram schematically illustrating an example of the edge device 330-1 shown in Fig. 13. As shown in Fig. 14, the edge device 330-1 has an addition unit 336, unlike the edge device 30. The addition unit 336 adds a code identifying the device itself to the processing data. The communication unit 34 transmits the code identifying the device itself to the server device 320 together with the intermediate output value, which is the processing data.

[0094] The edge device 330-2 has the same configuration as the edge device 330-1. In this case, the DNN1 included in each edge device 330 may be the same model.

[0095] Furthermore, the DNN1 possessed by each edge device 330 may be a model formed through multi-task learning, in which consensus is reached between models, resulting in a commonality up to a specified intermediate layer. Consensus between models refers to, for example, learning while consensus is reached between intermediate layers at the same level of multiple models. In other words, when different training data is given for each model, it can be said that two factors are simultaneously optimized: the cost term related to the problem set for the model itself, and the cost term for consensus reached between intermediate layers at the same level of other models. As a result, the DNN1 possessed by each edge device 330 may be a model trained so that weights are the same from the input layer to a specified intermediate layer. For example, the DNN1 possessed by each edge device 330 may be common up to the feature extraction layer for acoustic signals, with subsequent layers performing different processing. In this case, the intermediate output values ​​output by each edge device 330 are set to be the output values ​​from the common layer. Of course, the edge devices 330 may also transmit output values ​​from different intermediate layers to the server device 320.

[0096] Fig. 15 is a diagram schematically illustrating an example of the server device 320 shown in Fig. 13. As shown in Fig. 15, the server device 320, compared to the server device 20, has a storage unit 324 and an inference result database (DB) 325. The storage unit 324 stores in the inference result DB 325 the result (inference result) of the analysis of the intermediate output value by the inference unit 22 in association with a code that identifies the edge device 330 that transmitted the processing data.

[0097] In the processing system 300, the processing performed in the edge device 330 and the processing performed in the server device 320 are optimized so that inference processing can be performed on data transmitted from any of the edge devices 330. For example, the DNN2 of the server device 320 is optimized so that it can handle processing data transmitted from any of the edge devices 330.

[0098] [Processing system procedure] 16 is a sequence diagram showing the flow of processing in the processing system according to embodiment 3. Steps S41 to S49 shown in FIG. 16 are the same processes as steps S1 to S9 shown in FIG.

[0099] If the evaluation value does not satisfy the predetermined value (step S46: No), the adding unit 336 adds a code identifying the own device to the processing data (step S50). The communication unit 34 transmits the code identifying the own device together with the intermediate output value, which is the processing data, to the server device 320 (steps S51 and S52).

[0100] Steps S53 to S58 shown in Fig. 16 correspond to steps S11 to S16 shown in Fig. 5. In the server device 320, the storage unit 324 associates the inference result with a code that identifies the edge device 330 that transmitted the processing data, and stores the inference result in the inference result DB 325 (steps S59 to S61).

[0101] [Effects of the Third Embodiment] As described above, in the third embodiment, even when connected to a plurality of edge devices 330, the DNN2 of the server device 320 is optimized so as to be able to handle processing data transmitted from any of the edge devices 330. The edge device 330 transmits a code identifying the server device 320 together with the intermediate output value, which is the processing data, to the server device 320. Therefore, the DNN2 of the server device 320 can recognize the processing data transmitted from any of the edge devices 330, thereby appropriately executing inference processing using the processing data.

[0102] The processing system 300 may include the encoding unit 235 and the decoding unit 223 described in the second embodiment.

[0103] [Embodiment 4] Next, a fourth embodiment will be described. FIG. 17 is a diagram schematically illustrating an example of the configuration of a processing system according to the fourth embodiment. As shown in FIG. 17, the processing system 400 according to the fourth embodiment has a configuration in which an edge device 430 is connected to a plurality of server devices 420-1, 420-2 via a network N. The number of server devices is an example, and may be three or more. When the server devices 420-1, 420-2 are collectively referred to, they will be referred to as server device 420.

[0104] The DNN2s of the server devices 420 each perform a different task. For example, the DNN2 of the server device 420-1 classifies the type of target data (image or audio signal). The DNN2 of the server device 420-2 classifies the nature of the target data (e.g., human or vehicle in the case of an object recognition task). The DNN2s of the other server devices 420 classify the processing content of the target data (object recognition task or sound source enhancement task). For example, if the DNN1 of the edge device 430 extracts data features, the DNN2 of each server device 420 will be specialized for the task assigned to that server device 420. Note that when different tasks are to be performed, so-called multitask learning can be used. Specifically, for Task 1 and Task 2, the input layer to a predetermined intermediate layer, which has been trained so that the weights from the input layer to the predetermined intermediate layer are common, can be placed on the edge device 430, and the parts beyond the predetermined intermediate layer can be placed on the server device 420. In this way, it is possible to configure the system so that any task can be processed using a model placed in any server device while using a common model placed in edge device 430. Also, different tasks may have the same purpose but different estimation accuracies. For example, the estimation accuracies may be configured so that edge device 430<server device 420-1<server device 420-2.

[0105] Fig. 18 is a diagram schematically illustrating an example of the edge device 430 shown in Fig. 17. As shown in Fig. 18, the edge device 430 has a selection unit 437, unlike the edge device 30. The selection unit 437 selects a server device 420 to which to transmit processing data from among the multiple server devices 420, depending on the purpose of processing the inference data.

[0106] [Processing system procedure] 19 is a sequence diagram showing the flow of processing in the processing system according to Embodiment 4. Steps S71 to S79 ​​shown in FIG. 19 are the same processes as steps S1 to S9 shown in FIG.

[0107] If the evaluation value does not satisfy the predetermined value (step S76: No), the selection unit 437 selects the destination server device 420 according to the purpose and accuracy of processing the inference data (step S80). The communication unit 34 transmits the processing data to the server device 420 (e.g., server device 420-1) selected by the selection unit 437 (steps S81 and S82). Steps S83 to S88 shown in FIG. 19 are the same processes as steps S11 to S16 shown in FIG. 5. The selection unit 437 that selects the destination (step S80) may be physically and / or logically located on the edge device side or on the server device side. The selection unit 437 may also be located on the network (at a location that cannot be distinguished as either the server side or the edge side).

[0108] [Effects of the Fourth Embodiment] Thus, in the fourth embodiment, even if the edge device 430 is connected to multiple server devices 420, the inference process can be executed appropriately by selecting the destination server device 420 depending on the purpose of processing the inference data.

[0109] In the fourth embodiment, there may be a plurality of edge devices 430. In addition, the processing system 400 may include a selection unit 437 in a network device between the edge device and the server device. In addition, the processing system 400 may include the encoding unit 235 and the decoding unit 223 described in the second embodiment. In this case, the selection unit 237 may be located before or after the encoding unit 235.

[0110] [Variations] Next, modifications of the embodiments 1 to 4 will be described. Fig. 20 is a diagram showing an outline of a processing system in the modifications of the embodiments 1 to 4. Hereinafter, with reference to Fig. 21, variations in the functions of the DNN1, DNN2, determination unit 33, encoding unit 235, decoding unit 223, and DNN2 shown in Fig. 20, and variations in communication content will be described.

[0111] Fig. 21 is a diagram showing variations in the functions of DNN1, DNN2, the determination unit 33, the encoding unit 235, and the decoding unit 223, as well as variations in the content of communication. Of the columns shown in Fig. 21, (1-A) to (1-H) show variations in the functions of DNN1 and DNN2. Furthermore, (2-A) to (2-G) show variations in the function of the determination unit 33. Furthermore, (3^A) to (3-F) show variations in the encoding unit 235, the decoding unit 223, and the content of communication between the edge device and the server device. Each functional unit and communication content can be as shown in each of these columns.

[0112] In addition, it is also possible to combine and operate each functional unit and communication content. For example, when using independent DNN1a and DNN2a (see Figure 2), it is possible to combine the functions and communication content of (1-D), (2-C), and (3-A). Also, when using DNN1b and DNN2b, which are divided from a single DNN3 (see Figure 3), it is possible to combine either (1-D) or (1-G) with either (2-C) or (2-D) and either (3-B), (3-C), (3-D-1), (3-D-2), or (3-D-3).

[0113] The present invention can be applied to a variety of situations where users have a wide range of needs. Some specific examples are given below.

[0114] [Autonomous Driving] Here is an example of collaboration between a computing device such as a DSP (Digital Signal Processor) installed in a vehicle and the cloud, which is considered an edge device. For example, processes that tend to require a large amount of calculation and transmission, such as navigation that takes traffic congestion into account, but where slow response is acceptable, can be handled by a server device, while events related to the direct control of the vehicle and decisions on vehicle control in response to detected events require a certain degree of accuracy and fast response, and can be handled by the edge device.

[0115] Change Detection When a time-series image signal is the subject, the edge device may detect whether or not there is a change compared to normal times or the previous frame, and the server device may estimate what kind of change has occurred.

[0116] The time-series image signal may be from a surveillance camera, satellite image, or aerial photograph. In the case of a surveillance camera, an edge device may detect a change when a person passes in front of the surveillance camera, and a server device may estimate the type of person who passed. In the case of satellite imagery, an edge device may detect changes in the edges or texture of buildings, or the passing of ships or automobiles, and a server device may estimate the type of building that was built, its construction status, and the type of ship that passed. In this case, a computing device installed on an airplane or satellite may be treated as an edge.

[0117] [Crime Prevention] Relatively simple and lightweight inference (counting the number of people, estimating gender and age, rough clothing determination, etc.) is performed on edge devices, while more complex inference that requires more load (identifying people, estimating posture, detecting suspicious people, etc.) is performed on the cloud (server devices).

[0118] In addition, detection of known suspicious individuals such as VIPs (Virtual IPs), repeaters, and complainers, which requires quick response, is performed on edge devices, while more general person detection, extraction of the person's characteristics, database creation, etc., which can take time, are performed in the cloud.

[0119] [Agriculture] For unmanned tractors, the edge device (tractor itself) checks to see if there are any obstacles ahead, and inference and planning, including how to deal with any obstacles that may be present, is carried out in the cloud.

[0120] [Inference-based vision] Images from cameras are received at the station and processed (normal two-layer inference), and the processed results are sent to the cloud for more advanced processing or aggregation (multi-layer inference). If resources at station A are depleted and the neighboring station B has surplus resources, the cloud controls the sending of partially processed data from station A to station B, which then performs the remaining processing. This enables efficient resource utilization (robustness of the service, efficient use of resources). This also means that computing devices located at stations may be controlled as a so-called edge cloud.

[0121] [Control of drone cameras] The cloud controls and instructs the placement of individual drone cameras in accordance with the overall shooting plan for multiple drone cameras, as well as recovery responses between cameras depending on the situation, while the drone (edge ​​device) performs inferences and judgments related to the situational responses specific to each drone camera (such as avoiding an obstacle that suddenly appears in front of the camera). In this example, the many-to-one embodiment 3 is applied.

[0122] Next, a one-to-many application example of the fourth embodiment will be described. A single camera image is converted into features using edge (DNN1), and the features are passed to multiple clouds in parallel and used in common to perform a variety of task processes (people counting, person identification, class classification, posture estimation, etc.). This is a one-to-many example, and encoding processing is applied to protect privacy.

[0123] [System configuration, etc.] The components of each device shown in the figure are conceptual functional units and do not necessarily have to be physically configured as shown. In other words, the specific form of distribution and integration of each device is not limited to that shown, and all or part of each device can be functionally or physically distributed and integrated in any unit depending on various loads, usage conditions, etc. Furthermore, all or any part of the processing functions performed by each device can be realized by a CPU and a program analyzed and executed by the CPU, or can be realized as hardware using wired logic.

[0124] Furthermore, among the processes described in this embodiment, all or part of the processes described as being performed automatically can be performed manually, or all or part of the processes described as being performed manually can be performed automatically using a known method.In addition, the information including the processing procedures, control procedures, specific names, various data and parameters shown in the above documents and drawings can be changed as desired unless otherwise specified.

[0125] [program] 22 is a diagram showing an example of a computer in which edge devices 30, 230, 330, and 430 and server devices 20, 220, 320, and 420 are realized by executing a program. The computer 1000 includes, for example, a memory 1010 and a CPU 1020. The computer may also include the accelerator described above to assist in calculations. The computer 1000 also includes a hard disk drive interface 1030, a disk drive interface 1040, a serial port interface 1050, a video adapter 1060, and a network interface 1070. These components are connected by a bus 1080.

[0126] The memory 1010 includes a ROM (Read Only Memory) 1011 and a RAM 1012. The ROM 1011 stores a boot program such as a BIOS (Basic Input Output System). The hard disk drive interface 1030 is connected to a hard disk drive 1090. The disk drive interface 1040 is connected to a disk drive 1100. A removable storage medium such as a magnetic disk or optical disk is inserted into the disk drive 1100. The serial port interface 1050 is connected to a mouse 1110 and a keyboard 1120, for example. The video adapter 1060 is connected to a display 1130, for example.

[0127] The hard disk drive 1090 stores, for example, an OS (Operating System) 1091, an application program 1092, a program module 1093, and program data 1094. That is, the programs that define the processes of the edge devices 30, 230, 330, and 430 and the server devices 20, 220, 320, and 420 are implemented as program modules 1093 in which computer-executable code is written. The program modules 1093 are stored, for example, in the hard disk drive 1090. For example, the program modules 1093 for executing processes similar to those of the functional configurations of the edge devices 30, 230, 330, and 430 and the server devices 20, 220, 320, and 420 are stored in the hard disk drive 1090. The hard disk drive 1090 may be replaced by an SSD (Solid State Drive).

[0128] Furthermore, setting data used in the processing of the above-described embodiment is stored as program data 1094, for example, in the memory 1010 or the hard disk drive 1090. Then, the CPU 1020 reads the program module 1093 or the program data 1094 stored in the memory 1010 or the hard disk drive 1090 into the RAM 1012 as necessary and executes them.

[0129] The program module 1093 and program data 1094 are not limited to being stored in the hard disk drive 1090, but may also be stored in, for example, a removable storage medium and read by the CPU 1020 via the disk drive 1100 or the like. Alternatively, the program module 1093 and program data 1094 may be stored in another computer connected via a network (such as a local area network (LAN) or a wide area network (WAN)). The program module 1093 and program data 1094 may then be read by the CPU 1020 from the other computer via the network interface 1070.

[0130] Although the present invention has been described above as an embodiment, the present invention is not limited to the description and drawings that form part of the disclosure of the present invention. In other words, other embodiments, examples, and operational techniques that can be made by those skilled in the art based on the present invention are all included in the scope of the present invention. [Explanation of symbols]

[0131] 10 Learning Device 11 High-precision model learning unit 12 Lightweight Model Learning Unit 30,230,230A,330,430 Edge Device 20,220,220A,320,420 Server equipment 100, 200, 300, 400 processing systems 111,121 Estimation part 112,122 Loss calculation section 113,123 Update Department 114 High Precision Model Information 124 Lightweight Model Information 22,32 Reasoning part 33 Judgment section 34 Communications Department 223 Decoding Unit 235 Encoding section 240A,250A NW equipment 324 Storage Unit 325 Inference Results Database (DB) 336 Additional Section 437 Selection Section

Claims

1. A processing system performed using an edge device and a server device, The edge device an edge processing unit that processes processing target data and outputs a processing result of the processing target data; a determination unit that outputs the processing result when the evaluation value satisfies a predetermined value, and determines to cause the server device to execute processing related to the processing target data when the evaluation value does not satisfy the predetermined value; a transmitting unit that transmits data for causing the server device to execute the processing related to the processing target data when the determining unit determines that the server device should execute the processing related to the processing target data; and the evaluation value is a deadline for obtaining a processing result of the processing target data, When a high speed response of the processing is not required, the processing related to the processing target data is performed by the server device, and when a high speed response of the processing is required, the processing of the processing target data is performed by the edge device, A processing system, wherein the data to be executed by the server device is the processing result.

2. further comprising an encoding unit that encodes the processing result, the process of outputting the processing result of the processing target data is inference using a trained neural network, 2. The processing system according to claim 1, wherein the processing result is an output value of the intermediate layer of the trained neural network that has been trained to reduce the entropy of a desired output value of the intermediate layer.

3. There are a plurality of the edge devices, The processing system described in claim 1 or 2, characterized in that the processing performed by the edge device and the processing performed by the server device are optimized so that the server device performs processing related to the data to be processed regardless of which edge device of the multiple edge devices the data is sent from.

4. There are a plurality of the server devices, The edge device a selection unit that selects, from among the plurality of server devices, a server device to which data for causing the server device to execute processing related to the processing target data is to be transmitted in accordance with a purpose for processing the processing target data; 4. The processing system according to claim 1, further comprising:

5. A processing method executed by a processing system using an edge device and a server device, a processing step in which the edge device processes processing target data and outputs a processing result of the processing target data; a determination step in which the edge device outputs the processing result when the evaluation value satisfies a predetermined value, and determines to have the server device execute processing related to the processing target data when the evaluation value does not satisfy the predetermined value; a transmission step of transmitting data for causing the server device to execute the processing related to the processing target data when the edge device determines in the determination step that the server device should execute the processing related to the processing target data; Including, the evaluation value is a deadline for obtaining a processing result of the processing target data, the processing system is configured such that, when a high speed response to the processing is not required, the server device processes the data to be processed, and, when a high speed response to the processing is required, the edge device processes the data to be processed; The processing method, wherein the data to be executed by the server device is the processing result.

Images