Edge device, inference system, control method, and control program

Abstract

[PROBLEM] To provide an edge device capable of easily maintaining the performance of a model of an edge device connected to a closed network. [SOLUTION] This edge device includes: a model; a performance deterioration detection unit that detects performance deterioration of the model; an inference request unit that, when performance deterioration is detected, requests another first edge device to perform inference processing based on another model; an inference processing unit that performs inference processing requested from another second edge device and transmits the result of the inference processing to the other second edge device; a model update determination unit that receives the result of inference processing by the other first edge device and determines whether to update the model on the basis of the received result; a model request unit that requests the other first edge device to transmit the other model when a determination was made to update the model; a model transmission unit that transmits the model on the basis of the request by the other second edge device; and a model replacement unit that receives the requested other model from the other first edge device and replaces the model.

Description

Edge device, inference system, control method, and control program

[0001] The present invention relates to an edge device, an inference system, a control method, and a control program.

[0002] In recent years, various technologies using machine learning have been developed. Machine learning models can classify input data into predefined classes or predict values ​​from the input data. For example, machine learning models can classify products into good or bad products based on photographed images, or estimate joint points from photographed images of people.

[0003] An Internet of Things (IoT) platform that applies machine learning models has been developed. The IoT platform is a system board that connects edge devices via a network, manages the edge devices, and collects data from the edge devices. A machine learning model runs on each edge device.

[0004] The machine learning models running on each edge device need to be continuously updated because their inference performance may decline due to changes in the surrounding environment or deterioration of sensors over time.

[0005] The following prior art is disclosed in Patent Document 1: An edge device acquires sensor data indicating the environment in which it is placed, analyzes the sensor data to determine the environment in which it is placed, and determines a machine learning model suitable for the determined environment. The edge device then acquires the determined machine learning model from the cloud and switches from another machine learning model that has already been applied to the acquired machine learning model.

[0006] The following prior art is disclosed in Patent Document 2 listed below. In each robot system, information acquired from sensors is input into an execution model, which outputs a recognition result, and the robot is controlled based on the recognition result to execute a task. In the robot system, when a server publishes a new model, the published model is acquired from the server as a candidate model, and the performance of the candidate model is evaluated using managed learning data. Then, in the robot system, if it is determined that the performance of the evaluated candidate model meets a predetermined performance level, the execution model is replaced with the candidate model.

[0007] The following technology is disclosed in Patent Document 3: When a learning device provided in a vehicle detects the replacement of a vehicle part related to input data for a machine learning model, the learning device receives from a server a new machine learning model trained using a training dataset corresponding to the replaced vehicle part, and then applies the received new machine learning model to the vehicle.

[0008] JP 2020-522886 A JP 2019-206041 A JP 2022-035227 A

[0009] However, in a system configured with a group of edge devices, there are use cases in which, from the viewpoint of security, the edge devices are not always connected to external networks other than the LAN to which they are interconnected. The prior art described in each of the above patent documents all assumes a constant connection with a server and receives a machine learning model from the server. For this reason, each prior art cannot address the above use cases.

[0010] If an edge device cannot obtain a machine learning model from the server, an administrator or other person must manually update the machine learning model on the edge device, and the burden of updating the model increases as the number of edge devices increases.

[0011] The present invention has been made to solve the above-mentioned problems, and aims to provide an edge device, an inference system, a control method, and a control program that can easily maintain the inference performance of a machine learning model in each edge device connected to a closed network.

[0012] The above-mentioned problems of the present invention are solved by the following means.

[0013] (1) An edge device comprising: a model for performing inference processing; and a performance degradation detection unit for detecting performance degradation of the model; an inference request unit for, when performance degradation of the model is detected by the performance degradation detection unit, requesting another, communicable first edge device, which has another model compatible with the model, to perform inference processing using the other model; an inference processing unit for performing inference processing using the model requested by another, communicable second edge device, and transmitting the result of the inference processing to the other, second edge device; a model update determination unit for receiving the result of the inference processing by the other, first edge device, requested by the inference request unit, and determining whether to update the model based on the received result; a model request unit for requesting the other, first edge device to transmit the other model if the model update determination unit determines to update the model; a model transmission unit for transmitting the model based on the request from the other, second edge device; and a model replacement unit for receiving the other model requested to be transmitted by the model request unit from the other, first edge device, and replacing the model with the received other model.

[0014] (2) The edge device described in (1) above, wherein the inference request unit requests another first edge device with which communication is possible to perform inference processing using the other model, the inference processing unit performs inference processing using the model requested by the other second edge device, and transmits the result of the inference processing and information regarding the compatibility of the model to the other second edge device, and the model update determination unit determines whether to update the model based on the result of the inference processing by the other first edge device and the information regarding the compatibility of the other model.

[0015] (3) The edge device described in (1) above has an inference processing feasibility determination unit that, when receiving a request for inference processing from the other second edge device, determines whether inference processing using the model is possible, and if it determines that it is not possible, notifies the other second edge device that inference processing is not possible.

[0016] (4) The edge device described in (1) above, having a specific edge device storage control unit that communicates with other edge devices connected to a network with which it can communicate, identifies specific edge devices having other models that are compatible with the model, and stores a list of the specific edge devices in a storage unit.

[0017] (5) The edge device described in (1) above, further comprising an update necessity notification unit that notifies the need to update the model when the model update determination unit determines not to update the model based on the results of the inference processing by all the other first edge devices to which the inference request unit has requested inference processing.

[0018] (6) The edge device according to (1), further comprising a model replacement completion notification unit that notifies that the model replacement has been completed after the model replacement unit has replaced the model with the other model.

[0019] (7) The edge device described in (1) above, wherein when the model replacement unit replaces the model with the other model, the model before replacement is stored in a memory unit, and the model update determination unit targets the model before replacement stored in the memory unit in addition to the other model provided in the other first edge device, and determines whether to update the model.

[0020] (8) The edge device described in (7) above, wherein the inference processing unit performs inference processing using the model and inference processing using the model before replacement stored in the memory unit, and transmits the results of multiple inference processing to the other second edge device.

[0021] (9) The edge device described in (1) above, having an inference processing control unit that does not perform inference processing using the model before replacement while the model replacement unit is replacing the model, and requests inference processing from another edge device.

[0022] (10) A system includes a plurality of edge devices each having a model for performing inference processing, and when a performance degradation detection unit of one edge device detects a performance degradation of a model included in the one edge device, an inference request unit requests another edge device that can communicate with the one edge device and has another model that is compatible with the one model to perform inference processing using the other model, and a model update determination unit receives a result of the inference processing by the other edge device and determines whether to update the one model based on the received result, and it is determined that the one model should be updated. when a request is made from the other edge device, a model request unit requests the other edge device to transmit the other model, a model replacement unit receives the other model requested to be transmitted by the model request unit from the other edge device and replaces the one model with the received other model, an inference processing unit in the other edge device performs inference processing using the other model requested by the one edge device and transmits the result of the inference processing to the one edge device, and a model transmission unit transmits the other model to the one edge device based on the request from the one edge device.

[0023] (11) A control method executed by an edge device having an edge device equipped with a model for performing inference processing, the control method comprising: a step (a1) of, when a performance degradation of the model is detected, requesting another edge device that can communicate with the edge device and that has another model compatible with the model to perform inference processing using the other model; a step (b1) of receiving a result of the inference processing by the other edge device requested in the step (a1) from the other edge device, and determining whether to update the model based on the received result, and if it is determined that the model should be updated, requesting the other edge device to transmit the other model; a step (c1) of receiving the other model requested in the step (b1) from the other edge device and replacing the model with the received other model; a step (d1) of performing inference processing using the model requested by the other edge device and transmitting the result of the inference processing to the other edge device; and a step (e1) of transmitting the model based on a request from the other edge device.

[0024] (12) A control method executed by an inference system having a plurality of edge devices each equipped with a model for performing inference processing, the method including: a step (a2) of, when detecting a performance degradation of a model equipped in one edge device, requesting another edge device that can communicate with the one edge device and that is equipped with another model that is compatible with the one model to perform inference processing using the other model; a step (b2) of, in the other edge device, performing the inference processing using the other model requested by the one edge device and transmitting the result of the inference processing to the one edge device; a step (c2) of receiving, in a first edge device, a result of an inference process by the other edge device from the other edge device, determining whether to update the one model based on the received result, and, if it is determined that the one model should be updated, requesting the other edge device to transmit the other model; a step (d2) of transmitting, in the other edge device, the other model to the one edge device based on the request from the one edge device; and a step (e2) of receiving, in the one edge device, the other model and replacing the one model with the received other model.

[0025] (13) A control program executed by an edge device having an edge device equipped with a model for performing inference processing, the control program causing a computer to execute processing including the steps of: (a1) requesting, when a performance degradation of the model is detected, another edge device that can communicate with the edge device and that has another model compatible with the model to perform inference processing using the other model; (b1) receiving, from the other edge device, a result of the inference processing by the other edge device requested in the step (a1), and determining whether to update the model based on the received result, and if it is determined that the model should be updated, requesting the other edge device to transmit the other model; (c1) receiving, from the other edge device, the other model requested in the step (b1), and replacing the model with the received other model; (d1) performing inference processing using the model requested by the other edge device and transmitting the result of the inference processing to the other edge device; and (e1) transmitting the model based on a request from the other edge device.

[0026] (14) A control program executed by an inference system having a plurality of edge devices each equipped with a model for performing inference processing, the control program including: a step (a2) of, when detecting a performance degradation of a model equipped in one edge device, requesting another edge device that can communicate with the one edge device and that is equipped with another model that is compatible with the one model to perform inference processing using the other model; a step (b2) of, in the one edge device, performing the inference processing using the other model requested by the one edge device and transmitting a result of the inference processing to the one edge device; A control program for causing a computer to execute a process comprising the steps of: receiving, from another edge device, a result of an inference process performed by the other edge device; determining whether to update the one model based on the received result; and, if it is determined that the one model should be updated, requesting the other edge device to transmit the other model; in the other edge device, transmitting the other model to the one edge device based on the request from the one edge device; and, in the one edge device, receiving the other model and replacing the one model with the received other model.

[0027] If an edge device detects a decline in model performance, it requests another edge device equipped with a model that is compatible with the model to perform inference processing using the other model. If it determines that the model needs to be updated based on the results of the inference processing, it obtains the other model from the other edge device and replaces the model with the other model. This makes it easy to maintain the inference performance of the machine learning model in each edge device connected to a closed network.

[0028] Advantages and features provided by one or more embodiments of the present invention will be more fully understood from the following detailed description and the accompanying drawings, which are provided for illustrative purposes only and are not intended to limit the present invention.

[0023] Figure 1 is a diagram illustrating a schematic configuration of an inference system.

[0024] Figure 2 is a block diagram illustrating the hardware configuration of a device.

[0025] Figure 3 is a block diagram illustrating the functions of a control unit.

[0026] Figure 4 is a block diagram illustrating the functions of a control unit of a device related to updating a model of a device whose inference accuracy has decreased.

[0027] Figure 5 is a diagram illustrating the positions of each device before moving its placement location.

[0028] Figure 6 is a diagram illustrating the positions of each device after moving the placement locations of some devices.

[0029] Figure 7 is a diagram illustrating a state in which a device having a model with decreased performance requests another device to perform an inference process.

[0030] Figure 8 is a diagram illustrating a state in which a device that has received a request to transmit a model transmits a model 170 to a device that has requested the transmission.

[0031] Figure 9 is a diagram illustrating a state in which a device having a model with decreased performance requests another device to perform an inference process.

[0032] Figure 10 is a diagram illustrating a state in which a device that has received a request to transmit a model transmits a model to a device that has requested the transmission.

[0033] Figure 11 is an explanatory diagram illustrating a response when a device having a model with decreased performance cannot find another device with a model that is compatible with the model that has decreased performance and that is capable of communication.

[0034] Figure 12 is a flowchart illustrating device operation.

[0029] Hereinafter, an edge device, an inference system, a control method, and a control program according to an embodiment of the present invention will be described with reference to the accompanying drawings. However, the scope of the present invention is not limited to the disclosed embodiment. In the description of the drawings, the same elements are denoted by the same reference numerals, and duplicated explanations will be omitted. Furthermore, the dimensional proportions in the drawings are exaggerated for the convenience of explanation and may differ from the actual proportions.

[0030] Fig. 1 is a diagram showing a schematic configuration of an inference system 1. In Fig. 1, for ease of explanation, a server 20 that is not included in the inference system 1 is also shown.

[0031] As shown in FIG. 1 , the inference system 1 includes a plurality of devices 10. The devices 10 constitute edge devices. The devices 10 are connected to each other so that they can communicate with each other. The devices 10 are connected to a closed network. That is, the devices 10 are connected to a closed local area network (LAN) and are not connected to the server 20 at least constantly. Each device 10 has a model 170 (see FIG. 4 ), which is a trained model that has been machine-learned, and performs inference processing using the model 170.

[0032] The server 20 may be a model distribution server. The server 20 may transmit any of the models 170, which are trained models that have been machine-learned, to devices that can communicate with each other.

[0033] FIG. 2 is a block diagram showing the hardware configuration of the device 10. As shown in FIG.

[0034] 2, the device 10 includes a control unit 100, a storage unit 200, a display unit 300, an input unit 400, and a communication unit 500. These components are connected to each other via a bus 600. The device 10 does not necessarily have to include the display unit 300 and the input unit 400. The device 10 may be configured by a computer.

[0035] The control unit 100 is configured by a CPU (Central Processing Unit) and controls and processes each part of the device 10 according to a program. The function of the control unit 100 will be described later.

[0036] The storage unit 200 may be configured with a RAM (Random Access Memory), a ROM (Read Only Memory), and a flash memory. The RAM temporarily stores programs and data as a working area for the control unit 100. The ROM stores various programs and data in advance. The flash memory stores various programs and data, including the operating system. The storage unit stores the model 170.

[0037] The display unit 300 is, for example, a liquid crystal display, and displays various types of information.

[0038] The input unit 400 is configured with, for example, a touch panel and various keys, and is used for various operations and inputs.

[0039] The communication unit 500 is an interface for communicating with external devices. Network interfaces conforming to standards such as Ethernet (registered trademark), SATA, PCI Express, USB, and IEEE 1394 may be used for communication. Additionally, various local connection interfaces such as wireless communication interfaces such as Bluetooth (registered trademark) and IEEE 802.11 may also be used for communication.

[0040] The function of the control unit 100 will be described.

[0041] FIG. 3 is a block diagram showing the functions of the control unit 100.

[0042] 3, by executing a program, the control unit 100 functions as a performance degradation detection unit 101, an inference request unit 102, a model update determination unit 103, a model request unit 104, a model replacement unit 105, an inference processing unit 106, and a model transmission unit 107. Furthermore, by executing a program, the control unit 100 functions as an inference processing feasibility determination unit 108, a specific edge device storage control unit 109, an update necessity notification unit 110, a model replacement completion notification unit 111, and an inference processing control unit 112.

[0043] First, the performance degradation detection unit 101, the inference request unit 102, the model update determination unit 103, the model request unit 104, the model replacement unit 105, the inference processing unit 106, and the model transmission unit 107 will be described.

[0044] 4 is a block diagram for explaining the function of the control unit 100 of the device 10 related to updating the model 170 of the device 10 in which the inference accuracy of the model 170 has decreased. Hereinafter, the inference accuracy may also be simply referred to as "accuracy."

[0045] In FIG. 4, the functions of the control unit 100 related to updating the model 170 are shown as a performance degradation detection unit 101, an inference request unit 102, a model update determination unit 103, a model request unit 104, a model replacement unit 105, an inference processing unit 106, and a model transmission unit 107.

[0046] Device 10A is a device 10 equipped with model 170A that needs to be updated due to a decrease in accuracy. Device 10B is a device 10 that transmits its own model 170B to device 10B in response to a request from device 10A equipped with model 170A whose accuracy has decreased. Device 10A and device 10B can achieve the same functions using their respective control units 100. However, device 10A and device 10B can function such that the former requests the latter to transmit model 170B, and the latter transmits model 170B to the former. For this reason, in FIG. 4 , blocks in device 10A and device 10B that correspond to this function are indicated by solid lines, and blocks corresponding to other functions are indicated by dashed lines.

[0047] Device 10A constitutes an edge device and one edge device. Model 170A constitutes a model and one model. Device 10B constitutes another edge device and another first edge device. Model 170B constitutes another model. Note that device 10C (not shown), which is a device 10 other than device 10A and device 10B and requests device 10A to perform inference processing using model 170A, constitutes another second edge device.

[0048] The model 170 may be a trained neural network model that performs inference processing to classify input data into predetermined classes or to predict numerical values ​​from input data. For ease of explanation, the following description will be given using an example in which the model 170 performs inference processing to estimate the coordinates of each joint point of a person from image data that is input data. The results of the inference processing by the model 170 may include the estimated results of the joint point coordinates and the likelihood of the estimated joint point coordinates. Hereinafter, the results of the inference processing by the model 170 will also be simply referred to as "inference results."

[0049] 4, the sensor 160 detects light, temperature, etc., and outputs the detected data. For simplicity of explanation, the following description will be given taking as an example a case where the sensor 160 is a camera and the detected data is image data.

[0050] The inference processing unit 106 inputs image data output as detection data from the sensor 160 to the model 170, causing the model 170 to perform inference processing. The inference processing unit 106 can use the inference results from the model 170 to detect events such as a person falling or tripping over.

[0051] The performance degradation detection unit 101 detects a performance degradation of the model 170. The performance degradation detection unit 101 can detect a performance degradation of the model 170 when the likelihood in the inference result by the model 170 becomes equal to or less than a predetermined threshold. Specifically, for example, the performance degradation of the model 170 is detected when the likelihood of the inference result of any one of the time-series consecutive image data, the average value of the likelihood of the inference results, or the median value of the likelihood of the inference results becomes equal to or less than a predetermined threshold. The predetermined threshold is set to any value within the range of 0.5 to 0.8, for example.

[0052] When the performance degradation detection unit 101 detects a performance degradation of the model 170, the inference request unit 102 requests another communicable device 10, which includes another model 170 compatible with the model 170, to perform inference processing using the other model 170. Specifically, as shown in FIG. 4 , when the performance degradation detection unit 101 of the device 10A detects a performance degradation of the model 170A, the inference request unit 102 requests the device 10B, which includes another model 170B compatible with the model 170A, to perform inference processing using the model 170B. In FIG. 4 , the devices 10A and 10B are illustrated as being capable of communicating with each other, and the models 170A and 170B are illustrated as being compatible with each other. The inference request unit 102 of the device 10A can send image data to the device 10B and request inference processing using the model 170B, using the image data as input data. The inference request unit 102 of the device 10A can know in advance whether the model 170B included in the device 10B is compatible with the model 170A included in the device 10A by receiving the metadata of the model 170B from the device 10B.

[0053] The inference request unit 102 may request any other device 10 with which communication is possible to perform inference processing using the model 170 included in the other device 10 .

[0054] The inference processing unit 106 performs inference processing using the model 170 when requested by another device 10 with which it can communicate, and transmits the inference result to the other device 10. Specifically, as shown in Fig. 4, the inference processing unit 106 of the device 10B performs inference processing using the model 170B when requested by the device 10A, and transmits the inference result to the device 10A. The inference processing unit 106 of the device 10B can perform inference processing using the model 170B using image data received from the device 10A as input data.

[0055] The model update determination unit 103 receives the inference result from another device 10 requested by the inference request unit 102 and determines whether to update the model 170 based on the received inference result. Specifically, as shown in FIG. 4 , the model update determination unit 103 of the device 10A receives the inference result from the device 10B requested by the inference request unit 102 via the communication unit 500 and determines whether to update the model 170A based on the received inference result. The model update determination unit 103 may determine to update the model 170A if the likelihood of the inference result from the model 170B for the image data transmitted to the device 10B when requesting the inference process is greater than the likelihood when a performance degradation of the model 170A was detected. The model update determination unit 103 may determine to update the model 170A by replacing the model 170A with the model 170B. In addition, the model update determination unit 103 may determine to update the model 170A when the likelihood of the inference result by the model 170B of the image data sent to the device 10B when requesting the inference process exceeds a predetermined threshold.

[0056] The model update determination unit 103 may determine whether to update the model 170 based on the inference result of another device 10 requested to perform inference processing by the inference request unit 102 and information regarding the compatibility of the other model. The information regarding compatibility includes, for example, metadata of the model 170. In this case, the inference processing unit 106 transmits the information regarding model compatibility together with the inference result to the device 10 that requested the inference processing.

[0057] When the model update determination unit 103 determines to update the model 170, the model request unit 104 requests the other device 10 that requested the inference process to transmit the other model 170 included in the other device 10. Specifically, as shown in Fig. 4, when the model update determination unit 103 determines to update the model 170A, the model request unit 104 of the device 10A requests the device 10B to transmit the model 170B.

[0058] The model sending unit 107 sends the model 170 based on a request from another device 10. Specifically, as shown in Fig. 4, the model sending unit 107 of the device 10B sends the model 170B to the device 10A based on a request from the device 10A.

[0059] The model replacement unit 105 receives another model 170, the transmission of which has been requested by the model request unit 104, from another device 10, and replaces the model 170 with the received another model 170. Specifically, as shown in Fig. 4, the model replacement unit 105 of the device 10A receives the model 107B, the transmission of which has been requested by the model request unit 104, from the device 10B via the communication unit 500. Then, the model replacement unit 105 of the device 10A updates the model 170A by replacing it with the received model 170B.

[0060] Next, the inference processing possibility determination unit 108, the specific edge device storage control unit 109, the update necessity notification unit 110, the model replacement completion notification unit 111, and the inference processing control unit 112 will be described.

[0061] When receiving an inference processing request from another device 10, the inference processing feasibility determination unit 108 determines whether inference processing by the model 170 is possible, and if it determines that inference processing is impossible, notifies the other device 10 via the communication unit 500 that inference processing is impossible. Cases where inference processing by the model 170 is impossible include, for example, a case where inference processing based on image data from the sensor 180 is currently being executed by the model 170 and the ongoing inference processing must be stopped in order to execute the requested inference processing. In this case, inference processing by the model 170 provided in the device 10B based on the image data from the sensor 180 may be prioritized. Specifically, in FIG. 4 , the inference processing feasibility determination unit 108 (not shown in FIG. 4 ) of the device 10B does not execute the inference processing requested by the device 10A when inference processing based on image data from the sensor 180 is currently being executed by the model 170. In this case, the inference processing feasibility determination unit 108 of the device 10B notifies the device 10A that inference processing is impossible.

[0062] The specific edge device storage control unit 109 communicates with other devices 10 connected to a communicable network, and identifies other devices 10 that have models 170 compatible with the model 170 that the specific edge device storage control unit 109 has as specific edge devices. The specific edge device storage control unit 109 then stores a list of the identified specific edge devices in the storage unit 200. Specifically, the specific edge device storage control unit 109 identifies specific edge devices by, for example, receiving metadata of the model 170 that the other devices 10 have from the other devices 10. The list of specific edge devices stored in the storage unit 200 can be used by the inference request unit 102 to select a destination to which the inference process should be requested.

[0063] When the model update determination unit 103 determines not to update the model 170 based on the inference results of all other devices 10 requested by the inference request unit 102 to perform inference processing, the update necessity notification unit 110 notifies the administrator that the model 170 needs to be updated. In other words, when the likelihoods of the inference results of all other devices 10 requested to perform inference processing are equal to or lower than the likelihood when performance degradation of the model 170A is detected, updating the model 170 will not address the performance degradation of the model 170. For this reason, it is necessary to notify the administrator that the performance of the model 170 has deteriorated and therefore needs to be updated. Therefore, the update necessity notification unit 110 notifies the administrator that the model 170 needs to be updated. The update necessity notification unit 110 notifies the administrator that the model 170 needs to be updated, for example, by transmitting an alarm via the communication unit 500 to the administrator's terminal indicating that the performance of the model 170 has deteriorated and therefore needs to be updated.

[0064] The model replacement completion notifying unit 111 notifies that the replacement of the model 170 has been completed after the model replacement unit 105 has replaced the model 170 with another model 170. The model replacement completion notifying unit 111 notifies that the replacement of the model 170 has been completed, for example, by sending a message indicating that the replacement of the model 170 has been completed to the terminal of the administrator.

[0065] While the model replacement unit 105 is replacing the model 170, the inference processing control unit 112 may not perform inference processing using the model 170 before replacement, but may request inference processing from another device 10. Specifically, while the model 170 is being replaced, the inference processing control unit 112 does not stop or interrupt the replacement processing, and does not reject the request for inference processing, but instead requests inference processing from another device 10. The inference processing control unit 112 may request inference processing from a device 10 that includes another model 170 that is compatible with the model 170. Whether a device 10 includes another model 170 that is compatible with the model 170 can be determined by receiving metadata from the device 10.

[0066] (Modification) When the model replacement unit 105 replaces the model 170 with the model 170 received from another device 10 in response to a request from the model request unit 104 to the other device 10, the model replacement unit 105 may store the model 170 before replacement in the storage unit 200. In this case, the model update determination unit 103 may treat the model 170 before replacement stored in the storage unit 200 as a target for updating the model 170, in addition to the model 170 included in the other device 10, and determine whether to update the model 170. That is, the model update determination unit 103 may treat the model 170 stored in the storage unit 200 as a candidate for the model 170 to be replaced, in addition to the model 170 requested from the other device 10.

[0067] Furthermore, in the device 10 that has been requested to perform inference processing using the model 170, the inference processing unit 106 may perform inference processing using the model 170 for which the inference processing has been requested and the model 170 before replacement that is stored in the storage unit 200. Then, the inference processing unit 106 may transmit multiple inference results from these inference processing to the device 10 that has requested the inference processing.

[0068] (Example) Fig. 5 is a diagram showing the positions of each device 10 before the placement position is moved. Fig. 6 is a diagram showing the positions of each device 10 after the placement positions of some of the devices 10 have been moved. Fig. 7 is a diagram showing a state in which a device having a model 170 with degraded performance requests another device 10 to perform inference processing. Fig. 8 is a diagram showing a state in which a device 10 that has been requested to transmit the model 170 transmits the model 170 to the device 10 that requested the transmission. Fig. 9 is a diagram showing a state in which a device having a model 170 with degraded performance requests another device 10 to perform inference processing. Fig. 10 is a diagram showing a state in which a device 10 that has been requested to transmit the model 170 transmits the model 170 to the device 10 that requested the transmission.

[0069] As shown in FIG. 5 , devices 10A and 10B are placed in a bright location within a building, device 10C is placed in a dark location within the building, and devices 10D and E are placed outside the building. In this state, each device 10 except for device 10E is equipped with a model 170 appropriate for its placement environment. That is, for example, device 10C placed in a dark location is equipped with a dark-adaptive version of model 170, and device 10D placed outside the building is equipped with a noise-adaptive version of model 170. The noise-adaptive version of model 170 is appropriate for device 10D placed outside the building because image data may contain noise caused by the lens of sensor 180 becoming dirty outside the building, and this possibility increases over time. The noise-adaptive version of model 170 may be a model 170 trained using image data containing noise. On the other hand, device 10E is equipped with a basic model 170 despite being placed outside the building, and therefore cannot be said to be equipped with a model 170 appropriate for its placement environment.

[0070] 6, when device 10A, which was placed in a bright place inside a building, is moved to a dark place inside the building, the performance of device 10A deteriorates because device 10A is equipped with a basic model 170 that is not a dark place version. Similarly, when device 10B, which was placed in a bright place inside a building, is moved outside the building, the performance of device 10B gradually deteriorates because device 10B is equipped with a basic model 170 that is not a noise-adaptive version. As described above, device 10E, which is placed outside the building, also has the basic model 170, so the performance of device 10E also gradually deteriorates.

[0071] As shown in Figure 7, device 10A, which detects a performance degradation of model 170, requests a communicable device 10C, which has a model 170 compatible with the degraded model 170, to perform inference processing using the compatible model 170.

[0072] 8, device 10A receives an inference result from device 10C that requested the inference process, and when it determines to update model 170 based on the inference result, it requests device 10C to transmit model 170 and receives model 170. Device 10A updates model 170 by replacing model 170 with the received dark environment version of model 170 that is suitable for the dark environment inside the building where device 10A will be placed after movement.

[0073] 9 , device 10E, which has detected a performance degradation of model 170, requests devices 10D and 10B, which are communicable and equipped with models 170 compatible with the degraded model 170, to perform inference processing using the compatible models 170. Note that device 10B has been moved outside the building, and therefore is no longer equipped with a model 170 suited to its installation environment. However, because device 10B has just been moved outside the building, it has not detected the performance degradation of model 170.

[0074] 10 , device 10E receives inference results from devices 10D and 10B, which have requested inference processing. If device 10E determines, based on each inference result, to update model 170 by replacing it with model 170 included in device 10D, it requests device 10D to transmit model 170 and receives model 170. Device 10E updates model 170 by replacing model 170 with the received noise-adaptive version of model 170 that is suited to the environment outside the building where it will be placed after movement.

[0075] FIG. 11 is an explanatory diagram for explaining what to do when a device 10 having a model 170 with reduced performance cannot find another device 10 with a model 170 that is compatible with the reduced performance model 170 and with which it can communicate.

[0076] As shown in FIG. 11 , even though device 10F is located outside the building, the model 170 included in device 10F is the basic model 170. Device 10F detects a performance degradation of the model 170, but is unable to find another device 10 with which it can communicate and which includes a model 170 compatible with the degraded model 170. In this case, device 10F may temporarily communicate with a model distribution server that is not constantly connected and acquire a noise-adapted version of the model 170 from the model distribution server. Note that if device 10F is unable to connect to the model distribution server even temporarily, an administrator or the like who receives a notification from the update necessity notification unit 110 that the model 170 needs to be updated may manually replace the model 170 included in device 10F to perform the update.

[0077] The operation of the device 10 will now be described.

[0078] 12 and 13 are flowcharts showing the operation of the device 10. The flowchart in Fig. 12 can be executed by the control unit 100 of one device 10 in accordance with a program. The flowchart in Fig. 13 can be executed by the control unit 100 of another device 10 in accordance with a program.

[0079] In one device 10, the control unit 100 determines whether or not a performance degradation of the model 170 has been detected (S101). If the control unit 100 has not detected a performance degradation of the model 170 (S101: NO), the control unit 100 executes step S101 again.

[0080] When the control unit 100 detects a performance degradation of the model 170 (S101: YES), it searches for another device 10 with which communication is possible that includes another model 170 that is compatible with the model 170 (S102).

[0081] The control unit 100 requests the other devices 10 found by the search to perform inference processing using the other models 170 (S103).

[0082] The control unit 100 determines whether there is another model 170 with a higher performance than the model 170 (S104). If the control unit 100 determines that there is no other model 170 with a higher performance than the model 170 (S104: NO), it notifies the control unit 100 that the model 170 needs to be updated (S107).

[0083] If the control unit 100 determines that there is another model 170 with higher performance than the model 170 (S104: YES), it requests another device 10 that has the other model 170 with the highest performance to send the other model 170 (S105).

[0084] The control unit 100 updates the model 170 by replacing it with another model 170 received from another device 10 (S106).

[0085] In the other device 10, the control unit 100 determines whether or not there is a request for inference processing (S201). If the control unit 100 determines that there is no request for inference processing (S201: NO), it executes step S201 again.

[0086] When the control unit 100 determines that there is a request for inference processing (S201: YES), it executes the inference processing using the model 170 (S202) and transmits the inference result to the device 10 that requested the inference processing (S203).

[0087] The control unit 100 determines whether or not the device 10 that requested the inference process has requested transmission of the model 170 (S204). If the control unit 100 determines that the device 10 that requested the inference process has not requested transmission of the model 170 (S204: NO), it executes step S204 again.

[0088] If the control unit 100 determines that the device 10 that requested the inference process has requested the transmission of the model 170 (S204: YES), the control unit 100 transmits the model 170 to the device 10 that requested the transmission of the model 170 (S205).

[0089] The flowcharts shown in Figures 12 and 13 may be executed by any one of the devices 10 as a control device that controls the entire inference system 1.

[0090] This embodiment has the following advantages.

[0091] If an edge device detects a decline in model performance, it requests another edge device equipped with a model that is compatible with the model to perform inference processing using the other model. If it determines that the model needs to be updated based on the results of the inference processing, it obtains the other model from the other edge device and replaces the model with the other model. This makes it easy to maintain the inference performance of the machine learning model in each edge device connected to a closed network.

[0092] The edge device also requests other edge devices with which it can communicate to perform inference processing using other models. The edge device performs inference processing using the model requested by the other edge device and transmits the inference results and information regarding model compatibility to the other edge device. The edge device then determines whether to update the model based on the inference results from the other edge device and the information regarding compatibility. This makes it possible to more flexibly and simply maintain the inference performance of the machine learning model of each edge device connected to the closed network.

[0093] Furthermore, when an inference processing request is received from another edge device, the edge device determines whether inference processing using the model is possible, and if it determines that it is not possible, notifies the other edge device that inference processing is not possible. This allows the edge device to maintain its performance by continuing inference processing using the model even when it receives an inference processing request from another edge device.

[0094] The device also communicates with other edge devices connected to a network that can communicate with the device, identifies specific edge devices that have other models that are compatible with the model, and stores a list of the specific edge devices in a storage unit. This makes it possible to quickly identify edge devices that have models that are compatible with the model whose performance has deteriorated when the model's performance deteriorates.

[0095] Furthermore, if it is determined that the model should not be updated based on the results of inference processing by all other edge devices that have been requested to perform inference processing, a notification is sent that the model needs to be updated. This prevents degradation of the model performance in the edge device from being left unattended, and also urges the administrator or other person in charge to update the model.

[0096] Furthermore, after replacing a model with another model, the completion of the model replacement is notified, allowing the administrator or the like to know which devices have had their models updated.

[0097] Furthermore, when replacing a model with another model, the model before replacement is stored in a storage unit. Then, in addition to other models included in other edge devices, the model before replacement stored in the storage unit is used as the model update target, and a decision is made as to whether to update the model. This makes it possible to maintain the number of model types that can be acquired in the edge device group.

[0098] In addition, the edge device performs inference processing using the model and inference processing using the model before replacement stored in the storage unit, and transmits multiple inference results to the edge device that requested the inference processing, thereby maintaining the number of types of models that can be acquired in the edge device group.

[0099] Furthermore, while the model is being replaced, inference processing using the model before replacement is not performed, and inference processing is requested from another edge device. This prevents unauthorized use of the license when a license agreement stipulates that only one model can be run on each edge device. Furthermore, it prevents inference processing from being suspended during a model update.

[0100] The present invention is not limited to the above-described embodiments.

[0101] For example, the learning data may be text data or the like.

[0102] In addition, in the embodiments, part or all of the processing executed by the program may be replaced with hardware such as a circuit.

[0103] While embodiments of the present invention have been described and illustrated in detail, the disclosed embodiments are made for purposes of illustration and example only and are not intended to be limiting, and the scope of the present invention should be construed by the language of the appended claims.

[0104] This application is based on a Japanese patent application (Patent Application No. 2024-105973) filed on July 1, 2024, the disclosure of which is incorporated herein by reference in its entirety.

[0105] 1 Inference system, 10 Device, 100 Control unit, 101 Performance degradation detection unit, 102 Inference request unit, 103 Model update determination unit, 104 Model request unit, 105 Model replacement unit, 106 Inference processing unit, 107 Model transmission unit, 108 Inference processing feasibility determination unit, 109 Specific edge device storage control unit, 110 Update necessity notification unit, 111 Model replacement completion notification unit, 112 Inference processing control unit, 200 Storage unit, 300 Display unit, 400 Input unit, 500 Communication unit, 600 Bus, 20 Server.

Claims

1. An edge device comprising: a model for performing inference processing; a performance degradation detection unit that detects performance degradation of the model; an inference request unit that, when a performance degradation of the model is detected by the performance degradation detection unit, requests another, communicable first edge device that has another model compatible with the model to perform inference processing using the other model; an inference processing unit that performs inference processing using the model requested by another, communicable second edge device and transmits the result of the inference processing to the other, second edge device; a model update determination unit that receives the result of the inference processing by the other, first edge device requested by the inference request unit and determines whether to update the model based on the received result; a model request unit that, when the model update determination unit determines to update the model, requests the other, first edge device to transmit the other model; a model transmission unit that transmits the model based on a request from the other, second edge device; and a model replacement unit that receives the other model requested to be transmitted by the model request unit from the other, first edge device, and replaces the model with the received other model.

2. The edge device described in claim 1, wherein the inference request unit requests another first edge device with which communication is possible to perform inference processing using the other model, the inference processing unit performs inference processing using the model requested by the other second edge device, and transmits the result of the inference processing and information regarding the compatibility of the model to the other second edge device, and the model update determination unit determines whether to update the model based on the result of the inference processing by the other first edge device and the information regarding the compatibility of the other model.

3. An edge device as described in claim 1, having an inference processing feasibility determination unit that, when receiving a request for inference processing from the other second edge device, determines whether inference processing using the model is possible, and if it determines that it is not possible, notifies the other second edge device that inference processing is not possible.

4. The edge device of claim 1, further comprising a specific edge device storage control unit that communicates with other edge devices connected to a network with which it can communicate, identifies specific edge devices having other models that are compatible with the model, and stores a list of the specific edge devices in a storage unit.

5. The edge device of claim 1, further comprising an update necessity notification unit that notifies that the model needs to be updated when the model update determination unit determines not to update the model based on the results of the inference processing by all of the other first edge devices to which the inference request unit has requested inference processing.

6. The edge device according to claim 1, further comprising a model replacement completion notification unit that notifies that the model replacement has been completed after the model replacement unit has replaced the model with the other model.

7. The edge device according to claim 1, wherein when the model replacement unit replaces the model with the other model, the model before replacement is stored in a memory unit, and the model update determination unit targets the model before replacement stored in the memory unit in addition to the other model provided in the other first edge device, and determines whether to update the model.

8. The edge device described in claim 7, wherein the inference processing unit performs inference processing using the model and inference processing using the model before replacement stored in the memory unit, and transmits the results of the multiple inference processing to the other second edge device.

9. The edge device according to claim 1, further comprising an inference processing control unit that, while the model replacement unit is replacing the model, does not perform inference processing using the model before replacement, but instead requests inference processing from another edge device.

10. A system includes a plurality of edge devices each equipped with a model for performing inference processing, wherein, when a performance degradation detection unit detects a performance degradation of a model equipped in the one edge device, an inference request unit requests another edge device that can communicate with the one edge device and that has another model compatible with the one model to perform inference processing using the other model; a model update determination unit receives a result of the inference processing by the other edge device and determines whether to update the one model based on the received result, and when it is determined that the one model should be updated, a model request unit requests the other edge device to transmit the other model; a model replacement unit receives the other model requested to be transmitted by the model request unit from the other edge device and replaces the one model with the received other model; and, in the other edge device, an inference processing unit performs inference processing using the other model requested by the one edge device and transmits the result of the inference processing to the one edge device; An inference system in which a model sending unit sends the other model to the one edge device based on a request from the one edge device.

11. A control method executed by an edge device having an edge device equipped with a model for performing inference processing, comprising: a step (a1) of, when a performance degradation of the model is detected, requesting another edge device that can communicate with the edge device and that has another model compatible with the model to perform inference processing using the other model; a step (b1) of receiving from the other edge device a result of the inference processing requested in step (a1), and determining whether to update the model based on the received result, and if it is determined that the model should be updated, requesting the other edge device to transmit the other model; a step (c1) of receiving from the other edge device the other model requested in step (b1) and replacing the model with the received other model; a step (d1) of performing inference processing using the model requested by the other edge device and transmitting the result of the inference processing to the other edge device; and a step (e1) of transmitting the model based on a request from the other edge device.

12. A control method executed by an inference system having a plurality of edge devices each equipped with a model for performing inference processing, comprising: a step (a2) in one edge device, when detecting a performance degradation of a first model equipped in the first edge device, requesting another edge device that can communicate with the first edge device and that is equipped with another model compatible with the first model to perform inference processing using the other model; a step (b2) in the other edge device performing the inference processing using the other model requested by the first edge device and transmitting the result of the inference processing to the first edge device; a step (c2) in the one edge device receiving from the other edge device the result of the inference processing performed by the other edge device, determining whether to update the first model based on the received result, and if it is determined that the first model should be updated, requesting the other edge device to transmit the other model; and a step (d2) in the other edge device transmitting the other model to the first edge device based on the request from the first edge device. and (e2) receiving, in the one edge device, the other model and replacing the one model with the received other model.

13. A control program executed by an edge device having an edge device equipped with a model for performing inference processing, the control program causing a computer to execute processing comprising the steps of: (a1) requesting, when a performance degradation of the model is detected, another edge device that can communicate with the edge device and that has another model compatible with the model to perform inference processing using the other model; (b1) receiving from the other edge device a result of the inference processing requested in step (a1), determining whether to update the model based on the received result, and, if it is determined that the model should be updated, requesting the other edge device to transmit the other model; (c1) receiving from the other edge device the other model requested in step (b1) and replacing the model with the received other model; (d1) performing inference processing using the model requested by the other edge device and transmitting the result of the inference processing to the other edge device; and (e1) transmitting the model based on a request from the other edge device.

14. A control program executed by an inference system having a plurality of edge devices each equipped with a model for performing inference processing, comprising: a step (a2) in one edge device, when detecting a performance degradation of a first model equipped in the first edge device, requesting another edge device that can communicate with the first edge device and that is equipped with another model compatible with the first model to perform inference processing using the other model; a step (b2) in the other edge device, performing the inference processing using the other model requested by the first edge device and transmitting the result of the inference processing to the first edge device; a step (c2) in the one edge device, receiving from the other edge device the result of the inference processing by the other edge device, determining whether to update the first model based on the received result, and if it is determined that the first model should be updated, requesting the other edge device to transmit the other model; and a step (d2) in the other edge device, transmitting the other model to the first edge device based on the request from the first edge device. and (e2) receiving, in the one edge device, the other model and replacing the one model with the received other model.

Images