Systems, edge devices, and methods for controlling them.

Abstract

With conventional technologies, when linking edge devices each equipped with artificial intelligence inference capabilities, it took a long time to obtain an answer. [Solution] When question data is input from the client terminal 105 or the operation unit 206 of edge device A106, the determination unit 332 of device A106 determines whether the inference processing for the question data should be performed by the inference unit 333 of edge device A106, device B, or application service A103 (S721). If it determines that the inference processing should be performed by device B, it transmits the question data to device B without going through application service A103, and presents the answer data received from device B to the user via the client terminal 105 or operation unit 206, which was the source of the question data input (S725, S726).

Description

Technical Field

[0001] The present invention relates to systems, edge devices, and control methods thereof.

Background Art

[0002] In recent years, the introduction of artificial intelligence (AI) technology into devices has become an important means to enhance the value and competitiveness of devices. The scope supported by AI has become increasingly diverse day by day, contributing to the automation, efficiency improvement, and quality enhancement of operations using devices.

[0003] Conventionally, since the execution of inference processing by AI requires corresponding computing resources, it has often been limited to execution in a remote environment such as cloud services via a network.

[0004] However, in addition to the evolution of tuning technologies such as the lightweighting of machine learning models, with the improvement in the performance of information devices, it has become realistic to execute inference processing by AI on a single edge device. Furthermore, by combining the conventional execution in a remote environment and the local execution on an edge device, it has also become realistic to execute inference processing by AI on the edge device when responsiveness is required and in a remote environment when higher accuracy and versatility are required.

[0005] In Patent Document 1, a technique has been proposed in which the conventional execution in a remote environment and the local execution on an edge device are performed in parallel, and by obtaining the ensemble result of those inferences, a result in which each inference is harmonized is obtained.

Prior Art Documents

Patent Documents

[0006]

Patent Document 1

Summary of the Invention

[0007] When a device managed by one cloud service needs to be linked with another device managed by a different cloud service, it's common practice to link the devices between their respective cloud services rather than directly. The reason for this configuration is that, even after a device is released, linking with different types of devices becomes easy simply by updating the cloud service and making only minor changes to the device itself.

[0008] However, under this configuration, if the user's individual characteristics are clearly reflected in the machine learning models of other edge devices, and the response is obtained through inference, then coordinating via the cloud would result in a delay in obtaining the response.

[0009] This invention was made to solve the above-mentioned problems. The purpose of this invention is to provide a mechanism that enables edge devices, each equipped with artificial intelligence inference capabilities, to cooperate without degrading response performance and efficiently obtain inference results. [Means for solving the problem]

[0010] The present invention relates to a system including a plurality of edge devices, each edge device having an operation means for receiving user operations, a communication means for connecting to a network, and an inference means for performing inference processing of answer data to question data using a machine learning model, wherein at least one of the plurality of edge devices has a determination means for determining whether the inference processing of the question data should be performed by the inference means, another edge device, or a service that provides inference functionality to this edge device via the network, when question data is input from a client terminal that can communicate via the network or the operation means, a determination means for determining whether the inference processing of the question data should be performed by the inference means, another edge device, or a service that provides inference functionality to this edge device via the network, when it is determined that the inference processing should be performed by the other edge device, a transmission means for transmitting the question data to the other edge device without going through the service, and a presentation means for presenting the answer data to the question data that has been inferred and processed by the other edge device to the user via the client terminal or the operation means that is the source of the input of the question data, and the other edge device has a response means for inferring answer data to the question data using the inference means provided by the other edge device when it receives question data from another edge device, and responding with the answer data to the other edge device. [Effects of the Invention]

[0011] According to the present invention, edge devices, each equipped with artificial intelligence-based inference capabilities, can cooperate without degrading response performance to efficiently obtain inference results. [Brief explanation of the drawing]

[0012] [Figure 1] This figure shows an example of the overall configuration of the device inference system according to this embodiment. [Figure 2A] A block diagram showing an example of a typical hardware configuration for a device. [Figure 2B] A block diagram showing an example of a typical hardware configuration for a device. [Figure 2C] Block diagram showing an example of a general hardware configuration of a device management service and an application service. [Figure 3A] Block diagram showing an example of the software configuration of each component according to the present embodiment. [Figure 3B] Block diagram showing an example of the software configuration of each component according to the present embodiment. [Figure 4] Diagram showing a series of screens displayed on the display unit of the terminal in the device registration process according to the present embodiment. [Figure 5A] Diagram showing the sequence of registering a device to an application service. [Figure 5B] Diagram showing the sequence of registering a device to a device management service. [Figure 6A] Diagram showing an example of an inference request and an inference result display screen displayed on the display unit of the terminal. [Figure 6B] Diagram showing an example of an inference request and an inference result display screen displayed on the display unit of the terminal. [Figure 7A] Diagram showing an example of a sequence of performing inference processing in the present embodiment. [Figure 7B] Diagram showing an example of a sequence of performing inference processing in the present embodiment. [Figure 7C] Diagram showing an example of a sequence of performing inference processing in the present embodiment. [Figure 8] Diagram showing an example of a sequence of performing inference processing in the present embodiment. [Figure 9] Flowchart showing an example of a process for measuring the accuracy value of an inference model in the second embodiment.

Mode for Carrying Out the Invention

[0013] Hereinafter, embodiments for carrying out the present invention will be described with reference to the drawings. 〔First Embodiment〕 FIG. 1 is a diagram showing an example of the overall configuration of a device inference system according to an embodiment of the present invention.

[0014] Networks 100 to 101 are networks that connect each component of this system. Networks 100 to 101 are communication networks realized by, for example, LAN (Local Area Network), WAN (Wide Area Network), telephone lines, dedicated digital lines, ATM or frame relay lines, cable TV lines, wireless lines for data broadcasting, etc. Networks 100 to 101 can be of any type as long as data can be transmitted and received (communication is possible) between each component.

[0015] In this embodiment, in the connection between the device management service 102 and the application service A 103 and the application service B 104, the network 100 will be described as the Internet. Also, in the connection between the device A 106, the device B 107 and the terminal 105, the network 101 will be described as an intranet.

[0016] The device A 106 and the device B 1 are devices that have a function of connecting to the network 101 and a function of executing inference processing (inference processing by artificial intelligence (AI)) using a machine learning model alone. These devices include home appliances and office equipment. Specifically, home appliances include washing machines, air conditioners, refrigerators, and cooking appliances such as microwave ovens and ovens, while office equipment includes image forming machines. These devices may also be photographic devices such as still cameras and video cameras. These devices are equipped with the aforementioned functions in addition to their original functions. Specific functions include, for example, a washing machine estimating the completion time of washing, and an air conditioner inferring optimal temperature and humidity settings for drying laundry. Another example is an image forming machine inferring optimal print settings from the electronic data to be printed and user information. Cameras may also have functions to infer optimal shooting settings or to infer and analyze what the subject being photographed is. In this embodiment, the explanation uses the case where these devices are home appliances as an example, but the type of device is not limited as long as it satisfies the two conditions mentioned above: network connectivity and the ability to perform inference processing using machine learning models (artificial intelligence inference function). Hereinafter, these devices will also be referred to as devices or edge devices.

[0017] Furthermore, although devices A106 and B107 are described separately in this embodiment, they are considered to be different types of devices with different functions. It is assumed that each of these devices has the ability to perform network connectivity and inference processing using machine learning models. In addition, although only devices A106 and B107 are shown in the illustration, other such devices may also be included.

[0018] Terminal 105 is a client terminal that has the function of connecting to networks 100-101 and connects to devices A106, B107, and device management service 102. Specifically, it is a mobile terminal such as a smartphone, a tablet terminal, or a PC (personal computer). Terminal 105 may also be a smart speaker, which is a speaker equipped with network connectivity and an AI assistant capable of voice recognition and voice operation. In this embodiment, terminal 105 will be described as a smartphone, but the type of device is not limited as long as it has the aforementioned network connectivity and user interface functions.

[0019] Device management service 102 is a service that manages information about devices such as device A106 and device B107. Through user operation, terminal 105 registers information about devices A106 and B107, as well as information about terminal 105, with device management service 102. This allows the user to instruct the device to perform inference processing. Furthermore, application services A103 and B104 refer to this registered information to determine whether or not they can provide functionality to the target device.

[0020] Application services A103 and B104 are services that have the function of connecting to network 100 and the function of independently executing inference processing using machine learning models. Since these services exist according to the device, the service corresponding to device A is described as application service A, and the service corresponding to device B is described as application service B. Specifically, for example, if there are multiple devices of different types such as cooking appliances and air conditioning appliances, there will be application service A corresponding to cooking appliances and application service B corresponding to air conditioning appliances, and the respective functional services will be provided to devices A106 and B107. In addition, the application services also provide functional services to terminals 105 associated with devices in device management service 102.

[0021] In Figure 1, each component is shown as a single unit for the sake of simplicity in explaining this embodiment. However, there is no intention to limit or restrict the number of units of each element in terms of configuration. Each component may be composed of one or more units.

[0022] Figure 2A is a block diagram showing an example of a typical hardware configuration of device A106 and device B107 according to this embodiment. The CPU 200 starts the OS (Operating System) using the boot program stored in the ROM 201. The CPU 200 then executes various processes by running application programs stored in the external storage device 203 on the OS.

[0023] RAM202 is used as a workspace for CPU200. The external storage device 203 stores various programs such as the device application 330 and device control application 340 shown in Figure 3A (described later), machine learning models used by the inference execution unit 211 and the learning unit 212, and various data such as device settings and history information. The external storage device 203 can be an HDD (Hard Disk Drive) or an SSD (Solid State Drive).

[0024] The network unit 204 connects to the network 100 and communicates with each element that constitutes the device inference system in this embodiment. The CPU 200 is connected via the system bus 210 to the ROM 201, RAM 202, and external storage device 203, along with the network unit 204, operation interface 205, display interface 207, device control unit 209, inference execution unit 211, and learning unit 212.

[0025] The operation interface 205 is an interface that connects the device and the operation unit 206. The control unit 206 sends user input data received from a keyboard, hardware keys, microphone, etc., to the CPU 200 via the control interface 205.

[0026] The display I / F 207 is an interface that connects the device to the display unit 208. The display unit 208 outputs data sent from the CPU 200 via the display interface 207 using a display, speaker, etc. There are also units that combine the functions of both the operation unit 206 and the display unit 208, such as a touch panel, in which case they are connected to both interfaces 205 and 207.

[0027] The device control unit 209 controls the operation of the device for each process performed by the application program executed by the CPU 200. Assuming that device A106 is a microwave oven, for example, if the user selects the heating function, the CPU 200 sends control instructions via the device control unit 209 to the components necessary to perform the heating function, specifically the heater, fan, sensors, etc.

[0028] The inference execution unit 211 performs inference processing in the application program executed by the CPU 200. The inference execution unit 211 is a device suitable for inference processing using machine learning models, such as an NPU (Neural Processing Unit) or GPU (Graphics Processing Unit). Depending on the scale of the machine learning model, the inference processing may be performed by the CPU 200 instead of the inference execution unit 211.

[0029] The learning unit 212 obtains the results of the inference process performed by the inference execution unit 211 and performs a retraining process for the machine learning model stored in the external storage device 203. In this embodiment, details of the inference process and the retraining process of the machine learning model will not be mentioned.

[0030] Figure 2B is a block diagram showing an example of a typical hardware configuration of terminal 105 according to this embodiment. Since the basic components are the same as those of devices A106 and B107 described above, the explanation of overlapping components is omitted. Note that the external storage device 203 of terminal 105 stores programs such as the terminal application 320 shown in Figure 3A, which will be described later, instead of the device application 330 and device control application 340. Furthermore, while this embodiment assumes that terminal 105 is a mobile device such as a smartphone, it is not limited to this.

[0031] Figure 2C is a block diagram showing an example of a typical hardware configuration for the device management service 102, application service A103, and application service B104 according to this embodiment. Since the basic components are the same as those of devices A106 and B107 described above, the explanation of overlapping components will be omitted. Note that the external storage device 203 of the device management service 102 stores programs such as the device management application 300 shown in Figure 3B, instead of the device application 330 and device control application 340. Also, the external storage device 203 of application service A103 and application service B104 stores programs such as the service application 310 shown in Figure 3B, which will be described later, instead of the device application 330 and device control application 340.

[0032] Furthermore, a characteristic of each service, such as the device management service 102, application service A103, and application service B104, is that they exchange information with each element constituting the device inference system in this embodiment through communication via the network 100. Therefore, the display unit I / F 207 and display unit 208 are not essential for each service. Also, the device management service 102, application service A103, and application service B104 may be implemented by a single computer, by multiple computers, or may be cloud services implemented using cloud computing technology.

[0033] Figures 3A and 3B are block diagrams showing examples of the software configurations of each component according to this embodiment. Hereafter, Figures 3A and 3B will be collectively referred to as "Figure 3". Note that the block diagrams shown in Figure 3 only include software that is highly relevant to this embodiment. For example, assuming that device A106 is a microwave oven, there is actually software to implement device-specific functions such as heating and blowing applications, but these have been omitted.

[0034] The software configuration of the device management service 102 will be described below using Figure 3(a). In the software configuration shown in Figure 3(a), the CPU 200 reads and executes a program stored in either the RAM 202, ROM 201, or external storage device 203 of the device management service 102 shown in Figure 2C, thereby realizing those functions.

[0035] Device application 330 is an application that has the functionality to perform a series of inference processes on devices A106 and B107. Here, the explanation will focus on an example on device A106, with an example on device B107 shown in parentheses.

[0036] The communication unit 331 transmits and receives data between the terminal 105, the device management service 102, and the application service A103 (application service B104) via the network 100.

[0037] The determination unit 332, in response to the user's request (question data) received from the terminal 105 via the operation unit 206 or the network unit 204, determines, based on device registration information obtained from the terminal application 320 or device management application 300 (described later), which device—device A106 (device B107), application service A103 (application service B104), or device B107 (device A106)—is best suited for performing the inference process. If the determination unit 332 determines that inference on its own device is best, it sends an inference process execution request to the inference unit 333. If the determination unit 332 determines that inference on application service A103 (application service B104) is best, it sends an inference process execution request to the inference unit 312 of application service A103 (application service B104). Furthermore, if the determination unit 332 determines that inference on another device, device B107 (device A106), is optimal, it sends an inference processing execution request to the inference unit 333 of device B107 (device A106). The determination unit 332 also receives a user response from terminal 105 via the operation unit 206 or network unit 204 regarding the result of the inference processing performed by device A106, device B107, or application service A103, and determines whether to continue or interrupt the execution of the inference processing according to the user's response.

[0038] The inference unit 333 performs AI-based inference processing, for example, by using a machine learning model stored in the device's external storage device 203. The inference unit 333 displays the results of the inference processing on the display unit 208. The data management unit 334 manages the configuration information of devices A106 and B107, and also sends and receives configuration information to and from terminal 105 and device management service 102 via networks 100-101. Specific examples of configuration information will be described later.

[0039] Device control 340 is control software that operates on the equipment control unit 209 and controls the functions of the device. Specifically, if the device is a washing machine, it controls the washing process; if it is an air conditioning product, it controls the functions of adjusting the room temperature and humidity; and if it is an image forming apparatus, it controls the execution of the printing process.

[0040] The software configuration of terminal 105 will be described below using Figure 3(b). The software configuration shown in Figure 3(b) is realized when the CPU 200 reads and executes an application program stored in either the RAM 202, ROM 201, or external storage device 203 of terminal 105 shown in Figure 2B.

[0041] Terminal application 320 is an application on terminal 105 that has the function of instructing device A106 (device B107) to perform a series of inference processes. Terminal application 320 also has the function of sending and receiving information about device A106 and device B107 with device management service 102.

[0042] The communication unit 321 transmits and receives data between device A106, device B107, and device management service 102 via networks 100-101. The device management unit 322 searches for, registers, and manages devices A106 and B107, which are the communication targets of terminal 105. When registering a device, the device management unit 322 obtains the user information of the device owner as authentication information entered by the user and transmits it to the authentication application 302 of the device management application 300 of the device management service 102. Once authenticated, the device information is registered and managed. Information regarding the registered devices is managed by the data management unit 323. The device management unit 322 also transmits the device information managed by the data management unit 323 to the device management service 102 to share the data.

[0043] The software configuration of the device management service 102 will be described below using Figure 3(c). The software configuration shown in Figure 3(c) is realized when the CPU 200 reads and executes an application program stored in either the RAM 202, ROM 201, or external storage device 203 of the device management service 102 shown in Figure 2A.

[0044] The device management application 300 is an application program that has the functionality to perform a series of device management processes in the device management service 102. The communication unit 301 transmits and receives data between terminal 105 and devices A106 and B107 via the network 100-101. The authentication application 302 identifies the user who owns the device and performs authentication and authorization processing. The device user information is managed by the data management unit 304.

[0045] The device management unit 303 registers and manages information about users authenticated by the authentication application 302 and their owned devices, associating them with each other. In this embodiment, it is assumed that the device information registration process is performed by receiving data from the device management unit 322 of the terminal 105. However, device information may also be registered directly with the device management service 102 via a web browser or a dedicated application. The data management unit 304 manages user information and device information handled by the device management application 300.

[0046] The software configurations of application service A103 and application service B104 will be described below using Figure 3(d). The software configuration shown in Figure 3(d) is realized when the CPU 200 reads and executes an application program stored in either the RAM 202, ROM 201, or external storage device 203 of application service A103 or application service B104 as shown in Figure 2A.

[0047] Service application 310 is an application equipped with a function (inference unit 312) to perform inference processing for the corresponding device. Service application 310 of application service A103 is an application equipped with a function to perform inference processing for device A106. Service application 310 of application service B104 is an application equipped with a function to perform inference processing for device B107.

[0048] The device management unit 313 manages the device information of the corresponding device. The device management unit 313 of application service A103 manages the device information of device A106. The device management unit 313 of application service B104 manages the device information of device B107. The communication unit 311 transmits and receives data between device A106, device B107, terminal 105, and device management service 102 via networks 100-101.

[0049] The inference unit 312 receives a request from the determination unit 332 of the device application 330 and executes inference processing that is difficult to perform on the device. Inference processing that is difficult to perform on the device includes, for example, those that would take a long time to complete given the device's hardware performance. Generally, application services A103 and B104 often have better hardware performance than devices A106 and B107.

[0050] Figure 4 shows a series of screens displayed on the display unit 208 of the terminal 105 during the device registration process according to this embodiment.

[0051] <Device List Screen 400> The device list screen 400 is a screen that displays a list of device information registered in the terminal application 320. The device list screen 400 is displayed, for example, when the terminal application 320 is launched.

[0052] The device list display area 401 displays a list of device information registered in the terminal application 320. If no device information is registered, it will be displayed blank. In this embodiment, an example is shown where a list of display names that the user of the device can freely enter in the terminal application 320 is displayed, such as "Home washing machine," "Living room TV," and "Children's room air conditioner."

[0053] The device registration button 402 is a button that transitions to the device discovery screen 410. When button 402 is pressed, a series of device registration processes are started. The device editing button 403 is a button that transitions to the device information input screen 420. When button 403 is pressed while device information displayed in area 401 is selected, the screen transitions, and the selected device information becomes editable. The device delete button 404 is used to delete registered device information. When button 404 is pressed while device information displayed in area 401 is selected, the selected device information is deleted. At this time, the device information registered in the device management unit 322 is also deleted along with the display in the device list display area 401.

[0054] <Device discovery screen 410> The device discovery screen 410 is a screen for searching for devices that can be connected via the network 101.

[0055] When the search button 412 is pressed, the terminal application 320 sends a search request to all devices connected to the network 101 via the communication unit 321. The device discovery screen 410 is a screen that displays a list of the results of discovery requests received by the terminal application 320.

[0056] The search list display area 411 displays a list of device information included in the search request results. In this embodiment, for example, the model name, which is a fixed value for identifying the device, such as "refrigerator" or "air purifier," is displayed in a list for each device found in the search.

[0057] The information input button 413 is a button for transitioning to the device information input screen 420. When button 413 is pressed while device information is selected in area 411, the screen transitions, and it becomes possible to input the selected device information.

[0058] <Device Information Input Screen 420> The device information input screen 420 is a screen for users of a device to input additional information to easily identify the device information they register in the terminal application 320. This screen is also accessed when the device editing button 403 on the device list screen 400 is pressed.

[0059] The device information input area 421 includes a display name input field, a model name display field, and a location input field. The model name display field displays the model name, which is a fixed value used to identify the device, such as "refrigerator".

[0060] The display name input field and the location input field are fields where the user can enter a string of their choice. The display name input field allows the user to enter any string of text to be displayed on the device list screen 400. The location input field allows the user to enter any string of text to be entered to enter the installation location. These input fields are for entering information that makes it easy for the user to identify what the device is. Additionally, information that makes the device easier to use on the terminal application 320, such as who the device owner is, may also be entered.

[0061] In this embodiment, only field 421 is shown, but there is no intention to limit its number. For example, an input field such as the device's installation location may be displayed. The device registration button 422 is a button that transitions to the user authentication screen 430.

[0062] <User Authentication Screen 430> The user authentication screen 430 is where the device management service 102 performs authentication to verify whether the device owner has the authority to use the service.

[0063] User information input area 431 contains an account input field and a password input field. The account input field is for entering the account (e.g., username). The password input field is for entering the password.

[0064] When the authentication button 432 is pressed, the terminal application 320 sends an authentication request to the device management service 102. This authentication request includes the account (e.g., username) entered in the account input field of the user information input area 431 and the password entered in the password input field. When the device management service 102 receives the above authentication request, it performs authentication using the authentication application 302. If authentication is successful, the terminal application 320 registers the device information entered on the device information input screen 420 with the device management unit 322 and transitions the display unit 208 screen to the device list screen 400. If the device registration is successful, the device display name entered on the device information input screen 420 will be displayed in the device list display area 401 of the device list screen 400.

[0065] The sequence diagrams in Figures 5A and 5B below illustrate the flow of the device registration process related to this embodiment. Figure 5A shows an example of a sequence for registering devices A106 / B107 with application services A103 / B104.

[0066] In S501, the user sets network information for the network unit 204 of device A106 / B107, for example, from the operation unit 206 of device A106 / B107. If terminal 105 and device A106 / B107 can connect, for example via Bluetooth®, the network information may also be set for the network unit 204 of device A106 / B107 from terminal 105. Once the network unit 204 is configured, device A106 / B107 connects to network 101 through the network unit 204. If the network unit 204 is already configured, device A106 / B107 connects to network 101 at startup.

[0067] In S502, the device application 330 of device A106 / B107 transmits device information to the service application 310 of application service A103 / B104.

[0068] Table 1 shows an example of device information transmitted by S502. [Table 1] In Table 1, the serial column shows the serial number assigned to uniquely identify the device. The model name column is a column that shows the model name, which represents the type of device. The IP address column is the column that shows the IP address of the device.

[0069] Upon receiving this device information, the service application 310 registers the received device information with the device management unit 313. The above describes the registration sequence for each device and each device application service.

[0070] Figure 5B shows an example of a sequence in which a device found by discovery from terminal 105 is registered with the device management service 102. When the terminal application 320 of terminal 105 detects a press of the device registration button 402 on the device list screen 400 displayed on the operation unit 206 of terminal 105, it displays the device search screen 410 on the display unit 208. When the terminal application 320 detects a press of the search button 412 by the user, it proceeds to process S511.

[0071] In S511, the terminal application 320 sends a discovery request to all devices connected to the network 101. The sequence in Figure 5B shows the sending of discovery requests to devices A106 / B107, but in reality, discovery requests are sent to all devices connected to network 101.

[0072] In devices A106 / B107, when the data management unit 334 of the device application 330 receives the search request in S511, it proceeds to S512. In S512, devices A106 / B107 acquire device information managed by the data management unit 334 and transmit the device information to the terminal application 320. Table 2 shows an example of a search result that the data management unit 334 sends to the terminal application 320 as a response to a search request.

[0073] [Table 2] In Table 2, the serial column shows the serial number assigned to uniquely identify the device. The model name column is a column that shows the model name, which represents the type of device. The IP address column is the column that shows the IP address of the device. The service column contains information indicating the device application service that registered the device in S502.

[0074] When the terminal application 320 receives the above-mentioned device information response, it displays the received device information in the search list display area 411 of the search list screen 410. Furthermore, when the user selects a desired device from the search list display area 411 and presses the information input button 413, the terminal application 320 proceeds to process S513.

[0075] In S513, the terminal application 320 displays a device information input screen 420 to accept input of additional device details. The display name information and location information entered here are sent to the data management unit 323. Then, when the user presses the registration button 422, the terminal application 320 proceeds to process S514.

[0076] In S514, the terminal application 320 displays the user authentication screen 430 and accepts the user's authentication information. When the user presses the authentication button 432, the terminal application 320 proceeds to process S515.

[0077] In step S515, the terminal application 320 transmits the user authentication information entered on the user authentication screen 430 to the authentication application 302. Table 3 shows an example of authentication information that the terminal application 320 sends to the device management service 102.

[0078] [Table 3] The account column is a column that shows the username. The account (username) is issued when the application service A103 / B104 is contracted. In this embodiment, we will explain using an example where "UserA" has contracted for two application services A103 / B104, but the number of service contracts is not limited to two. The password column is the column that displays the password.

[0079] When the device management application 300 receives the authentication information in S515, it proceeds to process S516. In S516, the device management application 300 obtains the user authentication information received above, performs authentication processing with the authentication application 302, and responds with the authentication result to the terminal application 320.

[0080] If the authentication result in S516 is successful, the terminal application 320 proceeds to S517. In S517, the terminal application 320 sends the device information selected on the search list screen 410 and the additional device details entered on the device information input screen 420 to the device management service 102.

[0081] Table 4 shows an example of device registration information that the terminal application 320 transmits to the device management service 102. Note that the information shown in Table 4 is also data managed by the data management unit 323 of the terminal application 320.

[0082] [Table 4] The serial column is a column that displays the serial number assigned to uniquely identify the device. The model name column is a column that shows the model name, which represents the type of device. The service column contains information indicating the device application service that registered the device in step S502. The Display Name column and Location column are columns that display the arbitrary display name and location values ​​entered in field 421. The IP address column is the column that shows the IP address of the device. The account column is a column that indicates the account of the contracted user associated with the device.

[0083] When the device management application 300 receives the device information in S517 via the communication unit 301, it proceeds to process S518. In S518, the device management unit 303 of the device management application 300 stores the received device information in the data management unit 304.

[0084] Furthermore, in S519, the terminal application 320 sends the device registration information transmitted to the device management service 102 in S517, and the device management unit 322 stores this information in the data management unit 323. The terminal application 320 then displays the device list screen 400 on the display unit 208.

[0085] The series of inference processes of this embodiment will be explained below using Figures 6A, 6B, 7A to 7C, and 8. Figures 6A and 6B show examples of inference request and inference result display screens shown on the display unit 208 of terminal 105. Hereafter, Figures 6A and 6B will be collectively referred to as "Figure 6". Figures 7A to 7C and Figure 8 show an example of a sequence in which, in this embodiment, the determination unit 332 of device A106 determines which inference unit to execute the inference process, and the inference process is performed in the inference unit indicated by the determination result.

[0086] Figures 7A to 7C assume that the terminal application 320 of terminal 105 displays a screen as shown in Figure 6 on the display unit 208 of terminal 105, and that device A106 is selected and operated from the operation unit 206 of terminal 105. Figure 8 illustrates the case where a question is input using the operation unit 206 of device A106 and the inference result is output to the display unit 208. In the following explanation, we will assume that device A106 is a "washing machine" and device B107 is an "air conditioning product," but we are not limited to these.

[0087] In Figure 6, the device list screen 600 is a screen that displays a list of devices A106 / B107 managed by the device management unit 322 of the terminal application 320. On this screen, the user can select the desired device in the device list area 601 to transition to the inference request screen 610. Here, we will assume that device A106, which is the "house washing machine," has been selected for the following explanation.

[0088] The terminal application 320 displays the display name (in this case, "house washing machine") of the device registration information (Table 4) managed by the device management unit 322, corresponding to the device (in this case, device A106) selected by the user on the device list screen 600, at the top of the inference request screen 610. Furthermore, the terminal application 320 accepts user questions in the question input area 613 at the bottom of the inference request screen 610.

[0089] First, let's explain the sequence shown in Figure 7A. Figure 7A shows the sequence in which the determination unit 332 of device A106 determines where to perform the inference processing, and when device A106 performs the inference processing.

[0090] The user enters a question in the question input area 613 of the inference request screen 610 and presses the send button 614. When the terminal application 320 detects this operation, it displays the content entered in the question input area 613 on the inference request screen 610, as shown in the question display area 611. Furthermore, the device management unit 322 of the terminal application 320 receives the question information entered by the user in the question input area 613. The device management unit 322 then sends the question information, along with the device registration information (Table 4) list of all devices managed by the data management unit 323, to the selected device (in this case, device A106) via the communication unit 321 (S701).

[0091] When the communication unit 331 receives the device registration information (Table 4) list and query information for all devices, the device application 330 of device A106 proceeds to process S702. In this example, a configuration was described in which the device registration information (Table 4) list for all devices is received from the terminal application 320 along with the query information. However, a configuration in which the device management service 102 receives the query information from the device management application 300 in response to (or periodically) the receipt of the query information is also possible.

[0092] In S702, the device application 330 of device A106 inputs the received list of all device registration information (Table 4) and the question information as input parameters into the machine learning model at the determination unit 332, and determines where to perform the inference process. In the example in Figure 7A, it is explained that it is determined that the inference will be performed at the inference unit 333 of device A106 itself.

[0093] In S703, the inference unit 333 inputs the above-mentioned question information as input parameters to the machine learning model and performs inference processing for the question. Note that the machine learning model used by the determination unit 332 and the machine learning model used by the inference unit 333 may be the same or different. Next, in S704, the inference unit 333 responds to the terminal application 320, which is the source of the question data input, with the inference result via the communication unit 331.

[0094] When the terminal application 320 receives the inference result response from S704, it proceeds to processing S705. In S705, the terminal application 320 displays the received inference result on the display unit 208. Specifically, the terminal application 320 displays the inference result on the inference request screen 610, as shown in the inference result display area 612, as an answer to the question in the inference result input area 611, and presents it to the user.

[0095] Next, we will explain the sequence shown in Figure 7B. Figure 7B shows the sequence in which the determination unit 332 of device A106 determines where to perform the inference processing, and the application service A103 associated with device A106 performs the inference processing.

[0096] The user enters a question in the question input area 613 of the inference request screen 620 and presses the send button 614. When the terminal application 320 detects this operation, it displays the content entered in the question input area 613 on the inference request screen 620, as shown in the question display area 621. Furthermore, the device management unit 322 of the terminal application 320 receives the question information entered by the user in the question input area 613. The device management unit 322 then sends the question information, along with the device registration information (Table 4) list of all devices managed by the data management unit 323, to the selected device (in this case, device A106) via the communication unit 321 (S710).

[0097] When the communication unit 331 receives the device registration information (Table 4) list and query information for all devices, the device application 330 of device A106 proceeds to S711. In S711, the device application 330 of device A106 inputs the device registration information (Table 4) list information and question information received from all devices into the machine learning model as input parameters in the determination unit 332, and determines where to perform the inference process. In the example in Figure 7B, it is explained that it is determined that the inference will be performed in the inference unit 312 of application service A103.

[0098] In S712, the terminal application 320 sends the above question information to the application service A103. When application service A103 receives the question information from S712, it proceeds to process S713.

[0099] In S713, the inference unit 312 of application service A103 inputs the received question information as input parameters into the machine learning model and performs inference processing for the question. Next, in S714, the inference unit 312 transmits the inference result from S713 to the device application 330 of device A106.

[0100] When device application 330 of device A106 receives the inference result from S714, it proceeds to processing in S715. In step S715, the determination unit 332 of the device application 330 transmits the received inference result to the terminal application 320, which is the source of the question data input.

[0101] When the terminal application 320 receives the inference result from S715, it proceeds to process S716. In S716, the terminal application 320 presents the inference result to the user by displaying the received inference result on the display unit 208. Specifically, the terminal application 320 displays the inference result in the inference result display area 622 of the inference request screen 620, as an answer to the question in the inference result input area 621, similar to the inference result display area 623.

[0102] Next, we will explain the sequence shown in Figure 7C. Figure 7C shows the sequence when the determination unit 332 of device A106 determines where to perform the inference processing, and it determines that the inference processing will be performed on device B107, which is different from device A106.

[0103] The user enters a question in the question input area 613 of the inference request screen 630 / 640 and presses the send button 614. When the terminal application 320 detects this operation, it displays the content entered in the question input area 613 on the inference request screen 630 / 640, as shown in the question display area 631 / 641. Furthermore, the device management unit 322 of the terminal application 320 receives the question information entered by the user in the question input area 613. The device management unit 322 then sends the question information, along with the device registration information (Table 4) list of all devices managed by the data management unit 323, to the selected device (in this case, device A106) via the communication unit 321 (S720).

[0104] When the communication unit 331 receives the device registration information (Table 4) list and query information for all devices, the device application 330 of device A106 proceeds to process S721. In S721, the device application 330 of device A106 inputs the device registration information (Table 4) list information and question information received from all devices into the machine learning model using the determination unit 332 as input parameters, and determines where to perform the inference processing. Specifically, the determination unit 332 determines which device should perform the inference processing based on the question information and the information in the model name column and service column of the device registration information (Table 4) list for all devices. In the example in Figure 7C, it is explained that it has been determined that the inference should be performed in the inference unit 312 of device B107. Note that devices A106 and B107 may be on the same network as other devices. The result of the determination in S721 may result in a request for inference from both device B107 and yet another device (i.e., from multiple devices).

[0105] In S722, the terminal application 320 sends the above-mentioned question information to device B107. If the result of the determination in S721 indicates that inference is required from both device B107 and another device, the terminal application 320 sends the above-mentioned question information to both device B107 and the other device. When the device application 330 of device B107 receives the question information in S722, it proceeds to process S723.

[0106] In step S723, the inference unit 333 of device B107 inputs the received question information as input parameters into the machine learning model and performs inference processing for the question.

[0107] Next, in S724, the inference unit 333 of device B107 transmits the inference result from S723 to the device application 330 of device A106. Furthermore, if the inference result is to execute one of the device's functions, the inference unit 333 of device B107 sends an execution instruction to the device control 340. This execution instruction is processed by the device control unit 209. If the above query information has also been sent to another device, that other device also performs the same processing as in S723 and S724.

[0108] When device application 330 of device A106 receives the inference result from S724, it proceeds to process S725. In step S725, the determination unit 332 of the device application 330 transmits the received inference result to the terminal application 320 of terminal 105, which is the source of the question data input. If another device also receives the question information, the inference result from that other device is also transmitted to the terminal application 320 of terminal 105.

[0109] When the terminal application 320 of terminal 105 receives the inference result from S725, it proceeds to processing in S726. In S726, the terminal application 320 of terminal 105 displays the received inference result on the display unit 208 and presents it to the user. Specifically, the terminal application 320 displays the inference result in the inference result display area 632 of the inference request screen 630, as an answer to the question in the inference result input area 631, similar to the inference result display area 633. Furthermore, if the determination in S721 results in a request for inference from both device B107 and another device on the same network, the inference results will be displayed in the inference result display area 642 of the inference request screen 640, as shown in the inference result display areas 643 and 644, with the inference results being displayed as the answers from device B and the other device to the question in the inference result input area 641.

[0110] Figures 7A to 7C above illustrate the scenario where the terminal application 320 of terminal 105 displays a screen as shown in Figure 6 on the display unit 208 of terminal 105, and device A106 is selected and operated from the operation unit 206 of terminal 105. However, it is also possible to display a screen as shown in Figure 6 on the display unit 208 of device A106, and operate device A106 directly from the operation unit 206 of device A106.

[0111] Next, referring to Figure 8, we will explain the sequence when a question is entered at the operation unit 206 of device A106 and the inference result is output to the display unit 208. Figure 8 corresponds to the sequence when the determination unit 332 of device A106 determines where to perform the inference processing and determines that the inference processing will be performed at device B107, which is separate from device A106.

[0112] When a question is entered into the control panel 206 of device A106, the device application 330 of device A106 proceeds to process S801. In S801, the device application 330 obtains a list of device registration information (Table 4) for all devices from the device management application 300. The following steps S802 to S804 are processed in the same way as S721 to S723 in Figure 7C, so their explanation will be omitted.

[0113] Next, in S805, the inference unit 333 of device B107 transmits the inference result from S804 to the device management application 300. In addition, in S806, the inference unit 333 of device B107 transmits the inference result from S804 to the device application 330 of device A106. When device application 330 of device A106 receives the inference result from S724, it proceeds to process S807.

[0114] In S807, the device application 330 of device A106 displays the received inference result on the display unit 208 of device A106, which is the source of the question data input. The display screen at this time will be the same as the inference request screen 630 if inference is requested only from device B107, and the display screen will be the same as the inference request screen 640 if inference is requested from both device B107 and another device.

[0115] Furthermore, if device A106 is equipped with an audio output mechanism, in S808, the device application 330 of device A106 converts the received inference result into audio data and outputs the inference result as audio.

[0116] Note that, as shown in Figure 7A, when the determination result is that device A106 performs inference processing, the sequence diagram for when a question is input to the operation unit 206 of device A106 and the inference result is output to the display unit 208 is not shown, but it is as follows: In this case, instead of S803 to S807, the inference unit 333 of device A106 performs inference processing and displays the inference result on the operation unit of device A106. Furthermore, as shown in Figure 7B, when the result of the determination is to perform inference processing in application service A103 associated with device A106, the sequence diagram for when a question is input at the operation unit 206 of device A106 and the inference result is output to the display unit 208 is not shown, but it is as follows: In this case, instead of S803~S806, the sequence diagram shows that device A106 sends question information to application service A103, the inference unit 312 of application service A103 performs inference processing, and device A106 receives the inference result from application service A103.

[0117] Through the series of processes described above, the system determines which device or cloud service has the most suitable machine learning model for inference. If it determines that inference should be performed on another device, it directly queries that device to execute the inference. This allows for inference results using machine learning models that more accurately reflect the user's individuality, and also improves response performance.

[0118] It should be noted that the function of the determination unit 332 in the first embodiment does not need to be provided in all devices; it is sufficient if it is provided in at least one device. For example, device A106 may be provided with the function of the determination unit 332, but device B107 may not be provided with the function of the determination unit 332. Furthermore, when using a smart speaker as terminal 105, question data shall be input using voice input, and answer data shall be output using voice output.

[0119] [Second Embodiment] In the first embodiment described above, as shown in Figure 7C, there is a device A106 that determines where to perform inference, a device B107 that determines where to perform inference, and if there is another device on the same network, the determination unit 332 determines that inference should be performed on both devices, and in this example, inference is performed on both devices. Furthermore, if device B107 and the other device mentioned above are identical in both model and performance, it may suffice to perform inference on either device. In this case, the decision of which device to use for inference can be made based on the accuracy of the machine learning model, and the inference request should be sent to the device with the higher accuracy of the machine learning model. The data management and processing required to achieve this will be explained in detail below.

[0120] First, let's explain how to manage model accuracy values. Table 5 shows an example of a search result that the data management unit 334 of the device application 330 sends as a response to a search request from the terminal application 320 in the second embodiment.

[0121] [Table 5] The columns other than the model precision value are the same as those in Table 2, so their explanations are omitted. The model precision value is a number that represents the accuracy of the model. The closer this value is to 1, the better the accuracy.

[0122] Table 6 also shows an example of data managed by the device management unit 322 of the terminal application 320. This data is also an example of a list of device registration information for all devices that is sent along with the query information when an inference request is made. The device management unit 322 of the terminal application 320 periodically obtains the latest device registration information from the device management service 102 and updates the data managed by the device management unit 322 using the obtained data.

[0123] [Table 6] The columns other than the model accuracy value column are the same as those in Table 4, and the model accuracy value column is the same as the model accuracy column in Table 5, so its explanation is omitted.

[0124] The device management unit 322 of the terminal application 320 manages the model accuracy values ​​in a table like the one shown in Table 6. Then, using data like that in Table 6, the process of narrowing down the target for inference to a single option is as follows. The determination unit 332 of the terminal application 320 determines, for example, in S721 shown in Figure 7C, which device should be used for inference. If there are multiple devices of the same model, and the determination unit 332 determines that it is best to perform inference with one device, the determination unit 332 selects the device with the best model accuracy from among the devices included in the device registration information list (Table 6) of all devices sent from the terminal application 320 as the device to be used for inference.

[0125] Figure 9 is a flowchart showing an example of the process in which the determination unit 332 of the device application 330 running on each device A106, B107 periodically measures the accuracy value information of the machine learning model in the second embodiment and transmits it to the device management service 102. The process shown in this figure is realized by the CPU 200 of devices A106, B107 executing a program stored in the external storage device 203.

[0126] In S901, if the determination unit 332 of the device application 330 determines that it is a scheduled time, it proceeds to S902. In S902, the determination unit 332 determines whether the number of executions is less than a predetermined number (N times).

[0127] If the number of executions is less than a predetermined number (N times) (if the result is Yes in S902), the determination unit 332 proceeds to S903. In S903, the determination unit 332 of each device A106, B107 performs inference processing and returns processing to S902.

[0128] Then, when the number of executions reaches a predetermined number (N times) (i.e., the result is No in S902), the determination unit 332 proceeds to S904. In other words, the determination unit 332 repeatedly executes the inference process a specific number of times. The same question is used in the inference process each time.

[0129] In S904, the determination unit 332 calculates the accuracy rate of the results of repeating the same question a specific number of times, calculates this as the precision value, and proceeds to S905. In S905, the determination unit 332 transmits the accuracy value calculated in S904 to the device management service 102 via the communication unit 331. When the device management unit 303 of the device management service 102 receives the precision value from S905, it manages the precision value in the data management unit 304.

[0130] Furthermore, when question data is input (or periodically), the device determination unit 332 of each device acquires device information for each device, including the accuracy value, from the device management service 102 and uses it to determine which device performs the inference processing of the question data. If each device receives device information for each device, including the accuracy value, along with the question data from the client terminal 105, it may use the received device information. In this case, the client terminal 105 shall acquire the device information for each device from the device management service 102 before sending the question data to the device (or periodically). The client terminal 105 may also collect device information including the accuracy value from each device.

[0131] Using the accuracy values ​​obtained as described above, the device determination unit 332 determines that there are multiple devices of the same model and performance on the network and that inference can be performed on a single device, and then determines to send an inference request to the device with the highest accuracy value. This allows for efficient acquisition of inference results.

[0132] As described above, each embodiment makes it possible to obtain inference results from machine learning models that more accurately reflect the individuality of the user, and also to improve response performance. In other words, edge devices equipped with artificial intelligence inference capabilities can cooperate without degrading response performance to efficiently obtain inference results.

[0133] It should be noted that the structure and content of the various data described above are not limited to those mentioned, and it goes without saying that they can be composed of various structures and contents depending on the use and purpose. Although one embodiment has been described above, the present invention can take the form of, for example, a system, apparatus, method, program, or storage medium. Specifically, it may be applied to a system consisting of multiple devices, or to an apparatus consisting of a single device. Furthermore, any configurations combining the above embodiments are also included in the present invention.

[0134] [Other Embodiments] The present invention can also be realized by supplying a program that implements one or more of the functions of the above-described embodiments to a system or device via a network or storage medium, and by having one or more processors in the computer of that system or device read and execute the program. It can also be realized by a circuit (e.g., an ASIC) that implements one or more functions. Furthermore, the present invention may be applied to a system consisting of multiple devices or to a device consisting of a single device. The present invention is not limited to the embodiments described above, and various modifications (including organic combinations of each embodiment) are possible based on the spirit of the invention, and these are not excluded from the scope of the invention. That is, all configurations that combine the above-described embodiments and their modified forms are included in the present invention.

[0135] This embodiment includes the following configurations and methods. (Composition 1) A system including multiple edge devices, Each of the aforementioned edge devices is An operating means that accepts user input, A means of communication for connecting to a network, It includes an inference means that performs inference processing of answer data for question data using a machine learning model, At least one of the aforementioned multiple edge devices is When question data is input from a client terminal or the operating means that can communicate via the network, a determination means determines whether the inference processing for the question data should be performed on the inference means, another edge device, or a service that provides inference functionality to this edge device via the network. A transmission means that, when it is determined that inference processing should be performed on the other edge device, transmits the query data to the other edge device without going through the service, The system includes a presentation means that presents to the user, via the client terminal or the operating means that is the source of the input for the question data, the answer data to the question data that has been inferred and processed on the other edge device, which was received from the other edge device. The aforementioned other edge device is, The system includes a response means that, upon receiving question data from another edge device, infers answer data for the question data using inference means provided by the other edge device, and responds with the answer data to the other edge device. A system characterized by the following features. (Configuration 2) The system includes a management service for managing information about each of the edge devices, The at least one edge device has an acquisition means for acquiring information about each edge device from the management service, The system according to configuration 1, characterized in that the determination means uses the acquired information regarding each edge device for the determination. (Composition 3) The information relating to each edge device includes information indicating the accuracy of the inference processing performed by the inference means provided by each edge device. The information indicating the accuracy of the inference process included in the information about each edge device is registered with the management service based on the results of the inference process performed periodically on each edge device using the inference means provided by that edge device. The system according to configuration 2, characterized in that, when the determination means determines that inference processing should be performed on another edge device, the edge device among the other edge devices that has higher information indicating the accuracy of the inference processing is designated as the edge device to perform the inference processing. (Composition 4) The system according to Configuration 1, characterized in that when the determination means receives device information, which is information relating to each of the edge devices, from the client terminal, the determination means uses the received information relating to each of the edge devices for the determination. (Composition 5) The information relating to each edge device includes information indicating the accuracy of the inference processing performed by the inference means provided by each edge device. If the determination means determines that inference processing should be performed on another edge device, it selects the edge device among the other edge devices that has higher information indicating the accuracy of the inference processing as the edge device to perform the inference processing. The system according to configuration 4, characterized in that the information indicating the accuracy of the inference process included in the information about each edge device is based on the results of the inference process using the inference means provided by each edge device, which is performed periodically at each edge device. (Composition 6) The system according to any one of configurations 1 to 5, characterized in that each of the edge devices includes either a consumer electronics product or an office machine. (Composition 7) It is an edge device, An operating means that accepts user input, A means of communication for connecting to a network, An inference means that performs inference processing of response data to question data using a machine learning model, When question data is input from a client terminal or the operating means that can communicate via the network, a determination means determines whether the inference processing for the question data should be performed by the inference means, another edge device that can communicate with this edge device, or a service that provides inference functionality to this edge device via the network. A transmission means that, when it is determined that inference processing should be performed on the other edge device, transmits the query data to the other edge device without going through the service, A presentation means that presents to the user, via the client terminal or the operating means that is the source of the input for the question data, the answer data to the question data that has been inferred and processed on the other edge device, which has been received from the other edge device. An edge device characterized by having the following features. (Composition 8) The system includes an acquisition means for acquiring information about each edge device from a management service that manages information about the edge device and other edge devices, The edge device according to configuration 7, characterized in that the determination means uses the information about each edge device obtained for the determination. (Composition 9) The information relating to each edge device includes information indicating the accuracy of the inference processing performed by each edge device. The information indicating the accuracy of the inference process included in the information about each edge device is registered with the management service based on the results of the inference process performed periodically on each edge device using the inference means provided by that edge device. The edge device according to configuration 8, characterized in that, when the determination means determines that inference processing should be performed on another edge device, the edge device among the other edge devices that has higher information indicating the accuracy of the inference processing is designated as the edge device to perform the inference processing. (Composition 10) The edge device according to configuration 7, characterized in that when the determination means receives device information, which is information relating to each of the edge devices, from the client terminal, the determination means uses the received information relating to each of the edge devices for the determination. (Composition 11) The information relating to each edge device includes information indicating the accuracy of the inference processing performed by each edge device. If the determination means determines that inference processing should be performed on another edge device, it selects the edge device among the other edge devices that has higher information indicating the accuracy of the inference processing as the edge device to perform the inference processing. The edge device according to configuration 10, characterized in that the information indicating the accuracy of the inference process included in the information about each edge device is based on the results of the inference process performed periodically by each edge device. (Composition 12) The edge device according to any one of configurations 6 to 11, characterized in that the edge device is either a home appliance or an office machine. (Method 1) A control method for a system including multiple edge devices having an operating means for receiving user input, a communication means for connecting to a network, and an inference means for performing inference processing of answer data to question data using a machine learning model, Executed by at least one edge device among the aforementioned plurality of edge devices, When question data is input from a client terminal or the operating means that can communicate via the network, a determination step is made to determine whether the inference processing for the question data should be performed on the inference means, another edge device, or a service that provides inference functionality to this edge device via the network. A transmission step in which, if it is determined that inference processing should be performed on the other edge device, the query data is transmitted to the other edge device without going through the service, A presentation step of presenting to the user, via the client terminal or the operating means that is the source of the input for the question data, the answer data to the question data that has been inferred and processed by the other edge device, which has been received from the other edge device, The execution performed by the aforementioned other edge device, When question data is received from another edge device, the inference means provided by the other edge device infers answer data for the question data and responds with the answer data to the other edge device in a response step, A method for controlling a system characterized by having the following features. (Method 2) A control method for an edge device having an operating means for receiving user input, a communication means for connecting to a network, and an inference means for performing inference processing of answer data to question data using a machine learning model, When question data is input from a client terminal or the operating means that can communicate via the network, a determination step is made to determine whether the inference processing for the question data should be performed by the inference means, another edge device that can communicate with this edge device, or a service that provides inference functionality to this edge device via the network. A transmission step in which, if it is determined that inference processing should be performed on the other edge device, the query data is transmitted to the other edge device without going through the service, A presentation step of presenting to the user, via the client terminal or the operating means that is the source of the input for the question data, the answer data to the question data that has been inferred and processed by the other edge device, which has been received from the other edge device, A method for controlling an edge device, characterized by having the following features.

Claims

1. A system including multiple edge devices, Each of the aforementioned edge devices is An operating means that accepts user input, A means of communication for connecting to a network, It includes an inference means that performs inference processing of answer data for question data using a machine learning model, At least one of the aforementioned multiple edge devices is When question data is input from a client terminal or the operating means that can communicate via the network, a determination means determines whether the inference processing for the question data should be performed on the inference means, another edge device, or a service that provides inference functionality to this edge device via the network. A transmission means that, when it is determined that inference processing should be performed on the other edge device, transmits the query data to the other edge device without going through the service, The system includes a presentation means that presents to the user, via the client terminal or the operating means that is the source of the input for the question data, the answer data to the question data that has been inferred and processed on the other edge device, which was received from the other edge device. The aforementioned other edge device is, The system includes a response means that, upon receiving question data from another edge device, infers answer data for the question data using inference means provided by the other edge device, and responds with the answer data to the other edge device. A system characterized by the following features.

2. The system includes a management service for managing information about each of the edge devices, The at least one edge device has an acquisition means for acquiring information about each edge device from the management service, The system according to claim 1, characterized in that the determination means uses the information about each edge device obtained for the determination.

3. The information relating to each edge device includes information indicating the accuracy of the inference processing performed by the inference means provided by each edge device. The information indicating the accuracy of the inference process included in the information about each edge device is registered with the management service based on the results of the inference process performed periodically on each edge device using the inference means provided by that edge device. The system according to claim 2, characterized in that, when the determination means determines that inference processing should be performed on another edge device, the edge device among the other edge devices that has higher information indicating the accuracy of the inference processing is designated as the edge device to perform the inference processing.

4. The system according to claim 1, characterized in that when the determination means receives device information, which is information relating to each of the edge devices, from the client terminal, the determination means uses the received information relating to each of the edge devices for the determination.

5. The information relating to each edge device includes information indicating the accuracy of the inference processing performed by the inference means provided by each edge device. If the determination means determines that inference processing should be performed on another edge device, it selects the edge device among the other edge devices that has higher information indicating the accuracy of the inference processing as the edge device to perform the inference processing. The system according to claim 4, characterized in that the information indicating the accuracy of the inference process included in the information about each edge device is based on the results of an inference process performed periodically on each edge device using the inference means provided by the edge device.

6. The system according to any one of claims 1 to 5, characterized in that each of the edge devices includes either a consumer electronics product or an office machine.

7. It is an edge device, An operating means that accepts user input, A means of communication for connecting to a network, An inference means that performs inference processing of response data to question data using a machine learning model, When question data is input from a client terminal or the operating means that can communicate via the network, a determination means determines whether the inference processing for the question data should be performed by the inference means, another edge device that can communicate with this edge device, or a service that provides inference functionality to this edge device via the network. A transmission means that, when it is determined that inference processing should be performed on the other edge device, transmits the query data to the other edge device without going through the service, A presentation means that presents to the user, via the client terminal or the operating means that is the source of the input for the question data, the answer data to the question data that has been inferred and processed on the other edge device, which has been received from the other edge device. An edge device characterized by having the following features.

8. The system includes an acquisition means for acquiring information about each edge device from a management service that manages information about the edge device and other edge devices, The edge device according to claim 7, characterized in that the determination means uses the information about each edge device obtained for the determination.

9. The information relating to each edge device includes information indicating the accuracy of the inference processing performed by each edge device. The information indicating the accuracy of the inference process included in the information about each edge device is registered with the management service based on the results of the inference process performed periodically on each edge device using the inference means provided by that edge device. The edge device according to claim 8, characterized in that, when the determination means determines that inference processing should be performed on another edge device, the edge device among the other edge devices that has higher information indicating the accuracy of the inference processing is designated as the edge device to perform the inference processing.

10. The edge device according to claim 7, characterized in that when the determination means receives device information which is information relating to each of the edge devices from the client terminal, the determination means uses the received information relating to each of the edge devices for the determination.

11. The information relating to each edge device includes information indicating the accuracy of the inference processing performed by each edge device. If the determination means determines that inference processing should be performed on another edge device, it selects the edge device among the other edge devices that has higher information indicating the accuracy of the inference processing as the edge device to perform the inference processing. The edge device according to claim 10, characterized in that the information indicating the accuracy of the inference process included in the information about each edge device is based on the results of the inference process performed periodically by each edge device.

12. The edge device according to any one of claims 6 to 11, characterized in that the edge device is either a home appliance or an office machine.

13. A control method for a system including multiple edge devices having an operating means for receiving user input, a communication means for connecting to a network, and an inference means for performing inference processing of answer data to question data using a machine learning model, Executed by at least one edge device among the aforementioned plurality of edge devices, When question data is input from a client terminal or the operating means that can communicate via the network, a determination step is made to determine whether the inference processing for the question data should be performed on the inference means, another edge device, or a service that provides inference functionality to this edge device via the network. A transmission step in which, if it is determined that inference processing should be performed on the other edge device, the query data is transmitted to the other edge device without going through the service, A presentation step of presenting to the user, via the client terminal or the operating means that is the source of the input for the question data, the answer data to the question data that has been inferred and processed by the other edge device, which has been received from the other edge device, The execution performed by the aforementioned other edge device, When question data is received from another edge device, the inference means provided by the other edge device infers answer data for the question data and responds with the answer data to the other edge device in a response step, A method for controlling a system characterized by having the following features.

14. A control method for an edge device having an operating means for receiving user input, a communication means for connecting to a network, and an inference means for performing inference processing of answer data to question data using a machine learning model, When question data is input from a client terminal or the operating means that can communicate via the network, a determination step is made to determine whether the inference processing for the question data should be performed by the inference means, another edge device that can communicate with this edge device, or a service that provides inference functionality to this edge device via the network. A transmission step in which, if it is determined that inference processing should be performed on the other edge device, the query data is transmitted to the other edge device without going through the service, A presentation step of presenting to the user, via the client terminal or the operating means that is the source of the input for the question data, the answer data to the question data that has been inferred and processed by the other edge device, which has been received from the other edge device, A method for controlling an edge device, characterized by having the following features.

Images