Sign sensing device, sign sensing system, server device, sign sensing method, and sign sensing program

Abstract

A sign detection device (100) uses sensor data (51) and a local model (52) that have been acquired with respect to a sensing target to sense signs in the sensing target. A screen display unit (130) displays an information selection screen to allow a user to select a data type. The information selection screen presents: a data classification of information to be transmitted to a server device that updates the local model (52); data types belonging to the data classification; and explanatory information that explains advantages and disadvantages of transmitting information corresponding to the data types to the server device. The explanatory information includes advantages and disadvantages related to security. An information transmission unit (140) transmits, to the server device, information corresponding to the data type selected by the user using the information selection screen.

Description

Sign detection device, sign detection system, server device, sign detection method, and sign detection program

[0001] The present disclosure relates to a sign detection device, a sign detection system, a server device, a sign detection method, and a sign detection program.

[0002] An information processing system has been provided that uses a determination model trained by machine learning to suit the application and determines signs of a measurement target according to the application. Patent Literature 1 discloses a system that detects disease or its signs by using biological information measured by a wearable device such as a smart watch and medical information measured in a hospital.

[0003] International Publication No. 2021 / 075289

[0004] The technology of Patent Document 1 generates a determination model through machine learning using first measurement data measured by a medical device and second measurement data measured by a non-medical device. The determination model is then used to determine whether a user being measured has a disease or a symptom thereof as a prognosis. The technology of Patent Document 1 is based on a centralized system, and models and data are aggregated in a parent server.

[0005] However, there are users who do not want to send collected vital data (biological information) to a parent server such as a cloud. Such users cannot use the system of Patent Document 1, and various data measured about such users cannot be collected. Furthermore, if data collection is insufficient, the accuracy of the judgment model will not improve. As such, there is a trade-off between the security level desired by the user and the accuracy of predictive detection, and it is difficult to appropriately balance the two.

[0006] The predictive detection device of the present disclosure aims to appropriately balance the security level desired by the user with the accuracy of predictive detection by presenting explanatory information about security for each data classification and then allowing the user to select the information to send.

[0007] The sign detection device according to the present disclosure is an sign detection device that detects signs in a detection target using sensor data acquired about the detection target and a local model, and includes: a screen display unit that displays an information selection screen that presents a data classification indicating a classification of information to be sent to a server device that updates the local model, a data type belonging to the data classification, and explanatory information that explains advantages and disadvantages of sending information corresponding to the data type to the server device, the explanatory information including advantages and disadvantages related to security, and allows a user to select a data type as information to be sent to the server device via the information selection screen; and an information transmission unit that transmits information corresponding to the data type selected by the user via the information selection screen to the server device.

[0008] The sign detection device according to the present disclosure can present explanatory information about security for each data category and then allow the user to select the information to send. Thus, the sign detection device according to the present disclosure has the effect of being able to appropriately balance the security level desired by the user with the accuracy of sign detection.

[0009] 1 is a diagram showing an example of the overall configuration of a sign detection system according to embodiment 1. FIG. 2 is a diagram showing an example of a basic processing flow in the sign detection system according to embodiment 1. FIG. 3 is a diagram showing an example of the configuration of a sign detection device according to embodiment 1. FIG. 4 is a diagram showing an example of the configuration of a server device according to embodiment 1. FIG. 5 is a diagram showing another example of the configuration of a sign detection device according to embodiment 1. FIG. 6 is a flow diagram showing an example of the operation of the sign detection system according to embodiment 1. FIG. 7 is a diagram showing an example of the configuration of a transmission necessity screen according to embodiment 1. FIG. 8 is a diagram showing an example of the configuration of an information selection screen in a terminal having a model update function according to embodiment 1. FIG. 9 is a diagram showing an example of the configuration of an information selection screen in a terminal not having a model update function according to embodiment 1. FIG. 10 is a schematic diagram for explaining explanatory information presented on the transmission necessity screen and the information selection screen according to embodiment 1. FIG. 11 is a schematic diagram for explaining explanatory information presented on the transmission necessity screen and the information selection screen according to embodiment 1. FIG. 12 is a diagram showing an example of the configuration of a sign detection device according to a modified example of embodiment 1.

[0010] The present embodiment will be described below with reference to the drawings. In each drawing, the same or corresponding parts are assigned the same reference numerals. In the description of the embodiment, the description of the same or corresponding parts will be omitted or simplified as appropriate. The arrows in the drawings mainly indicate the flow of data or the flow of processing.

[0011] Embodiment 1. *** Description of Configuration *** FIG. 1 is a diagram showing an example of the overall configuration of a sign detection system 500 according to this embodiment. The sign detection system 500 according to this embodiment includes a sign detection device 100 and a server device 200. A sign detection device 100 is provided for each detection target 10. In the sign detection system 500 according to this embodiment, the sign detection device 100 detects a sign in the detection target 10 using sensor data acquired about the detection target 10 and a local model. In this embodiment, the detection target 10 is a person for whom a disease or a symptom thereof is to be detected as a sign. In other words, the sign detection device 100 detects a disease or a symptom thereof in the person of the detection target 10 as a sign using the sensor data acquired about the detection target 10 and a local model.

[0012] The server device 200 generates or updates a parent model using information transmitted from each sign detection device 100. The server device 200 transmits the updated parent model to each sign detection device 100, thereby updating the local model of each sign detection device 100. The server device 200 is configured, for example, from a cloud. The server device 200 is also referred to as a parent server. The sign detection devices 100 and the server device 200 communicate with each other via a network 300.

[0013] 2 is a diagram showing an example of the basic processing flow in the sign detection system 500 according to this embodiment. The sign detection device 100 is a user's terminal, which is the detection target 10. For example, the sign detection device 100 is a device capable of calculation processing, such as a wearable terminal such as a smart watch, a smartphone, a notebook PC, a desktop PC, or a microcomputer. PC is an abbreviation for personal computer.

[0014] The sensor data 51 is various data acquired about the detection target 10. The sensor data 51 is information that can be input to the local model. The sensor data 51 includes general measurement data such as biometric data and other signal data. The sensor data 51 is information sensed by a wearable device worn by or carried by the detection target 10, or various sensor devices such as a camera and a microphone located near the detection target 10. Specifically, the sensor data 51 includes biometric information sensed by a smartwatch or the like (e.g., heart rate data, pulse rate, blood pressure, body temperature, respiratory rate, blood oxygen concentration, level of consciousness, etc.), a behavior log, and information such as sound, image, temperature, and vibration measured by various sensors. The sensor data 51 also includes information obtained by non-contact sensors such as a camera, an infrared sensor, or a driver monitor.

[0015] The local model 52 is a machine learning model that is generated or updated by machine learning in the server device 200 using information transmitted from each sign detection device 100. The local model 52 is provided in the sign detection device 100. The parent model 41 is a machine learning model managed by the server device 200. The parent model 41 is updated by the server device 200 and distributed to each sign detection device 100. The parent model 41 is configured by a machine learning model such as a neural network, for example. The server device 200 updates the parent model 41 using information transmitted from each sign detection device 100. The server device 200 transmits the parent model 41 to each sign detection device 100, thereby updating the local model 52 of each sign detection device 100.

[0016] An example of the basic processing flow of the sign detection system 500 according to this embodiment will be described with reference to FIG. 2 . (1) Each user's terminal (sign detection device 100), which is the detection target 10, stores a machine learning model for sign estimation as a local model 52. The local model 52 may be downloaded from the server device 200 or may be pre-stored. (2) The sign detection device 100 receives sensor data 51, such as sensed biometric information, as input and performs sign detection processing using the local model 52. (3) If the sign detection device 100 has the function of updating the local model 52, it updates the local model 52 using the sensor data 51 stored in the storage unit 150. In the case of a neural network, the parameters of the local model 52 are updated using, for example, a stochastic gradient descent method. (4) If the sign detection device 100 has the function of updating the local model 52, it transmits only information related to the update to the server device 200. Hereinafter, information related to the update will be referred to as update information. The server device 200 updates the parent model 41 using the update information transmitted from the sign detection devices 100. In the case of a neural network, the update information may be parameters of the local model 52 that represents the network, or gradient information associated with parameter updates. In the case of a neural network, the update method may be, for example, federated averaging. In the federated averaging method, the parameters transmitted from each sign detection device 100 are weighted by the number of data points and averaged to obtain the parameters of the parent model 41.

[0017] For example, by performing the above processes (1) to (4), the parent model 41 can be updated without transmitting highly confidential information to the server device 200. Furthermore, by receiving the updated parent model 41, the local model 52 can be updated.

[0018] Fig. 3 is a diagram showing an example of the configuration of the sign detection device 100 according to this embodiment. Fig. 4 is a diagram showing an example of the configuration of the server device 200 according to this embodiment.

[0019] The sign detection device 100 is a computer. The sign detection device 100 includes a processor 910, as well as other hardware such as a memory 921, an auxiliary storage device 922, an input interface 930, an output interface 940, and a communication device 950. The device may also include various sensor devices. The processor 910 is connected to the other hardware via signal lines and controls the other hardware.

[0020] The sign detection device 100 includes, as functional elements, a sensor data acquisition unit 110, a detection unit 120, a screen display unit 130, an information transmission unit 140, and a storage unit 150. The storage unit 150 stores sensor data 51, a local model 52, update information 53, and attribute information 54.

[0021] The server device 200 is a computer. The server device 200 includes a processor 910, as well as other hardware such as a memory 921, an auxiliary storage device 922, an input interface 930, an output interface 940, and a communication device 950. The processor 910 is connected to the other hardware via signal lines and controls the other hardware.

[0022] The server device 200 includes, as functional elements, an update unit 210, a model transmission unit 220, and a storage unit 240. The storage unit 240 stores a parent model 41.

[0023] Hereinafter, the individual devices of the sign detection device 100 and the server device 200 may be referred to as individual devices of the sign detection system 500. For simplicity of explanation, the same reference numerals are used to denote the hardware of the individual devices of the sign detection system 500, but it is clear that each device of the sign detection system 500 has its own individual hardware. Also, below, the hardware will be explained using the sign detection device 100 as an example. The same explanation as for the sign detection device 100 can also be applied to the hardware of the server device 200.

[0024] The functions of the sensor data acquisition unit 110, the detection unit 120, the screen display unit 130, and the information transmission unit 140 are realized by software. The storage unit 150 is provided in the memory 921. Note that the storage unit 150 may be provided in the auxiliary storage device 922, or may be provided separately in the memory 921 and the auxiliary storage device 922.

[0025] The processor 910 is a device that executes a sign detection program. The sign detection program is a program that realizes the functions of the sensor data acquisition unit 110, the detection unit 120, the screen display unit 130, and the information transmission unit 140. The processor 910 is an IC that performs arithmetic processing. Specific examples of the processor 910 are a CPU, a DSP, and a GPU. IC is an abbreviation for Integrated Circuit. CPU is an abbreviation for Central Processing Unit. DSP is an abbreviation for Digital Signal Processor. GPU is an abbreviation for Graphics Processing Unit.

[0026] The memory 921 is a storage device that temporarily stores data. Specific examples of the memory 921 are SRAM and DRAM. SRAM is an abbreviation for Static Random Access Memory. DRAM is an abbreviation for Dynamic Random Access Memory. The auxiliary storage device 922 is a storage device that saves data. A specific example of the auxiliary storage device 922 is an HDD. The auxiliary storage device 922 may also be a portable storage medium such as an SD (registered trademark) memory card, CF, NAND flash, flexible disk, optical disk, compact disk, Blu-ray (registered trademark) disk, or DVD. Note that HDD is an abbreviation for Hard Disk Drive. SD (registered trademark) is an abbreviation for Secure Digital. CF is an abbreviation for CompactFlash (registered trademark). DVD is an abbreviation for Digital Versatile Disk.

[0027] The input interface 930 is a port connected to an input device such as a mouse, keyboard, or touch panel. The input interface 930 is also a port connected to various sensor devices. Specifically, the input interface 930 is a USB terminal. The input interface 930 may also be a port connected to a LAN. USB is an abbreviation for Universal Serial Bus. LAN is an abbreviation for Local Area Network. Although one input interface 930 is shown in FIG. 1 , multiple input interfaces 930 may be present.

[0028] The output interface 940 is a port to which a cable of an output device such as a display is connected. Specifically, the output interface 940 is a USB terminal or an HDMI (registered trademark) terminal. Specifically, the display is an LCD. The output interface 940 is also called a display interface. HDMI (registered trademark) is an abbreviation for High Definition Multimedia Interface. LCD is an abbreviation for Liquid Crystal Display. Although one output interface 940 is shown in FIG. 1 , multiple output interfaces 940 may be present.

[0029] The communication device 950 has a receiver and a transmitter. The communication device 950 is connected to a communication network such as a LAN, the Internet, a telephone line, or Wi-Fi (registered trademark). Specifically, the communication device 950 is a communication chip or NIC. NIC is an abbreviation for Network Interface Card.

[0030] The sign detection program is executed in the sign detection device 100. The sign detection program is loaded into the processor 910 and executed by the processor 910. In addition to the sign detection program, the memory 921 also stores an OS. OS is an abbreviation for Operating System. The processor 910 executes the sign detection program while running the OS. The sign detection program and the OS may be stored in an auxiliary storage device 922. The sign detection program and the OS stored in the auxiliary storage device 922 are loaded into the memory 921 and executed by the processor 910. Note that part or all of the sign detection program may be incorporated into the OS.

[0031] The sign detection device 100 may include multiple processors that replace the processor 910. These multiple processors share the task of executing the sign detection program. Each processor is a device that executes the sign detection program in the same way as the processor 910.

[0032] The data, information, signal values ​​and variable values ​​used, processed or output by the symptom detection program are stored in the memory 921, the auxiliary storage device 922, or a register or cache memory within the processor 910.

[0033] The "unit" of each of the sensor data acquisition unit 110, the detection unit 120, the screen display unit 130, and the information transmission unit 140 may be interpreted as a "circuit," a "step," a "procedure," a "process," or a "circuitry." The sign detection program causes a computer to execute a sensor data acquisition process, a detection process, a screen display process, and an information transmission process. The "processing" of the sensor data acquisition process, the detection process, the screen display process, and the information transmission process may be interpreted as a "program," a "program product," a "computer-readable storage medium storing a program," or a "computer-readable recording medium recording a program." Furthermore, the sign detection method is a method performed by the sign detection device 100 executing the sign detection program. The sign detection program may be provided by being stored in a computer-readable recording medium. Furthermore, the sign detection program may be provided as a program product.

[0034] FIG. 5 is a diagram showing another example of the configuration of the sign detection device 100 according to the present embodiment. The sign detection device 100 shown in FIG. 3 is a high-spec terminal that has the function of updating the local model 52. The update information 53 is information that is used when updating the local model 52. On the other hand, the sign detection device 100 shown in FIG. 5 does not have the function of updating the local model 52. Therefore, the sign detection device 100 shown in FIG. 5 does not have the update information 53 that is used when updating the local model 52. The rest of the configuration is the same as that of the sign detection device 100 shown in FIG. 4.

[0035] ***Description of Operation*** Next, the operation of each device of the sign detection system 500 according to this embodiment will be described. The operating procedure of each device of the sign detection system 500 corresponds to a sign detection method. Furthermore, the program that realizes the operation of each device of the sign detection system 500 corresponds to a sign detection program.

[0036] FIG. 6 is a flow diagram showing an example of the operation of the sign detection system 500 according to this embodiment.

[0037] <Display Process of Transmission Necessity Screen 61> In step S101, the screen display unit 130 displays the transmission necessity screen 61, which allows the user to select whether or not to transmit information to the server device 200. The transmission necessity screen 61 is a UI (User Interface) screen. The screen display unit 130 displays the transmission necessity screen 61 before displaying the information selection screen 62, which allows the user to select information to transmit to the server device 200. The transmission necessity screen 61 presents explanatory information 611 that explains the advantages and disadvantages of each case of whether or not to transmit information to the server device 200. The explanatory information 611 includes advantages and disadvantages related to security.

[0038] 7 is a diagram showing an example of the configuration of a transmission necessity screen 61 according to this embodiment. The transmission necessity screen 61 presents options for whether or not to transmit information to the server device 200. The transmission necessity screen 61 also presents explanatory information 611 that explains the advantages and disadvantages of selecting each option. The explanatory information 611 includes advantages and disadvantages related to security. Specifically, the following is provided.

[0039] The transmission necessity screen 61 presents "Option 1: No information will be sent," which is an option for not sending information to the server device 200, and explanatory information 611 that explains the advantages and disadvantages of not sending information to the server device 200. The transmission necessity screen 61 also presents "Option 2: Send information," which is an option for sending information to the server device 200, and explanatory information 611 that explains the advantages and disadvantages of sending information to the server device 200.

[0040] The explanatory information 611 on the transmission necessity screen 61 will be described in detail later.

[0041] If it is selected not to transmit information to the server device 200 (NO in step S102), the process ends. If it is selected to transmit information to the server device 200 (YES in step S102), the process proceeds to step S103.

[0042] <Display process of information selection screen 62> The screen display unit 130 displays the information selection screen 62 that allows the user to select information to be transmitted to the server device 200. The information selection screen 62 is a UI screen. The screen display unit 130 determines whether the device itself has a function for updating the local model 52, and displays the information selection screen 62 according to the determination result.

[0043] The information selection screen 62 presents a data classification 621, a data type 622 belonging to the data classification 621, and explanatory information 623 that explains the advantages and disadvantages of transmitting information corresponding to the data type 622 to the server device 200. The explanatory information 623 includes advantages and disadvantages related to security. The data classification 621 indicates the classification of information to be transmitted to the server device 200, which updates the local model 52. The screen display unit 130 prompts the user to select the data type 622 as information to be transmitted to the server device 200 via the information selection screen 62. Specifically, this is as follows:

[0044] In step S103, the screen display unit 130 determines whether the sign detection device 100, which is the device itself, has the function of updating the local model 52. Whether the sign detection device 100 has the function of updating the local model 52 refers to whether the specifications of the sign detection device 100 are such that the device can perform learning processing on its own. If it is determined that the sign detection device 100 has the function of updating the local model 52 (YES in step S103), the process proceeds to step S104. If it is determined that the sign detection device 100 does not have the function of updating the local model 52 (NO in step S103), the process proceeds to step S105.

[0045] In step S104, the screen display unit 130 displays the information selection screen 62 in the case where the device has the function of updating the local model 52. In addition, in step S105, the screen display unit 130 displays the information selection screen 62 in the case where the device does not have the function of updating the local model 52. The information selection screen 62 is a screen that allows the user to select information to be transmitted to the server device 200.

[0046] 8 is a diagram showing an example of the configuration of the information selection screen 62 in a terminal having the model update function according to this embodiment. When the screen display unit 130 determines that the terminal has the function to update the local model 52, it presents, on the information selection screen 62, data classifications 621 consisting of update classifications, attribute classifications, and detailed classifications, as well as data types 622 and explanation information 623 belonging to each data classification.

[0047] The update classification in the data classification 621 is a classification to which update information used to update the local model 52 belongs. Information belonging to the update classification is information that can be acquired when the sign detection device 100 is a high-spec terminal with a model update function. The attribute classification in the data classification 621 is a classification to which information indicating the attributes of the detection target 10 belongs. For example, information belonging to the attribute classification is information such as the gender, height, weight, or age of the detection target 10. The detailed classification in the data classification 621 is a classification to which sensor data acquired about the detection target 10 belongs. For example, information belonging to the detailed classification is information such as heart rate data, activity log, blood oxygen level, face image, or voice. Specifically, the information is as follows.

[0048] In FIG. 8, "Information for model update" is displayed as the update classification, "User information" as the attribute classification, and "Data details" as the detailed classification. The user can select each classification. For the update classification "Information for model update," the user can choose whether or not to send the "Information for model update." Since the "Information for model update" is update information used to update the local model 52, the user cannot select its details. Furthermore, explanatory information 623 is presented, explaining the advantages and disadvantages of selecting the update classification "Information for model update."

[0049] For the attribute classification "user information," data types 622 such as gender, height / weight, age, and medical history can be selected. An option "send all" for sending all data types belonging to "user information" may also be presented. Furthermore, explanatory information 623 is presented that explains the advantages and disadvantages of selecting the attribute classification "user information."

[0050] For the detailed classification "Data Details," data types 622 such as heart rate data, activity log, blood oxygen level, face image, and audio can be selected. An option "Send All" for sending all data types belonging to "Data Details" may also be presented. Furthermore, explanatory information 623 is presented that explains the advantages and disadvantages of selecting the detailed classification "Data Details."

[0051] The explanation information 623 on the information selection screen 62 will be explained in detail later.

[0052] 9 is a diagram showing an example of the configuration of the information selection screen 62 in a terminal that does not have the model update function according to this embodiment. When the screen display unit 130 determines that the terminal does not have the function to update the local model 52, the screen display unit 130 presents, on the information selection screen 62, attribute classifications and detailed classifications, as well as data types 622 and explanation information 623 that belong to each data classification.

[0053] If the sign detection device 100 is a terminal that does not have a model update function, the sign detection device 100 is unable to acquire the update information used to update the local model 52. Therefore, the sign detection device 100 presents only options for information related to attribute classification and detailed classification on the information selection screen 62, and does not present options for information related to update classification.

[0054] Information regarding update classification is not presented in the configuration example of the information selection screen 62 in Fig. 9. Information regarding other attribute classifications and detailed classifications is the same as that of the information selection screen 62 in Fig. 8.

[0055] <Transmitting Information to Server Device 200> In step S106, the information transmitting unit 140 transmits, to the server device 200, information corresponding to the data type 622 selected by the user via the information selection screen 62. When the update classification "Information for Model Update" is selected, the information transmitting unit 140 transmits, to the server device 200, update information used to update the local model 52. This process is similar to the example (4) of the basic process flow in FIG. 2 . When the attribute classification "User Information" is selected, the information transmitting unit 140 transmits, to the server device 200, information corresponding to the data type selected by the user from among the data types 622 belonging to "User Information." For example, all or part of the attribute information of the detection target 10, such as the gender, height / weight, age, and medical history, is transmitted to the server device 200. When the detailed classification "Data Details" is selected, the information transmitting unit 140 transmits, to the server device 200, information corresponding to the data type selected by the user from among the data types 622 belonging to "Data Details." For example, all or part of the sensor data, such as heart rate data, activity log, blood oxygen concentration, face image, and voice of the detection target 10, is transmitted to the server device 200. The sensor data is collected by the sensor data acquisition unit 110 from various sensor devices.

[0056] <Updating the Parent Model 41 in the Server Device 200> In step S107, the update unit 210 of the server device 200 updates the parent model 41 using the information transmitted from the sign detection device 100. The update unit 210 updates the parent model 41 depending on the type of information transmitted from the sign detection device 100.

[0057] <Transmitting the parent model 41 from the server device 200 to the sign detection device 100> In step S108, the model transmission unit 220 of the server device 200 distributes the parent model 41 to each sign detection device 100. When the parent model 41 is distributed from the server device 200, each sign detection device 100 updates its local model 52 by replacing it with the parent model 41 distributed from the server device 200.

[0058] <Sign Detection Process> In step S109 , the detection unit 120 detects a sign of the detection target 10 using the sensor data 51 and the local model 52 .

[0059] 6, the parent model 41 is updated (step S107), the updated parent model 41 is transmitted (step S108), and a sign is detected (step S109). However, the process of updating the parent model 41, the process of transmitting the updated parent model 41, and the process of detecting a sign may be executed asynchronously.

[0060] 10 and 11 are schematic diagrams for explaining explanatory information presented on the transmission necessity screen 61 and the information selection screen 62 according to the present embodiment. Examples of explanatory information presented on the transmission necessity screen 61 and the information selection screen 62 will be described using Fig. 7 to Fig. 11. Here, the sign detection device may be referred to as a terminal in the following description.

[0061] In a predictive detection system that performs predictive detection processing using a local model and sensor data on a device, highly confidential information is not sent to a server device. However, because a local model is used within the device, devices without a model update function, such as smartwatches or low-spec smartphones, cannot retrain or fine-tune the model using sensor data. As a result, performance does not improve from the parent model. Furthermore, for devices with a model update function, such as PCs with GPUs or high-spec smartphones, performance can be expected to improve by retraining and fine-tuning the model using sensor data on the device. However, if the parent model is changed, the updates made through retraining and fine-tuning are reset.

[0062] <Explanatory Information on the Transmission Necessity Screen 61> First, an example of explanatory information on the transmission necessity screen 61 will be described. The advantage of "Option 1: No information will be sent" is that security is completely ensured. Therefore, as shown in FIG. 7, "Security is completely ensured" is displayed.

[0063] On the other hand, the drawbacks of "Option 1: No information is sent" vary depending on whether the terminal has a model update function, as shown in the upper part of Figure 10. Without the model update function, the provided parent model is simply used as the local model, meaning that model performance will not improve until a new parent model is provided. Furthermore, with the model update function, the local model in the terminal can be updated using only the user's own sensor data. However, there is a drawback in that the update is reset when the parent model is changed. As shown in Figure 7, the explanatory information may include a description of drawbacks both with and without the model update function. Alternatively, the screen display unit may first determine whether the model update function is available and then present drawbacks depending on whether the model update function is available.

[0064] The advantages of "Option 2: Send information" are presented as "Frequent updates to the estimation model are expected to improve estimation accuracy" and "Data analysis results related to the user can be obtained," as shown in FIG. 7. On the other hand, "Option 2: Send information" has the disadvantage of security in that periodic data communication is performed. Therefore, as shown in FIG. 7, "Periodic data communication will be performed" is presented. Note that the transmission necessity screen 61 may be configured to present "*Data communication frequency can be changed from the settings," allowing the data communication frequency setting to be changed from the transmission necessity screen 61.

[0065] <Explanatory Information of Information Selection Screen 62> Next, the explanatory information of the information selection screen 62 will be described.

[0066] <<In the case of the option "Information for model update">> As described above, the option "Information for model update" is only presented on terminals with a model update function. When "Information for model update" is sent, the local model is updated within the terminal, and update information accompanying the model update is sent to the server device. The server device updates the parent model based on the update information and distributes it to the terminal. This process allows the local model using information sensed by the terminal to be updated via the latest parent model. Therefore, the update reset does not occur. Furthermore, since the local model can be updated based on the latest parent model, accuracy is further improved.

[0067] 8, the advantage of sending "information for model update" is presented as "Frequent updates of the estimation model are expected to improve estimation accuracy." On the other hand, the disadvantage of sending "information for model update" is presented as "periodic data communication will be performed." Note that it may also be configured to present "*Data communication frequency can be changed from settings" so that the data communication frequency setting can be changed from the information selection screen 62.

[0068] <<In the case of the option "User Information">> In the case of the option "User Information," attribute information indicating the attributes of the detection target, such as gender, age, body shape information, or medical history, is transmitted. The data type of attribute information to be transmitted can be selected individually. By transmitting attribute information, it becomes possible to provide a model and analysis results trained on data of people with similar characteristics to the user or similar cases. As shown in the upper part of Figure 11, there is a concern that personal information may be transmitted, but the level of personal information is not very high and useful results are fed back to the user.

[0069] 8 and 9, the advantages of sending "user information" are presented as "You can compare your results with people similar to you based on the information you select" and "The model's performance will be improved based on data from users similar to you." Also, the disadvantages of sending "user information" are presented as "We will take security into consideration, but your data will remain in the cloud."

[0070] <<When "Data Details" is selected as the option>> When "Data Details" is selected as the option, sensor data including waveform information of the sensed information, or information such as indices and features calculated from the waveform, is transmitted. The data type of sensor data to be transmitted can be selected individually. This makes it possible to provide detailed analysis results of the sensor data obtained from the detection target. Furthermore, as shown in the lower part of Figure 11, even if the terminal does not have a model update function, the sensor data of the device itself is reflected in the parent model. Therefore, updating the parent model improves performance.

[0071] 8 and 9, the advantages of sending "Detailed Data" are presented as "We will provide detailed data analysis results for the selected information" and "We will improve the model performance based on the selected information." In addition, the disadvantages of sending "Detailed Data" are presented as "We will take security into consideration, but your data will remain in the cloud."

[0072] If the terminal has a model update function, the model can be updated within the terminal without sending "data details." Therefore, for terminals with model update function, it may be possible to present a message such as "*If model updates are possible within the terminal, model updates are possible without using this method," as shown in Figure 11.

[0073] If the terminal has a model update function, the update classification, attribute classification, and detailed classification can be used in any combination. The selection is also left to the user. By presenting the advantages of each selection on the UI, it is possible to make the user's data selection easier. If the terminal does not have a model update function, the attribute classification and detailed classification can be used in any combination. The selection is also left to the user. By presenting the advantages of each selection on the UI, it is possible to make the user's data selection easier.

[0074] ***Other Configurations*** <Variation 1> In the present embodiment, an example has been described in which a person is the target of sign detection and signs of disease or the like are detected using biometric data, other signal data, and measurement data in general. In Variation 1 of the present embodiment, the sign detection device may be used to detect signs of abnormalities or failures in production equipment in a factory or the like as the detection target.

[0075] A controller connected to production equipment is used as a predictive detection device, and a local model for estimation is stored. The predictive detection device then uses the local model to input sensor data from failure detection sensors (e.g., microphones, vibration sensors, cameras, infrared sensors, etc.) and performs predictive detection processing. The user selects one of the following representative cases and receives feedback or services based on the information sent to the server device. The user may be an operator who operates the equipment or an administrator who manages the equipment. For highly confidential information, only the downloaded local model is used. For highly confidential information, only update information for the local model belonging to the update classification is sent to improve the estimation accuracy specific to the equipment. For model or age information belonging to the attribute classification is sent to refer to information from similar cases. For detailed sensor data, performance is constantly improved.

[0076] <Modification 2> In the present embodiment, an example has been described in which the functions of each device in the sign detection system 500 are implemented by software. As a modification 2, the functions of each device in the sign detection system 500 may be implemented by hardware. Specifically, each device in the sign detection system 500 has an electronic circuit 909 instead of a processor 910.

[0077] FIG. 12 is a diagram showing an example of the configuration of a sign detection device 100 according to a modified example of this embodiment. The electronic circuit 909 is a dedicated electronic circuit that realizes the functions of the sensor data acquisition unit 110, the detection unit 120, the screen display unit 130, and the information transmission unit 140. Specifically, the electronic circuit 909 is a single circuit, a composite circuit, a programmed processor, a parallel programmed processor, a logic IC, a GA, an ASIC, or an FPGA. GA is an abbreviation for Gate Array. ASIC is an abbreviation for Application Specific Integrated Circuit. FPGA is an abbreviation for Field-Programmable Gate Array.

[0078] The functions of the sensor data acquisition unit 110, the detection unit 120, the screen display unit 130, and the information transmission unit 140 may be realized by a single electronic circuit, or may be realized by distributing them across multiple electronic circuits.

[0079] As another modification, some of the functions of sensor data acquisition unit 110, detection unit 120, screen display unit 130, and information transmission unit 140 may be realized by electronic circuits, and the remaining functions may be realized by software. Also, some or all of the functions of sensor data acquisition unit 110, detection unit 120, screen display unit 130, and information transmission unit 140 may be realized by firmware.

[0080] Each of the processor and electronic circuit is also referred to as processing circuitry. In other words, the functions of the sensor data acquisition unit 110, the detection unit 120, the screen display unit 130, and the information transmission unit 140 are realized by the processing circuitry. Note that the same explanation as in Fig. 12 can also be applied to the server device 200 in Fig. 4 and the sign detection device 100 in Fig. 5. In other words, by providing an electronic circuit 909 instead of the processor 910, the functions can be realized in hardware.

[0081] ***Description of Effects of the Present Embodiment*** In the symptom detection system according to the present embodiment, a system is constructed in which data is distributed rather than aggregated. In this embodiment, a symptom detection system is constructed that allows real-time updates of models without aggregating data in a server device. This makes it possible to realize applications that do not require highly confidential information to be sent to a server device. Furthermore, in this embodiment, the user's wishes can be reflected by allowing the user to specify the information to be sent to the server device. Furthermore, in this embodiment, incentives according to the level of information to be sent to the server device can be presented, thereby increasing the user's motivation to send data.

[0082] The sign detection system according to this embodiment presents security concerns about sending highly confidential information and the benefits of sending that information. This makes it possible to provide an information selection screen that makes it easy for the user to select the highly confidential information to send. Furthermore, the information to be received and the information and services to be provided can be changed according to the disclosure level of the highly confidential information selected by the user. In this way, the sign detection system according to this embodiment presents explanatory information about security to the user, and then allows the user to select the type of data to be sent to the server device. Therefore, the sign detection system according to this embodiment has the effect of being able to appropriately balance the security level desired by the user and the accuracy of sign detection.

[0083] In the first embodiment described above, each section of each device of the sign detection system has been described as an independent functional block. However, the configuration of each device of the sign detection system does not have to be the same as that of the above-described embodiment. The functional blocks of each device of the sign detection system may have any configuration as long as they can realize the functions described in the above-described embodiment. Furthermore, each device of the sign detection system may not be a single device, but may be a system composed of multiple devices. Furthermore, multiple parts of the first embodiment may be combined to implement the system. Alternatively, only one part of the first embodiment may be implemented. In addition, the first embodiment may be combined in any way, either as a whole or in part. That is, in the first embodiment, parts of the first embodiment may be freely combined, or any component of the first embodiment may be modified, or any component of the first embodiment may be omitted.

[0084] The above-described embodiments are essentially preferred examples and are not intended to limit the scope of the present disclosure, the scope of application of the present disclosure, or the scope of use of the present disclosure. The above-described embodiments can be modified in various ways as needed. For example, the procedures described using flow charts or sequence diagrams may be modified as appropriate.

[0085] 10 Detection target, 41 Parent model, 51 Sensor data, 52 Local model, 53 Update information, 54 Attribute information, 61 Transmission necessity screen, 611, 623 Explanation information, 62 Information selection screen, 621 Data classification, 622 Data type, 100 Sign detection device, 110 Sensor data acquisition unit, 120 Detection unit, 130 Screen display unit, 140 Information transmission unit, 150, 240 Storage unit, 200 Server device, 210 Update unit, 220 Model transmission unit, 300 Network, 500 Sign detection system, 909 Electronic circuit, 910 Processor, 921 Memory, 922 Auxiliary storage device, 930 Input interface, 940 Output interface, 950 Communication device.

Claims

1. An early warning detection device that detects signs in a detection target using sensor data and a local model acquired about the detection target, the early warning detection device comprising: a screen display unit that displays an information selection screen presenting a data classification indicating a classification of information to be sent to a server device that updates the local model, a data type belonging to the data classification, and explanatory information explaining advantages and disadvantages of sending information corresponding to the data type to the server device, the explanatory information including advantages and disadvantages related to security, and allows a user to select a data type as information to be sent to the server device via the information selection screen; and an information transmission unit that transmits information corresponding to the data type selected by the user via the information selection screen to the server device.

2. The screen display unit of the predictive detection device of claim 1 displays a transmission necessity screen that allows the user to select whether or not to send information to the server device before displaying the information selection screen, and terminates processing when it is selected not to send information to the server device, and displays the information selection screen when it is selected to send information to the server device.

3. The predictor detection device of claim 2, wherein the screen display unit presents explanatory information on the transmission necessity screen that explains the advantages and disadvantages of each case of whether or not to transmit information to the server device, the explanatory information including advantages and disadvantages regarding security.

4. The data classification includes an update classification to which update information used to update the local model belongs, an attribute classification to which attributes of the detection object belong, and a detailed classification to which sensor data acquired about the detection object belongs, and the screen display unit determines whether or not the device has a function to update the local model, and displays the information selection screen in accordance with the determination result. An early warning detection device as described in any one of claims 1 to 3.

5. The predictive detection device of claim 4, wherein when the screen display unit determines that it has the function of updating the local model, it presents the update classification, the attribute classification, and the detailed classification, as well as the data type and explanatory information belonging to each classification, on the information selection screen.

6. The predictive detection device of claim 4, wherein when the screen display unit determines that it does not have the function of updating the local model, it presents the attribute classification and the detailed classification, as well as the data type and explanatory information belonging to each classification, on the information selection screen.

7. An indication detection system comprising an indication detection device that detects indications in a detection target using sensor data acquired about the detection target and a local model, and a server device that updates the local model, wherein the indication detection device comprises: a screen display unit that displays an information selection screen presenting a data classification that indicates a classification of information to be transmitted to the server device, a data type belonging to the data classification, and explanatory information that describes advantages and disadvantages of transmitting information corresponding to the data type to the server device, and allows a user to select a data type as information to be transmitted to the server device via the information selection screen; and an information transmission unit that transmits to the server device information corresponding to the data type selected by the user via the information selection screen, and the server device comprises: an update unit that updates a parent model using the information transmitted from the indication detection device, and a model transmission unit that transmits the parent model to the indication detection device as the updated local model.

8. The sign detection system of claim 7, wherein the data classification includes an update classification to which update information used to update the local model belongs, an attribute classification to which attributes of the detection object belong, and a detailed classification to which sensor data acquired about the detection object belongs, and wherein the update unit, upon receiving update information belonging to the update classification from the sign detection device, updates the parent model using the update information, upon receiving information corresponding to a data type belonging to the attribute classification, updates the parent model using the information corresponding to the data type belonging to the attribute classification, and upon receiving information corresponding to a data type belonging to the detailed classification, updates the parent model using the information corresponding to the data type belonging to the detailed classification.

9. A server device that communicates with an early detection device that detects signs in a detection target using sensor data and a local model acquired about the detection target and updates the local model, the server device comprising: a screen display unit that displays an information selection screen presenting a data classification indicating a classification of information to be transmitted to the server device, a data type belonging to the data classification, and explanatory information that describes advantages and disadvantages of transmitting information corresponding to the data type to the server device, and allows a user to select a data type as information to be transmitted to the server device via the information selection screen; an information transmission unit that transmits information corresponding to the data type selected by the user via the information selection screen to the server device; and an update unit that updates a parent model using information transmitted from the early detection device; and a model transmission unit that transmits the parent model to the early detection device as the updated local model.

10. An indication detection method used by a computer that detects indications in a detection target using sensor data and a local model acquired about the detection target, the method comprising: displaying an information selection screen presenting a data classification indicating a classification of information to be sent to a server device that updates the local model, a data type belonging to the data classification, and explanatory information explaining advantages and disadvantages of sending information corresponding to the data type to the server device, having a user select a data type as information to be sent to the server device via the information selection screen; and transmitting information corresponding to the data type selected by the user to the server device by the computer.

11. An indication detection program used in an indication detection device that detects indications in a detection target using sensor data and a local model acquired about the detection target, the indication detection program causing the indication detection device, which is a computer, to execute the following steps: a screen display process that displays an information selection screen presenting a data classification indicating a classification of information to be transmitted to a server device that updates the local model, a data type belonging to the data classification, and explanatory information that explains the advantages and disadvantages of transmitting information corresponding to the data type to the server device, and allows a user to select a data type as information to be transmitted to the server device via the information selection screen; and an information transmission process that transmits information corresponding to the data type selected by the user via the information selection screen to the server device.

Images