Information processing device, machine learning device, inference device, information processing method, machine learning method, inference method, and lighting fixture.

Abstract

To provide an information processing device capable of detecting accidents in bathrooms with high accuracy. [Solution] The information processing device comprises a dynamic information acquisition unit that acquires dynamic information within the bathroom, a static information acquisition unit that acquires static information within the bathroom, a bathtub distance information acquisition unit that acquires bathtub distance information relating to the position of the water surface in the bathtub, and a bathing state estimation unit that estimates the state of a bather by inputting bathing state inference model input information based on the dynamic information, the static information, and the bathtub distance information into a bathing state inference model. The bathing state inference model is a trained inference model that has learned the correlation between the bathing state inference model input information and information relating to the state of the bather through machine learning.

Description

Technical Field

[0001] The present invention relates to an information processing apparatus, a machine learning apparatus, an inference apparatus, an information processing method, a machine learning method, an inference method, and a lighting fixture.

Background Art

[0002] A system for detecting the occurrence of accidents indoors has been developed. For example, Document 1 discloses a system for estimating an abnormal situation of a person in a room based on the opening / closing state of a door at an entrance / exit to the room.

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0004] On the other hand, in the conventional technology, there has been a problem that false alarms occur in detecting accidents in the bathroom.

[0005] The present invention has been made by paying attention to the above problems, and an object thereof is to provide an information processing apparatus, a machine learning apparatus, an inference apparatus, an information processing method, a machine learning method, an inference method, and a lighting fixture capable of detecting accidents in a bathroom with high accuracy.

Means for Solving the Problems

[0006] To achieve the above objective, an information processing device according to one aspect of the present invention comprises: a dynamic information acquisition unit that acquires dynamic information in a bathroom; a static information acquisition unit that acquires static information in a bathroom; a bathtub distance information acquisition unit that acquires bathtub distance information relating to the position of the water surface in the bathtub; and a bathing state estimation unit that estimates the state of a bather by inputting bathing state inference model input information based on the dynamic information, the static information, and the bathtub distance information into a bathing state inference model. The bathing state inference model is a trained inference model that has learned the correlation between the bathing state inference model input information and information relating to the state of the bather by machine learning. [Effects of the Invention]

[0007] According to one aspect of the present invention, an information processing device can detect accidents in a bathroom immediately with high accuracy.

[0008] Other issues, configurations, and effects will be clarified in the embodiments for carrying out the invention described later. [Brief explanation of the drawing]

[0009] [Figure 1] This is an overall configuration diagram showing an overview of the bathtub sinking detection system 1 according to an embodiment. [Figure 2] This is a diagram showing the configuration of the lighting fixture 100. [Figure 3] This is a block diagram showing the functions of the information processing device 40. [Figure 4] This is an explanatory diagram showing the processing of the inference model learning unit 413. [Figure 5] This is an explanatory diagram showing the processing of the bathroom information processing unit 414. [Figure 6] This figure shows an example of dynamic information D11. [Figure 7] This figure shows an example of static information D12 and bathtub distance information D13. [Figure 8] This diagram shows the hardware configuration of computer 900. [Figure 9]This is a flowchart illustrating an example of the operation (information processing method) of the information processing device 40. [Modes for carrying out the invention]

[0010] Hereinafter, embodiments for carrying out the present invention will be described with reference to the drawings. In the following, the scope necessary for explaining how to achieve the objectives of the present invention will be schematically shown, and the scope necessary for explaining the relevant parts of the present invention will be mainly explained, with any parts that are omitted from explanation being based on prior art.

[0011] Figure 1 is an overall configuration diagram showing an overview of the bathtub sinking detection system 1 according to an embodiment. Figure 2 is a diagram showing the configuration of the lighting fixture 100.

[0012] The Bathtub Submersion Detection System 1 is a system for detecting a submersion state in a bathtub, which occurs as an accident in the bathroom R of a house H. A submersion state is defined as a condition in which the bather's face or nose is in contact with the water surface or submerged below the water surface due to fainting or falling asleep while bathing. The Bathtub Submersion Detection System 1 comprises a lighting fixture 100 installed in the bathroom R and a user terminal device 200. The lighting fixture 100 and the user terminal device 200 are connected to a wired or wireless network N and configured to send and receive various types of data to and from each other. Note that the number of information processing devices 40 and user terminal devices 200 and the connection configuration of the network N are not limited to the example in Figure 1 and may be changed as appropriate.

[0013] The lighting fixture 100 has the function of so-called edge AI, which operates artificial intelligence on an edge device installed at the end of the network. The information processing device 40 of the lighting fixture 100 acquires information about the bathroom while bathing and detects whether or not the bather U1 is submerged.

[0014] If the information processing device 40 estimates that a sinking has occurred, it will notify the user terminal device 200 used by the emergency contact user U2 of an alert. The emergency contact user U2 may be a family member living with or separately from the bather U1, or a security company, etc.

[0015] The user terminal device 200 is a client-type computer and is a general-purpose terminal such as a smartphone, a tablet terminal, a personal computer, or a wearable device. Hereinafter, the details of the lighting fixture 100 will be described.

[0016] (Lighting fixture 100) As shown in FIG. 2, the lighting fixture 100 includes a light-emitting unit 10, an imaging device 20, a depth data sensor 30, and an information processing device 40.

[0017] The light-emitting unit 10 is a component that converts the electric power supplied from the power source into light and irradiates the surroundings, and functions as lighting in the bathroom. For example, an LED chip, a fluorescent lamp, an incandescent lamp, a halogen lamp, etc. are used for the light-emitting unit 10.

[0018] The imaging device 20 is a camera that acquires imaging data (including still images or moving images) in the bathroom, receives the light reflected or radiated from the subject, and records the information as image data on a recording medium. Note that the imaging device 20 may have a function of a thermography camera that acquires an image regarding the temperature distribution in the bathroom as imaging data.

[0019] The depth data sensor 30 is a sensor that detects the distance to an object in space as depth data. For example, a stereo camera, a ToF (Time of Flight) sensor, a LiDAR (Light Detection and Ranging) sensor, etc. are used for the depth data sensor 30.

[0020] The information processing device 40 is an edge device including a control unit such as a microcomputer, and is composed of a general-purpose or dedicated computer (see FIG. 9 described later) or the like.

[0021] (Configuration of the information processing device 40) Figure 3 is a block diagram showing an example of an information processing device 40. The information processing device 40 comprises a control unit 41, a data storage unit 42, a learned model storage unit 43, and a communication unit 44.

[0022] The communication unit 44 is connected to an external device (for example, a user terminal device 200, etc.) via the network N and functions as a communication interface for sending and receiving various types of data.

[0023] The data storage unit 42 stores a database 421 and an information processing program 422. Multiple bathroom information D10 (i.e., dynamic information D11, static information D12, and bathtub distance information D13) are registered in the database 421. Details of the bathroom information D10 will be described later.

[0024] The trained model storage unit 43 stores a trained bathing state inference model 431. The bathing state inference model 431 is a trained inference model that has learned the correlation between the bathing state inference model input information and the bathing state information D15 through machine learning. Details about the bathing state information D15 will be described later. The bathing state inference model 431 stored in the trained model storage unit 43 may be provided to other devices via a network N or recording medium. Furthermore, the number of bathing state inference models 431 stored in the trained model storage unit 43 is not limited to one each; for example, multiple inference models with different conditions, such as differences in machine learning methods or data, may be stored and used selectively or in parallel.

[0025] In Figure 3, the data storage unit 42 and the trained model storage unit 43 are shown as two separate storage units, but they may be composed of a single storage unit or three or more storage units. Furthermore, although the data storage unit 42 and the trained model storage unit 43 are assumed to be implemented on an edge device, at least one of them may be configured as a storage unit on an external computer (for example, a server computer or a cloud computer).

[0026] The control unit 41 functions as a transmission / reception control unit 411, a database management unit 412, an inference model learning unit 413, and a bathroom information processing unit 414 by executing the information processing program 422 recorded in the data storage unit 42.

[0027] (Transmit / receive control unit 411) The transmission / reception control unit 411 transmits and receives various types of data to and from external devices (e.g., a user terminal device 200). For example, the transmission / reception control unit 411 sends display information to the user terminal device 200 to output an alert notification. The transmission / reception control unit 411 also works in conjunction with the database management unit 412, the inference model learning unit 413, and the bathroom information processing unit 414 to transmit and receive various types of information.

[0028] (Database Management Department 412) The database management unit 412 registers the bathroom information D10 (i.e., dynamic information D11, static information D12, and bathtub distance information D13) acquired by the bathroom information processing unit 414 into the database 421. This increases the number of data entries in the database 421, thereby improving the accuracy of the trained model generated in the trained model storage process described later.

[0029] (Inference model learning unit 413) Figure 4 is an explanatory diagram showing the processing of the inference model learning unit 413. The inference model learning unit 413 comprises a training data acquisition unit 413A and a machine learning unit 413B.

[0030] The training data acquisition unit 413A refers to the database 421 and acquires training data D20, which consists of input variable data and target variable data.

[0031] The input variables that make up the training data D20 are the dynamic information D21, static information D22, and bathtub distance information D23 registered in the database 421. The target variable that makes up the training data D20 is the bathing status information D25 registered in the database 421.

[0032] Dynamic information refers to the time-series difference information of multiple images taken inside the bathroom. An example of dynamic information is body movement information regarding the movements of bathers inside the bathroom. Static information refers to image information taken inside the bathroom. An example of static information is behavioral information regarding the posture of bathers inside the bathroom.

[0033] Bathtub distance information refers to depth data regarding the position of the water surface inside the bathtub. An example of bathtub distance information is water surface distance information, which refers to the distance between the water surface inside the bathtub and a part of the bather's face. Bathing status information is label data indicating whether or not the bather is submerged.

[0034] The training data D20 is used as training data, validation data, and test data in supervised learning. The target variable data that makes up the training data D20 is used as the correct label in supervised learning.

[0035] The learning data acquisition unit 413A may acquire learning data D20 from data already registered in the database 421 that meets predetermined conditions (for example, conditions regarding the time the data was registered). Alternatively, the learning data acquisition unit 413A may acquire learning data D20 by other methods, either in place of or in addition to the database 421. The learning data acquisition unit 413A may also acquire learning data D20 in cooperation with an external device connected via the network N, for example.

[0036] The machine learning unit 413B uses multiple sets of training data D20 acquired by the training data acquisition unit 413A to perform machine learning in which it trains the bathing state inference model 431 on the correlation between input variable data and target variable data.

[0037] Furthermore, the bathing state inference model 431 may be, for example, an inference model pre-trained using an image recognition dataset, or an inference model with randomly initialized parameters. In this case, the machine learning unit 413B may perform additional training such as fine-tuning or transfer learning on the bathing state inference model 431.

[0038] The timing at which the machine learning unit 413B performs machine learning on the inference model learning unit 413 may be when the number of newly registered data in the database 421 exceeds a predetermined number, or it may be at predetermined intervals, or it may not be limited to these.

[0039] (Bathroom information processing unit 414) Figure 5 is an explanatory diagram showing the processing of the bathroom information processing unit 414. The bathroom information processing unit 414 comprises a dynamic information acquisition unit 414A, a static information acquisition unit 414B, a bathtub distance information acquisition unit 414C, and a bathing state estimation unit 414D.

[0040] The dynamic information acquisition unit 414A acquires dynamic information D11 within the bathroom based on the imaging data acquired by the imaging device 20. For example, the dynamic information acquisition unit 414A acquires body movement information regarding the movements of bathers in the bathroom from the imaging data acquired by the imaging device 20 using known image analysis techniques. As another example, the dynamic information acquisition unit 414A acquires information regarding changes in the background other than the bathers in the bathroom from the imaging data acquired by the imaging device 20 using known image analysis techniques.

[0041] Figure 6 shows an example of dynamic information D11 obtained by background subtraction. In background subtraction, images are compared frame by frame, and changed areas are extracted as difference information in white. In the image shown in Figure 6A, more white areas are detected than in the image shown in Figure 6B, indicating that the subject is moving more. Using such dynamic information D11, it may be possible to specify that an abnormal state is assumed if there is no movement of the subject above a certain amount.

[0042] The static information acquisition unit 414B acquires static information D12 of the bathroom based on the imaging data acquired by the imaging device 20. For example, the static information acquisition unit 414B acquires behavioral information regarding the bather's position in the bathroom from the imaging data acquired by the imaging device 20. This behavioral information includes information such as which parts of the bather's body are in contact with the water surface in the bathtub, and which parts of the bather's body are submerged in the water.

[0043] The bathtub distance information acquisition unit 414C acquires bathtub distance information D13 regarding the position of the water surface in the bathtub based on the depth data acquired by the depth data sensor 30. For example, the bathtub distance information acquisition unit 414C acquires water surface distance information regarding the distance between the water surface in the bathtub and a part of the bather's face (for example, nose or mouth) from the depth data acquired by the depth data sensor 30.

[0044] Specifically, if the depth data sensor 30 is a LiDAR sensor, water surface distance information regarding the distance between the water surface in the bathtub and a part of the bather's face may be acquired based on the first reflected light from the water surface of the bathtub and the second reflected light from a part of the bather's face. Alternatively, if the depth data sensor 30 is a Time-of-Flight (Tof) sensor, water surface distance information regarding the distance between the water surface in the bathtub and a part of the bather's face may be acquired based on the first time from when the infrared light is emitted until it reflects back from the water surface of the bathtub and returns, and the second time until it reflects back from a part of the bather's face and returns. Furthermore, if the depth data sensor 30 is a stereo camera, the distance between the position of the water surface calculated from the two acquired images and the position of a part of the bather's face may be acquired as water surface distance information.

[0045] Figure 7 shows an example of static information D12 and bathtub distance information D13. Figures 7A and 7B show images relating to the bather's position in the bathroom. In the example shown in Figure 7A, the bather's nose position Hn and mouth position Hm are higher than the water surface position Hw, maintaining a predetermined distance. On the other hand, in the example shown in Figure 7B, the bather's nose position Hn and mouth position Hm are lower than the water surface position Hw. Using such static information D12 and bathtub distance information D13, it may be specified that an abnormal state is estimated when the bather's nose position Hn and mouth position Hm are lower than the water surface position Hw.

[0046] Furthermore, the static information D12, which includes images of bathers in the bathroom, may be subject to image processing such as blurring or mosaic effects on areas where faces are detected, in order to prevent the identification of individuals. By adopting such a configuration, it is possible to obtain information necessary for estimating the bathing state while protecting the privacy of bathers.

[0047] The bathing state estimation unit 414D estimates the state of a bather by inputting bathing state inference model input information based on dynamic information, static information, and distance information within the bathtub into the bathing state inference model 431. As an example, the bathing state estimation unit 414D estimates whether or not the bather is submerged. As an example, the bathing state inference model 431, upon receiving the bathing state inference model input information, outputs information regarding the bather's submersion as a probability. The bathing state estimation unit 414D may be configured to estimate that the bather is submerged if the probability output from the bathing state inference model 431 is above a predetermined threshold. Alternatively, the bathing state estimation unit 414D may be configured to estimate that the bather is submerged if the probability output from the bathing state inference model 431 is below a predetermined threshold.

[0048] Furthermore, dynamic information, static information, and information about distance within the bathtub can be combined in any way to form the input information for the bathing state inference model. That is, As dynamic information, • Time-series difference information of multiple images taken inside the bathroom • Body movement information regarding the movements of bathers in the bathroom Either one of them, and as static information, • Image information taken inside the bathroom • Information regarding the behavior of bathers in the bathroom Either one of them, and as distance information within the bathtub, • Depth data regarding the water level in the bathtub • Water surface distance information regarding the distance between the water surface in the bathtub and a portion of the bather's face. Either one of these can be combined with the other in any way.

[0049] (Hardware configuration of each device) Figure 8 is a hardware configuration diagram showing an example of computer 900. The information processing device 40 and user terminal device 200 in the bathtub sinking detection system 1 are composed of a general-purpose or dedicated computer 900.

[0050] As shown in Figure 8, the computer 900 comprises, as its main components, a bus 910, a processor 912, memory 914, an input device 916, an output device 917, a display device 918, a storage device 920, a communication interface unit 922, an external device interface unit 924, an I / O device interface unit 926, and a media input / output unit 928. Note that the above components may be omitted as appropriate depending on the intended use of the computer 900.

[0051] The processor 912 consists of one or more arithmetic processing units (CPU (Central Processing Unit), MPU (Micro-processing unit), DSP (digital signal processor), GPU (Graphics Processing Unit), etc.) and operates as a control unit that oversees the entire computer 900. The memory 914 stores various data and programs 930 and consists of volatile memory (DRAM, SRAM, etc.) that functions as main memory, and non-volatile memory (ROM), flash memory, etc.

[0052] The input device 916 consists of, for example, a keyboard, mouse, numeric keypad, electronic pen, microphone, etc., and functions as an input unit. The output device 917 consists of, for example, a sound (voice) output device, a vibration device, etc., and functions as an output unit. The display device 918 consists of, for example, a liquid crystal display, an organic EL display, electronic paper, a projector, etc., and functions as an output unit. The input device 916 and the display device 918 may be configured as an integrated unit, such as a touch panel display. The storage device 920 consists of, for example, an HDD, SSD, etc., and functions as a storage unit. The storage device 920 stores various data necessary for the execution of the operating system and program 930.

[0053] The communication I / F unit 922 is connected by wire or wireless to a network 940 such as the Internet or an intranet (which may be the same as network N in Figure 1) and functions as a communication unit that sends and receives data with other computers according to a predetermined communication standard. The external device I / F unit 924 is connected by wire or wireless to external devices 950 such as cameras, printers, scanners, and reader / writers and functions as a communication unit that sends and receives data with external devices 950 according to a predetermined communication standard. The I / O device I / F unit 926 is connected to I / O devices 960 such as various sensors and actuators and functions as a communication unit that sends and receives various signals and data with the I / O devices 960, for example, detection signals from sensors and control signals to actuators. The media input / output unit 928 is composed of a drive device such as a DVD drive or CD drive and reads and writes data to media (non-temporary storage medium) 970 such as DVDs and CDs.

[0054] In the computer 900 having the above configuration, the processor 912 calls and executes the program 930 stored in the storage device 920 in the memory 914, and controls various parts of the computer 900 via the bus 910. The program 930 may also be stored in the memory 914 instead of the storage device 920. The program 930 may be recorded on the media 970 in an installable or executable file format and provided to the computer 900 via the media input / output unit 928. The program 930 may also be provided to the computer 900 by downloading it via the network 940 through the communication interface unit 922. Furthermore, the computer 900 may implement the various functions realized by the processor 912 executing the program 930 using hardware such as an FPGA or ASIC.

[0055] Computer 900 is an electronic device of any form, consisting of, for example, a stationary computer or a portable computer. Computer 900 may be a client computer, a server computer, a cloud computer, or an embedded computer such as a control panel or controller (including microcontrollers, programmable logic controllers, and sequencers).

[0056] (Operation of the information processing device 40) Figure 9 is a flowchart illustrating an example of the operation (information processing method) of the information processing device 40. The series of information processing methods by the information processing device 40 shown in Figure 9 will be explained starting from the pre-training process of the bathing state inference model 431. Furthermore, it will be explained that the database 421 contains multiple entries of bathroom information registered by the database management unit 412.

[0057] First, in step S100 (training data acquisition step), the training data acquisition unit 413A acquires multiple sets of training data D20 based on registered bathroom information stored in the database 421. The training data D20 includes dynamic information D21, static information D22, and bathtub distance information D23.

[0058] Next, in step S110 (machine learning process), the machine learning unit 413B performs machine learning on the bathing state inference model 431 using the acquired training data D20. This process constructs a model for inferring information about the bathing state from information within the bathroom.

[0059] Next, in step S120, the bathroom information processing unit 414 acquires real-time bathroom information D10 from the imaging device 20 and the depth data sensor. The bathroom information D10 includes dynamic information D11, static information D12, and bathtub distance information D13.

[0060] Next, in step S130, the bathing state estimation unit 414D inputs the bathing state inference model input information based on real-time bathroom information D10 into the trained bathing state inference model 431 to obtain information about the bathing state.

[0061] Next, in step S140 (bathing state estimation step), the bathing state estimation unit 414D estimates whether or not the bath is submerged based on the acquired information regarding the bathing state.

[0062] Next, in step S150, if the bathing state estimation step (S140) determines that the person has sunk, the transmission / reception control unit 411 outputs an alert to the user terminal device 200. This allows the emergency contact user U2 to take appropriate action immediately if the bather U1 sinks in the bathroom.

[0063] As described above, the information processing device 40 according to this embodiment acquires bathroom information including dynamic information, static information, and distance information within the bathtub, and estimates the state of the bather by inputting the bathing state inference model input information into the bathing state inference model. This makes it possible to detect sinking in the bathtub with high accuracy and immediately.

[0064] Furthermore, the information processing device 40 acquires different types of data, such as dynamic information, static information, and distance information within the bathtub, in a multimodal manner and performs machine learning processing and inference processing. This enables multifaceted and complex detection, thereby reducing false alarms.

[0065] Furthermore, since the information processing device 40 has the functionality of an edge AI that operates artificial intelligence on an edge device, it can detect the sinking state in real time and respond immediately to abnormal situations. In addition, it can process highly private information such as image data from inside the bathroom without transmitting it to the outside from the edge device, thus protecting privacy.

[0066] Furthermore, since it is implemented as a lighting fixture 100 equipped with an information processing device 40, it is easy to install in the bathroom considering power supply and other factors, and it can reduce the psychological resistance of bathers to having their bathrooms imaged.

[0067] (Other embodiments) The present invention is not limited to the embodiments described above, and can be implemented with various modifications without departing from the spirit of the invention. All such modifications are included in the technical concept of the present invention.

[0068] For example, in the above embodiment, the bathing state estimation unit 414D estimated whether the bather was submerged or not as bathing state information D15, but it is not limited to this example. For example, the bathing state estimation unit 414D may estimate the probability that the bather will submerge in the future. In this case, the bathing state inference model 431, which receives the bathing state inference model input information, outputs information regarding the bather's submersion as a probability along with a future time. The bathing state estimation unit 414D may be configured to output an alert if it estimates that there is a high probability that the bather will submerge at a time when the probability output from the bathing state inference model 431 is above a predetermined threshold.

[0069] Furthermore, in the above embodiment, if the input state estimation unit 424D cannot acquire any one of the dynamic information D11, static information D12, and bathtub distance information D13, it may estimate the bather's state using only the information that has been acquired. For example, if the static information D12 and bathtub distance information D13 cannot be acquired due to the conditions in the bathroom, the input state estimation unit 424D may estimate the bather's state using only the dynamic information D11.

[0070] Furthermore, the machine learning unit 413B may perform preprocessing, such as feature calculation, on the training data D20 acquired by the training data acquisition unit 413A, and then perform the machine learning process. In this case, the features of the dynamic information D21, the static information D22, and the distance information in the bathtub D23 may be calculated and each of the three features may be used as input variable data, or the three features may be combined and used as input variable data.

[0071] Furthermore, although the above embodiment includes an information processing device 40 in the lighting fixture 100, the invention is not limited to this example. Specifically, the information processing device 40 may be included in other electrical equipment installed in the bathroom (for example, a television or audio system). In such an embodiment, the technical concept of this disclosure can be applied in the same way as in the above embodiment.

[0072] In the above embodiment, the information processing device 40 was described as being composed of a single device, but it may be composed of multiple devices. For example, by distributing the parts 411 to 414 of the information processing device 40 across multiple devices, it may be composed of a machine learning device equipped with an inference model learning unit 413 that performs an input state inference model learning process, and an information processing device equipped with a bathroom information processing unit 414 that performs a bathing state estimation process. In this case, each part (each process) of each of the above devices may be implemented by a program (information processing program or machine learning program) that can be executed on the computer 900.

[0073] Each of the above devices can be configured as follows, for example. Note that the configuration and operation of each part of each device, and the various data handled by each device, are the same as in the above embodiments, so a detailed explanation is omitted.

[0074] The machine learning device that performs the input state inference model learning process includes a learning data acquisition unit 413A that acquires multiple sets of learning data D20 consisting of input data, a machine learning unit 413B that uses the multiple sets of learning data D20 acquired by the learning data acquisition unit 413A to train the bathing state inference model 431 on the correlation between input data and output data, and a trained model storage unit 43 that stores the bathing state inference model 431 on which the correlation has been learned by the machine learning unit 413B. The input variable data is bathing state inference model input information based on dynamic information D21, static information D22, and bathtub distance information D23. The target variable data is bathing state information D25.

[0075] Furthermore, the transmission / reception control unit 411 and the database management unit 412 may be provided in each of the above devices. Also, although the information processing device 40 is assumed to be implemented on an edge device, the database 421 only needs to be configured to be accessible from each of the above devices, and may be stored in the storage unit of any of the devices, or in the storage unit of an external computer. In addition, some of the above devices may be implemented in the user terminal device 200.

[0076] (Inference device, inference method, and inference program) The present invention can be provided not only in the form of the information processing device 40 (information processing method or information processing program) according to the above embodiment, but also in the form of an inference device (inference method or inference program) used to infer feature quantities. In that case, the inference device (inference method or inference program) may include a memory and a processor, the processor of which may execute a series of processes. The series of processes includes an information acquisition process (information acquisition step) for acquiring bathroom information D10 and an inference process (inference step) for inferring information about the bathing state based on the bathroom information D10.

[0077] By providing the inference device (inference method or inference program) in the form of an inference device, it becomes easier to apply to various devices compared to implementing an information processing device. It will be obvious to those skilled in the art that when the inference device (inference method or inference program) infers information about the bathing state, it may apply the inference method performed by the bathroom information processing unit 414 using a trained inference model generated by the machine learning device and machine learning method according to the above embodiment. [Explanation of Symbols]

[0078] 1...Bathtub submersion detection system, 10...Light-emitting unit, 20...Imaging device, 30...Depth data sensor, 40...Information processing device, 41...Control unit, 42...Data storage unit, 43...Model memory unit, 44...Communication unit, 100...Lighting fixture, 200...User terminal device, 411...Transmission / reception control unit, 412...Database management unit, 413...Inference model learning unit, 413A...Training data acquisition unit, 413B...Machine learning unit, 414...Bathroom information processing unit, 414A... Dynamic information acquisition unit, 414B... Static information acquisition unit, 414C...Bathtub distance information acquisition unit, 414D...Bath state estimation unit, 421...Database, 422...Information processing program, 431...Bath state inference model

Claims

1. A dynamic information acquisition unit that acquires dynamic information within the bathroom, A static information acquisition unit that acquires static information inside the bathroom, A unit for acquiring distance information inside the bathtub that acquires distance information inside the bathtub regarding the position of the water surface inside the bathtub, The system includes a bathing state estimation unit that estimates the state of a bather by inputting bathing state estimation model input information based on the dynamic information, the static information, and the distance information inside the bathtub into the bathing state estimation model, and The bathing state inference model is a trained inference model that has learned the correlation between the input information for the bathing state inference model and information regarding the bather's state through machine learning, and is an information processing device.

2. The dynamic information acquisition unit uses the following as the dynamic information: The difference information in chronological order of multiple images taken inside the aforementioned bathroom, or To acquire body movement information regarding the movements of bathers in the aforementioned bathroom, The information processing apparatus according to claim 1.

3. The static information acquisition unit uses the following as the static information: Image information taken inside the aforementioned bathroom, or To acquire behavioral information regarding the behavior of bathers in the aforementioned bathroom, The information processing apparatus according to claim 1.

4. The unit that acquires the distance information inside the bathtub uses the following as the distance information inside the bathtub: The depth data inside the bathtub, or The system acquires water surface distance information relating to the distance between the water surface in the bathtub and a part of the bather's face. The information processing apparatus according to claim 1.

5. The bathing state estimation unit determines the bather's state as follows: To estimate whether the bather is submerged or not, The information processing apparatus according to claim 1.

6. The bathing state estimation unit determines the bather's state as follows: To estimate the probability that the bather in question will sink in the future, The information processing apparatus according to claim 1.

7. An information processing device according to any one of claims 1 to 6, An imaging device that acquires images of the inside of the bathroom, A depth data sensor that detects depth data in the bathtub, A lighting fixture comprising a light-emitting unit.

8. A training data acquisition unit that acquires multiple sets of training data consisting of input data and output data, A machine learning unit uses multiple sets of training data acquired by the training data acquisition unit to train a bathing state inference model on the correlation between the input data and the output data, The system includes a trained model storage unit that stores the bathing state inference model, which has been trained by the machine learning unit to learn the correlation, The aforementioned input data is input information for a bathing state inference model, based on dynamic information within the bathroom, static information within the bathroom, and distance information within the bathtub regarding the position of the water surface in the bathtub. The output data mentioned above is information about the bather's condition, according to the machine learning device.

9. An inference device comprising memory and a processor, The aforementioned processor, Dynamic information acquisition process to acquire dynamic information within the bathroom, Static information acquisition process to obtain static information within the bathroom, A process for acquiring distance information within the bathtub, which obtains information about the position of the water surface inside the bathtub, An inference device that performs an inference process for inferring information about the state of a bather based on the dynamic information, the static information, and the distance information inside the bathtub.

10. A dynamic information acquisition process to acquire dynamic information within the bathroom, A static information acquisition process to acquire static information within the bathroom, A process for acquiring distance information inside the bathtub, which acquires information about the position of the water surface inside the bathtub, The system includes a bathing state estimation step, which estimates the state of a bather by inputting bathing state inference model input information based on the dynamic information, the static information, and the distance information inside the bathtub into a bathing state inference model. The bathing state inference model is a trained inference model that has learned the correlation between the input information for the bathing state inference model and the information regarding the bather's state through machine learning, in this information processing method.

11. A training data acquisition process involves acquiring multiple sets of training data consisting of input data and output data, A machine learning process in which a bathing state inference model learns the correlation between the input data and the output data using multiple sets of training data acquired in the training data acquisition process, The system includes a trained model storage step for storing the bathing state inference model, which has learned the correlation relationship through the machine learning step. The aforementioned input data is input information for a bathing state inference model, based on dynamic information within the bathroom, static information within the bathroom, and distance information within the bathtub regarding the position of the water surface in the bathtub. The output data described above is information about the bather's condition, according to a machine learning method.

12. An inference method performed by an inference device comprising memory and a processor, The aforementioned processor, Dynamic information acquisition process to acquire dynamic information within the bathroom, Static information acquisition process to obtain static information within the bathroom, A process for acquiring distance information within the bathtub, which obtains information about the position of the water surface inside the bathtub, An inference device that performs an inference process for inferring information about the state of a bather based on the dynamic information, the static information, and the distance information inside the bathtub.

Images