Information processing systems, information processing methods, and computer programs

Abstract

[Problem] To provide an information processing method, a computer program, and an information processing device that are capable of highly accurately estimating the weight of an object-to-be-measured in a non-contact manner. [SOLUTION] An information processing system according to the present disclosure comprises one or a plurality of computer processors. The information processing system comprises: a measurement device that includes an imaging device, an infrared sensor unit, and an ultrasonic sensor unit; and an information processing device that is connected to the measurement device. The one or plurality of computer processors are characterized by including: a reception unit that receives data acquired by the measurement device; and a processing unit that calculates the weight of the object-to-be-measured by processing the data received by the reception unit, wherein the processing unit calculates the volume of each portion of the object-to-be-measured on the basis of the data received by the reception unit, and calculates the weight of the object-to-be-measured on the basis of the volume of each portion, information pertaining to the configuration of each portion, and the specific gravity of an element pertaining to the configuration.

Description

Technical Field

[0001] The present invention relates to an information processing system, an information processing method, and a computer program.

Background Art

[0002] Conventionally, when a user wants to know their own weight, it is common to measure the weight by stepping on a weighing scale.

[0003] However, when sharing a weighing scale with other users, for example, management in terms of hygiene is necessary.

[0004] In addition, the act of stepping on a weighing scale not only places a physical burden on the user, but there are also some people who feel psychological resistance to such an act.

[0005] Patent Document 1 discloses a technique for estimating the weight of an animal that is difficult to measure with a weighing scale by a non-contact method, but there are still problems with the measurement accuracy.

Prior Art Documents

Patent Documents

[0006]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0007] An object of the present disclosure is to provide a technical improvement that solves or alleviates at least some of the problems of the above-described conventional techniques. One more specific object of the present disclosure is to provide an information processing method, a computer program, and an information processing apparatus capable of measuring the weight of a measurement target non-contact and with high accuracy.

Means for Solving the Problems

[0008] The information processing system in this disclosure is an information processing system comprising one or more computer processors, wherein the information processing system comprises a measuring device including an imaging device, an infrared sensor unit and an ultrasonic sensor unit, and an information processing device connected to the measuring device, wherein one or more computer processors have a receiving unit that receives data acquired by the measuring device and a processing unit that calculates the weight of the object to be measured by processing the data received by the receiving unit, wherein the processing unit calculates the volume of each part of the object to be measured based on the data received by the receiving unit, and calculates the weight of the object to be measured based on the volume of each part, information regarding the configuration of each part and the specific gravity of the elements related to said configuration.

[0009] The above processing unit can estimate the composition of the object to be measured based on the reflection data from the object to be measured, which is received by the receiving unit and acquired by the ultrasonic sensor unit.

[0010] The above processing unit can generate 3D data of the object to be measured based on the reflection data and / or radiation data from the object, including the object to be measured, which are received by the receiving unit and acquired by the infrared sensor unit.

[0011] The above-described processing unit can generate 3D data of the object to be measured based on image data including the object to be measured, which is received by the receiving unit and acquired by the imaging device.

[0012] The above processing unit divides a three-dimensional object having the shape corresponding to the object to be measured into multiple parts of a predetermined size based on the 3D data of the object to be measured. Based on information regarding the composition of each part and the specific gravity of the elements related to that composition, it calculates the weight of each part and obtains the weight of the object to be measured by summing the weights of all parts.

[0013] The above processing unit can further identify the measurement target by analyzing the image data, including the measurement target, that has been received by the receiving unit and acquired by the imaging device.

[0014] The above processing unit can further perform background removal from image data by analyzing the image data, including the object to be measured, that has been received by the receiving unit and acquired by the imaging device.

[0015] The above infrared sensor unit may include an infrared camera and an infrared sensor composed of an infrared light emitter, an infrared light receiver, and a reflector.

[0016] The ultrasonic sensor unit described above may comprise a main unit consisting of an ultrasonic light emitter and an ultrasonic light receiver, and a sub-unit consisting of an ultrasonic light receiver and a reflector.

[0017] The ultrasonic sensor unit described above comprises a main unit and two sub-units, and can be arranged such that the horizontal angle of the straight line connecting the main unit and each of the sub-units, and the vertical angle of the straight line connecting the main unit and each of the sub-units, are within a predetermined value.

[0018] The above measuring device can be used to be positioned either directly in front of the object to be measured or above the object to be measured.

[0019] The above one or more computer processors further have a storage unit that stores user information of the user to be measured, and the processing unit determines whether the image data of the user's face to be measured, which has been received by the receiving unit and acquired by the imaging device, is included in the user information stored in the storage unit, and if it is determined as a result of the determination that the image data of the user's face is included in the user information, the user can be identified as the target of measurement.

[0020] The memory unit also stores information about the user's actual weight. The processing unit compares the calculated weight of the measured object, which is the user's weight, with the user's actual weight stored in the memory unit. Based on the comparison, it can determine whether there is a difference of more than a predetermined amount between the user's weight and the user's actual weight.

[0021] If, as a result of the determination, there is a difference of a predetermined value or more between the user's weight and the user's actual weight, the processing unit can request the measuring device to acquire data again.

[0022] The information processing method in the present disclosure is an information processing method in an information processing system including one or more computer processors. The information processing system includes a measuring device including an imaging device, an infrared sensor unit, and an ultrasonic sensor unit, and an information processing device connected to the measuring device. The one or more computer processors are caused to execute steps of receiving data acquired by the measuring device, calculating the volume of each part of the measurement target based on the data, and calculating the weight of the measurement target based on the volume of each part, information on the composition of each part, and the specific gravity of the elements related to the composition.

[0023] The computer program in the present disclosure is a computer program executed in an information processing system including one or more computer processors. The information processing system includes a measuring device including an imaging device, an infrared sensor unit, and an ultrasonic sensor unit, and an information processing device connected to the measuring device. The one or more computer processors are caused to realize a receiving function of receiving data acquired by the measuring device and a processing function of calculating the weight of the measurement target by processing the data received by the receiving function. The processing function calculates the volume of each part of the measurement target based on the data received by the receiving function, and calculates the weight of the measurement target based on the volume of each part, information on the composition of each part, and the specific gravity of the elements related to the composition.

Advantages of the Invention

[0024] According to the present disclosure, it is possible to provide a technical improvement that solves or alleviates at least part of the problems of the above-described conventional technologies. Specifically, it is possible to provide an information processing method, a computer program, and an information processing device capable of non-contact and highly accurate measurement of the weight of a measurement target.

Brief Description of the Drawings

[0025] [Figure 1] This is a system configuration diagram showing an example of the system configuration of the information processing system in the present disclosure. [Figure 2] This is a configuration diagram showing an example of the hardware configuration of the information processing device in the present disclosure. [Figure 3] This is a configuration diagram showing an example of the functional configuration of the information processing device in the present disclosure. [Figure 4] This is a conceptual diagram showing an example of the arrangement of the information processing system in the present disclosure. [Figure 5] This is a conceptual diagram for explaining the measurement algorithm of the weight of the measurement target in the present disclosure. [Figure 6] This is a conceptual diagram showing another example of the arrangement of the information processing system in the present disclosure. [Figure 7] This is a conceptual diagram showing another example of the arrangement of the information processing system in the present disclosure. [Figure 8] This is a conceptual diagram showing another example of the arrangement of the information processing system in the present disclosure. [Figure 9] This is a configuration diagram showing another example of the functional configuration of the information processing device in the present disclosure. [Figure 10] This is a system configuration diagram showing another example of the system configuration of the information processing system in the present disclosure. [Figure 11] This is a flowchart showing an example of the user registration flow in the present disclosure. [Figure 12] This is a flowchart showing an example of the measurement flow in the present disclosure. [Figure 13] This is a flowchart showing an example of the information processing method in the present disclosure. [Figure 14] This is a circuit configuration diagram showing an example of the circuit configuration for realizing the function of the computer program in the present disclosure. [Figure 15] This is a system configuration diagram showing an example of the system configuration of the information processing system in another embodiment of the present disclosure. [Figure 16] ​ [Figure 17] This diagram shows an example of the functional configuration of the first information processing device in this disclosure. [Figure 18] This is a conceptual diagram showing an image of the screen displayed on the display of the first information processing device in this disclosure. [Figure 19] This is a system configuration diagram showing an example of the system configuration of an information processing system in another embodiment of the present disclosure. [Figure 20] This diagram shows an example of the functional configuration of the third information processing device in this disclosure. [Figure 21] This diagram shows another example of the functional configuration of the first information processing device in this disclosure. [Figure 22] This flowchart shows an example of the flow of an information processing method in another embodiment of the present disclosure. [Figure 23] This is a circuit diagram showing an example of a circuit configuration for realizing the functionality of a computer program in other embodiments of this disclosure. [Figure 24] This is a system configuration diagram showing an example of the system configuration of an information processing system in another embodiment of the present disclosure. [Modes for carrying out the invention]

[0026] Embodiments of the information processing system described herein will be explained with reference to the drawings.

[0027] Figure 1 shows an example of the system configuration of the information processing system 1000 in this disclosure.

[0028] The information processing system 1000 in this disclosure comprises an information processing device 100 and a measuring device 200, as shown in Figure 1. The information processing device 100 and the measuring device 200 can be directly connected by wire or wireless.

[0029] Here, the hardware configuration of the information processing device 100 will be described using Figure 2. The information processing device 100 may include a processor 101, memory 102, storage 103, input / output interface (input / output I / F) 104, and communication interface (communication I / F) 105. Each component is interconnected via bus B.

[0030] The information processing device 100 can realize the functions and methods described in this embodiment through the cooperation of the processor 101, memory 102, storage 103, input / output I / F 104, and communication I / F 105.

[0031] The processor 101 executes functions and / or methods realized by code or instructions contained in a program stored in the storage 103. The processor 101 may include, for example, a central processing unit (CPU), an MPU (Micro Processing Unit), a GPU (Graphics Processing Unit), a microprocessor, a processor core, a multiprocessor, an ASIC (Application-Specific Integrated Circuit), an FPGA (Field Programmable Gate Array), etc., and may realize each process disclosed in each embodiment by logic circuits (hardware) or dedicated circuits formed on an integrated circuit (IC (Integrated Circuit) chip, LSI (Large Scale Integration)), etc. Furthermore, these circuits may be realized by one or more integrated circuits, and the multiple processes shown in each embodiment may be realized by a single integrated circuit. In addition, LSIs may be referred to as VLSI, Super LSI, Ultra LSI, etc., depending on the degree of integration.

[0032] Memory 102 temporarily stores the program loaded from storage 103 and provides a workspace for the processor 101. Various data generated while the processor 101 is executing the program are also temporarily stored in memory 102. Memory 102 includes, for example, RAM (Random Access Memory) and ROM (Read Only Memory).

[0033] Storage 103 stores programs. Storage 103 includes, for example, an HDD (Hard Disk Drive), an SSD (Solid State Drive), flash memory, etc.

[0034] The communication interface 105 is implemented as hardware such as a network adapter, communication software, or a combination thereof, and transmits and receives various types of data over the network. This communication may be performed via wired or wireless connection, and any communication protocol may be used as long as communication between the two devices is possible. The communication interface 105 communicates with other information processing devices over the network. The communication interface 105 transmits various types of data to other information processing devices according to instructions from the processor 101. The communication interface 105 also receives various types of data transmitted from other information processing devices and transmits them to the processor 101.

[0035] The input / output interface 104 includes an input device for inputting various operations to the information processing device 100, and an output device for outputting processing results processed by the information processing device 100. The input / output interface 104 may have the input device and output device integrated, or they may be separated into an input device and an output device.

[0036] The input device is implemented by any or a combination of any type of device capable of receiving input from a user and transmitting the information related to said input to the processor 101. The input device includes, for example, hardware keys such as touch panels, touch displays, and keyboards, pointing devices such as mice, cameras (for image-based operation input), and microphones (for voice-based operation input).

[0037] The output device outputs the processing results processed by the processor 101. The output device includes, for example, a touch panel, a speaker, etc.

[0038] Returning to Figure 1, the measuring device 200 includes an imaging device 210, an infrared sensor unit 220, and an ultrasonic sensor unit 230.

[0039] In this disclosure, the one or more computer processors provided in the information processing system 1000 are described as being provided in the information processing device 100, but are not limited to this, and may be provided in cooperation with other devices connected to the information processing device 100.

[0040] Furthermore, one or more computer processors in this disclosure include a receiving unit 110 and a processing unit 120, as shown in Figure 3.

[0041] The receiving unit 110 receives the data acquired by the measuring device 200.

[0042] Furthermore, data acquisition by the measuring device 200 may be triggered when the user enters a predetermined measurement area. For example, the user entering the predetermined measurement area can be determined by image analysis performed by the imaging device 210, which is one of the measuring devices 200. In this case, the measurement by the infrared sensor unit 220 and the ultrasonic sensor unit 230 may be initiated according to the results of the image analysis.

[0043] The predetermined measurement area can, for example, be the range that the imaging device 210 can capture.

[0044] Alternatively, data acquisition by the measuring device 200 may be performed continuously, while data acquisition by the receiving unit 110 may be triggered when the user enters a predetermined measurement area.

[0045] The processing unit 120 calculates the weight of the object to be measured by processing the data received by the receiving unit 110.

[0046] The object of measurement is not particularly limited, but in the following examples, the object of measurement will be a human being, and the weight of the object of measurement will be the human body weight. It is preferable that the weight referred to here does not include the weight of the clothing the person is wearing, as will be explained later.

[0047] The processing unit 120 calculates the weight of each part of the object to be measured based on the data received by the receiving unit 110.

[0048] As an example, the processing unit 120 divides a three-dimensional object having an outline corresponding to the object to be measured into multiple parts (faults) of predetermined sizes based on the 3D (three-dimensional) data of the object to be measured. For each divided part, the weight of each part is obtained by integrating the area of ​​the cross-section. Note that the predetermined size does not have to be irregular.

[0049] The processing unit 120 then calculates the weight of the object to be measured based on the volume of each part, information regarding the composition of each part, and the specific gravity of the elements related to that composition.

[0050] Information regarding the composition of each body part refers, for example, to the proportion of elements that make up each body part. These elements are what is commonly known as the body's component composition. The human body is said to be mainly composed of skeletal muscle, adipose tissue, bone, and blood, while organs are mainly composed of water, fat, and protein.

[0051] Information regarding the composition of the above-mentioned parts may be obtained by using pre-prepared reference data on the proportion of elements, or by using data obtained by analyzing data from the measuring device 200.

[0052] Furthermore, the weighting of the elements involved in the composition shall be determined using pre-prepared standard data regarding the weighting of elements.

[0053] The weight of the object to be measured, calculated by the processing unit 120, may be displayed on a display provided by or connected to the information processing system 1000. Preferably, the display referred to here is a user terminal 400 (such as a mobile phone or smartphone) held by the user.

[0054] The above configuration provides a technical improvement that solves or mitigates at least some of the problems of the conventional technology described above. Specifically, it becomes possible to estimate the weight of the object to be measured with high accuracy without contact.

[0055] In particular, the COVID-19 pandemic has heightened people's vigilance regarding contact.

[0056] However, conventional weighing scales and similar devices use springs or pressure sensors, and require the user to directly step onto the device for measurement. There was a need for a technology that could measure weight "non-contact".

[0057] According to the invention disclosed herein, various data of the object to be measured are measured non-contact, eliminating the need for hygienic management.

[0058] Furthermore, according to the invention disclosed herein, since there is no need to step onto the measuring device, the physical burden on the user can be reduced.

[0059] In addition, by eliminating the need to step onto the measuring device, measurements can be taken discreetly without interfering with daily life.

[0060] According to the present invention, by adopting a novel configuration that calculates weight using data acquired by an imaging device, an infrared sensor unit, and an ultrasonic sensor unit, it becomes possible to estimate the weight of the object to be measured with higher accuracy than in the conventional method.

[0061] Furthermore, the processing unit 120 can estimate the composition of the object to be measured based on the transmission and / or reflection data from the object to be measured acquired by the ultrasonic sensor unit 230, which is received by the receiving unit 110.

[0062] The composition estimated by this method may be used as information regarding the composition of the above-mentioned part. For estimating the composition of the object to be measured by the ultrasonic sensor unit 230, known estimation techniques can be used.

[0063] With this configuration, it becomes possible to calculate the weight of the object being measured with higher accuracy than when using reference data.

[0064] Furthermore, the processing unit 120 can generate 3D data of the object to be measured based on the reflection data and / or radiation data from the object, including the object to be measured, which are received by the receiving unit 110 and acquired by the infrared sensor unit 220.

[0065] Here, the infrared sensor unit 220 can be configured to include an infrared camera 221 and an infrared sensor 222, as shown in Figure 4.

[0066] The infrared camera 211 is an infrared thermography camera that can detect heat radiated from the object U being measured. The processing unit 120 uses this heat as radiation data.

[0067] The infrared sensor 222 consists of an infrared light emitter 223, an infrared light receiver 224, and a reflector 225. Specifically, the infrared sensor 222 emits infrared light from the infrared light emitter 223 toward the object to be measured U, which passes through the object to be measured U, and the transmitted light reflected by the reflector 225 is received by the infrared light receiver 224. The processing unit 120 uses this transmitted light as reflection data.

[0068] In this context, the measurement target U refers to a person without clothes (a naked person), while the object refers to a person wearing clothes. In other words, with this configuration, it becomes possible to generate 3D data of only the body, excluding the clothes, even for a person wearing clothes.

[0069] As described above, the processing unit 120 divides a three-dimensional object having an outline corresponding to the object to be measured into multiple parts of a predetermined size based on the 3D data of the object to be measured, calculates the weight of each part based on information about the composition of each part and the specific gravity of the elements related to that composition, and obtains the weight of the object to be measured by summing the weights of all parts.

[0070] As an example, as described above, the processing unit 120 divides a three-dimensional object having an outline corresponding to the object to be measured into multiple parts of a predetermined size based on the 3D data of the object to be measured. Then, for each divided part, as shown in Figure 5, the area of ​​the cross-section of the divided part is calculated, and the volume of each part is obtained by integrating these areas for each part.

[0071] Alternatively, the processing unit 120 can generate 3D data of the object to be measured based on image data including the object to be measured, which has been received by the receiving unit 110 and acquired by the imaging device 210.

[0072] In other words, the processing unit 120 can generate 3D data from 2D image data. This configuration may also be realized by an external service that can be linked with the information processing system of the present invention, such as automatic generation by AI.

[0073] Furthermore, the processing unit 120 can identify the measurement target by analyzing the image data, including the measurement target, that has been received by the receiving unit 110 and acquired by the imaging device 210.

[0074] The imaging device 210 can be, for example, a visible light camera.

[0075] Identifying the measurement target U involves, for example, identifying attribute information such as the gender and age of the user who is the measurement target U. Such attribute information is obtained by analyzing the image data of the user's face and / or body captured by the imaging device 210.

[0076] Alternatively, the identification of the measurement target is the identification of a registered user. Such identification can be performed by determining whether the measurement target is a registered user based on the image data of the user's face captured by the imaging device 210. In the case of a registered user, the attribute information is pre-registered based on the user's input.

[0077] The processing unit 120 can further perform background removal from image data by analyzing the image data, including the object to be measured, that has been received by the receiving unit 110 and acquired by the imaging device 210.

[0078] For separating humans from the background in image data, known techniques can be used.

[0079] The ultrasonic sensor unit 230 may comprise a main unit 231 and two sub-units 232, as shown in Figure 6.

[0080] The main unit 231 consists of an ultrasonic light emitting unit 233 and an ultrasonic light receiving unit 234.

[0081] The sub-unit 232 consists of an ultrasonic light receiving unit 235 and a reflecting unit 236.

[0082] The main unit 231 and the two slave units 232 can be arranged such that the horizontal angle of the straight line connecting the main unit 231 and each of the slave units 232, and the vertical angle of the straight line connecting the main unit 231 and each of the slave units 232, are within a predetermined value.

[0083] As an example, it is preferable to set the predetermined value to 80°.

[0084] Furthermore, the measuring device 200 can be positioned either directly in front of the object to be measured U (Figure 6) or above the object to be measured (Figure 7).

[0085] Although Figures 6 and 7 only show the infrared camera 221 and the main unit 231 of the ultrasonic sensor unit 230 as examples of the measuring device 200, it can also include the other components mentioned above.

[0086] One or more computer processors in this disclosure may further have a storage unit 130, as shown in Figure 9.

[0087] The memory unit 130 stores the user's user information.

[0088] At this time, the processing unit 120 determines whether the image data of the user's face, which is the target of measurement and was acquired by the imaging device 210 and received by the receiving unit 110, is included in the user information stored in the storage unit 130.

[0089] Then, if the determination results in a finding that the user's facial image data is included in the user information, the processing unit 120 identifies the user as the measurement target U.

[0090] The measuring device 200 may be configured to perform measurements only on users who have been certified as the target of measurement U.

[0091] Furthermore, for users who are not certified as measurement target U, the measuring device 200 (in this case, the infrared sensor unit 220 and the ultrasonic sensor unit 230) may perform measurements only with the user's permission. For example, a confirmation screen may be displayed on a display connected to the information processing system 1000 in this disclosure to indicate whether or not to perform the measurement. Note that the display referred to here may be provided by the information processing system 1000.

[0092] Furthermore, the above confirmation screen may be displayed on the user's terminal 400 (such as a mobile phone or smartphone; details will be described later). In this case, information may be transmitted from the information processing device 100 to the user terminal 400 via push notification or the like.

[0093] With this configuration, it is possible to appropriately respond when an unregistered user enters the measurement area.

[0094] The memory unit 130 may also store information regarding the user's actual weight.

[0095] At this time, the processing unit 120 compares the user's weight, which is the calculated weight of the object to be measured U, with the user's actual weight stored in the storage unit 130.

[0096] Then, based on the comparison, it is determined whether there is a difference of more than a predetermined amount between the user's weight and the user's actual weight.

[0097] If the determination shows that there is a difference of more than a predetermined amount between the user's weight and the user's actual weight, the processing unit 120 requests the measuring device 200 to acquire the data again.

[0098] Alternatively, the processing unit 120 corrects the user's weight based on the actual weight.

[0099] Furthermore, as shown in Figure 10, the information processing device 100 may be connected to the server device 300 via the Internet. In this case, the storage unit 130 may be provided by the server device 300. Also, the calculated user weight may be stored in association with the user information stored in the storage unit 130.

[0100] Furthermore, the information processing device 100 may be connected to a user terminal 400. The connection can be made via the internet, or by Bluetooth®, wireless LAN, or the like.

[0101] The information processing device 100 can transmit the calculated weight data to the user terminal 400, and the user terminal 400 can store that data.

[0102] Here, we will explain the user registration flow for the information processing system described in this disclosure using Figure 12.

[0103] First, the processing unit 120 acquires an image of the user who wishes to register (S11). Such user images may be acquired and registered using both a visible light camera and an infrared camera.

[0104] Next, the processing unit 120 registers an image of the identified user's face (S12). The user's face image registered here can be automatically displayed on the aforementioned display during or after the measurement.

[0105] Then, the processing unit 120 acquires and registers information regarding the user's name, gender, age, weight, blood type, and height based on the user's operation (S13). The weight registered here will be managed as the actual weight, as described later.

[0106] Finally, the processing unit 120 completes the user registration after the user confirms the registration details (S14).

[0107] Next, Figure 12 will be used to explain the weight measurement flow in the information processing system disclosed herein.

[0108] First, the processing unit 120 acquires an image of the user (S15). Based on this image, it is determined whether the user is a registered user. Furthermore, this image is used with the user's body and background separated, and the background removed.

[0109] Next, the processing unit 120 acquires data from the ultrasonic sensor unit 230 and the infrared sensor unit 220 (S16, S17). The order of these steps is not particularly limited.

[0110] Then, the processing unit 120 performs image analysis and processing on the above data (S18). Here, the image captured by the visible light camera and the image projected by the infrared camera are analyzed, and noise reduction processing can be performed.

[0111] Next, the processing unit 120 displays the data in a distribution chart (S19).

[0112] Then, the processing unit 120 calculates the volume (S21).

[0113] If necessary, the processing unit 120 corrects the weight error (S22).

[0114] Finally, the calculated weight information is displayed on the display connected to the information processing device 100 (S23).

[0115] Furthermore, the calculated weight information may be stored in the server device 300 described above (S24).

[0116] Next, embodiments of the information processing method described herein will be explained with reference to the drawings.

[0117] The information processing method in this disclosure is an information processing method in an information processing system 1000 comprising one or more computer processors.

[0118] The above-described information processing system 1000 comprises a measuring device 200 including an imaging device 210, an infrared sensor unit 220, and an ultrasonic sensor unit 230, and an information processing device 100 connected to the measuring device 200. Details of these devices are as described above.

[0119] The information processing method in this disclosure causes one or more computer processors to execute steps S110 to S130 shown in Figure 13.

[0120] In step S110, the data acquired by the measuring device 200 is received. This step S110 can be performed by the receiving unit 110 described above. Details of the receiving unit 110 are as described above.

[0121] In step S120, the volume of each part to be measured is calculated based on the received data. This step S120 can be performed by the processing unit 120 described above. Details of the processing unit 120 are as described above.

[0122] In step S130, the weight of the object to be measured is calculated based on the volume of each part, information on the composition of each part, and the specific gravity of the elements related to that composition. This step S130 can be performed by the processing unit 120 described above. Details of the processing unit 120 are as described above.

[0123] The above configuration provides a technical improvement that solves or mitigates at least some of the problems of the conventional technology described above. Specifically, it becomes possible to estimate the weight of the object to be measured with high accuracy without contact.

[0124] The information processing system described in this disclosure can improve measurement accuracy compared to conventional methods by analyzing and applying data obtained from ultrasonic waves emitted from an ultrasonic sensor and reflected back by a reflector, data from three-dimensional body measurements taken by an infrared sensor, and gender and age data obtained by a visible light camera.

[0125] The system consists of a main sensor and sensors installed in three, two, or one location on the reflective section. The sensors 3D scan the human body and calculate its area.

[0126] It can measure only the body by passing through clothing, calculate a distribution map, and then make estimations.

[0127] • Scanning method 1) Measuring distance using ultrasound 2) Scan 3D with an infrared sensor 3) Measure only the body while passing through clothing, calculate volume in 3D form, and create a volume distribution map of the body's contours in centimeters to calculate volume with greater accuracy.

[0128] ·Weight measurement calculation method The body is composed of skeletal muscle, adipose tissue, bone, blood, and other organs, all of which are made up of water, fat, and protein. Therefore, the volume is calculated, and each component is assumed to have a specific gravity, which is then calculated as follows.

[0129] Since the human body is composed of water, bone, muscle, fat, and protein, its volume is calculated, and the specific gravity of each component is assumed and calculated using the following formulas. The standard range for blood specific gravity in Japanese people is 1.052 to 1.060 for men and 1.049 to 1.056 for women. The calculations can be performed using the following values: bone = approximately 2.0, muscle = 1.06, brain = 1.04, and fat = 0.94.

[0130] Ultrasonic and infrared sensors can measure body composition by analyzing the difference in transmittance as they pass through the human body. This allows for the quantitative determination of bone mass, muscle mass, and fat mass.

[0131] • Technical points 1. Basic information input section (age, gender, weight, blood type, height) 2. Algorithm for estimating gender and age using a visible light camera 3. Algorithm for estimating height 4. Algorithm for estimating body structure using ultrasonic sensors 5. Algorithm for estimating area using 3D scanning 6. Algorithm for calculating weight from area 7. Algorithm for correcting errors 8. How to display weight 9. Wi-Fi (registered trademark) and Bluetooth (registered trademark) communication methods 10. How to provide the service using a dedicated app

[0132] • Measurement algorithm 1. External shape detection The 3D-measured body is divided into irregularly sized segments, and a horizontal and vertical image distribution map is created as shown in Figure 5. The curved surface is then integrated to perform calculations. 2. Estimate each part To estimate body weight, separate water, bone, muscle, and fat. 3. Noise Reduction 4. Estimating weight

[0133] • 3D scanning of measurements 1. Measure only the body, passing through clothing. 2. Separate the body from the background. The volume of the body is represented by a distribution map with irregular sizes, and the 3D-measured body is represented by a point distribution map with irregular sizes. This allows for the calculation of more precise curved surfaces. 3. Recognize gender 4. It can be automatically retrieved after identity verification.

[0134] Next, embodiments of the computer program described herein will be explained with reference to the drawings.

[0135] The computer program in this disclosure is a computer program that is executed in an information processing system 1000 having one or more computer processors.

[0136] The above-described information processing system 1000 comprises a measuring device 200 including an imaging device 210, an infrared sensor unit 220, and an ultrasonic sensor unit 230, and an information processing device 100 connected to the measuring device 200. Details of these devices are as described above.

[0137] Then, the receiving function and processing function are implemented in one or more of the above-mentioned computer processors.

[0138] The receiving function receives data acquired by the measuring device.

[0139] The processing function calculates the weight of the object being measured by processing the received data.

[0140] Specifically, the processing function calculates the volume of each part of the object to be measured based on the data received by the receiving function, and then calculates the weight of the object based on the volume of each part, information about the composition of each part, and the specific gravity of the elements related to that composition.

[0141] The above receiving and processing functions can be realized by the receiving circuit 111 and processing circuit 121 shown in Figure 14. The receiving circuit 111 and processing circuit 121 are realized by the receiving unit 110 and processing unit 120 described above, respectively. Details of each unit are as described above.

[0142] The above configuration provides a technical improvement that solves or mitigates at least some of the problems of the conventional technology described above. Specifically, it becomes possible to estimate the weight of the object to be measured with high accuracy without contact.

[0143] Other embodiments of the information processing system in this disclosure will be described below. In this embodiment, one or more processors provided in the information processing device 100 may be provided in the first information processing device 1100 described below.

[0144] The inventions described below, according to the embodiments, relate to an information processing system and an information processing device equipped with an analysis unit as an AI platform.

[0145] Biological information such as blood pressure and body temperature is important for detecting abnormalities in health. Traditionally, the main method for obtaining this biological information has been to attach measuring devices to the body and directly acquire the values.

[0146] In recent years, the widespread use of non-contact measuring devices and wearable devices has made it easier to acquire biometric information than before.

[0147] In the conventional technology described above, biological information acquired by a measuring instrument was processed by an information processing device connected via the Internet.

[0148] However, the above configuration has the problem that it cannot be used with devices that cannot connect to the internet or in locations where internet access is unavailable. Furthermore, due to security issues, there is a risk that measurement data of biometric information, which is personal information, and the results of its analysis, may be leaked.

[0149] Therefore, the purpose of the disclosure in this embodiment is to provide a technical improvement that solves or mitigates at least some of the problems of the prior art described above. One of the more specific purposes of this disclosure is to provide an information processing system, information processing method, computer program, and information processing device that can analyze a user's health status even in an environment that is not connected to the Internet.

[0150] The information processing system of the present invention in this embodiment is characterized by comprising a first information processing device having a storage unit for storing the user's genome information, a biometric information acquisition unit for acquiring the user's biological information, an image acquisition unit for acquiring at least an image of the user's face, an environmental information acquisition unit for acquiring environmental information, a display unit for displaying at least a portion of the biometric information acquired by the biometric information acquisition unit and at least a portion of the environmental information acquired by the environmental information acquisition unit, a first analysis unit for analyzing the user's health status based on the biometric information acquired by the biometric information acquisition unit and the genome information stored in the storage unit, a determination unit for determining whether or not there is an abnormality in the health status analyzed by the first analysis unit as an AI base, and an output unit for outputting the result of the determination by the determination unit.

[0151] The above information processing system further includes a biometric information acquisition device connected to the first information processing device, and the biometric information acquisition unit can acquire the user's biometric information acquired by the biometric information acquisition device.

[0152] The first information processing device described above is connected to multiple biometric information acquisition devices, each of which is installed in a room used by different users in a residential or shared facility, and the biometric information acquisition unit can acquire the user's biometric information acquired by the biometric information acquisition devices, along with the user's identification number.

[0153] The above-mentioned biometric information acquisition device can be installed embedded in the walls and / or ceilings of residential or shared facilities.

[0154] The above-mentioned biological information acquisition unit and / or biological information acquisition device can acquire biological information at predetermined times or at predetermined time intervals.

[0155] The above-mentioned biometric information acquisition unit and / or biometric information acquisition device may be designed to acquire the user's respiration, heart rate, and blood pressure in a non-contact manner.

[0156] The above-mentioned biometric information acquisition unit and / or biometric information acquisition device may acquire the user's brainwaves by contact or non-contact.

[0157] The above-mentioned biological information acquisition unit and / or biological information acquisition device may be designed to acquire the user's body surface temperature in a non-contact manner.

[0158] The above-mentioned biometric information acquisition unit and / or biometric information acquisition device may acquire the user's blood glucose level by contact or non-contact.

[0159] The above-mentioned biometric information acquisition unit and / or biometric information acquisition device may be capable of acquiring information through short-range communication with a vital sign measurement sensor embedded in a part of the user's body.

[0160] The above-mentioned environmental information acquisition unit may acquire at least one of temperature, humidity, atmospheric pressure, and oxygen concentration.

[0161] The first analysis unit described above can further analyze the user's facial image acquired by the image information acquisition unit to acquire skin tone information and / or facial expression information, and analyze the user's health status based on the skin tone information and / or facial expression information, the biometric information acquired by the biometric information acquisition unit, and the genomic information stored in the memory unit.

[0162] The first information processing device described above may further include a user identification unit that identifies the user based on an image of the user's face acquired by the image acquisition unit.

[0163] The first information processing device described above may further include a communication unit that can communicate with the outside world only under specific conditions.

[0164] The above information processing system further comprises a second information processing device that can be connected to the first information processing device by short-range wireless communication, and a third information processing device that can be connected to the second information processing device by an internet line. The third information processing device may have a second analysis unit that analyzes the results of the determination obtained via the second information processing device and determines the action to be taken, and an execution unit that executes the action determined by the second analysis unit.

[0165] The above information processing system further includes a third information processing device and a fourth information processing device that can be connected via an internet line, and the action may be the notification of the result output by the output unit to the fourth information processing device.

[0166] The above information processing system further includes a third information processing device and a fifth information processing device that can be connected via an internet line, and the action may be defined as the transmission of an operation command to the fifth information processing device.

[0167] The third information processing device described above can acquire health checkup data for which access has been permitted in advance, and its action can be to transmit information on lifestyle improvements corresponding to the health checkup data to the second information processing device.

[0168] The above information processing system further comprises a second information processing device that can be connected to the first information processing device by short-range wireless communication, and a sixth information processing device that can be connected to the second information processing device by short-range wireless communication. The sixth information processing device is installed on a predetermined means of transport, and the sixth information processing device acquires the user's biometric information during transport and transmits the acquired biometric information during transport to the second information processing device. The second information processing device transmits the biometric information during transport to the first information processing device when it is within short-range wireless communication distance from the first information processing device, and the first information processing device can pass the acquired biometric information during transport to the first analysis unit.

[0169] The above information processing system further comprises a second information processing device that can be connected to the first information processing device by short-range wireless communication, and a seventh information processing device that can be connected to the second information processing device by short-range wireless communication. The seventh information processing device is equipped with a predetermined means of movement. The seventh information processing device acquires the user's biometric information, transmits the acquired biometric information during movement to the second information processing device, and when the second information processing device is within a distance where short-range wireless communication is possible with the first information processing device, it transmits the biometric information during movement to the first information processing device. The first information processing device can then pass the acquired biometric information to the first analysis unit.

[0170] The first information processing device described above can utilize a rechargeable battery, a single-use battery, or new energy sources as its power source.

[0171] The information processing method in this embodiment is characterized by causing one or more computer processors to execute a biometric information acquisition step of acquiring the user's biometric information, an image acquisition step of acquiring at least an image of the user's face, an environmental information acquisition step of acquiring environmental information, a display step of displaying at least a portion of the biometric information acquired in the biometric information acquisition step and at least a portion of the environmental information acquired in the environmental information acquisition step, a first analysis step of analyzing the user's health status based on the biometric information acquired in the biometric information acquisition step and the user's genome information stored in a predetermined storage unit, a determination step of determining whether or not there is an abnormality in the health status analyzed in the first analysis step, and an output step of outputting the result of the determination in the determination step.

[0172] The computer program in this embodiment is characterized by implementing on one or more computer processors: a biometric information acquisition function for acquiring the user's biometric information; an image acquisition function for acquiring at least an image of the user's face; an environmental information acquisition function for acquiring environmental information; a display function for displaying at least a portion of the biometric information acquired by the biometric information acquisition function and at least a portion of the environmental information acquired by the environmental information acquisition function; a first analysis function for analyzing the user's health status based on the biometric information acquired by the biometric information acquisition function and the user's genome information stored in a predetermined memory unit; a determination function for determining whether or not there is an abnormality in the health status analyzed by the first analysis function; and an output function for outputting the result of the determination by the determination function.

[0173] The information processing device in this embodiment is characterized by comprising: a storage unit for storing the user's genome information; a biometric information acquisition unit for acquiring the user's biological information; an image acquisition unit for acquiring at least an image of the user's face; an environmental information acquisition unit for acquiring environmental information; a display unit for displaying at least a portion of the biometric information acquired by the biometric information acquisition unit and at least a portion of the environmental information acquired by the environmental information acquisition unit; a first analysis unit as an AI platform for analyzing the user's health status based on the biometric information acquired by the biometric information acquisition unit and the genome information stored in the storage unit; a determination unit for determining whether or not there is an abnormality in the health status analyzed by the first analysis unit; and an output unit for outputting the result of the determination by the determination unit.

[0174] The invention in this embodiment provides a technical improvement that solves or mitigates at least some of the problems of the prior art described above. Specifically, it provides an information processing system, information processing method, computer program, and information processing device that can analyze a user's health status based on acquired biometric information and output the analysis results, even without being connected to the internet.

[0175] An embodiment of the information processing system in this embodiment will be described with reference to the drawings.

[0176] Figure 15 shows an example of the system configuration of the information processing system 2000 in this embodiment.

[0177] In this embodiment, the information processing system 2000 includes a first information processing device 2100, as shown in Figure 15.

[0178] Furthermore, the information processing system 2000 may include a biological information acquisition device 10 that can be connected to the first information processing device 2100, or the functions of the biological information acquisition device 10 may be provided by the first information processing device 2100. Details of the biological information acquisition device 10 will be described later.

[0179] The hardware configuration of the first information processing device 2100 can be the same as that of the information processing device 100 described above.

[0180] Furthermore, the second information processing device 2200, the third information processing device 2300, the fourth information processing device 2400, the fifth information processing device 2500, the sixth information processing device, and the seventh information processing device, which will be described later, can also have the same hardware configuration as the first information processing device 2100.

[0181] Figure 16 shows an image of the external appearance of the first information processing device 2100.

[0182] As shown in Figure 16 as an example, the first information processing device 2100 consists of a housing 106 and a display 107 provided on the front of the housing 106. A camera 108 and a speaker 109 may also be provided on the front of the housing 106. Furthermore, the first information processing device 2100 may be equipped with various sensors, a rechargeable battery, a power cable connection, a SIM card insertion slot, an SD card memory insertion slot, a USB insertion slot, etc. (not shown).

[0183] The first information processing device 2100, as shown as an example in Figure 17, includes a storage unit 2110, a biological information acquisition unit 2120, an image acquisition unit 2130, an environmental information acquisition unit 2140, a display unit 2150, a first analysis unit 2160, a determination unit 2170, and an output unit 2180.

[0184] The memory unit 2110 stores the user's genome information.

[0185] The memory unit 2110 may be the storage 103 of the first information processing device 2100, or it may be a recording medium (USB memory, SD card, etc.) that can be removed from the first information processing device 2100.

[0186] Genomic information may include, for example, data showing the genetic predisposition to a user's diseases and physical characteristics, obtained by analyzing saliva or blood samples taken from the user.

[0187] Furthermore, the memory unit 2110 may include basic information such as the user's address, name, and age; medical history information including a history of illnesses the user has suffered in the past; performance information including the user's daily respiratory rate and heart rate; abnormality history information regarding abnormalities in the user's health condition; and contact information including a list of caregivers who will notify the user of their condition.

[0188] Initially, the performance information registers daily respiratory rate and heart rate values ​​included in the user's application. However, after the first information processing device 2100 is put into use, the measured normal data is saved chronologically so that changes over time can be retrieved.

[0189] The biometric information acquisition unit 2120 acquires the user's biometric information.

[0190] User biometric information includes, for example, at least one of the following: blood pressure, pulse, respiration, consciousness, body temperature, and blood oxygen saturation. Of these, blood pressure, pulse, and respiration are important parameters for analyzing the user's health status.

[0191] Furthermore, the biometric information acquisition unit 2120 can be made capable of acquiring the above-mentioned biometric information from the user in a non-contact manner.

[0192] For example, the biometric information acquisition unit 2120 has a transmitting unit that emits microwaves and a receiving unit that receives microwaves reflected by the user. The user's movement can be detected by the phase difference created by the Doppler effect between the emitted microwaves and the received microwaves.

[0193] This technology allows for the detection of unconscious biological activities such as breathing and heartbeat, based on subtle movements on the body surface associated with respiration and heartbeat. Furthermore, it can also estimate blood pressure fluctuations.

[0194] The biological information acquisition unit 2120 may further include a processing unit that detects the time change in the phase difference associated with the transmission and reception of microwaves and extracts frequency components from the time change to determine the respiratory rate and heart rate.

[0195] The microwaves are emitted from the transmitter at an irradiation angle of approximately 180°, and biological information can be detected within a distance range of approximately 10m from the transmitter.

[0196] The image acquisition unit 2130 acquires at least an image of the user's face.

[0197] The image acquisition unit 2130 acquires an image of the location where the first information processing device 2100 is installed. The image acquisition unit 2130 acquires the image using the camera 108 described above. The image may be a still image or a video. The image acquisition unit 2130 can be the receiving unit 110 described above.

[0198] To avoid privacy issues, the image acquisition unit 2130 is locked in the control program so that it does not operate under normal circumstances. It can also be controlled so that the lock is released and image acquisition becomes possible when an abnormality above a predetermined level is detected.

[0199] Alternatively, the system could be configured to activate camera 108 when a user shouts.

[0200] The environmental information acquisition unit 2140 acquires environmental information.

[0201] The environmental information acquisition unit 2140 acquires environmental information of the space in which the first information processing device 2100 is installed. Here, environmental information refers to information related to the living space, such as room temperature and humidity.

[0202] In one embodiment, the environmental information acquisition unit 2140 includes a temperature sensor and a humidity sensor. In other embodiments, it may include a smoke sensor or a gas leak detection sensor. The temperature sensor may use a thermistor, a resistance thermometer, or any other type. The humidity sensor may use polymer resistance, polymer capacitance, or any other type, but an electrical sensor that can output an electrical signal is used.

[0203] While temperature and humidity sensors can continuously measure room temperature and humidity, these environmental indicators generally do not change rapidly in a short period of time. Therefore, a configuration that intermittently acquires environmental information at predetermined time intervals is also acceptable.

[0204] The system may also include a radio wave receiving sensor that receives standard radio waves carrying time and calendar information, and a barometric pressure sensor. These allow for the acquisition of environmental information such as time and weather.

[0205] The display unit 2150 displays at least a portion of the biological information acquired by the biological information acquisition unit 2120, and at least a portion of the environmental information acquired by the environmental information acquisition unit 2140. The display unit 2150 can be the display described above.

[0206] Figure 18 shows an example of a display screen 151 that the display unit 2150 displays on the display 107, and the screen is provided with a biometric information display area 152 and an environmental information display area 153.

[0207] The biometric information display area 152 displays numerical values ​​and graphs for respiration, heart rate, and blood pressure, as shown in Figure 18, but is not limited to these.

[0208] The environmental information display area 153 displays the time and weather, as shown in Figure 18, but is not limited to these.

[0209] The first analysis unit 2160 analyzes the user's health status based on the biological information acquired by the biological information acquisition unit 2120 and the genomic information stored in the storage unit 2110. In other words, the first analysis unit 2160, acting as an AI platform, uses AI to analyze the user's health status.

[0210] The first analysis unit 2160 has training data of factors highly associated with the onset of specific diseases, and analyzes whether pre-onset tendencies for diseases that the user is genetically likely to develop can be seen in the user's biometric information.

[0211] The determination unit 2170 determines whether or not there is an abnormality in the health condition analyzed by the first analysis unit 2160.

[0212] Specifically, the determination unit 2170 first determines whether or not there is a first abnormality in the health condition. A first abnormality refers to a case where the numerical value of the biological information exceeds a predetermined threshold, regardless of the genomic information. Specifically, a first abnormality refers to a level where the numerical value of the biological information requires immediate medical attention.

[0213] Next, the determination unit 2170 determines whether or not there is a second abnormality in the health condition. A second abnormality refers to a case where, although the numerical values ​​of the biological information are below a predetermined threshold, the genomic information shows a pre-symptomatic tendency for a disease that the user is genetically likely to develop. Specifically, a second abnormality refers to a level where, although the numerical values ​​of the biological information do not require immediate medical attention, the genomic information shows a pre-symptomatic tendency for a disease that the user is genetically likely to develop.

[0214] The criteria used for these determinations are assumed to be stored in advance by the memory unit 2110.

[0215] If no abnormality is detected, the acquired biological information is determined to be the normal data described above and stored in the storage unit 2110. Since the storage unit 2110 has limited capacity, the normal data may be stored by converting spot data at regular time intervals into numerical values ​​to show the passage of time, or it may be stored as numerical data obtained by averaging interval data at regular time intervals. Alternatively, data that has been in storage for a certain period of time may be overwritten with the latest data, so that only the latest data for a certain period of time is always stored, thereby suppressing an increase in storage capacity.

[0216] The determination unit 2170 may rank the abnormality based on the results of the above determination. For example, in one embodiment, the abnormality is ranked into three levels: normal level, level that does not require immediate attention but requires confirmation (second abnormality), and emergency level that requires immediate action (first abnormality).

[0217] The output unit 2180 outputs the result of the determination made by the determination unit 2170.

[0218] Specifically, if the determination unit 2170 determines that there is a first abnormality in the health condition, the output unit 2180 can output a predetermined warning sound and / or warning display from the first information processing device 2100.

[0219] The predetermined warning sound will be output from the speaker 109 of the first information processing device 2100, and the predetermined warning display will be displayed on the display 107 of the first information processing device 2100.

[0220] Furthermore, if the determination unit 2170 determines that there is a first abnormality in the health condition, the output unit 2180 may output a warning sound and / or a warning display from the first information processing device 2100 to other devices, and may also be configured to enable emergency calls. Details will be described later.

[0221] Other devices may include information processing devices owned by the user's family, information processing devices owned by caregivers, and information processing devices owned by hospitals, fire departments, police, security companies, etc. Further details will be provided later.

[0222] Furthermore, if the determination unit 2170 determines that there is a second abnormality in the health condition, the output unit 2180 can output a predetermined notification sound or notification display from the first information processing device 2100.

[0223] The predetermined notification sound will be output from the speaker 109 of the first information processing device 2100, and the predetermined notification display will be displayed on the display 107 of the first information processing device 2100.

[0224] Furthermore, the output unit 2180 may display an icon on the display 107 corresponding to the rank determined by the determination unit 2170. It is preferable to change the shape and color of the icon so that the urgency can be judged at a glance. For example, by displaying a normal level as green, a level that does not require immediate attention but needs to be checked as yellow, and an emergency level requiring immediate action as red, users can intuitively grasp the situation.

[0225] Furthermore, the ranking by the judgment unit 2170 may be performed on both biological information and environmental information, and each ranking may be displayed on the display 107 using icons. This allows users to easily determine what kind of abnormality has occurred.

[0226] The above configuration provides a technical improvement that solves or mitigates at least some of the problems of the conventional technology described above. Specifically, since the first information processing device 2100 itself has the function of holding the user's genome information and performing AI analysis of the information, it is possible to analyze the user's health status and output the analysis results even in an environment that is not connected to the internet (even if it operates standalone without a network).

[0227] Next, other embodiments of the information processing system 2000 in this disclosure will be described.

[0228] As shown in Figure 19, the information processing system 2000 in this disclosure may further include a second information processing device 2200 and a third information processing device 2300.

[0229] The second information processing device 2200 is capable of connecting to the first information processing device 2100 via short-range wireless communication.

[0230] The second information processing device 2200 can be, for example, a smartphone or tablet PC, and can be a device capable of communicating with the outside world using various communication standards. The hardware configuration of the second information processing device 2200 is as described above.

[0231] Furthermore, the first information processing device 2100 and the second information processing device 2200 can be connected using short-range wireless communication such as BLE (registered trademark).

[0232] The third information processing device 2300 is capable of connecting to the second information processing device 2200 via an internet connection.

[0233] The third information processing device 2300 can be, for example, a server or other device capable of communicating with the outside world. The hardware configuration of the third information processing device 2300 is as described above.

[0234] Furthermore, the second information processing device 2200 and the third information processing device 2300 can be connected via an internet connection.

[0235] The third information processing device 2300 has a second analysis unit 2310 and an execution unit 2320, as shown in Figure 20.

[0236] The second analysis unit 2310 analyzes the result of the determination obtained from the first information processing device 2100 via the second information processing device 2200 and determines the appropriate action to take.

[0237] Depending on whether the judgment includes information indicating a primary health abnormality, information indicating a secondary health abnormality, or information indicating no abnormalities at all, the second analysis unit 2310 may analyze the results of its analysis to determine the appropriate action to take.

[0238] The execution unit 2320 performs actions based on the analysis results from the second analysis unit 2310.

[0239] With the above configuration, the first analysis unit 2160 analyzes information in a closed space for in-house use, while the second analysis unit 2310 can perform actions necessary to maintain or improve the user's health by coordinating with other devices connected to the internet.

[0240] Next, I will explain some specific examples of actions.

[0241] As shown in Figure 19, the information processing system 2000 in this disclosure may further include a fourth information processing device 2400.

[0242] The fourth information processing device 2400 is a device that can be connected to the third information processing device 2300 via an internet connection.

[0243] The fourth type of information processing device, 2400, refers to information processing devices owned by the user's family, the user's caregiver, and the user's regular hospital, nearby fire departments, police stations, security companies, etc.

[0244] In this case, the above action is to notify the fourth information processing device 2400 of the result output by the output unit 2180.

[0245] Specifically, the second analysis unit 2310 determines that if it contains information indicating a first type of abnormality in the health condition, it will output a warning sound and / or a warning display to the fourth information processing device 2400.

[0246] Furthermore, as shown in Figure 19, the information processing system 1000 in this disclosure may further include a fifth information processing device 2500.

[0247] The fifth information processing device 2500 is capable of connecting to the third information processing device 2300 via an internet connection.

[0248] Specifically, the fifth information processing device 2500 can be an IoT device located around the user. Examples include internet-connected air conditioners, automatic windows, electronic locks, humidifiers, dehumidifiers, electric devices, aroma diffusers, speakers, and refrigerators.

[0249] At this time, the above action is the transmission of an operation command to the fifth information processing device 2500.

[0250] Examples of operation commands include turning the air conditioner on / off and adjusting the temperature.

[0251] Furthermore, the third information processing device 2300 may control the fifth information processing device 2500 in accordance with the biological information, environmental information, and / or image information obtained from the first information processing device 2100 by the second information processing device 2200.

[0252] Specifically, the AI ​​analysis of the second analysis unit 2310 can determine facial color and expression to provide an optimal space, coordinate with lighting color and relaxing scents, and provide music in conjunction with speakers. Furthermore, it can maintain comfort by operating the air conditioner in conjunction with the AI ​​based on temperature and humidity, check indoor oxygen concentration, and provide notifications and automatically open / close windows with ventilation motors in conjunction with the AI. It may also have functions to notify pre-registered email addresses when the temperature exceeds a certain level, to notify the disaster prevention center or management office, and to automatically unlock doors in conjunction with door locks. Additionally, it may be able to provide guidance on dietary restrictions and recipes in conjunction with an IoT refrigerator.

[0253] Furthermore, if the necessary equipment is not IoT-connected, the system may send advice to the second information processing device 2200 suggesting that it would be better to operate a nearby device.

[0254] Furthermore, the above action can be used to transmit information about lifestyle improvements to the second information processing device 2200.

[0255] At this time, the third information processing device 2300 can acquire pre-approved health checkup data. Lifestyle improvement information will then be generated according to the health checkup data.

[0256] Lifestyle improvement information specifically includes advice on diet, and users can send recommended recipes.

[0257] Furthermore, as shown in Figures 15 and 19, the information processing system 2000 may also include a biological information acquisition device 10 connected to the first information processing device 2100.

[0258] The biological information acquisition device 10 is a separate device that controls the functions of the biological information acquisition unit 2120.

[0259] At this time, the biometric information acquisition unit 2120 acquires the user's biometric information acquired by the biometric information acquisition device 10.

[0260] By separating the biometric information acquisition device 10 from the first information processing device 2100, the biometric information acquisition device 10 can be used in a variety of ways, such as embedding the sensor in the wall of a building, decorating the wall, or attaching it to the back of wallpaper to automatically acquire vital information.

[0261] This configuration can be adopted in facilities with multiple users, such as nursing homes.

[0262] With this configuration, body temperature, blood pressure, pulse, and respiration can be automatically monitored in real time, eliminating the need for frequent measurements by nurses or administrators.

[0263] When a user resides in a nursing home, their primary living space is often a single private room, and the first information processing device 2100 can be installed in one location for monitoring. However, when a user lives in an individual residence such as an apartment or detached house, their living space is often divided into multiple rooms such as a living room and bedroom, requiring the acquisition of biometric information in each room.

[0264] Therefore, in another embodiment of this disclosure, the information processing system 2000 may include one master unit (first information processing device 2100) and multiple slave units (biometric information acquisition devices 10).

[0265] The configuration of the master device is the same as that of the first information processing apparatus 2100 described above. However, the biological information acquisition unit 2120 of the first information processing apparatus 2100 manages, in addition to the biological information acquired by its own biological information acquisition unit 2120 and the environmental information acquired by the environmental information acquisition unit 2140, the biological information and environmental information transmitted from the biological information acquisition device 10, and determines the presence or absence of abnormalities according to a preset abnormality determination criterion.

[0266] If there are multiple biological information acquisition devices 10, there is a risk that it may not be possible to identify which slave device the transmitted biological information and environmental information are from. Therefore, each slave device is assigned a unique ID number, and when biological information and environmental information are transmitted, they are transmitted in association with the unique ID number.

[0267] The biological information acquisition device 10 includes at least a biological information acquisition unit and a communication unit. Since the transmission of information from the slave device to the master device is performed using short-range wireless communication, the slave device can be installed at a free position within the range where the wireless signal can reach. If there are wired terminals installed in each room in the house, they may be connected by wire.

[0268] Note that the slave device does not include an element corresponding to the analysis unit. This is because the analysis is performed by the master device, and thus the slave device can be miniaturized. Also, for slave devices installed in places where privacy is particularly required, such as toilets and bathrooms, a configuration excluding the image acquisition unit may be adopted. Thus, slave devices with different configurations depending on the installation location may be mixed. Also, the installation location and shape may be changed according to the installation location. For example, for a slave device installed in the bedroom of a user who spends a long time sleeping in bed, it may be installed under the bed in order to accurately acquire biological information. Also, the slave device can be installed by embedding it in the wall or ceiling. According to these configurations, it is possible to install the device without making the user aware of the acquisition of biological information more than necessary and without damaging the aesthetics of the installation location.

[0269] The biological information acquisition unit 2120 and / or the biological information acquisition device 10 can acquire the respiration, heartbeat, and blood pressure of the user in a non-contact manner.

[0270] These can be obtained in a non-contact manner by the transmission and reception of the above-described micro waves.

[0271] In addition, the biological information acquisition unit 2120 and / or the biological information acquisition device 10 can acquire the electroencephalogram of the user in a contact or non-contact manner.

[0272] The electroencephalogram can be obtained by a known electroencephalogram sensor. Also, the technology for measuring the electroencephalogram in a non-contact manner may be realized by known or future technologies.

[0273] In addition, the biological information acquisition unit 2120 and / or the biological information acquisition device 10 can acquire the body surface temperature of the user in a non-contact manner.

[0274] The body surface temperature can be obtained by a known thermosensor. As an example, the above-described infrared camera can be used.

[0275] In addition, the biological information acquisition unit 2120 and / or the biological information acquisition device 10 can acquire the blood glucose level of the user in a contact or non-contact manner.

[0276] The blood glucose level can be obtained by a known blood glucose sensor. Also, the technology for measuring the blood glucose level in a non-contact manner may be realized by known or future technologies.

[0277] The biological information acquisition unit 2120 and / or the biological information acquisition device 10 can be configured to be able to acquire information by short-range communication with a vital measurement sensor embedded in a part of the user's body.

[0278] And the environmental information acquisition unit 2140 can acquire at least one of the temperature, humidity, atmospheric pressure, and oxygen concentration at the location where the first information processing device 2100 is disposed.

[0279] These can be obtained by the temperature sensor, humidity sensor, pressure sensor, and oxygen concentration sensor provided in the first information processing device 2100.

[0280] Furthermore, the values ​​obtained from the barometric pressure sensor can be used to display weather forecasts.

[0281] Furthermore, the environmental information acquisition unit 2140 may acquire the time. In this case, the time can be acquired by the radio wave receiving sensor provided in the first information processing device 2100. Note that the time set in the first information processing device 2100 may be used.

[0282] These may all be displayed on display 107, or only those that have been configured may be displayed. This configuration may be made possible by the user.

[0283] With the above configuration, it is possible to create a comfortable space that helps maintain or improve the health of users.

[0284] Furthermore, the environmental information acquisition unit 2140 may acquire values ​​from fire sensors and sound sensors. These sensors may be provided by the first information processing device 2100, or they may be obtained from other connectable devices.

[0285] The first analysis unit 2160 can further analyze the user's facial image to obtain skin tone information and / or facial expression information, and analyze the user's health status based on the skin tone information and / or facial expression information, the biometric information obtained by the biometric information acquisition unit, and the genomic information stored in the memory unit.

[0286] Facial color and facial expression information are important decision-making factors, alongside biological information, and when combined with genome analysis data, they can enable the early detection of certain diseases.

[0287] In addition, facial information and expression information can be used as indicators for measuring the comfort level of the environment. Based on such information, the fifth information processing device 500 can be operated.

[0288] As an example, an optimal space (environment) can be provided by coordinating with electric light color, providing in cooperation with a relaxation scent, and providing music in cooperation with a speaker.

[0289] As shown in FIG. 21, the first information processing device 2100 can further include a user identification unit 2190.

[0290] The user identification unit 2190 identifies the user based on the image acquired by the image acquisition unit 2130.

[0291] Although it has been described that the image acquired by the image acquisition unit 2130 is used for analyzing the user's expression and complexion, here, it can be used to implement a face authentication function for identifying the user.

[0292] Known techniques can be used for face authentication technology.

[0293] According to the above configuration, it becomes possible to realize usage by a plurality of users.

[0294] Also, as shown in FIG. 21, the first information processing device 2100 in the present disclosure can further include a communication unit 2195. [[ID=3,0]]

[0295] The communication unit 2195 enables the first information processing device 2100 to communicate with an external device only under specific conditions.

[0296] As an example, the specific conditions refer to the case where it is determined by the determination unit 2170 that there is a first abnormality in the health state.

[0297] To enable the above-mentioned communication by the communication unit 2195, the first information processing device 2100 may be equipped with a wired LAN connection terminal and / or a wireless LAN card. Furthermore, it may be equipped with a SIM card insertion terminal to enable communication even when an internet connection environment is not available. By inserting a SIM card, it becomes possible to communicate with external devices via the mobile phone's wireless communication network.

[0298] Specifically, the first information processing device 2100 will be able to communicate directly with the fourth information processing device 2400 without going through the second information processing device 2200 and / or the third information processing device 2300.

[0299] With the above configuration, it becomes possible to respond quickly in emergencies.

[0300] Next, embodiments of the information processing method described herein will be explained with reference to the drawings.

[0301] As shown in Figure 22 as an example, the information processing method in this disclosure is characterized by causing one or more computer processors to execute a biometric information acquisition step S2120, an image acquisition step S2130, an environmental information acquisition step S2140, a display step S2150, a first analysis step S2160, a determination step S2170, and an output step S2180.

[0302] In the biometric information acquisition step S2120, the user's biometric information is acquired. This biometric information acquisition step S2120 can be performed by the biometric information acquisition unit 2120 described above. Details of the biometric information acquisition unit 2120 are as described above.

[0303] In the image acquisition step S2130, an image of at least the user's face is acquired. This image acquisition step S2130 can be performed by the image acquisition unit 2130 described above. Details of the image acquisition unit 2130 are as described above.

[0304] In the environmental information acquisition step S2140, environmental information is acquired. This environmental information acquisition step S2140 can be performed by the environmental information acquisition unit 2140 described above. Details of the environmental information acquisition unit 2140 are as described above.

[0305] In the display step S2150, at least a portion of the biological information acquired in the biological information acquisition step S2120 and at least a portion of the environmental information acquired in the environmental information acquisition step S2140 are displayed. This display step S2150 can be performed by the display unit 2150 described above. Details of the display unit 2150 are as described above.

[0306] In the first analysis step S2160, the user's health status is analyzed based on the biological information acquired in the biological information acquisition step S2120 and the user's genome information stored in a predetermined memory unit. This first analysis step S2160 can be performed by the first analysis unit 2160 described above. Details of the first analysis unit 2160 are as described above.

[0307] In the determination step S2170, it is determined whether or not there is an abnormality in the health condition analyzed in the first analysis step S2160. This determination step S2170 can be performed by the determination unit 2170 described above. Details of the determination unit 2170 are as described above.

[0308] In the output step S2180, the result of the determination in the determination step S2170 is output. This output step S2180 can be performed by the output unit 2180 described above. Details of the output unit 2180 are as described above.

[0309] The above configuration provides a technical improvement that solves or mitigates at least some of the problems of the conventional technology described above. Specifically, since the first information processing device itself holds the user's genome information and has the function of analyzing the information with AI, it becomes possible to analyze the user's health status and output the analysis results even in an environment that is not connected to the internet.

[0310] Next, embodiments of the computer program described herein will be explained with reference to the drawings.

[0311] The computer program in this disclosure is characterized by enabling one or more computer processors to implement a biological information acquisition function, an image acquisition function, an environmental information acquisition function, a display function, a first analysis function, a judgment function, and an output function.

[0312] The biometric information acquisition function acquires the user's biometric information.

[0313] The image acquisition function acquires at least an image of the user's face.

[0314] The environmental information acquisition function acquires environmental information.

[0315] The display function displays at least a portion of the biometric information acquired by the biometric information acquisition function, and at least a portion of the environmental information acquired by the environmental information acquisition function.

[0316] The first analysis function analyzes the user's health status based on biometric information acquired by the biometric information acquisition function and the user's genome information stored in a designated memory unit.

[0317] The judgment function determines whether or not there is an abnormality in the health condition analyzed by the first analysis function.

[0318] The output function outputs the result of the judgment made by the judgment function.

[0319] The above-mentioned biological information acquisition function, image acquisition function, environmental information acquisition function, display function, first analysis function, judgment function, and output function can be realized by the biological information acquisition circuit 2121, image acquisition circuit 2131, environmental information acquisition circuit 2141, display circuit 2151, first analysis circuit 2161, judgment circuit 2171, and output circuit 2181 shown in Figure 23. The biological information acquisition circuit 2121, image acquisition circuit 2131, environmental information acquisition circuit 2141, display circuit 2151, first analysis circuit 2161, judgment circuit 2171, and output circuit 2181 are realized by the biological information acquisition unit 2120, image acquisition unit 2130, environmental information acquisition unit 2140, display unit 2150, first analysis unit 2160, judgment unit 2170, and output unit 2180, respectively. Details of each unit are as described above.

[0320] The above configuration provides a technical improvement that solves or mitigates at least some of the problems of the conventional technology described above. Specifically, since the first information processing device itself has the function of storing the user's genome information and performing AI analysis on the acquired information, it becomes possible to analyze the user's health status and output the analysis results even in an environment that is not connected to the internet.

[0321] Finally, we will explain the circumstances that led the inventors to create the present invention.

[0322] As the aging rate of society rapidly increases, medical benefit expenditures due to the expansion of social security are expected to reach approximately 54 trillion yen in fiscal year 2025, up from the current approximately 36 trillion yen. Per capita medical expenses increase with age, except for infancy, but increase rapidly after age 65, and after age 80, the proportion of expenses related to hospitalization (hospitalization + meals and living expenses) becomes high. Currently, more than one-third of medical treatment expenses (fiscal year 2013) are related to lifestyle-related diseases, and diseases associated with aging and mental and neurological disorders account for a large proportion of these expenses.

[0323] Amidst the economic and employment uncertainty caused by the spread of the novel coronavirus, which has led to restrictions on contact with others and self-isolation due to health concerns, working styles are changing. As a result, awareness of health is higher than ever before, including health management through remote work and caregiving due to the aging population.

[0324] In response to these societal lifecycle changes and technological advancements driven by AI, this invention aims to provide a service that utilizes personal genome data to predict bodily abnormalities and manage health through contactless healthcare, thereby contributing to improved lifestyles and a more comfortable living environment.

[0325] Currently, home-use electronic blood pressure monitors are broadly divided into two main types: "upper arm type," which measures blood pressure on the upper arm, and "wrist type," which measures blood pressure on the wrist. Furthermore, "upper arm type" monitors can be divided into two types: those in which the user wraps a cuff (armband) around their upper arm themselves, and those in which the user puts their arm through a cylindrical cuff that is integrated with the blood pressure monitor body.

[0326] Healthcare devices include smartwatches, finger-worn devices, and devices that connect to smartphones, etc. People with pre-existing conditions related to blood pressure, such as hypertension and diabetes, need to measure their blood pressure and have blood tests done daily or regularly.

[0327] With contact-type sensors, data is often not collected due to the stress of wearing them or the tendency to forget to wear them, which are natural aspects of human daily life. By creating a non-contact type, vital data can be acquired simply by being present without wearing anything, and non-contact vital sensors can contribute to human health without interfering with daily life.

[0328] Furthermore, by inputting genomic information, we aim to provide effective personalized services tailored to each individual. Additionally, it can be operated as a standalone device anywhere, without needing an internet connection. If realized, it possesses the elements to be recognized as a business that contributes to society.

[0329] Furthermore, using the individual's genomic data, vital data, complexion and facial expressions, and brainwave information, a standalone AI and a dedicated app can work in conjunction with AI services to provide an optimal living environment, enabling early detection and notification of symptoms, monitoring of lifestyle habits, and guidance for improvement.

[0330] Examples of non-contact vital signs sensors include the following: • Respiration, heart rate, blood pressure (estimated) • Face recognition sensor • Identifying individuals through facial recognition • Estimating the state based on complexion and facial expression • Thermosensor (body surface temperature) • Electroencephalogram (EEG) measurement (connected or non-contact) • Diabetes test sensor

[0331] In one embodiment of this disclosure, a server device, data processing system, data processing method, and program are provided that acquire a person's health status and inform a dedicated display device of ways to improve lifestyle and environment, using a device programmed with AI that analyzes the person's genome analysis data acquired in advance, facial expressions and behavioral patterns (movements), a device that acquires non-contact vital information (heart rate, respiration, blood pressure, body temperature, etc.) using microwaves, a device that can measure brain waves, a device that measures temperature using infrared rays, a device that measures oxygen concentration, and a device that displays the weather forecast and time.

[0332] The invention disclosed herein detects early signs of illness in users. However, network connectivity for disease prediction presents security challenges such as the leakage of measurement data, and it cannot be used in locations without network connectivity. Furthermore, it is not useful for improving lifestyle habits, as it does not predict illness.

[0333] For example, AI can optimize health conditions by performing automatic ventilation control, automatic lighting control, fragrance control, and door control based on a comprehensive assessment, thereby enabling lifestyle improvements.

[0334] The invention described herein involves loading genome analysis data, which has been previously confirmed by the individual, into a standalone device via a dedicated app.

[0335] If you collapse at home, vital signs such as heart rate and respiration can be measured to understand changes in your body. However, due to the accuracy of the sensors, there is a possibility of malfunctions. Therefore, if the AI ​​behavior pattern (movement) is not natural, the AI ​​equipped with vital data will make a comprehensive judgment and send an alert.

[0336] According to this invention, accumulated data can be analyzed by AI to pre-control a comfortable environment. For example, the AI ​​can optimize health conditions by performing automatic ventilation control, automatic lighting control, fragrance control, door control, etc.

[0337] Furthermore, the information processing system disclosed herein can be controlled in conjunction with a smart home in a residential or shared facility.

[0338] In another embodiment, the information processing system of the present disclosure may further include a second information processing device that can be connected to the first information processing device by short-range wireless communication, and a sixth information processing device that can be connected to the second information processing device by short-range wireless communication.

[0339] The sixth information processing device shall be installed on a designated means of transportation. Specifically, the designated means of transportation may be a vehicle such as a bus or an airplane.

[0340] In this configuration, the sixth information processing device acquires the user's biometric information during movement and transmits the acquired biometric information during movement to the second information processing device. The second information processing device, when within short-range wireless communication distance of the first information processing device, transmits the biometric information during movement to the first information processing device. The first information processing device then passes the acquired biometric information during movement to the first analysis unit.

[0341] With this configuration, passengers or other crew members in limited spaces on modes of transport such as buses and airplanes can perform health checks by scanning a dedicated identification number on their smartphone with a dedicated app.

[0342] In another embodiment, the information processing system may further include a second information processing device that can be connected to the first information processing device by short-range wireless communication, and a seventh information processing device that can be connected to the second information processing device by short-range wireless communication.

[0343] The seventh information processing device shall be equipped with a predetermined means of transportation. Specifically, the seventh information processing device may be a mobile robot.

[0344] In this configuration, the seventh information processing device acquires the user's biometric information, transmits the acquired biometric information during movement to the second information processing device, the second information processing device transmits the biometric information during movement to the first information processing device when it is within range of short-range wireless communication with the first information processing device, and the first information processing device passes the acquired biometric information to the first analysis unit.

[0345] With this configuration, the robot will be able to move around in a specific location while monitoring the user's health condition.

[0346] The notification method may include a standalone device for notification, a function to notify a third party via a dedicated app on the user's smartphone that connects to an external AI, and a function to view the history of acquired data.

[0347] Furthermore, the display screen can be made more convenient by showing weather information and the time, which are among the first things people want to know when they wake up each day.

[0348] Furthermore, by linking with sensors that measure oxygen and PM concentrations, the surrounding environment can be made more comfortable.

[0349] The information processing system 1000 in this disclosure may be equipped with a rechargeable battery, a single-use battery, or a new energy source (solar power, wind power, etc.) and capable of taking measurements anywhere.

[0350] The information processing system 1000 in this disclosure may be configured to acquire vital data in residential or shared facilities in conjunction with a slave unit.

[0351] The information processing system 1000 in this disclosure may be embedded in the wall or ceiling of a residential facility or other facility, linked to each person's ID that has been registered in advance, and may be used to obtain the data of the person in question in advance, and to check and notify them of their condition.

[0352] The information processing system 1000 in this disclosure may be equipped with a function that allows a physician to view data remotely, and may also be used to reduce the time it takes for a physician to perform remote medical consultations using a camera.

[0353] In the information processing system 1000 described herein, in order to save power, various sensors may be activated when a person is detected by a motion sensor.

[0354] Furthermore, the biological information acquisition unit and / or biological information acquisition device can acquire biological information at predetermined times or at predetermined time intervals.

[0355] Next, another embodiment of the information processing system in this disclosure will be described. In this embodiment, the information processing device 100 may include an information processing unit, a control unit, a communication unit, a recording unit, a display unit, and an input unit, as shown in Figure 24.

[0356] The input section accepts data for registering and correcting user information. Users can register their age, gender, actual weight, blood type, height, etc., through the input section.

[0357] The control unit controls the entire system.

[0358] The information processing unit performs noise reduction, surface area calculation, and volume calculation on the data acquired from the sensor.

[0359] Sensors that can be connected include visible light cameras, infrared sensors, and ultrasonic sensors.

[0360] Visible light cameras are used to measure the entire body and acquire data to determine gender and age categories.

[0361] Infrared sensors are used to measure the entire body from body heat, even while wearing clothes, and to acquire data for 3D modeling.

[0362] Ultrasonic sensors are used to acquire data for defining the structural divisions of the human body (bones, muscles, etc.).

[0363] The reflective portion reflects specific wavelengths of infrared and ultrasonic waves.

[0364] The recording unit is responsible for recording all data.

[0365] The display unit shows a user interface (UI) that the user can select.

[0366] The communications unit is responsible for connecting to the network. Connection is possible using methods such as Wi-Fi (registered trademark) and Bluetooth (registered trademark).

[0367] In the cloud, data is managed.

[0368] The mobile device is used to manage weight data using a dedicated app.

[0369] Furthermore, a computer or mobile phone or other information processing device can be suitably used to function as a server or terminal device according to the above-described embodiment. Such an information processing device can be realized by storing a program describing the processing content that realizes each function of the server or terminal device according to the embodiment in the storage unit of the information processing device, and having the CPU of the information processing device read and execute the program.

[0370] While several embodiments of the present invention have been described, these embodiments are presented as examples only and are not intended to limit the scope of the invention. These novel embodiments can be carried out in a variety of other forms, and various omissions, substitutions, and modifications can be made without departing from the spirit of the invention. These embodiments and their variations are included in the scope and spirit of the invention, as well as in the claims of the invention and its equivalents.

[0371] Furthermore, the methods described in the embodiments can be distributed by storing the program, which can be executed by a computer, on recording media such as magnetic disks (floppy disks, hard disks, etc.), optical disks (CD-ROMs, DVDs, MOs, etc.), and semiconductor memory (ROMs, RAMs, flash memory, etc.), and by transmitting them via communication media. The program stored on the media also includes a configuration program that configures the software means (including not only the executable program but also tables and data structures) to be executed by the computer. The computer implementing this device reads the program recorded on the recording media and, if necessary, constructs the software means using the configuration program, and executes the above-described process by controlling its operation with this software means. The recording media referred to in this specification are not limited to those for distribution, but also include storage media such as magnetic disks and semiconductor memory provided inside the computer or in devices connected via a network. The storage unit may function as, for example, main memory, auxiliary memory, or cache memory. [Explanation of symbols]

[0372] 1000 Information Processing Systems 100 Information Processing Devices 110 Receiver 120 Processing Unit 200 measuring devices 210 Imaging device 220 Infrared Sensor Unit 230 Ultrasonic Sensor Unit

Claims

1. An information processing system comprising one or more computer processors, The information processing system comprises a measuring device including an imaging device, an infrared sensor unit, and an ultrasonic sensor unit, and an information processing device connected to the measuring device. The one or more computer processors described above are: A determination unit that analyzes the image captured by the aforementioned imaging device to determine whether the object to be measured has entered a predetermined measurement area, When the determination unit determines that the object to be measured has entered a predetermined measurement area, the facial recognition unit determines whether or not the object to be measured is a registered user. If the facial recognition unit determines that the subject to be measured is not a registered user, a confirmation unit confirms whether or not to proceed with the measurement for that subject. A receiving unit that receives the data acquired by the measuring device, A processing unit that processes the data received by the receiving unit to calculate the weight of the object to be measured. It has, The processing unit is an information processing system that calculates the volume of each part of the object to be measured based on the data received by the receiving unit, and calculates the weight of the object to be measured based on the volume of each part, information regarding the configuration of each part, and the specific gravity of the elements related to the configuration.

2. The information processing system according to claim 1, characterized in that the acquisition of data by the measuring device is initiated when the verification unit confirms that the measurement can be performed.

3. The information processing system according to claim 1, characterized in that the measurement by the infrared sensor unit and the ultrasonic sensor unit is initiated when the confirmation unit confirms that the measurement can be performed.

4. The information processing system according to claim 1, characterized in that, when the verification unit determines that the measurement target is not a registered user, it displays a verification screen on a display provided by the information processing system or on the display of the user's information processing terminal.

5. The aforementioned processing unit, The information processing system according to claim 1, characterized in that the receiving unit estimates the composition of the object to be measured based on the reflection data from the object to be measured acquired by the ultrasonic sensor unit, which is received by the receiving unit.

6. The information processing system according to claim 1, characterized in that the processing unit generates 3D data of the object to be measured based on the reflection data and / or radiation data from the object, including the object to be measured, which has been received by the receiving unit and acquired by the infrared sensor unit.

7. The information processing system according to claim 1, characterized in that the processing unit generates 3D data of the object to be measured based on image data including the object to be measured, which has been received by the receiving unit and acquired by the imaging device.

8. The aforementioned processing unit, The information processing system according to claim 6 or 7, characterized in that, based on the 3D data of the object to be measured, a three-dimensional object having an external shape corresponding to the object to be measured is divided into a plurality of parts of a predetermined size, the weight of each part is calculated based on the information regarding the configuration of each part and the specific gravity of the elements related to said configuration, and the weight of the object to be measured is obtained by summing the weights of all parts.

9. The information processing system according to claim 1, further characterized in that the processing unit performs background removal from the image data by analyzing the image data including the object to be measured, which has been received by the receiving unit and acquired by the imaging device.

10. The information processing system according to claim 1, characterized in that the infrared sensor unit comprises an infrared camera and an infrared sensor composed of an infrared light emitter, an infrared light receiver, and a reflector.

11. The information processing system according to claim 1, characterized in that the ultrasonic sensor unit comprises a main unit consisting of an ultrasonic light emitting unit and an ultrasonic light receiving unit, and a sub-unit consisting of an ultrasonic light receiving unit and a reflecting unit.

12. The information processing system according to claim 11, wherein the ultrasonic sensor unit comprises a main unit and two sub-units, and is arranged such that the horizontal angle of the angle formed by the straight line connecting the main unit and each of the sub-units, and the vertical angle of the straight line connecting the main unit and each of the sub-units, are within a predetermined value.

13. The information processing system according to claim 1, characterized in that the measuring device is used to be positioned in front of the object to be measured or above the object to be measured.

14. The information processing system according to claim 1, wherein the one or more computer processors further have a storage unit for storing user information of the user to be measured, and the processing unit determines whether the image data of the user's face to be measured, which has been received by the receiving unit and acquired by the imaging device, is included in the user information stored in the storage unit, and if, as a result of the determination, it is determined that the image data of the user's face is included in the user information, the processing unit designates the user as the target of measurement.

15. The memory unit further stores information regarding the user's actual weight. The information processing system according to claim 14, characterized in that the processing unit compares the user's weight, which is the calculated weight of the object to be measured, with the user's actual weight stored in the storage unit, and determines whether there is a difference of a predetermined amount or more between the user's weight and the user's actual weight as a result of the comparison.

16. The information processing system according to claim 14, characterized in that, if the processing unit determines that there is a difference of a predetermined amount or more between the user's weight and the user's actual weight, it requests the measuring device to acquire data again.

17. An information processing method in an information processing system comprising one or more computer processors, The information processing system comprises a measuring device including an imaging device, an infrared sensor unit, and an ultrasonic sensor unit, and an information processing device connected to the measuring device. The one or more computer processors: A determination step in which the object to be measured enters a predetermined measurement area by analyzing the image captured by the aforementioned imaging device, If it is determined in the aforementioned determination step that the object to be measured has entered a predetermined measurement area, a facial recognition step is performed to determine whether or not the object to be measured is a registered user. In the facial recognition step, if it is determined that the subject to be measured is not a registered user, a confirmation step is taken to confirm whether or not to perform the measurement on the subject. The steps include receiving the data acquired by the measuring device, A step of calculating the volume of each part of the object to be measured based on the aforementioned data, A step of calculating the weight of the object to be measured based on the volume of each of the aforementioned parts, information regarding the composition of each of the aforementioned parts, and the specific gravity of the elements related to said composition. An information processing method that enables execution of [the specified action].

18. A computer program executed in an information processing system comprising one or more computer processors, The information processing system comprises a measuring device including an imaging device, an infrared sensor unit, and an ultrasonic sensor unit, and an information processing device connected to the measuring device. The one or more computer processors: The imaging device analyzes the images captured by the aforementioned imaging device to determine whether the object to be measured has entered a predetermined measurement area. When the aforementioned determination function determines that the subject to be measured has entered a predetermined measurement area, a facial recognition function determines whether or not the subject to be measured is a registered user. The facial recognition function determines that the subject to be measured is not a registered user, and a confirmation function is provided to confirm whether or not to proceed with the measurement for that subject. The measuring device includes a receiving function for receiving data acquired by the measuring device, and a processing function for calculating the weight of the object to be measured by processing the data received by the receiving function. To make it happen, The aforementioned processing function is Based on the data received by the receiving function, the volume of each part of the object to be measured is calculated. A computer program that calculates the weight of the object to be measured based on the volume of each of the aforementioned parts, information regarding the composition of each of the aforementioned parts, and the specific gravity of the elements related to said composition.

Images