Computer program, information processing method, information processing device, and information processing system

Abstract

This information processing method for determining an abnormality in the heart or a blood vessel of a living body comprises: identifying a displacement site at which a surface of the living body is displaced due to the heart or the blood vessel and that is detectable from a first direction; identifying a displacement site at which the surface of the living body is displaced due to the heart or the blood vessel and that is detectable from a second direction; detecting the pulsation of the heart or the blood vessel on the basis of the displacements of the surface of the living body at the identified displacement sites; and determining an abnormality in the heart or the blood vessel on the basis of information related to the detected pulsation of the heart or the blood vessel.

Description

[Technical Field]

[0001] The present invention relates to a computer program, an information processing method, an information processing device, and an information processing system. [Background technology]

[0002] Patent Document 1 discloses a biological information monitoring device that monitors biological information such as pulse rate by emitting high-frequency electromagnetic waves, detecting reflected waves scattered on the surface of a human body, and calculating the temporal fluctuations of the surface of the body. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Application Laid-Open No. 2005-270570 Summary of the Invention [Problem to be solved by the invention]

[0004] However, Patent Document 1 does not disclose a technique for determining specific abnormalities in the heart or blood vessels.

[0005] An object of the present invention is to provide a computer program, an information processing method, an information processing device, and an information processing system that can detect the pulsation of the heart or blood vessels of a living body and determine abnormalities in the heart or blood vessels. [Means for solving the problem]

[0006] The computer program of this embodiment is a computer program for causing a computer to determine abnormalities in the heart or blood vessels of a living body, and causes the computer to perform the following processes: identify a displacement site detectable from a first direction where the body surface is displaced by the heart or blood vessels; identify a displacement site detectable from a second direction where the body surface is displaced by the heart or blood vessels; detect the pulsation of the heart or blood vessels based on the displacement of the body surface at the identified displacement site; and determine abnormalities in the heart or blood vessels based on information related to the detected pulsation of the heart or blood vessels.

[0007] The information processing method of this aspect is an information processing method for determining abnormalities in the heart or blood vessels of a living body, and includes identifying a displacement site detectable from a first direction where the body surface is displaced by the heart or blood vessels, identifying a displacement site detectable from a second direction where the body surface is displaced by the heart or blood vessels, detecting the pulsation of the heart or blood vessels based on the displacement of the body surface at the identified displacement site, and determining abnormalities in the heart or blood vessels based on information related to the detected pulsation of the heart or blood vessels.

[0008] The information processing device of this embodiment is an information processing device that determines abnormalities in the heart or blood vessels of a living body, and includes a first identification unit that identifies a displacement site that is detectable from a first direction where the body surface is displaced by the heart or blood vessels, a second identification unit that identifies a displacement site that is detectable from a second direction where the body surface is displaced by the heart or blood vessels, a detection unit that detects the pulsation of the heart or blood vessels based on the displacement of the body surface at the displacement site identified by the first identification unit and the second identification unit, and a determination unit that determines abnormalities in the heart or blood vessels based on information related to the detected pulsation of the heart or blood vessels.

[0009] The information processing system according to this aspect is an information processing system including an information processing device and a sensor device for determining abnormalities in the heart or blood vessels of a living body, wherein the information processing device includes a first infrared sensor or visible light sensor for identifying a displacement site where the surface of the living body is displaced by the heart or blood vessels from a first direction, and a first millimeter wave sensor or terahertz band sensor for irradiating the living body with millimeter wave or terahertz band electromagnetic waves in the first direction and receiving reflected waves from the displacement site of the living body, and the sensor device includes a second infrared sensor or visible light sensor for identifying a displacement site where the surface of the living body is displaced by the heart or blood vessels from a second direction, and a second millimeter wave sensor or terahertz band sensor for irradiating the living body with millimeter wave or terahertz band electromagnetic waves in the second direction and receiving reflected waves from the displacement site of the living body. and a second millimeter wave sensor or terahertz band sensor that receives waves, and an identification unit that identifies the displacement site based on signal data output from the first infrared sensor or visible light sensor and the second infrared sensor or visible light sensor; a detection unit that irradiates the displacement site identified by the identification unit with millimeter wave or terahertz band electromagnetic waves from the first millimeter wave sensor or terahertz band sensor and the second millimeter wave sensor or terahertz band sensor, and detects heart or blood vessel pulsation based on signal data output from the first millimeter wave sensor or terahertz band sensor and the second millimeter wave sensor or terahertz band sensor; and a determination unit that determines abnormality in the heart or blood vessel based on information related to the detected heart or blood vessel pulsation. [Effects of the Invention]

[0010] Based on the above, it is possible to provide a computer program, an information processing method, an information processing device, and an information processing system that can detect the pulsation of the heart or blood vessels of a living body and determine abnormalities in the heart or blood vessels. [Brief explanation of the drawings]

[0011] [Figure 1] FIG. 1 is an explanatory diagram illustrating an example of the configuration of an information processing system according to a first embodiment. [Figure 2] 1 is a block diagram showing an example of the configuration of an information processing device according to a first embodiment. [Figure 3] FIG. 2 is a conceptual diagram illustrating an example of a learning model according to the first embodiment. [Figure 4] FIG. 1 is a conceptual diagram showing a displaced area, the heart, and blood vessels. [Figure 5] 1 is a block diagram showing an example of the configuration of a sensor device according to a first embodiment. [Figure 6] 4 is a flowchart showing an information processing procedure according to the first embodiment. [Figure 7] 4 is a flowchart showing an information processing procedure according to the first embodiment. [Figure 8] 10 is a flowchart showing a procedure for identifying a displaced portion. [Figure 9] 10 is a flowchart showing a procedure for identifying a displaced portion. [Figure 10] 4 is a flowchart showing a processing procedure of the sensor device according to the first embodiment. [Figure 11] 10 is a flowchart showing an abnormality determination process procedure. [Figure 12] FIG. 10 is a schematic diagram showing an example of a determination result display image. [Figure 13] FIG. 10 is a schematic diagram showing an example of a determination result display image. [Figure 14] FIG. 10 is a schematic diagram showing an example of a determination result display image. [Figure 15] 10 is a flowchart showing an information processing procedure according to the second embodiment. [Figure 16] 11 is a flowchart showing an information processing procedure according to the third embodiment. [Figure 17] FIG. 10 is an explanatory diagram illustrating an example of the configuration of an information processing system according to a fourth embodiment. [Figure 18] 10 is a flowchart showing an information processing procedure according to the fourth embodiment. DETAILED DESCRIPTION OF THE INVENTION

[0012] Specific examples of a computer program, an information processing method, an information processing device, and an information processing system according to embodiments of the present invention will be described below with reference to the drawings. Note that the present invention is not limited to these examples, but is defined by the claims, and is intended to include all modifications within the meaning and scope equivalent to the claims. Furthermore, at least some of the embodiments described below may be combined in any manner.

[0013] (Embodiment 1) 1 is an explanatory diagram showing an example of the configuration of an information processing system according to embodiment 1. The information processing system includes an information processing device 1 according to embodiment 1 and a sensor device 9. The information processing device 1 is wirelessly connected to the sensor device 9, a first communication terminal 2, a second communication terminal 3, a server 4, etc., and can transmit and receive various types of information. The information processing device 1 is a device that uses infrared rays and millimeter waves to detect the pulsation of the heart and blood vessels of a user (living body) and determine whether or not there is an abnormality in the heart and blood vessels. The information processing device 1 is assumed to be installed in a room R where the user is present. The sensor device 9 is a device that detects the pulsation of the heart and blood vessels of the user (living body) in the same manner as the information processing device 1 and transmits the detection results to the information processing device 1. The sensor device 9 is provided, for example, in a lighting device L installed on the ceiling of the room R. The first communication terminal 2 is a communication device used by the user's family. The second communication terminal 3 is a communication terminal used by medical personnel. The server 4 is a device that provides information related to the environment that affects the user's heart rate, such as temperature and humidity. Note that each device may be configured to be connected by a wired cable. In this embodiment, cardiac abnormalities refer to diseases of the heart itself, and vascular abnormalities include abnormalities of the blood vessels themselves, such as arteriosclerosis, and abnormalities of organs, tissues, and parts due to abnormal blood flow, such as cerebral infarction and leg infarction (severe lower limb ischemia).

[0014] 2 is a block diagram showing an example of the configuration of the information processing device 1 according to embodiment 1. The information processing device 1 is a computer including a processing unit 11, a storage unit 12, an infrared sensor 13, a millimeter wave sensor 14, a communication unit 15, an operation unit 16, and a display unit 17. Note that the information processing device 1 may be a multi-computer consisting of multiple computers, or may be a virtual machine virtually constructed by software.

[0015] The processing unit 11 is an arithmetic processing device having one or more central processing units (CPUs), micro-processing units (MPUs), graphics processing units (GPUs), general-purpose computing on graphics processing units (GPGPUs), tensor processing units (TPUs), etc. The processing unit 11 reads and executes a computer program P1 stored in the storage unit 12, thereby executing a process for determining abnormalities in the user's heart and blood vessels.

[0016] The storage unit 12 is a storage device such as a hard disk, an EEPROM (Electrically Erasable Programmable ROM), a flash memory, etc. The storage unit 12 stores a computer program P1, a user DB 18, and a learning model 19, which the processing unit 11 uses to execute a process of causing a computer to determine abnormalities in the user's heart and blood vessels.

[0017] The computer program P1 is a program for causing a computer to function as the information processing device 1 according to the present embodiment 1 and for executing the information processing method according to the present embodiment 1. The computer program P1 causes the computer to execute the following processes: identify a displacement site where the living body surface is displaced by the user's heart or blood vessels; detect the pulsation of the heart or blood vessels based on the displacement of the living body surface at the identified displacement site; and determine abnormalities in the heart or blood vessels based on information related to the detected pulsation of the heart or blood vessels.

[0018] The computer program P1 may be recorded in a computer-readable manner on a recording medium 10. The storage unit 12 stores the computer program P1 read from the recording medium 10 by a reading device (not shown). The recording medium 10 may be a semiconductor memory such as a flash memory, an optical disk, a magnetic disk, a magneto-optical disk, or the like. Alternatively, the computer program P1 according to this embodiment may be downloaded from a program providing server (not shown) connected to a communication network and stored in the storage unit 12.

[0019] The user DB 18 stores basic information about users, such as identification information for identifying users, authentication information for authenticating each individual user, and their names, genders, and ages. The user DB 18 also stores environmental information, such as the user's pulse and heart rate, the detection date and time, temperature, and humidity, detected by the information processing device 1, in association with the user's identification information. The user DB 18 also stores the determination result by the information processing device 1, i.e., information indicating whether the user's heart or blood vessels are normal, in association with the user's identification information. The user DB 18 may be a cloud database.

[0020] The infrared sensor 13 is, for example, an infrared laser such as LiDAR, an infrared camera, or the like, and is a sensor that uses infrared light to non-contactly identify each part of the user's body, or a displacement part of the biological surface that is displaced due to the pulsation of the heart or blood vessels. The infrared sensor 13 is an example of a non-contact sensor that identifies a displacement part of the biological surface from a first direction. The first direction is, for example, a substantially horizontal direction. In other words, the infrared sensor 13 is a sensor that identifies a displacement part of the user from a substantially horizontal direction. The infrared sensor 13 has high spatial resolution and is suitable for capturing the structure of an object. However, it has the disadvantage that it is easily absorbed by clothing, etc., and cannot capture displacement of the biological surface that is hidden by clothing. The substantially horizontal direction does not necessarily have to be a strictly horizontal direction, but may be a vertical direction or a direction oblique to the horizontal plane as long as it is a direction that allows detection of a frontal part of a user standing upright.

[0021] An infrared camera is a camera equipped with a lens and a CMOS image sensor that receives infrared light reflected from the user's biological surface or clothing, and outputs infrared image data (signal data) as two-dimensional information about the user. The infrared image data is image data consisting of multiple pixels arranged approximately horizontally and vertically. On the other hand, the infrared sensor 13, which is a LiDAR, includes a light-emitting element that irradiates infrared rays toward the user and a light-receiving element that receives the infrared rays irradiated and reflected from the user. The light-emitting element is, for example, an infrared laser such as a vertical cavity surface-emitting laser (VCSEL), and irradiates the user with a dot pattern arranged vertically and horizontally. The light-receiving element is, for example, a CMOS image sensor. The infrared sensor 13 calculates the distance to the user based on the round-trip time it takes for the light to be irradiated toward the user, reflected, and returned. The infrared sensor 13 calculates the distance to each dot pattern and outputs point cloud data (signal data) that is three-dimensional information about the user. The point cloud data is, for example, a representation of multiple points on the user's body surface or clothing surface using three-dimensional coordinates. The processing unit 11 can convert the point cloud data into voxel data. The processing unit 11 can also convert the point cloud data or voxel data into two-dimensional infrared image data. For simplicity of explanation, an example will be described below in which two-dimensional infrared image data is used to identify each part of the user's body and displaced parts of the living body surface that are displaced due to pulsation of the heart and blood vessels.

[0022] The millimeter wave sensor 14 is a sensor that uses millimeter waves to detect the pulsation at the displaced part of the user. Although the millimeter wave sensor 14 has inferior spatial resolution compared to the infrared sensor 13, millimeter waves are electromagnetic waves that pass through the user's clothing without being absorbed and are reflected by the body surface, making it suitable for applications that capture displacement of the body surface. In particular, the millimeter wave sensor 14 detects the pulsation at the displaced part of the user by emitting millimeter waves to the user from a first direction and receiving reflected waves from the displaced part of the user. The first direction is a substantially horizontal direction. In other words, the millimeter wave sensor 14 detects the pulsation at the displaced part of the user by emitting millimeter waves in a substantially horizontal direction toward the user and receiving reflected waves from the displaced part of the user. The millimeter-wave sensor 14 includes a synthesizer for generating millimeter-wave signals, a transmitting antenna, a receiving antenna, a mixer, and the like. The transmitting antenna transmits millimeter-wave electromagnetic waves generated by the synthesizer. The receiving antenna receives millimeter-wave electromagnetic waves reflected by the user's biological surface. The mixer is a circuit that mixes the transmitted and received waves to generate an intermediate-frequency signal. The processing unit 11 can calculate the distance to the user based on the intermediate-frequency signal data. The processing unit 11 can particularly calculate the change in distance to a displaced part of the user, i.e., the displacement of the biological surface, and detect the pulsation of the heart or blood vessels at the displaced part. The millimeter-wave sensor 14 can irradiate millimeter waves by electronic scanning, aiming at any displaced part, and the processing unit 11 can detect the pulsation at the displaced part.

[0023] The communication unit 15 includes a processing circuit, a communication circuit, etc. for performing wireless communication processing, and transmits and receives various information between the sensor device 9, the first communication terminal 2, the second communication terminal 3, and the server 4 via a router not shown.

[0024] The operation unit 16 is an input device that accepts user operations on the information processing device 1. The input device is, for example, a pointing device such as a touch panel, or a keyboard.

[0025] The display unit 17 is an output device that outputs the abnormality determination result of the user's heart or blood vessels, and is, for example, a liquid crystal display or an EL display.

[0026] FIG. 3 is a conceptual diagram illustrating an example of the learning model 19 according to the first embodiment. The learning model 19 is a model that recognizes a predetermined object included in an infrared image. For example, by utilizing an image recognition technology using semantic segmentation, the learning model 19 can classify objects pixel by pixel and recognize each part of the human body included in the infrared image as an object. Specifically, the learning model 19 recognizes the face, right head, left head, right neck, left neck, right carotid triangle, left carotid triangle, right chest, left chest, right upper arm, left upper arm, right forearm, left forearm, right carpal, left carpal, right dorsum of the hand, left dorsum of the hand, right dorsum of the foot, left dorsum of the foot, etc. of the human body in the infrared image pixel by pixel. The upper arm, forearm, wrist, palm, and dorsum of the hand constitute the upper limb.

[0027] The learning model 19 is, for example, a convolutional neural network (CNN) that has been trained using deep learning. The learning model 19 has an input layer 19a to which infrared image data is input, an intermediate layer 19b that extracts and restores features of the infrared image, and an output layer 19c that outputs part-extracted image data that indicates objects included in the infrared image in pixel units. The learning model 19 is, for example, a U-Net.

[0028] The input layer 19a of the learning model 19 has a plurality of neurons that accept input of infrared image data, i.e., the pixel values ​​of each pixel that constitutes an infrared image, and passes the input pixel values ​​to the intermediate layer 19b. The intermediate layer 19b has a convolutional layer (CONV layer) and a deconvolutional layer (DECONV layer). The convolutional layer is a layer that compresses the dimensionality of the infrared image data. Object features are extracted through dimensional compression. The deconvolutional layer performs deconvolution processing to restore the original dimensions. The restoration processing in the deconvolutional layer generates part-extracted image data in which each pixel has a pixel value (class data) that corresponds to the object class. The output layer 19c has a plurality of neurons that output the part-extracted image data. The part-extracted image is an image that is classified by part of the human body, for example, color-coded.

[0029] The learning model 19 can be generated by preparing training data including infrared image data obtained by the infrared sensor 13 and part-extracted image data in which class data corresponding to the type of part of the human body is assigned to each pixel of the infrared image, and using the training data to train an untrained neural network.

[0030] According to the learning model 19 trained in this manner, by inputting infrared image data of a human body obtained by the infrared sensor 13 into the learning model 19 as shown in Figure 3, extracted image data of parts of the human body is obtained, in which each part is classified on a pixel-by-pixel basis.

[0031] While the above example describes an example of recognizing each part of the human body in an infrared image, which is two-dimensional information, it may also be configured to recognize each part of the human body in three-dimensional point cloud data or voxel data obtained by LiDAR. The processing unit 11, for example, converts the point cloud data into voxel data, generates multiple two-dimensional image data based on the voxel data, performs image recognition processing on each piece of two-dimensional image data using the same method as above, and reverse-converts the multiple images of the two-dimensional part-extracted image data into voxel data or point cloud data, thereby obtaining data indicating the type of each part of the human body in the three-dimensional data in voxel units or point data units. Furthermore, the type of each part of the human body may be recognized using a learning model 19 such as 3D U-Net, which can directly recognize each part of the user in the voxel data. Alternatively, each part of the human body in the 3D information may be recognized using a known machine learning method.

[0032] The processing unit 11 can recognize each part of the user's body in the infrared image by using the learning model 19. There are displacement parts on the surface of the living body where the pulsation of the heart or blood vessels propagates uniquely and appears as periodic displacement on the surface of the living body. The processing unit 11 identifies the displacement parts based on the recognition result using the learning model 19.

[0033] Figure 4 is a conceptual diagram showing the displaced areas, heart, and blood vessels. Displaced areas include, for example, the neck, temporal region, upper arm, the inner part of the carpus near the thumb, the inner part of the carpus near the little finger, the dorsum of the foot, or the chest. The carotid artery is located in the neck, the superficial temporal artery in the temporal region, the brachial artery in the upper arm, the radial artery in the inner part of the carpus near the thumb, the ulnar artery in the inner part of the carpus near the little finger, the dorsum of the foot, and the heart in the chest.

[0034] 5 is a block diagram showing an example of the configuration of the sensor device 9 according to embodiment 1. The sensor device 9 is a computer including a processing unit 91, a storage unit 92, an infrared sensor 93, a millimeter wave sensor 94, and a communication unit 95, similar to those of the information processing device 1.

[0035] The processing unit 91 is an arithmetic processing device having one or more CPUs, GPUs, GPGPUs, TPUs, etc. The processing unit 91 reads and executes a computer program P2 stored in the storage unit 92, thereby executing a process of detecting the pulsations of the user's heart and blood vessels.

[0036] The memory unit 92 is a storage device such as a hard disk, EEPROM, or flash memory. The memory unit 92 stores a computer program P2 and a learning model 99 that cause the processing unit 91 to execute a process of causing a computer to determine a process for detecting the pulsation of the user's heart and blood vessels. The structure of the learning model 99 is similar to that of the learning model 19. However, the learning model 99 may be trained to recognize each part of the human body from an infrared image of the user captured from above. For example, the learning model 99 may be configured to recognize the head, ears, shoulders, etc., as viewed from above, in addition to the parts that the learning model 19 can recognize.

[0037] The computer program P2 is a program for causing a computer to function as the sensor device 9 according to the present embodiment 1. The computer program P2 is for causing a computer to execute a process of identifying a displacement site where the body surface is displaced by the user's heart or blood vessels, and detecting the pulsation of the heart or blood vessels based on the displacement of the body surface at the identified displacement site.

[0038] The computer program P2 may be recorded in a computer-readable manner on a recording medium 90. The storage unit 92 stores the computer program P2 read from the recording medium 90 by a reading device (not shown). The recording medium 90 may be a semiconductor memory such as a flash memory, an optical disk, a magnetic disk, a magneto-optical disk, or the like. Alternatively, the computer program P2 according to this embodiment may be downloaded from a program providing server (not shown) connected to a communication network and stored in the storage unit 92.

[0039] The infrared sensor 93 is, for example, an infrared laser such as LiDAR, an infrared camera, or the like, and is a sensor that uses infrared rays to identify, in a non-contact manner, each part of the user's body, or a displaced part of the biological surface that is displaced due to the pulsation of the heart or blood vessels, from a second direction. The second direction is, for example, a substantially vertical direction. That is, the infrared sensor 93 is a sensor that identifies a displaced part of the user from above. The infrared sensor 93 is an example of a non-contact sensor for identifying a displaced part of the biological surface, and the structure of the infrared sensor 93 is similar to that of the infrared sensor 13. Note that the substantially vertical direction does not necessarily mean a strict vertical direction, but may be a direction oblique to the vertical direction and the horizontal plane, as long as the part of the user standing upright can be detected from the top of the head.

[0040] The millimeter wave sensor 94 is a sensor for detecting pulsation at the displaced part of the user using millimeter waves. The structure of the millimeter wave sensor 94 is similar to that of the millimeter wave sensor 14. In particular, the millimeter wave sensor 94 detects pulsation at the displaced part of the user by irradiating the user with millimeter waves from a second direction and receiving reflected waves from the displaced part of the user. The second direction is approximately vertical. In other words, the millimeter wave sensor 94 detects pulsation at the displaced part of the user by irradiating the user with millimeter waves from above downward and receiving reflected waves from the displaced part of the user.

[0041] The communication unit 95 includes a processing circuit, a communication circuit, and the like for performing wireless communication processing, and transmits and receives various information to and from the information processing device 1 via a router (not shown).

[0042] 6 and 7 are flowcharts showing information processing procedures according to the first embodiment. The information processing device 1 executes the following processing periodically at any timing, for example, three times a day. The processing unit 11 transmits an instruction signal to the sensor device 9 to instruct the sensor device 9 to identify the user's displaced part and detect the pulse (step S110). That is, the processing unit 11 detects the user's displaced part and pulse captured from above, and requests the sensor device 9 to transmit information on the pulse detection result indicating the pulse at the detected displaced part. Details of the processing on the sensor device 9 side will be described later.

[0043] The processing unit 11 detects the user by infrared rays using the infrared sensor 13 (step S111). Specifically, if the infrared sensor 13 is an infrared camera, the processing unit 11 captures an image of the user using the infrared camera and acquires infrared image data of the user. If the infrared sensor 13 is a LiDAR, the processing unit 11 acquires point cloud data of the user using the LiDAR. The processing unit 11 converts the point cloud data into two-dimensional infrared image data.

[0044] Next, the processing unit 11 identifies the individual through face authentication processing using the result of infrared detection (step S112). For example, the processing unit 11 extracts predetermined feature amounts from the infrared image data or point cloud data of the user, and identifies the individual user by comparing the extracted feature amounts with authentication information registered in the user DB 18.

[0045] Next, the processing unit 11 inputs the infrared image data into the learning model 19 to recognize each part of the user's body in the infrared image (step S113). Then, the processing unit 11 identifies a displacement part where the biological surface is periodically displaced due to the pulsation of the heart or blood vessels (step S114). The process of identifying the displacement part will be described in detail later. The processing unit 11 that executes the process of step S114 functions as a first identification unit that identifies a displacement part where the biological surface is displaced due to the heart or blood vessels.

[0046] Next, based on the identification result of step S114, the processing unit 11 sequentially aims and irradiates millimeter waves at each identified displacement site, receives reflected waves (step S115), and detects the pulsation of the heart or blood vessels at each displacement site (step S116). For example, the processing unit 11 detects the time change of the pulse, the pulse rate, the heart rate, the pulse rhythm, or the magnitude of the pulse fluctuation, the peak time of the pulse, etc. The time change of the pulse is, in other words, the time change of the amount of displacement of the body surface at the displacement site. The pulse rate is, for example, the number of times the artery pulses per minute. The heart rate is, for example, the number of times the heart beats per minute. The pulse rhythm is, for example, a value indicating the pulse period at the displacement site or the regularity of the pulse period. The magnitude of the pulse fluctuation is the amplitude of the displacement of the body surface at the displacement site. The peak time of the pulse is the time when the amount of displacement at the displacement site is maximum. It is also possible to detect the time when the amount of displacement is minimum. The processing unit 11 that executes the processes of steps S115 to S116 functions as a detection unit that detects the pulsation of the heart or blood vessels based on the displacement of the living body surface at the identified displacement site.

[0047] Next, the processing unit 11 determines whether the user is in a resting state based on the detected pulse period (step S117). For example, the processing unit 11 can determine whether the user is in a resting state by reading information related to the past pulses of an individual identified by face authentication from the user DB 18 and comparing the currently detected pulse period with the past pulse period. If the current pulse period is extremely shorter than the past pulse period, it can be determined that the user is not in a resting state.

[0048] If it is determined that the subject is not in a resting state (step S117: NO), the processing unit 11 returns the process to step S113. If it is determined that the subject is in a resting state (step S117: YES), the processing unit 11 determines whether pulsation at each displaced site has been detected for a predetermined time (step S118). The predetermined time is, for example, several times the average pulsation cycle of the heart and blood vessels. If it is determined that the pulsation has been detected for less than the predetermined time (step S118: NO), the processing unit 11 returns the process to step S113. If it is determined that pulsation has been detected for more than the predetermined time (step S118: YES), the processing unit 11 accesses the server 4 and acquires environmental information such as the current temperature and humidity (step S119).

[0049] Furthermore, the processing unit 11 receives information on the pulse detection result transmitted from the sensor device 9 (step S120).

[0050] Next, the processing unit 11 stores the pulse detection result indicating the pulse at each displacement site, the date and time when the pulse was detected, and the environmental information acquired in step S119 in association with the user's identification information in the user DB 18 (step S121). The pulse detection result includes the pulse detection result detected by the information processing device 1 and the pulse detection result detected by the sensor device 9.

[0051] Next, the processing unit 11 determines whether there is an abnormality in the heart or blood vessels based on the detection results of the pulsation at each of the identified multiple displacement sites (step S122). Specifically, the abnormality in the heart or blood vessels is determined by comparing the time change in the pulsation, the pulse rate, the heart rate, the pulse rhythm, the magnitude of the pulse fluctuation, the peak time of the pulsation, etc. at each of the identified multiple displacement sites. Details of the abnormality determination process will be described later. The processing unit 11 that executes the process of step S122 functions as a determination unit that determines whether there is an abnormality in the heart or blood vessels based on information related to the detected pulsation of the heart or blood vessels.

[0052] The processing unit 11 determines whether or not there is an abnormality in the heart or blood vessels (step S123). If it is determined that there is no abnormality (step S123: NO), the processing unit 11 determines whether or not it is a predetermined notification timing (step S124). If there is normal, it is considered unnecessary to display the determination result and notify relevant parties each time an abnormality is determined, and therefore the notification timing is determined in step S124. Note that display and notification may be performed each time a determination process is performed.

[0053] If it is determined that it is not the timing for notification (step S124: NO), the processing ends. If it is determined that it is the timing for notification (step S124: YES), the processing unit 11 generates a determination result display image 171 (see FIG. 12) indicating that the result is normal, and displays it on the display unit 17 (step S125). Then, the processing unit 11 notifies the family member's first communication terminal 2 and the medical professional's first communication terminal 2 of the determination result. No. 2communication terminal 3 (Step S126). The processing unit 11 may transmit the determination result together with information such as the user's name, identification information, and contact details to the first communication terminal 2 and the second communication terminal 3. The processing unit 11 may be configured not to transmit personal information such as the user's name to the second communication terminal 3 of the medical professional.

[0054] If it is determined in step S123 that there is an abnormality (step S123: YES), the processing unit 11 generates a determination result display image 171 (see FIGS. 13 and 14) indicating that there is an abnormality, and displays it on the display unit 17 (step S127). Then, the processing unit 11 transmits the determination result to the first communication terminal 2 of the family member and the second communication terminal 3 of the medical professional (step S128).

[0055] The medical personnel can receive the information on the determination result at the second communication terminal 3, and transmit to the information processing device 1 the finding information indicating the finding on the determination result.

[0056] The processing unit 11 of the information processing device 1 receives the finding information transmitted from the second communication terminal 3 of the medical professional via the communication unit 15 (step S129), and displays the received finding information of the medical professional on the display unit 17 (step S130). The processing unit 11 also transmits the finding information of the medical professional to the first communication terminal 2 of the family member (step S131), and then ends the processing.

[0057] 8 and 9 are flowcharts showing the procedure for identifying a displaced portion. The processing unit 11 determines whether or not the area of ​​the carotid triangle has been recognized by the processing of step S113 (step S151). If the area of ​​the carotid triangle has been recognized (step S151: YES), the processing unit 11 identifies the area as a displaced portion where the carotid artery and jugular vein are located (step S152).

[0058] When the processing of step S152 is completed or when it is determined in step S151 that the carotid triangle region has not been recognized (step S151: NO), the processing unit 11 determines whether the user's face has been recognized by the processing of step S113 (step S153). When it is determined that the user's face has been recognized (step S153: YES), the processing unit 11 detects any one of the facial contour, eyes, eyebrows, nose, mouth, nasolabial folds, ears, and chin, and identifies the displacement region where the carotid artery is located based on the amount of deviation of each part, such as the eyes, from the midline (step S154). The processing unit 11 detects the facial contour, eyes, eyebrows, nose, mouth, nasolabial folds, ears, or chin, for example, using the learning model 19. Alternatively, the processing unit 11 may be configured to extract the image portion of the face from the infrared image and detect the eyes, etc. of the face based on rules by binarization, pattern matching, or the like. Meanwhile, by performing the processing in step S113, the processing unit 11 can recognize left and right body parts, such as the right and left chest, right and left upper arms, etc., and can therefore identify the line separating the left and right body parts as the midline. From the positional relationship between this midline and each part, such as the left and right eyes, it is possible to estimate whether the face is facing right or left, the angle of neck rotation, etc. If the face is facing right, the left side of the neck recognized by the learning model 19 can be identified as the displacement part where the carotid artery is located. If the face is facing left, the right side of the neck can be identified as the displacement part where the carotid artery is located. Furthermore, the displacement part may be further narrowed down and identified from the part corresponding to the right or left carotid artery depending on the amount of neck rotation.

[0059] When the processing of step S154 is completed, or when it is determined in step S153 that the face has not been recognized (step S153: NO), the processing unit 11 determines whether or not the temporal region of the user has been recognized by the processing of step S113 (step S155). When it is determined that the temporal region has been recognized (step S155: YES), the processing unit 11 identifies the measurement region as a displacement region where the superficial temporal artery is located (step S156).

[0060] When the processing of step S156 is completed, or when it is determined in step S155 that the temporal region has not been recognized (step S155: NO), the processing unit 11 determines whether or not the upper arm region has been recognized by the processing of step S113 (step S157). When it is determined that the upper arm region has been recognized (step S157: YES), the processing unit 11 identifies the upper arm region as a displacement region where the brachial artery is located (step S158).

[0061] When the processing of step S158 is completed, or when it is determined in step S157 that the upper arm has not been recognized (step S157: NO), the processing unit 11 determines whether or not the palm has been recognized by the processing of step S113 (step S159). When it is determined that the palm has been recognized (step S159: YES), the processing unit 11 recognizes the position of the thumb from the image of the palm (step S160), and identifies the portion of the carpus near the thumb recognized in the processing of step S113 as the displaced region where the radial artery is located (step S161). Furthermore, the processing unit 11 identifies the portion of the carpus near the little finger recognized in the processing of step S113 as the displaced region where the ulnar artery is located (step S162).

[0062] When the palm of the hand is recognized, that is, when the palm is facing the infrared sensor 13, the learning model 19 recognizes the medial side of the carpus as the carpus. The medial side of the carpus is a part of the body where the surface of the body is displaced by the pulsation of the radial artery and the ulnar artery. When the dorsum of the hand is facing the infrared sensor 13, the learning model 19 recognizes the lateral side of the carpus as the carpus. Furthermore, if the learning model 19 is trained to distinguish between the thumb side of the medial carpus, the little finger side of the medial carpus, and the lateral carpus, the above-mentioned recognition process for the palm orientation and thumb position is unnecessary. The learning model 19 can directly recognize the displacement area where the radial artery and the displacement area where the ulnar artery are located.

[0063] When the processing of step S162 is completed, or when it is determined in step S159 that the palm has not been recognized (step S159: NO), the processing unit 11 determines whether or not the dorsum of the foot has been recognized by the processing of step S113 (step S163). When it is determined that the dorsum of the foot has been recognized (step S163: YES), the processing unit 11 identifies the dorsum of the foot as a displacement site where the dorsum of the foot artery is located (step S164).

[0064] When the processing of step S164 is completed, or when it is determined in step S163 that the dorsum of the foot has not been recognized (step S163: NO), the processing unit 11 determines whether or not the chest has been recognized by the processing of step S113 (step S165). When it is determined that the chest has not been recognized (step S165: NO), the processing unit 11 ends the processing of identifying the displaced part. When it is determined that the chest has been recognized (step S165: YES), the processing unit 11 identifies the chest as the displaced part where the heart is located (step S166), and ends the processing of identifying the displaced part.

[0065] According to the above processing, it is possible to identify displacement areas where the living body surface is displaced due to the propagation of vibration or vascular pulsation. Specifically, the processing unit 11 can identify displacement areas where the living body surface is displaced due to the carotid artery, temporal artery, brachial artery, radial artery, ulnar artery, dorsalis pedis artery, and heart pulsation. The processing unit 11 can also identify the area where the jugular vein is located.

[0066] 10 is a flowchart showing the processing procedure of the sensor device 9 according to the first embodiment. When the processing unit 91 of the sensor device 9 receives a detection instruction transmitted from the information processing device 1, the processing unit 91 executes the following processing. The processing unit 91 detects infrared rays of the user using the infrared sensor 93 (step S141). Specifically, if the infrared sensor 93 is an infrared camera, the processing unit 91 captures an image of the user using the infrared camera and acquires infrared image data of the user. If the infrared sensor 93 is a LiDAR, the processing unit 91 acquires point cloud data of the user using the LiDAR. The processing unit 91 converts the point cloud data into two-dimensional infrared image data.

[0067] Next, the processing unit 91 inputs the infrared image data into the learning model 99 to recognize each part of the user's body in the infrared image (step S142). Then, the processing unit 91 identifies displaced parts where the biological surface is periodically displaced due to the pulsation of the heart or blood vessels (step S143). Details of the process for identifying displaced parts are similar to the process for identifying displaced parts by the information processing device 1. Note that the processing unit 91 that executes the process of step S143 functions as a second identification unit that identifies displaced parts where the biological surface is displaced due to the heart or blood vessels.

[0068] Next, based on the identification result of step S143, the processing unit 91 sequentially aims and irradiates millimeter waves at each identified displacement site, receives reflected waves (step S144), and detects the pulsation of the heart or blood vessels at each displacement site (step S145). The processing unit 91 that executes the processes of steps S144 and S145 functions as a detection unit that detects the pulsation of the heart or blood vessels based on the displacement of the living body surface at the identified displacement site.

[0069] Next, the processing unit 91 determines whether or not the user is in a resting state based on the detected pulsation period (step S146).

[0070] rest If it is determined that the state is not the same (step S146: NO), the processing unit 91 returns the process to step S142. rest If it is determined that the state is correct (step S146: YES), the processing unit 91 determines whether pulsation at each displaced site has been detected for a predetermined time (step S147). The predetermined time is, for example, several times the average pulsation cycle of the heart and blood vessels. If it is determined that the period is shorter than the predetermined time (step S147: NO), the processing unit 91 returns the process to step S142. If it is determined that pulsation has been detected for a predetermined time or longer (step S147: YES), the processing unit 91 transmits information on the pulsation detection result to the information processing device 1 (step S148) and ends the process.

[0071] FIG. 11 is a flowchart showing the abnormality determination process. The processing unit 11 determines a blood vessel or cardiac abnormality based on the difference between the pulse rhythm deviation and the magnitude of the oscillation between the left and right carotid arteries (step S171). If the pulse rhythm deviation is equal to or greater than a predetermined threshold, the processing unit 11 determines a blood vessel or cardiac abnormality. In other words, if the time difference between the peak time of the pulsation at the first displacement site and the peak time of the pulsation at the second displacement site is equal to or greater than a predetermined threshold, the processing unit 11 determines a blood vessel or cardiac abnormality. Similarly, if the difference in the magnitude of the oscillation is equal to or greater than a predetermined threshold, the processing unit 11 determines a blood vessel or cardiac abnormality. If the difference in the pulse rhythm deviation and the magnitude of the pulse oscillation is large, abnormalities such as arteriosclerosis and vascular stenosis are suspected. For example, the processing unit 11 determines an ischemic or hemorrhagic cerebrovascular abnormality. That is, the processing unit 11 determines an abnormality related to stroke, cerebral infarction, or cerebral hemorrhage. The same applies below.

[0072] Next, the processing unit 11 determines a blood vessel or cardiac abnormality based on the difference in the amount of deviation or the magnitude of fluctuation between the pulse rhythms of the left temporal artery and the right temporal artery (step S172). For example, the processing unit 11 determines an ischemic or hemorrhagic cerebrovascular abnormality. That is, the processing unit 11 determines an abnormality related to stroke, cerebral infarction, or cerebral hemorrhage.

[0073] The processing unit 11 determines whether there is an abnormality in the blood vessels or the heart based on the difference in the amount of deviation or the magnitude of the fluctuation between the pulse rhythms of the left brachial artery and the right brachial artery (step S173).

[0074] The processing unit 11 determines whether there is an abnormality in the blood vessels or the heart based on the difference in the amount of deviation or the magnitude of the fluctuation between the pulse rhythms of the left dorsalis pedis artery and the right dorsalis pedis artery (step S174). For example, the processing unit 11 determines whether there is an abnormality in the blood vessels of the legs.

[0075] Next, the processing unit 11 determines a blood vessel or cardiac abnormality based on the difference between the pulsation propagation velocity between the heart and the left carotid artery and the pulsation propagation velocity between the heart and the right carotid artery (step S175). If the difference between the propagation velocities is equal to or greater than a predetermined threshold, some cardiac or vascular abnormality is suspected. For example, the processing unit 11 determines an ischemic or hemorrhagic cerebrovascular abnormality. That is, the processing unit 11 determines an abnormality related to stroke, cerebral infarction, or cerebral hemorrhage.

[0076] Next, the processing unit 11 determines whether there is an abnormality related to critical limb ischemia based on the magnitude of the fluctuation in the pulse of the dorsalis pedis artery (step S176). If the fluctuation in the pulse of the dorsalis pedis artery is less than a predetermined threshold, the processing unit 11 determines that there is an abnormality related to critical limb ischemia.

[0077] Next, the processing unit 11 performs a process to detect jugular vein distension by analyzing infrared image data in the area of ​​the carotid triangle or by analyzing point cloud data in the area of ​​the carotid triangle, and determines abnormalities related to heart failure based on the presence or absence of jugular vein distension (step S177).

[0078] In the above description, an example has been described in which cardiac or vascular abnormalities are determined mainly based on the difference in the amount of deviation or magnitude of oscillation between the pulse rhythms of the left and right arteries. However, cardiac or vascular abnormalities may also be determined based on the time change of any two pulses of the carotid artery, superficial temporal artery, brachial artery, radial artery, ulnar artery, or dorsalis pedis artery, or the difference in the amount of deviation or magnitude of oscillation between the pulse peaks, pulse rate, heart rate, pulse rhythm, or the like. Regarding the presence or absence of jugular vein distension, an abnormality related to heart failure may also be determined based on the size and shape of the bulging jugular vein. Furthermore, the presence or absence of jugular vein distension and an abnormality related to heart failure based thereon may also be determined by comparing the magnitude of oscillation of the pulse or heart rate of the arteries.

[0079] In the above description, abnormality determination is performed based on the current pulsation of the heart and blood vessels. However, the abnormality of the heart or blood vessels may be determined by comparing the information on the user's past pulsation stored in the user DB 18 with the information on the current pulsation. For example, the processing unit 11 may determine an increase or decrease in the user's heart rate as an abnormality of the heart or blood vessels. Furthermore, it is preferable to compare information on past pulsations detected in an environment similar to the current environment with information on the current pulsation. This allows for more accurate determination of abnormalities of the heart or blood vessels.

[0080] 12, 13, and 14 are schematic diagrams showing an example of the determination result display image 171. The processing unit 11 generates the determination result display image 171 as shown in FIGS. 12 to 14 by the processes of step S125 and step S127. The determination result display image 171 displays, for example, a human body image 172 depicting a human body and various arteries and the heart that are the detection targets. The human body image 172 includes text images indicating the names of the various arteries: "(1) Temporal Artery," "(2) Carotid Artery," "(3) Brachial Artery," "(4) Radial Artery," "(5) Ulnar Artery," and "(6) Dorsalis Pedis Artery." The processing unit 11 may display the character images corresponding to the arteries in which the pulsation was detected in different ways from the character images corresponding to the arteries in which the pulsation was not detected in. For example, the processing unit 11 may highlight the character images corresponding to the arteries in which the pulsation was detected and display the character images corresponding to the arteries in which the pulsation was not detected in light text.

[0081] The determination result display image 171 includes graphs 173a and 173b that show changes over time in the pulsations of a plurality of arteries. In the examples shown in Figures 12 to 14, two graphs 173a and 173b that show changes over time in the pulsations of the carotid artery and the radial artery are displayed. The graphs 173a and 173b may display the state of pulsation in real time, or may statically display the state of pulsation for a certain period of time. The multiple graphs 173a, 173b showing the pulsation of each artery may be displayed at predetermined screen positions corresponding to each artery and the heart, or only the graphs 173a, 173b showing the pulsation of the detected artery may be displayed. The processing unit 11 may accept the selection of the graph to be displayed via the operation unit 16, and display the graphs 173a, 173b showing the pulsation of the selected artery. It is also preferable to display the peak of the pulsation as the zero point.

[0082] If the determination result is normal, the processing unit 11 may display graphs 173a and 173b showing the pulsation states of one or more representative arteries, as shown in Fig. 12. For example, graphs 173a and 173b showing the pulsation states of the carotid artery and the radial artery may be displayed. Furthermore, the processing unit 11 may display the text image corresponding to the artery being displayed in a different manner from other text images. For example, the text image may be highlighted.

[0083] If the determination result is abnormal, the processing unit 11 displays graphs 173a and 173b showing the time change in pulsation of the two arteries that formed the basis for the abnormal determination on the determination result display image 171, as shown in Figures 13 and 14. Furthermore, the processing unit 11 may highlight the text images indicating the arteries that formed the basis for the abnormal determination in a manner that differs from the normal state. For example, the processing unit 11 may highlight the text images indicating the detected arteries in green when the normal determination is made, and highlight the arteries that formed the basis for the abnormal determination in red.

[0084] Furthermore, as shown in FIGS. 12 and 13, the determination result display image 171 includes a message image 174 indicating whether the determination result is normal or not.

[0085] Furthermore, when finding information from a medical professional is received, the determination result display image 171 includes a finding message image 175 showing the finding information, as shown in FIG.

[0086] 12 to 14, the user can know the pulsation state of the heart and blood vessels and whether the heart or blood vessels are normal. Needless to say, the determination result display image 171 may be transmitted as the determination result to the first communication terminal 2 and the second communication terminal 3.

[0087] The above-described determination result display image 171 is merely an example, and other information may be displayed. For example, a graph showing the time change in the heart or blood vessel pulsation from one day ago, one week ago, or one year ago may be displayed next to or superimposed on the current graph for comparison. Information such as heart rate and pulse rate may also be displayed.

[0088] As described above, the information processing system according to the first embodiment can detect the pulsation of the user's heart or blood vessels and determine cardiac or vascular abnormalities. Specifically, the processing unit 11 can determine cardiac or vascular abnormalities such as arteriosclerosis and stenosis by comparing the pulse rate, heartbeat, pulse rhythm, amplitude of oscillation, and the like at multiple displacement sites. More specifically, cardiac or vascular abnormalities can be determined based on the time difference between the peak pulsation points and the amplitude difference between the carotid artery and the radial artery, or between paired left and right arteries. Furthermore, ischemic and hemorrhagic cerebrovascular abnormalities can be determined based on the amplitude of oscillation of the temporal artery, carotid artery, etc. In other words, abnormalities related to stroke, cerebral infarction, and cerebral hemorrhage can be determined. Furthermore, the presence or absence of critical limb ischemia can be determined based on the amplitude of oscillation of the dorsalis pedis artery. Furthermore, abnormalities related to heart failure can be determined by detecting jugular vein distension.

[0089] Furthermore, since the infrared sensor 13 is used to identify the displaced area and then irradiates the displaced area with millimeter waves, it is possible to detect the pulsation of blood vessels and the heart at the displaced area more accurately and efficiently. The infrared sensor 13 can identify the displaced area more accurately than millimeter waves. On the other hand, the millimeter wave sensor 14 can detect the pulsation at the displaced area that is hidden by clothing and cannot be detected by the infrared sensor 13. In this way, by compensating for the advantages and disadvantages of the infrared sensor 13 and the millimeter wave sensor 14, it is possible to accurately and efficiently detect the pulsation of various arteries and the heart, and to determine abnormalities in the blood vessels and the heart.

[0090] Furthermore, the information processing system recognizes each part of the user's body from two different directions, and Millimeter waves By irradiating the body with light, the pulsation of the heart and blood vessels can be detected, and abnormalities in the heart or blood vessels can be detected with higher accuracy.

[0091] Furthermore, the information processing device 1 is configured to detect pulsation by irradiating millimeter waves at the displacement site identified by the infrared sensor 13, and therefore can detect pulsation of the heart or each artery efficiently and accurately.

[0092] Furthermore, according to the first embodiment, the determination result can be notified to family members or medical professionals. Furthermore, when there is an abnormality in the heart or blood vessels and there is information on the findings of a medical professional, the information processing device 1 can display the information on the findings of the medical professional on the display unit 17. Furthermore, the information on the findings of the medical professional can be transmitted to the first communication terminal 2 of the family member. Therefore, after detecting an abnormality in the heart or blood vessels early, it is possible to notify the user and family members of highly reliable information from the medical professional.

[0093] In the first embodiment, an example has been described in which the infrared sensors 13, 93 are used to recognize each part of the user's body and the displaced part, but an imaging device that captures an image of the user using visible light may be provided instead of the infrared sensors 13, 93. The processing unit 11 can similarly recognize each part of the human body and identify the displaced part based on image data of the user captured using visible light.

[0094] Although the millimeter wave sensors 14 and 94 are used to detect pulsation at a displaced part of the human body, a sensor that transmits and receives electromagnetic waves in the terahertz band may be provided. By irradiating the displaced part with electromagnetic waves in the terahertz band and receiving reflected waves from the displaced part, pulsation at the displaced part can be detected in the same manner as the millimeter wave sensors 14 and 94.

[0095] Furthermore, the information processing device 1 has been described as including the infrared sensors 13, 93 and the millimeter wave sensors 14, 94, but the infrared sensors 13, 93 or the millimeter wave sensors 14, 94 may be devices that are externally connected by wire or wirelessly.

[0096] Furthermore, in the first embodiment, an example has been described in which the infrared sensors 13, 93 and the millimeter wave sensors 14, 94 are used to detect the displacement site and pulse of the user's body from the horizontal and vertical directions, but the detection directions are just an example, and the configuration may be such that the displacement site and pulse of the user are detected from an oblique direction relative to the horizontal plane.

[0097] Furthermore, in this embodiment 1, an example has been described in which the infrared sensors 13, 93 and the millimeter wave sensors 14, 94 are used to detect the displacement site and pulsation in the user's body from two directions, but the detection directions are not limited to two directions, and the configuration may be such that the displacement site is identified and the pulsation is detected from three or more directions.

[0098] Furthermore, in the first embodiment, an example has been described in which a home computer executes the computer programs P1 and P2 according to the first embodiment, but a cloud computer may be configured to execute the computer programs P1 and P2 according to the first embodiment and implement the information processing method. Needless to say, the computer programs P1 and P2 may be distributed and executed on multiple server computers.

[0099] Furthermore, in this embodiment 1, the information processing device 1 and the sensor device 9 share the processing load, but the sensor device 9 may execute all or part of the processing based on the computer program P1, and the information processing device 1 may execute all or part of the processing based on the computer program P2. In the first embodiment, the information processing device 1 includes a first identification unit, a detection unit, and a determination unit, and the sensor device 9 includes a second identification unit and a detection unit. However, the arrangement of each unit is an example. The first identification unit, the second identification unit, the detection unit, and the determination unit may be provided in either or both of the information processing device 1 and the sensor device 9, as appropriate.

[0100] (Embodiment 2) The information processing system according to the second embodiment differs from the information processing system according to the first embodiment in terms of the information processing procedure. Since the other configurations of the information processing system are the same as those of the information processing system according to the first embodiment, the same reference numerals are used for the same parts and detailed description thereof will be omitted.

[0101] 15 is a flowchart showing an information processing procedure according to the second embodiment. The information processing device 1 executes the following processing at any timing. As in the first embodiment, the processing unit 11 transmits an instruction signal to the sensor device 9 to instruct the processing of identifying the user's displaced part and detecting the pulse (step S210), detects the user's infrared rays using the infrared sensor 13 (step S211), identifies the individual through face recognition processing using the infrared detection result (step S212), and recognizes each part of the user's body in the infrared image by inputting the infrared image data into the learning model 19 (step S213). That is, the processing unit 11 recognizes each part of the user's body as viewed from a substantially horizontal direction. In other words, the processing unit 11 performs image recognition of each part of the user in the infrared image obtained by detection from a substantially horizontal direction using the infrared sensor 13.

[0102] Meanwhile, the processing unit 91 of the sensor device 9 receives the instruction signal transmitted from the information processing device 1 (step S214). Having received the instruction signal, the processing unit 91 detects the user's infrared rays using the infrared sensor 93 (step S215), and inputs the infrared image data into the learning model 99 to recognize each part of the user's body in the infrared image (step S216). That is, the processing unit 11 recognizes each part of the user's body as seen from above. In other words, the processing unit 11 uses the infrared sensor 93 Each part of the user's body is recognized in an infrared image obtained by detecting the user from approximately vertically above using the infrared camera. Then, the processing unit 91 transmits information on the recognition result obtained by the process of step S216 to the information processing device 1 via the communication unit 95 (step S217).

[0103] The processing unit 11 of the information processing device 1 receives the information on the recognition result transmitted from the sensor device 9 via the communication unit 15 (step S218). Then, based on the recognition result of each part of the user by the information processing device 1 and the recognition result of each part of the user by the sensor device 9, the processing unit 11 identifies a displacement part where the surface of the living body is periodically displaced due to the pulsation of the heart or blood vessels (step S219).

[0104] The processing unit 11 may identify one displaced portion using only the recognition result from the information processing device 1, or may identify one displaced portion using only the recognition result from the sensor device 9. The method for identifying the displaced portion is the same as in the first embodiment.

[0105] For example, if the information processing device 1 cannot recognize the dorsum of the foot but the sensor device 9 can, the processing unit 11 identifies the dorsum of the foot recognized by the sensor device 9 as the displacement site where the dorsal artery is located. In this case, the processing unit 11 temporarily stores information indicating that the sensor device 9 is capable of detecting the pulsation of the dorsal artery. Conversely, if the information processing device 1 can recognize the temporal region but the sensor device 9 cannot recognize the temporal region, the processing unit 11 identifies the temporal region recognized by the information processing device 1 as the displaced region where the superficial temporal artery is located. In this case, the processing unit 11 temporarily stores information indicating that it is the information processing device 1 that can detect the pulsation of the superficial temporal artery. In this way, when either the information processing device 1 or the sensor device 9 can recognize a region corresponding to a displacement region where an artery whose pulsation is to be detected is located, the processing unit 11 can identify the region recognized by either the information processing device 1 or the sensor device 9 as the displacement region. The processing unit 11 temporarily stores information indicating the information processing device 1 or the sensor device 9 that can recognize the region as a device that can detect the pulsation of the region.

[0106] For a portion that can be recognized by both the information processing device 1 and the sensor device 9, the displaced portion may be identified based on the recognition result of the information processing device 1. Alternatively, the size of the portion that can be recognized by the information processing device 1 may be compared with the size of the corresponding portion that can be recognized by the sensor device 9, and the displaced portion may be identified using the recognition result of the device that can recognize the larger portion.

[0107] Furthermore, the information processing device 1 may identify a displaced portion using both the recognition results of the information processing device 1 and the sensor device 9. For example, a displaced portion where the carotid artery is located can be recognized using both recognition results. In step S154 of the first embodiment, the amount of twisting of the neck is estimated based on the amount of deviation of each part of the face from the midline, and the displaced portion where the carotid artery is located is identified. However, in the second embodiment, the amount of twisting of the neck can be directly estimated based on the recognition result of the user's head or ears and shoulders or chest recognized by the sensor device 9, and the displaced portion in the neck recognized by the information processing device 1 can be identified based on the amount of twisting. Here, a method for recognizing a displacement site where a carotid artery is present has been exemplified, but the displacement site may be identified by appropriately combining the recognition results of the information processing device 1 and the sensor device 9.

[0108] After completing the process of step S219, the processing unit 11 transmits information indicating the displacement site where the sensor device 9 should detect the pulsation to the sensor device 9 via the communication unit 15 (step S220).

[0109] Then, based on the identification result of step S219, the processing unit 11 sequentially aims and irradiates millimeter waves at each displacement site identified on the information processing device 1 side as a site where pulsation should be detected, receives reflected waves (step S221), and detects the pulsation of the heart or blood vessels at each displacement site (step S222).

[0110] Meanwhile, the sensor device 9 receives information indicating the displacement site via the communication unit 95 (step S223). Having received the information, the sensor device 9 sequentially aims and irradiates millimeter waves at each displacement site identified by the sensor device 9 as a site where pulsation should be detected, based on the information, and receives reflected waves (step S224), thereby detecting the pulsation of the heart or blood vessels at each displacement site (step S225). Then, the processing unit 91 transmits information on the pulsation detection result, indicating the pulsation at each detected displacement site, to the information processing device 1 via the communication unit 95 (step S226).

[0111] The information processing device 1 receives the information on the pulse detection result transmitted from the sensor device 9 via the communication unit 15 (step S227). Thereafter, by executing the same processes as steps S121 to S131 described in the first embodiment, it is possible to determine and notify abnormalities in the user's heart and blood vessels.

[0112] According to the information processing system etc. of embodiment 2, by using both the recognition results of each part of the user's body by the information processing device 1 and the recognition results of each part of the user's body by the sensor device 9, it is possible to identify displaced parts to be irradiated with millimeter waves, detect the pulsation at each displaced part, and determine abnormalities in the user's heart or blood vessels.

[0113] According to the second embodiment, the information processing device 1 is configured to identify the displacement region where the pulsation should be detected and the sensor device 9 is configured to identify the displacement region where the pulsation should be detected. strange rank rank That is, the information processing device 1 and the sensor device 9 can detect the displacement of the part to be detected by the device itself. rank This configuration allows for efficient detection of the pulsation of each displacement site. strange rank rank The heartbeat can be detected.

[0114] (Embodiment 3) The information processing system according to the third embodiment differs from the information processing system according to the first embodiment in terms of the information processing procedure. Since the other configurations of the information processing system are the same as those of the information processing system according to the first embodiment, the same reference numerals are used for the same parts, and detailed description thereof will be omitted.

[0115] The infrared sensors 13 and 93 according to the third embodiment are devices capable of three-dimensionally detecting the user's body surface and clothing surface, such as an infrared laser such as LiDAR. The information processing device 1 recognizes each part of the user's body three-dimensionally using three-dimensional information, that is, point cloud data or voxel data.

[0116] 16 is a flowchart showing an information processing procedure according to embodiment 3. The information processing device 1 executes the following processes at any timing. As in embodiments 1 and 2, the processing unit 11 transmits an instruction signal to the sensor device 9 to instruct the processing of identifying the user's displaced part and detecting the pulse (step S310), detects the user's infrared rays using the infrared sensor 13 (step S311), and identifies the individual by face recognition processing using the result of the infrared detection (step S312).

[0117] Meanwhile, the processing unit 91 of the sensor device 9 receives the instruction signal transmitted from the information processing device 1 (step S313). Having received the instruction signal, the processing unit 91 detects the user by infrared rays using the infrared sensor 93 (step S314), and transmits infrared signal data output from the infrared sensor 93 to the information processing device 1 via the communication unit 95 (step S315). The infrared signal data is, for example, point cloud data that is three-dimensional information about the user.

[0118] The processing unit 11 of the information processing device 1 receives the infrared signal data transmitted from the sensor device 9 at the communication unit 15 (step S316). Then, the processing unit 11 integrates the infrared signal data output from the infrared sensor 13 and the infrared signal data received in step S316 (step S317). The point cloud data obtained by the infrared sensor 13 represents the positions of a large number of points on the surface of the user's body or clothing in three-dimensional coordinates in a coordinate system based on the position of the infrared sensor 13. The point cloud data obtained by the infrared sensor 93 represents the positions of a large number of points on the surface of the user's body or clothing in three-dimensional coordinates in a coordinate system based on the position of the infrared sensor 93. The point cloud data obtained by the infrared sensor 13 and the point cloud data obtained by the infrared sensor 93 are data represented in different coordinate systems, but by performing coordinate transformation so that the multiple points detected by both infrared sensors 13 and 93 match, the point cloud data can be integrated and more detailed three-dimensional information about the surface of the user's body or clothing can be obtained.

[0119] Then, the processing unit 11 recognizes each part of the user's body using the point cloud data obtained in step S317 (step S318). In step S318, the processing unit 11 may recognize each part of the user's body in two dimensions or three dimensions. The processing unit 11 can recognize each part of the user's body by, for example, converting the point cloud data into three-dimensional voxel data or a two-dimensional infrared image and inputting the converted voxel data or infrared image into the learning model 19. The two-dimensional infrared image is an image obtained when the user is detected from a substantially horizontal direction using the infrared sensor 13, and an image obtained when the user is detected from a substantially vertical direction using the infrared sensor 93.

[0120] Next, the processing unit 11 identifies a displacement part where the surface of the living body is periodically displaced due to the pulsation of the heart or blood vessels, based on the recognition result of each part of the user by the information processing device 1 (step S319).

[0121] Thereafter, the processing unit 11 executes the same processes as steps S220 to S227 in embodiment 2 and steps S121 to S131 described in embodiment 1 to determine and notify abnormalities in the user's heart and blood vessels.

[0122] According to the information processing system etc. of embodiment 3, the infrared signal data output from the infrared sensor 13 and the infrared sensor 93 are integrated to recognize each part of the user's body, identify the displaced parts, and detect the pulsation at each displaced part, so that abnormalities in the user's heart or blood vessels can be determined more accurately.

[0123] Alternatively, the pulsation of the body surface at the displaced part may be detected three-dimensionally using information on the displaced part detected and recognized three-dimensionally by the infrared sensors 13 and 93, the detection results by the millimeter-wave sensors 14 and 94. This makes it possible to detect the pulsation of the heart or blood vessels at the displaced part with greater accuracy.

[0124] (Embodiment 4) The information processing device 1 according to the fourth embodiment differs from the first to third embodiments in that it detects the user's body movements and heartbeat using an acceleration sensor 5 and a contact sensor 6. The other configurations of the information processing device 1 are the same as those of the information processing device 1 according to the first to third embodiments, and therefore the same reference numerals are used for the same parts and detailed description will be omitted.

[0125] 17 is an explanatory diagram showing an example of the configuration of an information processing system according to a fourth embodiment. The information processing system according to the fourth embodiment includes an information processing device 1 similar to those of the first to third embodiments, and further includes an acceleration sensor 5 attached to the user, and a contact sensor 6 that detects the pulsation of the heart or blood vessels. The acceleration sensor 5 transmits acceleration signal data indicating the acceleration corresponding to the movement of the user's body to the information processing device 1. The contact sensor 6 is attached to a part of the body where the surface of the living body is displaced by the pulsation of the heart or blood vessels, and transmits pulsation signal data indicating the pulsation to the information processing device 1. It is preferable that the contact sensor 6 be attached to a part that is unlikely to be irradiated with millimeter waves from the information processing device 1.

[0126] 18 is a flowchart showing an information processing procedure according to the fourth embodiment. The processing unit 11 of the information processing device 1 determines whether it is a predetermined monitoring timing (step S441). The predetermined monitoring timing is an arbitrary timing and can be set appropriately by the user. If it is determined that it is not a monitoring timing (step S441: NO), the processing unit 11 returns the process to step S441 and waits.

[0127] If it is determined that it is time to monitor (step S441: YES), the processing unit 11 receives acceleration signal data transmitted from the acceleration sensor 5 (step S442) and receives pulsation signal data transmitted from the contact sensor 6 (step S443).

[0128] Next, the processing unit 11 determines whether the magnitude of the body movement is less than a predetermined value based on the acceleration signal data, thereby determining whether the user is in a resting state (step S444). If it is determined that the magnitude of the body movement is equal to or greater than the predetermined value and the user is not in a resting state (step S444: NO), the processing unit 11 returns the process to step S442.

[0129] If it is determined that the magnitude of the body movement is less than a predetermined value and that the patient is in a resting state (step S444: YES), the processing unit 11 identifies the displaced area using the same processing procedures as in the first to third embodiments, detects the pulsations of the heart and blood vessels, and executes the process of determining abnormalities in the heart and blood vessels. However, the processing unit 11 according to the fourth embodiment determines abnormalities in the heart or blood vessels using the pulsations detected by the millimeter wave sensor 14, which is a non-contact sensor, and the pulsations indicated by the pulsation signal data transmitted from the contact sensor 6.

[0130] As described above, according to the information processing system etc. of the fourth embodiment, by attaching the contact sensor 6 to a location where it is difficult for the millimeter wave sensor 14 to irradiate, it is possible to detect abnormalities in the heart or blood vessels based on the pulsation of arteries in more locations. For example, by attaching the contact sensor 6 under the armpit, it is possible to detect the pulsation of the axillary artery. Furthermore, by attaching the contact sensor 6 to the groin, the back of the knee, or behind the inner ankle, it is possible to detect the pulsation of the femoral artery, popliteal artery, posterior tibial artery, etc., and it is possible to more accurately determine abnormalities in the heart or blood vessels.

[0131] Furthermore, since the acceleration sensor 5 is used to confirm the user's resting state, the displacement site is identified and the pulsation is detected, so that the abnormality determination process can be performed efficiently. [Explanation of symbols]

[0132] 1. Information processing equipment 2. First communication terminal 3 Second communication terminal 4 Server 5. Accelerometer 6 Contact Sensors 9 Sensor Device 10,90 Recording media 11,91 Processing section 12,92 Storage part 13,93 Infrared sensor 14,94 mmWave sensor 15,95 Communications Department 16 Control section 17 Display 18 User DB 19,99 Learning Model 19a Input layer 19b Middle class 19c Output layer 171 Judgment result display image P1, P2 computer programs Room R L Lighting device

Claims

1. A computer program for causing a computer to determine abnormalities in the heart or blood vessels of a living body, a displacement site where the surface of the living body is displaced by the heart or blood vessels, the displacement site being detectable from a first direction that is a direction capturing a front side site of the living body, is identified by infrared light or visible light from a first infrared sensor or a visible light sensor; a displacement site where the surface of the living body is displaced by the heart or blood vessels, the displacement site being detectable from a second direction that is a direction capturing the site viewed from above the living body, is identified by infrared light or visible light from a second infrared sensor or visible light sensor; irradiating the displacement site identified by both the first direction and the second direction with millimeter waves or terahertz band electromagnetic waves from a millimeter wave sensor or a terahertz band sensor, and detecting heart or blood vessel pulsation based on reflected waves from the displacement site of the living body; Determining abnormalities in the heart or blood vessels based on information related to the detected heart or blood vessel pulsations A computer program for causing the computer to execute a process.

2. Identifying a plurality of the displacement regions to be identified by both the first direction and the second direction; The pulse rate, pulse rhythm, or magnitude of pulse fluctuation at each of the identified multiple displacement sites is compared to determine abnormalities in the heart or blood vessels. The computer program product according to claim 1, for causing the computer to execute a process.

3. identifying a plurality of said mutation sites; If the time difference between the peak of the pulsation at the first displacement site and the peak of the pulsation at the second displacement site is equal to or greater than a threshold, it is determined that there is an abnormality in the heart or blood vessels.

3. A computer program according to claim 1 or 2, for causing a computer to execute a process.

4. Identifying the displacement site based on signal data output from the first infrared sensor or the visible light sensor; The displacement portion is identified based on signal data output from the second infrared sensor or the visible light sensor. The computer program according to any one of claims 1 to 3, for causing the computer to execute processing.

5. identifying the displacement site based on signal data output from a first infrared sensor or a visible light sensor facing the first direction with respect to the living body; The displacement site is identified based on signal data output from a second infrared sensor or a visible light sensor facing the second direction with respect to the living body.

5. A computer program according to claim 1, for causing a computer to execute a process.

6. The displacement site is identified based on signal data output from a first infrared sensor or visible light sensor facing the first direction with respect to the living body and signal data output from a second infrared sensor or visible light sensor facing the second direction with respect to the living body.

6. A computer program according to claim 1, for causing a computer to execute a process.

7. irradiating the living body with millimeter waves or terahertz band electromagnetic waves in the first direction, and detecting heart or blood vessel pulsation based on signal data output from a first millimeter wave sensor or terahertz band sensor that receives reflected waves from the displaced part of the living body; A second millimeter wave sensor or a terahertz band sensor irradiates the living body with millimeter waves or terahertz band electromagnetic waves in the second direction, and receives reflected waves from the displaced part of the living body, and based on signal data output from the second millimeter wave sensor or the terahertz band sensor, detects the pulsation of the heart or blood vessels.

7. A computer program product according to claim 1, for causing a computer to execute a process.

8. Millimeter wave or terahertz wave electromagnetic waves are irradiated onto the identified displacement site, and the pulsation of the heart or blood vessels is detected based on the reflected waves from the displacement site. The computer program according to any one of claims 1 to 7, for causing a computer to execute a process.

9. The pulsation of the heart or blood vessels at the displacement site is detected based on the signal data output from the millimeter wave sensor or the terahertz band sensor, and the pulsation of the heart or blood vessels at at least one of the displacement sites at which the surface of the living body is displaced by the heart or blood vessels, excluding the displacement site detected based on the signal data output from the millimeter wave sensor or the terahertz band sensor, is detected based on the signal data output from a contact sensor attached to the living body.

9. A computer program product according to claim 1, for causing a computer to execute a process.

10. determining whether or not the magnitude of the body movement is less than a predetermined value based on signal data output from an acceleration sensor attached to the living body; If the magnitude of the body movement is less than a predetermined value, the pulsation of the heart or blood vessels is detected based on the displacement of the living body surface at the identified displacement site.

10. A computer program product according to claim 1, for causing a computer to execute a process.

11. The displacement site is the neck, the temporal region, the upper limb, the dorsum of the foot, or the chest, and the pulse of the carotid artery in the neck, the superficial temporal artery in the temporal region, the brachial artery, the radial artery, or the ulnar artery in the upper limb, the dorsum of the foot, or the heart in the chest is detected. A computer program according to any one of claims 1 to 10, for causing a computer to execute a process.

12. Identifying the area of ​​the carotid trigone as the displacement site; Detecting carotid pulse 12. A computer program product according to claim 11, for causing a computer to execute a process.

13. Detect one of the facial contours, eyes, eyebrows, nose, mouth, nasolabial folds, ears, and chin, Identifying the displacement site where the carotid artery is located based on the amount of displacement of the detected site from the midline; Detecting carotid pulse 13. A computer program according to claim 11 or 12, for causing a computer to execute a process.

14. The upper arm is identified as the displacement site, and the pulsation of the brachial artery is detected.

14. A computer program product according to any one of claims 11 to 13, for causing a computer to execute a process.

15. The inner part of the carpus near the thumb is identified as the displacement site, and the pulsation of the radial artery is detected.

15. A computer program product according to any one of claims 11 to 14, for causing a computer to execute a process.

16. The part of the inner part of the wrist near the little finger is identified as the displacement site, and the pulsation of the ulnar artery is detected.

16. A computer program product according to any one of claims 11 to 15, for causing a computer to execute a process.

17. Determine cardiac or vascular abnormalities based on any two of the pulses or heart rates of the carotid artery, superficial temporal artery, brachial artery, radial artery, ulnar artery, and dorsalis pedis artery.

17. A computer program product according to any one of claims 11 to 16, for causing a computer to execute a process.

18. Determine cardiac or vascular abnormalities based on the pulse and heart rate of the carotid artery, superficial temporal artery, brachial artery, radial artery, ulnar artery, or dorsalis pedis artery A computer program product according to any one of claims 11 to 17, for causing a computer to execute a process.

19. Determine cardiac or vascular abnormalities based on deviations in the pulse peaks of the carotid artery, superficial temporal artery, brachial artery, radial artery, ulnar artery, or dorsalis pedis artery 19. A computer program product according to any one of claims 11 to 18, for causing a computer to execute a process.

20. Diagnose abnormalities related to critical limb ischemia based on the magnitude of pulse fluctuations in the dorsalis pedis artery 20. A computer program product according to any one of claims 11 to 19, for causing a computer to execute a process.

21. Determine abnormalities in the heart or blood vessels based on the pulse of the left carotid artery and the pulse of the right carotid artery 21. A computer program product according to any one of claims 11 to 20, for causing a computer to execute a process.

22. Determine cardiac or vascular abnormalities based on the pulse of the left temporal artery and the pulse of the right temporal artery 22. A computer program product according to any one of claims 11 to 21, for causing a computer to carry out a process.

23. Determine cardiac or vascular abnormalities based on the pulse of the left brachial artery and the pulse of the right brachial artery 23. A computer program product according to any one of claims 11 to 22, for causing a computer to carry out a process.

24. Diagnosing cardiac or vascular abnormalities based on the pulse of the left dorsalis pedis artery and the pulse of the right dorsalis pedis artery 24. A computer program product according to any one of claims 11 to 23, for causing a computer to carry out a process.

25. Identifying the area of ​​the carotid trigone as the displacement site; Detect jugular vein distention, Determine abnormalities related to heart failure A computer program product according to any one of claims 1 to 24, for causing a computer to carry out a process.

26. An information processing method for determining abnormalities in the heart or blood vessels of a living body by a computer, comprising: the computer identifies a displacement site where the surface of the living body is displaced by the heart or blood vessels, the displacement site being detectable from a first direction that is a direction capturing a front side site of the living body, using infrared light or visible light from a first infrared sensor or a visible light sensor; the computer identifies a displacement site where the surface of the living body is displaced by the heart or blood vessels, and which can be detected from a second direction that is a direction for capturing the site viewed from above the living body, using infrared light or visible light from a second infrared sensor or visible light sensor; the computer irradiates the identified displacement site with millimeter waves or terahertz band electromagnetic waves from a millimeter wave sensor or a terahertz band sensor, and detects heart or blood vessel pulsation based on the reflected waves from the displacement site of the living body; The computer determines abnormalities in the heart or blood vessels based on information related to the detected heart or blood vessel pulsations. Information processing methods.

27. An information processing device for determining abnormalities in the heart or blood vessels of a living body, a first identifying unit that identifies a displacement site where a surface of a living body is displaced by a heart or a blood vessel, the displacement site being detectable from a first direction that is a direction capturing a front side site of the living body, using infrared light or visible light from a first infrared sensor or a visible light sensor; a second identifying unit that identifies a displacement site, which is a displacement site on the surface of the living body caused by the heart or blood vessels and can be detected from a second direction, which is a direction in which the displacement site is captured as viewed from above the living body, using infrared light or visible light from a second infrared sensor or visible light sensor; a detection unit that irradiates the displacement site identified by the first identification unit and the second identification unit with millimeter waves or terahertz band electromagnetic waves from a millimeter wave sensor or a terahertz band sensor, and detects heart or blood vessel pulsation based on reflected waves from the displacement site of the living body; a determination unit that determines abnormalities in the heart or blood vessels based on information related to the detected pulsation of the heart or blood vessels; An information processing device comprising:

28. An information processing system including an information processing device and a sensor device for determining abnormalities in the heart or blood vessels of a living body, The information processing device includes: a first infrared sensor or visible light sensor for identifying a displacement site of a living body surface displaced by a heart or a blood vessel from a first direction that captures a front side site of the living body; a first millimeter wave sensor or a terahertz band sensor that irradiates the living body with millimeter waves or terahertz band electromagnetic waves in a first direction and receives reflected waves from the displaced part of the living body; Equipped with The sensor device includes: a second infrared sensor or visible light sensor for identifying a displacement site of the living body surface displaced by the heart or blood vessels from a second direction that captures the site as seen from above the living body; a second millimeter wave sensor or a terahertz band sensor that irradiates the living body with millimeter waves or terahertz band electromagnetic waves in a second direction and receives reflected waves from the displaced part of the living body; Equipped with an identifying unit that identifies the displacement portion based on signal data output from the first infrared sensor or visible light sensor and the second infrared sensor or visible light sensor; a detection unit that irradiates the displacement site identified by the identification unit with electromagnetic waves in the millimeter wave or terahertz band from the first millimeter wave sensor or terahertz band sensor and the second millimeter wave sensor or terahertz band sensor, and detects heart or blood vessel pulsation based on signal data output from the first millimeter wave sensor or terahertz band sensor and the second millimeter wave sensor or terahertz band sensor; a determination unit that determines abnormalities in the heart or blood vessels based on information related to the detected pulsation of the heart or blood vessels; An information processing system comprising:

Images