Information processing method, information processing system, and information processing program

Abstract

An information processing method according to the present disclosure includes calculating sound-collection characteristics on the basis of: sound-collection data obtained by collecting reference sounds with a sound-collection device; and reference sounds. The information processing method includes correcting, on the basis of the calculated sound-collection characteristics, the vital sounds collected by the sound-collection device.

Description

Information processing method, information processing system, and information processing program

[0001] The present disclosure relates to an information processing method, an information processing system, and an information processing program.

[0002] In recent years, progress has been made in realizing remote medical care such as online consultations, providing medical services to remote locations, and collaboration between multiple medical institutions. In addition, digital therapeutics (DTx), which provides prevention, diagnosis, and treatment of diseases, has been attracting attention as an application service or IT service.

[0003] In telemedicine, for example, when a doctor and a patient are far apart, the patient can talk to the doctor via a microphone, etc. In addition, in DTx, a user inputs information to be used for prevention, diagnosis, treatment, etc. into a user terminal such as a smartphone.

[0004] Japanese Patent Application Laid-Open No. 2020-162112

[0005] In remote medical care, doctors may not only talk to patients but also provide medical care based on the patient's vital signs. In this case, the patient needs to sense the vital signs using a dedicated sensor, such as a stethoscope sensor. In the following, the patient may be referred to simply as the user.

[0006] Furthermore, in DTx, when prevention, diagnosis, treatment, etc. of diseases are performed based on vital sounds, etc., the user needs to sense his or her own vital sounds using a dedicated sensor.

[0007] As described above, in telemedicine and DTx, users may need to prepare sensors to acquire vital signs. Therefore, a system that allows users to more easily receive telemedicine or DTx services is desired.

[0008] Therefore, the present disclosure proposes an information processing method, an information processing system, and an information processing program that allow users to more easily receive remote medical care or DTx services.

[0009] It should be noted that the above problem or object is merely one of multiple problems or objects that can be solved or achieved by multiple embodiments disclosed in this specification.

[0010] The information processing method of the present disclosure includes calculating sound collection characteristics based on sound collection data obtained by collecting a reference sound with a sound collection device and the reference sound, and correcting the vital sound collected by the sound collection device based on the calculated sound collection characteristics.

[0011] 1 is a diagram illustrating an example of a schematic configuration of an information processing system according to the proposed technology of the present disclosure. FIG. 1 is a diagram illustrating an example of sensors and sensing data according to the proposed technology of the present disclosure. FIG. 2 is a diagram illustrating an example of functional blocks of an information processing system according to the proposed technology of the present disclosure. FIG. 3 is a diagram illustrating an example of a method for acquiring vital sound according to the proposed technology of the present disclosure. FIG. 4 is a block diagram illustrating an example of a configuration of a sound source generating device according to the first embodiment of the present disclosure. FIG. 5 is a flowchart illustrating an example of a flow of correction processing according to the first embodiment of the present disclosure. FIG. 6 is a diagram illustrating an example of an information processing system according to the second embodiment of the present disclosure. FIG. 7 is a diagram illustrating an example of an information processing system according to the third embodiment of the present disclosure. FIG. 8 is a flowchart illustrating an example of a flow of correction processing according to the third embodiment of the present disclosure. FIG. 9 is a diagram illustrating an example of an information processing system according to the fourth embodiment of the present disclosure. FIG. 10 is a block diagram illustrating an example of a configuration of a sound collecting device according to the fourth embodiment of the present disclosure. FIG. 11 is a diagram illustrating an example of an information processing system according to the fifth embodiment of the present disclosure. FIG. 12 is a flowchart illustrating an example of a flow of correction processing according to the fifth embodiment of the present disclosure. FIG. 13 is a diagram illustrating an example of an information processing system according to the sixth embodiment of the present disclosure. FIG. 14 is a diagram illustrating another example of an information processing system according to the sixth embodiment of the present disclosure. FIG. 15 is a hardware configuration diagram illustrating an example of a computer that realizes the functions of a user terminal according to an embodiment of the present disclosure.

[0012] Hereinafter, embodiments of the present disclosure will be described in detail with reference to the accompanying drawings. In this specification and drawings, components having substantially the same functional configurations are designated by the same reference numerals, and redundant description will be omitted.

[0013] In this specification and drawings, similar components of the embodiments may be distinguished by adding at least one different alphabet and / or number after the same reference numeral. However, if there is no need to particularly distinguish between the similar components, only the same reference numeral will be used.

[0014] One or more embodiments (including examples, modifications, and application examples) described below can be implemented independently. However, at least a portion of the embodiments described below may be implemented in appropriate combination with at least a portion of another embodiment. These embodiments may include novel features that are different from each other. Therefore, these embodiments may contribute to solving different purposes or problems and may produce different effects from each other.

[0015] <<1. Introduction>> <1-1. Schematic Configuration of Information Processing System> First, a schematic configuration of an information processing system 100 according to the proposed technology of the present disclosure will be described. FIG. 1 is a diagram illustrating an example of a schematic configuration of the information processing system 100 according to the proposed technology of the present disclosure. A user of the information processing system 100 is referred to as user U1 and is illustrated. FIG. 1 illustrates a configuration common to the information processing systems 100 of the embodiments described below.

[0016] User U1 may be a patient with a disease. Examples of diseases include neurological diseases, muscular degenerative diseases, cardiovascular diseases, mental diseases, diabetes, immune / allergic diseases, and diseases specific to the elderly. More specifically, examples of diseases include Parkinson's disease, muscular dystrophy, cerebellar degenerative diseases, amyotrophic lateral sclerosis (ALS), arrhythmia, heart failure, high blood pressure, depression, dementia, sleep disorders, asthma, hay fever, sarcopenia / frailty, and the like. Note that user U1 is not limited to a patient with a disease. User U1 may also be a person who receives a service provided by the information processing system 100, for example, for the purpose of maintaining health.

[0017] The information processing system 100 includes an information acquisition device 1, an information processing device 2, and one or more terminals. Examples of the terminals include a medical professional terminal 3, a community member terminal 4, a life insurance / health insurance terminal 5, a product / service provider terminal 6, and an analysis terminal 7. These devices and terminals are configured to be able to communicate via a network N.

[0018] The information acquisition device 1 includes a sensor 11 and a user terminal 12. The sensor 11 is provided for the user U1. The sensing data may be data at a specific point in time or time-series data. The sensor 11 may be a wearable sensor attached to the user U1's body, or may be an image sensor (e.g., a surveillance camera) or a sound collection sensor (e.g., the sound collection unit of a smart speaker) that captures images of the user U1 and detects the user U1's sounds. Examples of body parts of the user U1 to which the sensor 11 may be attached include the wrist Ua1, forearm Ub1, upper arm Uc1, neck Ud1, waist Ue1, thigh Uf1, chest Ug1, fingers Uh1, face Ui1 (eyes, mouth, etc.), and ear Uj1, each of which is designated by a symbol. Specific examples of sensor devices include a smartphone, a smartwatch, a ring, earphones, a hearing aid, a head-mounted display, etc.

[0019] 2 is a diagram illustrating an example of a sensor 11 and sensing data according to the proposed technique of the present disclosure. Examples of the sensor 11 include an acceleration sensor, an angular velocity sensor, a position sensor, an imaging sensor, a sound collection sensor, and a vital sign sensor. Various known sensors may be used as these sensors.

[0020] An acceleration sensor detects acceleration. Sensing data of an angular velocity sensor is referred to as acceleration data. An angular velocity sensor is also called a gyro sensor, and detects angular velocity. Sensing data of an angular velocity sensor is referred to as angular velocity data. A position sensor detects position. Sensing data of a position sensor is referred to as position data.

[0021] The imaging sensor is an image sensor or the like, and captures an image. The sensing data of the imaging sensor is referred to as image data. Note that the image may be interpreted to include a video, and the imaging may be interpreted to include photography. The sound collection sensor is a microphone or the like, and detects sound. The sound collection sensor may be an external sound sensor provided in a hearable device such as a hearing aid or earphone. The sensing data of the sound collection sensor is referred to as sound data.

[0022] A vital sensor detects biological information. The sensed data of a vital sensor is called vital data. Examples of sensed data include heart rate, brain waves, electrocardiogram, blood flow, blood pressure, respiration, lung capacity, electromyography, body temperature, blood sugar level, weight, vibration, impact, and sweat. The above-mentioned image sensor and sound collection sensor may also be used as a non-contact vital sensor. For example, blood pressure and the like are detected from image data of the image sensor. Heart rate and the like are detected from sound data of the sound collection sensor.

[0023] Various information other than the above may also be included in the sensing data as sensing targets. For example, the results of operation of the user terminal 12 by user U1 may be included in the sensing data. Examples of operation results include operation time, operation time (operation timing), input voice, input document, etc. Medication / meal information regarding user U1's medication and meals may also be included in the sensing data. Medication / meal information may be input and acquired by operating the user terminal 12, or detected by a sensor provided on a medicine box, tray, etc. Health check results of user U1 may also be included in the sensing data.

[0024] Returning to FIG. 1 , the user terminal 12 is a terminal used by the user U1. Examples of the user terminal 12 include a smartphone, a tablet terminal, a PC, a television, etc., and the same may apply to terminals other than the user terminal 12 described below. The sensing data of the sensor 11 is transmitted from the sensor 11 to the user terminal 12 and collected. Note that a portion of the sensor 11 may be incorporated into the user terminal 12. The user terminal 12 transmits the collected sensing data to the information processing device 2. The information processing device 2 receives the sensing data.

[0025] The information processing device 2 processes the received sensing data. The details of the processing will be described later. Information obtained by the processing in the information processing device 2 (corresponding to "result information" in FIG. 3 described later) is transmitted to other devices or terminals.

[0026] The medical staff terminal 3 is a terminal used by a medical staff C1. Examples of the medical staff C1 include doctors, nurses, pharmacists, physical therapists, and caregivers.

[0027] The community member terminal 4 is a terminal used by a member M1. The member M1 belongs to the same community as the user U1, and is, for example, a family member of the user U1 or another patient with the same disease as the user U1.

[0028] The life insurance / health insurance terminal 5, the product / service provider terminal 6 and the analysis terminal 7 will be described later.

[0029] 3 is a diagram illustrating an example of functional blocks of an information processing system 100 according to the proposed technique of the present disclosure. In the information acquisition device 1, sensing data from a sensor 11 is collected by a user terminal 12.

[0030] The user terminal 12 includes a communication unit 13, a user interface unit 14, a processing unit 15, and a storage unit 16. An example of information stored in the storage unit 16 is an application program 161 (application software). Execution of the application program 161 provides an application used by the user U1. Examples of the application include an application that supports the rehabilitation of the user U1, an application that supports treatment for a disease, and an application that records disease symptoms, diet, etc. More specifically, examples of the application include a rehabilitation application used during rehabilitation, an application that provides cognitive behavioral therapy, and an application that records symptoms of cardiac disease or respiratory disease.

[0031] The communication unit 13 communicates with other devices, etc. For example, the communication unit 13 receives sensing data from the sensor 11 and result information (described later) from the information processing device 2. The communication unit 13 also transmits the sensing data to the information processing device 2.

[0032] The user interface unit 14 accepts operations of the user terminal 12 by the user U1 and presents information to the user U1.

[0033] The processing unit 15 functions as a control unit that controls each element of the user terminal 12 and executes various processes. For example, the processing unit 15 executes an application program 161, thereby providing the various applications described above. As an example of a rehabilitation application, a rehabilitation menu showing the contents of rehabilitation, for example, is presented (displayed, etc.) by the user interface unit 14. The user U1 performs rehabilitation according to the presented rehabilitation menu.

[0034] The information processing device 2 receives and processes the sensing data from the information acquisition device 1. The information processing device 2 includes a communication unit 21, an estimation unit 22, a storage unit 23, a recommendation unit 24, and a processing unit 25. Examples of information stored in the storage unit 23 include patient information 231, community information 232, an algorithm DB 233, a trained model 234, recommendation information 235, and anonymously processed information 236.

[0035] The patient information 231 includes information about the patient user U1. Examples of the patient information 231 include disease information, diagnosis information, medical records, checkup information, medication information, hospital visit history information, rehabilitation history information, etc., of the user U1. The disease information includes the name of the disease of the user U1, as described above. The diagnosis information, medical records, checkup information, medication information, and hospital visit history information are information about the diagnosis, medical treatment, checkup, medication, and hospital visit history of the user U1, and are provided, for example, from outside the information processing device 2 (such as the medical professional terminal 3). The rehabilitation history information is information about past rehabilitation performed by the user U1, and is obtained, for example, by the rehabilitation application described above and transmitted from the user terminal 12 to the information processing device 2.

[0036] The community information 232 is information about the community to which the user U1 belongs, and includes information about the member M1 and the community member terminal 4.

[0037] The algorithm DB 233, the trained model 234, the recommendation information 235, and the anonymously processed information 236 will be described later.

[0038] The communication unit 21 communicates with other devices, etc. For example, the communication unit 21 receives sensing data from the user terminal 12.

[0039] The estimation unit 22 performs estimation processing based on the sensing data. For example, the estimation unit 22 calculates various indices that can be used for estimation based on the sensing data. Examples of the indices include an index related to the user U1's physical function, an index related to the user's behavior, an index related to a disease, and an index related to the user's emotions. Various algorithms may be used to calculate the indices. The algorithm may be a trained model generated by machine learning using training data. The algorithm may be designed to output an index when sensing data is input, or may be designed to output an index when feature quantities obtained from the sensing data are input. The feature quantities may be calculated by the estimation unit 22 based on the sensing data. The calculation of feature quantities may be interpreted to include the generation, extraction, etc. of feature quantities.

[0040] Information including the estimation result of the estimation unit 22 is referred to as "result information" and is illustrated. The communication unit 21 transmits the result information to other devices and terminals, in this example, the user terminal 12, the medical worker terminal 3, the community member terminal 4, and the life insurance / health insurance terminal 5. The recommendation unit 24 and processing unit 25 of the information processing device 2 will be described later.

[0041] In the user terminal 12, the result information is presented by the user interface unit 14. The user U1 can know various indicators related to his / her own physical function, behavior, disease, etc. The result information transmitted from the information processing device 2 to the user terminal 12 may include content to be presented to the user U1. This makes it possible to present individual content based on the estimation result.

[0042] The medical staff terminal 3 includes a communication unit 31, a user interface unit 32, and a storage unit 33. An example of information stored in the storage unit 33 is medical information 331. The medical information 331 includes, for example, medical record information of the user U1, and is used by the medical staff C1 to diagnose the user U1, etc.

[0043] The communication unit 31 communicates with other devices, etc. For example, the communication unit 31 receives result information from the information processing device 2.

[0044] The user interface unit 32 accepts operations of the medical worker terminal 3 by the medical worker C1 and presents information to the medical worker C1. For example, result information from the information processing device 2 is presented, and the medical information 331 is updated accordingly, such as by adding medical record information. Intervention by the medical worker C1 is also possible. For example, individual content such as a rehabilitation menu customized by the medical worker C1 to suit the user U1 is generated. The content is transmitted to the user terminal 12 by the communication unit 31 and presented to the user U1.

[0045] The community member terminal 4 includes a communication unit 41 and a user interface unit 42. The communication unit 41 communicates with other devices, etc. For example, the communication unit 41 receives result information from the information processing device 2.

[0046] The user interface unit 42 accepts operations of the community member terminal 4 by the member M1 and presents information to the member M1. For example, result information from the information processing device 2 is presented and shared with the member M1. The result information sent from the information processing device 2 to the community member terminal 4 may include content to be presented to the member M1. This makes it possible to present individual content based on the estimation results.

[0047] The life insurance / health insurance terminal 5 is a terminal used by insurance companies, health insurance companies, etc. The life insurance / health insurance terminal 5 includes a communication unit 51, an analysis unit 52, and a storage unit 53. An example of information stored in the storage unit 53 is customer / employee information 531. The customer / employee information 531 includes information related to the life insurance, health insurance, etc. of user U1.

[0048] The communication unit 51 receives the result information from the information processing device 2. The analysis unit 52 analyzes the result information and identifies (calculates, etc.) insurance premiums and rewards. The identification may involve the work, judgment, etc. of employees of the life insurance company or health insurance company. Insurance premiums may be reduced or changed to a limited plan, etc. The communication unit 51 transmits the identified insurance premiums and recommended insurance premium / reward information to the user terminal 12. The insurance premium / reward information is presented by the user interface unit 14 of the user terminal 12.

[0049] The product / service provider terminal 6 is a terminal used by a company or the like that provides a product / service. Examples of products include wheelchairs, walking aids, rehabilitation equipment, health foods, health equipment, etc. An example of a service is a health application that can be executed on the user terminal 12.

[0050] The product / service provider terminal 6 includes a communication unit 61 and a user interface unit 62. For example, product / service information that associates the result information with products and services is input or generated via the user interface unit 62. For example, a condition indicating the association between an index shown in the result information and a product or service is input, and information including this condition is generated as product / service information. The communication unit 61 transmits the product / service information to the information processing device 2.

[0051] The communication unit 21 of the information processing device 2 receives product / service information from the product / service provider terminal 6 .

[0052] The recommendation unit 24 and processing unit 25 of the information processing device 2 will be described. The recommendation unit 24 generates recommendation information 235 including information on products and services to be recommended to user U1 based on the product / service information from the product / service provider terminal 6. For example, based on the conditions input at the product / service provider terminal 6, products and services corresponding to the result information are determined, and recommendation information 235 recommending them is generated. The communication unit 21 transmits the recommendation information 235 to the user terminal 12 and the community member terminal 4. The recommendation information 235 is presented by the user interface unit 14 of the user terminal 12 or by the user interface unit 42 of the community member terminal 4.

[0053] The processing unit 25 anonymizes the result information to generate anonymous processed information 236. The anonymous processed information 236 describes the anonymized personal information and the result information in association with each other.

[0054] The communication unit 21 of the information processing device 2 transmits the anonymously processed information 236 to the analysis terminal 7 .

[0055] The analysis terminal 7 is a terminal used by, for example, a company that provides the above-mentioned products / services, a pharmaceutical company that conducts clinical development, etc. The analysis terminal 7 includes a communication unit 71, an analysis unit 72, and a user interface unit 73.

[0056] The communication unit 71 receives the anonymously processed information 236 from the information processing device 2. The analysis unit 72 performs data analysis based on the anonymously processed information 236. The analysis may involve the work, judgment, etc. of company employees, etc. The user interface unit 73 presents information related to the data analysis, etc. Examples of analysis include analysis of user demographics for products such as health foods and health equipment, and data analysis for clinical development. The anonymously processed information 236 can be utilized for various services, such as marketing analysis by manufacturers, analysis of the proportion, age, gender, etc. of users with specific symptoms, and symptom monitoring for patients taking specific medications.

[0057] <1-2. Modified Examples of Information Processing System> In the above embodiment, an example has been described in which sensing data from the sensor 11 is transmitted to the information processing device 2 via the user terminal 12. However, some or all of the sensing data from the sensor 11 may be transmitted directly from the sensor 11 to the information processing device 2 without going through the user terminal 12.

[0058] In the above embodiment, an example has been described in which functions and information related to various services at the life insurance / health insurance terminal 5, product / service provider terminal 6, and analysis terminal 7, such as the recommendation unit 24, processing unit 25, recommendation information 235, and anonymously processed information 236, are provided in the information processing device 2. However, these functions and information may also be provided in a server device or the like (service provider server device) managed by the user of the corresponding terminal. The life insurance / health insurance terminal 5, product / service provider terminal 6, and analysis terminal 7 communicate with the corresponding server device or the like to use its functions. This can reduce the processing load on the information processing device 2 and simplify functions to reduce costs.

[0059] Some of the functions of the terminal may be provided in a server device or the like managed by the user of the corresponding terminal. For example, the functions of the analysis unit 52 of the life insurance / health insurance terminal 5 may be provided in a server device or the like managed by an insurance company, health insurance company, etc. The life insurance / health insurance terminal 5 communicates with the server device or the like to use its functions. The functions of the analysis unit 72 of the analysis terminal 7 may be provided in a server device or the like managed by a pharmaceutical company, etc. The analysis terminal 7 communicates with the server device or the like to use its functions. This can reduce the processing burden on the life insurance / health insurance terminal 5 and the analysis terminal 7, and simplify functions to reduce costs.

[0060] <1-3. Overview of Proposed Technology> As described above, the information processing system 100 of the present disclosure acquires sensing data such as vital sounds using the sensor 11. The sensor 11 may be, for example, a smartphone.

[0061] In the proposed technology of the present disclosure, for example, the information processing system 100 acquires vital sounds using a sound collection sensor (an example of a sound collection device, hereinafter also referred to simply as a microphone) such as a smartphone. As described above, the user U1 uses a user terminal 12 such as a smartphone. That is, in the proposed technology of the present disclosure, for example, the vital sounds are acquired using a sensor 11 mounted on the user terminal 12.

[0062] This allows the user U1 to more easily acquire vital sounds without having to prepare a dedicated sensor for acquiring vital sounds. Note that the user terminal 12, such as a smartphone, itself may serve as the sound collection device, or a sound collection sensor mounted on the user terminal 12 may serve as the sound collection device.

[0063] 4 is a diagram illustrating an example of a method for acquiring vital sounds according to the proposed technique of the present disclosure. In the example of FIG. 4, a case is illustrated in which snoring sounds of a user U1 are acquired as vital sounds.

[0064] 4, the user terminal 12 acquires the snoring sound of the user U1 as a vital sound via a microphone (not shown) mounted on the user terminal 12. Based on the acquired vital sound, the user terminal 12 performs, for example, a check for sleep apnea syndrome and presents the result to the user U1.

[0065] In this way, vital signs can be more easily acquired by using the user terminal 12 that the user U1 normally uses. However, the microphone of the user terminal 12, such as a smartphone, may have characteristics that are specialized for acquiring specific sounds (e.g., conversations, music, etc.).

[0066] The microphone mounted on the user terminal 12 has a characteristic of attenuating or amplifying sounds of specific frequencies (hereinafter also referred to as microphone characteristic).

[0067] The microphone characteristics may be characteristics according to, for example, the model or version of the operating system (OS) of the user terminal 12. Furthermore, an application that acquires vital signs sounds cannot generally know the microphone characteristics.

[0068] Therefore, if vital signs sounds are acquired using a microphone mounted on the user terminal 12, there is a risk that sounds different from the actual vital signs sounds of the user U1 will be acquired. Even if an analysis is performed using these vital signs sounds, there is a risk that an accurate analysis result will not be obtained.

[0069] Therefore, in the proposed technology of the present disclosure, the user terminal 12 (an example of an information processing device) corrects the vital sounds collected by the microphone using sound collection characteristics based on the reference sound, thereby enabling the user terminal 12 to acquire more accurate vital sounds, in other words, vital sounds closer to RAW data.

[0070] Fig. 5 is a diagram showing an example of a method for correcting vital sounds according to the proposed technique of the present disclosure. Fig. 5 illustrates a case in which a sound generator 200 generates a reference sound. That is, an information processing system 100 according to the proposed technique of the present disclosure may include the sound generator 200. Note that the sound generator 200 here is a device for generating a reference sound, and is assumed to store data of the reference sound that will serve as a sound source in advance.

[0071] 5A, the sound generator 200 generates a reference sound (step S1). For example, the sound generator 200 generates the reference sound from a speaker in accordance with an instruction from the user U1 or the user terminal 12.

[0072] Examples of the reference sound include environmental sounds around the user terminal 12, sweep sounds, and white noise. The reference sound may also be a sound specialized for a specific vital sound.

[0073] The user terminal 12 collects the reference sound generated by the sound source generating device 200 and acquires collected sound data (step S2). The user terminal 12 collects the reference sound using, for example, a microphone mounted thereon and generates collected sound data.

[0074] The user terminal 12 calculates sound collection characteristics based on the sound collection data and the reference sound (step S3). For example, the user terminal 12 calculates sound collection characteristics (functions) for generating sound collection data from the reference sound according to the difference between the sound collection data and the reference sound.

[0075] 5B, the user terminal 12 acquires vital sounds of the user U1 (step S4). Here, the user terminal 12 acquires snoring sounds as vital sounds of the user U1.

[0076] The user terminal 12 corrects the acquired vital sound using the sound collection characteristics (step S5). For example, the user terminal 12 generates the corrected vital sound by multiplying the acquired vital sound by the inverse characteristic (inverse function) of the sound collection characteristics.

[0077] The user terminal 12 checks the symptoms of the user U1 using the corrected vital signs sounds (step S6).

[0078] In this way, the user terminal 12 according to the proposed technology of the present disclosure calculates the sound collection characteristics of the microphone using the reference sound and corrects the vital sound based on the calculated sound collection characteristics, thereby enabling the user terminal 12 to acquire vital sound that is closer to the actual sound.

[0079] 4 and 5, snoring is given as an example of a vital sound, but the vital sound is not limited to this. The vital sound may be any sound (or sound data) that is used for diagnosis, analysis, etc. in telemedicine or DTx.

[0080] Examples of vital sounds include breathing sounds, coughing sounds, bowel sounds, heartbeat sounds, and speech sounds in addition to snoring sounds. The heartbeat sounds may be sounds of the user U1 himself / herself or sounds of a fetus or the like.

[0081] 4 and 5, the user U1 acquires vital signs sounds using a user terminal 12 such as a smartphone, but the device for acquiring vital signs sounds is not limited to the user terminal 12. For example, the user U1 may acquire vital signs sounds using an external device (not shown) such as an external microphone.

[0082] The external device in this case is not a dedicated device for acquiring vital sounds such as a stethoscope sensor, but a general device (microphone) such as a headset that is used for recording conversations, etc.

[0083] Even when such an external device is connected to the user terminal 12 and used, there is a risk that a particular sound may be amplified or attenuated by the user terminal 12 or the external device. In this case, the user terminal 12 can acquire more accurate vital sounds by identifying the sound collection characteristics and correcting the vital sounds.

[0084] 5, the reference sound is generated by a dedicated sound source generating device 200, but the device that generates the reference sound is not limited to this. For example, the reference sound may be generated by the user terminal 12, or by a general-purpose external speaker (not shown).

[0085] 5, the user terminal 12 checks the symptoms of the user U1 using the corrected vital sign sounds, but the symptom check, etc. may be performed by something other than the user terminal 12. For example, the operation using the corrected vital sign sounds (here, the symptom check, etc.) may be performed by a doctor in a remote location, or may be performed by an external server (not shown), etc.

[0086] <<2. First embodiment>> <2-1. Configuration example> <2-1-1. Configuration example of user terminal> As shown in Fig. 5, the information processing system 100 according to the first embodiment includes a user terminal 12 and a sound source generating device 200. A detailed configuration example of the user terminal 12 will be described using Fig. 6. Note that the same components as those in the user terminal 12 in Fig. 3 are denoted by the same reference numerals, and some explanations may be omitted.

[0087] 6 is a block diagram showing an example configuration of a user terminal 12 according to the first embodiment of the present disclosure. The user terminal 12 in FIG. 6 includes a communication unit 13, a user interface unit 14, a processing unit 15, a storage unit 16, and an input / output unit 17.

[0088] (Communication Unit 13) The communication unit 13 is a communication interface for communicating with other devices. The communication unit 13 may be a wired interface or a wireless interface.

[0089] For example, the communication unit 13 is a LAN (Local Area Network) interface such as a Network Interface Card (NIC), or the communication unit 13 communicates with other devices using wireless communication technology such as Bluetooth (registered trademark).

[0090] The communication unit 13 may be configured to be able to use wireless access technologies (wireless communication methods) other than Bluetooth, such as LTE (Long Term Evolution), NR (New Radio), Wi-Fi (registered trademark), etc. For example, the user terminal 12 may be configured to be able to use Bluetooth and Wi-Fi.

[0091] (Storage Unit 16) The storage unit 16 is a data readable / writable storage device such as a dynamic random access memory (DRAM), a static random access memory (SRAM), a flash memory, a hard disk, etc. The storage unit 16 stores, for example, sound source data of a reference sound output from the sound source generating device 200.

[0092] (Input / Output Unit 17) The input / output unit 17 is a user interface for exchanging information with the user U1. The input / output unit 17 includes a microphone 171 and a speaker 172.

[0093] Furthermore, for example, the input / output unit 17 may include an operation device such as a keyboard, a mouse, operation keys, a touch panel, etc., which allows the user U1 to perform various operations. Alternatively, the input / output unit 17 may include a display device such as a liquid crystal display (LCD) or an organic electroluminescence display (OLED).

[0094] The input / output unit 17 may include an acoustic device such as a buzzer in addition to the speaker 172. The input / output unit 17 may also include a lighting device such as an LED (Light Emitting Diode) lamp.

[0095] The microphone 171 is a sound collection sensor (microphone) that collects surrounding sounds. The speaker 172 is an acoustic device that generates sounds. In this embodiment, the sound source generating device 200 generates the reference sound, so the input / output unit 17 does not need to include the speaker 172.

[0096] (Processing Unit 15) The processing unit 15 executes various processes of the user terminal 12. The processing unit 15 may be a controller that controls each unit of the user terminal 12. The processing unit 15 is realized by a processor such as a CPU (Central Processing Unit) or an MPU (Micro Processing Unit). For example, the processing unit 15 is realized by the processor executing various programs stored in a storage device inside the user terminal 12 using RAM (Random Access Memory) or the like as a working area.

[0097] The processing unit 15 may be realized by an integrated circuit such as an application specific integrated circuit (ASIC) or a field programmable gate array (FPGA). A CPU, an MPU, an ASIC, and an FPGA can all be considered as controllers.

[0098] The processing unit 15 includes an acquisition unit 151, a calculation unit 152, and a correction unit 153. Each block constituting the processing unit 15 (acquisition unit 151 to correction unit 153) is a functional block indicating a function of the processing unit 15.

[0099] These functional blocks may be software blocks or hardware blocks. For example, each of the above-mentioned functional blocks may be a software module implemented by software (including a microprogram), or may be a circuit block on a semiconductor chip (die). Of course, each functional block may be a processor or an integrated circuit. The functional blocks may be configured in any manner. Note that the processing unit 15 may be configured by functional units different from the above-mentioned functional blocks.

[0100] (Acquisition unit 151) The acquisition unit 151 acquires various pieces of information to be used by the calculation unit 152 and the correction unit 153. For example, the acquisition unit 151 acquires sound data collected by the microphone 171 from the microphone 171. Examples of the sound data include collected sound data in which the microphone 171 records a reference sound, and vital sound data in which the microphone 171 records the vital sounds of the user U1.

[0101] The acquisition unit 151 outputs the acquired collected sound data to the calculation unit 152. The acquisition unit 151 outputs the acquired vital sound data to the correction unit 153.

[0102] Furthermore, the acquisition unit 151 acquires sound source data of a reference sound that is used by the calculation unit 152 to calculate sound collection characteristics. When the storage unit 16 stores the sound source data, the acquisition unit 151 acquires the sound source data from the storage unit 16. Alternatively, the acquisition unit 151 may acquire the sound source data from an external device (not shown) such as a cloud server.

[0103] The acquisition unit 151 outputs the acquired sound source data to the calculation unit 152 .

[0104] The acquisition unit 151 acquires the sound collection characteristics calculated by the calculation unit 152. The acquisition unit 151 acquires the sound collection characteristics directly from the calculation unit 152. Alternatively, if the sound collection characteristics calculated by the calculation unit 152 are stored in the storage unit 16, the acquisition unit 151 may acquire the sound collection characteristics from the storage unit 16.

[0105] The acquisition unit 151 outputs the acquired sound collection characteristics to the correction unit 153 .

[0106] (Calculation Unit 152) The calculation unit 152 compares the sound collection data and the sound source data to calculate the sound collection characteristics. For example, the calculation unit 152 performs an algorithm calculation to minimize the difference between the sound collection data and the sound source data.

[0107] The calculation unit 152 calculates, for example, sound collection characteristics that reduce the difference between the sound collection data and the sound source data. For example, the calculation unit 152 calculates sound collection characteristics that make the sound collection data closer to the sound source data. More specifically, the calculation unit 152 calculates, as the sound collection characteristics, a transfer function that receives the sound collection data as input and outputs the sound source data.

[0108] Alternatively, the calculation unit 152 may calculate a transfer function for obtaining sound collection data from sound source data. In this case, the calculation unit 152 may use the inverse function of the calculated transfer function as the sound collection characteristic.

[0109] The calculation unit 152 may calculate sound collection characteristics according to the vital sounds acquired by the user terminal 12. For example, if the vital sounds are snoring sounds, the calculation unit 152 may calculate sound collection characteristics (transfer function) that amplify (emphasize) sounds of specific frequencies according to the frequency distribution of the snoring sounds.

[0110] In this way, the calculation unit 152 may calculate sound collection characteristics that amplify sounds in a frequency band that includes vital sounds, in accordance with vital sounds acquired in response to a service (such as remote medical care or DTx) provided by the information processing system 100. In this case, the calculation unit 152 may also calculate sound collection characteristics that attenuate sounds other than those in a frequency band that includes vital sounds.

[0111] The calculation unit 152 outputs the calculated sound collection characteristics to the correction unit 153. Alternatively, the calculation unit 152 causes the storage unit 16 to store the sound collection characteristics.

[0112] (Correction Unit 153) The correction unit 153 corrects the vital sound data acquired by the acquisition unit 151 using the sound collection characteristics calculated by the calculation unit 152, and generates corrected data.

[0113] The correction unit 153 inputs vital sound data into a sound collection characteristic, which is a transfer function, and acquires the output as correction data, for example.

[0114] The correction unit 153 outputs the correction data to the processing unit 15 that performs subsequent processing. For example, in the case of DTx, the correction unit 153 outputs the correction data to an application (not shown) that analyzes the correction data. Alternatively, the correction unit 153 transmits the correction data to the information processing device 2 (see FIG. 3 ) or the like via the communication unit 13.

[0115] In addition, in the case of remote medical treatment, the correction unit 153 transmits the correction data via the communication unit 13 to the medical staff terminal 3 used by the doctor, the information processing device 2, etc.

[0116] 7 is a block diagram showing an example of the configuration of a sound generator 200 according to the first embodiment of the present disclosure. The sound generator 200 in FIG. 7 includes a communication unit 210, a user interface unit 220, a processing unit 230, a storage unit 240, and an input / output unit 250.

[0117] (Communication Unit 210) The communication unit 210 is a communication interface for communicating with other devices. The communication unit 210 may be a wired interface or a wireless interface.

[0118] For example, the communication unit 210 is a LAN interface such as a NIC, or the communication unit 210 communicates with other devices using wireless communication technology such as Bluetooth.

[0119] The communication unit 210 may be configured to be able to use wireless access technologies (wireless communication methods) other than Bluetooth, such as LTE, NR, Wi-Fi, etc. For example, the sound source generation device 200 may be configured to be able to use Bluetooth and Wi-Fi.

[0120] (Storage Unit 240) The storage unit 240 is a data readable / writable storage device such as a DRAM, an SRAM, a flash memory, a hard disk, etc. The storage unit 240 stores, for example, sound source data of the reference sound output from the sound source generating device 200.

[0121] (Input / Output Unit 250) The input / output unit 250 is a user interface for exchanging information with the user. The input / output unit 250 includes a speaker 2501.

[0122] Furthermore, for example, the input / output unit 250 may include an operation device for allowing a user to perform various operations, such as a keyboard, a mouse, operation keys, a touch panel, etc. Alternatively, the input / output unit 250 may include a display device, such as a liquid crystal display or an organic EL display.

[0123] The input / output unit 250 may include an audio device such as a buzzer in addition to the speaker 2501. The input / output unit 250 may also include a lighting device such as an LED lamp. The input / output unit 250 may also include an input device such as a microphone.

[0124] (Processing Unit 230) The processing unit 230 executes various processes of the sound generation device 200. The processing unit 230 may be a controller that controls each unit of the sound generation device 200. The processing unit 230 is realized by a processor such as a CPU or an MPU. For example, the processing unit 230 is realized by the processor executing various programs stored in a storage device inside the sound generation device 200 using a RAM or the like as a working area.

[0125] The processing unit 230 may be realized by an integrated circuit such as an ASIC or an FPGA. A CPU, an MPU, an ASIC, and an FPGA can all be considered as a controller.

[0126] The processing unit 230 includes a playback unit 2301. The block (playback unit 2301) that constitutes the processing unit 230 is a functional block that indicates the function of the processing unit 230.

[0127] This functional block may be a software block or a hardware block. For example, the above-mentioned functional block may be a software module realized by software (including a microprogram), or may be a circuit block on a semiconductor chip (die). Of course, the functional block may be a processor or an integrated circuit. The functional block may be configured in any manner. Note that the processing unit 230 may be configured by functional units different from the above-mentioned functional blocks.

[0128] (Playback unit 2301) The playback unit 2301 outputs the reference sound from the speaker 2501. The playback unit 2301, for example, acquires sound source data of the reference sound from the storage unit 240 and plays back the sound source data, thereby generating the reference sound from the speaker 2501.

[0129] For example, when the memory unit 240 does not store sound source data, the playback unit 2301 may acquire the sound source data from another device (e.g., a user terminal 12 or a server device (not shown)) via the communication unit 210.

[0130] Furthermore, if the sound source generating device 200 does not communicate with other devices, such as if the playback unit 2301 does not acquire sound source data from other devices, the communication unit 210 may be omitted.

[0131] 8 is a flowchart showing an example of the flow of the correction process according to the first embodiment of the present disclosure. The correction process shown in FIG. 8 is performed by an application that provides a service such as DTx or telemedicine when the application is launched. Alternatively, the correction process may be performed in accordance with an instruction from user U1.

[0132] The application corresponds to the acquisition unit 151, calculation unit 152, and correction unit 153 of the processing unit 15 of the user terminal 12 described above, or is a function (or program) executed by these functional blocks.

[0133] 8, the application determines whether or not to perform calibration (step S101). Here, calibration means, for example, calculation or updating of sound collection characteristics.

[0134] For example, the application may determine to perform calibration when an application providing a service such as DTx or telemedicine is launched for the first time (first launch). Alternatively, the application may determine to perform calibration when the application is launched for the first time after an OS version update (first launch after an OS update). Furthermore, the application may determine to perform calibration when a predetermined period of time has passed since calibration was performed.

[0135] If it is determined that calibration is not to be performed (step S101; No), the application executes the process of step S105.

[0136] On the other hand, if it is determined that calibration is to be performed (step S101; Yes), the application requests output of a reference sound (step S102).

[0137] For example, the application may request the user U1, by display, voice, or the like, to output a reference sound from the sound generator 200. Alternatively, if the user terminal 12 can communicate with the sound generator 200, the application may directly request (or instruct) the sound generator 200 to output a reference sound.

[0138] The application acquires collected sound data of the reference sound (step S103). The application acquires collected sound data of the sound collected by the microphone 171.

[0139] For example, when the user U1 presses a recording start button or the like displayed on the display, the application may start collecting the reference sound output from the sound source generating device 200. Alternatively, the application may start collecting the reference sound after a predetermined period has elapsed since the application requested the output of the reference sound.

[0140] This collected sound data is data that has been subjected to specific signal processing by the user terminal 12. The application cannot know what type of specific signal processing this is.

[0141] The application calculates sound collection characteristics using the acquired sound collection data and sound source data of the reference sound (step S104). For example, when the application receives the sound collection data as input, it calculates a transfer function for outputting the reference sound as the sound collection characteristics.

[0142] Next, the application acquires vital sound of the user U1 (step S105). The application acquires data of the vital sound collected by the microphone 171 (vital sound data).

[0143] The application acquires the vital sounds of the user U1, for example, when the user U1 presses a recording start button or the like displayed on the display. Alternatively, the application may start acquiring the vital sounds immediately after calibration is performed (or after the application is started if calibration is not performed).

[0144] Alternatively, the application may start acquiring vital sounds when a predetermined time (for example, a bedtime set by the user U1) arrives.

[0145] The application corrects the acquired vital sound (step S106). The application corrects the acquired vital sound data using the sound collection characteristics to generate corrected data. If the sound collection characteristics are the transfer function described above, the application inputs the acquired vital sound data into the transfer function to acquire the corrected data.

[0146] The application may correct the acquired vital sign sounds in real time, or may correct them after the acquisition (recording) of the vital sign sounds is completed. The acquisition of the vital sign sounds may be terminated, for example, by an instruction from user U1. User U1 may instruct the end of the acquisition of the vital sign sounds by, for example, pressing a recording stop (end) button or the like displayed on the display.

[0147] 8, the application performs both the calculation of the sound collection characteristics and the correction of the vital sounds, but the processing performed by the application is not limited to this. For example, the application may perform the calculation processing for calculating the sound collection characteristics and the correction processing for correcting the vital sounds as separate processing.

[0148] As described above, the user terminal 12 according to the first embodiment of the present disclosure collects the reference sound output from the sound source generating device 200 via the microphone 171 and acquires sound collection data. The user terminal 12 calculates sound collection characteristics based on the acquired sound collection data and the reference sound. The user terminal 12 corrects the vital sounds of the user U1 based on the sound collection characteristics.

[0149] This allows the user terminal 12 to acquire vital signs sounds that are closer to the actual sounds. Furthermore, the user U1 can acquire vital signs sounds using the user terminal 12 that he or she is using, and can more easily enjoy services such as DTx and remote medical care.

[0150] Furthermore, by using a dedicated sound source generating device 200 that generates a reference sound, the user terminal 12 can calculate the sound collection characteristics with higher accuracy.

[0151] <<3. Second Embodiment>> In the first embodiment described above, the information processing system 100 includes the user terminal 12 and the sound generator 200. That is, the sound generator 200 is an acoustic device different from the user terminal 12 (an example of a device that corrects vital sounds). The sound generator 200 is also a device dedicated to generating a reference sound. However, the sound generator 200 is not limited to a device dedicated to generating a reference sound. For example, the sound generator 200 may be an acoustic device (e.g., an external speaker) that communicates with the user terminal 12.

[0152] Fig. 9 is a diagram illustrating an example of an information processing system 100 according to the second embodiment of the present disclosure. In the example of Fig. 9, the sound source generating device 200 is an external speaker wirelessly connected to the user terminal 12. Note that Fig. 9 omits illustration of some components of the information processing system 100.

[0153] 9, the user terminal 12 collects the reference sound output from the sound source generating device 200, which is an external speaker, and calculates the sound collection characteristics. The user terminal 12 corrects the vital sounds of the user U1 based on the calculated sound collection characteristics.

[0154] In this case, the configurations of the user terminal 12 and the sound generator 200 are the same as those in Figures 6 and 7. However, the sound generator 200 according to this embodiment differs from the sound generator 200 according to the first embodiment in that it does not store sound source data of the reference sound. In this case, the sound generator 200 according to this embodiment does not need to have the storage unit 240.

[0155] The sound generator 200 is connected to the user terminal 12 via a wired or wireless connection. As described above, the sound generator 200 according to this embodiment does not store sound source data. Therefore, the user terminal 12 according to this embodiment transmits sound source data to the sound generator 200.

[0156] The sound generator 200 generates a reference sound by playing back the sound source data received from the user terminal 12. When the sound generator 200 plays back the received sound source data as is, the processing unit 230 of the sound generator 200 (or a part of the functions of the processing unit 230) may be omitted.

[0157] 8, when requesting the sound generator 200 to output a reference sound, the user terminal 12 transmits sound source data to the sound generator 200. For example, the user terminal 12 transmits the sound source data to the sound generator 200 to request (demand) the output of a reference sound.

[0158] In this way, by using an acoustic device such as an external speaker connected to the user terminal 12 as the sound source generating device 200, the user U1 does not need to prepare a dedicated sound source generating device 200. This allows the user terminal 12 to more easily calculate the sound collection characteristics of the microphone 171.

[0159] However, since the sound source generating device 200 according to this embodiment is not a dedicated device but a general-purpose device (for example, an external speaker), it is not necessarily guaranteed that it can output a reference sound for calculating sound collection characteristics with a predetermined accuracy. Therefore, when it is required that the user terminal 12 calculates sound collection characteristics with higher accuracy more reliably, it is desirable to use the dedicated sound source generating device 200 as shown in the first embodiment.

[0160] Here, it is assumed that the sound source generating device 200, which is an external speaker or the like, is connected to the user terminal 12 that calculates the sound collection characteristics, but the device to which the sound source generating device 200 is connected may be connected to another user terminal 12. In other words, the other user terminal 12 that does not calculate the sound collection characteristics may transmit sound source data to the sound source generating device 200.

[0161] For example, the first user terminal 12 that calculates the sound collection characteristics may be a smartphone, and the second user terminal 12 that transmits the sound source data may be a PC (Personal Computer).

[0162] In this case, for example, a user U1 who has received a request from a first user terminal 12 to output a reference sound operates the second user terminal 12 to transmit sound source data to the sound generator 200. This causes the reference sound to be output from the sound generator 200. The sound source data may be held by the second user terminal 12, or may be acquired by the second user terminal 12 from a cloud server or the like.

[0163] In this case, the sound generator 200 may be a speaker 172 mounted on the second user terminal 12 (i.e., an internal speaker of the second user terminal 12).

[0164] <<4. Third Embodiment>> In the first and second embodiments described above, the sound source generating device 200 generates the reference sound, but the device that generates the reference sound is not limited to the sound source generating device 200. For example, the user terminal 12 (an example of a device that corrects vital sounds) may generate the reference sound.

[0165] Fig. 10 is a diagram illustrating an example of an information processing system 100 according to a third embodiment of the present disclosure. In the example of Fig. 10, the user terminal 12 generates the reference sound. In other words, the sound source generating device 200 may be installed in the user terminal 12. Note that Fig. 10 omits illustration of some components of the information processing system 100.

[0166] 10 , a reference sound output from a speaker 172 of the user terminal 12 is collected by a microphone 171 of the user terminal 12. The user terminal 12 calculates sound collection characteristics using the collected reference sound. The user terminal 12 corrects the vital sounds of the user U1 based on the calculated sound collection characteristics.

[0167] In this case, the configuration of the user terminal 12 is the same as that shown in Fig. 6. In this case, for example, it can be said that the speaker 172 of the input / output unit 17 of the user terminal 12 corresponds to the sound source generating device 200 according to the first and second embodiments.

[0168] 11 is a flowchart showing an example of the flow of a correction process according to the third embodiment of the present disclosure. The correction process shown in FIG. 11 is performed by an application that provides a service such as DTx or telemedicine when the application is launched. Alternatively, the correction process may be performed in accordance with an instruction from user U1.

[0169] 11, the same steps as the correction process in FIG. 8 are denoted by the same reference numerals, and the description thereof will be omitted.

[0170] The application that has determined in step S101 to perform calibration outputs a reference sound (step S201). For example, the application plays back the sound source data stored in the storage unit 16, and outputs the reference sound from the speaker 172.

[0171] In this way, by having the user terminal 12 play back the reference sound, the user terminal 12 can calculate the sound collection characteristics and correct the vital sounds without the user U1 having to prepare the sound source generating device 200. This allows the user U1 to more easily have the user terminal 12 acquire more accurate vital sounds.

[0172] <<5. Fourth Embodiment>> In the first to third embodiments described above, the user terminal 12 collects the reference sound, but the device that collects the reference sound is not limited to the user terminal 12. For example, an external sound collection sensor (sound collection device) such as an external microphone that is different from the user terminal 12 (an example of a device that corrects vital sounds) may collect the reference sound.

[0173] Fig. 12 is a diagram illustrating an example of an information processing system 100 according to a fourth embodiment of the present disclosure. The information processing system 100 illustrated in Fig. 12 differs from the information processing system 100 according to the first embodiment in that it includes a sound collection device 300 instead of the sound source generation device 200. Note that Fig. 12 omits illustration of some components of the information processing system 100.

[0174] 12 , the user terminal 12 outputs a reference sound from the speaker 172. The sound collection device 300 collects the reference sound and generates collected sound data. The sound collection device 300 transmits the collected sound data to the user terminal 12.

[0175] The user terminal 12 calculates the sound collection characteristics based on the received sound collection data, and corrects the vital sounds of the user U1 using the calculated sound collection characteristics.

[0176] In this way, the user terminal 12 can calculate the sound collection characteristics based on the reference sound collected using the sound collection device 300 other than the microphone 171 mounted on the user terminal 12 .

[0177] For example, it is conceivable that the user U1 does not use the microphone 171 of the user terminal 12, but collects vital sounds using the sound collection device 300 that communicates with the user terminal 12. In such a case, the user terminal 12 can calculate the sound collection characteristics with higher accuracy by calculating the sound collection characteristics using the sound collection device 300 that actually collects the vital sounds.

[0178] Fig. 13 is a block diagram showing an example configuration of a sound collection device 300 according to the fourth embodiment of the present disclosure. Note that the configuration of the user terminal 12 is the same as that of the user terminal 12 shown in Fig. 6, and therefore description thereof will be omitted.

[0179] The sound collection device 300 in FIG. 13 includes a communication unit 310 , a user interface unit 320 , a processing unit 330 , a storage unit 340 , and an input / output unit 350 .

[0180] (Communication Unit 310) The communication unit 310 is a communication interface for communicating with other devices. The communication unit 310 may be a wired interface or a wireless interface.

[0181] For example, the communication unit 310 is a LAN interface such as a NIC, or the communication unit 310 communicates with other devices using wireless communication technology such as Bluetooth.

[0182] The communication unit 310 may be configured to be able to use wireless access technologies (wireless communication methods) other than Bluetooth, such as LTE, NR, Wi-Fi, etc. For example, the sound collection device 300 may be configured to be able to use Bluetooth and Wi-Fi.

[0183] (Storage Unit 340) The storage unit 340 is a data readable / writable storage device such as a DRAM, an SRAM, a flash memory, or a hard disk.

[0184] (Input / Output Unit 350) The input / output unit 350 is a user interface for exchanging information with the user U1. The input / output unit 350 includes a microphone 3502.

[0185] Furthermore, for example, the input / output unit 350 may include an operation device for allowing a user to perform various operations, such as a keyboard, a mouse, operation keys, a touch panel, etc. Alternatively, the input / output unit 350 may include a display device, such as a liquid crystal display or an organic EL display.

[0186] The input / output unit 350 may include an audio device such as a speaker or a buzzer. The input / output unit 350 may also include a lighting device such as an LED lamp.

[0187] (Processing Unit 330) The processing unit 330 executes various processes of the sound collection device 300. The processing unit 330 may be a controller that controls each unit of the sound collection device 300. The processing unit 330 is realized by a processor such as a CPU or an MPU. For example, the processing unit 330 is realized by the processor executing various programs stored in a storage device inside the sound collection device 300 using a RAM or the like as a working area.

[0188] The processing unit 330 may be realized by an integrated circuit such as an ASIC or an FPGA. A CPU, an MPU, an ASIC, and an FPGA can all be considered as a controller.

[0189] Furthermore, when the sound collection device 300 is configured to simply collect sound and notify the user terminal 12, some components such as the processing unit 330 and the storage unit 340 may be omitted.

[0190] The processing of the user terminal 12 is the same as the correction processing of FIG. 11 except that in step S103, the collected sound data is acquired from the sound collection device 300 instead of the microphone 171, and therefore description thereof will be omitted here.

[0191] In this way, the user terminal 12 according to this embodiment can acquire sound collection data by collecting a reference sound using the sound collection device 300, which is an external device.

[0192] For example, when the user U1 acquires vital sounds using a sound collection device 300 that is specialized for acquiring vital sounds, the user terminal 12 may apply a signal to the vital sounds. Even in such a case, the application of the user terminal 12 can calculate the sound collection characteristics and correct the vital sounds, thereby acquiring more accurate vital sounds.

[0193] <<6. Fifth Embodiment>> In the above-described fourth embodiment, the user terminal 12 outputs a reference sound and the sound collection device 300 collects the sound, but the sound collection device 300 does not have to collect the reference sound output by the user terminal 12. For example, the sound collection device 300 may store sound collection data in which the reference sound is collected in advance. In this case, the user terminal 12 acquires this sound collection data from the sound collection device 300 and calculates sound collection characteristics.

[0194] Fig. 14 is a diagram illustrating an example of an information processing system 100 according to a fifth embodiment of the present disclosure. The information processing system 100 illustrated in Fig. 14 includes a user terminal 12 and a sound collection device 300. Note that Fig. 14 omits illustration of some components of the information processing system 100.

[0195] 14 , the user terminal 12 does not output the reference sound from the speaker 172. The user terminal 12 communicates with the sound collection device 300. The user terminal 12 acquires, from the sound collection device 300, collected sound data in which the sound collection device 300 records the reference sound.

[0196] The user terminal 12 calculates the sound collection characteristics based on the received sound collection data, and corrects the vital sounds of the user U1 using the calculated sound collection characteristics.

[0197] In this way, the user terminal 12 calculates the sound collection characteristics based on the reference sound recorded in advance by the sound collection device 300. This saves the sound collection device 300 the trouble of collecting the reference sound, and allows the user terminal 12 to calculate the sound collection characteristics in a shorter time.

[0198] Furthermore, since the user terminal 12 does not need to output a reference sound, the user terminal 12 can calculate the sound collection characteristics even in an environment where quietness is required, for example.

[0199] It should be noted that the sound collection device 300 is assumed to hold sound collection data in which a reference sound is recorded in advance at the time of manufacture, shipping, etc. This sound collection data may be data in which the reference sound is collected by the sound collection device 300, or may be data in which the reference sound is collected using a device having the same characteristics (or the same performance) as the sound collection device 300.

[0200] The configuration of the sound collection device 300 is the same as that of the sound collection device 300 shown in Figure 13, except that the memory unit 340 has sound collection data in which reference sounds have been pre-recorded, so a description thereof will be omitted here.

[0201] 15 is a flowchart showing an example of the flow of a correction process according to the fifth embodiment of the present disclosure. The correction process shown in FIG. 15 is performed by an application that provides a service such as DTx or telemedicine when the application is launched. Alternatively, the correction process may be performed in accordance with an instruction from user U1.

[0202] The correction process shown in Fig. 15 differs from the process in Fig. 11 in that it does not include step S201 of outputting a reference sound. Of the other processes, the same processes as those in the correction process in Fig. 11 are denoted by the same reference numerals, and descriptions thereof will be omitted.

[0203] The application that has determined in step S101 to perform calibration acquires sound collection data from the sound collection device 300 (step S301). The application that has acquired the sound collection data executes processing such as calculation of sound collection characteristics.

[0204] In this way, the application can calculate the sound collection characteristics using the sound collection data stored in advance in the sound collection device 300 and correct the vital sounds.

[0205] <<7. Sixth embodiment>> In the fourth embodiment described above, the user terminal 12 outputs a reference sound and the sound collection device 300 collects the sound, but the sound collection device 300 may also collect a reference sound output by a device other than the user terminal 12 (for example, an external device such as the sound source generating device 200).

[0206] Fig. 16 is a diagram illustrating an example of an information processing system 100 according to a sixth embodiment of the present disclosure. The information processing system 100 illustrated in Fig. 16 includes a user terminal 12, a sound source generating device 200, and a sound collecting device 300. Note that some components of the information processing system 100 are not illustrated in Fig. 16.

[0207] In FIG. 16, for example, a sound generator 200 (an external speaker in the example of FIG. 16) that communicates with the user terminal 12 acquires sound source data from the user terminal 12 and outputs a reference sound.

[0208] The sound collection device 300 collects the reference sound output by the sound source generating device 200 and generates collected sound data. The sound collection device 300 transmits the collected sound data to the user terminal 12.

[0209] The user terminal 12 calculates the sound collection characteristics based on the received sound collection data, and corrects the vital sounds of the user U1 using the calculated sound collection characteristics.

[0210] In this way, the sound collection device 300 collects the reference sound output by the sound source generating device 200 to generate sound collection data, and the user terminal 12 can calculate the sound collection characteristics based on this sound collection data.

[0211] In addition, in Figure 16, the sound source generating device 200 is shown as an external speaker connected to the user terminal 12, but the sound source generating device 200 may also be a dedicated device for outputting reference sounds as in the first embodiment.

[0212] The configuration of the user terminal 12 is the same as the configuration of the user terminal 12 shown in the first and second embodiments. The configuration of the sound source generation device 200 is the same as the configuration of the sound source generation device 200 shown in the first and second embodiments. The configuration of the sound collection device 300 is the same as the configuration of the sound collection device 300 shown in the fourth embodiment.

[0213] The correction process executed by the application of the user terminal 12 is the same as the correction process shown in FIG. 8 except that the sound collection data is acquired from the sound collection device 300 in step S103.

[0214] In this way, the user terminal 12 may output and collect the reference sound using external devices (here, the sound source generating device 200 and the sound collecting device 300).

[0215] Here, the sound source generating device 200 and the sound collecting device 300 are described as separate devices, but the sound source generating device 200 and the sound collecting device 300 may be mounted in a single device. Examples of such a device (hereinafter also simply referred to as an integrated device) include a headset and a smart speaker.

[0216] 17 is a diagram illustrating another example of the information processing system 100 according to the sixth embodiment of the present disclosure. The information processing system 100 illustrated in FIG. 17 includes a user terminal 12 and an all-in-one device 400.

[0217] The all-in-one device 400 shown in Fig. 17 is, for example, a headset. Although the all-in-one device 400 is a headset in Fig. 17, the all-in-one device 400 is not limited to a headset. The all-in-one device 400 may be, for example, a smart speaker, as long as it is equipped with the sound source generating device 200 and the sound collecting device 300.

[0218] The all-in-one device 400 is equipped with, for example, headphones as the sound source generating device 200. The all-in-one device 400 is also equipped with a microphone as the sound collecting device 300.

[0219] The user terminal 12 outputs a reference sound from the sound source generating device 200, which is a headphone, and acquires collected sound data collected by the sound collecting device 300, which is a microphone.

[0220] Alternatively, for example, similar to the sound collection device 300 according to the fifth embodiment, if the integrated device 400 holds sound collection data in which a reference sound has been collected in advance, the user terminal 12 may acquire this sound collection data from the integrated device 400.

[0221] In this way, the user terminal 12 may output and collect the reference sound using the all-in-one device 400 (here, the sound source generating device 200 and the sound collecting device 300). In other words, the sound source generating device 200 and the sound collecting device 300 may be different devices or the same device.

[0222] When no particular distinction is made between the sound source generating device 200, the sound collecting device 300, and the all-in-one device 400, they are also simply referred to as external devices (of the user terminal 12).

[0223] <<8. Other Embodiments>> The processing according to each of the above-described embodiments may be implemented in various different forms other than the above-described embodiments.

[0224] For example, when an application running on the user terminal 12 executes a correction process, it may select a device that outputs a reference sound and a device that collects the reference sound.

[0225] For example, when the application determines to perform calibration in step S101 of the correction process described above (for example, the correction process shown in FIGS. 8, 11, and 15), it selects a device that outputs a reference sound. The application may select, for example, the sound source generating device 200, the all-in-one device 400, or the user terminal 12 as the device that outputs the reference sound.

[0226] Furthermore, for example, when the application determines to perform calibration in step S101 of the above-described correction process (for example, the correction process shown in FIGS. 8, 11, and 15), the application selects a device that collects the reference sound. The application can select, for example, the sound collection device 300, the all-in-one device 400, or the user terminal 12 as the device that collects the reference sound.

[0227] The application performs the correction process according to the selected device.

[0228] In this way, the application can select whether or not to use the devices (for example, the sound source generating device 200, the sound collecting device 300, and the all-in-one device 400, etc.) prepared by the user U1.

[0229] The application may select these external devices in accordance with, for example, an instruction from the user U1. Alternatively, the application may select a device to output and / or collect a reference sound depending on whether an external device is connected to the user terminal 12.

[0230] It is desirable that the device used to collect the reference sound is the same device that the user U1 uses to collect vital sounds.

[0231] Of the processes described in the above embodiments, all or part of the processes described as being performed automatically can be performed manually, or all or part of the processes described as being performed manually can be performed automatically using a known method. In addition, the information including the processing procedures, specific names, various data, and parameters shown in the above documents and drawings can be changed as desired unless otherwise specified. The various information shown in each drawing is not limited to the information shown in the drawings.

[0232] The components of each device shown in the figure are conceptual functional components and do not necessarily have to be physically configured as shown. In other words, the specific form of distribution and integration of each device is not limited to that shown in the figure, and all or part of them can be functionally or physically distributed and integrated in any unit depending on various loads and usage conditions.

[0233] The above-described embodiments and modifications can be combined as appropriate within the scope of not causing any contradiction in the processing content.

[0234] The effects described in this specification are merely examples and are not limiting, and other effects may also be present.

[0235] <<9. Hardware Configuration>> Information devices such as the user terminal 12 according to each of the above-described embodiments are realized by a computer 1000 configured as shown in Fig. 18. Fig. 18 is a hardware configuration diagram showing an example of the computer 1000 that realizes the functions of the user terminal 12 according to an embodiment of the present disclosure. The computer 1000 includes a CPU 1100, a RAM 1200, a ROM 1300, a HDD (Hard Disk Drive) 1400, a communication interface 1500, and an input / output interface 1600. The components of the computer 1000 are connected by a bus 1050.

[0236] The CPU 1100 operates and controls each unit based on programs stored in the ROM 1300 or the HDD 1400. The CPU 1100 loads the programs stored in the ROM 1300 or the HDD 1400 into the RAM 1200 and executes processing corresponding to the various programs.

[0237] The ROM 1300 stores boot programs such as a Basic Input Output System (BIOS) that is executed by the CPU 1100 when the computer 1000 starts up, and programs that depend on the hardware of the computer 1000 .

[0238] HDD 1400 is a recording medium readable by computer 1000 that non-temporarily records programs executed by CPU 1100 and data used by such programs. Specifically, HDD 1400 is a recording medium that records the acoustic processing program according to the present disclosure, which is an example of program data 1450.

[0239] The communication interface 1500 is an interface for connecting the computer 1000 to an external network 1550 (such as the Internet). The CPU 1100 receives data from other devices and transmits data generated by the CPU 1100 to other devices via the communication interface 1500.

[0240] The input / output interface 1600 is an interface for connecting the input / output device 1650 and the computer 1000. The CPU 1100 receives data from input devices such as a keyboard and a mouse via the input / output interface 1600. The CPU 1100 transmits data to output devices such as a display, a speaker, and a printer via the input / output interface 1600. The input / output interface 1600 can also function as a media interface for reading a program recorded on a predetermined recording medium.

[0241] The media may be optical recording media such as DVDs (Digital Versatile Discs) and PDs (Phase Change Rewritable Discs), magneto-optical recording media such as MOs (Magneto-Optical disks), tape media, magnetic recording media, or semiconductor memories.

[0242] When the computer 1000 functions as the user terminal 12 according to the embodiment, the CPU 1100 of the computer 1000 executes an information processing program loaded onto the RAM 1200 to realize the functions of the control unit 130. The HDD 1400 stores the information processing program according to the present disclosure and data in the storage unit 120.

[0243] The CPU 1100 reads and executes the program data 1450 from the HDD 1400. However, as another example, the CPU 1100 can also obtain these programs from other devices via an external network 1550.

[0244] <<10. Supplementary Information>> The present technology may also be configured as follows. (1) An information processing method including: calculating sound collection characteristics based on sound collection data obtained by collecting a reference sound with a sound collection device and the reference sound; and correcting vital sounds collected by the sound collection device based on the calculated sound collection characteristics. (2) The information processing method according to (1), in which the vital sounds include at least one of breathing sounds, coughing sounds, bowel sounds, heartbeat sounds, snoring sounds, and speech. (3) The information processing method according to (1) or (2), in which the reference sound includes at least one of environmental sounds around the sound collection device, sweep sounds, and white noise. (4) The information processing method according to any one of (1) to (3), in which calculating the sound collection characteristics includes determining a characteristic transfer function that minimizes a difference between the sound collection data and the reference sound. (5) The information processing method according to any one of (1) to (4), wherein the sound collection data is data collected by the sound collection device mounted on the device that corrects the vital sounds. (6) The information processing method according to any one of (1) to (4), wherein the sound collection data is data collected by the sound collection device different from the device that corrects the vital sounds. (7) The information processing method according to any one of (1) to (6), wherein the reference sound is output from an acoustic device mounted on the device that corrects the vital sounds. (8) The information processing method according to any one of (1) to (6), wherein the reference sound is output from an acoustic device different from the device that corrects the vital sounds. (9) The information processing method according to (8), wherein the reference sound is output from the acoustic device based on a reference sound source stored in the device that corrects the vital sounds. (10) The information processing method according to (8), wherein the reference sound is output from the acoustic device based on a reference sound source stored in the acoustic device. (11) The information processing method according to any one of (1) to (10), wherein the calculation of the sound collection characteristics is performed before the timing at which the vital sound correction is performed for the first time or after an operating system of the device that corrects the vital sound is updated.(12) An information processing system comprising: an acoustic device that outputs a reference sound; a sound collection device that collects the reference sound; and an information processing device, wherein the information processing device comprises: a control unit that identifies sound collection characteristics based on sound collection data collected by the sound collection device and the reference sound, and corrects the vital sound collected by the sound collection device based on the calculated sound collection characteristics. (13) An information processing program that causes a computer to function as an information processing device comprising: a control unit that calculates sound collection characteristics based on sound collection data collected by a sound collection device of a reference sound and the reference sound, and corrects the vital sound collected by the sound collection device based on the calculated sound collection characteristics. (14) An information processing device comprising: a control unit that calculates sound collection characteristics based on sound collection data collected by a sound collection device of a reference sound and the reference sound, and corrects the vital sound collected by the sound collection device based on the calculated sound collection characteristics.

[0245] REFERENCE SIGNS LIST 1 Information acquisition device 2 Information processing device 3 Medical professional terminal 4 Community member terminal 5 Life insurance / health insurance terminal 6 Product / service provider terminal 7 Analysis terminal 11 Sensor 12 User terminal 13, 21, 31, 41, 51, 61, 71, 210, 310 Communication unit 14, 32, 42, 62, 73, 220, 320 User interface unit 15, 230, 330 Processing unit 16, 23, 33, 53, 120, 240, 340 Memory unit 17, 250, 350 Input / output unit 100 Information processing system 130 Control unit 151 Acquisition unit 152 Calculation unit 153 Correction unit 171, 3502 Microphone 172, 2501 Speaker 200 Sound source generating device 300 Sound collection device 400 Integrated device 2301 Playback section

Claims

1. An information processing method comprising: calculating sound collection characteristics based on sound collection data obtained by collecting a reference sound with a sound collection device and the reference sound; and correcting vital sounds collected by the sound collection device based on the calculated sound collection characteristics.

2. The information processing method according to claim 1, wherein the vital sounds include at least one of breathing sounds, coughing sounds, bowel sounds, heartbeat sounds, snoring sounds, and speech sounds.

3. The information processing method according to claim 1, wherein the reference sound includes at least one of an environmental sound around the sound collection device, a sweep sound, and white noise.

4. The information processing method according to claim 1, wherein calculating the sound collection characteristics includes determining a characteristic transfer function that minimizes the difference between the sound collection data and the reference sound.

5. The information processing method according to claim 1, wherein the collected sound data is data collected by the sound collection device mounted on the device that corrects the vital sounds.

6. The information processing method according to claim 1, wherein the collected sound data is data collected by a sound collection device different from the device that corrects the vital sounds.

7. The information processing method according to claim 1, wherein the reference sound is output from an acoustic device mounted on a device that corrects the vital sounds.

8. The information processing method according to claim 1, wherein the reference sound is output from an acoustic device different from a device that corrects the vital sound.

9. The information processing method according to claim 8, wherein the reference sound is output from the acoustic device based on a reference sound source stored in the device that corrects the vital sound.

10. The information processing method according to claim 8, wherein the reference sound is output from the acoustic device based on a reference sound source stored in the acoustic device.

11. An information processing method as described in claim 1, wherein the calculation of the sound collection characteristics is performed before the timing when the vital sound correction is performed for the first time, or after the operating system of the device correcting the vital sound is updated.

12. An information processing system comprising: an acoustic device that outputs a reference sound; a sound collection device that collects the reference sound; and an information processing device, wherein the information processing device comprises a control unit that identifies sound collection characteristics based on the sound collection data collected by the sound collection device and the reference sound, and corrects the vital sound collected by the sound collection device based on the calculated sound collection characteristics.

13. An information processing program for causing a computer to function as an information processing device having a control unit that calculates sound collection characteristics based on sound collection data obtained by collecting a reference sound with a sound collection device and the reference sound, and corrects the vital sound collected by the sound collection device based on the calculated sound collection characteristics.

Images