Program, data processing system, data processing method, and data processing terminal

Abstract

Improve the determination accuracy of the presence or absence of abnormalities in biological sounds acquired by a stethoscope. 【Solution means】The program of this embodiment causes the processor included in the data processing terminal 1 to function as a biological sound data acquisition unit 165 that acquires biological sound data based on auscultation sounds acquired by a stethoscope ST in contact with a subject, a position data acquisition unit 163 that acquires position data indicating the auscultation position where the stethoscope ST acquired the auscultation sound, and a communication processing unit 167 that transmits auscultation data in which the auscultation position indicated by the position data is associated with the biological sound data to an external device 2 in a state where it is possible to determine whether the auscultation sound is normal or abnormal.

Description

Technical Field

[0001] The present invention relates to a program, a data processing system, a data processing method, and a data processing terminal for processing biological sound data collected by a stethoscope.

Background Art

[0002] Conventionally, a technique for determining whether biological sounds acquired by a stethoscope are abnormal is known. Patent Document 1 discloses a technique for determining that there is an abnormality in biological sounds when a similarity determined by comparing data of biological sounds acquired by a stethoscope with sample data is equal to or greater than a threshold value.

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0004] Biological sounds acquired by a stethoscope vary depending on the position of the body with which the stethoscope is in contact (hereinafter sometimes referred to as the "auscultation position"). In the conventional technique, when determining whether there is an abnormality in biological sounds, comparison is made with sample data determined regardless of the auscultation position, resulting in a problem that false determination is likely to occur.

[0005] Therefore, the present invention has been made in view of these points, and an object thereof is to improve the determination accuracy of the presence or absence of an abnormality in biological sounds acquired by a stethoscope.

Means for Solving the Problems

[0006] The program according to the first aspect of the present invention causes a processor included in a data processing terminal to function as a biological sound data acquisition unit that acquires biological sound data based on auscultation sounds acquired by a stethoscope in contact with a subject, a position data acquisition unit that acquires position data indicating an auscultation position at which the stethoscope acquired the auscultation sounds, and a communication processing unit that transmits auscultation data in which the auscultation position indicated by the position data is associated with the biological sound data to an external device in a state where it is possible to determine whether the auscultation sounds are normal or abnormal.

[0007] The communication processing unit may transmit the auscultation data to the external device that creates a machine learning model for determining the presence or absence of an abnormality based on the biological sound data acquired at each of the plurality of auscultation positions.

[0008] The program may further cause the processor to function as a diagnostic data acquisition unit that acquires diagnostic data indicating a diagnostic result in association with the auscultation position, and the communication processing unit may transmit the auscultation data in which the auscultation position, the biological sound data, and the diagnostic data are associated.

[0009] The program may further cause the processor to function as a display processing unit that causes a display unit to display a body image for a user of the stethoscope to set the auscultation position, and an operation reception unit that receives an operation for the user to set the auscultation position. The position data acquisition unit may acquire the position data based on the operation received by the operation reception unit, and the communication processing unit may transmit the auscultation data in which label data indicating the auscultation position corresponding to the position data is associated with the biological sound data.

[0010] The program may further cause the processor to function as a display processing unit that causes the display unit to display a body image for the user of the stethoscope to set the auscultation position, and an operation reception unit that receives an operation for the user to set the auscultation position. The position data acquisition unit may acquire, as the position data, data indicating the reference position closest to the position on the body image set by the user among a plurality of reference positions in the body image.

[0011] The program may further cause the processor to function as a display processing unit that causes the display unit to display a body image for the user of the stethoscope to set the auscultation position and instruction text to notify the user, and an operation reception unit that receives the operation of the user. The display processing unit may determine the positional relationship between the body image and the instruction text to be displayed on the display unit in response to the operation reception unit receiving an operation for designating the hand holding the stethoscope or the hand operating the data processing terminal.

[0012] The program may further cause the processor to function as a display processing unit that causes the display unit to display timing information indicating a first timing for causing the subject to inhale and a second timing for causing the subject to exhale. The communication processing unit may transmit the first biometric sound data acquired by the biometric sound data acquisition unit during at least a part of the period from the first timing to the second timing in association with label data indicating a state of inhaling, and transmit the second biometric sound data acquired by the biometric sound data acquisition unit during at least a part of the period from the second timing to the first timing in association with label data indicating a state of exhaling.

[0013] The program may further cause the processor to function as a state detection unit that detects the state of the subject's body. The communication processing unit may transmit the biometric sound data during the period when the state detection unit detects the state in association with label data indicating the state detected by the state detection unit.

[0014] The state detection unit may detect the state of the body by detecting the voice uttered by the user of the stethoscope.

[0015] The state detection unit may cause the display unit to display one or more candidates for the state of the body, and detect, as the state of the body, the state corresponding to the candidate selected by the user of the stethoscope from the one or more candidates.

[0016] The program may further cause the processor to function as an operation reception unit that receives an operation for the user of the stethoscope to select a diagnostic mode to be used among a plurality of diagnostic modes or an attribute of the subject, and the state detection unit may cause the display unit to display the one or more candidates corresponding to the diagnostic mode or the attribute corresponding to the operation received by the operation reception unit.

[0017] The state detection unit may analyze the biological sound data and cause the display unit to display the one or more candidates corresponding to the analysis result.

[0018] The program may further cause the processor to function as an operation reception unit that receives an operation for switching the operation mode of the communication processing unit to either a first mode or a second mode, and the communication processing unit transmits the auscultation data when the operation reception unit receives an operation for setting the first mode, and transmits the auscultation data to obtain a determination result from the external device that determines the presence or absence of an abnormality based on the biological sound data when the operation reception unit receives an operation for setting the second mode.

[0019] The program may further cause the processor to function as a display processing unit that causes the display unit to display information indicating that a consideration has occurred for the user of the stethoscope when the communication processing unit transmits the auscultation data in the first mode, and causes the display unit to display information indicating that a charge has occurred for the user when the communication processing unit transmits the auscultation data in the second mode.

[0020] The data processing system according to the second aspect of the present invention includes a data processing device and a data processing terminal. The data processing terminal includes a biological sound data acquisition unit that acquires biological sound data based on the auscultation sound acquired by a stethoscope that has come into contact with a subject, a position data acquisition unit that acquires position data indicating the auscultation position where the stethoscope has acquired the auscultation sound, and a communication processing unit that transmits auscultation data in which the auscultation position indicated by the position data is associated with the biological sound data to the data processing device in a state where it is possible to determine whether the auscultation sound is normal or abnormal. The data processing device includes a data reception unit that receives the auscultation data, and a learning unit that trains a machine learning model using, as teacher data indicating abnormality or teacher data indicating normality, the biological sound data in association with the auscultation position.

[0021] The data processing method according to the third aspect of the present invention includes steps executed by a computer: a step of acquiring biological sound data based on the auscultation sound acquired by a stethoscope that has come into contact with a subject; a step of acquiring position data indicating the auscultation position where the stethoscope has acquired the auscultation sound; and a step of transmitting auscultation data in which the auscultation position indicated by the position data is associated with the biological sound data to an external device in a state where it is possible to determine whether the auscultation sound is normal or abnormal.

[0022] The data processing terminal according to the fourth aspect of the present invention includes a biological sound data acquisition unit that acquires biological sound data based on the auscultation sound acquired by a stethoscope that has come into contact with a subject, a position data acquisition unit that acquires position data indicating the auscultation position where the stethoscope has acquired the auscultation sound, and a communication processing unit that transmits auscultation data in which the auscultation position indicated by the position data is associated with the biological sound data to an external device in a state where it is possible to determine whether the auscultation sound is normal or abnormal.

Advantages of the Invention

[0023] According to the present invention, there is an effect that the determination accuracy of the presence or absence of abnormality in the biological sound acquired by the stethoscope can be improved.

Brief Description of the Drawings

[0024]

Figure 1

Figure 2

Figure 3

Figure 4

Figure 5

Figure 6

Figure 7

Figure 8

Figure 9

Figure 10

Figure 11

Figure 12

Figure 13

Figure 14

Embodiments for Carrying Out the Invention

[0025] [Overview of Information Processing System S] Using FIG. 1, the overview of the information processing system S according to the present embodiment will be described. FIG. 1 is a diagram showing an overview of the operation of the information processing system S. The information processing system S includes a data processing terminal 1 and an external device 2. The information processing system S may include other devices such as servers and terminals.

[0026] The information processing system S is a system for learning the relationship between a biological sound (hereinafter sometimes referred to as "biological sound") obtained by a stethoscope that has come into contact with a subject, the stethoscope position, and the diagnosis result. Further, the information processing system S is a system for determining a diagnosis result based on a biological sound and a stethoscope position using the learning result.

[0027] The data processing terminal 1 is a terminal used by the user U of the stethoscope. The user U is, for example, a doctor who examines a subject, but may also be a family member of the subject or the subject himself / herself. The data processing terminal 1 is preferably a portable terminal such as a smartphone or a tablet personal computer, but may also be a stationary terminal such as a desktop personal computer.

[0028] The external device 2 is a computer such as a server that performs the above-described learning. The data processing terminal 1 and the external device 2 can communicate with each other. It is assumed that the external device 2 is provided at a location different from the location where the user U uses the data processing terminal 1 (for example, in the cloud), but it may also be provided at the location where the user U uses the data processing terminal 1.

[0029] In recent years, due to changes in people's lifestyles (intake of high-calorie diets, lack of exercise, excessive alcohol intake), aging, and the spread of the novel coronavirus, diseases represented by heart diseases and lung diseases have been increasing. Whether a subject has a disease can often be determined based on the biological sound of the subject obtained from a stethoscope. However, conventionally, there has been a problem that only a specialist in the disease that the subject has can make this determination. In addition, there has also been a problem that even if a doctor other than a specialist or a non-doctor (such as a family member of the subject) determines the presence or absence of a disease in a subject based on a biological sound using a conventional device, misjudgment is likely to occur.

[0030] Therefore, in the first mode, which is the diagnosis mode used by a specialist as the user U, the information processing system S generates data for learning the relationship between the biological sound, the auscultation position, and the diagnosis result by the specialist to create a machine learning model. Further, in the second mode, which is the diagnosis mode used by a doctor other than the specialist or a person who is not a doctor (hereinafter sometimes referred to as "a person other than the specialist"), the information processing system S uses the machine learning model created using the data generated in the first mode to provide a determination result based on the biological sound and the auscultation position. By using the above machine learning model in the second mode, even a person other than the specialist can accurately determine the presence or absence of a disease in the subject. Hereinafter, the outlines of the first mode and the second mode will be described.

[0031] First, the case where the data processing terminal 1 is used by a specialist and the specialist selects the first mode in the data processing terminal 1 will be described. The data processing terminal 1 receives an input of the auscultation position from the specialist. The data processing terminal 1 acquires biological sound data based on the auscultation sound acquired by the stethoscope in contact with the input auscultation position. While auscultating, the specialist inputs the disease name as the diagnosis result to the data processing terminal 1 if there is an abnormality in the auscultation sound and determines that there is an abnormality.

[0032] Next, as shown in FIG. 1, the data processing terminal 1 transmits auscultation data in which the auscultation position, the biological sound data, and the diagnosis result are associated to the external device 2. Then, the external device 2 that has received the auscultation data learns the machine learning model using the biological sound data as teacher data indicating that there is an abnormality or teacher data indicating that it is normal. Although one specialist is shown in FIG. 1, in order to create a more accurate machine learning model, it is preferable to collect auscultation data from more specialists. Therefore, the external device 2 may collect auscultation data from the data processing terminals 1 used by a number of specialists.

[0033] Incidentally, in order for the external device 2 to collect a large number of auscultation data, it is necessary to have a specialist perform an operation to transmit the auscultation data. Therefore, in order to motivate the specialist to transmit the auscultation data, the specialist who has transmitted the auscultation data may be given a consideration. Specifically, as shown in FIG. 1, the external device 2 may transmit consideration information indicating that consideration has occurred to the data processing terminal 1 of the specialist who has transmitted the auscultation data.

[0034] Note that the data processing terminal 1 used by the specialist may transmit auscultation data in which the auscultation position indicated by the position data is associated with the bioacoustic data to the external device 2. Then, the external device 2 that has received the auscultation data may use the machine learning model created in the first mode to transmit a determination result that determines the presence or absence of an abnormality in the auscultation sound and, if there is an abnormality, the disease name, to the data processing terminal 1 used by the specialist. Thereby, the specialist can use the received determination result as reference information when making a final diagnosis result.

[0035] When the data processing terminal 1 used by the specialist receives the determination result from the external device 2, it may also receive charge information indicating that a charge has occurred. Thereby, for the specialist, the consideration for transmitting the auscultation data in which the auscultation position, the bioacoustic data, and the diagnosis result are associated to the external device 2, and the charge for transmitting the auscultation data in which the auscultation position and the bioacoustic data are associated to the external device 2 and receiving the determination result from the external device 2 will occur simultaneously. For this reason, the specialist can use the determination result by the external device 2 at an amount obtained by subtracting the consideration amount from the charge amount (free of charge if the charge amount and the consideration amount are equal).

[0036] Next, the data processing terminal 1 is used by a person other than a specialist doctor, and the case where a person other than a specialist doctor selects the second mode in the data processing terminal 1 will be described. First, the data processing terminal 1 acquires position data indicating the auscultation position and biological sound data in the same manner as in the method of the first mode. Next, as shown in FIG. 1, the data processing terminal 1 transmits auscultation data in which the auscultation position indicated by the position data is associated with the biological sound data to the external device 2. Then, the external device 2 that has received the auscultation data uses the machine learning model created in the first mode to determine the presence or absence of an abnormality in the auscultation sound and, if there is an abnormality, the disease name. The external device 2 transmits the determined result to the data processing terminal 1 as shown in FIG. 1. At this time, the external device 2 may transmit charging information indicating that charging has occurred to the data processing terminal 1 together with the determination result.

[0037] In this way, the information processing system S can learn the relationship between the auscultation position, the biological sound data, and the diagnosis result by a specialist doctor auscultating the subject, and create a machine learning model. Since teacher data for learning the machine learning model is created by a specialist doctor diagnosing the subject, the machine learning model can be learned in a short period of time.

[0038] In addition, the information processing system S can use the created machine learning model to provide a determination result on the presence or absence of an abnormality in the auscultation sound and the disease name based on the learning result. As a result, even a person other than a specialist doctor can accurately determine the presence or absence of a disease and the disease name of the subject if the auscultation position and biological sound data are acquired. Hereinafter, the configurations and operations of the stethoscope, the data processing terminal 1, and the external device 2 will be described in detail.

[0039] [Configuration and Operation of Stethoscope ST] FIG. 2 is a diagram showing the configuration of the stethoscope ST. The stethoscope ST includes a data processing terminal 1, a chest piece C, earphones E, and a tube T. The data processing terminal 1 can be connected to the tube T and acquires an auscultation sound through the tube T. The tube T may be provided with an AD converter that converts the auscultation sound into digital data.

[0040] The chest piece C is the part that contacts the subject. Contact includes both directly contacting the subject's body by contacting the subject's skin, and indirectly contacting the subject's body by contacting the subject's clothing or undergarments. The chest piece C picks up auscultation sounds from the subject's body.

[0041] The earphone E is a device used by a medical professional or a non-medical professional to listen to auscultation sounds. The earphone E can be connected to the tube T. The tube T is a device for connecting the data processing terminal 1, the chest piece C, and the earphone E.

[0042] The auscultation sounds picked up by the chest piece C are transmitted through the tube T to the data processing terminal 1 and the earphone E. When the auscultation sound, which is an analog signal, is transmitted to the data processing terminal 1, it is converted into biological sound data, which is a digital signal. The biological sound data transmitted to the data processing terminal 1 may be stored in the memory or disk within the data processing terminal 1, or may be stored in the cloud outside the data processing terminal 1.

[0043] [Configuration and Operation of Data Processing Terminal 1] Figure 3 is a diagram showing the configuration of the data processing terminal 1. The data processing terminal 1 includes an operation unit 11, a terminal communication unit 12, a display 13, a microphone 14, a storage unit 15, and a control unit 16.

[0044] The operation unit 11 is a device that receives the operations of the user U of the stethoscope ST, and is, for example, a touch panel. The terminal communication unit 12 is a communication interface for communicating with an external device 2 via a communication network such as the Internet.

[0045] The display 13 is a display unit for displaying information. The microphone 14 is a device for inputting the voice of the user U of the stethoscope ST.

[0046] The storage unit 15 is a storage medium including a ROM (Read Only Memory), a RAM (Random Access Memory), and the like. The storage unit 15 stores programs executed by the control unit 16. For example, the storage unit 15 stores an information processing program that causes the control unit 16 to function as an operation reception unit 161, a display processing unit 162, a position data acquisition unit 163, a state detection unit 164, a biological sound data acquisition unit 165, a diagnosis data acquisition unit 166, and a communication processing unit 167. The storage unit 15 stores auscultation data.

[0047] FIG. 4 is a diagram showing an example of auscultation data. In the auscultation data shown in FIG. 4, biological sound data, label data, and diagnosis data are associated with each other. The biological sound data is data obtained by converting the auscultation sound of the analog signal acquired by the stethoscope ST into a digital signal.

[0048] The label data includes data indicating at least any one of the auscultation position, the state of breathing, and the state of the body. The auscultation position is the position of the body where the stethoscope ST has contacted as described above. The state of breathing is either the state where the subject is inhaling or exhaling. The state of the body is an auscultation state (for example, the state of the subject's breathing) that is the state of the subject during auscultation or an auscultation result (for example, the name or characteristics of the auscultation sound that can be heard when the subject has a disease) that is the result of auscultating the subject.

[0049] The diagnosis data is data indicating the diagnosis result by a specialist regarding the subject. The diagnosis result includes the presence or absence of an abnormality and the disease name. In the auscultation data shown in FIG. 4, the diagnosis result is recorded for each auscultation position.

[0050] FIG. 5 is a diagram showing another example of auscultation data. Also in the auscultation data shown in FIG. 5, biological sound data, label data, and diagnosis data are associated with each other, but the diagnosis result includes the presence or absence of an abnormality for each auscultation position, the presence or absence of an abnormality as a comprehensive judgment, and the disease name as a comprehensive judgment. That is, in the auscultation data shown in FIG. 5, the presence or absence of an abnormality and the disease name as a result of a comprehensive judgment by a specialist based on the presence or absence of an abnormality at each auscultation position are recorded.

[0051] The control unit 16 is, for example, a CPU (Central Processing Unit). By executing the information processing program stored in the storage unit 15, the control unit 16 functions as an operation reception unit 161, a display processing unit 162, a position data acquisition unit 163, a state detection unit 164, a biological sound data acquisition unit 165, a diagnostic data acquisition unit 166, and a communication processing unit 167.

[0052] The operation reception unit 161 receives various operations by the user U. The operation reception unit 161 receives, for example, an operation for switching the operation mode of the communication processing unit 167 to either the first mode or the second mode. The operation reception unit 161 receives, for example, an operation in which the user U selects a diagnostic mode to be used from among a plurality of diagnostic modes.

[0053] FIG. 6 is a diagram showing an example of a mode selection screen. On the mode selection screen, a specific mode can be selected as the operation mode and the diagnostic mode. When the user U, who is a specialist, performs an operation of selecting "specialist mode" as the operation mode, the operation reception unit 161 receives the operation as an operation for switching the operation mode to the first mode. On the other hand, when the user U, who is a non-specialist, performs an operation of selecting "non-specialist mode" as the operation mode, the operation reception unit 161 receives the operation as an operation for switching the operation mode to the second mode. The content displayed when the user U performs an operation of selecting each mode of the diagnostic mode will be described in the description section of the display processing unit 162.

[0054] The operation reception unit 161 may receive an operation in which the user U of the stethoscope ST selects the attributes of the subject. The operation reception unit 161 may receive, for example, an operation in which the user U inputs the attributes (such as gender, age, medical history, or smoking history) of the subject from the display 13.

[0055] The operation reception unit 161 may receive an operation in which the user U sets the auscultation position. For example, the operation reception unit 161 may receive an operation in which the user U taps the auscultation position on the body image displayed on the auscultation screen described later.

[0056] Unless otherwise specified, the following description of the processes executed by each unit other than the operation reception unit 161 in the control unit 16 is the description of the process when the first mode is selected. That is, the user U of the stethoscope ST is assumed to be a medical professional.

[0057] The display processing unit 162 causes various types of information to be displayed on the display 13 which is the display unit. For example, the display processing unit 162 causes a body image for the user U to set the auscultation position to be displayed on the display 13. For example, for each of the front, side, and back of the body, the display processing unit 162 causes a body image on which the user U can select the auscultation position to be displayed on the display 13.

[0058] When the user U who is a medical professional selects the "free diagnosis mode" as the diagnosis mode, the display processing unit 162 may display a body image in which candidates for the auscultation position are not shown so that the user U can freely select the auscultation position. Also, when the user U selects the "health check mode" as the diagnosis mode, the display processing unit 162 may display a body image in which the auscultation positions defined as health check items are shown. Further, when the user U selects a mode for a specific disease other than these as the diagnosis mode, the display processing unit 162 may display a body image in which the auscultation positions corresponding to the selected disease name are shown.

[0059] The display processing unit 162 may display on the display 13 a body image for the user U of the stethoscope ST to set the auscultation position and instruction text to notify the user U. The display processing unit 162 may display on the display 13, for example, text instructing the user U to place the stethoscope ST at a specific position on the body of the subject, text instructing the user U to confirm whether the auscultation sound can be heard when the user U places the stethoscope ST on the subject, text instructing the user U to press the recording button if the auscultation sound can be heard, and any one of the text instructing the user U to keep the stethoscope ST in contact with the auscultation position until the recording ends when the user U starts recording. Thereby, the user U can clearly understand the operations to be performed on the data processing terminal 1 by himself / herself.

[0060] FIG. 7 is a diagram showing an example of a front auscultation screen displayed when auscultating the front of the subject's body. On the body image on the left side of the screen, the display 13 displays "front chest" as a candidate for the auscultation position. Also, on the instruction text on the right side of the screen, the display 13 displays text instructing the user U to place the stethoscope ST on the front chest and to press the recording button if the auscultation sound can be heard.

[0061] FIG. 8 is a diagram showing an example of a back auscultation screen displayed when auscultating the back of the subject's body. On the body image on the left side of the screen, the display 13 displays "center of the back chest" as a candidate for the auscultation position. Also, on the instruction text on the right side of the screen, the display 13 displays text instructing the user U to keep the stethoscope ST in contact and wait until the recording ends because the auscultation sound of the center of the back chest is being recorded.

[0062] Incidentally, for example, when user U holds the data processing terminal 1 with the right hand and operates the data processing terminal 1 with the thumb of the right hand, if a body image is displayed on the left side as in the auscultation screens shown in FIGS. 7 and 8, user U may not be able to tap the body image with the thumb of the right hand. Therefore, in response to the operation reception unit 161 receiving an operation that designates the hand holding the stethoscope ST or the hand operating the data processing terminal 1, the display processing unit 162 determines the positional relationship between the body image and the instruction text to be displayed on the display unit 13 (display 13).

[0063] For example, assuming that the operation reception unit 161 receives an operation that designates the hand holding the chest piece C of the stethoscope ST, the display processing unit 162 displays the body image at a position on the side opposite to the hand holding the chest piece C and displays the instruction text at a position on the same side as the hand holding the chest piece C on the auscultation screen. Also, for example, assuming that the operation reception unit 161 receives an operation that designates the hand operating the data processing terminal 1, the display processing unit 162 displays the body image at a position on the same side as the hand operating the data processing terminal 1 and displays the instruction text at a position on the side opposite to the hand operating the data processing terminal 1 on the auscultation screen.

[0064] Specifically, since user U is right-handed, when the right hand is designated as the hand holding the chest piece C of the stethoscope ST and the left hand is designated as the hand operating the data processing terminal 1, the display processing unit 162 displays the body image on the left side and the instruction text on the right side as shown in the auscultation screens of FIGS. 7 and 8. By displaying in this way, right-handed user U can hold the stethoscope ST with the right hand, which is the dominant hand, hold the data processing terminal 1 with the left hand, and select the body image displayed on the left side with the thumb of the left hand.

[0065] On the other hand, when the left hand is designated as the hand holding the chest piece C of the stethoscope ST because the user U is left-handed and the right hand is designated as the hand operating the data processing terminal 1, the display processing unit 162 displays the body image on the right side and the instruction text on the left side as shown in the auscultation screen of FIG. 9. FIG. 9 is a diagram showing a screen in which the display positions of the body image and the instruction text in the front auscultation screen of FIG. 7 are swapped.

[0066] In this way, by changing the display positions of the body image and the instruction text on the auscultation screen according to whether the user U holds the chest piece C of the stethoscope ST or operates the data processing terminal 1, the display processing unit 162 makes the auscultation screen easier to operate for the user U. As a result, the user U can perform the auscultation work more accurately and quickly.

[0067] The position data acquisition unit 163 acquires position data indicating the auscultation position where the stethoscope ST has acquired an auscultation sound. The position data acquisition unit 163 acquires position data based on, for example, the operation received by the operation reception unit 161. Specifically, the position data acquisition unit 163 acquires position data indicating the position set by the user U in the body image displayed on the auscultation screen as the auscultation position. The user U can set the position, for example, by tapping the body image displayed on the auscultation screen. The user U performs this tap, for example, before recording the auscultation sound by applying the stethoscope ST to the subject.

[0068] Incidentally, the position tapped by the user U in the body image displayed on the auscultation screen may deviate from the reference position for auscultation due to hand tremors or the like of the user U. In this case, it is preferable that the position obtained by correcting the tap position to the reference position is acquired as the auscultation position. Therefore, the position data acquisition unit 163 acquires, as position data, data indicating the reference position closest to the position on the body image set by the user U among a plurality of reference positions in the body image.

[0069] Figure 10 is a diagram showing a plurality of reference positions in a body image. In Figure 10, eight points labeled A through H are the reference positions. A reference range indicated by a dotted line is set around the reference positions. The reference ranges surrounding A and B are the apical lung fields, the reference ranges surrounding C and D are the upper lung fields, the reference ranges surrounding E and F are the middle lung fields, and the reference ranges surrounding G and H are the lower lung fields. Assuming that the position indicated by the star mark is the position tapped by the user U, the position data acquisition unit 163 acquires position data indicating the position of C, which is the reference position corresponding to the reference range in which the position tapped by the user U is included, as the auscultation position.

[0070] In this way, by the position data acquisition unit 163 acquiring data indicating the reference position as the position data, even when the user U accidentally taps a position deviated from the position where auscultation should originally be performed, the position data acquisition unit 163 can acquire position data indicating the correct auscultation position.

[0071] If the user U taps a position that is too far from the reference position, there is a possibility that the user U has mistaken the auscultation position in the first place. In order for the user U to recognize this mistake, the position data acquisition unit 163 may display an error on the auscultation screen when the position tapped by the user U is separated from the reference position by a predetermined distance or more. For example, in Figure 10, if the position tapped by the user U is not included in any of the reference ranges including the reference positions A through H, the position data acquisition unit 163 may display an error on the auscultation screen.

[0072] The state detection unit 164 detects the state of the subject's body. The state detection unit 164 detects, for example, the auscultation state, which is the state of the subject during auscultation, as the state of the subject's body.

[0073] The state detection unit 164 detects the physical state of, for example, the user U of the stethoscope ST by detecting the voice uttered by the user U. The state detection unit 164 detects the physical state of the subject by detecting, for example, the voice about the auscultation state (e.g., the state of the subject's breath) uttered by the medical professional who is the user U of the stethoscope ST, through the microphone 14.

[0074] Specifically, when the state detection unit 164 detects the voice (such as the voice of "Inhale--") instructing the subject to inhale uttered by the medical professional who is the user U, it detects the physical state that the subject is inhaling. On the other hand, when the state detection unit 164 detects the voice (such as the voice of "Exhale--") instructing the subject to exhale uttered by the medical professional who is the user U, it detects the physical state that the subject is exhaling.

[0075] The state detection unit 164 may display one or more candidates of the physical state on the display 13, and detect the state corresponding to the candidate selected by the user of the stethoscope ST from the one or more candidates as the physical state. For example, as one or more candidates of the physical state, the state detection unit 164 displays, via the display processing unit 162, the name or characteristics of the auscultation sound that can be heard when the subject has a disease on the display 13, and detects the state corresponding to the name or characteristics of the auscultation sound selected by the user U of the stethoscope ST as the physical state of the subject.

[0076] Specific examples of the auscultation sound that can be heard when the subject has a disease will be described. As heart sounds, examples include heart murmur (when there is blood backflow to the heart), atrial fibrillation (when there is arrhythmia), and gallop sound (when there is an extra heart sound), etc. As breath sounds, examples include abnormalities in the loudness of the auscultation sound, left-right differences in the auscultation sound, and abnormalities in the location where the auscultation sound can be heard (e.g., the auscultation sound that should be heard from the bronchus is heard from the lung). As complex breath sounds, examples include intermittent rattling sounds, continuous rattling sounds, and differences due to body position or coughing. Also, as the auscultation sound that can be heard when the subject has interstitial pneumonia, examples include crepitus and squawk sounds.

[0077] In this way, by having the state detection unit 164 display one or more candidates for the state of the subject's body on the display 13, the user U of the stethoscope ST only needs to select the displayed candidates, eliminating the need to input the state of the subject's body one by one. As a result, when the user U feels something abnormal while listening to the auscultation sound, the user U can easily record the state of the subject's body.

[0078] By the way, when a large number of candidates for the state of the body are displayed on the display 13, there is a possibility that it will take time for the user U to search for the candidates and a possibility that the user U will select the wrong candidate. Therefore, the state detection unit 164 causes the display 13 to display one or more candidates corresponding to the diagnostic mode or attribute corresponding to the operation received by the operation reception unit 161.

[0079] The state detection unit 164, for example, refers to data in which one or more candidates for the state of the body are associated for each diagnostic mode or each attribute, identifies one or more candidates for the state of the body associated with the diagnostic mode or attribute received by the operation reception unit 161, and causes the identified candidates to be displayed on the auscultation screen via the display processing unit 162. The state detection unit 164 may cause one or more candidates for the state of the body identified based on the attribute received by the operation reception unit 161 among one or more candidates for the state of the body identified based on the diagnostic mode received by the operation reception unit 161 to be displayed on the auscultation screen. Thereby, the state detection unit 164 can further narrow down the candidates to be displayed on the auscultation screen.

[0080] FIG. 11 is a diagram showing an example of an auscultation screen on which one or more candidates for the state of the body are displayed. For example, assume that the operation reception unit 161 receives an operation of user U who selects the "interstitial pneumonia mode" from a plurality of diagnostic modes displayed on the mode selection screen (FIG. 6). In this case, the state detection unit 164 causes the auscultation screen to display, as the names of auscultation sounds corresponding to interstitial pneumonia, crackles, bubbling sounds, wheezing sounds, snoring sounds, strider sounds, and squawk sounds, as shown enclosed by a dotted line frame in the auscultation screen of FIG. 11. When user U presses a button corresponding to the auscultation sound heard while the auscultation sound is being heard, the state detection unit 164 causes the storage unit 15 to store the biological sound data in association with the type of sound corresponding to the pressed button.

[0081] In this way, by causing the state detection unit 164 to display one or more candidates for the state of the body according to the diagnostic mode or the attributes of the subject on the auscultation screen, user U can select a candidate corresponding to the auscultation sound of the subject from a limited number of candidates rather than a large number of candidates. As a result, user U can quickly find the corresponding candidate, and the probability of selecting an incorrect candidate also decreases.

[0082] Incidentally, one or more candidates for the state of the body according to the aforementioned diagnostic mode or the attributes of the subject may be different from the current state of the subject. Therefore, the state detection unit 164 may analyze the biological sound data and cause the display 13 to display one or more candidates corresponding to the analysis result.

[0083] The state detection unit 164 identifies the characteristics of the biological sound data, for example, by analyzing the biological sound data. The state detection unit 164 refers to data in which the characteristics of the biological sound data are associated with one or more candidates for the state of the body, for example, and identifies one or more candidates for the state of the body associated with the characteristics of the acquired biological sound data, and causes the display 13 to display the identified candidates via the display processing unit 162.

[0084] In this way, by causing the state detection unit 164 to display one or more candidates for the physical state according to the result of analyzing the biological sound data of the subject on the auscultation screen, the user U can select the physical state of the subject from among appropriate options according to the current state of each subject.

[0085] In addition, when the user U selects the recording button, the state detection unit 164 may determine whether the auscultation sound (for example, heartbeat) can actually be obtained from the stethoscope ST by comparing the model of the auscultation sound stored in the storage unit 15 in advance with the biological sound data. When the state detection unit 164 determines that the auscultation sound cannot be obtained, it may prevent the user U from pressing the recording button by displaying an error on the display 13 via the display processing unit 162. Thereby, it is possible to prevent data based on sounds other than the auscultation sound or no sound from being recorded as biological sound data. Further, when the state detection unit 164 detects that the stethoscope ST is placed at a position different from the instructed position by comparing the model of the auscultation sound stored in the storage unit 15 in association with the auscultation position with the biological sound data, it may prevent the user U from pressing the recording button by displaying an error on the display 13. Thereby, it is possible to prevent biological sound data based on the auscultation sound at the wrong position from being obtained.

[0086] The biological sound data acquisition unit 165 acquires biological sound data based on the auscultation sound acquired by the stethoscope ST in contact with the body of the subject. The biological sound data acquisition unit 165 acquires, as biological sound data, data obtained by converting the auscultation sound of the analog signal transmitted through the tube T from the chest piece C of the stethoscope ST in contact with the body of the subject into a digital signal.

[0087] The biological sound data acquisition unit 165 associates the biological sound data with at least one of the auscultation position indicated by the position data acquired by the position data acquisition unit 163 and the physical state detected by the state detection unit 164, and stores it in the storage unit 15 as auscultation data.

[0088] Note that the process of the position data acquisition unit 163 acquiring position data and the process of the biological sound data acquisition unit 165 acquiring biological sound data may be performed in either order, or both may be performed simultaneously.

[0089] The diagnosis data acquisition unit 166 acquires diagnosis data indicating a diagnosis result in association with the auscultation position. For example, after the acquisition of the biological sound by the biological sound data acquisition unit 165 and the acquisition of the position data by the position data acquisition unit 163 are completed, the diagnosis data acquisition unit 166 displays a diagnosis result input screen for allowing the user U to input a diagnosis result. For example, when the operation reception unit 161 receives an operation in which the user U inputs a diagnosis result on the diagnosis result input screen, the diagnosis data acquisition unit 166 acquires diagnosis data indicating the received diagnosis result.

[0090] Note that the user U may input, as a diagnosis result, the presence or absence of an abnormality and the disease name for each auscultation position from the diagnosis result input screen. In this case, the format of the diagnosis data is as included in the auscultation data shown in FIG. 4. Alternatively, the user U may input, as a diagnosis result, the presence or absence of an abnormality for each auscultation position and the presence or absence of an abnormality and the disease name as a result of a comprehensive determination based on the presence or absence of an abnormality at each auscultation position from the diagnosis result input screen. In this case, the format of the diagnosis data is as included in the auscultation data shown in FIG. 5.

[0091] The communication processing unit 167 transmits auscultation data in which the auscultation position indicated by the position data is associated with the biometric sound data to the external device 2 in a state where it is possible to determine whether the auscultation sound is normal or abnormal. The external device 2 is, for example, a computer that creates a machine learning model for determining the presence or absence of an abnormality based on the biometric sound data acquired at each of a plurality of auscultation positions. The communication processing unit 167 transmits, for example, auscultation data in which label data indicating the auscultation position corresponding to the position data is associated with the biometric sound data to the external device 2. The label data may be anything that can identify the auscultation position, and is, for example, a name, symbol, or mark indicating the auscultation position. The communication processing unit 167 transmits, for example, auscultation data in which label data indicating the auscultation position and the biometric sound data are associated, as shown in the auscultation data of FIGS. 4 and 5, to the external device 2.

[0092] The communication processing unit 167 transmits auscultation data that does not include information indicating the presence or absence of an abnormality to the external device 2 only when the diagnostic data indicates that it is normal, so that the external device 2 can recognize that the received auscultation data indicates a normal auscultation sound. The communication processing unit 167 may transmit auscultation data that does not include information indicating the presence or absence of an abnormality to the external device 2 only when the diagnostic data indicates that it is abnormal, so that the external device 2 can recognize that the received auscultation data indicates an abnormal auscultation sound.

[0093] The communication processing unit 167 may transmit auscultation data in which the auscultation position, the biometric sound data, and the diagnostic data are associated. The communication processing unit 167 generates, for example, auscultation data by associating the auscultation position indicated by the position data acquired by the position data acquisition unit 163, the biometric sound data acquired by the biometric sound data acquisition unit 165 corresponding to the auscultation position, and diagnostic data indicating the presence or absence of an abnormality in the auscultation sound corresponding to the combination of the auscultation position and the biometric sound data, and transmits the generated auscultation data to the external device 2 via the terminal communication unit 12. Note that the communication processing unit 167 may transmit the auscultation data to the external device 2 when the operation reception unit 161 receives an operation for setting the first mode.

[0094] In this way, by the communication processing unit 167 transmitting the auscultation data in which the auscultation position and the biological sound data are associated to the external device 2 in a state where it is possible to determine whether the auscultation sound is normal or abnormal, the external device 2 can learn the relationship among the auscultation position, the biological sound data, and the presence or absence of abnormality of the auscultation sound. As a result, even a person other than a specialist can accurately determine the presence or absence of a disease of the subject if the auscultation position and the biological sound data are acquired.

[0095] The communication processing unit 167 may transmit the first biological sound data acquired by the biological sound data acquisition unit 165 during at least a part of the period from the first timing to the second timing in association with label data indicating a state of inhaling breath. Further, the communication processing unit 167 may transmit the second biological sound data acquired by the biological sound data acquisition unit 165 during at least a part of the period from the second timing to the first timing in association with label data indicating a state of exhaling breath. The label data may be anything that can identify the state of the subject's breath, and for example, it is a name, a symbol, a mark, etc. indicating the state of breath.

[0096] In order for the communication processing unit 167 to be able to transmit the biological sound data in association with such label data, the display processing unit 162 causes the display unit 13 (display 13) to display timing information indicating the first timing for causing the subject to inhale breath and the second timing for causing the subject to exhale breath. The display processing unit 162, for example, displays, on the auscultation screen, text instructing the user U to inhale breath at the first timing for causing the subject to inhale breath, and displays, on the auscultation screen, text instructing the user U to exhale breath at the second timing for causing the subject to exhale breath.

[0097] The communication processing unit 167 transmits, for example, the first biometric sound data acquired by the biometric sound data acquisition unit 165 during at least a partial period from when the display processing unit 162 displays on the auscultation screen the text for instructing the user U to inhale to when the display processing unit 162 displays on the auscultation screen the text for instructing the user U to exhale, in association with the label data indicating that the subject is in the state of inhaling. Further, the communication processing unit 167 transmits, for example, the second biometric sound data acquired by the biometric sound data acquisition unit 165 during at least a partial period from when the display processing unit 162 displays on the auscultation screen the text for instructing the user U to exhale to when the display processing unit 162 displays on the auscultation screen the text for instructing the user U to inhale, in association with the label data indicating that the subject is in the state of exhaling.

[0098] The communication processing unit 167 may transmit the biometric sound data during the period when the state detection unit 164 detects the state, in association with the label data indicating the state detected by the state detection unit 164. The label data may be anything that can identify the state of the subject's body, and is, for example, the name, symbol, and mark indicating the state of the body. As described above, the state of the subject's body is the auscultation state (for example, the state of the subject's breath) or the auscultation result (for example, the name or characteristics of the auscultation sound heard when the subject has a disease). The communication processing unit 167 transmits, for example, the auscultation data in which the biometric sound data, the auscultation position, the state of the breath, the state of the body, and the diagnostic data are associated to the external device 2, as shown in the auscultation data of FIGS. 4 and 5.

[0099] In this way, by the communication processing unit 167 transmitting the biometric sound data to the external device 2 in association with the label data indicating at least any one of the auscultation position, the state of the subject's breath, and the state of the body, the external device 2 can create a machine learning model capable of determining the abnormality of the auscultation sound for each of at least any one of the auscultation position, the state of the breath, and the state of the body.

[0100] In the explanations so far, the processing when the first mode used by a specialist is selected has been described. Here, the processing when the second mode used by a person other than a specialist is selected will be described.

[0101] The position data acquisition unit 163 and the biological sound data acquisition unit 165 acquire position data and biological sound data, respectively, in the same manner as the processing when the first mode is selected.

[0102] When the operation reception unit 161 receives an operation to switch to the second mode, the communication processing unit 167 transmits auscultation data in which the auscultation position indicated by the position data is associated with the biological sound data to the external device 2 in order to obtain a determination result from the external device 2 that determines the presence or absence of an abnormality based on the biological sound data. For example, the communication processing unit 167 generates auscultation data by associating the auscultation position indicated by the position data acquired by the position data acquisition unit 163 with the biological sound data acquired by the biological sound data acquisition unit 165 corresponding to the auscultation position, and transmits the generated auscultation data to the external device 2 via the terminal communication unit 12.

[0103] In this way, by the communication processing unit 167 transmitting the auscultation data in which the auscultation position and the biological sound data are associated to the external device 2, the external device 2 can determine the presence or absence of an abnormality in the auscultation sound by inputting the auscultation position and the biological sound data into the machine learning model created in the first mode. The external device 2 transmits the determination result to the data processing terminal 1, and the terminal communication unit 12 receives the determination result.

[0104] FIG. 12 is a diagram showing an example of a risk determination result screen indicating the determination results of the risks of various diseases. As shown in FIG. 12, the display processing unit 162 displays on the risk determination result screen a bar indicating the degree of risk for each disease and characters (high, medium, low) indicating the degree of risk for each disease. By displaying such a determination result on the display 13, even a person other than a specialist can accurately determine the presence or absence of a disease in the subject.

[0105] By the way, in order to create a more accurate machine learning model, it is necessary to collect more auscultation data from specialists. In order to motivate specialists to send auscultation data, when a specialist selects the first mode and sends auscultation data to the external device 2, it is preferable to give a reward to the specialist. On the other hand, when a person other than a specialist uses the second mode to send auscultation data to the external device 2, since the person other than the specialist uses the knowledge of the specialist, it is preferable to charge the person other than the specialist.

[0106] Therefore, when the communication processing unit 167 sends auscultation data in the first mode, the display processing unit 162 causes the display 13 to display information indicating that a reward has occurred for the user U of the stethoscope ST. On the other hand, when the communication processing unit 167 sends auscultation data in the second mode, the display processing unit 162 causes the display 13 to display information indicating that a charge has occurred for the user U.

[0107] For example, after the communication processing unit 167 sends auscultation data to the external device 2, the terminal communication unit 12 receives a notice indicating that a reward or a charge has occurred, and the amount of the reward or the charge. The display processing unit 162 causes the display 13 to display a notice indicating that a reward or a charge has occurred, which the terminal communication unit 12 has received, and the amount of the reward or the charge. Note that the external device 2 may determine an amount corresponding to the data volume of the auscultation data transmitted by the communication processing unit 167 as the amount of the reward or the charge.

[0108] In this way, by generating a reward when the communication processing unit 167 sends auscultation data to the external device 2 in the first mode, more auscultation data from specialists can be collected, so that a more accurate machine learning model can be created. On the other hand, by generating a charge when the communication processing unit 167 sends auscultation data to the external device 2 in the second mode, the operator providing the information processing system S can receive a reward from a person other than a specialist, and can also collect the amount necessary for paying the reward to the specialist.

[0109] [Configuration and Operation of External Device 2] FIG. 13 is a diagram showing the configuration of the external device 2 (data processing device 2). The external device 2 includes a device communication unit 21, a storage unit 22, and a control unit 23.

[0110] The device communication unit 21 is a communication interface for communicating with the data processing terminal 1 via a communication network such as the Internet.

[0111] The storage unit 22 is a storage medium including a ROM, a RAM, and the like. The storage unit 22 stores an information processing program that causes the control unit 23 to function as a data reception unit 231 and a learning unit 232. The storage unit 22 stores the learned machine learning model.

[0112] The control unit 23 is, for example, a CPU. The control unit 23 functions as a data reception unit 231 and a learning unit 232 by executing the information processing program stored in the storage unit 22.

[0113] The data reception unit 231 receives auscultation data. The data reception unit 231 receives, for example, auscultation data in which an auscultation position, biological sound data, and diagnostic data are associated, transmitted from the data processing terminal 1, via the device communication unit 21.

[0114] The learning unit 232 learns (trains) a machine learning model using the biological sound data as teacher data indicating abnormality or teacher data indicating normality, in association with the auscultation position. For example, when the diagnostic data in the auscultation data indicates that the auscultation sound is abnormal, the learning unit 232 learns the machine learning model using the biological sound data as teacher data indicating abnormality. On the other hand, for example, when the diagnostic data in the auscultation data indicates that the auscultation sound is normal, the learning unit 232 learns the machine learning model using the biological sound data as teacher data indicating normality.

[0115] When the biological sound data in the auscultation data is associated with label data (label data indicating the auscultation position, the state of the subject's breath, the auscultation state, or the auscultation result), for example, the learning unit 232 may perform learning for each content of the label data or use the label data as a feature amount for learning. As a result, the external device 2 can create a more accurate machine learning model.

[0116] The machine learning performed by the learning unit 232 may be either classification or regression. In the case of classification, the learning unit 232 learns a machine learning model for determining the presence or absence of an abnormality in the auscultation sound or the presence or absence of a disease in the subject. In the case of regression, a machine learning model for determining the degree of abnormality of the auscultation sound or the probability that the subject has a disease is learned.

[0117] The learning unit 232 can perform machine learning by a known method. Known methods are, for example, methods such as logistic regression, support vector machine (SVM), decision tree, random forest, and neural network.

[0118] [Processing flow in the data processing terminal 1] FIG. 14 is a flowchart showing the processing flow in the data processing terminal 1. Hereinafter, with reference to FIG. 14, the processing flow in the data processing terminal 1 will be described.

[0119] The operation reception unit 161 determines whether the operation mode selected by the user U on the mode selection screen (FIG. 6) is the first mode used by a medical professional (S1).

[0120] When the operation reception unit 161 determines that the operation mode selected by the user U is the first mode used by a medical specialist (S1: YES), the data processing terminal 1 executes the following processes S2 to S5. The biological sound data acquisition unit 165 acquires biological sound data based on the auscultation sound acquired by the stethoscope ST in contact with the subject (S2). Also, the position data acquisition unit 163 acquires position data indicating the auscultation position where the stethoscope ST acquired the auscultation sound (S3). Also, the diagnostic data acquisition unit 166 acquires diagnostic data indicating a diagnostic result in association with the auscultation position (S4).

[0121] Then, the communication processing unit 167 transmits auscultation data in which the biological sound data acquired in S2, the auscultation position acquired in S3, and the diagnostic data acquired in S4 are associated to the external device 2 (S5). Thereby, the external device 2 can learn a machine learning model for diagnosing whether the auscultation sound is abnormal using the biological sound data associated with the auscultation position as teacher data.

[0122] When the operation reception unit 161 determines that the operation mode selected by the user U is not the first mode used by a medical specialist (S1: NO), it is determined that the operation mode is the second mode used by non-medical specialists. In this case, the data processing terminal 1 executes the following processes S6 to S9. The biological sound data acquisition unit 165 acquires biological sound data based on the auscultation sound acquired by the stethoscope ST in contact with the subject (S6). Also, the position data acquisition unit 163 acquires position data indicating the auscultation position where the stethoscope ST acquired the auscultation sound (S7).

[0123] Then, the communication processing unit 167 transmits auscultation data in which the biological sound data acquired in S6 and the auscultation position acquired in S7 are associated to the external device 2 (S8). Thereby, the external device 2 can determine whether the auscultation sound is abnormal using the learned machine learning model. The terminal communication unit 12 acquires the determination result by receiving the determination result from the external device 2 (S9).

[0124] [Modification example] In the above description, the case where the external device 2 to which the data processing terminal 1 transmits biometric sound data in the first mode and the external device 2 to which the data processing terminal 1 transmits biometric sound data in the second mode are the same device is illustrated. However, the external device 2 to which the data processing terminal 1 transmits biometric sound data may be different between the first mode and the second mode. Specifically, in the first mode, the data processing terminal 1 transmits biometric sound data to a first external device having a function of creating a machine learning model. In the second mode, the data processing terminal 1 transmits biometric sound data to a second external device having a function of determining a disease name using the created machine learning model. By operating in this way, a large number of second external devices can use the machine learning model created by the first external device to determine the disease name based on the biometric sound data.

[0125] [Effect by data processing terminal 1] As described above, the data processing terminal 1 according to the present embodiment transmits position data indicating the auscultation position and biometric sound data obtained when a medical specialist auscultates a subject to the external device 2 in a state where it is possible to determine whether the auscultation sound is normal or abnormal. In this way, since the data processing terminal 1 transmits teacher data for learning a machine learning model to the external device 2 when a medical specialist diagnoses a subject, the external device 2 can learn the machine learning model in a short period of time.

[0126] In addition, the information processing system S can provide a determination result of the presence or absence of an abnormality in the auscultation sound and the disease name based on the learning result by using the created machine learning model. As a result, even a person other than a medical specialist can accurately determine the presence or absence of a disease and the disease name of a subject by auscultating using the data processing terminal 1.

[0127] As described above, the present invention has been explained using embodiments. However, the technical scope of the present invention is not limited to the scope described in the above embodiments, and various modifications and changes are possible within the scope of the gist. For example, all or part of the device can be configured by being functionally or physically dispersed and integrated in any unit. Also, new embodiments resulting from any combination of multiple embodiments are included in the embodiments of the present invention. The effects of the new embodiments resulting from the combination have the effects of the original embodiments combined.

Explanation of Signs

[0128] 1 Data processing terminal 11 Operation unit 12 Terminal communication unit 13 Display 14 Microphone 15 Storage unit 16 Control unit 161 Operation reception unit 162 Display processing unit 163 Position data acquisition unit 164 State detection unit 165 Biometric sound data acquisition unit 166 Diagnosis data acquisition unit 167 Communication processing unit 2 External device (data processing device) 21 Device communication unit 22 Storage unit 23 Control unit 231 Data reception unit 232 Learning unit S Information processing system ST Stethoscope C Chest piece E Earphone T Tube

Claims

1. A processor included in a data processing terminal is caused to function as: a biological sound data acquisition unit that acquires biological sound data based on auscultation sounds acquired by a stethoscope that has come into contact with a subject; a position data acquisition unit that acquires position data indicating an auscultation position at which the stethoscope has acquired the auscultation sounds; a state detection unit that detects a state of the subject's breath based on a timing of an instruction to change the state of the subject's breath; a communication processing unit that transmits auscultation data in which the auscultation position indicated by the position data and the biological sound data are associated to an external device in a state where it is possible to determine whether the auscultation sounds are normal or abnormal; a program for causing the above to function, wherein the communication processing unit transmits the biological sound data for a period corresponding to the state of the breath in association with label data indicating the state of the subject's breath detected by the state detection unit; a program for the above.

2. The state detection unit detects whether the subject is in a state of inhaling or exhaling by detecting a voice instructing the subject to inhale or a voice instructing the subject to exhale, and the communication processing unit transmits the biological sound data for at least a part of a period from a first timing when the subject is instructed to inhale to a second timing when the subject is instructed to exhale in association with the label data indicating that the subject is in a state of inhaling detected by the state detection unit, and transmits the biological sound data for at least a part of a period from the second timing to the first timing in association with the label data indicating that the subject is in a state of exhaling detected by the state detection unit. The program according to claim 1.

3. The processor is further caused to function as a display processing unit that causes a display unit to display timing information indicating a first timing when the subject is instructed to inhale and a second timing when the subject is instructed to exhale, and the communication processing unit transmits the biological sound data for at least a part of a period from the first timing to the second timing in association with the label data indicating that the subject is in a state of inhaling, and transmits the biological sound data for at least a part of a period from the second timing to the first timing in association with the label data indicating that the subject is in a state of exhaling. The program according to claim 1.

4. ​ The communication processing unit transmits the auscultation data to an external device that creates a machine learning model for determining the presence or absence of an abnormality based on the biological sound data acquired at each of the plurality of auscultation positions. The program according to claim 1.

5. The processor is further caused to function as a diagnostic data acquisition unit that acquires diagnostic data indicating a diagnostic result in association with the auscultation position. The communication processing unit transmits the auscultation data in which the auscultation position, the biological sound data, and the diagnostic data are associated. The program according to claim 1.

6. The processor is further caused to function as a display processing unit that causes a display unit to display a body image for a user of the stethoscope to set the auscultation position, and an operation reception unit that receives an operation for the user to set the auscultation position, and the position data acquisition unit acquires the position data based on the operation received by the operation reception unit, and the communication processing unit transmits the auscultation data in which label data indicating the auscultation position corresponding to the position data and the biological sound data are associated. The program according to claim 1.

7. The processor is further caused to function as a display processing unit that causes a display unit to display a body image for a user of the stethoscope to set the auscultation position, and an operation reception unit that receives an operation for the user to set the auscultation position, and the position data acquisition unit acquires, as the position data, data indicating the reference position among a plurality of reference positions in the body image that is closest to the position set by the user on the body image. The program according to claim 1.

8. The state detection unit detects the state of the subject's body by detecting the voice uttered by the user of the stethoscope. The program according to claim 1.

9. The state detection unit causes the display unit to display one or more candidates for the state of the subject's body, and detects, as the state of the body, the state corresponding to the candidate selected by the user of the stethoscope from the one or more candidates. The program according to claim 1.

10. The processor is further caused to function as an operation reception unit that receives an operation for the user of the stethoscope to select a diagnostic mode to be used or an attribute of the subject from among a plurality of diagnostic modes, and the state detection unit causes the display unit to display the one or more candidates corresponding to the diagnostic mode or the attribute corresponding to the operation received by the operation reception unit. The program according to claim 9.

11. The state detection unit analyzes the biological sound data and causes the display unit to display the one or more candidates corresponding to the analysis result. The program according to claim 9.

12. The processor is further caused to function as an operation reception unit that receives an operation for switching the operation mode of the communication processing unit to either a first mode or a second mode. When the operation reception unit receives an operation for setting the first mode, the communication processing unit transmits the auscultation data. When the operation reception unit receives an operation for setting the second mode, the communication processing unit transmits the auscultation data in order to obtain a determination result from the external device that determines the presence or absence of an abnormality based on the biological sound data. The program according to claim 1.

13. When the communication processing unit transmits the auscultation data in the first mode, the processor is further caused to function as a display processing unit that causes the display unit to display information indicating that a consideration has occurred for the user of the stethoscope. When the communication processing unit transmits the auscultation data in the second mode, the processor is further caused to function as a display processing unit that causes the display unit to display information indicating that a charge has occurred for the user. The program according to claim 12.

14. A processor included in a data processing terminal is a display processing unit that causes the display unit to display a body image for a user of a stethoscope to set an auscultation position and instruction text to be notified to the user, an operation reception unit that receives an operation of the user, a biological sound data acquisition unit that acquires biological sound data based on an auscultation sound acquired by the stethoscope that has contacted the subject, a position data acquisition unit that acquires position data indicating the auscultation position at which the stethoscope has acquired the auscultation sound, a communication processing unit that transmits auscultation data in which the auscultation position indicated by the position data is associated with the biological sound data to an external device in a state where it is possible to determine whether the auscultation sound is normal or abnormal, caused to function as, The display processing unit determines the positional relationship between the body image and the instruction text to be displayed on the display unit in response to the operation reception unit receiving an operation for designating the hand holding the stethoscope or the hand operating the data processing terminal. for a program.

15. A data processing apparatus and a data processing terminal are provided, wherein the data processing terminal includes a biological sound data acquisition unit that acquires biological sound data based on an auscultation sound acquired by a stethoscope that has contacted a subject. A position data acquisition unit that acquires position data indicating the auscultation position where the stethoscope has acquired the auscultation sound; A state detection unit that detects the state of the subject's breath based on the timing of an instruction to change the state of the subject's breath; A communication processing unit that transmits auscultation data in which the auscultation position indicated by the position data is associated with the biological sound data to the data processing device in a state where it is possible to determine whether the auscultation sound is normal or abnormal; having; The communication processing unit transmits the biological sound data for a period corresponding to the state of the breath in association with label data indicating the state of the subject's breath detected by the state detection unit; The data processing device; A data reception unit that receives the auscultation data; A learning unit that learns (trains) a machine learning model using the biological sound data as teacher data indicating abnormality or teacher data indicating normality in association with the auscultation position and the state of the breath; A data processing system having.

16. Steps executed by a computer, A step of acquiring biological sound data based on auscultation sound acquired by a stethoscope in contact with a subject; A step of acquiring position data indicating the auscultation position where the stethoscope has acquired the auscultation sound; A step of detecting the state of the subject's breath based on the timing of an instruction to change the state of the subject's breath; A step of transmitting auscultation data in which the auscultation position indicated by the position data is associated with the biological sound data to an external device in a state where it is possible to determine whether the auscultation sound is normal or abnormal; having; In the step of transmitting, the biological sound data for a period corresponding to the state of the breath is transmitted in association with label data indicating the state of the subject's breath detected in the step of detecting; A data processing method.

17. A biological sound data acquisition unit that acquires biological sound data based on auscultation sound acquired by a stethoscope in contact with a subject; A position data acquisition unit that acquires position data indicating the auscultation position where the stethoscope has acquired the auscultation sound; A state detection unit that detects the state of the subject's breath based on the timing of an instruction to change the state of the subject's breath; A communication processing unit that transmits auscultation data in which the auscultation position indicated by the position data is associated with the biological sound data to an external device in a state where it is possible to determine whether the auscultation sound is normal or abnormal; having; The communication processing unit transmits the biometric sound data for a period corresponding to the breath state in association with label data indicating the breath state of the subject detected by the state detection unit. Data processing terminal.

Images