Information processing device, information processing system, information processing method, and control program

Abstract

The present invention provides an information processing device, an information processing system, an information processing method, and a control program that make it possible to more accurately notify a user of the condition of intestines. The information processing device comprises: an acquisition unit that acquires a plurality of consecutive sound frames which have been collected from the abdomen of a user; a determination unit that determines whether or not an intestinal sound is included for each of the plurality of sound frames; a correction unit that corrects a determination result from the determination unit on the basis of the number of consecutive sound frames which have been determined to include the intestinal sound or the number of consecutive sound frames which have been determined not to include the intestinal sound; a detection unit that detects, on the basis of the determination result from the determination unit and a determination result which has been obtained by the correction made by the correction unit, an intestinal sound frame group which consists of consecutive sound frames which have been determined to include that the intestinal sound in the plurality of sound frames; an identification unit that identifies intestinal information pertaining to the condition of intestines of the user on the basis of the intestinal sound frame group; and an output unit that outputs the intestinal information.

Description

Information processing device, information processing system, information processing method, and control program

[0001] The present disclosure relates to an information processing device, an information processing system, an information processing method, and a control program.

[0002] The intestines (small and large intestines) are important organs for the digestion and absorption of nutrients and for symbiosis with intestinal bacteria. In recent years, there has been an increasing need for information about intestinal health, and systems that provide users with information about their intestinal health have been developed.

[0003] Japanese Patent Laid-Open No. 2003-144993 discloses a health management support system that acquires and records a user's health management information and outputs messages relating to health management to the user based on the contents of the information.

[0004] Patent Document 2 discloses an analysis device that predicts the type of biological sound contained in acoustic data obtained from a subject by a sound collection device according to a prediction algorithm learned by a machine learning device, and evaluates the presence or absence of intestinal disease based on the predicted type of biological sound.

[0005] JP 2017-041035 A International Publication No. 2019 / 216320

[0006] There is a demand for more accurate notification of intestinal conditions to users.

[0007] An object of the information processing device, information processing system, information processing method, and control program is to enable more accurate notification of the user's intestinal condition.

[0008] The information processing device according to the embodiment has an acquisition unit that acquires multiple consecutive sound frames collected from the abdomen of a user, a determination unit that determines whether or not each of the multiple sound frames contains bowel sounds, a correction unit that corrects the determination result made by the determination unit based on the number of consecutive sound frames that have been determined to contain bowel sounds or the number of consecutive sound frames that have been determined not to contain bowel sounds, a detection unit that detects a group of bowel sound frames made up of consecutive sound frames that have been determined to contain bowel sounds among the multiple sound frames based on the determination result made by the determination unit and the determination result corrected by the correction unit, an identification unit that identifies intestinal information related to the condition of the user's intestines based on the group of bowel sound frames, and an output unit that outputs the intestinal information.

[0009] In the information processing device according to the embodiment, it is preferable that the information processing device further includes a calculation unit that calculates frequency information regarding the frequency of bowel sounds occurring in a plurality of sound frames based on the bowel sound frame group, and the identification unit identifies the bowel information based on the frequency information.

[0010] In the information processing device according to the embodiment, the correction unit preferably considers that a sound frame group in which the number of consecutive frames determined to contain bowel sounds is equal to or less than a first threshold does not contain bowel sounds.

[0011] In the information processing device according to the embodiment, the correction unit preferably considers a sound frame group containing bowel sounds if the number of consecutive sound frames determined not to contain bowel sounds is equal to or less than a second threshold value.

[0012] In the information processing device according to the embodiment, it is preferable that the determination unit determines whether or not each sound frame contains bowel sounds by inputting information based on each sound frame acquired by the acquisition unit into a trained model that has been trained to output information indicating whether or not a sound frame collected from the abdomen of a user is included when the sound frame is input.

[0013] The information processing system according to the embodiment has an acquisition unit that acquires multiple consecutive sound frames collected from the abdomen of a user, a determination unit that determines whether or not each of the multiple sound frames contains bowel sounds, a correction unit that corrects the determination result made by the determination unit based on the number of consecutive sound frames that have been determined to contain bowel sounds or the number of consecutive sound frames that have been determined not to contain bowel sounds, a detection unit that detects a group of bowel sound frames made up of consecutive sound frames that have been determined to contain bowel sounds among the multiple sound frames based on the determination result made by the determination unit and the determination result corrected by the correction unit, an identification unit that identifies intestinal information related to the condition of the user's intestines based on the group of bowel sound frames, and an output unit that outputs the intestinal information.

[0014] The information processing method according to the embodiment acquires a plurality of consecutive sound frames collected from the abdomen of a user, determines whether or not each of the plurality of sound frames contains bowel sounds, corrects the determination result of whether or not bowel sounds are contained based on the number of consecutive sound frames determined to contain bowel sounds or the number of consecutive sound frames determined not to contain bowel sounds, detects a group of bowel sound frames consisting of consecutive sound frames determined to contain bowel sounds among the plurality of sound frames based on the determination result and the corrected determination result, identifies intestinal information relating to the condition of the user's intestines based on the group of bowel sound frames, and outputs the intestinal information.

[0015] The control program according to the embodiment is a computer control program that causes the computer to acquire multiple consecutive sound frames collected from the abdomen of a user, determine whether or not each of the multiple sound frames contains bowel sounds, correct the determination result of whether or not bowel sounds are contained based on the number of consecutive sound frames that are determined to contain bowel sounds or the number of consecutive sound frames that are determined not to contain bowel sounds, detect a group of bowel sound frames consisting of consecutive sound frames that are determined to contain bowel sounds among the multiple sound frames based on the determination result and the corrected determination result, identify intestinal information related to the condition of the user's intestines based on the group of bowel sound frames, and output the intestinal information.

[0016] The information processing device, information processing system, information processing method, and control program can more accurately notify the user of the state of their intestines.

[0017] The objects and advantages of the invention will be realized and obtained by means of the elements and combinations particularly pointed out in the claims. Both the foregoing general description and the following detailed description are exemplary and explanatory and are not restrictive of the invention as claimed.

[0018] 1 is a diagram illustrating a schematic configuration of an information processing system 1 according to an embodiment. FIG. 2 is a diagram illustrating a schematic configuration of a terminal device 100. FIG. 3 is a diagram illustrating a schematic configuration of a server device 200. FIG. 4 is a sequence illustrating an example of the operation of intestinal information output processing. FIG. 5 is a schematic diagram illustrating an example of a sound collection preparation screen 500. FIG. 6 is a schematic diagram illustrating an example of a sound collection status screen 510. FIG. 7 is a schematic diagram illustrating an example of a notification screen 520. FIG. 8 is a flowchart illustrating an example of the operation of a specific process. FIG. 9 is a schematic diagram illustrating correction of a bowel sound determination result. FIG. 10 is a schematic diagram illustrating correction of a bowel sound determination result. FIG. 11 is a schematic diagram illustrating correction of a bowel sound determination result.

[0019] Hereinafter, an information processing device, an information processing system, an information processing method, and a control program according to one aspect of an embodiment will be described with reference to the drawings. However, please note that the technical scope of the present invention is not limited to the embodiment, but extends to the inventions set forth in the claims and their equivalents.

[0020] FIG. 1 is a diagram showing a schematic configuration of an information processing system 1 according to an embodiment.

[0021] 1, the information processing system 1 includes one or more terminal devices 100 and a server device 200. The terminal devices 100 and the server device 200 are connected to each other so as to be able to communicate with each other via a network N. The network N is a wired network such as the Internet or an intranet. The network N may also be a wireless network such as a wireless LAN (Local Area Network).

[0022] FIG. 2 is a diagram showing a schematic configuration of the terminal device 100. As shown in FIG.

[0023] The terminal device 100 is an example of an information processing device. The terminal device 100 is a multi-function mobile phone (a so-called smartphone). The terminal device 100 may be a personal computer, a notebook PC, a tablet PC, or the like. The terminal device 100 includes a first communication device 101, an input device 102, a display device 103, a sound collection device 104, a first storage device 110, and a first processing device 120. The first communication device 101, the input device 102, the display device 103, the sound collection device 104, the first storage device 110, and the first processing device 120 are connected to each other via a CPU (Central Processing Unit) bus or the like.

[0024] The first communication device 101 has an antenna for transmitting and receiving wireless signals and a wireless communication interface circuit conforming to a communication protocol such as wireless LAN, and is communicatively connected to the network N in accordance with the communication standard such as wireless LAN. The first communication device 101 sends data received from the server device 200 or the like via the network N to the first processing device 120. The first communication device 101 also transmits data received from the first processing device 120 to the server device 200 or the like via the network N. The first communication device 101 may have a wireless communication interface circuit conforming to a communication standard such as LTE (Long Term Evolution) or 5G, and may be communicatively connected to the network N via a base station. The first communication device 101 may also have a wired communication interface circuit conforming to a communication protocol such as TCP / IP (Transmission Control Protocol / Internet Protocol), and may be communicatively connected to the network N in accordance with a communication standard such as Ethernet (registered trademark).

[0025] The input device 102 has an input device such as a touch panel type input device, a keyboard, or a mouse, and an interface circuit that acquires signals from the input device, and outputs operation signals in response to input operations by the user.

[0026] The display device 103 is an example of an output unit. The display device 103 has a display such as a liquid crystal display or an organic electroluminescence (EL) display, and an interface circuit that outputs image data to the display, and displays the image data on the display.

[0027] The sound collection device 104 has a microphone that receives ambient sound and outputs an electrical signal corresponding to the input sound, and an A / D converter that amplifies the electrical signal output from the microphone and performs analog-to-digital (A / D) conversion. The sound collection device 104 converts the input sound into a digital signal every predetermined time (e.g., 50 msec) to generate a sound frame and output it to the first processing device 120. Note that the sound collection device 104 may be an external device connected to the terminal device 100 via a wired interface circuit such as a USB (Universal Serial Bus) or a wireless interface circuit such as Bluetooth (registered trademark).

[0028] The first storage device 110 is an example of a storage unit. The first storage device 110 includes a memory device such as a RAM (Random Access Memory) or a ROM (Read Only Memory), a fixed disk device such as a hard disk, or a portable storage device such as a flexible disk or an optical disk. The first storage device 110 also stores computer programs, databases, tables, and the like used for various processes of the terminal device 100. The computer programs may be installed into the first storage device 110 from a computer-readable portable recording medium using a known setup program or the like. The portable recording medium is, for example, a CD-ROM (Compact Disc Read Only Memory), a DVD-ROM (Digital Versatile Disc Read Only Memory), or the like. The computer programs may be stored in a recording medium owned by a predetermined server and installed via a network N.

[0029] The first processing device 120 operates based on a program stored in advance in the first storage device 110. The first processing device 120 is, for example, a CPU. A digital signal processor (DSP), a large scale integration (LSI), an application specific integrated circuit (ASIC), a field-programmable gate array (FPGA), or the like may be used as the first processing device 120. The first processing device 120 is connected to the first communication device 101, the input device 102, the display device 103, the sound collection device 104, the first storage device 110, and the like, and controls each device. The first processing device 120 acquires multiple consecutive sound frames collected from the user's abdomen from the sound collection device 104, and transmits them to the server device 200 via the first communication device 101.

[0030] The first processing device 120 reads the computer program stored in the first storage device 110 and operates in accordance with the read computer program. As a result, the first processing device 120 functions as a first acquisition unit 121 and a first output control unit 122. The first acquisition unit 121 and the first output control unit 122 are examples of an acquisition unit and an output control unit, respectively.

[0031] FIG. 3 is a diagram showing a schematic configuration of the server device 200.

[0032] The server device 200 includes a second communication device 201, a second storage device 210, and a second processing device 220. The second communication device 201, the second storage device 210, and the second processing device 220 are connected to each other via a CPU bus or the like.

[0033] The second communication device 201 is an example of an output unit. The second communication device 201 has a wired communication interface circuit that complies with a communication protocol such as TCP / IP. The second communication device 201 is communicatively connected to the network N in accordance with a communication standard such as Ethernet (registered trademark). The second communication device 201 sends data received from the terminal device 100 or the like via the network N to the second processing device 220. The second communication device 201 transmits data received from the second processing device 220 to the terminal device 100 or the like via the network N. The second communication device 201 may have an antenna that transmits and receives wireless signals and a wireless communication interface circuit that complies with a communication protocol such as wireless LAN, and may be communicatively connected to the network N in accordance with a communication standard such as wireless LAN.

[0034] The second storage device 210 includes a memory device such as RAM or ROM, a fixed disk device such as a hard disk, or a portable storage device such as a flexible disk or optical disk. The second storage device 210 also stores computer programs, databases, tables, and the like used for various processes of the server device 200. The computer programs may be installed into the second storage device 210 from a computer-readable portable recording medium such as a CD-ROM or DVD-ROM using a known setup program or the like. The computer programs may be stored in a recording medium owned by a predetermined server and installed via the network N.

[0035] The second storage device 210 stores data such as a trained model 211. The trained model 211 is a model that has been pre-trained to output information indicating whether or not bowel sounds emitted from the intestines are included in a sound frame when information based on the sound frame collected from the user's abdomen is input. The information based on the sound frame is, for example, a spectrum image generated from the sound frame or the sound frame itself. Bowel sounds include sounds emitted by intestinal peristalsis. The trained model 211 is generated by the second processing device 220 or another server device. The trained model 211 is pre-trained, for example, by deep learning such as a convolutional neural network (CNN) or supervised learning such as a random forest or support vector regression. The trained model 211 is pre-trained using multiple sets of information based on the sound frame collected from the user's abdomen and ground truth data indicating whether or not each sound frame includes bowel sounds, and is stored in the second storage device 210 in advance. As the trained model 211, functions in various machine learning frameworks such as fastText, tinySVM, random forest, and TensorFlow, or various existing libraries may be used.

[0036] The second processing device 220 operates based on a program stored in advance in the second storage device 210. The second processing device 220 is, for example, a CPU. A DSP, an LSI, an ASIC, an FPGA, etc. may also be used as the second processing device 220. The second processing device 220 is connected to the second communication device 201, the second storage device 210, etc., and controls each device. The second processing device 220 determines whether or not each of the multiple sound frames acquired from the terminal device 100 contains bowel sounds, corrects the determination result, and identifies bowel information related to the user's bowel condition based on the corrected determination result.

[0037] The second processing device 220 reads the computer program stored in the second storage device 210 and operates in accordance with the read computer program. As a result, the second processing device 220 functions as a second acquisition unit 221, a determination unit 222, a correction unit 223, a detection unit 224, a calculation unit 225, an identification unit 226, and a second output control unit 227. The second acquisition unit 221 and the second output control unit 227 are examples of an acquisition unit and an output control unit, respectively.

[0038] FIG. 4 is a sequence diagram showing an example of the operation of the intestinal information output process in the information processing system 1.

[0039] An example of the operation of the intestinal information output process will be described below with reference to the sequence shown in Fig. 4. The operation flow described below is executed mainly by the processing devices of each device in cooperation with the elements of each device, based on a program stored in advance in each storage device of each device included in the information processing system 1.

[0040] First, when a command to output intestinal information is input by the user of the terminal device 100 using the input device 102, the first acquisition unit 121 of the terminal device 100 acquires environmental sounds (ambient sounds) of the terminal device 100 from the sound collection device 104 (step S101). The first acquisition unit 121 causes the sound collection device 104 to start collecting sound while the sound collection device 104 is not in contact with the user's abdomen, sequentially acquires multiple consecutive sound frames collected by the sound collection device 104 as sound frames of environmental sounds, and stores the acquired frames in the first storage device 110. The first acquisition unit 121 measures the intensity (sound pressure) of the sounds included in the sound frames acquired from the sound collection device 104. The first acquisition unit 121 also generates display data for a sound collection preparation screen indicating that preparation for sound collection is underway, and displays the sound collection preparation screen on the display device 103.

[0041] FIG. 5A is a schematic diagram showing an example of a sound collection preparation screen 500. As shown in FIG.

[0042] As shown in FIG. 5A , the sound collection preparation screen 500 displays an environmental sound intensity 501, a sound collection method 502, a sound collection button 503, an end button 504, and the like. The environmental sound intensity 501 indicates the intensity of the environmental sound of the terminal device 100. The sound collection method 502 indicates a method by which the user collects abdominal sounds (such as contacting the sound collection device 104 with the part of the abdomen facing the intestines). The sound collection button 503 is a button for instructing the start of sound collection. The end button 504 is a button for instructing the end (cancellation) of sound collection. When the end button 504 is pressed on the sound collection preparation screen 500, the first acquisition unit 121 ends the intestinal information output process.

[0043] Next, when the user of the terminal device 100 inputs an instruction to collect abdominal sounds using the input device 102, the first acquisition unit 121 acquires abdominal sounds emitted from the user's abdomen from the sound collection device 104 (step S102). The abdominal sounds include bowel sounds emitted from the intestines. The abdominal sounds may also include sounds emitted by blood flow in the abdomen (e.g., abdominal aortic sounds) or sounds emitted from organs such as the stomach. The first acquisition unit 121 determines that an instruction to collect abdominal sounds has been input when the sound collection button 503 is pressed on the sound collection preparation screen 500. The first acquisition unit 121 causes the sound collection device 104 to start collecting sound while the sound collection device 104 is in contact with the user's abdomen, and sequentially acquires multiple consecutive sound frames collected from the user's abdomen by the sound collection device 104 as sound frames of abdominal sounds and stores them in the first storage device 110. The first acquisition unit 121 measures the intensity (sound pressure) of the sound included in the sound frame acquired from the sound collection device 104. The first acquisition unit 121 also generates display data for a sound collection status screen indicating that abdominal sounds are being collected, and causes the display device 103 to display the sound collection status screen.

[0044] FIG. 5B is a schematic diagram showing an example of a sound collection status screen 510.

[0045] 5B , the sound collection status screen 510 displays a sound collection status 511, a remaining time 512, an abdominal sound intensity 513, a stop button 514, and the like. The sound collection status 511 indicates the current sound collection status (abdominal sounds are being collected). The remaining time 512 indicates the time remaining until sound collection is completed. The sound collection time is set in advance to, for example, 60 seconds, and the first acquisition unit 121 calculates the remaining time 512 by subtracting the elapsed time from the start of sound collection from the predetermined sound collection time. The abdominal sound intensity 513 indicates the intensity of abdominal sounds collected by the sound collection device 104. The stop button 514 is a button for instructing the stopping (cancellation) of sound collection. When the stop button 514 is pressed on the sound collection status screen 510, the first acquisition unit 121 ends the intestinal information output process.

[0046] In addition, the first acquisition unit 121 may acquire abdominal sounds by reading them from the first storage device 110, rather than having the sound collection device 104 collect abdominal sounds in real time. The abdominal sounds have been collected in advance and stored (recorded) in the first storage device 110.

[0047] Next, when the collection of abdominal sounds is completed, first acquisition unit 121 causes sound collection device 104 to end sound collection, and transmits the acquired sound frames of abdominal sounds and environmental sounds to server device 200 via first communication device 101 (step S103). When the sound collection time has elapsed since the start of sound collection, first acquisition unit 121 determines that the collection of abdominal sounds is completed.

[0048] Next, the second acquisition unit 221 of the server device 200 receives sound frames of abdominal sounds and environmental sounds from the terminal device 100 via the second communication device 201 and stores them in the second storage device 210. As a result, the second acquisition unit 221 acquires multiple consecutive sound frames collected from the user's abdomen. Next, the second processing device 220 executes identification processing (step S104). In the identification processing, the second processing device 220 identifies intestinal information related to the user's intestinal condition based on the sound frames acquired by the second acquisition unit 221. The identification processing will be described in detail below.

[0049] Next, the second output control unit 227 outputs the intestinal information identified in the identification process by transmitting it to the terminal device 100 via the second communication device 201 (step S105).

[0050] Next, the first output control unit 122 of the terminal device 100 acquires the intestinal information by receiving it from the server device 200 via the first communication device 101. Next, the first output control unit 122 generates display data for a notification screen for notifying the user of the acquired intestinal information, and outputs the notification screen by displaying it on the display device 103 (step S106).

[0051] FIG. 5C is a schematic diagram showing an example of a notification screen 520.

[0052] As shown in FIG. 5C , the notification screen 520 displays a condition 521, a score 522, an end button 523, and the like. The condition 521 indicates the user's intestinal condition. The condition 521 may indicate a rank corresponding to the user's intestinal condition. The score 522 indicates the score of abdominal sounds acquired by the terminal device 100. The end button 523 is a button for instructing the end of the display of the notification screen 520. When the end button 523 is pressed on the notification screen 520, the first output control unit 122 ends the intestinal information output process. By viewing the notification screen 520, the user can easily and accurately recognize their own intestinal condition.

[0053] FIG. 6 is a flowchart showing an example of the operation of the identification process.

[0054] The identification process shown in FIG. 6 is executed in step S104 in FIG.

[0055] First, the second acquisition unit 221 of the server device 200 generates a spectrum image corresponding to each acquired abdominal sound frame (step S201). The second acquisition unit 221 performs frequency transformation such as a Fourier transform or a fast Fourier transform on each sound frame to generate a spectrum image showing a wave (graph) with the horizontal axis representing frequency and the vertical axis representing the intensity of each frequency.

[0056] The second acquisition unit 221 may normalize each sound frame and then generate a spectrum image corresponding to the normalized sound frame. For example, the second acquisition unit 221 normalizes each sound frame so that the minimum and maximum sound intensities included in sound frames for a certain period of time are 0 and 1, respectively. Alternatively, the second acquisition unit 221 normalizes each sound frame by converting the sound intensities included in each sound frame according to a predetermined conversion formula. This allows the server device 200 to more accurately determine whether or not each sound frame contains bowel sounds.

[0057] Next, the determination unit 222 determines whether or not each sound frame contains bowel sounds for each sound frame of abdominal sounds acquired by the second acquisition unit 221 (step S202). The determination unit 222 inputs spectrum images generated from each sound frame into the trained model 211 and acquires information output from the trained model 211, thereby determining whether or not each sound frame contains bowel sounds. By utilizing the trained model 211, the information processing system 1 can determine with high accuracy whether or not each sound frame contains bowel sounds. Hereinafter, sound frames of abdominal sounds determined to contain bowel sounds may be referred to as bowel sound frames, and sound frames of abdominal sounds determined not to contain bowel sounds may be referred to as non-bow sound frames.

[0058] Next, the correction unit 223 determines whether or not there is a sound frame group in which the number of consecutive bowel sound frames is equal to or less than a first threshold value among the abdominal sound sound frames acquired by the second acquisition unit 221 (step S203). The first threshold value is set in advance to, for example, the number of frames corresponding to the minimum, maximum, average, or median value of the length of sound that a human can hear (distinguish). In other words, a sound frame group in which the number of consecutive bowel sound frames is equal to or less than the first threshold value is considered to be bowel sounds that a human cannot hear.

[0059] The correction unit 223 may set the first threshold based on the sound frames of the environmental sound acquired by the second acquisition unit 221. In this case, for example, the correction unit 223 measures the intensity of the sound included in the sound frames of the environmental sound, and corrects the first threshold so that the greater the intensity of the environmental sound, the larger the first threshold, and the smaller the intensity of the environmental sound, the smaller the first threshold. This allows the information processing system 1 to correct the determination result by the determination unit 222 taking into account the influence of the environmental sound, and to determine the user's intestinal condition with higher accuracy.

[0060] If there is no sound frame group in which the number of consecutive bowel sound frames is equal to or less than the first threshold, the correction unit 223 does not perform any particular process and proceeds to step S205.

[0061] On the other hand, if there is a sound frame group in which the number of consecutive bowel sound frames is equal to or less than the first threshold, the correction unit 223 corrects the bowel sound frames included in that sound frame group to non-bow sound frames (step S204). That is, the correction unit 223 considers that a sound frame group in which the number of consecutive bowel sound frames determined by the determination unit 222 to contain bowel sounds is equal to or less than the first threshold does not contain bowel sounds.

[0062] Next, the correction unit 223 determines whether or not there is a sound frame group in which the number of consecutive non-bow sound frames is equal to or less than a second threshold among the abdominal sound sound frames acquired by the second acquisition unit 221 (step S205). The second threshold is set in advance to, for example, the number of frames corresponding to the minimum, maximum, average, or median of the length of sound that a person can recognize as being interrupted. In other words, a sound frame group in which the number of consecutive non-bow sound frames is equal to or less than the second threshold is considered to be a bowel sound that a person cannot recognize as being interrupted. The second threshold is set to a value greater than the first threshold. Note that the second threshold may be set to a value equal to or less than the first threshold.

[0063] The correction unit 223 may set the second threshold based on the sound frames of the environmental sound acquired by the second acquisition unit 221. In this case, for example, the correction unit 223 measures the intensity of the sound included in the sound frames of the environmental sound, and corrects the second threshold so that the greater the intensity of the environmental sound, the smaller the second threshold becomes, and vice versa. This allows the information processing system 1 to correct the determination result by the determination unit 222 taking into account the influence of the environmental sound, and to more accurately determine the user's intestinal condition.

[0064] If there is no sound frame group in which the number of consecutive non-bowel sound frames is equal to or less than the second threshold, the correction unit 223 does not perform any particular process and proceeds to step S207.

[0065] On the other hand, if there is a sound frame group in which the number of consecutive non-bow sound frames is equal to or less than the second threshold, the correction unit 223 corrects the non-bow sound frames included in that sound frame group to bowel sound frames (step S206). That is, the correction unit 223 considers a sound frame group in which the number of consecutive frames determined by the determination unit 222 not to contain bowel sounds to be equal to or less than the second threshold to contain bowel sounds.

[0066] In this way, the correction unit 223 corrects the determination result by the determination unit 222 based on the number of consecutive times that it is determined that bowel sounds are included or the number of consecutive times that it is determined that bowel sounds are not included.

[0067] Next, the detection unit 224 detects a bowel sound frame group and a non-bow sound frame group from the plurality of abdominal sound frames acquired by the second acquisition unit 221 based on the determination result by the determination unit 222 and the determination result corrected by the correction unit 223 (step S207). The detection unit 224 detects, as a bowel sound frame group, a sound frame group consisting of consecutive sound frames determined to contain bowel sounds among the plurality of abdominal sound frames acquired by the second acquisition unit 221. The detection unit 224 also detects, as a non-bow sound frame group, a sound frame group consisting of consecutive sound frames not containing bowel sounds among the plurality of abdominal sound frames acquired by the second acquisition unit 221.

[0068] Next, the calculation unit 225 calculates frequency information relating to the frequency of bowel sounds in the multiple abdominal sound frames acquired by the second acquisition unit 221, based on the bowel sound frame groups and non-bow sound frame groups detected by the detection unit 224 (step S208). For example, the calculation unit 225 calculates the number of bowel sound frame groups included in each predetermined period (i.e., the number of times a non-bow sound frame group changes to a bowel sound frame group when scanning the sound frames in a predetermined period in chronological order) as frequency information. The predetermined period is set to any period (e.g., 5 seconds). The calculation unit 225 may also calculate the interval between adjacent bowel sound frame groups (the time during which a non-bow sound frame group exists between adjacent bowel sound frame groups) (SSI: Sound-Sound Interval) as frequency information.

[0069] Next, the identification unit 226 identifies bowel information related to the user's bowel condition based on the occurrence frequency calculated by the calculation unit 225 (step S209), and the series of steps ends. The identification unit 226 calculates an abdominal sound score as the bowel information. For example, if the frequency information indicates the number of bowel sound frame groups included in each period, the identification unit 226 calculates the number of periods in which the number of bowel sound frame groups falls within a predetermined range as the abdominal sound score. The predetermined range is preset to the range of the number of bowel sound frame groups within a predetermined period when the bowel condition is good. For example, if the sound frame length is 50 msec and the predetermined period is 5 sec, the predetermined range is set to 5 to 20, etc. If the frequency information indicates the spacing between adjacent bowel sound frame groups, the identification unit 226 calculates the abdominal sound score as the number of bowel sound frame groups whose spacing from their adjacent bowel sound frame group on the rear side falls within a predetermined range. The predetermined range is preset to the range of spacing between bowel sound frame groups when the bowel condition is good. For example, the predetermined range is set to a time period equal to or greater than two frames and equal to or less than 15 frames.

[0070] Furthermore, the determination unit 226 determines whether the intestinal condition (intestinal peristalsis condition) is good or not as intestinal information based on the calculated abdominal sound score. If the abdominal sound score is equal to or greater than the determination threshold, the determination unit 226 determines that the intestinal condition (intestinal peristalsis condition) is good, and if the abdominal sound score is less than the determination threshold, the determination unit 226 determines that the intestinal condition (intestinal peristalsis condition) is poor. The determination threshold is set, through prior experiments, to a value between the score calculated from the abdominal sounds of a person with good intestinal condition and the score calculated from the abdominal sounds of a person with poor intestinal condition.

[0071] In this way, the identification unit 226 identifies intestinal information related to the user's intestinal condition based on the bowel sound frame group and non-bow sound frame group detected by the detection unit 224. By using the user's abdominal sounds, the information processing system 1 can identify the state of the user's intestinal peristalsis with high accuracy, and can identify the user's intestinal condition with high accuracy. Furthermore, the information processing system 1 can identify the user's intestinal condition with even higher accuracy by appropriately correcting the determination result of whether the user's abdominal sounds include bowel sounds in order to identify the user's intestinal condition.

[0072] In particular, the identification unit 226 identifies intestinal information related to the condition of the user's intestines based on the frequency of bowel sounds calculated by the calculation unit 225. By using the frequency of bowel sounds of the user, the information processing system 1 can identify the state of the user's intestinal peristalsis with high accuracy, and can identify the condition of the user's intestines with high accuracy.

[0073] 7A to 7D are schematic diagrams for explaining correction of the bowel sound determination results.

[0074] 7A and 7B show a series of sound frames F01 to F20. In the example shown in FIG. 7A, sound frames F02 to F09 and F12 to F19 are determined by the determination unit 222 to be non-bow sound frames, while sound frames F01, F10 to F11, and F20 are determined by the determination unit 222 to be bowel sound frames. The inventors conducted extensive experiments and discovered that if a group of short sound frames is considered to be a single group of bowel sounds, the interval between successive bowel sounds may become excessively short, potentially resulting in an inaccurate identification of the user's bowel condition. In the information processing system 1, as shown in FIG. 7B, sound frames F10 to F11, for which the number of consecutive frames determined to contain bowel sounds is equal to or less than a first threshold, are deemed not to contain bowel sounds, and sound frames F02 to F19 are concatenated as a non-bow sound frame group. This allows the information processing system 1 to prevent the interval between successive bowel sounds from becoming excessively short, thereby preventing the user's bowel condition from being erroneously identified.

[0075] The inventors have further discovered that if a group of sound frames with a relatively long duration is deemed not to be bowel sounds, the interval between successive bowel sounds may become excessively long, and the user's bowel condition may not be correctly identified. As described above, the first threshold is preferably set to a number of frames corresponding to the length of sound that a human can hear (distinguish), and is preferably set to a range of 3 to 7 frames, for example.

[0076] 7C and 7D show multiple consecutive sound frames F21-F40. In the example shown in FIG. 7C, sound frames F21, F30-F32, and F40 are determined by the determination unit 222 to be non-bow sound frames, while sound frames F22-F29 and F333-F39 are determined by the determination unit 222 to be bowel sound frames. The inventors conducted extensive experiments and discovered that if bowel sounds are divided by short silent periods, the interval between consecutive bowel sounds may become excessively short, potentially leading to an inaccurate identification of the user's bowel condition. In the information processing system 1, as shown in FIG. 7D, sound frames F30-F32, for which the number of consecutive frames determined not to contain bowel sounds is equal to or less than a second threshold, are deemed to contain bowel sounds, and sound frames F22-F39 are concatenated as a bowel sound frame group. This allows the information processing system 1 to prevent the interval between consecutive bowel sounds from becoming excessively short, thereby preventing the user's bowel condition from being erroneously identified.

[0077] Furthermore, the inventors conducted extensive experiments and discovered that when adjacent bowel sounds with a certain large interval between them are combined, the interval between successive bowel sounds becomes excessively long, which may result in the user's bowel condition not being correctly identified. As described above, the second threshold is preferably set to a number of frames corresponding to the length of a sound that a person can recognize as being interrupted, and is preferably set to a range of, for example, 30 to 40 frames.

[0078] As described above in detail, the information processing system 1 corrects the determination result of whether or not bowel sounds are included based on the number of consecutive sound frames determined to include bowel sounds or the number of consecutive sound frames determined not to include bowel sounds, and identifies the user's bowel condition based on the corrected determination result. This enables the information processing system 1 to more accurately determine the user's bowel condition and notify the user more accurately.

[0079] The identification process shown in FIG. 6 may be executed by the terminal device 100 instead of the server device 200. In this case, the terminal device 100 includes a determination unit, correction unit, detection unit, calculation unit, and / or identification unit similar to the determination unit 222, correction unit 223, detection unit 224, calculation unit 225, and / or identification unit 226. The terminal device 100 also stores a trained model similar to the trained model 211 in the first storage device 110. The process of step S103 in FIG. 4 is omitted, and the first processing device 120 of the terminal device 100 executes the identification process in step S104. That is, the determination unit of the terminal device 100 determines whether bowel sounds are included in each of a plurality of sound frames using the trained model stored in the first storage device 110. The correction unit of the terminal device 100 corrects the determination result made by the determination unit based on the number of consecutive sound frames determined to include bowel sounds or the number of consecutive sound frames determined not to include bowel sounds. A detection unit of the terminal device 100 detects a group of bowel sound frames based on the determination result by the determination unit and the determination result corrected by the correction unit. A calculation unit of the terminal device 100 calculates frequency information based on the group of bowel sound frames detected by the detection unit. An identification unit of the terminal device 100 identifies bowel information based on the frequency information. The processing of step S105 is omitted, and in step S105 the first output control unit 122 generates display data for a notification screen for notifying the user of the bowel information identified in the identification process, and displays the notification screen on the display device 103.

[0080] Alternatively, the server device 200 may execute a part of the specific process, and the terminal device 100 may execute the remaining part of the specific process. In addition, in the information processing system 1, a plurality of terminal devices 100 and / or a plurality of server devices 200 may cooperate to share the respective steps of the above-described processes.

[0081] In addition, the processing of step S201 in Figure 6 may be omitted, and in step S202, the judgment unit may input each sound frame itself into the trained model 211 as information based on the sound frame, and determine whether or not each sound frame contains bowel sounds.

[0082] Furthermore, in step S202, the determination unit may determine whether or not each sound frame contains bowel sounds using a trained model stored in an external server device, instead of using the trained model stored in the device itself. In this case, the determination unit transmits information based on the sound frames to the external server device. The external server device inputs the received information into the trained model and returns information output from the trained model. The determination unit determines whether or not each sound frame contains bowel sounds based on the information received from the external server device.

[0083] By utilizing the trained model stored in the external server device, the terminal device 100 or the server device 200 can accurately determine whether or not each sound frame contains bowel sounds using the latest trained model updated by the external server device. Furthermore, the information processing system 1 can reduce the storage capacity of the terminal device 100 or the server device 200. Meanwhile, by utilizing the trained model stored therein, the terminal device 100 or the server device 200 can determine whether or not each sound frame contains bowel sounds even when communication with the external server device is disconnected. Furthermore, the information processing system 1 can reduce the amount of communication between the terminal device 100 or the server device 200 and the external server device.

[0084] Furthermore, in step S202, the determination unit may determine whether each sound frame contains bowel sounds without using a trained model. For example, frequencies corresponding to bowel sounds are set in advance. The determination unit determines a sound frame whose spectrum image has an intensity at a frequency corresponding to bowel sounds equal to or greater than a predetermined level as a bowel sound frame, and determines a sound frame whose spectrum image has an intensity at a frequency corresponding to bowel sounds less than the predetermined level as a non-bow sound frame. Alternatively, a plurality of spectrum images generated from various abdominal sounds containing bowel sounds are stored in advance in a storage device. The determination unit calculates the similarity between the spectrum image generated in step S201 and each spectrum image stored in advance in the storage device. The similarity may be, for example, normalized cross-correlation. The determination unit compares each calculated similarity with a threshold, and determines that the sound frame contains bowel sounds if any of the similarities indicates that the spectrum images are similar, and determines that the sound frame does not contain bowel sounds if all of the similarities indicate that the spectrum images are not similar.

[0085] In addition, either the processing in steps S203 and S204 or the processing in steps S205 and S206 may be omitted.

[0086] 1 Information processing system, 100 Terminal device, 121 First acquisition unit, 122 First output control unit, 200 Server device, 211 Trained model, 221 Second acquisition unit, 222 Determination unit, 223 Correction unit, 224 Detection unit, 225 Calculation unit, 226 Identification unit, 227 Second output control unit

Claims

1. An information processing device comprising: an acquisition unit that acquires multiple consecutive sound frames collected from a user's abdomen; a determination unit that determines whether or not each of the multiple sound frames contains bowel sounds; a correction unit that corrects the determination result made by the determination unit based on the number of consecutive sound frames that are determined to contain bowel sounds or the number of consecutive sound frames that are determined not to contain bowel sounds; a detection unit that detects a group of bowel sound frames made up of consecutive sound frames that are determined to contain bowel sounds among the multiple sound frames based on the determination result made by the determination unit and the determination result corrected by the correction unit; an identification unit that identifies intestinal information related to the user's intestinal condition based on the group of bowel sound frames; and an output unit that outputs the intestinal information.

2. The information processing device according to claim 1, further comprising a calculation unit that calculates frequency information regarding the frequency of bowel sounds occurring in the plurality of sound frames based on the group of bowel sound frames, and the identification unit identifies the bowel information based on the frequency information.

3. The information processing device according to claim 1 or 2, wherein the correction unit considers that a group of sound frames in which the number of consecutive frames determined to contain bowel sounds is equal to or less than a first threshold does not contain bowel sounds.

4. The information processing device according to claim 1 or 2, wherein the correction unit considers a group of sound frames containing no bowel sounds to contain bowel sounds if the number of consecutive sound frames determined not to contain bowel sounds is equal to or less than a second threshold value.

5. An information processing device as described in claim 1 or 2, wherein the determination unit determines whether or not each sound frame contains bowel sounds by inputting information based on each sound frame acquired by the acquisition unit into a trained model that is trained to output information indicating whether or not a sound frame collected from the user's abdomen is included in the input sound frame.

6. An information processing system comprising: an acquisition unit that acquires a plurality of consecutive sound frames collected from the abdomen of a user; a determination unit that determines whether or not each of the plurality of sound frames contains bowel sounds; a correction unit that corrects the determination result made by the determination unit based on the number of consecutive sound frames that are determined to contain bowel sounds or the number of consecutive sound frames that are determined not to contain bowel sounds; a detection unit that detects a group of bowel sound frames made up of consecutive sound frames that are determined to contain bowel sounds among the plurality of sound frames based on the determination result made by the determination unit and the determination result corrected by the correction unit; an identification unit that identifies intestinal information related to the condition of the user's intestines based on the group of bowel sound frames; and an output unit that outputs the intestinal information.

7. An information processing method comprising: acquiring a plurality of consecutive sound frames collected from a user's abdomen; determining whether or not bowel sounds are included for each of the plurality of sound frames; correcting the determination result of whether or not bowel sounds are included based on the number of consecutive sound frames determined to contain bowel sounds or the number of consecutive sound frames determined not to contain bowel sounds; detecting a group of bowel sound frames consisting of consecutive sound frames determined to contain bowel sounds from among the plurality of sound frames based on the determination result and the corrected determination result; identifying intestinal information related to the user's intestinal condition based on the group of bowel sound frames; and outputting the intestinal information.

8. A computer control program that causes a computer to perform the following operations: acquire a plurality of consecutive sound frames collected from the abdomen of a user; determine for each of the plurality of sound frames whether or not bowel sounds are included; correct the determination result of whether or not bowel sounds are included based on the number of consecutive sound frames determined to contain bowel sounds or the number of consecutive sound frames determined not to contain bowel sounds; detect a group of bowel sound frames consisting of consecutive sound frames determined to contain bowel sounds from among the plurality of sound frames based on the determination result and the corrected determination result; identify intestinal information related to the user's intestinal condition based on the group of bowel sound frames; and output the intestinal information.

Images