Information processing method, computer program, information processing system, and information processing device

Abstract

This information processing method, which is executed by a computer, includes: acquiring a reflected sound which is picked up by a sound reception unit (12) and which is generated by a sound output from a sound generation unit (11) hitting a subject and being reflected of the same; calculating the walking speed of the subject on the basis of the acquired reflected sound; and outputting first information that, if the amount of change in the calculated walking speed is at least a predetermined value, indicates that the amount of change is at least a predetermined value.

Description

Information processing method, computer program, information processing system, and information processing device

[0001] The present disclosure relates to an information processing method and the like.

[0002] Conventionally, there has been known a fall detection method (information processing method) for detecting a fall of a target person. Patent Literature 1 discloses a method for detecting a fall of a target person in a target area based on an abnormal sound.

[0003] Japanese Patent Application Laid-Open No. 2019-079472

[0004] However, with the method disclosed in Patent Document 1, it may be difficult to accurately detect a fall of a subject.

[0005] Therefore, an object of the present disclosure is to provide an information processing method and the like that can accurately detect a fall of a subject.

[0006] An information processing method according to one aspect of the present disclosure is an information processing method executed by a computer, and includes acquiring reflected sound that is generated when sound output from a sound generating unit hits a target person and is received by a sound receiving unit, calculating the walking speed of the target person based on the acquired reflected sound, and, if a change in the calculated walking speed is greater than or equal to a predetermined value, outputting first information indicating that the change is greater than or equal to the predetermined value.

[0007] Furthermore, a program according to one aspect of the present disclosure causes a computer to execute the above information processing method.

[0008] In addition, an information processing system according to one aspect of the present disclosure includes a sound receiving unit that acquires reflected sound that is reflected when sound output from a sound generating unit hits a target person, a calculation unit that calculates the walking speed of the target person based on the acquired reflected sound, and an output unit that outputs first information indicating that the amount of change in the calculated walking speed is greater than or equal to a predetermined value when the amount of change is greater than or equal to the predetermined value.

[0009] In addition, an information processing device according to one aspect of the present disclosure includes a communication unit that acquires reflected sound that is generated when sound output from a sound generator hits a target person and is received by a sound receiver, a calculation unit that calculates the walking speed of the target person based on the acquired reflected sound, and an output unit that outputs first information indicating that the amount of change in the calculated walking speed is greater than or equal to a predetermined value when the amount of change is greater than or equal to the predetermined value.

[0010] These comprehensive or specific aspects may be realized as a system, an apparatus, a method, an integrated circuit, a computer program, or a non-transitory recording medium such as a computer-readable CD-ROM, or may be realized as any combination of a system, an apparatus, a method, an integrated circuit, a computer program, and a recording medium.

[0011] According to an information processing method according to one aspect of the present disclosure, a fall of a subject can be detected with high accuracy.

[0012] FIG. 1 is a block diagram showing a functional configuration of an information processing system according to an embodiment. FIG. 2 is a diagram showing a space in which the information processing system according to an embodiment is used. FIG. 3 is a flowchart of an operation example 1 of the information processing system according to an embodiment. FIG. 4 is a diagram showing a space when a first method for calculating relative position coordinates according to an embodiment is used. FIG. 5 is a diagram showing a space viewed from the ceiling when the first method for calculating relative position coordinates according to an embodiment is used. FIG. 6 is a diagram showing a received signal of a sound receiving unit when the first method for calculating relative position coordinates according to an embodiment is used. FIG. 7 is a diagram showing a space when a second method for calculating relative position coordinates according to an embodiment is used. FIG. 8 is a diagram showing a space viewed from the ceiling when the second method for calculating relative position coordinates according to an embodiment is used. FIG. 9 is a diagram for explaining delay-and-sum beamforming when the second method for calculating relative position coordinates according to an embodiment is used. FIG. 10 is a diagram showing coordinates (R ｘ , R ｙ) and the intensity of reflected sound at the coordinates. FIG. 11 is a diagram showing a spatial sound signal indicating spatial sound acquired by a sound receiving unit according to an embodiment. FIG. 12 is a diagram showing a walking sound signal according to an embodiment. FIG. 13 is a diagram showing a spectrogram of walking sound according to an embodiment. FIG. 14 is a flowchart of operation example 2 of an information processing system according to an embodiment. FIG. 15 is a flowchart of operation example 3 of an information processing system according to an embodiment. FIG. 16 is a flowchart of operation example 4 of an information processing system according to an embodiment. FIG. 17 is a flowchart of operation example 5 of an information processing system according to an embodiment.

[0013] (Findings that Form the Basis of the Present Disclosure) Conventionally, a fall detection method (information processing method) for detecting a fall of a subject has been known.

[0014] Patent Document 1 discloses a method for detecting a fall of a subject within a target area based on abnormal sounds.

[0015] More specifically, in Patent Document 1, a sound of a predetermined volume or more is an abnormal sound, and when this abnormal sound is heard, it is determined that the subject has fallen.

[0016] However, even if a subject falls, there are cases where the fall does not produce a sound below a predetermined volume, and the abnormal sound is not detected. Also, even if a subject falls and produces a sound above a predetermined volume, there are cases where the subject is surrounded by objects (such as furniture) that contain materials that easily absorb sound, and the sound is absorbed and becomes less than the predetermined volume, and the abnormal sound is not detected. In these cases, it is difficult to accurately detect the subject's fall.

[0017] Therefore, there is a demand for an information processing method that can accurately detect a subject's fall.

[0018] Furthermore, Patent Document 1 discloses a method for detecting the position of a target person based on footsteps (walking sounds).

[0019] However, depending on the condition of the floor or the condition of the subject's footwear, the walking sound may not be detected. For example, if the floor on which the subject walks is covered with soft carpet or if the subject is wearing socks, the volume of the walking sound will be low and it will be difficult to detect. As a result, it will be difficult to accurately detect the subject's position.

[0020] In a fall detection method (information processing method), the position at which a subject has fallen is important information. Therefore, there is a demand for an information processing method that can accurately detect the position of a subject.

[0021] The information processing method according to the first aspect is an information processing method executed by a computer, and includes acquiring reflected sound that is generated when sound output from a sound generating unit hits a target person and is received by a sound receiving unit, calculating the walking speed of the target person based on the acquired reflected sound, and, if a change in the calculated walking speed is greater than or equal to a predetermined value, outputting first information indicating that the change is greater than or equal to the predetermined value.

[0022] As a result, the walking speed of the subject is calculated based on the reflected sound, and when the amount of change in walking speed is equal to or greater than a predetermined value, first information is output. When the amount of change in walking speed is equal to or greater than a predetermined value, this corresponds to when the subject has fallen while walking. In this way, whether or not the subject has fallen is determined based on the reflected sound, so that the subject's fall can be accurately detected even when, for example, the volume of the sound caused by the subject's fall is less than a predetermined volume. In other words, an information processing method that can accurately detect when a subject has fallen is realized.

[0023] In the information processing method according to the second aspect, in the first aspect, the sound receiving unit is placed in a space in which the subject is present, and the calculating step calculates relative position coordinates between the subject and the sound receiving unit based on the acquired reflected sound, and calculates the walking speed based on the calculated relative position coordinates.

[0024] This allows the relative position coordinates to be calculated based on the reflected sound, thereby realizing an information processing method that can accurately detect the position (relative position coordinates) of the target person even when, for example, the volume of the target person's walking sound is low.

[0025] In the information processing method according to the third aspect, in the second aspect, when the calculated amount of change in the walking speed is equal to or greater than the predetermined value, the first information indicating that the amount of change in the calculated relative position coordinates is equal to or greater than the predetermined value is output.

[0026] As a result, first information indicating that the amount of change in walking speed at the calculated relative position coordinates is equal to or greater than a predetermined value is output, i.e., an information processing method is realized that can output first information indicating that the subject has fallen while walking at the calculated relative position coordinates.

[0027] The information processing method according to the fourth aspect is any one of the first to third aspects and includes determining, in which the acquiring step acquires the walking sound of the subject, the determining step determines whether the acquired walking sound includes the sound of a fall, and the outputting step outputs the first information indicating that the walking sound also includes the sound of a fall if the acquired walking sound includes the sound of a fall.

[0028] This determines whether the walking sounds include the sound of a fall, and if the walking sounds include the sound of a fall, the first information is further output. A case where the walking sounds include the sound of a fall corresponds to a case where the subject has fallen while walking. In this way, whether the subject has fallen is determined based on the walking sounds as well as the reflected sounds, so that the subject's fall can be detected with greater accuracy. In other words, an information processing method is realized that can detect the subject's fall with greater accuracy.

[0029] In the information processing method according to the fifth aspect, in the fourth aspect, the determination and the output are performed when the calculated amount of change in the walking speed of the subject is less than the predetermined value.

[0030] As a result, even if it is determined based on the reflected sound that the subject has not fallen, it is possible to determine whether or not the subject has fallen based on the walking sound, thereby making it possible to more accurately detect whether the subject has fallen.In other words, an information processing method is realized that can more accurately detect whether the subject has fallen.

[0031] In the information processing method according to the sixth aspect, in the fourth aspect, when the amplitude of the waveform of a walking sound signal indicating the acquired walking sound exceeds a predetermined threshold, it is determined that the acquired walking sound includes the falling sound.

[0032] This makes it possible to determine whether or not the subject has fallen based on the waveform of the walking sound signal.

[0033] In the information processing method according to the seventh aspect, in the fourth aspect, the determination involves determining whether the acquired walking sound includes the falling sound using a trained first learning model that inputs a spectrogram of the acquired walking sound and outputs second information indicating whether the acquired walking sound includes the falling sound.

[0034] This makes it possible to determine whether or not the subject has fallen based on the spectrogram of the walking sound.

[0035] An information processing method according to an eighth aspect is any one of the first to seventh aspects, wherein the sound is ultrasonic.

[0036] This allows for the determination of whether or not the subject has fallen using ultrasound, which is a sound that the subject cannot hear. By using ultrasound, the subject is less likely to feel annoyed by the sound than when, for example, audible sound is used.

[0037] In the information processing method according to a ninth aspect, in the third aspect, the outputting includes outputting the first information based on the calculated relative position coordinates being a dangerous walking location.

[0038] This realizes an information processing method that can output first information indicating that a subject has fallen while walking at the calculated relative position coordinates and that the relative position coordinates are a dangerous walking location.

[0039] In the information processing method according to the tenth aspect, in the third aspect, the outputting includes outputting the first information indicating that the calculated relative position coordinates are a candidate pedestrian dangerous area each time the amount of change is equal to or greater than the predetermined value, and if the number of times that the relative position coordinates are the candidate pedestrian dangerous area in the outputted first information is equal to or greater than a predetermined number, outputting the first information as if the calculated relative position coordinates are a pedestrian dangerous area.

[0040] As a result, when the amount of change in the relative position coordinates is equal to or greater than a predetermined value a predetermined number of times or more, first information indicating that the relative position coordinates are a dangerous place for walking is output. That is, when a predetermined number of falls or more have occurred at the relative position coordinates, an information processing method is realized in which the relative position coordinates can be determined to be a dangerous place for walking and output.

[0041] The information processing method according to the eleventh aspect includes, in the sixth aspect, determining the characteristics using a trained second learning model that inputs the acquired spectrogram of the walking sound and outputs the walking characteristics of the subject, and determining the predetermined threshold based on the determined characteristics.

[0042] This allows a predetermined threshold value according to the walking of the subject to be determined based on the spectrogram of the walking sound.

[0043] In the information processing method according to the twelfth aspect, in the ninth aspect, after the first information is output indicating that the calculated relative position coordinates are the pedestrian danger area, the calculating of the relative position coordinates is performed again, and the information processing method includes notifying the target person when the distance between the recalculated relative position coordinates and the relative position coordinates that are determined to be the pedestrian danger area based on the output first information is equal to or shorter than a predetermined distance.

[0044] This realizes an information processing method that can notify the subject when the distance is equal to or less than a predetermined distance. The relative position coordinates that are determined to be a dangerous walking location are the coordinates where the change in the subject's walking speed previously exceeded a predetermined value, and are the location where the subject fell. Furthermore, the recalculated relative position coordinates are the current location of the subject. Therefore, a case where the distance is equal to or less than a predetermined distance corresponds to a case where the subject is approaching the location where he or she previously fell. In such a case, a notification is issued to notify the subject of danger. This realizes an information processing method that can make the subject aware of danger.

[0045] The information processing method according to the thirteenth aspect includes a first change that changes the manner of the notification when the amount of change in the relative position coordinates recalculated after the notification is performed is equal to or greater than the predetermined value in the twelfth aspect.

[0046] This realizes an information processing method that can more easily alert a target person to danger by changing the notification mode, for example, by increasing the volume of the notification sound.

[0047] The information processing method according to the fourteenth aspect includes, in the twelfth or thirteenth aspect, a second change that increases the frequency of the sound output by the sound generator after the notification is performed.

[0048] This allows the reflected sound to be acquired again with the frequency of sound output increased, thereby realizing an information processing method that can detect the subject's fall more frequently and accurately.

[0049] A computer program according to a fifteenth aspect causes a computer to execute the information processing method according to any one of the first to fourteenth aspects.

[0050] This allows the computer to execute the above-described information processing method in accordance with the computer program.

[0051] The information processing system according to the sixteenth aspect includes a sound receiving unit that acquires reflected sound that is generated when sound output from a sound generating unit hits a target person, a calculation unit that calculates the walking speed of the target person based on the acquired reflected sound, and an output unit that outputs first information indicating that the amount of change in the calculated walking speed is greater than or equal to a predetermined value when the amount of change is greater than or equal to the predetermined value.

[0052] As a result, the walking speed of the subject is calculated based on the reflected sound, and when the amount of change in walking speed is equal to or greater than a predetermined value, first information is output. When the amount of change in walking speed is equal to or greater than a predetermined value, this corresponds to when the subject has fallen while walking. In this way, whether or not the subject has fallen is determined based on the reflected sound, so that the subject's fall can be accurately detected even when, for example, the volume of the sound caused by the subject's fall is less than a predetermined volume. In other words, an information processing system that can accurately detect when a subject has fallen is realized.

[0053] The information processing device according to the seventeenth aspect includes a communication unit that acquires reflected sound that is generated when sound output from a sound generator hits a target person and is received by a sound receiver, a calculation unit that calculates the walking speed of the target person based on the acquired reflected sound, and an output unit that outputs first information indicating that the amount of change in the calculated walking speed is greater than or equal to a predetermined value when the amount of change is greater than or equal to the predetermined value.

[0054] As a result, the walking speed of the subject is calculated based on the reflected sound, and when the amount of change in walking speed is equal to or greater than a predetermined value, first information is output. When the amount of change in walking speed is equal to or greater than a predetermined value, this corresponds to when the subject has fallen while walking. In this way, whether or not the subject has fallen is determined based on the reflected sound, so that the subject's fall can be accurately detected even when, for example, the volume of the sound caused by the subject's fall is less than a predetermined volume. In other words, an information processing device that can accurately detect when the subject has fallen is realized.

[0055] Furthermore, these comprehensive or specific aspects may be realized as a system, an apparatus, a method, an integrated circuit, a computer program, or a non-transitory recording medium such as a computer-readable CD-ROM, or may be realized as any combination of a system, an apparatus, a method, an integrated circuit, a computer program, and a recording medium.

[0056] Hereinafter, the embodiments will be specifically described with reference to the drawings.

[0057] The embodiments described below are all comprehensive or specific examples, and the numerical values, shapes, materials, components, arrangement and connection of the components, steps, and order of steps shown in the following embodiments are merely examples and are not intended to limit the scope of the claims.

[0058] In the following description, elements may be assigned ordinal numbers such as first and second. These ordinal numbers are assigned to elements in order to identify them and do not necessarily correspond to a meaningful order. These ordinal numbers may be rearranged, newly added, or removed as appropriate.

[0059] Furthermore, each figure is a schematic diagram and is not necessarily an exact illustration. Therefore, the scales and the like do not necessarily match in each figure. In each figure, the same reference numerals are used to denote substantially the same components, and redundant explanations will be omitted or simplified.

[0060] In this specification, terms indicating relationships between elements such as verticality, and numerical ranges, are not expressions that only express a strict meaning, but also expressions that include a substantially equivalent range, for example, a difference of about a few percent.

[0061] (Embodiment 1) [Configuration] First, a configuration of an information processing system 100 according to embodiment 1 will be described. Fig. 1 is a block diagram showing the functional configuration of the information processing system 100 according to this embodiment. Fig. 2 is a diagram showing a space 90 in which the information processing system 100 according to this embodiment is used.

[0062] The information processing system 100 according to this embodiment is a system that detects a fall of a subject T present in a space 90. The information processing system 100 is also a system that detects the position of the subject T, and outputs and stores the position where the subject T has fallen.

[0063] The space 90 is, for example, an indoor space. The space 90 according to the present embodiment is a residence, but is not limited to this. The space 90 may be an office, a public facility such as a community center or a library, a facility for the elderly, a store, or a commercial facility.

[0064] The subject T may be any person as long as he or she is present in the space 90. If the space 90 is a residence, the subject T may be, for example, a resident of the residence.

[0065] The information processing system 100 includes a detection device 10 and an information processing device 20 .

[0066] The detection device 10 is a device installed in a space 90. The detection device 10 communicates with an information processing device 20 via an information communication network 80 such as the Internet. The information processing device 20 may be installed in the space 90, but in this embodiment, it is installed outside the space 90, and more specifically, it is installed in a remote location different from the space 90.

[0067] The detection device 10 is a device having a sound generating unit 11 , a sound receiving unit 12 , a first communication unit 13 , a control unit 14 , and a first storage unit 15 .

[0068] The sound output unit 11 is a speaker that outputs sound. The sound output by the sound output unit 11 may be an infrasound, an audible sound, or an ultrasonic sound, but in this embodiment, it is an ultrasonic sound.

[0069] The ultrasonic waves output by the sound generator 11 are chirp signals or burst waves of 20 kHz or higher. The sound generator 11 outputs ultrasonic waves at a predetermined cycle. The predetermined cycle is, for example, 1 ms to 30 ms, e.g., 15 ms.

[0070] Furthermore, when a predetermined condition is satisfied, the sound output unit 11 issues a notification, that is, outputs a notification sound. The sound output unit 11 is an example of a notification unit.

[0071] The sound receiving unit 12 is a microphone that receives reflected sound, which is sound that is reflected when the sound (ultrasound) output by the sound generating unit 11 hits the subject T. The sound receiving unit 12 is preferably a microphone with two or more channels, and is preferably a directional microphone.

[0072] The sound receiving unit 12 also receives spatial sound, which is sound generated in the space 90. The spatial sound includes the walking sound of the subject T, that is, the sound receiving unit 12 receives the walking sound of the subject T. The walking sound of the subject T is the footsteps of the subject T.

[0073] The first communication unit 13 is a communication module (communication circuit) that enables the detection device 10 to communicate with the information processing device 20 via the information communication network 80. The first communication unit 13, for example, outputs reflected sound information indicating reflected sound received by the sound receiving unit 12 to the information processing device 20. The first communication unit 13 may output reflected sound information each time reflected sound is received by the sound receiving unit 12, i.e., sequentially outputs reflected sound information. The first communication unit 13 also outputs spatial sound information indicating spatial sound received by the sound receiving unit 12 to the information processing device 20. The first communication unit 13 may output spatial sound information each time spatial sound is received by the sound receiving unit 12, i.e., sequentially outputs spatial sound information. As described above, the spatial sound includes the walking sound of the subject T, and therefore the spatial sound information includes the walking sound. The communication performed by the first communication unit 13 is, for example, wireless communication, but may also be wired communication. The communication standard used for communication is not particularly limited.

[0074] The control unit 14 controls the sound generating unit 11, the sound receiving unit 12, the first communication unit 13, and the first storage unit 15. The control unit 14, for example, causes the sound generating unit 11 to output sound, causes the sound receiving unit 12 to receive reflected sound, causes the first communication unit 13 to output reflected sound information and spatial sound information, or causes the first storage unit 15 to store predetermined information. The control unit 14 is realized, for example, by a microcomputer, but may also be realized by a processor.

[0075] The first storage unit 15 is a storage device that stores programs and the like executed by the control unit 14. The first storage unit 15 is realized by, for example, a semiconductor memory.

[0076] The information processing device 20 is a device that includes a second communication unit 21, an information processing unit 22, and a second storage unit 23, and is an example of a computer.

[0077] The second communication unit 21 is a communication module (communication circuit) that enables the information processing device 20 to communicate with the detection device 10 via the information communication network 80. The second communication unit 21, for example, acquires reflected sound information and spatial sound information output from the first communication unit 13. More specifically, the second communication unit 21, for example, sequentially acquires reflected sound information and sequentially acquires spatial sound information. In this way, the second communication unit 21 acquires reflected sound and spatial sound. The communication performed by the second communication unit 21 is, for example, wireless communication, but may also be wired communication. There are no particular limitations on the communication standard used for the communication.

[0078] The information processing unit 22 controls the second communication unit 21 and the second storage unit 23. For example, the information processing unit 22 causes the second communication unit 21 to acquire reflected sound information and spatial sound information, and stores predetermined information in the second storage unit 23. The information processing unit 22 is realized by, for example, a microcomputer, but may also be realized by a processor.

[0079] The information processing unit 22 includes a calculation unit 221 , a determination unit 222 , a decision unit 223 , and an output unit 224 .

[0080] The calculation unit 221 calculates the walking speed of the subject T based on the reflected sound acquired by the sound receiving unit 12. Here, the calculation unit 221 calculates the walking speed of the subject T based on the reflected sound indicated by the reflected sound information acquired by the second communication unit 21.

[0081] Furthermore, the calculation unit 221 determines whether the calculated amount of change in walking speed is equal to or greater than a predetermined value. The amount of change in walking speed is a value obtained by subjecting the walking speed to a time derivative, and can be said to be the acceleration of the subject T.

[0082] Here, the relationship between the amount of change in the walking speed of the subject T and falls will be explained. When the subject T is walking normally, that is, when the subject T is walking without falling, the walking speed falls within a predetermined range, and the amount of change in the walking speed is less than a predetermined value. However, when the subject T falls while walking, the walking speed changes suddenly, that is, the amount of change in the walking speed increases suddenly and becomes equal to or exceeds a predetermined value. In other words, when the calculation unit 221 determines that the amount of change in the walking speed is equal to or greater than the predetermined value, this corresponds to a case in which the subject T has fallen.

[0083] The determination unit 222 determines whether the acquired walking sounds include the sound of a fall. Here, the determination unit 222 determines whether the walking sounds included in the spatial sound information acquired by the second communication unit 21 include the sound of a fall.

[0084] The determination unit 223 determines the walking characteristics of the subject T. The determination unit 223 determines that the walking characteristics of the subject T are, for example, shuffling or normal walking.

[0085] When the amount of change in the walking speed calculated by the calculation unit 221 is equal to or greater than a predetermined value, the output unit 224 outputs first information indicating that the amount of change is equal to or greater than a predetermined value.

[0086] The second storage unit 23 is a storage device that stores programs executed by the information processing unit 22. The second storage unit 23 is realized by, for example, a semiconductor memory. The second storage unit 23 may store, for example, the predetermined value.

[0087] The second storage unit 23 stores the first information output from the output unit 224. That is, when the amount of change in walking speed calculated by the information processing unit 22 (more specifically, the calculation unit 221) is equal to or greater than a predetermined value, the second storage unit 23 stores the first information indicating that the amount of change is equal to or greater than the predetermined value.

[0088] In the present embodiment, unless otherwise specified, the first information output from the output unit 224 is stored in the second storage unit 23. In addition, the first information output from the output unit 224 may be displayed on a display unit (not shown) or the like.

[0089] Hereinafter, operation examples 1 to 5 of the information processing method performed by the information processing system 100 will be described.

[0090] [Operation Example 1] FIG. 3 is a flowchart of operation example 1 of the information processing system 100 according to this embodiment.

[0091] First, the sound generator 11 outputs sound (ultrasound waves) (S10). The sound generator 11 outputs ultrasound waves at a predetermined cycle. The output ultrasound waves are reflected by the subject T present in the space 90.

[0092] Next, the sound receiving unit 12 acquires reflected sound and spatial sound reflected by hitting the target person T (S12). Because the sound generating unit 11 outputs ultrasonic waves at a predetermined cycle, the sound receiving unit 12 acquires reflected sound at the predetermined cycle. Furthermore, the sound receiving unit 12 acquires spatial sound including walking sound at the predetermined cycle, but the cycle for acquiring walking sound is not limited to this.

[0093] The first communication unit 13 sequentially outputs reflected sound information and sequentially outputs spatial sound information. The second communication unit 21 sequentially acquires reflected sound information and sequentially acquires spatial sound information. In other words, the second communication unit 21 acquires reflected sound and spatial sound.

[0094] The information processing unit 22 (more specifically, the calculation unit 221) calculates the walking speed of the subject T based on the reflected sound indicated by the reflected sound information acquired by the second communication unit 21. The method by which the calculation unit 221 calculates the walking speed is not particularly limited, and a known method can be used, but for example, a method utilizing the Doppler effect may also be used. Furthermore, in this embodiment, the calculation unit 221 calculates the walking speed using the following method.

[0095] The calculation unit 221 calculates the relative position coordinates between the subject T and the sound receiving unit 12 based on the acquired reflected sound, and calculates the walking speed based on the calculated relative position coordinates (S14).

[0096] First, the calculation of relative position coordinates will be described. Here, the relative position coordinates are calculated using the first method or the second method. First, the first method will be described.

[0097] Fig. 4 is a diagram showing a space 90 when the first method for calculating relative position coordinates according to the present embodiment is used. Fig. 5 is a diagram showing the space 90 when the first method for calculating relative position coordinates according to the present embodiment is used, as viewed from the ceiling.

[0098] The first method calculates relative position coordinates by calculating the time difference due to rising detection. In the first method, two-channel microphones M1 and M2, which are the sound receiving unit 12, are arranged in a straight line on the ceiling of the space 90. Each of the microphones M1 and M2 picks up reflected sounds.

[0099] 6 is a diagram showing a signal received by the sound receiving unit 12 when the first method for calculating relative position coordinates according to the present embodiment is used. More specifically, FIG. 6 is a diagram showing reflected sound signals indicating reflected sounds acquired by each of the microphones M1 and M2 of the sound receiving unit 12.

[0100] Here, the distance between microphones M1 and M2 is d [m], the speed of sound is c [m / s], the direction from which the reflected sound arrives is θ, and the difference between the rise time of the reflected sound signal at microphone M1 and the rise time of the reflected sound signal at microphone M2 is Δt [s]. In this case, θ satisfies the following formula:

[0101] θ = cos －１ (Δtc / d)

[0102] That is, since θ is calculated from Δt, c, and d, the arrival direction of the reflected sound to the sound receiving unit 12 (microphones M1 and M2) becomes clear. Since the subject T is standing or sitting on the floor of the space 90, the relative position coordinates between the subject T and the sound receiving unit 12 are calculated when viewed from the ceiling, that is, on a plan view of the space 90, as shown in Fig. 5. More specifically, the position where the arrival direction of the reflected sound on the straight line on which the microphones M1 and M2 are arranged intersects with the floor of the space 90 is calculated as the relative position coordinate. Furthermore, even when microphones of three or more channels are arranged in a straight line, the relative position coordinates are calculated in a similar manner.

[0103] Next, a second method for calculating relative position coordinates will be described.

[0104] Fig. 7 is a diagram showing a space 90 when the second method for calculating relative position coordinates according to the present embodiment is used. Fig. 8 is a diagram showing the space 90 when the second method for calculating relative position coordinates according to the present embodiment is used, as viewed from the ceiling.

[0105] The second method calculates relative position coordinates by calculating the arrival position of the reflected sound using beamforming. In the second method, microphones Mi (i is an integer equal to or greater than 3) of multiple channels, which are the sound receiving unit 12, are arranged in a non-linear manner on the ceiling of the space 90. The microphones Mi may be arranged, for example, randomly or in a matrix on the ceiling of the space 90. Each microphone Mi acquires reflected sound. Also, FIG. 8 illustrates multiple microphones M1, M2, M3 to Mm (m is an example of i).

[0106] FIG. 9 is a diagram for explaining delay-and-sum beamforming when the second method for calculating relative position coordinates according to this embodiment is used.

[0107] Delay-and-sum beamforming forms a beam by adding a specific delay to the received signal of each microphone Mi. Here, the output signal is denoted by y, and the reflected sound signal (received signal) indicating the reflected sound acquired by the i-th microphone (i.e., microphone Mi) is denoted by x. ｉ , x ｉ The delay time of τ ｉ , and time is t. In this case, y satisfies the following formula.

[0108]

[0109] Furthermore, the view (plan view) of the space 90 shown in FIG. 8 as seen from the ceiling is divided into a plurality of analysis target points, and the coordinates of the analysis target points are calculated as (R ｘ , R ｙ ) in FIG. 10. The coordinates of the analysis target point according to this embodiment are (R ｘ , R ｙ ) and the intensity of the reflected sound at the coordinates. ｘ , R ｙ10B shows the intensity of the reflected sound at the coordinates, more specifically, the intensity distribution of the reflected sound in a planar view of the space 90.

[0110] Also, the i-th microphone (i.e., microphone M ｉ ) coordinates (M ｉｘ , M ｉｙ ) In this case, the following equation holds:

[0111]

[0112] In the second method, the position showing the maximum intensity in the intensity distribution of the reflected sound calculated by the above two equations and shown in FIG. 10(b) is calculated as the relative position coordinate.

[0113] As described above, in this embodiment, the relative position coordinates may be calculated using the first method or the second method.

[0114] Then, each time the second communication unit 21 acquires reflected sound information, the calculation unit 221 calculates the relative position coordinates between the subject T and the sound receiving unit 12. The calculation unit 221 calculates the walking speed of the subject T by performing a time differentiation process on the relative position coordinates.

[0115] Returning to FIG. 3 again, the explanation will be given.

[0116] Next, the information processing unit 22 (more specifically, the calculation unit 221) determines whether the calculated amount of change in walking speed is equal to or greater than a predetermined value (S16).

[0117] Then, if the calculated amount of change in walking speed is equal to or greater than a predetermined value (Yes in step S16), the output unit 224 outputs first information indicating that the amount of change is equal to or greater than the predetermined value (S18). Then, the first information is stored in the second storage unit 23. A case in which it is determined that the amount of change in walking speed is equal to or greater than the predetermined value corresponds to a case in which the subject T has fallen, and therefore the first information also indicates that the subject T has fallen in the space 90. In other words, by performing the processing of step S18, information (first information) indicating that the subject T has fallen in the space 90 is output and stored.

[0118] Next, the process performed in step S18 will be described in detail.

[0119] As shown in this operation example, when the walking speed is calculated based on the relative position coordinates and the amount of change in the calculated walking speed is equal to or greater than a predetermined value, the output unit 224 outputs first information indicating that the amount of change in the calculated relative position coordinates is equal to or greater than the predetermined value (S18a). Then, the first information is stored in the second storage unit 23. This first information is information indicating that the subject T has fallen at the relative position coordinates in the space 90.

[0120] Furthermore, the output unit 224 determines that the relative position coordinates indicated by this first information are a dangerous walking location and outputs the first information (S18b). Then, the first information is stored in the second storage unit 23. Since the relative position coordinates are calculated each time reflected sound information is acquired, multiple relative position coordinates are stored in the second storage unit 23. In step S18b, the one relative position coordinate where the subject T fell is determined to be a dangerous walking location, and the first information is stored in the second storage unit 23.

[0121] Furthermore, if the amount of change in walking speed calculated by the information processing unit 22 is less than a predetermined value (No in step S16), the judgment unit 222 judges whether the acquired walking sounds include the sound of a fall (S20).

[0122] Here, a method for determining whether or not a walking sound includes a falling sound will be described. Here, the third or fourth method is used to determine whether or not a walking sound includes a falling sound.

[0123] First, the third method will be described. Fig. 11 is a diagram showing a spatial sound signal indicating spatial sound acquired by the sound receiving unit 12 according to this embodiment. The spatial sound includes the walking sound of the subject T. The determination unit 222 calculates (extracts) a walking sound signal indicating the walking sound by performing low-pass filtering processing on the spatial sound signal. Fig. 12 is a diagram showing the walking sound signal according to this embodiment.

[0124] The walking sound signal shown in FIG. 12 is a signal indicating six falls. When a fall occurs, the intensity of the walking sound signal changes; more specifically, the intensity of the walking sound signal increases. The determination unit 222 determines that the acquired walking sound includes a falling sound when the amplitude of the waveform of the walking sound signal indicating the acquired walking sound exceeds a predetermined threshold. For example, if the predetermined threshold is +0.0025, the determination unit 222 determines that a fall has occurred, that is, the walking sound includes a falling sound, when the amplitude exceeds +0.0025. Alternatively, the determination unit 222 may determine that the walking sound includes a falling sound when the absolute value of the amplitude of the waveform exceeds a predetermined threshold. In this case, when the predetermined threshold is +0.0025, the determination unit 222 may determine that a fall has occurred, that is, the walking sound includes a falling sound, when the amplitude exceeds +0.0025 or is smaller than -0.0025.

[0125] Next, a fourth method will be described. In the fourth method, the determination unit 222 determines whether the walking sound includes the falling sound by using the trained first learning model.

[0126] The trained first learning model is a trained model, more specifically, a deep learning model, that receives a spectrogram of the acquired walking sound as input and outputs second information indicating whether the acquired walking sound includes a fall sound. In the fourth method, the determination unit 222 holds this first learning model.

[0127] Fig. 13 is a diagram showing a spectrogram of walking sounds according to this embodiment. The diagram shown in Fig. 13 is obtained by performing short-time Fourier transform processing on the diagram shown in Fig. 12.

[0128] This first learning model is a model for generating second information indicating whether or not the walking sound includes the sound of a fall.

[0129] In this embodiment, the first learning model is a model constructed by machine learning using one or more datasets as training data, and is a pre-trained model. Here, one dataset is composed of a combination of a spectrogram of walking sounds and information on the occurrence of falls in the spectrogram.

[0130] In other words, the first learning model is a recognition model constructed by machine learning using one or more datasets as training data. More specifically, the first learning model is a recognition model constructed using, as input data, spectrograms of walking sounds belonging to each of the one or more datasets that are the training data, and using, as output data, second information indicating whether the walking sounds in the spectrograms belonging to the datasets include falling sounds. The first learning model is trained using machine learning as described above, as an example.

[0131] The first learning model may be trained using, for example, XGBoost, Multi-Layer Perceptron (MLP), Random Forest, etc. The first learning model may also be trained using a learning algorithm other than those described above.

[0132] The determination unit 222 inputs the acquired spectrogram of the walking sound into the first learning model. As a result, second information is output from the first learning model. If the walking sound in the second information output from the first learning model includes a falling sound, the determination unit 222 determines that the walking sound includes a falling sound.

[0133] As described above, in this embodiment, whether or not walking sounds include falling sounds is determined using the third or fourth method.

[0134] If it is determined that the walking sounds include the sound of a fall (Yes in step S20), the process of step S18 is performed. At this time, the output unit 224 outputs first information indicating that the walking sounds include the sound of a fall. Then, the first information is stored in the second storage unit 23. In other words, information (first information) indicating that the subject T has fallen in the space 90 is stored.

[0135] Furthermore, if it is determined that the walking sounds do not include the falling sounds (No in step S20), the process of step S12 is performed again.

[0136] In this way, in Operation Example 1, the walking speed of the subject T is calculated based on the reflected sound, and when the amount of change in walking speed is equal to or greater than a predetermined value, the first information is output and stored. A case where the amount of change in walking speed is equal to or greater than a predetermined value corresponds to a case where the subject T has fallen while walking. In Operation Example 1, because it is determined whether or not the subject T has fallen based on the reflected sound, it is possible to accurately detect the subject T's fall even if, for example, the volume of the sound caused by the subject T's fall is less than a predetermined volume. In other words, an information processing method is realized that can accurately detect the subject T's fall.

[0137] [Operation Example 2] FIG. 14 is a flowchart of operation example 2 of the information processing system 100 according to this embodiment.

[0138] In the second operation example, unlike the first operation example, when a predetermined number of falls or more have occurred at a certain relative position coordinate, the relative position coordinate is determined to be a dangerous walking location and output.

[0139] 14, the process is performed up to step S16 shown in Operation Example 1. Also in Operation Example 2, similarly to Operation Example 1, the walking speed is calculated based on the relative position coordinates.

[0140] If the calculated change in walking speed is equal to or greater than the predetermined value (Yes in step S16), the output unit 224 outputs first information indicating that the change is equal to or greater than the predetermined value (S118).Then, the second storage unit 23 stores the first information.

[0141] Next, the process performed in step S118 will be described in detail.

[0142] As shown in this operation example, when the walking speed is calculated based on the relative position coordinates and the amount of change in the calculated walking speed is equal to or greater than a predetermined value, the output unit 224 outputs first information indicating that the amount of change in the calculated relative position coordinates is equal to or greater than a predetermined value (S18a).Then, the second storage unit 23 stores the first information.

[0143] Furthermore, in this case, the output unit 224 outputs first information indicating that the amount of change in the calculated relative position coordinates is equal to or greater than a predetermined value, and indicating that the relative position coordinates are a candidate pedestrian dangerous area (S18c). The second storage unit 23 stores the first information. That is, each time the amount of change equal to or greater than the predetermined value occurs, first information indicating that the calculated relative position coordinates are a candidate pedestrian dangerous area is output and stored. In other words, the number of times that the amount of change equal to or greater than the predetermined value occurs is the same as the number of times that the relative position coordinates indicated in the first information are a candidate pedestrian dangerous area. For example, when the amount of change equal to or greater than a predetermined value occurs once in a certain relative position coordinate, the relative position coordinates are output and stored once to indicate that the relative position coordinates are a candidate pedestrian dangerous area. Furthermore, when the amount of change equal to or greater than the predetermined value occurs again in the relative position coordinates, the first information indicating that the relative position coordinates are a candidate pedestrian dangerous area is output and stored once more, for a total of two times.

[0144] Next, the information processing unit 22 determines whether the number of times that the relative position coordinates indicate that the relative position coordinates are a candidate for a dangerous walking location in the first information output in step SS18c is equal to or greater than a predetermined number (S18d). The predetermined number may be, for example, a number determined by the administrator of the information processing system 100, but is not limited to this, and may be, for example, 2 to 20.

[0145] If the number of times that the relative position coordinates in the output first information indicate that the location is a potential dangerous location is equal to or greater than a predetermined number (Yes in step S18d), the output unit 224 determines that the relative position coordinates indicated by the first information are a dangerous location and outputs the first information (S18b).Then, the second storage unit 23 stores the first information.

[0146] In Operation Example 1, if the amount of change in relative position coordinates is greater than or equal to a predetermined value once, first information indicating that the relative position coordinates are a dangerous place to walk is output and stored. In Operation Example 2, if the amount of change in relative position coordinates is greater than or equal to a predetermined value a predetermined number of times or more, first information indicating that the relative position coordinates are a dangerous place to walk is output and stored. That is, in Operation Example 2, if a predetermined number of falls or more occur at the relative position coordinates, the relative position coordinates are determined to be a dangerous place to walk, output, and store. That is, in Operation Example 2, a more dangerous relative position coordinate is determined to be a dangerous place to walk, output, and store.

[0147] Also, if step S20 is performed and the answer is Yes, the process of step S118 is performed.

[0148] In addition, if the number of times that the relative position coordinates indicate that the relative position coordinates are a candidate for a dangerous walking location in the output first information is less than the predetermined number of times (No in step S18d), the process of step S12 is performed again.

[0149] [Operation Example 3] FIG. 15 is a flowchart of operation example 3 of the information processing system 100 according to this embodiment.

[0150] In the third operation example, the predetermined threshold value used in the third method according to the first operation example is determined by the determination unit 223 .

[0151] As shown in FIG. 15, the processes up to step S14 shown in the first operational example are carried out.

[0152] Then, the determination unit 223 determines the walking characteristics of the subject T using a trained second learning model that inputs the acquired spectrogram of the walking sound and outputs the walking characteristics of the subject T (S222). The walking characteristics of the subject T include, for example, shuffling or normal walking.

[0153] The trained second learning model is a trained model, more specifically, a deep learning model, that takes the acquired spectrogram of walking sounds as input and outputs the walking characteristics of the subject T. In this operation example, the determination unit 223 holds this second learning model. In the second learning model, the spectrogram that is used as input is the same as the spectrogram shown in FIG. 13 .

[0154] This second learning model is a model for generating the walking characteristics of subject T.

[0155] In this embodiment, the second learning model is a model constructed by machine learning using one or more datasets as training data, and is a pre-trained model. Here, one dataset is composed of a combination of a spectrogram of walking sounds and the walking characteristics of the subject T in the spectrogram.

[0156] In other words, the second learning model is a recognition model constructed by machine learning using one or more datasets as training data. More specifically, the second learning model is a recognition model constructed using spectrograms of walking sounds belonging to each of the one or more datasets that are the training data as input data and gait characteristics of the subject T in the spectrograms belonging to the datasets as output data. The second learning model is trained using machine learning as described above, as an example.

[0157] The second learning model may be trained using, for example, XGBoost, Multi-Layer Perceptron (MLP), Random Forest, etc. The second learning model may also be trained using a learning algorithm other than those described above.

[0158] The determination unit 223 inputs the acquired spectrogram of the walking sound into the second learning model. As a result, the walking characteristics of the subject T are output from the second learning model. The determination unit 223 determines that the characteristics of the subject T output from the second learning model are the walking characteristics of the subject T.

[0159] Furthermore, the determination unit 223 determines a predetermined threshold value based on the walking characteristics of the subject T determined in step S222 (S224).

[0160] Then, the process of step S16 is performed, and if the answer is No at step S16, the determination unit 222 determines whether the acquired walking sound includes a falling sound (S220). In this operation example, at step S220, whether the walking sound includes a falling sound is determined using the third method. That is, the determination unit 222 determines that the acquired walking sound includes a falling sound when the amplitude of the waveform of a walking sound signal indicating the acquired walking sound exceeds a predetermined threshold.

[0161] Here, the processes in steps S222 and S224 will be described again.

[0162] For example, if it is determined in step S222 that the walking characteristic of subject T is normal, then in step S224 the determination unit 223 determines that the predetermined threshold is +0.0025, as in step S20 of operation example 1. Note that if it is determined that the walking characteristic of subject T is shuffling, the determination unit 223 determines the predetermined threshold to be smaller than the predetermined threshold when it is determined that the walking characteristic of subject T is normal. For example, if it is determined that the walking characteristic of subject T is shuffling, then the determination unit 223 determines that the predetermined threshold is +0.001.

[0163] In step S220, the determination unit 222 determines whether the walking sound includes a fall sound using the predetermined threshold determined in this manner. When the walking characteristic of the subject T is shuffling, the subject T is more likely to fall than when the walking characteristic of the subject T is normal. When the walking characteristic of the subject T is shuffling, by reducing the predetermined threshold, the determination unit 222 can easily determine that the acquired walking sound includes a fall sound, even if the fall sound is quiet. In other words, it can be easily determined whether the subject T, who is at high risk of falling, has fallen.

[0164] If it is determined that the walking sound includes the sound of a fall (Yes in step S220), the process of step S18 is performed.

[0165] Furthermore, if it is determined that the walking sounds do not include the falling sounds (No in step S220), the process of step S12 is performed again.

[0166] [Operation Example 4] FIG. 16 is a flowchart of operation example 4 of the information processing system 100 according to this embodiment.

[0167] Operation example 4 is performed after the processing of operation example 1 is completed. In other words, before the processing of operation example 4 is performed, the output unit 224 has finished outputting the first information, determining that the relative position coordinates indicated by the first information are a pedestrian danger area. The first information is then stored in the second storage unit 23. For identification purposes, the relative position coordinates determined to be a pedestrian danger area may be referred to as relative position coordinate A.

[0168] In the fourth operational example, first, the sound generator 11 outputs sound (ultrasonic waves) (S310).

[0169] Next, the sound receiving unit 12 acquires the reflected sound reflected by hitting the target person T and the spatial sound (S312).

[0170] Then, the calculation unit 221 calculates the relative position coordinates between the subject T and the sound receiving unit 12 based on the acquired reflected sound (S314). The method by which the calculation unit 221 calculates the relative position coordinates is as described in Operation Example 1. Furthermore, the relative position coordinates calculated in step S314, that is, the recalculated relative position coordinates, may be referred to as relative position coordinates B for identification purposes. At this time, as in step S14 in Operation Example 1, the calculation unit 221 may calculate the walking speed based on the calculated relative position coordinates.

[0171] Next, the calculation unit 221 determines whether the distance between the recalculated relative position coordinate (relative position coordinate B) and the relative position coordinate (relative position coordinate A) that was determined to be a dangerous walking area based on the first information output and stored in step S18 of operation example 1 is less than a predetermined distance (S326).

[0172] The predetermined distance may be, for example, a distance determined by the administrator of the information processing system 100, but is not limited to this and may be, for example, a value between several tens of centimeters and several meters.

[0173] Relative position coordinate A is the coordinate where the change in walking speed of subject T becomes equal to or greater than a predetermined value before the processing of Operation Example 4 is performed, i.e., the position where subject T fell. Relative position coordinate B is the position of subject T while the processing of Operation Example 4 is being performed, i.e., the current position of subject T.

[0174] Therefore, when the distance between the relative position coordinate B and the relative position coordinate A is equal to or less than the predetermined distance, it corresponds to when the subject T is approaching the position where he or she previously fell.

[0175] If the distance between the relative position coordinate B and the relative position coordinate A is equal to or shorter than the predetermined distance (Yes in step S326), the information processing unit 22 controls the sound output unit 11, which is a notification unit, to notify the subject T (S328). For example, the information processing unit 22 causes the second communication unit 21 to output an instruction indicating that the sound output unit 11 will notify the subject T. The first communication unit 13 acquires the output instruction, and the control unit 14 controls the sound output unit 11 to notify the subject T. As a result, the sound output unit 11 notifies the subject T of danger, more specifically, of the risk of falling. For example, the sound output unit 11 may output an alarm sound or a buzzer sound as the notification sound, or may output a voice such as, "You are approaching the location where you previously fell."

[0176] Then, the processes of steps S10, S12, S14, S16 and S20 are performed in operation example 1. As in operation example 1, if step S16 is No, the process of step S20 is performed.

[0177] Then, when the calculated change in walking speed is equal to or greater than a predetermined value (Yes in step S16), the information processing unit 22 controls the sound output unit 11 to change the mode of notification (S330). For example, the information processing unit 22 causes the second communication unit 21 to output an instruction indicating that the sound output unit 11 should change the mode of notification. The first communication unit 13 acquires the output instruction, and the control unit 14 controls the sound output unit 11 to change the mode of notification. In this way, when the subject T falls, the control unit 14 changes the mode of notification provided by the sound output unit 11.

[0178] The information processing unit 22 of the information processing device 20 causes the second communication unit 21 to output information indicating that the calculated amount of change in walking speed is equal to or greater than a predetermined value to the first communication unit 13 of the detection device 10. The control unit 14 changes the mode of notification performed by the sound output unit 11 in accordance with the information acquired by the first communication unit 13.

[0179] For example, the control unit 14 changes the notification mode so as to increase the volume of the alarm sound, buzzer sound, or voice. As a result, when the process of Operation Example 4 is performed again, the sound output unit 11 can issue a notification at a higher volume to notify the user of the risk of falling.

[0180] Furthermore, if it is determined that the walking sound includes the sound of a fall (Yes in step S20), that is, if the subject T has fallen, the process of step S330 is performed.

[0181] If the calculated change in walking speed is less than a predetermined value (No in step S16) and it is determined that the walking sound does not include the sound of falling (No in step S20), the processing of step S330 is not performed.

[0182] [Operation Example 5] FIG. 17 is a flowchart of operation example 5 of the information processing system 100 according to this embodiment.

[0183] In the fifth operation example, first, the process of step S328 in the fourth operation example is performed.

[0184] Then, the information processing unit 22 changes the frequency of the sound (ultrasound) output by the sound output unit 11 to increase it (S430). For example, the information processing unit 22 causes the second communication unit 21 to output an instruction to change the frequency of the sound output by the sound output unit 11 to increase it. The first communication unit 13 acquires the output instruction, and the control unit 14 changes the frequency of the sound output by the sound output unit 11 to increase it.

[0185] Then, the sound generator 11 outputs sounds (ultrasound waves) at the increased frequency (S410). The ultrasound waves output at the increased frequency hit the subject T present in the space 90 and are reflected.

[0186] Then, the processes of steps S12, S14, S16, S18 and S20 of Operation Example 1 are performed.

[0187] As described above, when the subject T is approaching a position where the subject T previously fell, in step S328 the sound generator 11 notifies the subject T. In this way, when the subject T is approaching a position where the subject T previously fell, the frequency of the sound (ultrasound) output by the sound generator 11 is increased, thereby making it possible to more frequently and accurately detect falls by the subject T.

[0188] [Effects, etc.] The information processing method according to this embodiment is an information processing method executed by a computer (e.g., information processing device 20), and includes acquiring reflected sound that is generated when sound output by sound generator 11 hits subject T and is received by sound receiver 12, calculating the walking speed of subject T based on the acquired reflected sound, and, if the amount of change in the calculated walking speed is equal to or greater than a predetermined value, outputting first information indicating that the amount of change is equal to or greater than the predetermined value.

[0189] As a result, the walking speed of the subject T is calculated based on the reflected sound, and when the amount of change in walking speed is equal to or greater than a predetermined value, first information is output. A case where the amount of change in walking speed is equal to or greater than a predetermined value corresponds to a case where the subject T has fallen while walking. In this way, because it is determined whether or not the subject T has fallen based on the reflected sound, it is possible to accurately detect the subject T's fall even if, for example, the volume of the sound caused by the subject T's fall is low. In other words, an information processing method is realized that can accurately detect the subject T's fall.

[0190] Furthermore, in this embodiment, when the amount of change in walking speed is equal to or greater than a predetermined value, first information is output and stored. That is, information (first information) indicating that the subject T has fallen in the space 90 is stored. By storing and accumulating such information, it is possible to issue a notification to warn of danger, as shown in Operation Example 4, for example.

[0191] In the information processing method according to this embodiment, the sound receiving unit 12 is placed in a space 90 in which the subject T is located, and the relative position coordinates between the subject T and the sound receiving unit 12 are calculated based on the acquired reflected sound, and the walking speed is calculated based on the calculated relative position coordinates.

[0192] This allows the relative position coordinates to be calculated based on the reflected sound, thereby realizing an information processing method that can accurately detect the position (relative position coordinates) of the subject T even when, for example, the volume of the walking sound of the subject T is low.

[0193] In the information processing method according to this embodiment, when the amount of change in the calculated walking speed is greater than or equal to a predetermined value, first information indicating that the amount of change in the calculated relative position coordinates is greater than or equal to a predetermined value is output.

[0194] As a result, first information indicating that the amount of change in walking speed at the calculated relative position coordinates is equal to or greater than a predetermined value is output, i.e., an information processing method is realized that can output first information indicating that the subject T has fallen while walking at the calculated relative position coordinates.

[0195] The information processing method of this embodiment includes judging, acquiring the walking sound of the subject T, judging whether the acquired walking sound includes the sound of a fall, and outputting, if the acquired walking sound includes the sound of a fall, outputting first information indicating that the walking sound also includes the sound of a fall.

[0196] This determines whether the walking sounds include the sound of a fall, and if the walking sounds include the sound of a fall, the first information is further output. A case where the walking sounds include the sound of a fall corresponds to a case where the subject T has fallen while walking. In this way, whether the subject T has fallen is determined based on the walking sounds as well as the reflected sounds, so that the subject T's fall can be detected with greater accuracy. In other words, an information processing method is realized that can detect the subject T's fall with greater accuracy.

[0197] In the information processing method according to the present embodiment, when the calculated amount of change in the walking speed of the subject T is less than a predetermined value, a determination is made and an output is made.

[0198] As a result, even if it is determined based on the reflected sound that the subject T has not fallen, it is possible to determine whether or not the subject T has fallen based on the walking sound, thereby making it possible to more accurately detect a fall of the subject T. In other words, an information processing method is realized that can more accurately detect a fall of the subject T.

[0199] In the information processing method according to the present embodiment, when the amplitude of the waveform of the walking sound signal indicating the acquired walking sound exceeds a predetermined threshold, it is determined that the acquired walking sound includes the sound of a fall.

[0200] This makes it possible to determine whether or not the subject T has fallen based on the waveform of the walking sound signal.

[0201] In the information processing method according to this embodiment, the determination as to whether the acquired walking sounds include the sound of a fall is made using a trained first learning model that inputs a spectrogram of the acquired walking sounds and outputs second information indicating whether the acquired walking sounds include the sound of a fall.

[0202] This makes it possible to determine whether or not the subject T has fallen based on the spectrogram of the walking sound.

[0203] In the information processing method according to the present embodiment, the sound is ultrasonic.

[0204] This makes it possible to determine whether or not the subject T has fallen using ultrasound, which is a sound that cannot be heard by the subject T. By using ultrasound, the subject T is less likely to feel annoyed by the sound compared to when, for example, audible sound is used.

[0205] In the information processing method according to the present embodiment, the first information is output, with the calculated relative position coordinates being determined to be a dangerous walking location.

[0206] This realizes an information processing method that can output first information indicating that the subject T has fallen while walking at the calculated relative position coordinates and that the relative position coordinates are a dangerous walking location.

[0207] In the information processing method according to this embodiment, the output includes outputting first information indicating that the calculated relative position coordinates are a candidate pedestrian danger area each time the amount of change is greater than or equal to a predetermined value, and if the number of times the output first information indicates that the relative position coordinates are a candidate pedestrian danger area is greater than or equal to a predetermined number, outputting the first information indicating that the calculated relative position coordinates are a pedestrian danger area.

[0208] As a result, when the amount of change in the relative position coordinates is equal to or greater than a predetermined value a predetermined number of times or more, first information indicating that the relative position coordinates are a dangerous place for walking is output. That is, when a predetermined number of falls or more have occurred at the relative position coordinates, an information processing method is realized in which the relative position coordinates can be determined to be a dangerous place for walking and output.

[0209] The information processing method according to this embodiment includes determining features using a trained second learning model that inputs a spectrogram of the acquired walking sounds and outputs the walking features of the subject T, and determining a predetermined threshold value based on the determined features.

[0210] This allows a predetermined threshold value according to the walking of the subject T to be determined based on the spectrogram of the walking sound.

[0211] For example, as shown in Operation Example 3, when the walking characteristic of the subject T is shuffling, the predetermined threshold is determined so as to be small. When the walking characteristic of the subject T is shuffling, the subject T is more likely to fall than when the walking characteristic of the subject T is normal. When the walking characteristic of the subject T is shuffling, by reducing the predetermined threshold, it is possible to easily determine that the acquired walking sound includes the sound of a fall, even if the sound of a fall is small. In other words, it is possible to easily determine whether the subject T, who is at high risk of falling, has fallen.

[0212] In the information processing method according to this embodiment, in the outputting, after the first information is output indicating that the calculated relative position coordinates are a pedestrian danger area, the relative position coordinates are calculated again, and the information processing method includes notifying the subject T when the distance between the recalculated relative position coordinates and the relative position coordinates that are determined to be a pedestrian danger area based on the output first information is less than a predetermined distance.

[0213] This realizes an information processing method that can notify the subject T when the distance is equal to or less than a predetermined distance. The relative position coordinates that are determined to be a dangerous walking location are the coordinates where the change in the walking speed of the subject T previously exceeded a predetermined value, and are the location where the subject T fell. Furthermore, the recalculated relative position coordinates are the current location of the subject T. Therefore, a case where the distance is equal to or less than a predetermined distance corresponds to a case where the subject T is approaching the location where he or she previously fell. In such a case, a notification is issued to notify the subject T of the danger. This realizes an information processing method that can make the subject T aware of the danger.

[0214] The information processing method according to this embodiment includes a first change that changes the mode of notification when, after notification has been performed, the amount of change in the recalculated relative position coordinates is greater than or equal to a predetermined value.

[0215] This realizes an information processing method that can more easily alert the subject T to danger by changing the notification mode, for example, by increasing the volume of the notification sound.

[0216] The information processing method according to the present embodiment includes a second change that changes the frequency of sound output by sound output unit 11 to be increased after the notification is performed.

[0217] This allows the reflected sound to be acquired again with the frequency of sound output increased, thereby realizing an information processing method that can detect falls of the subject T more frequently and accurately.

[0218] A computer program according to the present embodiment causes a computer to execute any one of the above information processing methods.

[0219] This allows the computer to execute the above-described information processing method in accordance with the computer program.

[0220] The information processing system 100 according to this embodiment includes a sound receiving unit 12 that acquires reflected sound that is generated when the sound output by the sound generating unit 11 hits the subject T, a calculation unit 221 that calculates the walking speed of the subject T based on the acquired reflected sound, and an output unit 224 that outputs first information indicating that the amount of change in the calculated walking speed is greater than or equal to a predetermined value when the amount of change is greater than or equal to a predetermined value.

[0221] As a result, the walking speed of the subject T is calculated based on the reflected sound, and when the amount of change in walking speed is equal to or greater than a predetermined value, the first information is output. When the amount of change in walking speed is equal to or greater than a predetermined value, this corresponds to when the subject T has fallen while walking. In this way, because it is determined whether or not the subject T has fallen based on the reflected sound, it is possible to accurately detect the subject T's fall even when, for example, the volume of the sound caused by the subject T's fall is less than a predetermined volume. In other words, an information processing system 100 is realized that can accurately detect the subject T's fall.

[0222] The information processing device 20 according to this embodiment includes a communication unit (second communication unit 21) that acquires reflected sound that is generated when the sound output by the sound generation unit 11 hits the subject T and is received by the sound receiving unit 12, a calculation unit 221 that calculates the walking speed of the subject T based on the acquired reflected sound, and an output unit 224 that outputs first information indicating that the change in the calculated walking speed is greater than or equal to a predetermined value when the change is greater than or equal to a predetermined value.

[0223] As a result, the walking speed of the subject T is calculated based on the reflected sound, and when the amount of change in walking speed is equal to or greater than a predetermined value, the first information is output. When the amount of change in walking speed is equal to or greater than a predetermined value, this corresponds to when the subject T has fallen while walking. In this way, because it is determined whether or not the subject T has fallen based on the reflected sound, it is possible to accurately detect the subject T's fall even when, for example, the volume of the sound caused by the subject T's fall is less than a predetermined volume. In other words, an information processing device 20 is realized that can accurately detect the subject T's fall.

[0224] (Other Embodiments) The information processing system 100 and information processing method according to aspects of the present disclosure have been described above based on embodiments, but the present disclosure is not limited to these embodiments. For example, other embodiments realized by any combination of the components described in this specification or by excluding some of the components may also be considered as embodiments of the present disclosure. Furthermore, the present disclosure also includes variations obtained by applying various modifications to the above embodiments that would occur to a person skilled in the art within the scope of the gist of the present disclosure, i.e., the meaning indicated by the wording of the claims.

[0225] In the third example of operation, the processes of steps S222 and S224 are performed after the process of step S14, but this is not limiting. For example, the processes of steps S12 and S14 and the processes of steps S222 and S224 may be performed in parallel.

[0226] In the present embodiment, the sound output unit 11 is used as an example of a notification unit, but this is not limiting. For example, a light-emitting unit that emits light as a notification may be used as an example of a notification unit. The light-emitting unit may be a color lighting device that emits orange or red light, or may be a display device (display) that displays text such as "Danger." When the notification unit is a light-emitting unit, if the mode of notification is changed, processing such as changing the color of the emitted light or changing the text displayed is performed.

[0227] In the embodiment, the detection device 10 and the information processing device 20 are separate devices, but this is not limiting. For example, the detection device 10 and the information processing device 20 may be integrated, and as an example, the detection device 10 may include the information processing device 20 (i.e., the second communication unit 21, the information processing unit 22, and the second storage unit 23).

[0228] Note that the method by which the information processing unit 22 calculates the walking speed of the subject T is not limited to the above. For example, the information processing unit 22 may calculate the walking speed of the subject T using other sensor data, such as an image captured by a camera, data from a Time Of Flight (ToF) sensor, or data from a Light Detection And Ranging (LiDAR) sensor, in addition to the reflected sound. By combining the reflected sound with the other sensor data, the information processing unit 22 can accurately calculate the walking speed.

[0229] Furthermore, the method by which the information processing unit 22 calculates the relative position coordinates is not limited to the above. For example, the information processing unit 22 may calculate the relative position coordinates using other sensor data, such as an image captured by a camera, data from a ToF sensor, or data from a LiDAR sensor, in addition to the reflected sound. By combining the reflected sound with other sensor data, the information processing unit 22 can accurately calculate the relative position coordinates.

[0230] The following embodiments may also be included within the scope of one or more aspects of the present disclosure.

[0231] (1) Some of the components constituting the information processing system 100 may be a computer system comprising a microprocessor, ROM, RAM, hard disk unit, display unit, keyboard, mouse, etc. A computer program is stored in the RAM or hard disk unit. The microprocessor operates in accordance with the computer program to achieve its functions. Here, the computer program is composed of a combination of multiple instruction codes that indicate commands to a computer to achieve a predetermined function.

[0232] (2) Some of the components constituting the information processing system 100 may be configured as a single system LSI (Large Scale Integration). A system LSI is an ultra-multifunctional LSI manufactured by integrating multiple components on a single chip, and specifically, is a computer system configured to include a microprocessor, ROM, RAM, etc. A computer program is stored in the RAM. The system LSI achieves its functions by the microprocessor operating in accordance with the computer program.

[0233] (3) Some of the components constituting the information processing system 100 may be configured as an IC card or a standalone module that can be attached to or detached from each device. The IC card or module is a computer system configured with a microprocessor, ROM, RAM, etc. The IC card or module may include the ultra-multifunctional LSI. The IC card or module achieves its functions when the microprocessor operates in accordance with a computer program. The IC card or module may be tamper-resistant.

[0234] (4) Furthermore, some of the components constituting the information processing system 100 may be the computer program or the digital signal recorded on a computer-readable recording medium, such as a flexible disk, hard disk, CD-ROM, MO, DVD, DVD-ROM, DVD-RAM, BD (Blu-ray (registered trademark) Disc), semiconductor memory, etc. Alternatively, they may be digital signals recorded on such recording media.

[0235] Furthermore, some of the components constituting the information processing system 100 may transmit the computer program or the digital signal via a telecommunications line, a wireless or wired communication line, a network such as the Internet, data broadcasting, etc.

[0236] (5) The present disclosure may be embodied as the methods described above, a computer program that implements these methods on a computer, or a digital signal that includes the computer program.

[0237] (6) The present disclosure may also be a computer system having a microprocessor and a memory, the memory storing the computer program, and the microprocessor operating in accordance with the computer program.

[0238] (7) The program or the digital signal may also be implemented by another independent computer system by recording it on the recording medium and transferring it, or by transferring the program or the digital signal via the network, etc.

[0239] The present disclosure is applicable to an information processing method, an information processing system, and an information processing device, and is particularly applicable to detection of a fall, etc.

[0240] REFERENCE SIGNS LIST 10 Detection device 11 Sound generation unit 12 Sound receiving unit 13 First communication unit 14 Control unit 15 First storage unit 20 Information processing device 21 Second communication unit 22 Information processing unit 23 Second storage unit 80 Information communication network 90 Space 221 Calculation unit 222 Determination unit 223 Decision unit 224 Output unit M1, M2, M3, Mm, Mi Microphone T Subject

Claims

1. An information processing method executed by a computer, comprising: acquiring reflected sound that is received by a sound receiving unit and is generated when sound output from a sound generating unit hits a target person; calculating the walking speed of the target person based on the acquired reflected sound; and, if a change in the calculated walking speed is equal to or greater than a predetermined value, outputting first information indicating that the change is equal to or greater than the predetermined value.

2. The information processing method of claim 1, wherein the sound receiving unit is placed in a space where the subject is present, and the calculating step calculates relative position coordinates between the subject and the sound receiving unit based on the acquired reflected sound, and calculates the walking speed based on the calculated relative position coordinates.

3. The information processing method of claim 2, wherein when the calculated amount of change in the walking speed is equal to or greater than the predetermined value, the first information indicating that the amount of change in the calculated relative position coordinates is equal to or greater than the predetermined value is output.

4. The information processing method of claim 1, further comprising: determining, in said acquiring, acquiring walking sounds of the subject; determining, in said determining, whether the acquired walking sounds include the sound of a fall; and, in said outputting, if the acquired walking sounds include the sound of a fall, outputting the first information indicating that the walking sounds also include the sound of a fall.

5. The information processing method according to claim 4, wherein the determining and the outputting are performed when the calculated amount of change in the walking speed of the subject is less than the predetermined value.

6. The information processing method according to claim 4, wherein the determining step determines that the acquired walking sound includes the falling sound when the amplitude of the waveform of the walking sound signal indicating the acquired walking sound exceeds a predetermined threshold.

7. The information processing method of claim 4, wherein the determining step determines whether the acquired walking sound includes the falling sound using a trained first learning model that receives as input a spectrogram of the acquired walking sound and outputs second information indicating whether the acquired walking sound includes the falling sound.

8. The information processing method according to claim 1, wherein the sound is ultrasonic.

9. The information processing method according to claim 3, wherein the outputting comprises outputting the first information based on the calculated relative position coordinates being a dangerous walking location.

10. The information processing method of claim 3, wherein the outputting comprises outputting the first information indicating that the calculated relative position coordinates are a candidate pedestrian dangerous area each time the amount of change is equal to or greater than the predetermined value, and outputting the first information indicating that the calculated relative position coordinates are a pedestrian dangerous area if the number of times the output first information indicates that the relative position coordinates are a pedestrian dangerous area is equal to or greater than a predetermined number.

11. The information processing method according to claim 6, further comprising determining the characteristics using a trained second learning model that inputs the acquired spectrogram of the walking sound and outputs the walking characteristics of the subject, and determining the predetermined threshold based on the determined characteristics.

12. The information processing method according to claim 9, wherein in the outputting, after the first information is output indicating that the calculated relative position coordinates are the pedestrian danger area, the calculating of the relative position coordinates is performed again, and the information processing method includes notifying the subject if the distance between the recalculated relative position coordinates and the relative position coordinates that are determined to be the pedestrian danger area based on the output first information is less than a predetermined distance.

13. The information processing method according to claim 12, further comprising a first change for changing the manner of the notification if, after the notification has been performed, the amount of change in the recalculated relative position coordinates is equal to or greater than the predetermined value.

14. The information processing method according to claim 12, further comprising, after the notification, a second change of increasing the frequency of the sound output by the sound output unit.

15. A computer program for causing a computer to execute the information processing method according to any one of claims 1 to 14.

16. An information processing system comprising: a sound receiving unit that acquires reflected sound that is generated when sound output from a sound generating unit hits a subject and is reflected; a calculation unit that calculates the walking speed of the subject based on the acquired reflected sound; and an output unit that, when a change in the calculated walking speed is equal to or greater than a predetermined value, outputs first information indicating that the change is equal to or greater than the predetermined value.

17. An information processing device comprising: a communication unit that acquires reflected sound that is generated when sound output from a sound generating unit hits a target person and is received by a sound receiving unit; a calculation unit that calculates the walking speed of the target person based on the acquired reflected sound; and an output unit that, when a change in the calculated walking speed is equal to or greater than a predetermined value, outputs first information indicating that the change is equal to or greater than the predetermined value.

Images