Information processing method, information processing device, and program

Abstract

This information processing method, which is executed by a computer, comprises: acquiring information indicating a first reflected wave of an ultrasonic wave, which can be obtained by transmitting the ultrasonic wave to a space, and calculating a feature amount based on the difference between a first signal based on the reflected wave and a second signal based on information indicating a second reflected wave acquired prior to the first reflected wave (S13); and determining, on the basis of the calculated feature amount, whether a person is present in the space (S14).

Description

Information processing method, information processing device, and program

[0001] The present disclosure relates to an information processing method, an information processing device, and a program.

[0002] In recent years, devices with voice output capabilities have become widespread in living spaces. Examples of such devices include smart speakers and home appliances with voice output capabilities. It is believed that the effectiveness of such devices can be enhanced by determining the timing of their output based on the presence or absence of a person. Patent Literature 1 discloses a technology for detecting human movement using Doppler shifts generated by emitting ultrasonic signals into the environment.

[0003] U.S. Pat. No. 1,079,5018

[0004] However, the technology of Patent Document 1 has a problem in that it cannot detect the presence or absence of a person unless the person is moving, and therefore cannot improve the effectiveness of speech when the person is stationary. From the viewpoint of further improving the effectiveness of speech, it is desirable to be able to detect the presence or absence of a person regardless of the person's movement.

[0005] Therefore, the present disclosure provides an information processing method, an information processing device, and a program that can detect the presence or absence of a person regardless of the person's movement.

[0006] An information processing method according to one aspect of the present disclosure is an information processing method executed by a computer, which acquires information indicating a first reflected wave obtained by transmitting ultrasonic waves into a space, calculates a feature based on the difference between a first signal based on the first reflected wave and a second signal based on information indicating a second reflected wave acquired earlier than the first reflected wave, and determines whether or not a person is present in the space based on the calculated feature.

[0007] An information processing device according to one aspect of the present disclosure includes an acquisition unit that acquires information indicating a first reflected wave of an ultrasonic wave obtained by transmitting the ultrasonic wave into a space, a calculation unit that calculates a feature amount based on the difference between a first signal based on the first reflected wave and a second signal based on information indicating a second reflected wave acquired earlier than the first reflected wave, and a determination unit that determines whether or not a person is present in the space based on the calculated feature amount.

[0008] A program according to one aspect of the present disclosure is a program for causing a computer to execute the above-described information processing method.

[0009] According to one aspect of the present disclosure, it is possible to realize an information processing method or the like that can detect the presence or absence of a person regardless of the person's movement.

[0010] FIG. 1 is a block diagram showing a functional configuration of an information processing system according to Embodiment 1. FIG. 2 is a flowchart showing the operation of the information processing system according to Embodiment 1. FIG. 3 is a flowchart showing detailed operation of step S13 shown in FIG. 2. FIG. 4 is a diagram showing examples of various waveforms according to Embodiment 1. FIG. 5 is a flowchart showing the operation of an information processing device according to Modification 1 of Embodiment 1, corresponding to step S13 shown in FIG. 2. FIG. 6 is a diagram showing examples of various waveforms according to Modification 1 of Embodiment 1. FIG. 7 is a flowchart showing the operation of an information processing device according to Modification 2 of Embodiment 1, corresponding to step S13 shown in FIG. 2. FIG. 8 is a flowchart showing the operation of an information processing device according to Modification 3 of Embodiment 1, corresponding to step S13 shown in FIG. 2. FIG. 9 is a diagram showing examples of various waveforms according to Modification 3 of Embodiment 1. FIG. 10 is a block diagram showing a functional configuration for estimating an impulse response by an adaptive filter according to Modification 3 of Embodiment 1. FIG. 11 is a flowchart showing the operation of an information processing device according to Modification 4 of Embodiment 1, corresponding to step S13 shown in FIG. 2. FIG. 12 is a flowchart showing detailed operation of step S162 shown in FIG. 11. FIG. 13 is a flowchart showing detailed operation of step S163 shown in FIG. 11. Fig. 14 is a diagram showing an example of various waveforms according to Modification 4 of Embodiment 1. Fig. 15 is a flowchart showing the operation of an information processing device according to Modification 5 of Embodiment 1, corresponding to step S13 shown in Fig. 2. Fig. 16 is a block diagram showing the functional configuration of an information processing system according to Embodiment 2. Fig. 17 is a flowchart showing the operation of the information processing system according to Embodiment 2.

[0011] An information processing method according to a first aspect of the present disclosure is an information processing method executed by a computer, which acquires information indicating a first reflected wave obtained by transmitting ultrasonic waves into a space, calculates a feature based on the difference between a first signal based on the first reflected wave and a second signal based on information indicating a second reflected wave acquired earlier than the first reflected wave, and determines whether or not a person is present in the space based on the calculated feature.

[0012] As a result, since a feature based on the difference between the reflected wave and past data (second reflected wave) is used, even if a person is not moving at the present time, it is possible to detect the difference if the person has recently moved. In other words, it is possible to detect the presence of a person even if the person is not moving at the present time. Therefore, according to the information processing method, it is possible to detect the presence or absence of a person regardless of the person's current movement.

[0013] Also, for example, the information processing method according to the second aspect may be the information processing method according to the first aspect, wherein the first signal includes the first reflected wave, the second signal includes the second reflected wave, and the feature amount is a value based on a first standard deviation of the amplitude of a differential signal between the first reflected wave and the second reflected wave.

[0014] By calculating the variance of the reflected wave, a feature value that expresses the characteristics of the amplitude of the reflected wave can be obtained. Since the amplitude of the reflected wave can change depending on whether or not a person is moving, by using this feature value, it is possible to detect the presence or absence of a person regardless of their movement.

[0015] Also, for example, an information processing method according to a third aspect may be the information processing method according to the first aspect, in which a first direct wave of the ultrasonic wave is acquired, the first signal includes a first convolved signal obtained by convolving the first reflected wave and the first direct wave, the second signal includes a second convolved signal obtained by convolving the second reflected wave and a second direct wave acquired earlier than the first reflected wave, and the feature amount may be a value based on a first standard deviation of the amplitude of a difference signal between the first convolved signal and the second convolved signal.

[0016] This allows the direct wave and the reflected wave to be associated with each other by convolution, so that even if the transmitted sound is unstable, the influence on the calculation of the feature amount can be reduced.

[0017] Furthermore, for example, an information processing method according to a fourth aspect may be the information processing method according to the first aspect, which acquires a direct wave of the ultrasonic wave, generates a direct wave signal in which the amplitude of the first reflected wave in a received signal including the direct wave and the first reflected wave is replaced with zero, and generates a reflected wave signal in which the amplitude of the direct wave in the received signal is replaced with zero, and estimates a first impulse response of an adaptive filter based on the direct wave signal and the reflected wave signal, wherein the first signal includes the first impulse response and the second signal includes a second impulse response acquired earlier than the first impulse response.

[0018] This allows the estimation period of the impulse response to be the sampling period (presence / absence determination period), which can be made shorter than the emission period of the sound source. This improves the ability to follow slight changes in the spatial state, thereby improving robustness against external disturbances.

[0019] Also, for example, an information processing method according to a fifth aspect may be the information processing method according to the first aspect, wherein the first signal includes a signal indicating the change over time in amplitude of the first reflected wave, and the second signal includes a signal indicating the change over time in amplitude of the second reflected wave obtained earlier than the first reflected wave.

[0020] This allows us to focus only on the amplitude of the reflected wave by using a signal that indicates the time change in amplitude (for example, an envelope), which makes it possible to eliminate the influence of phase shifts in the reflected wave caused by disturbances such as air currents, thereby increasing robustness against disturbances.

[0021] Furthermore, for example, an information processing method according to a sixth aspect may be an information processing method according to the second or third aspect, wherein the first signal includes a first envelope signal indicating a change over time in the amplitude of the first reflected wave, the second signal includes a second envelope signal indicating a change over time in the amplitude of the second reflected wave acquired earlier than the first reflected wave, and the feature value may be a value calculated by calculating the maximum value of the amplitude of a difference signal between the first envelope signal and the second envelope signal and a value based on the first standard deviation.

[0022] As a result, the feature for determining whether a person is present or absent can be calculated using the two feature values, the maximum value and the first standard deviation. By using two feature values, the presence or absence of a person can be determined more accurately than when only one feature value is used.

[0023] Also, for example, an information processing method according to the seventh aspect is an information processing method according to any one of the first to sixth aspects, and the second reflected wave may be an average waveform of reflected waves received at multiple points in time in the past.

[0024] As a result, since an average waveform is used, it is possible to accurately determine whether a person is present or absent even if, for example, the most recent reflected wave contains noise.

[0025] Furthermore, for example, the information processing method according to the eighth aspect may be the information processing method according to the second or third aspect, in which one or more second standard deviations calculated in the past from the first standard deviation are obtained, and the feature amount may be a value obtained by smoothing the first standard deviation and the one or more second standard deviations.

[0026] This makes it possible to remove minute fluctuation components, thereby enabling accurate determination of the presence or absence of a person.

[0027] Furthermore, for example, an information processing method according to a ninth aspect is the information processing method according to the eighth aspect, in which the smoothing process may include moving average processing.

[0028] This makes it possible to remove small fluctuation components contained in the feature amount using the moving average.

[0029] Also, for example, an information processing method according to a tenth aspect may be the information processing method according to the eighth aspect, in which the smoothing process includes a peak hold process that stores a maximum value of the one or more second standard deviations obtained over a predetermined period, and the feature value may be a value based on the maximum value stored by the peak hold process and the first standard deviation.

[0030] This allows the peak value of the feature amount due to the most recent human movement in the past data to be maintained while attenuating it, so that even when there is little human movement, a higher feature amount value can be maintained. In other words, even when there is little human movement, the presence or absence of a person can be determined more accurately.

[0031] Furthermore, for example, an information processing method according to an eleventh aspect is the information processing method according to any one of the first to tenth aspects, wherein the ultrasonic waves may be emitted periodically.

[0032] This allows ultrasonic waves to be transmitted periodically, so that, for example, reflected waves at the present time and reflected waves at past times (waves reflected periodically from past transmitted signals) can be easily obtained.

[0033] Furthermore, for example, an information processing method according to a twelfth aspect is the information processing method according to any one of the first to eleventh aspects, and the ultrasonic waves may include burst waves.

[0034] As a result, since a burst wave is used, it is possible to easily obtain, for example, a reflected wave at the present time and a reflected wave at a past time point (a reflected wave periodically corresponding to a past transmitted signal).

[0035] Furthermore, for example, an information processing method according to a thirteenth aspect is the information processing method according to the twelfth aspect, wherein the burst wave may include a plurality of frequency components.

[0036] This allows the use of reflected waves for signals of multiple frequencies, thereby reducing the influence of noise of a specific frequency or noise caused by environmental changes such as temperature, humidity, and air currents.

[0037] Furthermore, for example, an information processing method according to a fourteenth aspect is the information processing method according to any one of the first to thirteenth aspects, wherein the ultrasonic waves may include a chirp signal.

[0038] This uses reflected waves from a chirp signal containing multiple frequencies, making it possible to reduce the effects of noise at a specific frequency or noise caused by environmental changes such as temperature, humidity, and airflow.

[0039] Furthermore, for example, an information processing method according to a fifteenth aspect may be an information processing method according to any one of the first to fourteenth aspects, in which if the feature amount is equal to or greater than a predetermined value, it is determined that a person is present in the space, and if the feature amount is less than the predetermined value, it is determined that no person is present in the space.

[0040] This makes it possible to determine whether a person is present or absent using a simple determination method that uses a predetermined value, thereby reducing the amount of processing that is required by the information processing device that executes the information processing method.

[0041] Furthermore, for example, an information processing method according to a sixteenth aspect may be an information processing method according to any one of the first to fourteenth aspects, and may obtain an inference result as to whether or not a person is present in the space by inputting the feature amount into a machine learning model.

[0042] This allows the presence or absence of people to be determined using a machine learning model.

[0043] Furthermore, for example, an information processing method according to a seventeenth aspect may be the information processing method according to the sixteenth aspect, in which a plurality of feature quantities and a plurality of judgment results as to whether or not a person is present are acquired, the acquired plurality of feature quantities are used as input information, and the acquired plurality of judgment results are used as correct answer data to train the machine learning model.

[0044] This allows for the generation of a machine learning model.

[0045] An information processing device according to an aspect of the present disclosure includes an acquisition unit that acquires information indicating a first reflected wave of an ultrasonic wave obtained by transmitting the ultrasonic wave into a space, a calculation unit that calculates a feature amount based on a difference between a first signal based on the first reflected wave and a second signal based on information indicating a second reflected wave acquired earlier than the first reflected wave, and a determination unit that determines whether or not a person is present in the space based on the calculated feature amount.A program according to an aspect of the present disclosure is a program for causing a computer to execute the above information processing method.

[0046] This provides the same effect as the above-described information processing method.

[0047] These general or specific aspects may be realized as a system, a method, an integrated circuit, a computer program, or a non-transitory recording medium such as a computer-readable CD-ROM, or as any combination of the system, method, integrated circuit, computer program, or recording medium. The program may be pre-stored in the recording medium, or may be supplied to the recording medium via a wide area communication network including the Internet.

[0048] Hereinafter, the embodiments will be specifically described with reference to the drawings.

[0049] The embodiments described below are all comprehensive or specific examples. The numerical values, shapes, components, component placement and connection configurations, steps, and step order shown in the following embodiments are merely examples and are not intended to limit the present disclosure. Furthermore, among the components in the following embodiments, components not described in independent claims are described as optional components.

[0050] Furthermore, each figure is a schematic diagram and is not necessarily an exact illustration. Therefore, for example, the scales of the figures do not necessarily match. Furthermore, in each figure, substantially the same components are given the same reference numerals, and redundant explanations are omitted or simplified.

[0051] Furthermore, in this specification, terms indicating relationships between elements such as "same," as well as numerical values ​​and numerical ranges, are not expressions that only express a strict meaning, but are expressions that also include a substantially equivalent range, for example, a difference of about several percent (or about 10%).

[0052] Furthermore, in this specification, ordinal numbers such as "first" and "second" do not refer to the number or order of components unless otherwise specified, but are used for the purpose of avoiding confusion and distinguishing between components of the same type.

[0053] First Embodiment An information processing system including an information processing device according to this embodiment will be described below with reference to FIGS.

[0054] [1-1. Configuration of Information Processing System] First, the configuration of an information processing system according to this embodiment will be described with reference to Fig. 1. Fig. 1 is a block diagram showing the functional configuration of an information processing system 1 according to this embodiment.

[0055] The information processing system 1 is a detection system for detecting the presence or absence of a person (whether or not a person is present) in a specified space, and is used to detect a person, for example, in an audio speech system including a device with an audio speech function. In the audio speech system, after audio corresponding to the speech content is generated, when the information processing device 20 detects a person, the audio is played from a speaker or the like. As will be described in detail later, the information processing device 20 can detect a person even if the person is not currently moving, thereby enabling more accurate detection of the presence or absence of a person. This allows the audio speech system to more accurately determine the timing of speech, thereby further improving the effectiveness of speech.

[0056] The predetermined space may be, for example, a space in which an audio speech system (for example, a speaker) is installed. The predetermined space may be an indoor space or an outdoor space.

[0057] As shown in FIG. 1 , the information processing system 1 includes a detection device 10 and an information processing device 20 .

[0058] The detection device 10 is a device that emits and receives ultrasonic waves to detect a person present in a predetermined space. The detection device 10 is configured to emit ultrasonic waves into a predetermined space and receive the sound reflected by the person. The detection device 10 may be realized as part of a voice speech system. The detection device 10 may also be a portable device that can be carried around freely, or may be a fixed device. The detection device 10 is communicably connected to an information processing device 20 via a network N, and transmits information acquired by the detection device 10 to the information processing device 20.

[0059] The detection device 10 includes a sound generating unit 11 , a sound receiving unit 12 , a control unit 15 , and a communication unit 16 .

[0060] The sound generator 11 is, for example, an ultrasonic transmitter having a speaker or the like, which emits ultrasonic waves into a predetermined space. The sound generator 11 may, for example, periodically emit ultrasonic waves. The sound generator 11 may, for example, emit burst waves of a frequency outside the audible range (for example, 20 kHz or higher) at a specific period. The period is, for example, but is not limited to, 25 ms. The sound generator 11 may also emit ultrasonic waves of a single frequency, or may emit ultrasonic waves of multiple frequencies synchronously and / or asynchronously. For example, the burst waves may include multiple frequency components. The emitted sound emitted from the sound generator 11 is reflected by a person and collected by the sound receiver 12 as reflected sound.

[0061] A burst wave is a signal that is transmitted at a predetermined time interval. In a burst wave, regions where a signal is present and regions where a signal is not present are repeated in the time domain. The use of a burst wave has the advantage of being able to capture direct waves and reflected waves separately. Note that the sound generation unit 11 is not limited to transmitting burst waves as long as it can periodically transmit ultrasonic waves. For example, instead of a burst wave, it may transmit a chirp signal whose frequency increases or decreases over time.

[0062] The sound receiving unit 12 is a receiver that has, for example, one or more microphones and receives reflected waves of ultrasonic waves emitted by the sound generating unit 11. The sound receiving unit 12 also receives direct waves of ultrasonic waves emitted by the sound generating unit 11. The direct waves are signals of ultrasonic waves emitted by the sound generating unit 11 that directly reach the sound receiving unit 12, and the reflected waves are signals received by the sound receiving unit 12 after the direct waves have arrived. The reflected waves are signals of ultrasonic waves emitted by the sound generating unit 11 that are reflected by an object such as a person.

[0063] The control unit 15 is a processor that controls the sound generating unit 11, the sound receiving unit 12, and the communication unit 16. The control unit 15 transmits information about a sound generating signal for transmitting a sound by the sound generating unit 11 and information about a sound receiving signal based on a reflected sound acquired by the sound receiving unit 12 to the information processing device 20 via the communication unit 16.

[0064] The communication unit 16 is a communication module that is communicably connected to the information processing device 20 via the network N. The communication unit 16 is connected to the network N wirelessly, for example.

[0065] The information processing device 20 is a device for detecting a person present in a predetermined space based on a signal from the detection device 10. Specifically, the information processing device 20 is a device for detecting a person present in a predetermined space based on information based on a sound signal obtained by receiving a reflected wave obtained by transmitting ultrasonic waves into the space using a sound receiving unit 12 such as a microphone.

[0066] The information processing device 20 has, as its functional configuration, a communication unit 21, a reflected wave extraction unit 22, an average calculation unit 23, a storage unit 24, a difference calculation unit 25, a smoothing processing unit 26, a determination unit 27, and an output unit 28. The information processing device 20 is realized by a non-volatile memory in which a program is stored, a volatile memory that is a temporary storage area for executing the program, an input / output port, a communication interface, a processor that executes the program, and the like. The information processing device 20 is realized by, for example, a server device. The information processing device 20 may be a stationary personal computer (PC), a portable terminal such as a portable PC, a smartphone, or a tablet, or a dedicated computer.

[0067] The communication unit 21 is a communication module, and is communicably connected to the detection device 10 via the network N. The communication unit 21 acquires information output from the detection device 10 and outputs it to the reflected wave extraction unit 22.

[0068] The reflected wave extraction unit 22 has, for example, a band-pass filter, and removes unnecessary frequencies from the signal received by the sound receiving unit 12 to extract a signal (reflected wave data) in the reflected wave section. The extracted reflected wave data is stored in the memory unit 24. The unnecessary frequencies are frequencies other than the frequency emitted by the sound generating unit 11. The extracted reflected wave is an example of a first signal.

[0069] The reflected wave extraction unit 22 may process only a specific frequency, such as the frequency of the transmitted signal, in the frequency domain. In this case, the reflected wave extraction unit 22 does not need to include a band-pass filter.

[0070] The average calculation unit 23 is a processing unit that calculates the average of a predetermined number of seconds of reflected wave data or a predetermined number of pieces of reflected wave data stored in the storage unit 24. The reflected wave data is represented by time on the horizontal axis and amplitude or normalized amplitude values ​​on the vertical axis, and the average calculation unit 23 calculates the average value of the amplitude or normalized amplitude values ​​for the same time period for each time period. In other words, the average calculation unit 23 calculates time-series data of the average value of the amplitude or normalized amplitude values ​​for the same time period.

[0071] Note that the average calculation unit 23 is not limited to calculating an average value, and may calculate a median, a mode, etc. Furthermore, the reflected wave data stored in the storage unit 24 is an example of a second signal based on a second reflected wave acquired earlier than the first reflected wave.

[0072] The average calculation unit 23 may process only a specific frequency, such as the frequency of the transmitted signal, in the frequency domain. In this case, the reflected wave extraction unit 22 does not need to include a band-pass filter.

[0073] When the difference calculation unit 25 calculates the difference using one piece of past reflected wave data (for example, the most recent reflected wave data), the average calculation unit 23 does not need to be provided.

[0074] The storage unit 24 stores past reflected wave data and the standard deviation (an example of a feature) of past differential signals. For example, the storage unit 24 may store only a predetermined number of seconds of reflected wave data or a predetermined number of pieces of reflected wave data as past reflected wave data. In other words, the storage unit 24 may delete reflected wave data older than a predetermined number of seconds or older reflected wave data when the number of pieces exceeds a predetermined number. Note that the storage unit 24 may also store the average of past reflected wave data calculated by the average calculation unit 23.

[0075] Furthermore, the storage unit 24 may store, for example, the standard deviation of the differential signal for a predetermined number of seconds or only the standard deviation of a predetermined number of differential signals as the standard deviation of the past differential signal. In other words, the storage unit 24 may delete the standard deviation of the differential signal older than a predetermined number of seconds, or delete the standard deviation of the older differential signal when the number of standard deviations exceeds a predetermined number. The storage unit 24 may be realized by, for example, a semiconductor memory or the like, but is not limited to this.

[0076] The difference calculation unit 25 is a processing unit that calculates the difference between the extracted reflected wave and the average of past reflected wave data. The difference calculation unit 25 finds the difference between the waveform of the received reflected wave and the average waveform of past reflected wave data. The difference calculation unit 25 calculates, for each time period, the difference between the amplitude or the normalized amplitude for the same time period.

[0077] The extracted reflected wave and past reflected wave data also include data on stationary objects in the space, such as desks. A differential signal (differential waveform) generated by taking the difference between the average of the extracted reflected wave and the past reflected wave data can be a signal in which the influence of stationary objects is reduced from the extracted reflected wave. Using such a differential waveform can increase the detection sensitivity for moving objects.

[0078] The difference calculation unit 25 also calculates the standard deviation of the amplitude of the difference signal. The standard deviation is a scalar value.

[0079] The difference calculation unit 25 may process only a specific frequency, such as the frequency of the transmitted signal, in the frequency domain. In this case, the reflected wave extraction unit 22 does not need to include a band-pass filter.

[0080] The smoothing processing unit 26 is a processing unit that performs averaging processing to remove minute fluctuation components. In this embodiment, the smoothing processing unit 26 performs moving average processing to calculate a moving average of a predetermined number of sample points. The smoothing processing unit 26 calculates the moving average of the standard deviation calculated by the difference calculation unit 25 and the past standard deviations stored in the storage unit 24 as a feature amount for detecting the presence or absence of a person. Note that the smoothing processing unit 26 is not an essential component. In other words, the smoothing processing does not have to be performed.

[0081] The smoothing processor 26 may process only a specific frequency, such as the frequency of the transmitted signal, in the frequency domain. In this case, the reflected wave extractor 22 does not need to include a band-pass filter.

[0082] The determination unit 27 is a processing unit that determines whether or not a person is present in the space based on the feature value output from the smoothing processing unit 26. In this embodiment, the determination unit 27 determines whether or not a person is present in the space based on the feature value and a preset threshold value.

[0083] The output unit 28 outputs the determination result obtained by the determination unit 27. The output unit 28 may transmit the determination result to, for example, a voice output system. The output unit 28 may include, for example, a communication interface.

[0084] [1-2. Operation of Information Processing System] Next, the operation of the information processing system 1 configured as described above will be described with reference to Figs. 2 to 4. Fig. 2 is a flowchart showing the operation (information processing method) of the information processing system 1 according to this embodiment. Steps S11 and S12 shown in Fig. 2 are operations performed by the detection device 10, and steps S13 to S17 are operations performed by the information processing device 20.

[0085] 2, first, the sound generator 11 emits ultrasonic waves (S11), and the sound receiver 12 receives the ultrasonic waves (S12). The sound receiver 12 receives, for example, direct waves and reflected waves. Note that it is sufficient for the sound receiver 12 to receive at least reflected waves corresponding to the detection range of a person.

[0086] The control unit 15 transmits information about the ultrasonic waves transmitted in step S11 and information about the ultrasonic waves received in step S12 to the information processing device 20 via the communication unit 16. Then, the information processing device 20 acquires the information transmitted from the detection device 10 via the communication unit 21. Note that the information about the received ultrasonic waves includes information indicating the reflected waves (waveform data of the reflected waves). The information about the received ultrasonic waves is an example of information indicating the first reflected waves. Furthermore, the communication unit 21 functions as an acquisition unit that acquires information indicating the first reflected waves via communication.

[0087] Next, the information processing device 20 calculates a feature amount based on at least the reflected wave (S13). Step S13 will be described later with reference to FIG.

[0088] Next, the determination unit 27 determines whether the feature amount is equal to or greater than a threshold value (S14). If the feature amount is equal to or greater than the threshold value (Yes in S14), the determination unit 27 determines that a person is present in the space, and if the feature amount is less than the threshold value (No in S14), the determination unit 27 determines that a person is not present in the space.

[0089] Next, if the determination unit 27 determines that the feature is greater than or equal to the threshold, the output unit 28 outputs a determination result of "presence" indicating that a person is present in the space (e.g., the detection target range within the space) (S15), and if the determination unit 27 determines that the feature is less than the threshold, the output unit 28 outputs a determination result of "absence" indicating that a person is not present in the space (e.g., the detection target range within the space) (S16).

[0090] Next, the information processing device 20 determines whether the presence / absence detection has been completed (S17), and if it is determined that the detection has been completed (Yes in S17), it terminates the processing, and if it is determined that the detection has not been completed (No in S17), it returns to step S11 and continues the processing. If it is determined No in step S17, the information processing device 20 transmits information to the detection device 10 instructing it to execute steps S11 and S12 again.

[0091] The execution timing of the process shown in FIG. 2 is not particularly limited, and the process may be executed at a predetermined timing or periodically.

[0092] Fig. 3 is a flowchart showing the detailed operation (information processing method) of step S13 shown in Fig. 2. Fig. 4 is a diagram showing examples of various waveforms according to this embodiment. In Fig. 4, the horizontal axis represents time and the vertical axis represents intensity.

[0093] As shown in Fig. 3, the reflected wave extraction unit 22 performs band-pass filtering on the signal received by the sound receiving unit 12 (S131). The reflected wave extraction unit 22 extracts a signal at the frequency of the ultrasonic waves emitted by the sound generating unit 11 from the signal received by the sound receiving unit 12. For example, the signal shown in Fig. 4(a) is extracted. Fig. 4(a) shows a signal including a direct wave with a large amplitude (for example, up to approximately 0.0025 s) and a reflected wave with a small amplitude (for example, after approximately 0.0025 s).

[0094] Referring back to FIG. 3 , the reflected wave extraction unit 22 then extracts a reflected wave from the band-pass filtered signal based on the elapsed time from the direct wave (S132). The difference between the arrival time of the direct wave and the arrival time of the reflected wave is multiplied by the speed of sound to convert the difference into the distance from the information processing device 20 (e.g., the sound generator 11) to the reflector. For example, if a detection range for detecting the presence or absence of a person is set, the elapsed time corresponding to the detection range can be calculated. Furthermore, the arrival time of the direct wave can be obtained in advance based on the relative positions of the sound generator 11 and the sound receiver 12. Therefore, the reflected wave extraction unit 22 extracts a signal from the detection range from the signal shown in FIG. 4A based on the elapsed time from the direct wave. In FIG. 4A, the time when the sound generator 11 emits ultrasonic waves is set to 0 s.

[0095] The detection target range can be arbitrarily set by changing the elapsed time. For example, it is also possible for the information processing device 20 to select a specific distance or distance section (a section for analyzing the presence or absence of a person) as the analysis target.

[0096] Fig. 4(b) shows the signal in the detection target range extracted by the reflected wave extraction unit 22. Note that in Fig. 4(b), the time corresponding to the closest distance in the detection target range is set to 0 ms. Also, in Fig. 4(b), the amplitude is shown as a normalized value, with the maximum value of the amplitude of the direct wave shown in Fig. 4(a) being set to 1. In Fig. 4(b), the amplitude is relatively small up to 4 ms, and increases after 4 ms.

[0097] 3 again, next, the reflected wave extraction unit 22 stores the reflected wave data shown in Fig. 4(b) in the storage unit 24 (S133). The stored reflected wave data will be used as past reflected wave data the next time the process shown in Fig. 3 is executed.

[0098] Next, the average calculation unit 23 acquires past data of the reflected wave from the storage unit 24 (S134) and calculates the average of the past data of the reflected wave. The past data of the reflected wave is an example of the second reflected wave. Note that the past here means a time before the reflected wave was acquired in step S12, and includes, for example, the last time or when the process shown in FIG. 2 was executed before the last time.

[0099] Next, the difference calculation unit 25 calculates the difference between the reflected wave shown in Fig. 4(b) and the average of the past data (S135). The reflected wave shown in Fig. 4(b) is an example of a first signal, and the average of the past data is an example of a second signal based on information indicating the second reflected wave.

[0100] FIG. 4C shows a signal (difference signal) indicating the difference between the reflected wave shown in FIG. 4B and the average of past data. In the differential signal shown in FIG. 4C, the amplitude of the reflected wave shown in FIG. 4B is significantly reduced after 4 ms. This is the result of reducing the influence of stationary objects in space. In other words, a stationary object exists in space at a position corresponding to a time after 4 ms, and the reflected wave shown in FIG. 4B includes a reflected wave from the stationary object. However, in the differential signal shown in FIG. 4C, the reflected wave from the stationary object has been removed by taking the difference from the past data.

[0101] Referring back to FIG. 3 , the difference calculation unit 25 then calculates the standard deviation of the difference signal (S136). The standard deviation here is the standard deviation in the time axis direction, and can also be said to be the standard deviation of the amplitude corresponding to the detection target range. Because the reflected wave contains the state of the space including the person, the amplitude of the reflected wave also reflects the state of the reflector. By calculating the standard deviation (variance) of the reflected wave, a feature quantity representing the characteristics of the amplitude of the reflected wave can be obtained. The difference calculation unit 25 stores the calculated standard deviation in the storage unit 24. The standard deviation is a value indicating the fluctuation of the amplitude and is a scalar value. The standard deviation, i.e., the feature quantity, may also be a dimensionless quantity. The standard deviation calculated in step S136 is an example of a first standard deviation.

[0102] Next, the smoothing processing unit 26 acquires past data of standard deviation from the storage unit 24 (S137), and calculates a moving average of the standard deviation calculated in step S136 and the past data of standard deviation (S138). The value of the moving average of standard deviation calculated in step S138 is used as a feature amount in step S14 and subsequent steps shown in Fig. 2. The past data of standard deviation is an example of one or more second standard deviations calculated earlier than the first standard deviation.

[0103] Next, the smoothing processing unit 26 stores the moving averaged standard deviation in the storage unit 24 (S139). The stored standard deviation will be used as past data of the standard deviation the next time the process shown in FIG. 3 is executed.

[0104] As described above, by using the differential signal, for example, if a person was moving at a past point in time and is not moving at the present point in time, the amplitude of the differential signal becomes large, and therefore the information processing device 20 can detect a person regardless of the person's movements.

[0105] (Variation 1 of Embodiment 1) An information processing system according to this variation will be described below with reference to Figures 5 and 6. Note that in each of the following variations, differences from embodiment 1 will be mainly described, and descriptions of content that is the same as or similar to embodiment 1 will be omitted or simplified. The configuration of the information processing system in each of the following variations may be the same as the information processing system 1 according to embodiment 1, and each variation will be described using the reference numerals of the information processing system 1 according to embodiment 1.

[0106] 5 is a flowchart showing the operation (information processing method) of the information processing device 20 according to this modification, which corresponds to step S13 shown in FIG. In this modification, the information processing device 20 is described, which is capable of detecting the presence of a person even when there is little human movement. Specifically, an example in which peak hold processing is used as the smoothing processing will be described.

[0107] 5, the smoothing processing unit 26 acquires past data of the standard deviation (S137) and then executes peak hold processing (S138a). The peak hold processing is processing for storing the maximum value of the standard deviation acquired during a predetermined period.

[0108] The smoothing processor 26 executes peak hold processing based on the following equations 1 to 3.

[0109] Ph=(1-αf)×x[n]+αf×Ph(x[n]>Ph)...Formula 1 Ph=(1-αs)×x[n]+αs×Ph(otherwise)...Formula 2 xh[n]=Ph...Formula 3

[0110] Here, Ph represents the peak hold value, x represents the original signal, xh represents the output signal of the peak hold process, αf represents the rise coefficient of the peak hold, αs represents the attenuation coefficient of the peak hold, and n represents the time index. The time index is a numerical value that indicates the ordinal number of the data in the data string. Equations 1 to 3 are stored in advance in the storage unit 24.

[0111] The smoothing processor 26 sequentially updates the peak hold value based on Equations 1 to 3. If the current value (amplitude of the differential signal) is higher than the peak hold value, the smoothing processor 26 updates the peak hold value by multiplying it by the rise coefficient αf, and if the current value is equal to or lower than the peak hold value, the smoothing processor 26 updates the peak hold value by multiplying it by the attenuation coefficient αs.

[0112] Furthermore, the smoothing processing unit 26 stores the peak-hold processed standard deviation in the storage unit 24 (S139a).

[0113] FIG. 6 shows examples of various waveforms according to this modification. The horizontal axis of FIG. 6 represents time, and the vertical axis represents the amplitude of the differential signal (Feature Value [a.u.] in FIG. 6). FIG. 6(a) shows a waveform obtained by performing moving average processing as a smoothing process (the waveform after processing shown in FIG. 6(a)). FIG. 6(b) shows a waveform obtained by performing peak hold processing as a smoothing process (the waveform after processing shown in FIG. 6(b)). Note that the pre-processing (solid line) and presence / absence labels (dashed dotted lines) shown in FIGS. 6(a) and 6(b) are the same. Furthermore, the pre-processing indicates the differential signal before the smoothing process, and the post-processing indicates the differential signal after the smoothing process, and the presence / absence labels indicate the results of the presence / absence determination by the determination unit 27.

[0114] As shown in the waveforms after processing in Figures 6(a) and 6(b), by using peak hold processing as the smoothing process, the peak value of the feature due to the most recent human movement can be maintained while attenuating it, so that even when there is little human movement, a high feature value can be maintained for a longer period than the moving average. This improves the accuracy of human detection when there is little human movement. For example, by using peak hold processing, it is possible to detect the presence of a person even when there is little human movement for a predetermined period (e.g., several seconds) or more.

[0115] It should be noted that the effect can be made longer (the previous value has an effect for a longer period of time) by making the value of the attenuation coefficient αs closer to 1. The attenuation coefficient αs and the rise coefficient αf are set in advance and stored in the storage unit 24.

[0116] (Second Modification of First Embodiment) Hereinafter, an information processing device 20 according to this modification will be described with reference to FIG.

[0117] 7 is a flowchart showing the operation (information processing method) of the information processing device 20 according to this modified example, which corresponds to step S13 shown in FIG. 2. In this modified example, an information processing device 20 that can reduce the influence on feature calculation even when the ultrasonic waves from the sound generation unit 11 are unstable will be described. Specifically, an example will be described in which the presence or absence of a person is detected using a signal obtained by convolving a direct wave and a reflected wave instead of a reflected wave. Note that in this modified example, in step S12 shown in FIG. 2, a direct wave (first direct wave) is also received in addition to the reflected wave.

[0118] As shown in Fig. 7, the reflected wave extraction unit 22 divides the band-pass filtered signal into a direct wave and a reflected wave (S141). Since the time at which the direct wave is received can be obtained in advance, the reflected wave extraction unit 22, for example, uses this time to divide the band-pass filtered signal into a direct wave and a reflected wave. For example, in the case of the signal shown in Fig. 4(a), the reflected wave extraction unit 22 may divide the signal into a direct wave and a reflected wave before 0.0025 s.

[0119] Next, the reflected wave extraction unit 22 performs convolution processing on the direct wave and the reflected wave (S142). The convolution processing here includes performing a binary operation in which one of the direct wave and the reflected wave is translated and then superimposed on the other of the direct wave and the reflected wave. The convolution processing on the direct wave and the reflected wave corresponds to calculating the impulse response of the direct wave and the reflected wave.

[0120] This makes it possible to extract reflected waves that are highly correlated with the direct wave, thereby reducing the influence of fluctuations in the direct wave. Fluctuations in the direct wave can be exemplified by, for example, changes in amplitude with each transmission. The signal generated by convolving the direct wave and the reflected wave is an example of a first convolved signal included in the first signal. The reflected wave extraction unit 22 may perform processing in the frequency domain. For example, when processing in the frequency domain, the reflected wave extraction unit 22 may process only a specific frequency, such as the frequency of the transmitted signal. In this case, the reflected wave extraction unit 22 does not need to include a band-pass filter.

[0121] Next, the reflected wave extraction unit 22 stores the convolution-processed data (first convolution signal) in the storage unit 24 (S133b). The stored data will be used as convolution-processed past data the next time the process shown in FIG. 7 is executed.

[0122] Next, the average calculation unit 23 acquires the convolution-processed past data from the storage unit 24 (S134b) and calculates the average of the convolution-processed past data. The past data, i.e., the signal that was convolution-processed in the past, is an example of a second convolution signal. The reflected wave and direct wave acquired when the process shown in FIG. 7 was previously performed are an example of a second reflected wave and a second direct wave, and the signal generated by convolving the second reflected wave and the second direct wave is an example of a second convolution signal included in the second signal.

[0123] Next, the difference calculation unit 25 calculates the difference between the first convolved signal and the average of past data (for example, the second convolved signal) (S135). Note that when one second convolved signal (for example, the most recent second convolved signal) is selected from the multiple second convolved signals, the difference calculation unit 25 may calculate the difference between the first convolved signal and the one second convolved signal.

[0124] Next, the difference calculation unit 25 executes the process of step S136 using the past data that has been subjected to the convolution process.

[0125] Next, the smoothing processing unit 26 acquires past data of standard deviation from the storage unit 24 (S137) and performs a smoothing process on the standard deviation calculated in step S136 and the past data of standard deviation (S138b). Examples of the smoothing process include, but are not limited to, moving average processing and peak hold processing. The smoothing processing unit 26 then stores the smoothed standard deviation in the storage unit 24 (S139b). The stored standard deviation is used as past data of standard deviation the next time the process shown in FIG. 7 is executed. The smoothed standard deviation is an example of a value based on the standard deviation of the amplitude of the differential signal. In this modification, in step S12 shown in FIG. 2, a direct wave is also received in addition to a reflected wave.

[0126] (Third Modification of First Embodiment) Hereinafter, an information processing device 20 according to this modification will be described with reference to FIGS. 8 to 10. FIG.

[0127] Fig. 8 is a flowchart showing the operation (information processing method) of the information processing device 20 according to this modified example, corresponding to step S13 shown in Fig. 2. Fig. 9 is a diagram showing examples of various waveforms according to this modified example. The vertical axis of Fig. 9 indicates amplitude, and the horizontal axis indicates the number of samples (number of samplings). In this modified example, an information processing device 20 is described that can reduce the deterioration in the accuracy of detecting the presence or absence of a person, which occurs when the value of a feature value fluctuates due to changes in the state of the space. An example of a change in the state of the space is a change in the air flow in the space, such as when an air conditioner is turned on.

[0128] As shown in FIG. 8, the reflected wave extraction unit 22 generates a direct wave signal and a reflected wave signal based on the signal that has been subjected to band-pass filtering (S151).

[0129] 9A shows a signal (original signal) that has been band-pass filtered. The original signal includes a direct wave within the dashed frame and a reflected wave outside the dashed frame. The dashed frame indicates the area corresponding to the direct wave, including the direct wave. The reflected wave outside the dashed frame is a reflected wave that corresponds to the detection target range. The sampling frequency of the original signal is higher than twice the transmission frequency of the sound source, for example, 96 kHz, but is not limited to this.

[0130] As shown in (b) of FIG. 9, the reflected wave extraction unit 22 generates a signal as a direct wave signal by replacing parts of the original signal other than the direct wave part (e.g., the reflected wave signal part) with zero. Replacing with zero is a process of replacing amplitude values ​​with zero. The direct wave signal may be, for example, a signal in which the amplitude of the reflected wave in a received signal including a direct wave and a reflected wave is set to zero. Furthermore, as shown in (c) of FIG. 9, the reflected wave extraction unit 22 generates a signal as a reflected wave signal by replacing the direct wave part of the original signal with zero. The reflected wave signal may be, for example, a signal in which the amplitude of the direct wave (e.g., the part corresponding to the direct wave) in a received signal including a direct wave and a reflected wave is set to zero.

[0131] This makes it possible to generate two waveform data sets with the same number of samples (same time width).

[0132] 8 again, the reflected wave extraction unit 22 estimates an impulse response by an adaptive filter for each sample of the direct wave signal and the reflected wave signal (S152). In the second modification of the first embodiment, one impulse response is estimated for one original signal, but in this modification, impulse responses for the number of samples are estimated for one original signal. The estimated impulse response is an example of the first impulse response included in the first signal.

[0133] This allows the calculation period of the impulse response to be the sampling period, which means that the calculation period of the impulse response is no longer restricted by the emission period of the sound source, and can be made shorter than the emission period of the sound source.

[0134] Next, the reflected wave extraction unit 22 stores the estimated impulse response data in the storage unit 24 (S133c). The estimated impulse response data will be used as past data of the estimated impulse response the next time the process shown in FIG. 8 is executed.

[0135] Next, the average calculation unit 23 acquires past data of the estimated impulse response from the storage unit 24 (S134c), and calculates the average of the past impulse response data. The past impulse response data is an impulse response acquired earlier than the first impulse response, and is an example of the second impulse response included in the second signal.

[0136] Next, the difference calculation unit 25 calculates the difference between the estimated impulse response and the average of the past data (S135).

[0137] The standard deviation calculated using this process is calculated at a time interval shorter than the emission period of the sound source. For example, when the spatial condition is changing, such as when wind is blowing in the space, the calculation period for the standard deviation is short, so the amplitude of the differential signal is smaller than when the differential signal is calculated at the emission period of the sound source. This prevents the feature value from being calculated as being larger than its actual value due to changes in the spatial condition, thereby preventing erroneous determinations of the presence or absence of a person. In other words, the system is more responsive to slight changes in the spatial condition and more robust against external disturbances.

[0138] Here, a functional configuration for estimating an impulse response for each sample will be described with reference to Fig. 10. Fig. 10 is a block diagram showing a functional configuration for estimating an impulse response by the adaptive filter 134 according to this modification. For example, each of the components shown in Fig. 10 may be included in the reflected wave extraction unit 22.

[0139] As shown in FIG. 10, the reflected wave extraction unit 22 (see FIG. 1) includes a bandpass filter 131 (BPF in FIG. 10), a reflected wave signal generation unit 132, a direct wave signal generation unit 133, and an adaptive filter 134.

[0140] The band-pass filter 131 is a filter for performing band-pass filtering in step S131.

[0141] The reflected wave signal generating unit 132 is a processing unit that generates a reflected wave signal by replacing the direct wave portion of the original signal with zero.

[0142] The direct wave signal generating unit 133 is a processing unit that generates a direct wave signal by replacing the portion of the original signal other than the direct wave portion (for example, the portion of the reflected wave signal) with zero.

[0143] When using the adaptive filter 134, the signals must have the same time (the same timing of 0), the same sample period, and the same signal length. The reflected wave signal generated by the reflected wave signal generator 132 and the direct wave signal generated by the direct wave signal generator 133 are signals that satisfy these conditions.

[0144] The adaptive filter 134 is a filter that self-adapts its transfer function according to a predetermined algorithm (e.g., an optimization algorithm). In the example of Fig. 10, the adaptive filter 134 is updated so that a signal obtained by convolving the impulse response of the adaptive filter 134 with the direct wave signal matches the reflected wave signal. If the two signals do not match, for example, an error is detected and the adaptive filter 134 is updated. The impulse response is determined by performing an inverse Fourier transform on the transfer function.

[0145] The sample interval for estimating the impulse response may be set arbitrarily taking into consideration the amount of calculation, etc. In other words, it is not necessary to estimate the impulse response for all of the samples included in the received signal.

[0146] (Fourth Modification of First Embodiment) The information processing device 20 according to this modification will be described below with reference to Figs. 11 to 14. In this modification, an example will be described in which a change over time in the amplitude shape of a reflected wave or a convoluted signal is added to a feature. Fig. 11 is a flowchart showing the operation (information processing method) of the information processing device 20 according to this modification, which corresponds to step S13 shown in Fig. 2.

[0147] 11 , the reflected wave extraction unit 22 extracts a reflected wave from the band-pass filtered signal, or generates a convoluted signal by convolving the direct wave and the reflected wave (S161), and stores the generated signal data in the storage unit 24 (S133d). The stored data is used as past signal data the next time the process shown in FIG. 11 is executed. An example in which a reflected wave is extracted in step S161 will be described below.

[0148] Next, the average calculation unit 23 acquires past data of the generated signal (here, past data of the reflected wave) from the storage unit 24 (S134d), and calculates the average of the past data of the generated signal.

[0149] Next, the difference calculation unit 25 executes a first feature amount calculation process (S162) and a second feature amount calculation process (S163).

[0150] FIG. 12 is a flowchart showing the detailed operation (information processing method) of step S162 shown in FIG.

[0151] As shown in FIG. 12, in the first feature amount calculation process, the processes of steps S135 and S136 described in the first embodiment and the like are executed.

[0152] FIG. 13 is a flowchart showing the detailed operation (information processing method) of step S163 shown in FIG.

[0153] As shown in FIG. 13 , in the second feature calculation process, the difference calculation unit 25 first calculates the envelope of the reflected wave extracted in step S161 (S163a). The envelope is a line connecting the maximum amplitudes of the reflected wave. Using the envelope allows attention to be focused on the amplitude of the reflected wave alone, thereby eliminating the influence of phase shifts in the reflected wave due to disturbances such as air currents. The envelope calculated in step S163a is an example of a first envelope signal. The envelope is a signal that indicates the temporal change in the amplitude of the reflected wave and is included in the first signal.

[0154] 14A and 14B are diagrams showing examples of various waveforms according to this modification. (a) of Fig. 14A shows an envelope curve connecting the maximum values ​​of the positive amplitudes of the reflected waves shown in (b) of Fig. 4. The difference calculation unit 25 calculates, for example, an envelope curve as shown in (a) of Fig. 14A.

[0155] Referring again to FIG. 13 , next, the difference calculation unit 25 calculates the difference with respect to the envelope of one block before (S163b). That is, the difference calculation unit 25 calculates the difference between the past envelope and the envelope calculated in step S163a. ​​For example, the difference calculation unit 25 calculates a difference signal as shown in FIG. 14(b). Note that the past envelope is an example of the second envelope signal. The past envelope is also a signal indicating the change over time in the amplitude of the second reflected wave, which was acquired earlier than the first reflected wave, and is included in the second signal.

[0156] Next, the difference calculation unit 25 calculates the maximum value of the difference signal (S163c). The difference calculation unit 25 calculates the maximum amplitude value of the difference signal shown in (b) of FIG. 14. The maximum amplitude value is, for example, the maximum absolute value of the difference (amplitude) in the time axis direction. Here, the maximum value is a scalar value. Furthermore, the maximum value, i.e., the feature amount, may be a dimensionless amount.

[0157] 11 , next, the smoothing processing unit 26 adds two feature amounts (S164). Here, the smoothing processing unit 26 adds the maximum value and the standard deviation. Note that the smoothing processing unit 26 may perform weighted addition of the maximum value and the standard deviation, or may calculate a single scalar value from the maximum value and the standard deviation using an operation other than addition (for example, at least one of subtraction, multiplication, and division).

[0158] In this way, information (envelope) indicating the time change in the amplitude shape of the reflected wave may be added to the feature. For example, the presence or absence of a person may be detected using two feature values. By observing the time change in the amplitude shape, it is possible to more easily capture changes in the reflected waveform due to the movement of the reflector. Furthermore, by using information indicating the time change in the amplitude shape of the reflected wave and by using two feature values, it is possible to improve robustness against amplitude fluctuations in the reflected wave due to refraction of the sound wave by airflow.

[0159] Although the above describes an example of calculating a feature for determining whether a person is present or absent by adding two feature amounts, it is also possible to calculate a feature for determining whether a person is present or absent by adding, for example, three or more feature amounts.

[0160] The information processing device 20 may determine the presence or absence of a person by using only the maximum value of the envelope out of the standard deviation and the maximum value of the envelope.

[0161] (Variation 5 of Embodiment 1) Hereinafter, an information processing device 20 according to this variation will be described with reference to FIG. 15. In this variation, a case where burst waves of multiple frequencies are used will be described. FIG. 15 is a flowchart showing the operation (information processing method) of the information processing device 20 according to this variation, corresponding to step S13 shown in FIG. 2. The multiple frequencies are not particularly limited, but examples include 20 kHz, 30 kHz, and 40 kHz. Furthermore, the number of frequencies is not particularly limited as long as it is two or more.

[0162] 15, in steps S131e to S136e, the information processing device 20 executes the processes of steps S131 to S136 shown in FIG. 3 for each frequency of the sound source (the frequency of the burst wave). That is, the processes of steps S131e to S136e are executed for the reflected waves for each frequency of the sound source. For example, in step S132e, the reflected waves of the sound source frequency value are extracted, in step S135e, the difference between the sound source frequency values ​​is calculated, and in step S136e, the standard deviation of the sound source frequency values ​​is calculated.

[0163] Next, the smoothing processing unit 26 adds up the standard deviations calculated for each frequency (S171). The smoothing processing unit 26 may perform weighted addition of each standard deviation, or may calculate one scalar value from each standard deviation using an operation other than addition (for example, at least one of subtraction, multiplication, and division).

[0164] In this way, the information processing device 20 may use burst waves of multiple frequencies as the sound source, calculate the feature amount (for example, standard deviation) of each frequency, and add them up to calculate one feature amount (the feature amount used to determine presence / absence). This makes it possible to reduce the influence of noise of a specific frequency or noise caused by environmental changes such as temperature, humidity, and air currents on the determination of presence / absence of a person.

[0165] In this modification, the sound generator 11 may emit burst waves of a plurality of frequencies simultaneously, or may emit the waves at different timings.

[0166] (Embodiment 2) An information processing system according to this embodiment will be described below with reference to Figures 16 and 17. Note that the following description will focus on differences from embodiment 1, and descriptions of content that is the same as or similar to embodiment 1 will be omitted or simplified.

[0167] [2-1. Configuration of Information Processing System] First, the configuration of an information processing device according to this embodiment will be described with reference to Fig. 16. Fig. 16 is a block diagram showing the functional configuration of an information processing system 1a according to this embodiment. In this embodiment, an example of detecting a person using a machine learning model that inputs feature amounts and outputs a detection result of the presence or absence of a person will be described.

[0168] 16 , the information processing device 20a includes, in addition to the components of the information processing device 20 according to the first embodiment, storage units 31 and 33 and a model construction unit 32. The determination unit 27 determines the presence or absence of a person using the machine learning model created by the model construction unit 32.

[0169] The storage unit 31 stores learning data used by the model construction unit 32 to learn a machine learning model. The learning data includes multiple pairs of feature amounts, which are input data, and teacher data on the presence or absence of people at that time. Here, the feature amount is one standard deviation, but it may also be time-series data of the standard deviation, waveform data of a differential signal, or the like. In other words, the generated machine learning model may be a model to which one standard deviation (i.e., one scalar value) is input as input information, a model to which time-series data of the standard deviation (e.g., time-series data of the standard deviation for two seconds), or a model to which the waveform data of the differential signal itself is input. The storage unit 31 may be realized, for example, by a semiconductor memory or the like, but is not limited thereto.

[0170] The model construction unit 32 generates a machine learning model trained on the training data. More specifically, the model construction unit 32 generates the machine learning model by performing supervised learning using the training data. When feature values ​​are input, the generated machine learning model (trained model) outputs an inference result regarding the presence or absence of a person in a space. The model construction unit 32 also stores the generated trained model in the storage unit 33.

[0171] The model construction unit 32 includes a computer including, for example, a memory and a processor (microprocessor), and realizes various functions by the processor executing a control program stored in the memory.

[0172] The storage unit 33 stores the trained model generated by the model construction unit 32. The storage unit 33 is realized by, for example, a semiconductor memory, but is not limited to this.

[0173] The judgment unit 27 reads out the trained model from the memory unit 33, and obtains the output (inference result) of the trained model obtained by inputting features into the read trained model as the judgment result of whether a person is present or absent.

[0174] The storage unit 31 and the model construction unit 32 may be realized by a device separate from the information processing device 20a. A learning device may be configured to include the storage unit 31 and the model construction unit 32, and the learning device may be communicably connected to the information processing device 20a.

[0175] In addition, when the waveform data of the differential signal itself is input to the machine learning model as a feature, the information processing device 20 a does not need to include the smoothing processing unit 26 .

[0176] 2-2. Operation of Information Processing System Next, the operation of the information processing system 1a configured as above will be described with reference to Fig. 17. Fig. 17 is a flowchart showing the operation (information processing method) of the information processing system 1a according to this embodiment.

[0177] 17 , after the feature amount is calculated (S13), the determination unit 27 inputs the feature amount into the trained model and obtains an inference result of "presence" or "absence" (S181). The determination unit 27 obtains the output of "presence" or "absence" obtained by inputting the feature amount into the trained model as the determination result of whether or not a person is present.

[0178] This allows the presence or absence of people to be determined using a machine learning model.

[0179] In this embodiment, a process of generating a machine learning model by the model construction unit 32 is executed as preprocessing before executing the operation shown in Fig. 17. The method of generating a machine learning model by the model construction unit 32 includes acquiring a plurality of feature amounts and a plurality of determination results as to whether or not a person is present, and learning the machine learning model using the acquired plurality of feature amounts as input information and the acquired plurality of determination results as correct answer data.

[0180] (Other Embodiments) The information processing method according to one or more aspects has been described above based on Embodiment 1, Modifications 1 to 5 of Embodiment 1, and Embodiment 2 (Embodiments, etc.), but the present disclosure is not limited to these Embodiments, etc. As long as it does not deviate from the spirit of the present disclosure, various modifications that a person skilled in the art can conceive of to the present embodiment, and embodiments constructed by combining components of different embodiments may also be included in the present disclosure.

[0181] For example, in the above-described first embodiment and the second modification of the first embodiment, it has been described that at least one of the reflected wave extraction unit, average calculation unit, difference calculation unit, and smoothing unit may perform processing in the frequency domain, but other processing units in the embodiments etc. may process only a specific frequency, such as the frequency of the transmitted signal, when performing processing in the frequency domain. In this case, the reflected wave extraction unit does not need to have a band-pass filter.

[0182] In the above embodiments, each component may be configured with dedicated hardware, or may be realized by executing a software program suitable for each component. Each component may be realized by a program execution unit such as a CPU or processor reading and executing a software program recorded on a recording medium such as a hard disk or semiconductor memory.

[0183] The order in which the steps in the flowchart are executed is merely an example for specifically explaining the present disclosure, and other orders may be used. Some of the steps may be executed simultaneously (in parallel) with other steps, or some of the steps may not be executed.

[0184] The division of functional blocks in the block diagram is an example, and multiple functional blocks may be realized as a single functional block, one functional block may be divided into multiple blocks, or some functions may be moved to another functional block.Furthermore, the functions of multiple functional blocks having similar functions may be processed in parallel or in time-sharing by a single piece of hardware or software.

[0185] Furthermore, the information processing device according to the above-described embodiments and the like may be realized as a single device or may be realized by multiple devices. When the information processing device is realized by multiple devices, the components of the information processing device may be distributed in any manner among the multiple devices. When the information processing device is realized by multiple devices, the communication method between the multiple devices is not particularly limited, and may be wireless communication or wired communication. Furthermore, wireless communication and wired communication may be combined between the devices. Furthermore, the information processing device according to the above-described embodiments and the like may be realized as a cloud server.

[0186] Furthermore, the detection device and the information processing device according to the above-described embodiments are not limited to being separate devices, and may be realized as a single device. When realized as a single device, the information indicating the first reflected wave may be a signal including the reflected wave received by the sound receiving unit, and the sound receiving unit may function as an acquisition unit that directly acquires (acquires by sensing) the information. Furthermore, the detection device may have some of the functions of the information processing device.

[0187] Furthermore, each component described in the above embodiments may be implemented as software or, typically, as an LSI, which is an integrated circuit. These components may be individually integrated into a single chip, or some or all of them may be integrated into a single chip. Here, the term "LSI" is used, but depending on the level of integration, it may also be referred to as an IC, system LSI, super LSI, or ultra LSI. Furthermore, the integrated circuit implementation method is not limited to LSI, and may be implemented using a dedicated circuit (a general-purpose circuit that executes a dedicated program) or a general-purpose processor. After LSI fabrication, a field programmable gate array (FPGA) that can be programmed or a reconfigurable processor that can reconfigure the connections or settings of circuit cells within the LSI may also be used. Furthermore, if an integrated circuit technology that replaces LSI emerges due to advances in semiconductor technology or a derivative technology, that technology may naturally be used to integrate the components.

[0188] A system LSI is an ultra-multifunctional LSI manufactured by integrating multiple processing units on a single chip. Specifically, it is a computer system that includes a microprocessor, ROM (Read Only Memory), RAM (Random Access Memory), etc. Computer programs are stored in the ROM. The system LSI achieves its functions when the microprocessor operates in accordance with the computer program.

[0189] Furthermore, one aspect of the present disclosure may be a computer program that causes a computer to execute each of the characteristic steps included in the information processing method shown in any of FIGS.

[0190] Furthermore, for example, the program may be a program to be executed by a computer. Another aspect of the present disclosure may be a computer-readable non-transitory recording medium on which such a program is recorded. For example, such a program may be recorded on a recording medium and distributed or circulated. For example, the distributed program may be installed in a device having another processor, and the program may be executed by the processor, thereby causing the device to perform each of the above processes.

[0191] The present disclosure is useful for an information processing device or the like for detecting the presence or absence of a person in a voice speech system or the like.

[0192] REFERENCE SIGNS LIST 1, 1a Information processing system 10 Detection device 11 Sound generation unit 12 Sound receiving unit (acquisition unit) 15 Control unit 16 Communication unit 20, 20a Information processing device 21 Communication unit (acquisition unit) 22 Reflected wave extraction unit 23 Average calculation unit 24, 31, 33 Storage unit 25 Difference calculation unit 26 Smoothing processing unit 27 Determination unit 28 Output unit 32 Model construction unit 131 Band-pass filter 132 Reflected wave signal generation unit 133 Direct wave signal generation unit 134 Adaptive filter N Network

Claims

1. An information processing method executed by a computer, comprising: acquiring information indicating a first reflected wave obtained by transmitting ultrasonic waves into a space; calculating a feature value based on the difference between a first signal based on the first reflected wave and a second signal based on information indicating a second reflected wave obtained before the first reflected wave; and determining whether or not a person is present in the space based on the calculated feature value.

2. The information processing method of claim 1, wherein the first signal includes the first reflected wave, the second signal includes the second reflected wave, and the feature is a value based on a first standard deviation of the amplitude of a differential signal between the first reflected wave and the second reflected wave.

3. The information processing method of claim 1, further comprising the steps of: acquiring a first direct wave of the ultrasonic wave; the first signal including a first convolved signal obtained by convolving the first reflected wave with the first direct wave; and the second signal including a second convolved signal obtained by convolving the second reflected wave with a second direct wave acquired before the first reflected wave; and the feature amount being a value based on a first standard deviation of the amplitude of a difference signal between the first convolved signal and the second convolved signal.

4. The information processing method of claim 1, further comprising: acquiring a direct wave of the ultrasonic wave; generating a direct wave signal in which the amplitude of the first reflected wave in a received signal including the direct wave and the first reflected wave is replaced with zero; and generating a reflected wave signal in which the amplitude of the direct wave in the received signal is replaced with zero; estimating a first impulse response of an adaptive filter based on the direct wave signal and the reflected wave signal; the first signal includes the first impulse response; and the second signal includes a second impulse response obtained earlier than the first impulse response.

5. The information processing method according to claim 1, wherein the first signal includes a signal indicating a change over time in the amplitude of the first reflected wave, and the second signal includes a signal indicating a change over time in the amplitude of the second reflected wave obtained earlier than the first reflected wave.

6. An information processing method as described in claim 2 or 3, wherein the first signal includes a first envelope signal indicating a change over time in the amplitude of the first reflected wave, the second signal includes a second envelope signal indicating a change over time in the amplitude of the second reflected wave acquired earlier than the first reflected wave, and the feature is a value calculated by calculating the maximum value of the amplitude of a difference signal between the first envelope signal and the second envelope signal and a value based on the first standard deviation.

7. The information processing method according to any one of claims 1 to 5, wherein the second reflected wave is an average waveform of reflected waves received at multiple past points in time.

8. The information processing method according to claim 2 or 3, further comprising obtaining one or more second standard deviations calculated in the past from the first standard deviation, and the feature amount being a value obtained by smoothing the first standard deviation and the one or more second standard deviations.

9. The information processing method according to claim 8, wherein the smoothing process includes a moving average process.

10. The information processing method according to claim 8, wherein the smoothing process includes a peak hold process that stores a maximum value of the one or more second standard deviations obtained in a predetermined period, and the feature is a value based on the maximum value stored by the peak hold process and the first standard deviation.

11. The information processing method according to any one of claims 1 to 5, wherein the ultrasonic waves are emitted periodically.

12. The information processing method according to any one of claims 1 to 5, wherein the ultrasonic waves include burst waves.

13. The information processing method according to claim 12, wherein the burst wave includes a plurality of frequency components.

14. The information processing method according to any one of claims 1 to 5, wherein the ultrasonic wave includes a chirp signal.

15. An information processing method according to any one of claims 1 to 5, further comprising determining that a person is present in the space if the feature amount is equal to or greater than a predetermined value, and determining that no person is present in the space if the feature amount is less than the predetermined value.

16. The information processing method according to any one of claims 1 to 5, further comprising inputting the feature amount into a machine learning model to obtain an inference result as to whether or not a person is present in the space.

17. The information processing method according to claim 16, further comprising acquiring a plurality of features and a plurality of judgment results as to whether or not a person is present, using the acquired plurality of features as input information, and using the acquired plurality of judgment results as correct answer data to train the machine learning model.

18. An information processing device comprising: an acquisition unit that acquires information indicating a first reflected wave of ultrasound obtained by transmitting ultrasound into a space; a calculation unit that calculates a feature value based on the difference between a first signal based on the first reflected wave and a second signal based on information indicating a second reflected wave acquired earlier than the first reflected wave; and a determination unit that determines whether or not a person is present in the space based on the calculated feature value.

19. A program for causing a computer to execute the information processing method according to any one of claims 1 to 5.

Images