Information processing apparatus, information processing method, and computer program product

Abstract

The invention provides an information processing apparatus, an information processing method, and a computer program product, which can efficiently acquire sound data of an object. Provided is an information processing device provided with: a data acquisition unit that acquires measurement data generated on the basis of a reflected wave from an object by irradiating the object with an electromagnetic wave; a conversion condition determination unit that determines whether or not a conversion condition for converting at least a portion of the measurement data into sound data is satisfied; and a data conversion unit that converts at least a portion of the measurement data into the sound data when the conversion condition is satisfied. The conversion condition determination unit may determine whether or not the conversion condition is satisfied on the basis of the measurement data.

Description

Technical Field

[0001] The present invention relates to an information processing device, an information processing method and a computer program product. Background Art

[0002] Conventionally, there is a known device for measuring the sound of an object (for example, see Patent Document 1). Also, there is a known method for sensing an object using an FMCW (Frequency Modulated Continuous Wave) radar (for example, see Patent Document 2).

[0003] Patent document 1: China Patent Application Publication No. 102961164

[0004] Patent Document 2: Japanese Patent Application Publication No. 2022-192005 Summary of the invention

[0005] In a first aspect of the present invention, an information processing device is provided. The information processing device may include a data acquisition unit that acquires measurement data generated by irradiating an object with electromagnetic waves and based on reflected waves from the object. Any of the information processing devices may include a conversion condition determination unit that determines whether a conversion condition for converting at least a part of the measurement data into sound data is satisfied. Any of the information processing devices may include a data conversion unit that converts at least a part of the measurement data into the sound data when the conversion condition is satisfied.

[0006] In any of the above-mentioned information processing devices, the conversion condition determination unit can output a control signal, and the control signal controls the electromagnetic wave and the information processing device in the following manner: before and after the conversion condition is met, at least one of the power consumption used to irradiate the electromagnetic wave and the power consumption of the information processing device remains unchanged.

[0007] In any of the above-mentioned information processing devices, the conversion condition determination unit can output a control signal, and the control signal controls at least one of the electromagnetic wave and the information processing device in the following manner: when the conversion condition is met, at least one of the power consumption used to irradiate the electromagnetic wave and the power consumption of the information processing device is increased compared with before the conversion condition is met.

[0008] In any of the above-mentioned information processing devices, the conversion condition determination unit can output a control signal, and the control signal controls the electromagnetic wave in the following manner: when the conversion condition is satisfied, the irradiation amount of the electromagnetic wave irradiated toward the object is increased compared with before the conversion condition is satisfied.

[0009] In any of the above-mentioned information processing devices, the conversion condition determination unit may determine whether the conversion condition is satisfied based on the measurement data.

[0010] In any of the above-mentioned information processing devices, the conversion condition determination unit may determine whether the conversion condition is satisfied based on an amplitude of the reflected wave included in the measurement data.

[0011] Any of the above-mentioned information processing devices may include a data processing unit that generates distance information based on the reflected wave included in the measurement data, the distance information indicating the distance between the information processing device and the object. In any of the above-mentioned information processing devices, the conversion condition determination unit may determine whether the distance information satisfies the conversion condition.

[0012] In any of the above information processing devices, the electromagnetic wave may include a chirp signal whose frequency gradually changes. In any of the above information processing devices, the measurement data may include an intermediate signal having a frequency corresponding to a difference between a frequency of the reflected wave and a frequency of the electromagnetic wave.

[0013] In any of the above information processing devices, the measurement data may include a plurality of the intermediate signals obtained for each transceiver channel, the transceiver channel being a combination of a transmitting antenna for the electromagnetic wave and a receiving antenna for the reflected wave. In any of the above information processing devices, the data processing unit may generate azimuth information based on the measurement data for each of the transceiver channels, the azimuth information indicating the azimuth of the object. In any of the above information processing devices, the conversion condition determination unit may determine whether the reflected waves at each azimuth satisfy the conversion condition based on the azimuth information.

[0014] In any of the above-mentioned information processing devices, the electromagnetic wave may repeatedly include the chirp signal. In any of the above-mentioned information processing devices, the measurement data may include the intermediate signal of each of the chirp signals. In any of the above-mentioned information processing devices, the data processing unit may generate the distance information for each of the chirp signals. In any of the above-mentioned information processing devices, the data processing unit may generate the speed information of the object based on the change of the distance information between the chirp signals. In any of the above-mentioned information processing devices, the conversion condition determination unit may determine whether the measurement data satisfies the conversion condition based on the speed information.

[0015] Any of the above-mentioned information processing devices may include a data processing unit, the data processing unit generating distance information based on the measurement data, generating phase shift information based on the measurement data, and calculating relative orientation determination information with respect to an object based on at least one of the amplitude of the reflected wave, the distance information, and the phase shift information, wherein the distance information indicates the distance between the information processing device and the object. In any of the above-mentioned information processing devices, the conversion condition determination unit may determine whether the relative orientation determination information satisfies a conversion condition for converting at least a part of the measurement data into sound data. In any of the above-mentioned information processing devices, the data conversion unit may convert at least a part of the measurement data into the sound data when the relative orientation determination information satisfies the conversion condition.

[0016] In any of the above-mentioned information processing devices, the conversion condition determination unit may generate notification information when the relative direction determination information does not satisfy the conversion condition, and the notification information notifies that the conversion condition is not satisfied.

[0017] In any of the above-mentioned information processing devices, the conversion condition determination unit may generate instruction information indicating a posture that the object should take in order to satisfy the conversion condition when the relative orientation determination information does not satisfy the conversion condition.

[0018] Any of the above-mentioned information processing devices may include a data processing unit that generates distance information and azimuth information based on the reflected wave included in the measurement data, and calculates the relative position of the object based on the distance information and the azimuth information, wherein the distance information indicates the distance between the information processing device and the object, and the azimuth information indicates the azimuth of the object. In any of the above-mentioned information processing devices, the conversion condition determination unit may generate instruction information that indicates the direction in which the object should move if the measurement data at the relative position does not satisfy the conversion condition, and convert at least a part of the measurement data into the sound data if the measurement data at the relative position satisfies the conversion condition.

[0019] In any of the above-mentioned information processing devices, the conversion condition determination unit may compare a time waveform represented by the sound data converted from the measurement data with a reference waveform to determine whether the sound data is a biological sound.

[0020] In any of the above-mentioned information processing devices, the conversion condition determination unit may determine whether the conversion condition is satisfied based on an image of a space in which the electromagnetic wave is irradiated.

[0021] In any of the above-mentioned information processing devices, the conversion condition determination unit may generate a data map indicating whether the sound data exists for each of the directions.

[0022] In any of the above-mentioned information processing devices, the conversion condition determination unit may generate the data maps for different timings, respectively, and generate a combined data map based on the plurality of data maps, the combined data map indicating whether the audio data exists at at least one timing.

[0023] In any of the above-mentioned information processing devices, the conversion condition determination unit may generate instruction information indicating a position to which the object should move in order to satisfy the conversion condition, when the measurement data does not satisfy the conversion condition.

[0024] In any of the above-mentioned information processing devices, the conversion condition determination unit may generate notification information for notifying that the conversion condition is not satisfied, when the measurement data does not satisfy the conversion condition.

[0025] Any of the above-mentioned information processing devices may include an identification information acquisition unit that acquires identification information for identifying the object. In any of the above-mentioned information processing devices, the data conversion unit may record the audio data and the identification information in association with each other.

[0026] In a second aspect of the present invention, an information processing method is provided. In the above information processing method, measurement data generated by irradiating an object with electromagnetic waves and based on reflected waves from the object can be obtained. In any of the above information processing methods, it can be determined whether a conversion condition for converting at least a part of the measurement data into sound data is satisfied. In any of the above information processing methods, at least a part of the measurement data can be converted into the sound data when the conversion condition is satisfied.

[0027] In a third aspect of the present invention, there is provided a computer program product including a computer program for causing a computer to execute the information processing method according to the second aspect.

[0028] The above summary of the invention does not list all the features of the present invention. In addition, subcombinations of these feature groups may also constitute inventions. BRIEF DESCRIPTION OF THE DRAWINGS

[0029] Figure 1A 1 is a diagram schematically showing a configuration of a system 300 for acquiring sound data of an object 12 .

[0030] Figure 1B It is a diagram for explaining the relative position of the object 12 and the measuring device 100.

[0031] Figure 1C This is a diagram for explaining an example of a representative surface of the object 12 .

[0032] Figure 1D This is a diagram illustrating an example of directions and relative directions.

[0033] Figure 2 1 is a schematic diagram showing an example of a measurement range 400 where the measurement device 100 irradiates the transmission wave.

[0034] Figure 3 It is a block diagram showing a configuration example of the information processing device 10 .

[0035] Figure 4 : is a block diagram showing a configuration example of the data processing unit 30 .

[0036] Figure 5 1 is a diagram showing an example of the transmission wave emitted by the transmission unit 120 .

[0037] Figure 6 1 is a diagram for explaining a transmission signal 220 , a reception signal 230 obtained from a reflected wave, and an intermediate signal 150 .

[0038] Figure 7 It is a diagram for explaining the distance R, the velocity V, and the orientation θ of the object 12 .

[0039] Figure 8 It is a diagram for explaining the distance R, the speed V, the orientation θ1, and the orientation θ2 of the object 12.

[0040] Fig. 9 This is a diagram for explaining the operating principle of the system 300 .

[0041] Fig.10 This is a diagram for explaining the principle of detecting the distance R to the object 12 .

[0042] Fig.11A This is a diagram for explaining the principle of detecting the orientation θ of the object 12 .

[0043] Fig. 11B 1 and 2 are diagrams for explaining a process of detecting the orientation θ of the object 12 by using the orientation FFT.

[0044] Fig.12 This is a diagram for explaining the principle of detecting the velocity V of the object 12 .

[0045] Fig.13 FIG. 1 is a diagram showing an example of a time waveform of an intermediate signal converted into a digital signal.

[0046] Fig.14 1 is a diagram for explaining an example of the operation of the switching condition determination unit 40 .

[0047] Fig.15 It is a diagram for explaining another example of the operation of the transition condition determination unit 40 .

[0048] Fig.16 This is a diagram showing an example of direction information.

[0049] Fig.17 1 is a diagram showing an example of an IQ diagram in a peak period PB of an intermediate signal.

[0050] Fig.18 This is a diagram showing an example of speed information.

[0051] Fig.19A This is a diagram showing an example of audio data converted from measurement data.

[0052] Fig.19B This is a diagram showing an example of audio data converted from measurement data.

[0053] Fig.19C This is a diagram for explaining the influence of the relative orientation of the object 12 and the measuring device 100 on the amplitude of the reflected wave.

[0054] Fig.19D and Fig.14 Likewise, time series data of the absolute value |max| of the maximum amplitude of the intermediate signal is shown.

[0055] Fig.19E 1 is a diagram for explaining the influence of the relative orientation θr between the object 12 and the measuring device 100 on the phase variation amount.

[0056] Fig.19F An example is shown in which the position of the representative surface 13 of the object 12 - 2 changes in the direction of the normal vector b.

[0057] Fig. 20 4 is a diagram for explaining an example of a data map 410 generated by the transition condition determination unit 40 .

[0058] Fig.21 1 is a diagram for explaining an example of the combined data map 420 generated by the conversion condition determination unit 40 .

[0059] Fig. 22 4 is a diagram showing another example of the data map 410 generated by the transition condition determination unit 40 .

[0060] Fig.23A : is a block diagram showing another configuration example of the information processing device 10 .

[0061] Fig. 23B1 is a diagram showing a temporal change in the relative position of the object 12 obtained from the distance information and the direction information.

[0062] Fig.24 This is a flowchart illustrating an example of the operation of the system 300 .

[0063] Fig.25 An example of a computer 1200 is shown that can implement various aspects of the present invention in whole or in part.

[0064] Description of symbols

[0065] 10…information processing device, 12…object, 13…representative surface, 20…data acquisition unit, 30…data processing unit, 32…distance information generation unit, 34…direction information generation unit, 36…speed information generation unit, 38…data cube, 40…conversion condition determination unit, 50…data conversion unit, 60…data output unit, 70…data data display unit, 80…identification information acquisition unit, 100…measuring device, 120…transmitting unit, 121…transmitting antenna, 140…receiving unit, 141…receiving antenna, 150…intermediate signal, 160…transmission and reception control unit, 170…intermediate signal generation unit, 180…filtering unit, 190…AD conversion unit, 220…transmitting unit signal, 230…received signal, 231…reflected wave, 300…system, 400…measurement range, 402…area, 410…data mapping, 420…combined data mapping, 430…display device, 432…indication information, 1200…computer, 1210…main controller, 1212…CPU, 1214…RAM, 1216…graphics controller, 1218…display device, 1220…input / output controller, 1222…communication interface, 1224…storage device, 1226…DVD-ROM drive, 1227…DVD-ROM, 1230…ROM, 1240…input / output chip, 1242…keyboard DETAILED DESCRIPTION

[0066] Hereinafter, the present invention will be described by way of the embodiments of the invention, but the following embodiments do not limit the invention according to the claims. In addition, not all combinations of features described in the embodiments are necessarily essential to the solution means of the invention.

[0067] Figure 1A1 is a diagram showing an overview of the structure of a system 300 for acquiring sound data of an object 12. The object 12 generates a sound of a measurement object. The sound may be generated periodically. At least a portion of the object 12 vibrates with the generation of the sound, and the position in space is displaced. As an example, the object 12 is a living being such as a human or an animal. In addition, as an example, the sound of the measurement object is a biological sound such as heart sounds or lung sounds generated by the activity of the living being. In this case, the skin near the heart or lungs of the object 12 vibrates according to the movement of the heart or lungs. However, the object 12 and the sound of the measurement object are not limited to the above examples. The biological sound of the measurement object may be airway breathing sounds, intestinal peristalsis sounds, or swallowing sounds. In these cases, the skin near the neck or intestines of the object 12 vibrates.

[0068] The system 300 includes a measuring device 100 and an information processing device 10. The measuring device 100 measures the time variation of the distance from the measuring device 100 to at least a portion of an object 12. The portion of the object 12 that is the object of the distance variation measurement is, for example, the surface of the object 12 near the portion of the object 12 that generates sound. The measuring device 100 measures the time variation of the spatial position of the surface. For example, the measuring device 100 measures the vibration of the skin of a person's chest. The measuring device 100 outputs measurement data indicating the measurement result. The frequency component of the displacement of the object 12 measured by the measuring device 100 includes the frequency component of the sound generated by the object 12. The information processing device 10 converts at least a portion of the measurement data of the measuring device 100 into sound data.

[0069] The measuring device 100 of this example irradiates an electromagnetic wave, i.e., a transmission wave, to an object 12, and measures a reflected wave from the object 12. The time from irradiating the transmission wave to receiving the reflected wave changes according to the distance from the measuring device 100 to the object 12. This time is also called TOF (Time of Flight). By measuring the reflected wave, the distance from the measuring device 100 to the object 12 can be measured.

[0070] The measuring device 100 may emit a transmission wave with a gradually changing frequency. In this case, the frequency of the transmission wave emitted when receiving the reflected wave changes according to the distance from the measuring device 100 to the object 12. Therefore, the frequency difference between the emitted transmission wave and the received reflected wave represents the distance from the measuring device 100 to the object 12. By observing the frequency difference and phase difference between the emitted transmission wave and the received reflected wave, the vibration of the object 12 can be measured.

[0071] The measurement data output by the measuring device 100 includes the result of measuring the reflected wave. The measurement result of the reflected wave includes information indicating the time variation of the distance to the object 12. As an example, the measurement data includes data indicating the time variation of the frequency difference between the transmission wave and the reflected wave. The measurement data may be a baseband signal. However, the measuring device 100 is not limited to gradually changing the frequency of the irradiated transmission wave as long as it can measure the distance to the object 12 based on the reflected wave.

[0072] The information processing device 10 converts at least a part of the measurement data into sound data. The sound data includes data for reproducing at least one of the time waveform and the spectrum of the sound of the measurement object. The sound data may also be at least one of the time waveform and the spectrum of the sound of the measurement object. The information processing device 10 can calculate the position change of the object 12 based on the measurement data. The information processing device 10 can extract the components of the frequency band that should be converted into sound data from the frequency components of the position change of the object 12 and convert them into sound data. The frequency band can be determined according to the type of sound that is the object. For example, in the case of measuring heart sounds, the information processing device 10 can set a frequency band including 100 Hz. In the case of measuring lung sounds, the information processing device 10 can set a frequency band including 100 Hz to 800 Hz.

[0073] The information processing device 10 can replay the sound based on the generated sound data, can display at least a part of the time waveform or spectrum of the sound, and can also send the sound data to the outside. For example, the information processing device 10 can send the sound data to a doctor's terminal at a distance. The doctor's terminal replays the sound based on the sound data. In this way, it is possible to support diagnosis and treatment at a distance.

[0074] The information processing device 10 of this example determines whether a conversion condition for converting at least part of the measurement data into sound data is satisfied. If the conversion condition is satisfied, the information processing device 10 converts at least part of the measurement data into sound data representing the sound of the measurement object, such as biological sound.

[0075] The conversion condition is, for example, a condition for determining whether appropriate sound data can be obtained. Depending on the relative position and relative orientation of the object 12 and the measuring device 100, the amplitude of the reflected wave is sometimes too small to obtain appropriate sound data. In addition, when only the position displacement irrelevant to the generation of the sound of the measuring object is measured, the position displacement represented by the reflected wave does not correspond to the sound of the measuring object, and appropriate sound data cannot be obtained. The relative position and relative orientation of the object 12 and the measuring device 100 will be described later.

[0076] The conversion condition may be a condition that specifies the amplitude of the reflected wave or another condition. The information processing device 10 generates audio data corresponding to the audio of the measurement target only when the conversion condition is satisfied, thereby suppressing generation of unnecessary audio data.

[0077] The generation of audio data when the conversion condition is satisfied refers to the generation of audio data provided to the user. The information processing device 10 may also determine whether the temporarily generated audio data satisfies the conversion condition and provide only the audio data that satisfies the conversion condition to the user. The audio data that does not satisfy the conversion condition may be discarded by the information processing device 10 or may be accumulated by the information processing device 10. The audio data generated when the conversion condition is satisfied may be recorded in the information processing device 10 as audio data that can be provided to the user.

[0078] The information processing device 10 can perform conversion processing from the measurement data to the sound data when the conversion conditions are met. The conversion processing from the measurement data to the sound data may include a process of generating information corresponding to the distance from the object 12 based on the measurement data. The conversion processing may also include a process of extracting the time variation of the phase of the signal contained in the measurement data. The conversion processing may also include a filtering process of the signal in the measurement data corresponding to the frequency band of the sound of the measurement object. The conversion processing may also include a Fourier transform of the signal contained in the measurement data. The conversion processing may also include at least a part of the determination processing 1 to 10 described later. The details of the conversion processing will be described in detail in. Figure 3 The conversion process from the measurement data to the sound data includes Figure 3 At least a portion of the processing described later.

[0079] By generating sound data only when the conversion condition is satisfied by the information processing device 10, the sound data can be efficiently provided to users such as doctors. For example, the trouble of users selecting appropriate sound data from a large amount of sound data can be reduced. In addition, it is also possible to continuously monitor whether the conversion condition is satisfied and automatically obtain sound data when the conversion condition is satisfied, thereby eliminating the need for medical staff to intervene in the acquisition of sound data. Medical staff can start diagnosis and treatment from the beginning of obtaining sound data, and can also perform diagnosis and treatment efficiently by analyzing the automatically obtained sound data afterwards.

[0080] The information processing device 10 can also analyze the automatically acquired sound data to determine whether or not an abnormality has occurred in heart sounds or the like. According to the information processing device 10, appropriate sound data of the object 12 can also be automatically acquired in the daily life of the object 12. For example, the measuring device 100 can be set in a device such as a mirror, a computer, a television, a chair, a car, etc., in which the object 12 is stationary at a predetermined relative position. The information processing device 10 can notify an external institution such as a medical institution when the abnormality is detected. The measuring device 100 can also be set outside the home of the object 12. For example, the measuring device 100 can also be set in a pharmacy or a retail store. In addition, the sound data of the object 12 can also be acquired in a space isolated from medical staff in the hospital. In this way, it is possible to suppress the infection of diseases from the object 12 to medical staff.

[0081] When the object 12 is an animal other than a human, it is difficult to still the object 12. Therefore, it is generally difficult to obtain sound data of biological sounds of animals. For example, in an animal hospital, the object 12 becomes excited and it is difficult to still the object 12. According to the information processing device 10, since the sound data of the object 12 can be obtained at home, etc., it is relatively easy to obtain appropriate sound data. For example, by obtaining sound data while the object 12 is sleeping, it is easy to obtain appropriate sound data.

[0082] The measuring device 100 can irradiate the transmission wave to the predetermined measuring range and measure the reflected waves from the multiple parts within the measuring range. Thus, the information processing device 10 can obtain the sound data of the multiple parts of the object 12. The information processing device 10 can generate information indicating whether the appropriate sound data is obtained for each of the multiple parts. Thus, diagnosis and treatment using sound data becomes easy.

[0083] When the switching condition is not satisfied, the information processing device 10 may suppress the power consumption of at least one of the measurement device 100 and the information processing device 10. This also reduces the power consumption.

[0084] The measuring device 100 of this example includes a transmitting unit 120 and a receiving unit 140. The transmitting unit 120 of this example irradiates the transmission wave to the object 12. The transmission wave of this example is a frequency modulated continuous wave (FMCW), but is not limited to this. The transmission wave of this example repeatedly includes chirps whose frequency is swept over time. The transmitting unit 120 may have a plurality of transmission antennas that irradiate the transmission wave respectively.

[0085] The receiving unit 140 receives a reflected wave after the transmission wave is reflected at the object 12. The receiving unit 140 can generate an electrical signal corresponding to the time waveform of the reflected wave. The receiving unit 140 can have a plurality of receiving antennas. By providing a plurality of combinations of the transmission antenna and the receiving antenna, it is possible to obtain information related to the azimuth θ of the object 12 observed from the measuring device 100. For example, when a plurality of receiving antennas are provided, the phase of the reflected wave reaching each receiving antenna changes according to the azimuth θ of the object 12 and the position of the receiving antenna. Since the position of each receiving antenna is known, the azimuth θ of the object 12 can be detected based on the phase of the reflected wave received by each receiving antenna.

[0086] The transmission and reception control unit 160 controls transmission and reception of signals by the transmission unit 120 and the reception unit 140. In one example, the transmission and reception control unit 160 controls at least one of the modulation width, cycle, and number of repetitions of the frequency of the chirp included in the transmission wave.

[0087] The measuring device 100 of this example further includes an intermediate signal generating unit 170, a filtering unit 180, and an AD conversion unit 190. The intermediate signal generating unit 170 generates an intermediate signal having a frequency corresponding to the difference between the frequency of the reflected wave and the frequency of the transmitted wave. The intermediate signal has an amplitude corresponding to the amplitude of the reflected wave. The maximum amplitude of the intermediate signal can be the same as the maximum amplitude of the reflected wave or can be proportional to it. The intermediate signal generating unit 170 can generate an intermediate signal for each combination of a transmitting antenna and a receiving antenna. For example, when there are A transmitting antennas and B receiving antennas, the intermediate signal generating unit 170 can generate A×B intermediate signals. In this case, there can also be B intermediate signal generating units 170.

[0088] The intermediate signal generating unit 170 of this example is a mixer that multiplies a transmission signal corresponding to a transmission wave transmitted by a corresponding transmission antenna and a reception signal corresponding to a reflected wave received by a corresponding reception antenna. By multiplying the two signals, an intermediate signal having a frequency that is a difference between the frequency of the reflected wave and the frequency of the transmission wave and a sum frequency signal having a frequency that is a sum of the frequency of the reflected wave and the frequency of the transmission wave can be generated. The farther the distance between the measuring device 100 and the object 12 is, the greater the frequency of the intermediate signal.

[0089] The filter unit 180 limits the frequency band of the signal output by the intermediate signal generating unit 170. The filter unit 180 of this example allows the intermediate signal in the signal output by the intermediate signal generating unit 170 to pass through and removes the sum frequency signal. In this way, the intermediate signal can be extracted. In addition, the filter unit 180 can also use a high-pass filter to remove short-range signals with relatively strong signal strength. In this way, saturation of the internal signal can be prevented, and the measurable distance range can be increased. As described above, the intermediate signal only needs to have a frequency corresponding to the difference between the frequency of the reflected wave and the frequency of the transmitted wave. In this way, the distance to the object 12 can be detected based on the intermediate signal. In this specification, a signal based on the intermediate signal obtained by applying at least one of the addition of arbitrary frequency characteristics and gain adjustment to the intermediate signal output by the intermediate signal generating unit 170 by the filter unit 180 or the like is also referred to as an intermediate signal.

[0090] The AD converter 190 converts the intermediate signal outputted from the filter 180 into a digital signal. The measuring device 100 outputs measurement data including the digital signal converted by the AD converter 190. The information processing device 10 of this example generates audio data based on at least a part of the frequency components of the intermediate signal included in the measurement data.

[0091] Figure 1B 1 is a diagram for explaining the relative position of the object 12 and the measuring device 100. In this specification, the relative position of the object 12 and the measuring device 100 means the relative positional relationship between the object 12 and the measuring device 100. For example, Figure 1B As shown in FIG. 1 , when an arbitrary orthogonal coordinate system (x-axis, y-axis, z-axis) is defined with the position of the measuring device 100 as the origin, the relative position is represented by the distance r of the object 12 from the origin and the angle θ and angle φ with two coordinate axes (e.g., the z-axis and the x-axis). In addition, in this specification, the above relative position is sometimes referred to as simply as position. In this specification, the relative orientation of the object 12 and the measuring device 100 is defined as the angle formed by the normal vector of an arbitrary representative surface defined for the object 12 and the direction vector from the measuring device 100 to the object 12.

[0092] Figure 1C 1 is a diagram for explaining an example of a representative plane of an object 12. The object 12 in this example is a person. A plane perpendicular to the ground and dividing the person into two parts, left and right, is set as a sagittal plane, a plane perpendicular to the ground and dividing the person into two parts, front and back, is set as a coronal plane, and a plane parallel to the ground and dividing the person into two parts, top and bottom, is set as a horizontal plane. In addition, an axis perpendicular to the ground is set as a vertical axis, an axis perpendicular to the coronal plane is set as a sagittal horizontal axis, and an axis perpendicular to the sagittal plane is set as a coronal horizontal axis.

[0093] When the object 12 is a person, the representative plane can be set to the coronal plane, and its normal vector can be set to the front direction of the sagittal horizontal axis. In addition, as a direction vector from the object 12 to the measuring device 100, a vector can be defined with the starting point being the center of gravity of the object 12 and the end point being the center of gravity of the measuring device 100. In addition, the magnitudes of the normal vector and the direction vector are arbitrary. At this time, the relative orientation changes due to the posture of the person serving as the object 12. The change in posture in this case includes not only the forward bending, backward bending, rotation, and lateral bending of the body, but also the movement of the arms, legs, and neck.

[0094] In this specification, the direction formed by the direction vector from the measuring device 100 to the object 12 is referred to as the orientation of the object 12 observed from the measuring device 100 and is simply referred to as the orientation, and the angle formed by the normal vector of any representative surface defined for the object 12 and the direction vector from the measuring device 100 to the object 12 is referred to as the relative orientation between the object 12 and the measuring device 100 and is simply referred to as the relative orientation.

[0095] Figure 1D is a diagram illustrating an example of directions and relative directions. Figure 1D In FIG. 1 , the representative surface 13 of the measuring device 100 and the object 12 are shown in the xyz coordinate system. Figure 1D The direction vector a from the measuring device 100 toward the object 12 and the normal vector b representing the surface 13 are shown. Figure 1D In the figure, an arrow is attached to the upper part of each vector symbol, but the arrow is omitted in the description.

[0096] For example, in Figure 1D In FIG. 1 , the orientation of the object 12 observed from the measuring device 100 is defined by the angle θ between the direction vector a and the z-axis and the angle φ between the straight line obtained by projecting the direction vector a onto the xy plane and the x-axis. The relative orientation of the object 12 and the measuring device 100 is defined by the angle θr between the normal vector b of the representative surface 13 of the object 12 and the direction vector a from the measuring device 100 to the object 12.

[0097] Figure 2 1 is a schematic diagram showing an example of a measurement range 400 where a transmission wave is irradiated by the measurement device 100. The object 12 in this example is a person. Figure 2 The measuring device 100 of the example is irradiating the transmission wave toward the vicinity of the chest of the object 12.

[0098] The measuring device 100 may continuously irradiate the transmission wave toward the predetermined measuring range 400 regardless of the presence or absence of the object 12, and measure the reflected wave from the direction of the measuring range 400. The transmission wave irradiated by the measuring device 100 may have a wavelength that can pass through clothes and be reflected by the skin of the living being. The transmission wave may be a so-called millimeter wave. The wavelength of the transmission wave may be greater than 1 mm and less than 10 mm.

[0099] When the measurement range 400 includes a part such as the chest that is displaced by the biological sound of the object 12, it is possible to generate sound data of the biological sound based on the reflected wave from the measurement range 400. On the other hand, when the measurement range 400 does not include the part, it is impossible to generate appropriate sound data based on the reflected wave from the measurement range 400. The system 300 of this example generates and records sound data based on the reflected wave when the conversion condition is satisfied, so that generation of inappropriate sound data can be suppressed.

[0100] The measurement range 400 may be divided into a plurality of regions 402. The displacement of each region 402 affects the phase of one or more reflected waves. The information processing device 10 can measure the displacement of the object 12 for each region 402 by analyzing the reflected waves received by the plurality of receiving antennas.

[0101] The information processing device 10 may determine whether the reflected wave satisfies the conversion condition for each area 402. In this case, the information processing device 10 generates sound data based on the reflected wave that satisfies the conversion condition. The information processing device 10 may record the generated sound data in association with the area 402.

[0102] The information processing device 10 may also determine whether the reflected wave received by one or more receiving antennas satisfies the conversion condition. For example, it may be determined whether the reflected wave received by the receiving antenna arranged in the center among the multiple receiving antennas arranged in a predetermined direction satisfies the conversion condition. In this case, the information processing device 10 may generate sound data for all regions 402 when the reflected wave received by one receiving antenna satisfies the conversion condition. The information processing device 10 may record the generated sound data in correspondence with each region 402.

[0103] In this specification, the plane parallel to the measurement range 400 is referred to as the XZ plane. In addition, the axis parallel to the direction from the measurement device 100 toward the object 12 is referred to as the Y axis. The X axis, the Y axis, and the Z axis are mutually orthogonal axes. As an example, the Z axis is parallel to the gravity direction.

[0104] Figure 31 is a block diagram showing a configuration example of the information processing device 10. The information processing device 10 includes a data acquisition unit 20, a conversion condition determination unit 40, and a data conversion unit 50. The information processing device 10 of this example also includes a data processing unit 30, a data output unit 60, and a data display unit 70. The information processing device 10 may be provided in the same housing as the measuring device 100, or may be provided in a housing different from the measuring device 100. The information processing device 10 may be a computer that processes digital signals. It may be a single computer as Figure 3 The components shown in the figure can function by multiple computers working together. Figure 3 Each of the components shown functions.

[0105] The data acquisition unit 20 acquires measurement data generated based on the reflected wave from the object 12 when the transmission wave is irradiated to the object 12. The measurement data may include data of a plurality of intermediate signals corresponding to a plurality of reflected waves obtained for each transmission and reception channel. The transmission and reception channel refers to a combination of a transmission antenna and a reception antenna. The data acquisition unit 20 of this example acquires the measurement data from the measurement device 100. The data acquisition unit 20 may include a communication device for communicating with the measurement device 100.

[0106] The data processing unit 30 performs predetermined data processing on the measurement data. For example, the data processing unit 30 calculates the change (i.e., displacement) of the position of the object 12 in each area 402 based on the measurement data. The data processing unit 30 may perform at least one of gain adjustment, fast Fourier transform (FFT), and filtering on each intermediate signal included in the measurement data. Details of the data processing in the data processing unit 30 will be described later.

[0107] The conversion condition determination unit 40 determines whether the conversion condition for converting at least a part of the measurement data into the sound data is satisfied. The conversion condition determination unit 40 of this example determines the conversion condition based on at least one of the reflected wave included in the measurement data, the intermediate signal included in the measurement data, the data output by the data processing unit 30, the data after at least a part of the processing is performed in the data processing unit 30, the data after at least a part of the processing is performed in the data conversion unit 50, and the sound data generated by the data conversion unit 50. The details of the determination process in the conversion condition determination unit 40 will be described later. The conversion condition determination unit 40 may also determine whether the conversion condition is satisfied based on information other than the measurement data such as the image of the object 12.

[0108] The conversion condition determination unit 40 of this example outputs a control signal that controls at least one of the transmission wave and the information processing device 10 in such a manner that, when the conversion condition is satisfied, at least one of the power consumption for irradiating the transmission wave and the power consumption of the information processing device 10 is increased compared to before the conversion condition is satisfied. The conversion condition determination unit 40 may make the frequency of irradiating the transmission wave after the conversion condition is satisfied greater than the frequency of irradiating the transmission wave before the conversion condition is satisfied. The conversion condition determination unit 40 may make the transmission power of the transmission wave greater than the transmission power before the conversion condition is satisfied after the conversion condition is satisfied. The conversion condition determination unit 40 may stop at least a part of data processing in at least one of the data processing unit 30 and the data conversion unit 50 before the conversion condition is satisfied. The conversion condition determination unit 40 may also stop the data processing unit 30 and the data conversion unit 50 before the conversion condition is satisfied, and start the data processing unit 30 and the data conversion unit 50 when the conversion condition is satisfied. The conversion condition determination unit 40 may also output a control signal that controls at least one of the transmission wave and the information processing device 10 in the following manner: when the conversion condition is satisfied and a certain time has passed, at least one of the power consumption for irradiating the transmission wave and the power consumption of the information processing device 10 is reduced. For example, when the conversion condition is satisfied, the conversion condition determination unit 40 increases at least one of the power consumption for irradiating the transmission wave and the power consumption of the information processing device 10 from an initial value. When the conversion condition is satisfied and a certain time has passed, the conversion condition determination unit 40 may restore the power consumption for irradiating the transmission wave and the power consumption of the information processing device 10 to an initial value. The certain time may be within 10 seconds, within 1 minute, or longer. The certain time may be a time required for a doctor's consultation. The certain time may be pre-set by a user such as a doctor.

[0109] In another example, the conversion condition determination unit 40 may output a control signal that controls the transmission wave and the information processing device in such a manner that at least one of the power consumption for irradiating the transmission wave and the power consumption of the information processing device remains unchanged before and after the conversion condition is satisfied. The conversion condition determination unit 40 may make the frequency of irradiating the transmission wave the same before and after the conversion condition is satisfied. The conversion condition determination unit 40 may make the transmission power of the transmission wave the same before and after the conversion condition is satisfied. The conversion condition determination unit 40 may not stop a part of the data processing in the data processing unit 30 and the data conversion unit 50 before the conversion condition is satisfied. The conversion condition determination unit 40 may not reduce the power supply to the data processing unit 30 and the data conversion unit 50 before the conversion condition is satisfied.

[0110] In other examples, the conversion condition determination unit 40 may output a control signal that controls the electromagnetic wave in the following manner: when the conversion condition is satisfied, the irradiation amount per unit time of the electromagnetic wave irradiated toward the object 12 increases compared to before the conversion condition is satisfied. The irradiation amount may be a time-integrated value of the intensity of the electromagnetic wave toward the object 12. In this example, when the conversion condition is not satisfied, the irradiation amount of the electromagnetic wave to the object 12 decreases. Thus, the influence of the electromagnetic wave on the object 12 can be reduced.

[0111] As described above, the conversion condition determination unit 40 can control the irradiation amount of the electromagnetic wave irradiated toward the object 12 by controlling at least one of the irradiation frequency of the transmission wave and the transmission power per irradiation. In this case, the power consumption for irradiating the transmission wave can be reduced while reducing the influence of the electromagnetic wave on the object 12.

[0112] The conversion condition determination unit 40 may also increase or decrease the irradiation amount of electromagnetic waves irradiated toward the object 12 before and after the conversion condition is satisfied, while maintaining at least one of the power consumption for irradiating the transmission wave and the power consumption of the information processing device. For example, the multiple transmission antennas of the measuring device 100 may include a dummy transmission antenna. The dummy transmission antenna irradiates electromagnetic waves in a direction not toward the object 12. The conversion condition determination unit 40 may increase or decrease the power consumption of the dummy transmission antenna before and after the conversion condition is satisfied, while maintaining the total power consumption of the multiple transmission antennas. More specifically, when the conversion condition is satisfied, the power consumption in the dummy transmission antenna is reduced, and the power consumption of the transmission antenna other than the dummy is increased, compared with the case where the conversion condition is not satisfied. The power consumption in the dummy transmission antenna when the conversion condition is satisfied may also be zero. According to this example, since the power consumption can be maintained, the noise generated by the change in power consumption can be reduced while reducing the influence of the electromagnetic wave on the object 12.

[0113] In the above example, the conversion condition determination unit 40 maintains or reduces the power consumption when the conversion condition is not satisfied compared to the case where the conversion condition is satisfied, in the control of increasing or decreasing the amount of electromagnetic wave irradiation to the object 12. In other examples, the conversion condition determination unit 40 may increase the power consumption when the conversion condition is not satisfied compared to the case where the conversion condition is satisfied, in the control of increasing or decreasing the amount of electromagnetic wave irradiation to the object 12. Through these controls, the amount of electromagnetic wave irradiation to the object 12 can be reduced.

[0114] The data conversion unit 50 converts at least a part of the measurement data into sound data when the conversion condition determination unit 40 determines that the predetermined conversion condition is satisfied. The intermediate signal in this example includes information indicating the position of the object 12. The data conversion unit 50 may convert the signal indicating the temporal variation of the position of the object 12 calculated from the intermediate signal into sound data. The data conversion unit 50 may convert a component of a predetermined frequency band in the signal indicating the temporal variation of the position of the object 12 into sound data.

[0115] When the measurement data includes a plurality of intermediate signals, if the conversion condition determination unit 40 determines that only a portion of the intermediate signals satisfy the conversion condition, the data conversion unit 50 may convert the intermediate signals satisfying the conversion condition into audio data, and not convert the other intermediate signals into audio data. In another example, if it is determined that a portion of the intermediate signals satisfy the conversion condition, the data conversion unit 50 may convert all the intermediate signals included in the measurement data into audio data.

[0116] The data output unit 60 outputs the sound data converted by the data conversion unit 50. The data output unit 60 can output the sound data to the data display unit 70. The data display unit 70 can include a display device. The data display unit 70 displays information related to the sound data. The data display unit 70 can display the time waveform of the sound data, can display the frequency spectrum, and can also display information indicating whether there is corresponding sound data for each area 402.

[0117] The data output unit 60 may also play back the sound based on the sound data recorded by the data conversion unit 50. The data output unit 60 may also transmit the sound data recorded by the data conversion unit 50 to an external device. For example, the data output unit 60 transmits the sound data to a terminal of a doctor who is far away from the object 12. The data output unit 60 may also transmit information indicating whether there is corresponding sound data for each area 402.

[0118] Figure 4 3 is a block diagram showing a configuration example of the data processing unit 30. The data processing unit 30 of this example includes a distance information generating unit 32, a direction information generating unit 34, and a speed information generating unit 36. It should be noted that the connection order of the direction information generating unit 34 and the speed information generating unit 36 ​​may also be the same as Figure 4 The connection is in reverse order.

[0119] The distance information generating unit 32 generates distance information indicating the distance to the object 12 based on the reflected wave included in the measurement data. The distance information generating unit 32 of this example generates distance information based on the intermediate signal generated from the reflected wave. In this specification, the intermediate signal is regarded as the signal of the reflected wave. The distance information generating unit 32 can generate distance information based on the frequency of the intermediate signal. The distance information generating unit 32 generates distance information for each intermediate signal.

[0120] The conversion condition determination unit 40 can determine whether the distance information satisfies the conversion condition. For example, the conversion condition determination unit 40 can determine whether the distance from the object 12 represented by the distance condition is within a prescribed range suitable for measuring sound. The range can be preset in the conversion condition determination unit 40.

[0121] The conversion condition determination unit 40 can determine whether the conversion condition is satisfied based on the distance information corresponding to any one or more intermediate signals. For example, it can be determined whether the distance information corresponding to the receiving antenna arranged in the center among the multiple receiving antennas satisfies the conversion condition. When the conversion condition determination unit 40 determines that the distance information does not satisfy the conversion condition, the information processing device 10 may not generate sound data for all the data included in the measurement data. When the conversion condition determination unit 40 determines that the distance information does not satisfy the conversion condition, the direction information generation unit 34 and the speed information generation unit 36 ​​may stop processing the measurement data. When the conversion condition determination unit 40 determines that the distance information satisfies the conversion condition, the direction information generation unit 34 may execute processing of the measurement data.

[0122] The orientation information generating unit 34 generates orientation information based on a plurality of distance information corresponding to a plurality of intermediate signals. The orientation information is information indicating the orientation of the object 12 relative to the measuring device 100. The orientation information may also be information indicating the orientation of the part of the object 12 that vibrates in response to sound. As described above, the reflected waves received by each receiving antenna produce a phase difference depending on the orientation of the object 12. The distance information in this example includes information on the phase of each intermediate signal. The orientation information generating unit 34 can calculate the orientation of the object 12 based on a plurality of distance information. The conversion condition determining unit 40 can extract the intermediate signal to be converted into sound data based on the orientation information.

[0123] The velocity information generating unit 36 ​​generates velocity information based on the time variation of the distance information corresponding to the azimuth of the reflected wave received from the object 12. Since the distance information includes information indicating the distance to the object 12, the velocity of the displacement of the object 12 can be calculated from the time variation of the distance to the object 12 indicated by the distance information.

[0124] The conversion condition determination unit 40 can extract the intermediate signal to be converted into the sound data based on the velocity information. The velocity information includes not only information on the displacement of the object 12 associated with the sound data of the measurement object, but also information on the displacement of the object 12 not associated with the sound data of the measurement object.

[0125] The conversion condition determination unit 40 may determine whether the information related to the displacement of the object 12 associated with the sound of the measurement object in the velocity information satisfies the conversion condition. The conversion condition determination unit 40 may determine whether the component of the frequency band corresponding to the sound of the measurement object in the velocity information satisfies the conversion condition. For example, in the case of measuring heart sounds, the frequency band may be 30 Hz or more and 100 Hz or less. The velocity information generation unit 36 ​​may output the intermediate signal determined as the velocity information satisfying the conversion condition to the data conversion unit 50.

[0126] Figure 5 1 is a diagram showing an example of a transmission wave emitted by the transmission unit 120. In this example, the measuring device 100 is a FMCW radar device, but the measuring device 100 is not limited to this. Figure 5 In FIG. 1 , a transmission wave emitted by one transmission antenna is shown. The transmission unit 120 of this example can repeatedly emit a transmission wave in a mode called a burst. Figure 5 , 1 pulse train is shown.

[0127] One pulse train of the transmission wave contains m chirp signals. m is an integer greater than 1. One chirp signal is an electromagnetic wave whose frequency is swept according to time. Figure 5 In the example of , each chirp signal includes a portion in which the frequency increases linearly with time. The manner in which the frequency changes in each chirp signal may be the same or different. By modulating the frequency of the chirp signal and analyzing the difference between the frequency of the irradiated transmission wave and the frequency of the received reflected wave, the distance R to the object 12, the speed V of the object 12, and the azimuth θ can be calculated. The vibration corresponding to the sound generated by the object 12 can be extracted from the time variation of the distance R.

[0128] Figure 6 1 is a diagram for explaining a transmission signal 220 , a reception signal 230 obtained from a reflected wave, and an intermediate signal 150 . Figure 6 The transmission signal 220 shows a portion of a chirp signal in which the frequency increases with time.

[0129] The frequency of the transmission signal 220 rises linearly during the period from time T0 to time Tm. The reflected wave is received after a delay time Td has passed since the transmission of the transmission signal 220, and a reception signal 230 is generated. The delay time Td varies according to the distance R from the measuring device 100 to the object 12. During the short period Td to Tm during which the transmission of the transmission signal 220 and the reception of the reception signal 230 are performed simultaneously, if it is assumed that the distance R is constant, the frequency difference f1 between the transmission signal 220 and the reception signal 230 is constant. The frequency difference f1 is determined by the delay time Td, so the frequency difference f1 represents the distance R from the object 12.

[0130] The intermediate signal 150 is a signal having a frequency f1 which is a difference between the frequency of the transmission signal 220 and the frequency of the reception signal 230. The intermediate signal generation unit 170 can generate the intermediate signal 150 based on the transmission signal 220 and the reception signal 230 in the period Td to Tm. The data processing unit 30 of this example performs FFT processing on the time waveform of the intermediate signal 150 to generate a spectrum of the intermediate signal 150. The frequency f1 indicating the peak in the spectrum corresponds to the distance R from the object 12.

[0131] Figure 7 1 and 2 are diagrams for explaining the distance R, the velocity V, and the azimuth θ of the object 12. In this example, for simplicity, the transmission unit 120 and the reception unit 140 of the measurement device 100 are assumed to be at the same position.

[0132] The object 12 is moving at a speed V at a distance R from the measuring device 100. The speed V is the relative speed between the measuring device 100 and the object 12. The azimuth θ is the angle of the object 12 observed from the measuring device 100. As an example, when the direction in which the plurality of receiving antennas of the receiving unit 140 are arranged is set as the x-axis direction and the direction in which the FMCW radar is emitted is set as the y-axis, the azimuth θ is the angle formed by the y-axis and the position of the object 12 in the xy plane.

[0133] Figure 8 1 is a diagram for explaining the distance R, speed V, azimuth θ1 and azimuth θ2 of the object 12. In this example, the position of the object 12 in the xyz space is detected. In this example, a plurality of receiving antennas can be arranged in both the x-axis direction and the z-axis direction. In addition to detecting the azimuth θ1 when the object 12 is projected onto the xy plane, the information processing device 10 of this example also detects the azimuth θ2 when the object 12 is projected onto the yz plane. When simply referred to as azimuth in this specification, both azimuth θ1 and azimuth θ2 may be included.

[0134] Fig. 9 3 is a diagram for explaining the operating principle of the system 300. The system 300 uses the data cube 38 to obtain information on the distance R, the speed V, and the orientation θ of the object 12.

[0135] The measuring device 100 has a plurality of transmission and reception channels. Each transmission and reception channel is a combination of a transmission antenna and a reception antenna. In one example, the measuring device 100 has one transmission antenna 121 and k reception antennas 141. k is an integer greater than 1. The measuring device 100 may also have a plurality of transmission antennas 121. Fig. 9 In FIG. 1 , a plurality of receiving antennas 141 are arranged in a row, but the receiving antennas 141 may be arranged in a row on both the x-axis and the z-axis. The measuring device 100 can detect the azimuth θ by having a plurality of transmission and reception channels. The reflected waves from the object 12 are input to each of the k receiving antennas.

[0136] The measurement data in this example includes multiple intermediate signals corresponding to multiple transceiver channels (in this example, multiple receiving antennas 141). In addition, the measurement data includes the intermediate signal of each chirp signal. Data cube 38 is a three-dimensional data string obtained by arranging the data string of distance information generated based on each intermediate signal for each transceiver channel and each number m of the chirp signal. The intermediate signal of each transceiver channel includes information related to the azimuth θ. Fig. 9 In the data cube 38, the direction in which the intermediate signal of each transceiver channel is arranged is represented by θ. In this specification, the direction in which the transceiver channels are arranged in the data cube 38 is sometimes referred to as the azimuth direction. Each piece of distance information corresponds to a distance R from the object 12. By performing FFT on the data of the predetermined bin number of the distance information in the azimuth direction, the azimuth of the object 12 relative to the measuring device 100 is obtained. By performing FFT on the data of the predetermined bin number of the distance information in the direction of the number m of the chirp signal, the speed of the object 12 is obtained.

[0137] The distance information generating unit 32 generates distance information for each intermediate signal corresponding to each transceiver channel (in this example, the receiving antenna 141). The distance information generating unit 32 of this example generates distance information by performing FFT processing on the digital data of the time waveform of the intermediate signal. The distance information of this example is the power spectrum of the intermediate signal. In this example, the number of intervals of the distance information is set to n / 2. In each distance information, the interval where the apex of the peak is located is set to the peak interval PB. The distance information includes the peak interval PB corresponding to the distance from the object 12. In addition, the distance information generating unit 32 of this example generates distance information for each chirp signal.

[0138] The azimuth information generating unit 34 performs azimuth FFT on the data string of the distance information corresponding to the position of the peak interval PB of the distance information to generate the azimuth information. The azimuth information is the power spectrum of the distance information in the peak interval PB. In this example, the number of intervals of the azimuth information is set to k. The azimuth FFT is to perform FFT processing on the values ​​of each distance information at the position of the peak interval PB in the azimuth direction. The FFT in the azimuth direction is, for example, to arrange the intermediate signals of the receiving antennas arranged in the x-axis or z-axis direction in sequence and perform FFT processing in sequence. The azimuth FFT can be performed on both the x-axis direction and the z-axis direction. The azimuth information generating unit 34 can generate the azimuth information in the x-axis direction based on the distance information corresponding to the receiving antennas arranged in the x-axis direction in the distance information corresponding to each transceiver channel, and generate the azimuth information in the z-axis direction based on the distance information corresponding to the receiving antennas arranged in the z-axis direction.

[0139] The position of the peak interval PB represents the value of the frequency of the intermediate signal. The azimuth information generating unit 34 may use the average value of the positions of the peak interval PB in the plurality of distance information, or may use the position of the peak interval PB in the distance information corresponding to a specific transceiver channel and a specific chirp signal. As an example, the azimuth information generating unit 34 may use the position of the peak interval PB of the distance information corresponding to the receiving antenna arranged in the center among the plurality of receiving antennas arranged on the x-axis and the z-axis.

[0140] The speed information generating unit 36 ​​performs speed FFT on the data string of the distance information at the position of the peak interval PB to generate speed information. The speed information is the power spectrum of the distance information in the peak interval PB. In this example, the number of intervals of the speed information is set to m. The speed FFT is to perform FFT processing on the values ​​of each distance information at the position of the peak interval PB in the time direction (in this example, the direction of the number m of the chirp signal in the data cube 38).

[0141] Fig.10 1 is a diagram for explaining the principle of detecting the distance R from the object 12. The distance R from the object 12 is calculated by performing a distance FFT on an intermediate signal corresponding to at least one chirp signal. Fig.10 , a transmission signal 220 corresponding to one transmission / reception channel and a reception signal 230 obtained from a reflected wave are shown. The solid line of the graph represents the transmission signal 220, and the dotted line represents the reception signal 230. The vertical axis represents frequency, and the horizontal axis represents time.

[0142] The IF frequency is the frequency of the intermediate signal 150 obtained by mixing the transmission signal 220 and the reception signal 230. The IF frequency increases as the distance R between the measurement device 100 and the object 12 increases. The information processing device 10 can obtain the distance R to the object 12 by analyzing the IF frequency.

[0143] The information processing device 10 performs distance FFT processing on each chirp signal to generate distance information. Distance FFT is also called range FFT in the signal processing of millimeter wave radar. For example, when the number of FFT points is n, the data strings of the real part and the imaginary part each obtain n / 2 points, and the number of intervals becomes n / 2. The information processing device 10 calculates the peak interval PB by obtaining the distance power spectrum from the result of the distance FFT. The information processing device 10 can generate distance information for each intermediate signal. The information processing device 10 can calculate the distance R from the object 12 based on the frequency of the peak interval PB.

[0144] Fig.11A This is a diagram for explaining the principle of detecting the direction θ of the object 12. The information processing device 10 of this example detects the direction θ of the object 12 based on K reflected waves 231-1 to 231-K received by K receiving antennas 141-1 to 141-K. The distance between the receiving antenna 141-1 and the receiving antenna 141-x is set to dx (x is an integer from 2 to K).

[0145] When the distance between the receiving antennas 141 is sufficiently small relative to the distance to the object 12, the angle θ at which the reflected wave 231 from the object 12 is incident on each receiving antenna 141 is substantially the same. On the other hand, the phase of the reflected wave 231 received by each receiving antenna 141 differs depending on the position of the receiving antenna 141. For example, the position of the reflected wave 231-K that is in phase with the reflected wave 231-1 received by the receiving antenna 141-1 is spaced apart by dKsinθ in the direction of the azimuth θ relative to the receiving antenna 141-K. In this way, the timing at which the in-phase wavefront of the reflected wave 231 reaches each receiving antenna 141 is determined by the position of the receiving antenna 141 and the azimuth θ.

[0146] The information processing device 10 can calculate the azimuth θ based on the position of each receiving antenna 141 and the phase of the reflected wave 231 received by each receiving antenna 141. The position of each receiving antenna 141 is a fixed value. Information related to the position of each receiving antenna 141 is pre-incorporated into the calculation processing of the information processing device 10. The information processing device 10 of this example calculates the azimuth θ based on the phase of the reflected wave 231 received by each receiving antenna 141.

[0147] The information processing device 10 can calculate the azimuth θ from the phase of the reflected wave 231 using azimuth FFT, Capon method, LP method (linear prediction method), MUSIC method, ESPRIT method, or compressed sensing method. In this example, an example of calculating the azimuth θ using azimuth FFT is described.

[0148] Fig. 11B1 and 2 are diagrams for explaining a process of detecting the orientation θ of the object 12 by using the orientation FFT. Fig. 11B The table shows the real part and imaginary part of the distance information obtained by the distance FFT for each transceiver channel (for example, the combination of the transmitting antenna 121 and the receiving antenna 141) of the measuring device 100. The direction information generating unit 34 extracts the data of the peak interval PB from the distance information of each transceiver channel. Fig. 11B In FIG. 1 , the data of the peak interval PB are surrounded by a dotted line.

[0149] The azimuth information generating unit 34 performs azimuth FFT on the data string of each distance information obtained by performing distance FFT on the chirp signal of each transceiver channel. The azimuth FFT is a process of performing FFT on the value of each distance information in the peak interval PB in the direction of the transceiver channel. In this example, there is one transmitting antenna 121, and the number of transceiver channels is equal to the number of receiving antennas 141. The number of the transceiver channel indicates the position of each receiving antenna 141 on the arrangement axis. That is, when the numbers of the transceiver channels are adjacent, the corresponding receiving antennas 141 are arranged adjacently on the arrangement axis. In this example, the number of receiving antennas 141 and the transceiver channels arranged in the x-axis direction is set to k. The measurement data includes k intermediate signals corresponding to k transceiver channels for calculating the azimuth θ in the x-axis direction for one chirp signal.

[0150] When the information processing device 10 detects the azimuth θ in the x-axis direction, the azimuth FFT processing is performed on k transceiver channels. In the case of k transceiver channels, the data strings of the real part and the imaginary part each obtain k points, and the number of intervals is k. In the reflected wave 231 received by each receiving antenna 141, a phase difference corresponding to the azimuth θ of the object 12 relative to the receiving antenna 141 is generated. By performing azimuth FFT on the value of the distance information in the peak interval PB in the direction in which the receiving antenna 141 is arranged (the direction of the transceiver channel), the frequency of the phase change of the intermediate signal between adjacent transceiver channels can be detected. In the spectrum obtained by the azimuth FFT, a peak appears at the interval position corresponding to the azimuth θ where the object 12 is located. Based on the position of the peak, the azimuth θ of the object 12 can be calculated. In this example, the azimuth θ in the x-axis direction is described, but the azimuth θ in the z-axis direction can also be calculated in the same way.

[0151] Fig.12 This is a diagram for explaining the principle of detecting the velocity V of the object 12 . Fig.12 Table 1 shows the real part and imaginary part of the distance information obtained by the distance FFT for the chirp signals 1 to m in the pulse train. The speed information generating unit 36 ​​extracts the data of the peak interval PB from each distance information. Fig.12In FIG. 1 , the data of the peak interval PB is surrounded by a dotted line. The velocity information generating unit 36 ​​performs FFT (velocity FFT) on the extracted data string in the direction of the chirp signal number. The velocity FFT is also called Doppler FFT in the signal processing of the millimeter wave radar.

[0152] If the time interval between each chirp signal is sufficiently small, the distance from the object 12 measured for each chirp signal does not change substantially. Therefore, each distance information has a peak value in the same peak interval PB. On the other hand, the phase of the intermediate signal changes due to the slight movement of the object 12 within the time interval of the chirp signal. By performing a velocity FFT on the value of the distance information in the peak interval PB in the numbering direction of the chirp signal, the frequency of the phase change of the intermediate signal between consecutive chirp signals can be detected. In the velocity power spectrum obtained by the velocity FFT, a peak value appears at the interval position corresponding to the frequency of the phase change. Based on the position of the peak value, the velocity V of the object 12 can be calculated.

[0153] The minute movement of the object 12 can be detected based on the change in the phase of the intermediate signal represented by the distance information in the time direction. The phase of the intermediate signal can be calculated based on the arc tangent of the ratio of the real part to the imaginary part of the distance information. The chirp number corresponds to the elapsed time. The speed information generating unit 36 ​​can obtain the time waveform data of the minute displacement of the object 12 by arranging the values ​​of the phase of the intermediate signal in the peak interval PB according to the chirp number. The time waveform of the minute displacement of the object 12 corresponds to the time waveform of the sound of the measurement object.

[0154] In this example, for the distance information corresponding to each chirp signal, the peak interval PB is kept constant and the velocity FFT and the time waveform of the micro-displacement are obtained. However, when the object 12 moves in whole or in part, the distance R to the object 12 will change significantly, so the peak interval PB of the distance information corresponding to each chirp signal may change.

[0155] The velocity information generating unit 36 ​​may detect a peak interval PB for each distance information corresponding to each chirp signal. When the distance information whose position does not change in the peak interval PB is detected for a predetermined reference number or more in the numbering direction of the chirp signal, the velocity information generating unit 36 ​​may obtain the time waveform of the micro displacement of the object 12 based on the distance information. When the distance information whose position does not change in the peak interval PB is detected for a predetermined reference number or more in the numbering direction of the chirp signal, the velocity information generating unit 36 ​​may perform velocity FFT on the distance information to generate a velocity power spectrum of the object 12.

[0156] The speed information generating unit 36 ​​may also use the position of the common peak interval PB to perform speed FFT and obtain the time waveform of the micro-displacement of the object 12 even when the position of the individual peak interval PB of each distance information changes. The position of the common peak interval PB may be, for example, the position of the peak interval PB that appears most frequently in each distance information. Between distance information with different positions of individual peak interval PB, the phase sometimes becomes discontinuous. The speed information generating unit 36 ​​may correct the phase of each intermediate signal in such a way that the continuity of the phase becomes higher. The speed information generating unit 36 ​​may perform phase correction based on the position of the individual peak interval PB as disclosed in Patent Document 2.

[0157] Fig.13 FIG. 1 is a diagram showing an example of a time waveform of an intermediate signal converted into a digital signal. Fig.13 The time waveform of the intermediate signal corresponding to one chirp signal is shown. The conversion condition determination unit 40 detects the maximum amplitude of the intermediate signal. The maximum amplitude may be the maximum amplitude on the positive side (max+), the maximum amplitude on the negative side (max-), the one with the larger absolute value between the maximum amplitude on the positive side and the maximum amplitude on the negative side, or the difference between the maximum amplitude on the positive side and the maximum amplitude on the negative side.

[0158] (Determination Process 1)

[0159] Fig.14 FIG. 4 is a diagram for explaining an example of the operation of the conversion condition determination unit 40. Fig.14 The determination process using the amplitude of the intermediate signal described in is referred to as determination process 1. Fig.14 The time series data of the absolute value |max| of the maximum amplitude of the intermediate signal is shown. The conversion condition determination unit 40 may detect the absolute value of the maximum amplitude of the intermediate signal corresponding to the plurality of chirp signals and make it the time series data.

[0160] The conversion condition determination unit 40 can determine whether the conversion condition is satisfied based on the measurement data. The conversion condition determination unit 40 of this example determines whether the conversion condition is satisfied based on the amplitude of the reflected wave represented by the measurement data. The amplitude of the intermediate signal is determined according to the amplitude of the reflected wave. The conversion condition determination unit 40 of this example indirectly determines whether the amplitude of the reflected wave satisfies the prescribed condition by determining whether the amplitude of the intermediate signal satisfies the prescribed condition.

[0161] The conversion condition determination unit 40 of this example determines that the intermediate signal satisfies the conversion condition when the absolute value |max| of the maximum amplitude of the intermediate signal is greater than the predetermined determination threshold TH1. The conversion condition determination unit 40 may also determine that the intermediate signal satisfies the conversion condition when the absolute value |max| of the maximum amplitude of the intermediate signal is continuously greater than the determination threshold TH1 within a predetermined determination period. The predetermined determination period is a period including a plurality of chirp signals. The determination period may also be a period including a chirp signal of one pulse train.

[0162] When the object 12 is too far from the measuring device 100, it is difficult to measure the minute displacement associated with the generation of sound. The amplitude of the reflected wave is attenuated according to the distance between the object 12 and the measuring device 100. Therefore, by determining the conversion condition based on the amplitude of the reflected wave, it is possible to determine whether appropriate sound data can be obtained. The conversion condition determination unit 40 can adjust the determination threshold TH1 based on the amplitude of the transmission wave transmitted by the transmission unit 120. For example, the larger the amplitude of the transmission wave, the larger the determination threshold TH1 is.

[0163] The conversion condition determination unit 40 may adjust the threshold value TH1 according to the attribute of the object 12. The attribute of the object 12 may include at least one of individual identification information such as the name of the individual identifying the object 12, gender identification information for identifying the gender of the object 12, age identification information for identifying the age of the object 12, size information indicating the size such as the height of the object 12, and classification information indicating the classification of the creature of the object 12. The conversion condition determination unit 40 may pre-set the threshold value TH1 for each attribute of the object 12. The conversion condition determination unit 40 may determine the attribute of the object 12 based on the image of the object 12. The measuring device 100 may include a photographing unit for acquiring the image of the object 12. The conversion condition determination unit 40 may also adjust the threshold value of the threshold value TH1 according to the orientation information described later. The conversion condition determination unit 40 may pre-set information indicating the relationship between the orientation information and the threshold value TH1.

[0164] The conversion condition determination unit 40 can stop the data processing in the data processing unit 30 and the data conversion unit 50 until the amplitude of the reflected wave satisfies the conversion condition. The amplitude of the reflected wave can be obtained without being processed in the data processing unit 30. Therefore, by determining the conversion condition based on the amplitude of the reflected wave, the power consumption of the data processing unit 30 and the data conversion unit 50 can be suppressed.

[0165] The conversion condition determination unit 40 may determine that the conversion condition is not satisfied even when the object 12 is too close to the measuring device 100. The conversion condition determination unit 40 may also determine that the intermediate signal satisfies the conversion condition when the absolute value of the maximum amplitude of the intermediate signal is within a predetermined determination range. The conversion condition determination unit 40 may adjust at least one of the upper limit value and the lower limit value of the determination range based on the amplitude of the transmission wave transmitted by the transmission unit 120.

[0166] When an object other than the object 12 exists in the space where the transmission wave is irradiated, the measuring device 100 detects the reflected wave from the object. Therefore, if the conversion condition is determined based on the amplitude of the reflected wave, the object may be mistaken for the object 12 and the conversion condition may be erroneously determined. The conversion condition determination unit 40 may cause at least a part of the data processing in the data processing unit 30 and the data conversion unit 50 to be executed when the amplitude of the reflected wave satisfies the conversion condition, and further determine whether the processing result satisfies the conversion condition. For example, after the intermediate signal is converted into sound data, it may be determined whether the sound data satisfies the prescribed condition.

[0167] The information processing device 10 may determine that the sound data cannot be properly obtained by using the current reflected wave when the sound data does not meet the prescribed conditions. The information processing device 10 may record the amplitude value of the reflected wave when it is determined that the data processing result in the data processing unit 30 and the data conversion unit 50 does not meet the prescribed conditions. The information processing device 10 may stop the data processing in the data processing unit 30 and the data conversion unit 50 until the amplitude value of the newly measured reflected wave changes by more than a prescribed reference value relative to the recorded amplitude value. By such processing, it is possible to suppress the conversion of reflected waves from objects such as chairs that are not the measurement object into sound data.

[0168] (Determination Process 2)

[0169] Fig.15 FIG. 4 is a diagram for explaining another example of the operation of the conversion condition determination unit 40. Fig.15 The determination process using the distance information described in is referred to as determination process 2. Fig.15 The spectrum of distance information corresponding to a predetermined chirp signal and a transmission / reception channel is shown. Figure 4 The distance information generating unit 32 of the data processing unit 30 shown generates distance information.

[0170] The conversion condition determination unit 40 of this example determines whether the distance information satisfies the conversion condition. Fig.14 In the determination process 1 described above, when the amplitude of the reflected wave satisfies a predetermined condition, the determination process 2 is further performed. In another example, the conversion condition determination unit 40 may perform the determination process 2 without performing the determination process 1.

[0171] The conversion condition determination unit 40 may perform determination processing 2 on one or more transmission and reception channels. For example, determination processing 2 may be performed on distance information corresponding to a receiving antenna arranged in the center. It may also be determined that the distance information of all transmission and reception channels satisfies the conversion condition when the distance information satisfies the conversion condition. The conversion condition determination unit 40 may also perform determination processing 2 on a spectrum obtained by overlapping a plurality of distance information corresponding to a plurality of transmission and reception channels. The conversion condition determination unit 40 may perform determination processing 2 on a spectrum generated by correlation integration of the distance information in the azimuth direction.

[0172] The conversion condition determination unit 40 may perform each determination process after removing noise or clutter components included in the distance information. The clutter components are, for example, reflection components from objects other than the target object 12.

[0173] like Figure 5 and Figure 6 As shown, the transmission signal 220 of this example includes a chirp signal whose frequency gradually changes. The measurement data includes an intermediate signal 150 having a frequency corresponding to the difference between the frequency of the received signal 230 and the frequency of the transmission signal 220. The data processing unit 30 performs Fourier transform (in this example, FFT) on the intermediate signal 150 to generate distance information. The distance information is a spectrum with a horizontal axis of range and a vertical axis of level. The range represents the interval of FFT and corresponds to the frequency of the intermediate signal 150.

[0174] The conversion condition determination unit 40 determines whether at least one of the magnitude of the peak value P1 of the distance information and the position (interval in this example) of the peak interval PB satisfies the conversion condition. The peak value P1 is a point having the maximum level in the distance information. The peak interval PB is an interval in which the peak value P1 is located. The position of the peak interval PB corresponds to the distance R from the measuring device 100 to the object 12. In this example, the greater the distance R, the greater the value of the position of the peak interval PB.

[0175] The conversion condition determination unit 40 may determine that the position of the peak section PB satisfies the conversion condition when the position of the peak section PB is equal to or smaller than a predetermined threshold value TH2. When the distance R is too large, it is relatively difficult to detect a minute displacement of the object 12 with high accuracy.

[0176] The conversion condition determination unit 40 may determine that the peak value P1 satisfies the conversion condition when the peak value P1 is greater than or equal to a predetermined threshold value TH3. If the peak value P1 is too small, the ratio of the sound component to the noise component (SN ratio) of the measurement target sound decreases.

[0177] The conversion condition determination unit 40 may determine whether the number of peaks having a level greater than or equal to a predetermined threshold value TH3 satisfies a predetermined conversion condition. For example, the conversion condition determination unit 40 may determine that the conversion condition is not satisfied when two or more peaks exist in one piece of distance information.

[0178] The conversion condition determination unit 40 may determine whether the half width of the peak P1 satisfies a predetermined conversion condition. For example, the conversion condition determination unit 40 may determine that the conversion condition is not satisfied when the half width of the peak P1 is equal to or larger than a predetermined threshold.

[0179] The conversion condition determination unit 40 may also adjust at least one of the threshold values ​​TH2 and TH3 according to the attribute of the object 12. The conversion condition determination unit 40 may pre-set each threshold value for each attribute of the object 12. The conversion condition determination unit 40 may also adjust at least one of the threshold values ​​TH2 and TH3 according to the orientation information described later. The conversion condition determination unit 40 may pre-set information indicating the relationship between the orientation information and the threshold values ​​TH2 and TH3.

[0180] (Determination Process 3)

[0181] Fig.16 This is a diagram showing an example of direction information. Figure 4 The direction information generating unit 34 shown generates direction information. Fig. 11B As described in , the direction information generating unit 34 may generate direction information by performing FFT in the direction of the distance information of the peak interval PB. The direction information generating unit 34 of this example generates direction information in two directions, the X-axis direction and the Z-axis direction. Fig.16 The orientation information in the X-axis direction and the Z-axis direction is schematically shown on the orthogonal axes. Fig.16 In the example, the higher the level of the azimuth information, the higher the density of the hatching. Fig.16 In, will be Figure 2 The measurement range 400 described in the above is displayed in an overlapping manner.

[0182] Fig.16 The coordinate diagram shows that the intermediate signal of the area 402 with a high level of the azimuth information is suitable for conversion of the sound data. The azimuth information generating unit 34 can determine whether the reflected waves (in this example, the intermediate signal) of each direction meet the conversion condition based on the azimuth information. The azimuth information generating unit 34 can determine that the intermediate signal of the area 402 meets the conversion condition when the level of the azimuth information becomes greater than a predetermined threshold. The conversion condition determining unit 40 can also adjust the threshold according to the attribute of the object 12. The conversion condition determining unit 40 can pre-set the threshold for each attribute of the object 12.

[0183] In this manual, sometimes Fig.16 The determination process using the orientation information described in the above is referred to as determination process 3. The conversion condition determination unit 40 may perform determination process 3 after performing at least one of determination process 1 and determination process 2. The conversion condition determination unit 40 may perform determination process 3 when it is determined that the conversion condition is satisfied in all the determination processes performed in determination process 1 and determination process 2.

[0184] (Determination Process 4)

[0185] Fig.17 FIG. 1 is a diagram showing an example of an IQ diagram in a peak interval PB of an intermediate signal. Fig.17 The determination process using the IQ diagram described in is referred to as determination process 4. Fig.17 In FIG. 1 , the time variation of the real part I and the imaginary part Q of the distance information of the peak interval PB is plotted on the orthogonal axis. When the phase of the intermediate signal of the peak interval PB corresponds to the appropriate sound data, Fig.17 The IQ graph has a circular or arc shape with a radius greater than a specified radius.

[0186] The conversion condition determination unit 40 may determine the Fig.16 The conversion condition determination unit 40 generates an IQ graph for the intermediate signal of each area 402 whose orientation information is determined to satisfy the conversion condition in the determination process 3 described in the above. The conversion condition determination unit 40 can determine whether the IQ graph of the intermediate signal presents a circular shape or an arc shape having a radius greater than the threshold value TH4. When the IQ graph satisfies the condition, the conversion condition determination unit 40 can determine that the intermediate signal satisfies the conversion condition. The conversion condition determination unit 40 can also adjust the threshold value TH4 according to the attribute of the object 12. The conversion condition determination unit 40 can pre-set the threshold value for each attribute of the object 12.

[0187] (Determination Process 5)

[0188] Fig.18 is a diagram showing an example of speed information. Fig.12 As described in , the speed information generating unit 36 ​​generates speed information by performing FFT on the phase of the intermediate signal in the peak interval PB in the numbering direction (i.e., the time direction) of the chirp signal. The speed information is a spectrum of the phase variation of the intermediate signal. The speed information generating unit 36 ​​of this example can generate speed information for the region 402 (i.e., the azimuth θ) determined by the determination process 3 to satisfy the conversion condition.

[0189] The level of the frequency component of the speed information corresponds to the level of the frequency component of the sound data. The conversion condition determination unit 40 determines whether the measurement data satisfies the conversion condition based on the speed information. Fig.18The determination process using the speed information described in is referred to as determination process 5.

[0190] The conversion condition determination unit 40 of this example can determine whether the conversion condition is satisfied based on the component of a predetermined frequency band F1 in the speed information. The frequency band F1 is a frequency band corresponding to the sound of the measurement object. When measuring heart sounds, the frequency band F1 can be 30 Hz or more and 100 Hz or less.

[0191] The conversion condition determination unit 40 may determine that the measurement data satisfies the conversion condition when there is a component greater than a predetermined threshold value TH5 in the frequency band F1. Thus, it is possible to exclude sound data having a frequency component that is too small for the sound of the measurement object. The conversion condition determination unit 40 may also adjust the threshold value TH5 according to the attribute of the object 12. The conversion condition determination unit 40 may predetermine the threshold value for each attribute of the object 12.

[0192] The conversion condition determination unit 40 may perform one of the determination process 4 and the determination process 5 on the intermediate signal of the region 402 extracted by the determination process 3. The conversion condition determination unit 40 may also perform both the determination process 4 and the determination process 5 on the intermediate signal of the region 402 extracted by the determination process 3. In this case, when it is determined that the conversion condition is satisfied in at least one of the determination process 4 and the determination process 5, it may be determined that the intermediate signal satisfies the conversion condition, or when it is determined that the conversion condition is satisfied in both the determination process 4 and the determination process 5, it may be determined that the intermediate signal satisfies the conversion condition.

[0193] The data conversion unit 50 converts the intermediate signal determined to satisfy the conversion condition in at least one of the determination processes 1 to 5 into sound data. The data conversion unit 50 may combine one or more of the determination processes 1 to 5 to extract the intermediate signal to be converted into sound data. The data conversion unit 50 may also convert the intermediate signal determined to satisfy the conversion condition in all of the determination processes 1 to 5 into sound data.

[0194] (Determination Process 6)

[0195] Fig.19A : is a diagram showing an example of sound data converted from measurement data. The measurement object in this example is heart sound. The data conversion unit 50 in this example extracts the time waveform of the phase value of the intermediate signal in the peak interval PB as sound data. The data conversion unit 50 can generate the waveform of the sound data by plotting the data of the phase value of the intermediate signal in the direction of the number of the chirp signal.

[0196] The conversion condition determination unit 40 of this example compares the time waveform represented by the sound data converted by the data conversion unit 50 with a predetermined reference waveform to determine whether the sound data satisfies the conversion condition. The conversion condition determination unit 40 can calculate the similarity between the reference waveform and the time waveform of the sound data, and determine that the sound data satisfies the conversion condition when the similarity is greater than a threshold. For example, the more similar the parameters such as the interval between the peaks of the time waveform, the ratio of the magnitudes of the peaks, and the half-peak width of the peaks are, the higher the similarity is calculated. In addition, various pattern matching using the Pearson correlation coefficient and the like can also be performed.

[0197] The conversion condition determination unit 40 records the audio data satisfying the conversion condition as the audio data to be provided to the user, and processes the audio data not satisfying the conversion condition as the audio data not to be provided to the user. In this specification, the processing using the time waveform of the audio data is sometimes referred to as determination processing 6.

[0198] The conversion condition determination unit 40 may also extract a component of a predetermined frequency band from the sound data converted from the measurement data and compare it with the reference waveform. The frequency band corresponds to the frequency band of the sound of the measurement object. The conversion condition determination unit 40 may compare a waveform obtained by removing high-frequency noise above a predetermined frequency from the sound data with the reference waveform.

[0199] (Determination Process 7)

[0200] Fig.19B : is a diagram showing an example of sound data converted from measurement data. The measurement object in this example is heart sound. The data conversion unit 50 in this example extracts the time waveform of the phase value of the intermediate signal in the peak interval PB as sound data. The data conversion unit 50 generates the waveform of the sound data by plotting the data of the phase value of the intermediate signal in the direction of the number of the chirp signal.

[0201] The conversion condition determination unit 40 of this example uses the amplitude of the time waveform represented by the audio data converted by the data conversion unit 50 to determine whether the conversion condition is satisfied. Fig.19B The determination process using the amplitude of the time waveform represented by the audio data described in the above is referred to as determination process 7.

[0202] The conversion condition determination unit 40 of this example determines that the conversion condition is satisfied when the amplitude of the audio data exceeds at least one of the predetermined determination thresholds TH7 and TH8. Fig.19BIn the example, the positive threshold of the displacement level corresponding to the sound data is set to TH7, and the negative threshold is set to TH8. In this specification, exceeding the threshold means being larger than the positive threshold or smaller than the negative threshold. The conversion condition determination unit 40 can extract the component of the specified frequency band in the sound data converted from the measurement data and determine the amplitude of the time waveform.

[0203] (Determination Process 8)

[0204] In determination processes 1 to 7, the conversion condition determination unit 40 determines whether the information included in the measurement data satisfies the conversion condition. In other examples, the conversion condition determination unit 40 may further determine whether the relative orientation between the measurement device 100 and the object 12 satisfies a predetermined conversion condition. In this way, the sound data of the object 12 can be acquired more efficiently.

[0205] Fig.19C and Fig.19D FIG. 4 is a diagram for explaining another example of the operation of the conversion condition determination unit 40. Fig.19C and Fig.19D The determination process using the amplitude and distance information of the intermediate signal described in is referred to as determination process 8.

[0206] Fig.19C 1 is a diagram for explaining the influence of the relative orientation of the object 12 and the measuring device 100 on the amplitude of the reflected wave. As an example, the object 12 is a person, and its representative plane 13 is set as the coronal plane of the human body, and its normal vector b is defined as the front direction of the human body of the sagittal horizontal axis. In addition, as an example, the direction vector a from the measuring device 100 to the object 12 is set as a vector with the center of gravity of the measuring device 100 as the starting point and the center of gravity of the object 12 as the end point. At this time, the relative orientation of the measuring device 100 and the object 12 changes according to the posture of the person as the object 12 with respect to the measuring device 100, and the intensity of the reflected wave received by the measuring device 100 also changes.

[0207] The conversion condition determination unit 40 of this example may also use the amplitude of the intermediate signal to preliminarily determine whether to perform the distance information determination. The conversion condition determination unit 40 may perform the distance information determination based on the determination process 2 when the amplitude of the intermediate signal satisfies the prescribed condition. The conversion condition determination unit 40 may determine whether the relative orientation between the measurement device 100 and the object 12 satisfies the conversion condition based on the amplitude of the intermediate signal and the distance information.

[0208] The conversion condition determination unit 40 of this example can perform the determination using the amplitude of the intermediate signal and the determination of the distance information in the same cycle. The determination cycle can be longer than the measurement cycle when the sound data is acquired when the conversion condition is satisfied. Thus, power consumption can be reduced. The conversion condition determination unit 40 can perform at least a part of the determination process using the amplitude of the intermediate signal and at least a part of the determination process of the distance information in parallel.

[0209] Fig.19D and Fig.14 Likewise, time series data of the absolute value |max| of the maximum amplitude of the intermediate signal is shown. Fig.19D The processed frames in the horizontal Fig.14 The time on the horizontal axis corresponds to the time on the horizontal axis. The conversion condition determination unit 40 can detect the absolute value of the maximum amplitude of the intermediate signal corresponding to the plurality of chirp signals and make it into time series data. The conversion condition determination unit 40 of this example can determine whether the preliminary conversion condition for determining whether to perform distance information determination is satisfied based on the amplitude of the reflected wave represented by the measurement data. The amplitude of the intermediate signal is determined according to the amplitude of the reflected wave. The conversion condition determination unit 40 of this example indirectly determines whether the amplitude of the reflected wave satisfies the prescribed condition by determining whether the amplitude of the intermediate signal satisfies the prescribed condition. In this specification, sometimes Fig.19D The determination process using the amplitude of the intermediate signal described in is referred to as determination process 8-1.

[0210] The conversion condition determination unit 40 of this example can determine that the intermediate signal satisfies the preliminary conversion condition when the absolute value |max| of the maximum amplitude of the intermediate signal is greater than the predetermined determination threshold TH6, and perform the determination of the distance information. The conversion condition determination unit 40 can also determine that the intermediate signal satisfies the preliminary conversion condition when the absolute value |max| of the maximum amplitude of the intermediate signal is continuously greater than the determination threshold TH6 within a predetermined determination period. The predetermined determination period is a period including a plurality of chirp signals. The determination period can also be a period including a chirp signal of one pulse train.

[0211] The conversion condition determination unit 40 of this example can determine that the intermediate signal satisfies the conversion condition when the absolute value |max| of the maximum amplitude of the intermediate signal is greater than the predetermined determination threshold TH1, similarly to the determination process 1. The conversion condition determination unit 40 can also determine that the intermediate signal satisfies the conversion condition when the absolute value |max| of the maximum amplitude of the intermediate signal is continuously greater than the determination threshold TH1 within a predetermined determination period. The predetermined determination period is a period including a plurality of chirp signals. The determination period can also be a period including a chirp signal of one pulse train.

[0212] The determination threshold TH6 can be smaller than the determination threshold TH1. Thus, the determination threshold 6 can suppress signal processing when the object 12 is not near the information processing device 10, and avoid inefficient measurement due to an inappropriate relative position even though the object 12 is near the information processing device 10.

[0213] The conversion condition determination unit 40 of this example determines whether the distance information satisfies the conversion condition through determination process 2 when the reflected wave amplitude satisfies the preliminary conversion condition. The conversion condition determination unit 40 may determine that the relative orientation of the object 12 and the measuring device 100 is inappropriate when the position of the peak interval PB is less than a predetermined threshold value TH2 and the absolute value |max| of the maximum amplitude of the intermediate signal is smaller than a predetermined determination threshold value TH1.

[0214] The conversion condition determination unit 40 can determine that the relative orientation of the object 12 and the measuring device 100 is inappropriate when the magnitude of the peak value P1 is greater than or equal to a predetermined threshold value TH3, the position of the peak interval PB is less than or equal to a predetermined threshold value TH2, and the absolute value |max| of the maximum amplitude of the intermediate signal is smaller than a predetermined determination threshold value TH1. When the peak value P1 is too small, the ratio of the sound component of the measuring object to the noise component (SN ratio) becomes small.

[0215] The conversion condition determination unit 40 may also adjust at least one of the threshold values ​​TH1 and TH6 according to the attribute of the object 12. The conversion condition determination unit 40 may pre-set each threshold value for each attribute of the object 12. The conversion condition determination unit 40 may also adjust at least one of the threshold values ​​TH1 and TH6 according to the orientation information described later. The conversion condition determination unit 40 may pre-set information indicating the relationship between the orientation information and the threshold values ​​TH1 and TH6.

[0216] (Determination Process 9)

[0217] In the determination process 8, the relative position of the measuring device 100 and the object 12 is determined based on the amplitude and distance information of the intermediate signal. In this example, the determination process using the amplitude and distance information of the intermediate signal and the phase variation is shown. In this way, the relative position of the measuring device 100 and the object 12 can be determined more accurately, so that the sound data of the object 12 can be obtained more efficiently.

[0218] Fig.19E1 is a diagram for explaining the influence of the relative orientation θr between the object 12 and the measuring device 100 on the phase variation. As an example, the object 12 is a person, and its representative plane 13 is set as the coronal plane of the human body, and its normal vector b is defined as the front direction of the human body of the sagittal horizontal axis. In addition, as an example, the direction vector a from the measuring device 100 to the object 12 is set as a vector starting from the center of gravity of the measuring device 100 and ending at the center of gravity of the object 12. At this time, the relative orientation θr between the measuring device 100 and the object 12 changes according to the posture of the person as the object 12 with respect to the measuring device 100, and the phase variation received by the measuring device 100 also changes.

[0219] exist Fig.19E , objects 12-1 and 12-2 are shown at two positions with different relative orientations with respect to the measuring device 100. As described above, the relative orientation θr is the angle θr between the normal vector b of the representative surface 13 and the direction vector a from the measuring device 100 to the object 12. Fig.19E In the example, the relative orientation θr of the object 12 - 1 is 180 degrees.

[0220] Fig.19F An example is shown in which the position of the representative surface 13 of the object 12-2 changes in the direction of the normal vector b. For example, the representative surface 13-2 is the skin of the object 12-2, and due to the breathing of the object 12-2, etc., the skin moves from the position of the representative surface 13-2a to the position of the representative surface 13-2b. For example, the vibration amplitude when the direction of the normal vector b of the representative surface 13 of the object 12 is consistent with the vibration direction corresponding to the sound to be measured is set to Δd. In this case, according to the angle θr between the normal vector b at the specified relative orientation θr and the direction vector a from the object 12 to the measuring device 100, the observable vibration component becomes Δd·|cos(θr)|. In the case Fig.19E When the relative orientation θr is 180 degrees, as in the case of the object 12-1 shown in FIG. 1 , the observable vibration component is Δd. Fig.19F As shown, when θr is not 0 degrees or 180 degrees, the observable vibration component decreases in accordance with |cos(θr)|.

[0221] Therefore, even when the object 12 is sufficiently close to the measuring device 100 and the conversion conditions of the reflected wave amplitude and the distance information are satisfied by the determination process 1 and the determination process 2, appropriate sound data may not be obtained due to the relative position θr of the object 12 with respect to the measuring device 100 caused by the orientation and posture of the object 12. In other words, the conversion condition determination unit 40 can determine whether the relative position of the object 12 and the measuring device 100 is appropriate based on at least one of the reflected wave amplitude and the distance information and the information of the phase change amount of the intermediate signal, that is, the phase shift information.

[0222] In this specification, the determination process related to the relative orientation using the amplitude and distance information and the phase variation of the intermediate signal is sometimes referred to as determination process 9. Specifically, the conversion condition determination unit 40 can determine that the relative orientation of the object 12 and the measuring device 100 is not appropriate when the conversion conditions of at least one of the determination processes 1 and 2 are satisfied and the conversion conditions of at least one of the determination processes 4, 5, and 7 are not satisfied.

[0223] In determination process 8 or 9, the data processing unit 30 can calculate the relative orientation determination information with respect to the object 12 based on at least one of the amplitude, distance information, and phase shift information of the reflected wave. Phase shift information is information indicating the phase change of the intermediate signal. The relative orientation determination information is an indicator for determining whether the relative orientation satisfies a specified conversion condition. The relative orientation determination information may not be information directly indicating the size of the relative orientation. For example, the relative orientation determination information may include at least one of the amplitude, distance information, and phase shift information of the reflected wave. The conversion condition determination unit 40 can determine whether at least one of the amplitude, distance information, and phase shift information of the reflected wave in the relative orientation determination information satisfies the conversion condition as described in determination process 8 or 9. In other examples, the relative orientation determination information may also be information indicating the size of the relative orientation θr.

[0224] (Determination Process 10)

[0225] In determination processes 1 to 9, the conversion condition determination unit 40 determines whether the information included in the measurement data satisfies the conversion condition and whether the relative orientation is appropriate. In other examples, the conversion condition determination unit 40 may also determine whether the conversion condition is satisfied based on an image of the space irradiated with the transmission wave. In this specification, processing based on a space image is sometimes referred to as determination process 10. The conversion condition determination unit 40 may determine that the conversion condition is satisfied when the object 12 is included in a predetermined position in the image. The measuring device 100 may include a photographing device for acquiring the image. The photographing device may acquire the image continuously or periodically in time.

[0226] The conversion condition determination unit 40 may calculate the distance R from the measurement device 100 to the object 12 based on the image. When the calculated distance R satisfies a predetermined condition, the conversion condition determination unit 40 may determine that the conversion condition is satisfied.

[0227] The conversion condition determination unit 40 can calculate the posture or orientation of the object 12 based on the image. The conversion condition determination unit 40 can determine whether the posture or orientation of the object 12 satisfies the conversion condition. For example, the conversion condition determination unit 40 can determine that the conversion condition is satisfied when the chest of the object 12 is facing the direction of the measurement device 100.

[0228] The conversion condition determination unit 40 may perform determination processing 10 in addition to the determination processing based on the measurement data. For example, the conversion condition determination unit 40 may perform determination processing based on the measurement data when it is determined in determination processing 10 that the conversion condition is satisfied. The conversion condition determination unit 40 may perform determination processing 10 instead of any one of determination processing 1 to 6. For example, the conversion condition determination unit 40 may perform determination processing 10 instead of determination processing 1.

[0229] Fig. 20 4 is a diagram for explaining an example of a data map 410 generated by the conversion condition determination unit 40. The data map 410 indicates whether there is sound data to be provided to the user for each orientation θ. The data map 410 of this example includes information indicating whether there is sound data to be provided to the user for each of the plurality of regions 402 included in the measurement range 400. The plurality of regions 402 correspond to the plurality of orientations θ.

[0230] The conversion condition determination unit 40 may generate information for displaying the data map 410. The data map 410 may include information indicating the positions of the plurality of regions 402 and information for displaying whether or not there is sound data for each region 402 ( Fig. 20 The conversion condition determination unit 40 may generate information for displaying an image representing at least a portion of the object 12 and an image representing the plurality of regions 402 in an overlapping manner. The image of the object 12 may be an image of the object 12 itself or a schematic image representing at least a portion of the object 12.

[0231] The conversion condition determination unit 40 may generate a data map 410 including audio data corresponding to each area 402. The conversion condition determination unit 40 may generate the data map 410 so that the corresponding audio data is reproduced when the circle mark of the area 402 is selected on the image displayed on the display device.

[0232] The data map 410 may also include information indicating the priority of the sound data of each area 402. For example, the conversion condition determination unit 40 may prioritize the corresponding sound data from high to low in the order of the amplitude of the reflected wave from large to small. The conversion condition determination unit 40 may prioritize the corresponding sound data from high to low in the order of the level of the frequency component in the frequency band of the sound of the measurement object from large to small. The data map 410 may be configured to Fig. 20 The circle symbol in the format of adding or replacing the circle symbol includes a numerical value indicating the priority of the sound data.

[0233] Fig.21 4 is a diagram for explaining an example of a combined data map 420 generated by the conversion condition determination unit 40. The conversion condition determination unit 40 of this example generates data maps 410 for different timings. For example, the conversion condition determination unit 40 generates data maps 410 for each pulse train of the transmission signal 220.

[0234] The conversion condition determination unit 40 generates a combined data map 420 based on the generated multiple data maps 410, and the combined data map 420 indicates whether there is sound data at at least one timing. For example, the conversion condition determination unit 40 determines whether there is sound data that should be provided to the user in at least one data map 410 for each area 402. The conversion condition determination unit 40 of this example also adds a circle mark to the area 402 with a circle mark in at least one data map 410 in the combined data map 420. In this way, the number of areas 402 where there is sound data that can be provided to the user can be increased. When there are multiple sound data that can be provided to the user in one area 402, the conversion condition determination unit 40 can establish a correspondence between the sound data with the highest priority and the area 402 and record it. In addition, the area 402 that is repeatedly determined to have sound data in multiple data maps 410 can be displayed as a region with high measurement reliability using a change in color or a numerical value. This reliability may be displayed together with the priority assigned by the transition condition determination unit 40 (by adding or multiplying the numerical values), or may be displayed separately.

[0235] Fig. 22 4 is a diagram showing another example of the data map 410 generated by the conversion condition determination unit 40. In this example, there is no audio data that can be provided to the user in all the areas 402. That is, the measurement data in this example does not satisfy the conversion condition.

[0236] The conversion condition determination unit 40 of this example generates notification information when the measurement data does not satisfy the conversion condition, and the notification information notifies that the conversion condition is not satisfied. The conversion condition determination unit 40 of this example generates notification information when the measurement data does not satisfy the conversion condition and at least one of the determination processes 8 or 9 determines that the relative orientation (or relative orientation determination information) between the object 12 and the measurement device 100 is not appropriate, and the notification information notifies that the relative orientation is not appropriate.

[0237] The conversion condition determination unit 40 of this example generates instruction information 432 indicating the position to which the object 12 should be moved in order to satisfy the conversion condition when the measurement data does not satisfy the conversion condition. The position of the object 12 includes the posture of the object 12, the relative position and relative orientation of the object 12 and the measuring device 100. The conversion condition determination unit 40 can determine the current position of the object 12 based on the image of the object 12. The conversion condition determination unit 40 can indicate the position to which the object 12 should be moved and the posture to be taken based on the current position of the object 12. For example, when the object 12 is facing the measuring device 100 sideways, the conversion condition determination unit 40 can generate instruction information 432 indicating that the object 12 should be facing the front relative to the object 12. The notification information and the instruction information 432 can be displayed on the display device 430 or output by voice. If the method of indicating to the user is voice, the position to be moved can be appropriately indicated even when the user is measuring the back side of himself. The display device 430 can be connected to the measuring device 100, or it can be another information device such as a smart phone or a smart watch. The method of indicating to the user may also use vibration, flashing of LEDs, etc., display on the screen, and localization of sound. In the notification based on vibration, the vibration may be strengthened (or weakened) according to the distance of the object 12 from the position suitable for measurement. In the flashing of LEDs, etc., the lighting interval may be changed according to the distance of the object 12 from the position suitable for measurement, and the lighting method such as the lighting interval of the LED may be changed according to the relative position of the object 12 with respect to the measurement device 100, the direction of the object 12, or the posture of the object 12. The lighting method may include, for example, which of the multiple LEDs is lit. In the display on the screen, an arrow may also be displayed. In the display on the screen, a measurement position image indicating the position where the object 12 should be moved may also be displayed superimposed on the current position image indicating the current position of the object 12. The current position image is, for example, an image of the object 12. The measurement position image is, for example, the outline of at least a portion of the object 12.

[0238] The conversion condition determination unit 40 may generate the instruction information 432 based on the distribution of the regions 402 where the sound data that can be provided exists. The conversion condition determination unit 40 may generate the instruction information 432 indicating the position to which the object 12 should move so that the region 402 where the sound data that can be provided exists is close to the center of the measurement range 400. When there are a plurality of regions 402, the conversion condition determination unit 40 may generate the instruction information 432 indicating the position to which the object 12 should move so that the center of the region where the regions 402 are distributed is close to the center of the measurement range 400.

[0239] The conversion condition determination unit 40 may generate instruction information 432 to search for a position satisfying the conversion condition. For example, the object 12 may be instructed to rotate once, and when the relative orientation thereof changes to a position satisfying the conversion condition, the object 12 may be instructed to rotate in the opposite direction, thereby guiding the object 12 to the position satisfying the conversion condition.

[0240] The conversion condition determination unit 40 of this example may move the position of the measurement device 100 to satisfy the conversion condition when the measurement data does not satisfy the conversion condition. The measurement device 100 may include a movable unit that changes the relative position or relative orientation with respect to the object 12 .

[0241] The conversion condition determination unit 40 may generate instruction information 432 for indicating the position to be moved to the object 12 when the relative orientation information at the relative position of the object 12 does not satisfy the conversion condition. The conversion condition determination unit 40 may convert at least a part of the measurement data into sound data when the relative orientation information at the relative position of the object 12 satisfies the conversion condition.

[0242] Fig.23A is a block diagram showing another configuration example of the information processing device 10. The information processing device 10 of this example is Figure 3 The structure of the information processing device 10 shown in the figure further includes an identification information acquisition unit 80. The identification information acquisition unit 80 acquires identification information of the identification object 12. The identification information is information that can distinguish the individual of the object 12. The data conversion unit 50 records the generated sound data and the identification information in correspondence. In this way, it is possible to manage from which object 12 the automatically acquired sound data is acquired.

[0243] The identification information acquisition unit 80 may acquire identification information related to the appearance of the object 12 from the image of the object 12. The identification information may include information related to the face of the object 12, or information related to the size of the object 12 such as height.

[0244] The identification information acquisition unit 80 can acquire identification information related to the movement of the object 12. The identification information can be acquired from an image of the object 12 or from measurement data of the measuring device 100. The identification information can include information related to the moving speed of the object 12. The identification information can include information related to the moving speed of the object 12 when approaching a predetermined object. The object is, for example, a mirror. In addition, the object can also be an electronic device such as a smartphone, a tablet, a computer, a display, a smart speaker, or an office / residential device such as a switch for a door, a window, a wall, or a lighting.

[0245] The identification information acquisition unit 80 may acquire identification information indicating a voiceprint of the object 12. In this case, the measurement device 100 may include a voice acquisition unit that acquires voice data of the object 12.

[0246] The identification information acquisition unit 80 can generate identification information based on the measurement data. The identification information acquisition unit 80 can determine which identification information the newly generated voice data corresponds to by comparing voice data generated in the past and associated with the identification information with the newly generated voice data.

[0247] The identification information acquisition unit 80 may acquire the amplitude or period of the displacement of the object 12 generated from the measurement data as identification information. For example, the amplitude and period of the displacement of the chest associated with breathing differ between individuals of the object 12 .

[0248] The identification information acquisition unit 80 can acquire the distance information and the azimuth information generated from the measurement data as identification information. The relative position of the object 12 relative to the measurement device 100 can be known from the distance information and the azimuth information. The individual object 12 can be identified based on the time change and the trajectory of the relative position. For example, the distance information is obtained based on the distance FFT, and the azimuth information is obtained based on the azimuth FFT.

[0249] Fig. 23B is a diagram showing the time change of the relative position of the object 12 obtained based on the distance information and the direction information. Fig. 23B In the example, the position of the object 12 is indicated by a circle using an arbitrary orthogonal coordinate system (x-axis and y-axis) with the measuring device 100 as the origin. At this time, the information processing device 10 can identify the individual of the object 12 based on the continuity of the temporal change of the position. Fig. 23B In the example, the plotted points of the plurality of circular marks can be classified into the object 12-A and the object 12-B. At this time, the information processing device 10 can simultaneously indicate whether the measurement data obtained at the position indicated by each circular mark satisfies the conversion condition. By indicating whether the conversion condition is satisfied together with the identification information, the object 12 can be guided to a measurable position, and efficient measurement can be performed.

[0250] Fig.24 This is a flowchart for explaining an example of the operation of the system 300. The operation of the system 300 is not limited to the operation of this example. The system 300 can perform the processing described in this specification.

[0251] In the measurement phase S502, the measurement device 100 emits a transmission wave and receives the reflected wave 231. The measurement device 100 generates measurement data based on the reflected wave 231.

[0252] In the reflected wave intensity determination stage S504, the switching condition determination unit 40 determines whether the intensity of the reflected wave 231 is greater than a predetermined threshold value. Fig.14 As in the determination process 1 described in the above, it is determined whether the absolute value |max| of the maximum amplitude of the intermediate signal 150 generated based on the reflected wave 231 is greater than the threshold value TH1. In this way, the intensity of the reflected wave 231 can be indirectly compared with the threshold value. As in the determination process 8, the conversion condition determination unit 40 can also set a threshold value TH6 for preliminary determination of whether to perform the subsequent process independently of the threshold value for determining the conversion condition. In this way, by combining it with the result of the subsequent distance condition determination stage S508, it is possible to determine whether the relative orientation of the object 12 and the measuring device 100 is appropriate.

[0253] When the intensity of the reflected wave 231 is less than the threshold value TH1, the conversion condition determination unit 40 determines that the conversion condition is not satisfied. In this case, the system 300 repeats the process of S502. When the intensity of the reflected wave 231 is greater than the threshold value, the conversion condition determination unit 40 determines that the first stage conversion condition is satisfied, and performs the process after S506.

[0254] The conversion condition determination unit 40 may not perform subsequent processing when the intensity of the reflected wave 231 is less than the threshold value TH6. In this case, the system 300 repeatedly performs the processing of S502. When the intensity of the reflected wave 231 is greater than the threshold value TH6, the conversion condition determination unit 40 determines that the conversion condition of the first stage is satisfied, and performs the processing after S506.

[0255] In the distance information acquisition stage S506 , the distance information generation unit 32 generates distance information based on the measurement data. The position of the peak interval PB in the distance information indicates the distance R from the measurement device 100 to the object 12 .

[0256] In the distance condition determination step S508, the conversion condition determination unit 40 determines whether the distance R to the object 12 is within a predetermined range. Fig.15 As in the determination process 2 described in , it is determined whether the position of the peak interval PB in the distance information is within the specified range. The position of the peak interval PB can use the average value of the positions of the peak intervals of multiple distance information, or can use the peak interval PB of the specified distance information such as the distance information corresponding to the area 402 configured in the center of the measurement range.

[0257] The conversion condition determination unit 40 determines that the conversion condition is not satisfied when the position of the peak interval PB is outside the prescribed range. In this case, the system 300 repeatedly performs the process of S502. The conversion condition determination unit 40 may determine that the conversion condition is satisfied when the position of the peak interval PB is within the prescribed range. The conversion condition determination unit 40 may also determine that the conversion condition is not satisfied when the size of the distance information in the peak interval PB is smaller than a prescribed threshold. The conversion condition determination unit 40 may determine that the conversion condition is satisfied when both the position of the peak interval PB and the size of the distance information in the peak interval PB satisfy prescribed conditions. When it is determined that the conversion condition is satisfied, the information processing device 10 performs the process after S510.

[0258] In the distance condition determination stage S508, the conversion condition determination unit 40 may also determine whether the relative orientation of the object 12 and the measuring device 100 is appropriate based on the reflected wave intensity determination result and the distance condition determination result. Fig.19C and Fig.19D As in the determination process 8 described in the above, the amplitude and distance information of the intermediate signal are used to determine whether the relative orientation of the object 12 and the measuring device 100 is appropriate.

[0259] The conversion condition determination unit 40 may determine that the relative orientation of the object 12 and the measurement device 100 is inappropriate when the position of the peak section PB is equal to or smaller than a predetermined threshold TH2 and the absolute value |max| of the maximum amplitude of the intermediate signal is smaller than a predetermined determination threshold TH1.

[0260] The conversion condition determination unit 40 may determine that the relative orientation of the object 12 and the measuring device 100 is inappropriate when the size of the peak P2 is greater than a specified threshold TH3, the position of the peak interval PB is less than a specified threshold TH2, and the absolute value |max| of the maximum amplitude of the intermediate signal is smaller than a specified determination threshold TH1.

[0261] In the direction information acquisition step S510, the direction information generating unit 34 acquires direction information indicating the direction θ at which the intermediate signal to be converted into audio data is received. The direction information generating unit 34 in this example generates direction information by performing FFT in the direction direction on the complex data of the distance information in the peak interval PB.

[0262] In the candidate coordinate extraction stage S512, the direction information generation unit 34 determines the direction θ at which the intermediate signal to be converted into sound data is received based on the direction information. Fig.16As described in , the direction information generating unit 34 determines the direction θ based on the position of the peak in the direction information. The direction information generating unit 34 may also determine the coordinates of the region 402 corresponding to the direction θ in the XZ plane. The direction information generating unit 34 extracts the sound data corresponding to the region 402 where the peak of the direction information is located as a candidate of the sound data to be provided to the user.

[0263] In the audio condition determination stage S514, the conversion condition determination unit 40 determines whether each audio data extracted as a candidate satisfies a predetermined conversion condition. The conversion condition determination unit 40 may determine whether each audio data extracted as a candidate satisfies a predetermined conversion condition. Fig.17 As in the determination process 4 described in the above, for each chirp signal, it is determined whether the IQ diagram obtained by plotting the phase in the peak interval PB has a predetermined shape. The conversion condition determination unit 40 can determine that the audio data satisfies the conversion condition when the IQ diagram has a circle or arc with a radius greater than a predetermined radius.

[0264] The conversion condition determination unit 40 may also be as in Fig.18 As in the determination process 5 described in the above, for each region 402 extracted in S512, it is determined whether the speed information obtained by performing FFT on the complex data at the peak interval PB in the time direction satisfies the prescribed conversion condition. Fig.19A and Fig.19B The conversion condition determination unit 40 may determine whether the time waveform of the sound data is similar to the reference waveform, or may determine whether the amplitude of the time waveform of the sound data exceeds a predetermined threshold value, as in the determination processes 6 and 7 described in the above. The conversion condition determination unit 40 may perform at least one of the determination processes 4 to 9. The conversion condition determination unit 40 may determine that the relative orientation of the object 12 and the measuring device 100 is inappropriate when the conversion conditions of at least one of the determination processes 1 and 2 are satisfied and the conversion conditions of at least one of the determination processes 4, 5, and 7 are not satisfied. The conversion condition determination unit 40 determines that the sound data of the region 402 satisfies the conversion conditions when the conditions are satisfied in all the determination processes performed.

[0265] The data conversion unit 50 generates and records all the sound data that satisfy the conversion conditions. The data conversion unit 50 may record each sound data and the coordinates of the area 402 in correspondence. The data conversion unit 50 may also record information indicating the time of measurement (for example, a combination of a pulse train number and a chirp signal number) in correspondence with the sound data.

[0266] When the conversion condition determination unit 40 determines that all the audio data candidates extracted in S512 do not satisfy the conversion condition, the process from S502 is repeated. When at least one audio data satisfies the conversion condition, the process from S516 is executed.

[0267] In the data output stage S516, the data output unit 60 outputs the sound data. The data output unit 60 can output the sound data according to the instructions from the user. Fig. 20 Information related to the audio data, such as the data map 410 described in the above, is displayed on the data display unit 70. The data output unit 60 may also reproduce the audio corresponding to the audio data in response to an instruction from a user or the like.

[0268] In the data transmission stage S518, the data output unit 60 transmits the sound data to an external device. The external device is, for example, a terminal of a doctor in a remote place. The data output unit 60 may transmit the data map 410 to the external device, or may transmit information used to generate the data map 410 to the external device. In this example, the sound data that should not be provided to the user among the sound data that can be generated based on the measurement data received by the information processing device 10 is not transmitted to the external device. Therefore, the amount of information transmitted by the data output unit 60 to the external device can be made less than the amount of information in the measurement data received by the information processing device 10.

[0269] Various embodiments of the present invention may be described with reference to flowcharts and block diagrams, where a block may represent (1) a stage of a process for performing an operation or (2) a portion of a device having the function of performing an operation. Specific stages and portions may be implemented by dedicated circuits, programmable circuits supplied with computer-readable instructions stored on a computer-readable medium, and / or processors supplied with computer-readable instructions stored on a computer-readable medium. Dedicated circuits may include digital and / or analog hardware circuits, which may include integrated circuits (ICs) and / or discrete circuits. Programmable circuits may include reconfigurable hardware circuits including storage elements such as logical AND, logical OR, logical XOR, logical NAND, logical NOR, and other logical operations, flip-flops, registers, field programmable gate arrays (FPGAs), programmable logic arrays (PLAs), and the like.

[0270] Computer readable media may include any tangible device capable of storing instructions to be executed by a suitable device, and as a result, a computer readable medium having instructions stored therein has a product containing instructions that can be executed in order to make a means for performing the operations specified by the flowchart or block diagram. Examples of computer readable media include electronic storage media, magnetic storage media, optical storage media, electromagnetic storage media, semiconductor storage media, etc. More specific examples of computer readable media include floppy disks (registered trademarks), floppy disks, hard disks, random access memories (RAM), read-only memories (ROM), erasable programmable read-only memories (EPROM or flash memory), electrically erasable programmable read-only memories (EEPROM), static random access memories (SRAM), compact disk read-only memories (CD-ROM), digital versatile disks (DVD), Blu-ray (RTM) disks, memory sticks, integrated circuit cards, etc.

[0271] Computer-readable instructions may include assembly instructions, instruction set architecture (ISA) instructions, machine instructions, machine-dependent instructions, microcode, firmware instructions, state setting data, or any source code or object code described in any combination of one or more programming languages ​​including object-oriented programming languages ​​such as Smalltalk (registered trademark), JAVA (registered trademark), C++, and conventional procedural programming languages ​​such as the "C" programming language or similar programming languages.

[0272] Computer-readable instructions can be provided to a processor or programmable circuit of a programmable data processing device such as a general-purpose computer, a special-purpose computer or other computer locally or via a wide area network (WAN) such as a local area network (LAN), the Internet, etc., and can be executed in order to make a means for performing the operation specified by a flowchart or a block diagram. Here, the computer can be a PC (personal computer), a tablet computer, a smart phone, a workstation, a server computer, a general-purpose computer or a special-purpose computer, etc., or a computer system formed by connecting multiple computers. Such a computer system formed by connecting multiple computers is also called a distributed computing system, which is a computer in a general sense. In a distributed computing system, multiple computers execute a part of a program respectively, and the data in the execution of the program is handed over between computers as needed, thereby executing the program collectively by multiple computers.

[0273] Examples of processors include computer processors, central processing units (CPUs), processing units, microprocessors, digital signal processors, controllers, microcontrollers, and the like. A computer may have one processor or multiple processors. In a multiprocessor system having multiple processors, each processor executes a portion of a program, and data in program execution is handed over between processors as needed, whereby multiple processors collectively execute the program. For example, in the execution of multiple tasks, multiple processors may each execute a portion of each task piecemeal by switching tasks every time slice. In this case, which portion of a program is executed by each processor may change dynamically. Which portion of a program is executed by each of the multiple processors may also be statically determined by programming with awareness of the multiple processors.

[0274] Fig.25 An example of a computer 1200 in which multiple schemes of the present invention can be embodied in whole or in part is shown. The program installed in the computer 1200 can make the computer 1200 function as an operation associated with the device involved in the embodiment of the present invention or one or more parts of the device, or can make it perform the operation or the one or more parts, and / or can make the computer 1200 perform the process involved in the embodiment of the present invention or the stage of the process. Such a program can be executed by the CPU 1212 to make the computer 1200 perform some of the blocks in the flowcharts and block diagrams described in this specification or specific operations that have been associated.

[0275] The computer 1200 of this embodiment includes a CPU 1212, a RAM 1214, a graphic controller 1216, and a display device 1218, which are connected to each other via a main controller 1210. In addition, the computer 1200 includes input / output units such as a communication interface 1222, a storage device 1224 such as a hard disk drive, a DVD-ROM drive 1226, and an IC card drive, which are connected to the main controller 1210 via an input / output controller 1220. In addition, the computer includes a conventional input / output unit such as a ROM 1230 and a keyboard 1242, which are connected to the input / output controller 1220 via an input / output chip 1240.

[0276] The CPU 1212 controls each unit by operating according to the program stored in the ROM 1230 and the RAM 1214. The graphic controller 1216 obtains image data generated by the CPU 1212 from a frame buffer or the like provided in the RAM 1214 or from itself, and displays the image data on the display device 1218.

[0277] The communication interface 1222 communicates with other electronic devices via a network. The storage device 1224 stores programs and data used by the CPU 1212 in the computer 1200. The DVD-ROM drive 1226 reads programs or data from the DVD-ROM 1227 and provides the programs or data to the storage device 1224 via the RAM 1214. The IC card drive reads programs and data from an IC card and / or writes programs and data to an IC card.

[0278] The ROM 1230 stores therein a boot program or the like executed by the computer 1200 at the time of activation and / or a program depending on the hardware of the computer 1200. In addition, the input / output chip 1240 can connect various input / output units to the input / output controller 1220 via a parallel port, a serial port, a keyboard port, a mouse port, and the like.

[0279] The program is provided by a computer-readable medium such as DVD-ROM 1227 or an IC card. The program is read from the computer-readable medium, installed in the storage device 1224, RAM 1214, or ROM 1230, which is also an example of a computer-readable medium, and executed by the CPU 1212. The information processing described in these programs is read by the computer 1200, resulting in cooperation between the program and the various types of hardware resources described above. The device or method can be configured by implementing the operation or processing of information according to the use of the computer 1200.

[0280] For example, when communication is performed between the computer 1200 and an external device, the CPU 1212 can execute a communication program loaded into the RAM 1214, and instruct the communication interface 1222 to perform communication processing based on the processing described in the communication program. Under the control of the CPU 1212, the communication interface 1222 reads transmission data stored in a transmission buffer processing area provided in a recording medium such as the RAM 1214, the storage device 1224, the DVD-ROM 1227, or an IC card, and transmits the read transmission data to the network, or writes reception data received from the network to a reception buffer processing area provided on the recording medium.

[0281] The CPU 1212 can read all or a necessary part of a file or database stored in an external recording medium such as the storage device 1224, the DVD-ROM drive 1226 (DVD-ROM 1227), or an IC card to the RAM 1214, and perform various types of processing on the data in the RAM 1214. The CPU 1212 then writes the processed data back to the external recording medium.

[0282] Various types of information such as various types of programs, data, tables, and databases can be stored in a recording medium and subjected to information processing. CPU1212 can perform various types of processing, including various types of operations, information processing, conditional judgments, conditional branches, unconditional branches, information retrieval / replacement, etc., which are recorded everywhere in this disclosure and specified by the instruction sequence of the program, on the data read from RAM1214, and write the results back to RAM1214. In addition, CPU1212 can retrieve information in files, databases, etc. in the recording medium. For example, in the case where multiple entries each having an attribute value of a first attribute associated with an attribute value of a second attribute are stored in a recording medium, CPU1212 can retrieve an entry in which the attribute value of the first attribute is specified and consistent with the condition from the multiple entries, read the attribute value of the second attribute stored in the entry, and thereby obtain the attribute value of the second attribute associated with the first attribute that satisfies a predetermined condition.

[0283] The program or software module described above may be stored in a computer-readable medium on or near the computer 1200. In addition, a recording medium such as a hard disk or RAM provided in a server system connected to a dedicated communication network or the Internet can be used as a computer-readable medium, thereby providing the program to the computer 1200 via the network.

[0284] The present invention has been described above using the embodiments, but the technical scope of the present invention is not limited to the scope described in the above embodiments. It is obvious to those skilled in the art that various changes or improvements can be made to the above embodiments. It is obvious from the description of the claims that the modes after such changes or improvements are also included in the technical scope of the present invention.

[0285] It should be noted that the execution order of each process such as actions, processes, steps and stages in the device, system, program and method shown in the claims, specifications and drawings can be implemented in any order as long as it is not specifically stated as "before" or "before" and the output of the earlier process is not used in the later process. Even if the action flow in the claims, specifications and drawings is described using "first" or "next" for convenience, it does not mean that it must be implemented in this order.

Claims

1. An information processing device comprising: A data acquisition unit that acquires measurement data generated based on a reflected wave from an object by irradiating an electromagnetic wave to the object; a conversion condition determination unit that determines whether a signal based on the measurement data satisfies a conversion condition for converting at least a part of the measurement data into sound data; and The data conversion unit converts at least a part of the measurement data into the audio data when a signal based on the measurement data satisfies the conversion condition.

2. The information processing device according to claim 1, wherein: The conversion condition determination unit outputs a control signal that controls the electromagnetic wave and the information processing device in such a manner that at least one of power consumption for irradiating the electromagnetic wave and power consumption of the information processing device remains unchanged before and after the conversion condition is satisfied.

3. The information processing device according to claim 1, wherein: The conversion condition determination unit outputs a control signal, which controls at least one of the electromagnetic wave and the information processing device in the following manner: when the conversion condition is met, at least one of the power consumption used to irradiate the electromagnetic wave and the power consumption of the information processing device is increased compared to before the conversion condition is met.

4. The information processing device according to claim 1, wherein: The conversion condition determination unit outputs a control signal that controls the electromagnetic wave in such a manner that, when the conversion condition is satisfied, an irradiation amount of the electromagnetic wave irradiated toward the object increases compared to before the conversion condition is satisfied.

5. The information processing device according to claim 1, wherein: The switching condition determination unit determines whether the switching condition is satisfied based on the measurement data.

6. The information processing device according to claim 5, wherein: The conversion condition determination unit determines whether the conversion condition is satisfied based on the amplitude of the reflected wave included in the measurement data.

7. The information processing device according to claim 5, wherein: The information processing device further includes a data processing unit that generates distance information based on the reflected wave included in the measurement data, the distance information indicating the distance between the information processing device and the object. The conversion condition determination unit determines whether the distance information satisfies the conversion condition.

8. The information processing device according to claim 7, wherein: The electromagnetic wave includes a chirp signal with a gradually changing frequency. The measurement data includes an intermediate signal having a frequency corresponding to a difference between a frequency of the reflected wave and a frequency of the electromagnetic wave.

9. The information processing device according to claim 8, wherein: The measurement data includes a plurality of the intermediate signals obtained for each transmission and reception channel, the transmission and reception channel being a combination of a transmission antenna for the electromagnetic wave and a reception antenna for the reflected wave, The data processing unit generates direction information based on the measurement data of each of the transmission and reception channels, the direction information indicating the direction of the object. The conversion condition determination unit determines whether the reflected waves in each direction satisfy the conversion condition based on the direction information.

10. The information processing device according to claim 8, wherein: The electromagnetic wave repeatedly includes the chirp signal, The measurement data includes the intermediate signal of each of the chirp signals, The data processing unit generates the distance information for each chirp signal. The data processing unit generates speed information of the object based on changes in the distance information between the chirp signals. The switching condition determination unit determines whether the measurement data satisfies the switching condition based on the speed information.

11. The information processing device according to claim 1, wherein: The information processing device further includes a data processing unit, the data processing unit generating distance information based on the measurement data, generating phase shift information based on the measurement data, and calculating relative orientation determination information with respect to the object based on at least one of the amplitude of the reflected wave, the distance information, and the phase shift information, wherein the distance information indicates the distance between the information processing device and the object. The conversion condition determination unit determines whether the relative direction determination information satisfies a conversion condition for converting at least a part of the measurement data into voice data. The data conversion unit converts at least a part of the measurement data into the voice data when the relative direction determination information satisfies the conversion condition.

12. The information processing device according to claim 11, wherein: The conversion condition determination unit generates notification information for notifying that the conversion condition is not satisfied, when the relative direction determination information does not satisfy the conversion condition.

13. The information processing device according to claim 11, wherein: The conversion condition determination unit generates instruction information indicating a posture that the object should take in order to satisfy the conversion condition, when the relative orientation determination information does not satisfy the conversion condition.

14. The information processing device according to claim 1, wherein: The information processing device further includes a data processing unit that generates distance information and azimuth information based on the reflected wave included in the measurement data, and calculates the relative position of the object based on the distance information and the azimuth information, wherein the distance information indicates the distance between the information processing device and the object, and the azimuth information indicates the azimuth of the object. The conversion condition determination unit generates instruction information for indicating a direction in which the object should move when the measurement data at the relative position does not satisfy the conversion condition. The conversion condition determination unit converts at least a part of the measurement data into the audio data when the measurement data at the relative position satisfies the conversion condition.

15. The information processing device according to any one of claims 1 to 14, wherein: The conversion condition determination unit compares a time waveform represented by the sound data converted from the measurement data with a reference waveform to determine whether the sound data is a biological sound.

16. The information processing device according to any one of claims 1 to 14, wherein: The conversion condition determination unit determines whether the conversion condition is satisfied based on an image of a space in which the electromagnetic wave is irradiated.

17. The information processing device according to claim 9, wherein: The conversion condition determination unit generates a data map indicating whether the audio data exists for each of the directions.

18. The information processing device according to claim 17, wherein: The conversion condition determination unit generates the data maps for different timings, respectively, and generates a combined data map based on the plurality of data maps, the combined data map indicating whether the audio data exists at at least one timing.

19. The information processing device according to any one of claims 1 to 14, wherein: The conversion condition determination unit generates instruction information indicating a position to which the object should move in order to satisfy the conversion condition, when the measurement data does not satisfy the conversion condition.

20. The information processing device according to any one of claims 1 to 14, wherein: The transition condition determination unit generates notification information for notifying that the transition condition is not satisfied, when the measurement data does not satisfy the transition condition.

21. The information processing device according to any one of claims 1 to 14, wherein: The information processing device further includes an identification information acquisition unit that acquires identification information for identifying the object. The data conversion unit records the audio data and the identification information in association with each other.

22. An information processing method, wherein: Acquire measurement data generated based on reflected waves from an object by irradiating an object with electromagnetic waves, determining whether a conversion condition for converting at least a part of the measurement data into audio data is satisfied, When the conversion condition is satisfied, at least a portion of the measurement data is converted into the audio data.

23. A computer program product, comprising a computer program, the computer program being configured to cause a computer to execute the information processing method according to claim 22.

Images