Substance identification device
The substance identification device uses ultrasonic waves with changing frequencies to analyze reflected waveforms, addressing the limitations of existing technologies by accurately identifying object materials and ensuring privacy, even in obstructed views.
Patent Information
- Application Number
- JP2020025739
- Authority / Receiving Office
- JP · JP
- Patent Type
- Patents
- Current Assignee / Owner
- Filing Date
- 2020-02-18
- Publication Date
- 2024-11-14
- Estimated Expiration
- 2040-02-18
AI Technical Summary
Existing material identification technologies, such as those using visible light or laser radars, struggle to accurately determine the material of an object and may pose privacy concerns due to image handling, while ultrasonic methods lack methods for determining material based on reflection coefficients.
A substance identification device using ultrasonic waves with a changing frequency generates and receives reflections, analyzing the waveform of the reflected waves to determine a substance identification index that reflects the object's hardness, density, and thickness, and compares these patterns with known references to identify the material.
Accurately identifies the material of an object by analyzing ultrasonic wave reflections, providing reliable material identification without privacy concerns and enabling detection through obstacles or in challenging environments.
Smart Images

Figure 0007583527000001 
Figure 0007583527000002 
Figure 0007583527000003
Abstract
Description
[Technical field]
[0001] The present invention relates to a material identifying device capable of acquiring information about a material constituting an object, and more particularly to a material identifying device that identifies the degree of hardness of an object. [Background technology]
[0002] A head-mounted device emits a localization ultrasonic pulse from an ultrasonic transmitter and receives the reflection from an obstacle with a two-channel ultrasonic receiver. The device uses descending FM ultrasound or the like as the localization ultrasonic pulse, and stretches the reflected sound by 50 times in time so that it can be heard as an audible sound. Research is being conducted to investigate the recognition characteristics of human echolocation in mimicry of bats (Non-Patent Document 1).
[0003] There is also research that uses a directional ultrasonic transmitter / receiver to obtain the frequency characteristics of the reflection coefficient from the sound reflected from an object, and shows that such characteristics change for plywood and plasterboard walls (Non-Patent Document 2). However, even if the reflection coefficient of the reflected sound changes depending on the object, there is no description of a method for determining the material of the target from the reflected sound. In addition, since the directionality of ultrasonic waves is assumed, there is no disclosure of the characteristics or trends of the reflection coefficient of the reflected sound, including measurements using a non-directional ultrasonic transmitter.
[0004] Recently, there are many technologies that use cameras and laser radars to recognize objects and detect the distance and position of objects ahead (for example, Patent Document 1). However, object recognition devices that use visible light, such as cameras and laser radars, cannot identify the material of an object, and there may be cases where it is not easy to grasp the substance and material of an identified object. Furthermore, for applications where it is important to consider the leakage of personal information, it may be necessary to handle images with care, and there may be cases where use of object recognition devices that use visible light, etc. is undesirable. [Prior art documents] [Non-patent literature]
[0005] [Non-Patent Document 1] Tadayuki Sasaki, Tatsuya Ifukube, "Obstacle Recognition Method Modeled on Bat Echolocation" (January 30, 1987), Noise Research Society Document No. H87-01-7, Hearing Research Society Document No. H-87-7 [Non-Patent Document 2] Masahiro Harazono, Kosuke Nogo, "Measurement of ultrasonic reflection coefficient of flat panels in real environments," Journal of the Acoustical Society of Japan, 72(9), 544550 (2016) [Patent documents]
[0006] [Patent Document 1] JP 2007-187618 A Summary of the Invention
[0007] The present invention has been made in view of the above-mentioned background art, and has an object to provide a material identifying device that can obtain information about a material that constitutes an object by using ultrasound.
[0008] In order to achieve the above-mentioned objective, the material identification device of the present invention comprises an ultrasonic transmitter that generates an ultrasonic transmission wave, an ultrasonic receiver that detects a reflected wave of the ultrasonic wave from the ultrasonic transmitter, and a processing device that analyzes and processes the waveform of the reflected wave, wherein the ultrasonic transmitter generates a transmission wave whose frequency changes according to a predetermined rule, and the processing device determines a material identification index that reflects the degree of hardness of the object based on the waveform of the reflected wave.
[0009] In the above-mentioned material identification device, the ultrasonic transmitter generates a transmission wave whose frequency changes according to a predetermined rule, so that the reflected wave detected by the ultrasonic receiver contains information about the material that constitutes the object that formed the reflected wave (for example, information about the hardness of the object) as a response to various frequencies. The processing device determines a material identification index that reflects the degree of hardness of the object based on the waveform of the reflected wave, and such a material identification index is information about the hardness, etc. of the material that constitutes the object, and serves as a reference for identifying the material of the object.
[0010] According to a specific aspect of the present invention, in the material identifying device, the material identifying indicator is information reflecting at least one of the hardness, density, and thickness of the object. In this case, the material identifying indicator includes information on not only hardness such as the hardness of the object but also density and thickness, and the state of the object can be accurately identified using the material identifying indicator.
[0011] According to yet another aspect of the present invention, the processing device compares the waveform pattern of the reflected wave obtained from the object with a plurality of reference waveform patterns obtained from a plurality of known substances, and identifies the substance type as a substance identification index of the object based on the degree of similarity. In this case, it becomes possible to determine which of the plurality of known substances the object that generated the reflected wave is.
[0012] According to yet another aspect of the present invention, the processing device compares the frequency spectrum of the reflected waves as the waveform patterns of the reflected waves, whereby the reflected waves as responses to various frequencies can be classified in terms of frequency distribution patterns, and the reliability of the material identification indicator can be improved in terms of identifying the material.
[0013] According to yet another aspect of the present invention, the processing device acquires one short-time spectrum having a time width equal to or less than twice the time width of the transmission wave, or acquires multiple short-time spectra having a time width narrower than the time width of the transmission wave at a predetermined time difference, and extracts features common to the multiple short-time spectra. In this case, focusing on the short-time spectrum makes it easier to extract and classify features, and thus simplifies the determination process.
[0014] According to yet another aspect of the present invention, the processing device detects the distance and direction to the object corresponding to the material identification indicator. In this case, it is assumed that some method for identifying the direction of the reflected wave is used. By confirming the distance and direction to the object for which the material identification indicator has been determined, information regarding the arrangement of the object, including information regarding hardness, etc., can be obtained.
[0015] According to yet another aspect of the invention, a processing device detects material identification indicators for at least one of a plurality of objects at overlapping orientations but at different distances, such as to provide information regarding a degree of hardness of an object obscuring a rear object, or to provide information regarding a degree of hardness of an object obscured by a front object.
[0016] According to yet another aspect of the present invention, a processing device detects material identification indicators for an object that transmits ultrasound and an object behind the object.
[0017] According to yet another aspect of the present invention, a plurality of ultrasonic receivers are disposed at a distance from the ultrasonic transmitter, and the processing device extracts reflected waves detected by the plurality of ultrasonic receivers that have a high correlation with the transmitted wave. In this case, noise can be reduced from the reflected waves, and the reliability of the determination of the material identification indicator can be increased.
[0018] According to yet another aspect of the present invention, the processing device determines the spatial arrangement pattern by mapping objects corresponding to the material identifying indicators. In this case, the arrangement state of the materials in the space of interest can be understood as the spatial arrangement pattern.
[0019] According to yet another aspect of the present invention, the processing device tracks changes in spatial layout patterns over time, allowing for the understanding of state changes and dynamic phenomena such as the movement or stationary state of an object of interest, which can be applied to the monitoring of various special events such as accidents, situations requiring assistance, and crime occurrences. [Brief description of the drawings]
[0020] [Figure 1] FIG. 1 is a block diagram illustrating a configuration of a material identifying apparatus according to the present invention. [Diagram 2] FIG. 1 is a conceptual diagram illustrating an example of installation of a material identifying device. [Diagram 3] FIG. 2 is a perspective view illustrating an example of the arrangement of an ultrasonic transmitter and an ultrasonic receiver. [Figure 4]1A and 1B are diagrams illustrating a transmission wave emitted from an acoustic wave transmitter. [Diagram 5] (A) shows an example of a reflected wave received by an ultrasonic receiver, (B) shows a waveform obtained by cross-correlating the reflected wave with the transmitted wave, and (C) shows the specific placement of the target when the reflected wave shown in (A) is obtained. [Figure 6] 1A to 1C are conceptual diagrams illustrating a method for calculating correlation function values. [Figure 7] FIG. 1 is a conceptual diagram illustrating a method for calculating a time spectrum pattern. [Figure 8] (A) to (D) show the time spectrum patterns obtained by conducting experiments in which a transmitted wave was directed at various samples and a reflected wave was detected. [Figure 9] (A) shows the time spectrum pattern obtained by a separate experiment, and (B) shows the pattern obtained by subtracting it from the case of iron. [Figure 10] FIG. 10 is a diagram for explaining the arrangement of ultrasonic transmitting / receiving units and samples in the experiment shown in FIG. [Figure 11] 11(A) to 11(C) are charts showing the results of measuring the time spectrum pattern of a low-density urethane foam. [Figure 12] 13(A) and 13(B) are diagrams for explaining an experiment conducted on the transparency of an ultrasonic transmitting and receiving device. [Figure 13] 10 is a flowchart illustrating an overview of the operation of the material identifying apparatus. [Figure 14] 10 is a flowchart illustrating details of a substance detection operation. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS
[0021] Hereinafter, a material identifying apparatus according to one embodiment of the present invention will be described with reference to the drawings.
[0022] 1 is a block diagram showing a material identifying device according to an embodiment. The material identifying device 100 includes an ultrasonic wave detecting transmitting / receiving device 20 that transmits and receives ultrasonic waves, and a processing device 80 that operates the ultrasonic wave detecting transmitting / receiving device 20 to analyze and process the waveform of a reflected wave RW that is a response to a transmitted wave SW.
[0023] The ultrasonic detection transmitting / receiving device 20 has an ultrasonic transmitter 31, first, second, third and fourth ultrasonic receivers 41, 42, 43 and 44, an amplifier device 50 and a microcomputer 60. The ultrasonic receivers 41 to 44 are arranged at a distance from the ultrasonic transmitter 31 in a predetermined direction, as will be described in detail later. The ultrasonic transmitter 31 and the ultrasonic receivers 41 to 44 are fixed integrally to a support 27 and constitute an ultrasonic transmitting / receiving unit 28.
[0024] As shown in FIG. 2, the ultrasonic wave detection transmitting / receiving device 20 is housed in a case 20a so that the ultrasonic wave transmission / reception unit 28 is exposed, and is installed facing the monitoring target via a mount device or other fixture 20b. In the illustrated case, the ultrasonic wave detection transmitting / receiving device 20 is fixed to the ceiling side of the indoor ID, and is arranged so that a human OB1, furniture, and other non-biological objects OB2, etc. present in the indoor ID can be observed while analyzing them with ultrasonic waves. In the illustrated case, a single ultrasonic wave detection transmitting / receiving device 20 is installed in the indoor ID, but it is also possible to install multiple ultrasonic wave detection transmitting / receiving devices 20 with different operation timings and ultrasonic characteristics, for example. In addition, the orientation and arrangement of the ultrasonic wave detection transmitting / receiving device 20 can be changed by the fixture 20b. It goes without saying that the ultrasonic wave detection transmitting / receiving device 20 can be installed outdoors, not limited to the indoor ID as illustrated.
[0025] Returning to Fig. 1, the ultrasonic transmitter 31 is an ultrasonic speaker with wide directivity. The ultrasonic transmitter 31 is a combination of a voice coil and a piezoelectric element, and converts an electric signal into ultrasonic waves. The ultrasonic transmitter 31 can generate sound waves of, for example, 20 Hz to 100 kHz, but in this embodiment, ultrasonic waves are emitted using a range of about 25 kHz to 100 kHz.
[0026] The first ultrasonic receiver 41 is an ultrasonic microphone with wide directivity. The first ultrasonic receiver 41 is a microphone using MEMS technology, and converts ultrasonic waves into electrical signals. The first ultrasonic receiver 41 may be a conventional capacitor type microphone, or may be one using a piezoelectric element. The second to fourth ultrasonic receivers 42 to 44 have the same structure as the first ultrasonic receiver 41. The first to fourth ultrasonic receivers 41 to 44 can detect sound waves of, for example, 20 Hz to 100 kHz, but in this embodiment, ultrasonic detection is performed using a range of about 25 kHz to 70 kHz.
[0027] The positional relationship between the ultrasonic transmitter 31 and the first to fourth ultrasonic receivers 41 to 44 will be described with reference to FIG. 3. For convenience, the center of the front surface of the ultrasonic transmitter 31 is assumed to be at the origin O of the XYZ coordinate system. The first and second ultrasonic receivers 41 and 42 are arranged at a predetermined distance in the X direction from the ultrasonic transmitter 31, and the third and fourth ultrasonic receivers 43 and 44 are arranged at a predetermined distance in the Y direction from the ultrasonic transmitter 31. The X direction corresponds to the horizontal direction on the paper, but does not correspond to the horizontal direction when the ultrasonic detection transmitting and receiving device 20 is installed at an angle. The X direction corresponds to the vertical direction on the paper, but does not correspond to the vertical direction when the ultrasonic detection transmitting and receiving device 20 is installed at an angle. When considered on the XY plane, the distance to the target OB can be known taking into account the speed of sound from the average value of the reception times ΔT1, ΔT2 of the reflected waves RW1, RW2 from the same target received by the first and second ultrasonic receivers 41, 42 based on the transmission time T0 of the transmission wave SW from the ultrasonic transmitter 31, and the direction of the target OB can be known taking into account the speed of sound from information such as the difference between the reception times ΔT1, ΔT2 and the interval between the first and second ultrasonic receivers 41, 42. When considered on the YZ plane, the distance to the target OB can be known taking into account the speed of sound from the average value of the reception times ΔT3, ΔT4 of the reflected waves RW3, RW4 from the same target received by the third and fourth ultrasonic receivers 43, 44 based on the transmission time T0 of the transmission wave SW from the ultrasonic transmitter 31, and the direction of the target OB can be known taking into account the speed of sound from information such as the difference between the reception times ΔT3, ΔT4 and the interval between the third and fourth ultrasonic receivers 43, 44. In the case of real three-dimensional space, the ultrasonic transmitter 31 and the first ultrasonic receiver 41 are considered as two foci, and a first ellipsoid whose sum of the distances from these two foci corresponds to the distance corresponding to the reception time ΔT1 is considered, and the ultrasonic transmitter 31 and the second ultrasonic receiver 42 are considered as two foci, and a second ellipsoid whose sum of the distances from these two foci corresponds to the distance corresponding to the reception time ΔT2 is considered.The target OB will be located on or near the intersection or circumference of these ellipsoids.Similarly, the ultrasonic transmitter 31 and the third ultrasonic receiver 43 are considered as two foci, and a third ellipsoid whose sum of distances from these two foci corresponds to the distance corresponding to the reception time ΔT3 is considered, and the ultrasonic transmitter 31 and the fourth ultrasonic receiver 44 are considered as two foci, and a fourth ellipsoid whose sum of distances from these two foci corresponds to the distance corresponding to the reception time ΔT4 is considered. The target OB is present on or near the intersection line or circumference of these ellipsoids. As a result, the target OB is present on or near the intersection point of the first to fourth ellipsoids as described above. Note that, when the target OB has a spatial spread, the time width of the reflected waves RW1 to RW4 is wider than the time width of the original transmission wave SW, and the reception times ΔT1 to ΔT4 have a certain width, and this width also provides information on the size or magnitude of the target OB. In the following, for the sake of simplicity, the ultrasonic transmitter 31 and the first and second ultrasonic receivers 41, 42 are sometimes focused on to describe the arrangement detection in the XY plane.
[0028] 1 includes a transmitting amplifier 55 that amplifies an electrical signal input to the ultrasonic transmitter 31 to a desired intensity, and receiving amplifiers 51-54 that amplify electrical signals output from the first to fourth ultrasonic receivers 41-44. The transmitting amplifier 55 is a high-power, high-speed amplifier, and the receiving amplifiers 51-54 are high-speed amplifiers.
[0029] The microcomputer 60 includes a DA converter 65, first to fourth AD converters 61 to 64, a microprocessor 68, and a memory 69. The DA converter 65 receives a command from the microprocessor 68 and sequentially converts a series of digital signals in a predetermined format corresponding to the transmission wave SW into analog signals at corresponding timing. The first to fourth AD converters 61 to 64 sequentially convert a series of analog signals corresponding to the reflected wave RW received from the first to fourth ultrasonic receivers 41 to 44 via the receiving amplifiers 51 to 54 into digital signals in a predetermined format. At this time, the microprocessor 68 temporarily distributes and stores the outputs of the first to fourth AD converters 61 to 64 in the memory 69 with time-related information attached thereto, and outputs the outputs to the processing device 80 at an appropriate timing.
[0030] In the ultrasonic detection transmitting / receiving device 20, a combination of the ultrasonic transmitter 31, the transmitting amplifier 55, and the DA converter 65 is called an ultrasonic transmitting device 25. Also, a combination of the first ultrasonic receiver 41, the receiving amplifier 51, and the first AD converter 61 is called a first ultrasonic receiving device 21, a combination of the second ultrasonic receiver 42, the receiving amplifier 52, and the second AD converter 62 is called a second ultrasonic receiving device 22, a combination of the third ultrasonic receiver 43, the receiving amplifier 53, and the third AD converter 63 is called a third ultrasonic receiving device 23, and a combination of the fourth ultrasonic receiver 44, the receiving amplifier 54, and the fourth AD converter 64 is called a fourth ultrasonic receiving device 24.
[0031] The processing device 80 is, for example, a personal computer, and includes an arithmetic unit 81, a storage device 82, an input / output device 83, a communication device 84, and a display 86. The processing device 80 is connected to the ultrasound detection transmitting / receiving device 20 via a communication cable 91 so as to be capable of digital communication. The processing device 80 is installed at a position away from the indoor ID shown in Fig. 2. In other words, the processing device 80 manages the ultrasound detection transmitting / receiving device 20 remotely.
[0032] The arithmetic unit 81 is a part that controls the operation of the processing device 80, controls the operation of the ultrasonic wave detection transmitting / receiving device 20, and obtains information on the monitoring target using ultrasonic waves. The storage device 82 is a part that stores programs that operate the processing device 80 itself and the ultrasonic wave detection transmitting / receiving device 20, and can also store information obtained on the monitoring target. The input / output device 83 is a part that is operated by an operator of the processing device 80, and can also output information related to the processing results by the arithmetic unit 81. The communication device 84 transmits and receives monitoring information, management information, command information, etc. to and from an external management server through a public communication network such as the Internet or a dedicated communication network that is independently constructed. The display 86 operates under the control of the arithmetic unit 81, and is a part that displays material spatial arrangement information, etc. obtained for the monitoring target, which will be described later.
[0033] The arithmetic device 81 operates the ultrasonic wave detection transmitting / receiving device 20 via the microcomputer 60, and generates a transmission wave SW whose frequency changes according to a predetermined rule (hereinafter, also referred to as FM type ultrasonic wave) SW from the ultrasonic transmitter 31 to the surroundings. The waveform of the transmission wave SW is, for example, one whose frequency increases linearly with time, but is not limited to this, and one whose frequency decreases linearly, one whose frequency changes nonlinearly, etc. can be used. The pattern of the transmission wave SW is stored in the data area 82a of the storage device 82 as ultrasonic transmission data US0. The ultrasonic transmission data US0 is not limited to one corresponding to a single waveform, and multiple waveforms can be prepared in advance, from which waveform data according to the purpose or situation can be selected. The arithmetic device 81 operates the ultrasonic wave detection transmitting / receiving device 20 via the microcomputer 60, and takes in the reflected wave RW as a response to the transmission wave SW as ultrasonic reflection data US1 to US4 for each of the first to fourth ultrasonic receivers 41 to 44, and stores it in the data area 82b of the storage device 82. The arithmetic device 81 extracts waveform data having a high correlation with the transmission wave SW from the ultrasonic reflection data US1 to US4, determines the distance and direction to the object (i.e., spatial arrangement), and stores it in the data area 82c of the storage device 82 as a labeled object, while analyzing the frequency distribution of the waveform data to determine a material identification index reflecting the degree of hardness of the object, and stores this material identification index in the data area 82c as information associated with the labeled object. The information stored in the data area 82c is material spatial arrangement information combining the spatial arrangement and the material identification index for the multiple labeled objects that form the reflected wave RW. That is, the arithmetic device 81 determines the spatial arrangement pattern by mapping the object corresponding to the material identification index. In this case, the arrangement state of the material in the space of interest (actually the arrangement state of the material identification index reflecting the degree of hardness) can be grasped as a spatial arrangement pattern. In this case, the spatial arrangement of the multiple labeled objects may include the spatial size. The calculation device 81 causes the ultrasonic wave detection transmitting / receiving device 20 to periodically emit a transmission wave SW at a predetermined timing, and the material spatial arrangement information is stored in the data area 82c in sequence as chronological monitoring information. The chronological monitoring information is a collection of continuous frame data, similar to a moving image.The computing device 81 can determine the identity of the labeled object between frame data of the monitoring information over time, and if the amount of displacement of objects of similar material or size is less than a predetermined amount, the computing device 81 can label the objects as the same object, and track or trace the movement of the labeled object according to the transition of the frame data, and store such tracking data in the data area 82d as object movement information. At this time, the computing device 81 can classify the importance of the tracked object according to its material identification index or size, and remove it from the monitoring target. In the above, the computing device 81 tracks the change over time in the spatial arrangement pattern of the object characterized by the material identification index stored in the data area 82c. In this case, it is possible to grasp the state change and dynamic phenomenon such as the movement or stationary of the target of interest. If the movement of the tracked object of the monitoring target shows abnormal behavior or moves to an unacceptable place, the computing device 81 can display this on the display 86, and if necessary, can report it to an external management server via the communication device 84. The material identification device 100 can be used to monitor various special events such as accidents, situations requiring assistance, and crime occurrences when installed in areas of public facilities where administrators are not present or in private spaces.
[0034] In the above explanation, a processing device 80 is provided separately from the microcomputer 60 to process the ultrasonic reflection data US1 to US4, obtain material spatial arrangement information, and track labeled objects. However, all or part of these functions of the processing device 80 can be incorporated into the microcomputer 60, and operations can be completed by the microcomputer 60 alone, for example.
[0035] FIG. 4(A) is a diagram for explaining an example of the generation of a transmission wave SW, where the horizontal axis indicates time and the vertical axis indicates frequency. In the illustrated example, a single transmission wave SW has a width of about 1 ms, and such a transmission wave SW is repeated in a 100 ms cycle. A single transmission wave SW is an FM type ultrasonic wave whose frequency decreases, and in the illustrated example, the frequency decreases abruptly but continuously from 80 kHz to 25 kHz. FIG. 4(B) shows the waveform of a single transmission wave SW, and it can be seen that the frequency decreases with the passage of time. Note that the transmission wave SW is not limited to an ultrasonic wave whose frequency decreases gradually as shown in FIG. 4(A) and the like, but may be an ultrasonic wave whose frequency increases gradually, and may have a region in which the frequency is kept constant in the initial time zone or the intermediate time zone. In addition, in FIG. 4(A), the transmission wave SW is repeated in a 100 ms cycle, but the repetition cycle of the transmission wave SW can be appropriately set according to the time until an echo is acquired depending on the size of the monitored object and its space as shown in FIG. 2.
[0036] Fig. 5(A) illustrates the reflected waves RW1, RW2 received by the first and second ultrasonic receivers 41, 42 or the ultrasonic reflection data US1, US2 obtained therefrom, and Fig. 5(B) illustrates a waveform obtained by cross-correlating the reflected waves RW1, RW2 or the ultrasonic reflection data US1, US2 obtained therefrom with the transmission data of the transmission wave SW. Fig. 5(C) illustrates an example of the placement of objects when the reflected waves RW1, RW2 shown in Fig. 5(A) etc. are obtained, that is, an example of an object placed in front of the ultrasonic transmission / reception unit 28 shown in Fig. 3. In this case, the first object PO1 located at the position (0,0.75) is cylindrical with a diameter of 5 mm extending in the Z direction, the second object PO2 located at the position (-0.5,1) is cylindrical with a diameter of 30 mm extending in the Z direction, the third object PO3 located at the position (0.4,1.2) is cylindrical with a diameter of 30 mm extending in the Z direction, and the fourth object PO4 located at the position (0,1.6) is cylindrical with a diameter of 50 mm extending in the Z direction. The cross-correlated waveform shown in FIG. 5(B) shows peaks RP11 to RP4 corresponding to the first to fourth objects PO1 to PO4. In other words, the distance and direction to the first to fourth objects PO1 to PO4 can be known by using the peaks RP11 to RP4.
[0037] As shown in Fig. 5(B), by taking the cross-correlation between the reflected wave RW1 or the reflected wave RW2 and the transmitted wave SW, the reflected waves RW1, RW2 that have a waveform similar to that of the transmitted wave SW are emphasized, and the waveforms different from that of the transmitted wave SW are reduced. In other words, the noise in the reflected waves RW1, RW2 is different from the transmitted wave SW, so it is reduced from the waveform shown in Fig. 5(A) obtained by the correlation function, and the reflected waves RW1, RW2 from objects such as objects PO1 to PO4 are enhanced because the transmitted wave SW is similar.
[0038] FIG. 6(A) is a conceptual diagram for explaining the reflected wave RW or ultrasonic reflection data obtained therefrom, and FIG. 6(B) is a conceptual diagram for explaining the transmitted wave SW or ultrasonic transmission data corresponding thereto. As shown by the dotted line, the reflected wave RW is sampled at regular intervals and converted into digital data (specifically, any of the ultrasonic reflection data US1 to US4). The sampling period is, for example, 1 μs, but is not limited to this. The sampling period of the transmitted wave SW coincides with that of the reflected wave RW at the stage of digital data (specifically, ultrasonic transmission data US0) before being converted into analog. When taking the cross-correlation between the reflected wave RW and the transmitted wave SW, in the first target section ID0 (corresponding to the time width IS0 of the transmitted wave SW) of the reflected wave RW, N multiplications are performed on the sample value of the reflected wave RW at each time point in the section ID0 and the sample value of the transmitted wave SW at each time point, and the sum of all the N multiplication values is normalized by the square root of the sum of the squares of each waveform, which is taken as the correlation function value at time t0. Next, as shown in Fig. 6(C), the transmitted wave SW is shifted by one point, and the sample value of the reflected wave RW at each time point in section ID1 is multiplied by the sample value of the transmitted wave SW at each time point N times, and the sum of all N multiplication values is normalized by the square root of the sum of squares of each waveform to obtain the correlation function value at time t1. This process is repeated to obtain correlation function values one after another, and when these values are plotted on the time axis, a correlation function waveform is obtained. The above is an example of a calculation to obtain the cross-correlation between the reflected wave RW and the transmitted wave SW, and the correlation can be obtained by various other calculation methods.
[0039] In the above, as shown in FIG. 5(B), when there are not many objects PO1-PO4 or when the objects PO1-PO4 are relatively far apart, the distance and direction to the first to fourth objects PO1-PO4 can be known. However, when the objects to be observed are close or there are many objects, interference may occur. Therefore, correlation waveforms are used to calculate the distance and direction. For example, when the receivers are on the left and right, the distance and direction to the cylinder, which is an object, are calculated from the correlation waveform (RP1R) between the transmitted wave SW and the right received wave (RW1R) and the correlation waveform (RP1L) between the transmitted wave SW and the left received wave (RW1L). Specifically, the distance from the object is calculated from the average value ((TR+TL) / 2) of the time difference (TR) from the transmitted wave SW to the right correlated waveform RP1R and the time difference (TL) from the transmitted wave SW to the left correlated waveform RP1L, and the direction is calculated from the difference (ΔT) between the time difference (TR) to the right and the time difference (TL) to the left. In other words, if TR and TL are large, the distance from the transmitter to the cylinder will be farther, and if the difference ΔT between TR and TL is the same, the cylinder will be in the forward direction, and the larger the difference ΔT, the further away from the forward direction it will be.
[0040] A method for analyzing the frequency distribution of the waveform data of the reflected wave RW will be described below. For a data region extracted in which the reflected wave RW1 has a correlation with the transmitted wave SW, an FFT (Fast Fourier Transform) is performed on the waveform data of interest in this data region to analyze the Fourier transform spectrum, i.e., the frequency distribution. It is also possible to perform an FFT on the entire region of the data of the reflected wave RW, which is the received wave. The analysis of the frequency distribution is not limited to FFT, and various methods such as wavelet transform can be used.
[0041] The waveform shown in FIG. 7(A) is assumed to be the reflected wave RW. For example, if the reflected wave RW is sampled over a period of 100 ms at a period of 1 μs immediately after the emission of the transmission wave SW, 100×1000 points, i.e., 100,000 sample values, are obtained. From among these sample values, for example, the first 100 sample values (shown as frame length in the figure) are subjected to FFT to obtain a spectrum SP1 (the leftmost side of FIG. 7(B)), then 50 points (shown as frame period in the figure) corresponding to a time shorter than the frame length are shifted, and the spectrum SP1 is obtained from the next 100 sample values by FFT (the next left side of FIG. 7(B)), then another 50 points are shifted, and the spectrum is obtained from the next 100 sample values by FFT, and so on. Here, the specific frame period is, for example, 50 μs, and the specific frame length is, for example, 100 μs=0.1 ms. When a spectrum is obtained by the above method, 100,000 sample values are shifted by 50 points each, so 2,000 short-time spectra are obtained. Such short-time spectra are hereinafter called short-time spectra. A collection of short-time spectra can be displayed as a three-dimensional diagram with the time axis as the horizontal axis, frequency as the depth axis from the front, and power (intensity) as the vertical axis, but by expressing power (intensity) as "density", it can be displayed as a two-dimensional pattern just like a "voiceprint", and such a pattern with visible density is called a density-type time spectrum pattern. In such a time spectrum pattern, the areas with high density are predicted to be reflected sound from an object, and the predicted reflected wave RW with a length of about 1 ms or more has a total of 20 or more short-time spectra obtained every 50 μs over a 0.1 ms period. Among these, the areas with high density are considered to be the reflected wave RW, and the short-time spectrum pattern there is used as the pattern matching data. Such short-time spectra or their averages are also called local time spectrum patterns or simply spectrum patterns. Calculating the entire time spectrum of the received wave would require a large amount of calculations, so for simplicity's sake, the waveform with high correlation (waveform of interest) is regarded as the reflected wave RW, and only its time spectrum is used as the pattern matching data.Focusing on the short-time spectrum makes it easier to extract features and classify them, and simplifies the determination process.
[0042] A single short-time spectrum can be used as the time spectrum pattern. For example, the point where the correlation function is maximized is calculated, and the waveform with the length of the transmission wave SW starting from that point is set as the waveform of interest. A frequency spectrum is analyzed or calculated in a range of 1 to 2 times the length of the transmission wave SW centered on the waveform of interest, and such a frequency spectrum can be used as a single short-time spectrum.
[0043] The short-time spectrum has a time width narrower than that of the transmitted wave SW. For short-time spectra, if the frame length (sample width) is short, the precision of the spectrum (more precisely, the frequency resolution) decreases, and there is a tendency for material identification to become difficult. Conversely, if the frame length (sample width) is long, the time resolution decreases, but the frequency resolution increases. In reality, the reflected wave RW is also about 1 ms, and if the sample width is made wider than that, signals other than the reflected wave will also be picked up, so the frame length (sample width) is set to 2 ms or less.
[0044] 8(A) to 8(D) show frequency spectra, i.e., time spectrum patterns, of reflected waves RW obtained by performing experiments in which the transmitted waves SW are applied to various samples and the reflected waves RW are detected. In order to obtain the time spectrum patterns shown in Figs. 8(A) to 8(D), plates of various materials, each measuring 25 cm x 30 cm, were placed 60 cm in front of the ultrasonic transmission / reception unit 28 shown in Fig. 3 (omitting the third and fourth ultrasonic receivers 43 and 44) with the unit facing horizontally. Fig. 8(A) shows the case of a 1 mm thick iron plate, Fig. 8(B) shows the case of a 2.3 mm thick styrene plate, Fig. 8(C) shows the case of a 5 mm thick lauan plywood, and Fig. 8(D) shows the case of a 5 mm thick urethane sponge. Note that Figs. 8(A) to 8(D) are arranged in order from hard to soft materials. As is clear from Figs. 8(A) to 8(D), it can be seen that the frequency peak patterns formed around 50 kHz differ for each material. In the steel plate of Fig. 8(A), there is a maximum peak at a frequency of 50kHz, and relatively large peaks around 30kHz and 70kHz. In the styrene plate of Fig. 8(B), in addition to a large peak around 50kHz, there is a maximum peak around 70kHz, and a relatively large peak below that around 30kHz. In the lauan plywood of Fig. 8(C), there is no peak around 50kHz, and the peak is flat from 30kHz to 70kHz. In the urethane sponge of Fig. 8(D), the frequency is weakened around 50kHz, and there is no noticeable peak overall in the frequency range from 30kHz to 70kHz, but large fluctuations in amplitude are observed.
[0045] FIG. 9(A) shows time spectrum patterns obtained by performing another experiment on various samples. FIG. 10 is a diagram for explaining the arrangement of the ultrasonic transmission / reception unit and the samples in the experiment shown in FIG. 9(A). The transmission wave SW output from the wave transmission / reception unit 28 was lowered in frequency from 80 kHz to 20 kHz in the pattern shown in FIG. 4(A). The wave transmission / reception unit 28 was set on a cylindrical support SM installed on a floor surface FL, and the ultrasonic transmission / reception unit 28 (without the third and fourth ultrasonic receivers 43 and 44) was placed above the sample SA. The distance from the ultrasonic transmission / reception unit 28 to the sample SA was 1 m, and the size of the sample SA was 20 cm x 30 cm. The thickness of the sample SA was 1 to 3 mm. Specifically, the samples SA were iron plate, plywood, vinyl, elastomer, styrene board, and plain rubber, and the hardness values were 100, 96.85, 85, 72, and 25, respectively. In Fig. 9(A), it can be seen that peak waveforms showing the difference in hardness are obtained in the frequency ranges shown as changes 1 to 6. For example, in the 25-30 kHz range, the level decreases for soft materials such as styrene board and plain rubber (change 1), while the level of soft materials such as vinyl increases above 35 kHz (changes 2 to 6). The level decreases in the 40-50 kHz range for hard materials such as plywood (between changes 3 and 4), but such a decrease is not observed for steel plates. Although the elastomer is buried in the figure and difficult to see, it is at a level that is intermediate between hard and soft. Fig. 9(B) is a spectrum pattern showing the difference with the spectrum of the steel plate, and it can be seen that the level increases and decreases depending on the frequency for soft and hard materials. However, the material should not be judged only by the local increase or decrease in the level in a specific frequency range, but by the overall difference in the increase and decrease pattern (frequency characteristics).
[0046] The above explanation is merely an example, and by conducting an experiment in which a transmitted wave SW is applied to samples SA of various sizes and materials other than those mentioned above and the reflected wave RW is detected, the time spectrum patterns of the reflected wave RW obtained are stored, and then analyzed and learned in advance, so that the processing device 80 can determine various information such as hardness and size of various targets OB.
[0047] 11(A) to 11(C) show the reflected sound spectrum from a 2 cm to 10 cm thick urethane foam (50 cm x 50 cm) placed 1 m in front of the ultrasonic transmitter 31. FIG. 11(A) shows the time spectrum pattern of a 2 cm thick urethane foam, FIG. 11(B) shows the time spectrum pattern of a 5 cm thick urethane foam, and FIG. 11(C) shows the time spectrum pattern of a 10 cm thick urethane foam. In general, the ultrasonic waves penetrated the urethane foam and the reflected sound was small, but the 46 to 47 kHz component weakened as the thickness increased. From this, it is inferred that the urethane foam used in this experiment has low density, so the emitted sound is generally transmitted, but absorption occurs around 46 to 47 kHz. In this way, it is inferred that the focus point differs depending on the material, and conversely, the difference in the material is predicted from the change in the focus point.
[0048] As is clear from the above, if the time spectrum patterns of the reflected wave RW are measured in advance for many predicted materials and registered as templates in the storage device 82, the material or material of the object or object OB present in front of the ultrasonic transmission / reception unit 28 can be estimated from the similarity of the time spectrum pattern of the reflected wave RW to the many registered templates. The material or material determined in this manner is a material identification index reflecting the degree of hardness of the object OB. In other words, in the waveform pattern of the reflected wave RW, a local time spectrum pattern (target waveform pattern) obtained for the object or object OB is compared with multiple templates (reference waveform patterns) obtained for multiple known types of materials, and the material type is identified as a material identification index of the object or object OB based on the high degree of similarity. The similarity or similarity of the local time spectrum patterns as described above can be determined by correlation and other various statistical methods. Furthermore, the similarity of the local time spectrum patterns as described above can be determined by using various machine learning methods such as deep learning that constructs a multilayered neural network, and pattern identification using AI is also possible.
[0049] The material identification index is not limited to identifying the material or material of the target OB. The material identification index may be something like hardness indicating the degree of hardness. The material identification index may be a hardness classification in which hardness is divided into multiple stages. For example, it is possible to classify hard plate-like objects such as (1) iron plates and plywood, (2) transparent vinyl boards and transparent vinyl boards, and (3) soft plate-like objects such as styrene boards and rubber. It has been experimentally confirmed that these classifications (1) to (3) have clearly different slopes of the entire time spectrum pattern (ratio of intensity at low frequency (around 30 kHz) to intensity at high frequency (50 to 60 kHz)). The material identification index may be something that distinguishes between categories of materials. For example, it may be possible to distinguish between metal solids, ceramic solids, organic solids, etc. From the viewpoint that organic solids include biological materials such as meat, it is considered that it is also possible to distinguish between living organisms and objects harder than them. The material identification index may include information on density such as porosity and fiber. The transmitted components are clearly larger for low density materials such as sponge and fiber, and the time spectrum pattern is significantly different from that of a high density material such as a rubber plate.
[0050] Although detailed explanation is omitted, it is believed that as the thickness of the target object or target OB increases, a downward shift occurs in which the power of the time spectrum pattern of the reflected wave RW decreases overall, and by calibrating the power value based on the distance to the target OB, it becomes possible to estimate the thickness of the target OB.
[0051] 12(A) and 12(B) are diagrams for explaining an experiment conducted on the see-through property of the ultrasonic wave detecting transmitting / receiving device 20. FIG. 12(A) shows a concentration-type time spectrum pattern when a glass plate is placed at a position 100 cm in front of the ultrasonic wave transmitting / receiving unit 28, and FIG. 12(B) shows a concentration-type time spectrum pattern when a glass plate is placed at a position 100 cm in front of the ultrasonic wave transmitting / receiving unit 28 and a thin furoshiki cloth is placed at a position 70 cm in front of the ultrasonic wave transmitting / receiving unit 28 so as to cover the glass plate. As is clear from FIG. 12(B), it can be seen that the presence of the furoshiki cloth and the glass plate can be detected simultaneously. From the above, it can be said that the material identifying device 100 of the embodiment can detect objects and people that are hidden by a partition or curtain such as a thin curtain and cannot be visually seen. Furthermore, the material identifying device 100 of the embodiment can detect materials such as glass and mirrors that are visually transparent and difficult to see.
[0052] By utilizing the above phenomenon, the processing device 80 can detect a material identification indicator for at least one of multiple targets that have overlapping orientations but different distances, and can detect the material identification indicators of the target through which the ultrasonic waves pass and the target behind it by comparing the transmitted waves SW and the reflected waves RW. Specifically, it is possible to detect not only the presence of a glass surface behind the furoshiki as described above, but also the presence of a hard object behind the urethane foam, and to detect differences in the thickness of the urethane foam by the reflected waves from a hard object behind the urethane foam.
[0053] Although a detailed description will be omitted, fine particles such as fog and smoke do not fundamentally block ultrasonic waves, so the material identification device 100 of the embodiment can be said to be capable of detecting objects or people in places that are difficult to see due to fog or smoke.
[0054] As described above, the material identifying device 100 of the embodiment does not perform object detection using the visible appearance of a material, and therefore can identify objects with the same pattern but different materials as different objects. In other words, when the material identifying device 100 of the embodiment is mounted on a moving body, for example, when the moving body moves toward an object ahead, it is possible to easily predict in advance the impact of a collision with the object ahead.
[0055] The material identifying apparatus 100 of the embodiment performs measurement using ultrasonic waves, and therefore, from the viewpoint of observing human movements in private spaces such as private rooms and medical facilities, it can be said to be highly reliable with respect to the leakage of personal information and the like.
[0056] Hereinafter, the operation of the material identifying device 100 will be described with reference to Fig. 13. The processing device 80 obtains information on the arrangement and material of one or more objects OB arranged in front of the ultrasonic transmitting / receiving unit 28 from data obtained using the ultrasonic detecting transmitting / receiving device 20 (step S1). That is, the processing device 80 obtains material spatial arrangement information.
[0057] Next, the processing device 80 refers to the storage device 82, and determines the identity of the target OB identified by the material spatial arrangement information while comparing it with past data, labels those determined to be the same object, and tracks or traces the movement of the labeled object or target OB (step S2).
[0058] The processing device 80 judges whether or not a notification event that requires an alarm or other notification has occurred, such as whether or not the movement or stop of the labeled object or target OB is abnormal, or whether or not the labeled object or target OB is moving to an unacceptable place (step S3). In a hospital or care facility, for example, abnormal behavior means a fall of a sick person or a person receiving care, and moving to an unacceptable place corresponds to, for example, the movement of an intruder from outside who has slipped through other security check mechanisms.
[0059] If the processing device 80 determines that the movement of the labeled object or target OB is behaving abnormally, or that the labeled object or target OB is moving to an unacceptable location, it displays an alarm on the display 86, etc., and reports the abnormal situation to an external management server via the communication device 84 (step S4).
[0060] The operation of step S1 in FIG. 13 will be described in detail below with reference to FIG. 14. The processing device 80 operates the ultrasonic wave detection transmitting / receiving device 20, generates a transmission wave SW, which is an FM type ultrasonic wave, from the ultrasonic transmitter 31 to the surroundings, and captures a reflected wave RW as a response to the transmission wave SW as ultrasonic reflection data for each of the first to fourth ultrasonic receivers 41 to 44 (step S11). After that, the processing device 80 performs a correlation process to extract waveform data that is correlated with the transmission wave SW from the ultrasonic reflection data (step S12). Next, the processing device 80 determines the distance and direction (i.e., spatial arrangement) to the target from the waveform data that is determined to be correlated in step S12, and stores it in the storage device 82 (step S13). Meanwhile, the processing device 80 analyzes the frequency distribution from the waveform data that is determined to be correlated in step S12 using a method such as FFT to obtain a spectrum pattern (step S14). Next, the processing device 80 performs pattern matching for the object whose spectral pattern was obtained in step S14, comparing it with the registered template stored in the storage device 82 (step S15), and determines a material identification index reflecting the degree of hardness of the object, such as estimating the material of the object OB and estimating the hardness level of the object OB (step S16). Finally, the processing device 80 performs integration by combining the spatial arrangement and the material identification index while labeling one or more objects OB corresponding to the reflected wave RW, to configure material spatial arrangement information (step S17).
[0061] As is clear from the above description, in the material identification device 100 of the above embodiment, the ultrasonic transmitter 31 generates a transmission wave SW whose frequency changes according to a predetermined rule, so that the reflected wave RW detected by the ultrasonic receivers 41-44 contains information about the material constituting the target OB that formed the reflected wave RW (for example, information about the hardness of the target OB) as a response to various frequencies. The processing device 80 determines a material identification index reflecting the degree of hardness of the target OB based on the waveform of the reflected wave RW, and such a material identification index is information about the hardness, etc. of the material constituting the target OB and serves as a reference for identifying the material of the target OB.
[0062] The present invention is not limited to the above embodiment, and can be implemented in various forms without departing from the spirit of the present invention. For example, the arrangement of the four ultrasonic receivers 41-44 shown in Fig. 3 is merely an example, and the distance and direction from the ultrasonic transmitter 31 can be changed as appropriate. Also, the spatial arrangement of the target OB can be determined by three ultrasonic receivers without providing four ultrasonic receivers 41-44.
[0063] The ultrasonic transmitter 31 may be directional, and the ultrasonic receivers 41 to 44 may also be independently directional. In this case, the direction of the target OB can be determined by moving the ultrasonic transmitter 31 and the ultrasonic receiver 41 in a scanning manner. [Explanation of symbols]
[0064] OB... object, OB1... human, OB2... object, PO1-PO4... object, RW... reflected wave, RW1-RW4... reflected wave, SW... transmitted wave, 1-4AD... number, 41-44... ultrasonic receiver, 51-54... receiving amplifier, 61-64... AD converter, 20... ultrasonic detection transmitting / receiving device, 21-24... ultrasonic receiving device, 25... ultrasonic transmitting device, 27... support, 28... ultrasonic transmitting / receiving unit, 31... ultrasonic transmitter, 41-44... ultrasonic receiver, 50... amplifier device, 60... microcomputer, 68... microprocessor, 69... memory, 80... processing device, 81... arithmetic unit, 82... storage device, 83... input / output device, 84... communication device, 86... display, 91... communication cable, 100... material identification device
Claims
1. an ultrasonic transmitter that generates an ultrasonic transmission wave; an ultrasonic receiver that detects a reflected wave of the ultrasonic wave from the ultrasonic transmitter; a processor that analyzes and processes the waveform of the reflected wave; The ultrasonic transmitter periodically generates the transmission wave whose frequency changes according to a predetermined rule, The processing device determines a material identification index reflecting a degree of hardness of the object based on a waveform of the reflected wave; The processing device acquires one short-time spectrum having a time width equal to or less than twice the time width of the transmission wave, or acquires a plurality of short-time spectra having a time width narrower than the time width of the transmission wave at a predetermined time difference, and extracts a feature common to the plurality of short-time spectra. Material identification device.
2. The material identification device according to claim 1 , wherein the processing device grasps a change over time in a spatial arrangement pattern of objects characterized by the material identification indicators and sizes corresponding to the common features.
3. 3. The material identification device according to claim 1, wherein the material identification index is information reflecting at least one of the hardness of the object, the denseness, and the thickness.
4. The material identification device according to any one of claims 1 to 3, wherein the processing device compares, with respect to the waveform pattern of the reflected wave, a target waveform pattern obtained for the target with a plurality of reference waveform patterns obtained for a plurality of known types of substances, and identifies a substance type as the material identification index of the target based on a high degree of similarity.
5. The material identifying device according to claim 4 , wherein the processing device compares a frequency spectrum of the reflected wave as a waveform pattern of the reflected wave.
6. 6. The material identifying device according to claim 1, wherein the processing device detects a distance and a direction to an object corresponding to the material identifying indicator.
7. The material identifying device according to claim 6 , wherein the processing device detects the material identifying indicator for at least one of a plurality of targets that have overlapping orientations but different distances.
8. 8. The material identifying device according to claim 7, wherein the processing device detects the material identifying indicator for an object through which ultrasonic waves pass and an object behind the object.
9. The ultrasonic receiver is disposed at a distance from the ultrasonic transmitter.
9. The material identifying device according to claim 6, wherein the processing device extracts reflected waves having a high correlation with the transmitted wave from the plurality of reflected waves detected by the plurality of ultrasonic receivers.
10. The material identifying device according to any one of claims 6 to 9, wherein the processing device determines a spatial arrangement pattern by mapping an object corresponding to the material identifying indicator.
11. The material identification apparatus of claim 10 , wherein the processing device tracks the movement of the spatial arrangement pattern.
Citation Information
Patent Citations
Ultrasonic sensor device
JP1988234107A
personal object detector
JP2000509493A
Object identifying device
JP2007187618A
Object discrimination device
JP2008058059A
Device and method for determining proximity of obstacle
JP2008292168A
Cited By
Ultrasound detection system and method
JP2025507090A
