Multi-channel artificial basilar membrane sensor system

Abstract

The present invention relates to a cochlear-mimicking multi-channel artificial basilar membrane sensor system that uses a design formula of a model for establishing a frequency band of an artificial basilar membrane sensor, and additionally combines with an artificial hair cell model for mimicking the signal detection and processing process of a biological cochlear, wherein, in order to separate and detect sound by frequency, the system comprises: an artificial basilar membrane of a uniform thickness having a width of a shape mathematically designed such that the characteristic frequencies thereof are uniformly distributed along the length; and a plurality of sensing units installed on the artificial basilar membrane for converting vibration into electrical signals.

Description

Multi-channel artificial basilar membrane sensor system

[0001] The present invention relates to a multi-channel artificial basilar membrane sensor system, and more particularly, to a cochlear-mimicking multi-channel artificial basilar membrane sensor system that utilizes a design formula of a model for designing a frequency band of an artificial basilar membrane sensor and additionally combines an artificial hair cell model for imitating a signal detection and processing process of a living cochlea.

[0002] Sound vibrates the basilar membrane of the cochlea, which is then converted into neural signals by the hair cells of the organ of Corti, which lie above the basilar membrane. In this way, the human cochlea detects sound through a mechanical-electrical conversion process.

[0003] The biological basilar membrane is a three-dimensional trapezoidal shape with a width of 100-500 μm, a thickness of 1.16-0.55 μm, and a length of 34 mm, and its stiffness varies depending on the location. As a result, the characteristic frequency of the basilar membrane varies exponentially with location, and when expressed on a logarithmic scale, it has linearity. The human cochlea has 24 critical bands divided into approximately 1.3 mm intervals according to the structural characteristics of the basilar membrane. Each critical band has an independent frequency band that can perceive sound, and these 24 frequency bands combined constitute the audible frequency range of 20 to 20,000 Hz.

[0004] Hair cells are also distributed in arrays of approximately 3,500 along the organ of Corti in the cochlea. Hair cells are divided into inner and outer hair cells. Inner hair cells generate neural signals as the basilar membrane vibrates, while outer hair cells amplify these signals. The neural signals generated by hair cells can be expressed using a sigmoid function, and the frequency selectivity of the basilar membrane is increased through bandpass filtering and normalization of the signals.

[0005] The present invention proposes a multi-channel artificial basilar membrane sensor system that combines an artificial basilar membrane, which is designed by utilizing structural variables and properties of the basilar membrane and the frequency band of the sensor, and a multi-channel artificial basilar membrane sensor that detects independent frequency bands depending on the location of the basilar membrane, and a signal processing model for normalization and band-filtering of additionally detected signals.

[0006] In addition, the present invention proposes a method for extracting and recognizing sound features by directly using signals of the above system.

[0007] In addition, the present invention can be usefully utilized to selectively detect only a desired voice signal in a noisy environment.

[0008] In order to solve the above-described problem, a multi-channel artificial basilar membrane sensor system according to one embodiment of the present invention comprises: an artificial basilar membrane having a width of a shape mathematically designed to have a constant thickness and a characteristic frequency uniformly distributed along the length in order to detect sounds by separating them according to frequency; and a plurality of sensing units installed in the artificial basilar membrane for converting vibrations into electrical signals.

[0009] According to one embodiment of the present invention, the outer portion of the artificial basement membrane is fixed and the inner portion is supported so as to be able to vibrate freely.

[0010] In addition, according to one embodiment of the present invention, the artificial basement membrane is characterized by having a width designed using the <mathematical formula> below according to structural variables, material properties, and target frequency band.

[0011] <Mathematical formula>

[0012]

[0013] Here, here, f L (or f low ) represents the lowest resonant frequency of the artificial basilar membrane, and f H (or f high) represents the maximum resonant frequency of the artificial basilar membrane, L represents the length of the artificial basilar membrane, and f CF (x) represents the resonant frequency according to the distance of the artificial basilar membrane, and ω(x) represents the width according to the distance of the artificial basilar membrane.

[0014] Additionally, according to one embodiment of the present invention, the structural variable is characterized in that it is a thickness of the artificial basement membrane.

[0015] Additionally, according to one embodiment of the present invention, the material properties are characterized by the density and modulus of the artificial basement membrane.

[0016] In addition, according to one embodiment of the present invention, the sensing part of the artificial basement membrane is characterized by being made of a piezoelectric material.

[0017] In addition, according to one embodiment of the present invention, it is characterized in that a signal processing model of artificial hair cells calculated by imitating biological hair cells is used to process electric signals generated from each sensing unit of the artificial basement membrane.

[0018] In addition, according to one embodiment of the present invention, the signal processing model is characterized in that it normalizes and band-filters the electric signal of the artificial basilar membrane detection unit by applying a sigmoid function as in the <mathematical formula> below.

[0019] <Mathematical formula>

[0020]

[0021] Here, V AHC,n is the output signal of channel n of the AHC model, and v ABM,max is the maximum output of the ABM sensor, and v ABM,n is the output signal of channel n of the ABM sensor, and α n and β n is the slope determination constant of channel n, and v ref,n is the maximum slope position of the nth channel, and can be expressed as the equation below in the above <Mathematical Formula 3>, and v ABM,max,nis the maximum output of channel n of the ABM sensor, and v ABM,min,n is the minimum output of channel n of the ABM sensor, and c is the position determination constant of the maximum slope of channel n.

[0022] Meanwhile, a multi-channel artificial basilar membrane sensor system according to one embodiment of the present invention is characterized in that it can detect sounds as electrical signals by dividing them into frequency bands by combining an artificial basilar membrane and artificial hair cells, and process the signals in real time without separate calculation.

[0023] In addition, according to one embodiment of the present invention, an artificial intelligence algorithm part is added based on a signal-processed electrical signal obtained from the artificial hair cell.

[0024] In addition, according to one embodiment of the present invention, the artificial intelligence algorithm is characterized in that at least one of a convolutional neural network (CNN), a recurrent neural network (RNN), an artificial neural network (ANN), a deep neural network (DNN), a Gaussian mixture model (GMM), a hidden Markov model (HMM), and a long short-term memory (LSTM) is used.

[0025] In addition, according to one embodiment of the present invention, as one of the feature extraction methods of the sound signal, a heat map image composed of a horizontal axis representing time information, a vertical axis representing the channel number of the sensing unit, and a color representing the size of the electric signal is used.

[0026] According to the present invention, a sensor manufactured according to the design formula of a multi-channel artificial basilar membrane sensor system similarly mimics the distribution of characteristic frequencies and critical bands and frequency selectivity according to the location of a biological basilar membrane.

[0027] In addition, according to the present invention, a sensor manufactured according to the design formula of a multi-channel artificial basilar membrane sensor system has a characteristic frequency that increases exponentially according to location.

[0028] In addition, according to the present invention, a sensor manufactured according to the design formula of an artificial basilar membrane sensor system has a wider threshold band that can be detected as it moves toward a detection section of a higher frequency band.

[0029] In addition, according to the present invention, the electrical signal of the sensor processed according to the signal processing model of the multi-channel artificial basilar membrane sensor system identically mimics the signal processing method and function of the neural signal of the biological hair cells.

[0030] In addition, according to the present invention, the electrical signal of the sensor processed according to the signal processing model of the multi-channel artificial basilar membrane sensor system is normalized by correcting the output difference according to the position.

[0031] In addition, according to the present invention, the electrical signal of the sensor processed according to the signal processing model of the multi-channel artificial basilar membrane sensor system amplifies only the output within the critical band and attenuates other signals, thereby acting as a band filter.

[0032] In addition, according to the present invention, sound features can be detected in the time-frequency domain without a frequency analysis process by using a signal processed according to a signal processing model of a multi-channel artificial basilar membrane sensor system.

[0033] In addition, according to the present invention, the detected sound characteristics of the multi-channel artificial basilar membrane sensor system are learned through an artificial intelligence algorithm, and sound information can be distinguished and recognized.

[0034] FIG. 1 is a schematic diagram showing the configuration of a multi-channel artificial basement membrane sensor system according to one embodiment of the present invention.

[0035] Figure 2 is a real photograph of an artificial basilar membrane sensor manufactured according to a design formula of one embodiment of the present invention.

[0036] FIG. 3 is a drawing of an artificial basement membrane for explaining the design formula and variables of the artificial basement membrane according to one embodiment of the present invention.

[0037] FIG. 4 is a graph for explaining the sigmoid function and variables of an artificial hair cell according to one embodiment of the present invention.

[0038] Figure 5 is a graph comparing simulation results of a conventional artificial basilar membrane and an artificial basilar membrane modeled according to a design formula of one embodiment of the present invention.

[0039] Figure 6 is a graph showing the vibration displacement measurement results of an artificial basilar membrane sensor manufactured according to a design formula of one embodiment of the present invention.

[0040] FIG. 7 is a graph showing a sigmoid function of each detection unit according to a signal processing model of one embodiment of the present invention and a graph showing a change in sensor output according to signal processing.

[0041] Figure 8 is a graph showing an electrical signal by applying a signal processing model and an electrical signal for each sensing unit of an artificial basement membrane sensor manufactured according to one embodiment of the present invention.

[0042] FIG. 9 is a real-time spectrogram that expresses the electrical signals of each sensing unit of an artificial basement membrane sensor manufactured according to one embodiment of the present invention as a heat map image, thereby eliminating the need for a separate STFT process.

[0043] Hereinafter, a multi-channel artificial basilar membrane sensor system according to a preferred embodiment will be described in detail with reference to the attached drawings. In this drawing, the same reference numerals will be used for identical components, and repetitive descriptions or detailed descriptions of well-known functions and components that may unnecessarily obscure the gist of the invention will be omitted. The embodiments of the invention are provided to more fully explain the present invention to those of ordinary skill in the art. Therefore, the shapes and sizes of elements in the drawings may be exaggerated for clearer description.

[0044] Conventional techniques have typically fabricated artificial basilar membrane sensors in a trapezoidal shape with a constant thickness for ease of fabrication. However, these simple structures fail to adequately reflect the complex structure of the biological basilar membrane and the variations in stiffness and characteristic frequency caused by thickness variations due to different cells. Consequently, the critical bands along the longitudinal axis of the trapezoidal basilar membrane are not properly separated, and resonant frequencies overlap. This simple basilar membrane shape makes it difficult to form a wide resonant frequency range.

[0045] Furthermore, since the magnitude of stiffness varies depending on the location of the basilar membrane, differences in vibration displacement occur. This means that the sensitivity of the electrical signal is not constant depending on the location of the artificial basilar membrane sensor. Since the electrical signal in the high frequency band appears weaker than the low frequency band, it is difficult to integrate and use the electrical signals detected at each location. Previous studies have also detected sounds by separating them into frequencies to some extent according to characteristic frequencies, but due to the above-mentioned problems, a separate additional computational process such as the Short Time Fourier Transform (STFT) is required to analyze the characteristics of the sound.

[0046] To solve the above problem, the present invention proposes an optimized design formula that reflects the change in stiffness due to the change in thickness of an artificial basement membrane into the change in width so as to mimic the location and distribution of characteristic frequencies of a biological basement membrane.

[0047] To achieve this, in one embodiment of the present invention, a multi-channel artificial basilar membrane sensor is proposed that mimics a critical band and detects independent frequency bands at each location of the sensor.

[0048] Furthermore, we propose a signal processing model that mimics the signal processing of hair cells to ensure that the signal amplitude is similar to the difference in vibration displacement across channels of a multi-channel artificial basilar membrane sensor. The signal processing model normalizes the electrical signal amplitude of each channel, and signals within the critical band have a band-filtering function through nonlinear amplification. This has the advantage of effectively detecting speech signals, as it isolates and amplifies sounds in the speech band, especially in environments with ambient noise with frequency components below 200 Hz.

[0049] In addition, the present invention proposes a method for recognizing and distinguishing sound features without a separate frequency analysis process by converting a signal of an artificial basilar membrane sensor system into a heat map image and using the same as learning data for a convolutional neural network algorithm, as one embodiment.

[0050] FIG. 1 is a schematic diagram showing the configuration of a multi-channel artificial basement membrane sensor system according to one embodiment of the present invention, FIG. 2 is a real photograph of an artificial basement membrane sensor manufactured according to a design formula of one embodiment of the present invention, and FIG. 3 is a diagram of an artificial basement membrane for explaining the design formula and variables of the artificial basement membrane according to one embodiment of the present invention.

[0051] Referring to FIGS. 1 to 3, a multi-channel artificial basement membrane sensor system (100) according to one embodiment of the present invention may include an artificial basement membrane sensor (110) and a signal processing unit (120).

[0052] First, referring to FIG. 1, the operation process of a multi-channel artificial basilar membrane sensor system (100) according to an embodiment of the present invention will be briefly examined. When a sound signal (S: Input Sound) is input to an artificial basilar membrane sensor (110), a signal (ν) detected in a plurality of channels (e.g., CH1 to CH24) arranged along the length direction of the artificial basilar membrane (ABM) (114) is generated. ABM) can be generated. These signals (V ABM ) can be signal processed in the signal processing unit (120). The signal processing model (M) of the signal processing unit (120) may be an artificial hair cell (AHC) model. The signal (V) calculated for each channel through the signal processing model (M) AHC ) can be integrated to generate an output signal (O).

[0053] As illustrated in FIG. 2, the artificial basement membrane sensor (110) may include a support member (112), an artificial basement membrane (114), and a detection member (117).

[0054] The artificial basilar membrane (114) can perform the function of distinguishing the frequency of sound at each position along its length when vibrating due to sound. The artificial basilar membrane (114) can have a spiral structure similar to a cochlea. The artificial basilar membrane (114) can have a thin membrane shape so that vibration due to sound can be easily generated. The artificial basilar membrane (114) can have a shape in which one end is narrow and the other end becomes wider. Here, one end of the artificial basilar membrane (114) can be referred to as a base, and the other end can be referred to as an apex.

[0055] The artificial basement membrane (114) can be composed of various materials such as polymer materials, ceramics, metals, and composites. For example, polymer materials with excellent flexibility and adhesion can be selected from various known materials such as PI (polyimide), PA (polyamide), PC (polycarbonate), PU (polyurethane), PP (polypropylen), PET (polyethylene terephthalate), PEN (polyethylene naphthalate), PP (polypropylene), PVDF (polyvinylidene fluoride), PPA (polyphthalamide), PPS (polyphenylene sulfide), PDMS (polydimethylsiloxane), PMMA (poly methyl methacrylate), and PET (polyethylene terephthalate).

[0056] Meanwhile, the material of the artificial basement membrane (114) may also be selected depending on the detection method of the detection unit (117). For example, if the material of the artificial basement membrane (114) is composed of a non-conductive material, a conductive material may be coated to detect vibration signals, and in order to perform both vibration and detection functions, the artificial basement membrane (114) may be manufactured in the form of a film having piezoelectric properties.

[0057] The support member (112) may have a housing shape and may have an internal space. An area to which an artificial basement membrane (114) can be fixed may be formed on one surface of the support member (110). The support member (112) may fix and support the edge (outer surface) of the artificial basement membrane (114) so ​​that the artificial basement membrane (114) is not deformed when vibrating. The thickness of the support member (112) may be greater than the thickness of the artificial basement membrane (114). When the artificial basement membrane (114) is supported by the support member (112), the artificial basement membrane (114) may be placed at a certain distance from the bottom of the support member (112). There is no limitation on the type of material of the support member (112).

[0058] The detection unit (117) is a detection sensor that can detect an electric signal generated when the artificial basement membrane (114) vibrates and transmit it to the outside. The detection unit (117) can be made of a conductive material so as to detect a potential difference generated by the vibration.

[0059] The sensing unit (117) converts vibrations into electrical signals, and various types of sensing methods can be used. The sensing method by the sensing unit (117) can be various methods such as a collector resistance method, a capacitive method, a piezoelectric method, a triboelectric method, and an electrochemical method. The sensing unit (117) can be made of a piezoelectric material that does not require power and has high sensitivity.

[0060] If the detection unit (117) can detect the vibration signal of the artificial basement membrane (114), the attachment method or detection method of the detection unit (117) can be formed in various ways.

[0061] The sensing unit (117) can be arranged along the longitudinal direction of the artificial basement membrane (114). The sensing unit (17) can be arranged at a specific location to form a channel. FIG. 2 illustrates a state in which 24 sensing units (117) are arranged along the longitudinal direction of the artificial basement membrane (114).

[0062] Hereinafter, a design formula of a multi-channel artificial basement membrane sensor system according to one embodiment of the present invention is described.

[0063] Referring to FIG. 3, according to one embodiment of the present invention, in the design of the multi-channel artificial basement membrane sensor system, the structural variables and material properties of the artificial basement membrane (114) are used as variables, and the characteristic frequency (쪉) is expressed as in <Mathematical Formula 1> below. CF ) can be calculated.

[0064] <Mathematical Formula 1>

[0065]

[0066] Here, δ represents the deflection of the central part of the beam, and Psound means negative pressure, F sound represents the force of the negative pressure, △x represents the length of the artificial basement membrane, which is a micro-section, w represents the width of the artificial basement membrane, t represents the thickness of the artificial basement membrane, E represents the Young's modulus of the artificial basement membrane, I represents the second moment of cross-section of the artificial basement membrane, which is a micro-section, and m BM refers to the mass of the artificial basement membrane, which is a microscopic section, and ρ BM refers to the density of the artificial basement membrane, and k BM refers to the spring constant of the artificial basilar membrane, which is a microscopic section, and f CF refers to the resonant frequency of the artificial basilar membrane, which is a microscopic section.

[0067] Referring to the above <Mathematical Formula 1> and the contents illustrated in Fig. 3, the characteristic frequency is It can be seen that it is proportional to . Here, w(x) means the width of the artificial basement membrane (114) according to the length of the artificial basement membrane (114), and t(x) means the thickness of the artificial basement membrane (114) according to the length of the artificial basement membrane (114).

[0068] Meanwhile, in the design of the artificial basement membrane sensor system according to one embodiment of the present invention, the width of the artificial basement membrane can be calculated as in <Mathematical Formula 2> below using the structural variables, material properties, and target frequency band of the artificial basement membrane (114) as variables.

[0069] <Mathematical Formula 2>

[0070]

[0071] Here, f L (or f low ) represents the lowest resonant frequency of the artificial basilar membrane, and f H (or f high ) represents the maximum resonant frequency of the artificial basilar membrane, L represents the length of the artificial basilar membrane, and f CF (x) represents the resonant frequency according to the distance of the artificial basilar membrane, and ω(x) represents the width according to the distance of the artificial basilar membrane.

[0072] Referring to the above <Mathematical Formula 1> and <Mathematical Formula 2>, in the design formula of the multi-channel artificial basement membrane sensor system according to one embodiment of the present invention, the thickness (t) of the basement membrane can be used as a structural variable in calculating the characteristic frequency or the width of the artificial basement membrane (114).

[0073] In addition, in the design formula of the multi-channel artificial basement membrane sensor system according to one embodiment of the present invention, in calculating the characteristic frequency or width of the artificial basement membrane (114), the density and Young's modulus of the artificial basement membrane (114) can be used as material properties.

[0074] In addition, in the design formula of the multi-channel artificial basement membrane sensor system according to one embodiment of the present invention, the maximum frequency (f) that the sensor can detect H ) and lowest frequency (f L ) can be used as the target frequency band.

[0075] Referring to the above <Mathematical Formula 1> or <Mathematical Formula 2>, the design formula of the multi-channel artificial basilar membrane sensor system may be characterized in that the width (ω) of the artificial basilar membrane increases exponentially. Accordingly, the artificial basilar membrane (114) designed according to one embodiment of the present invention may have a curved shape such that its width increases exponentially along its length direction.

[0076] Additionally, in the multi-channel artificial basilar membrane sensor system, the distribution of characteristic frequencies increases exponentially depending on the position (x) of the artificial basilar membrane.

[0077] FIG. 4 is a graph for explaining the sigmoid function and variables of an artificial hair cell according to one embodiment of the present invention.

[0078] The signal processing unit (120) processes the signal output from the detection unit (117) of the artificial basilar membrane sensor. The signal processing unit (120) according to one embodiment of the present invention may include a neural transmission signal model of an artificial hair cell as a signal processing model. The signal processing model may apply a sigmoid function as in <Mathematical Formula 3> below to process the electric signal output from the detection unit (117) of the artificial basilar membrane (114). Meanwhile, in <Mathematical Formula 3> below, the artificial basilar membrane sensor may be represented as an ABM sensor, and the artificial hair cell model to which the sigmoid function is applied may be represented as an AHC model.

[0079] <Mathematical Formula 3>

[0080]

[0081] Here, V AHC,n is the output signal of channel n of the AHC model, and v ABM,max is the maximum output of the ABM sensor, and v ABM,n is the output signal of channel n of the ABM sensor, and α n and β n is the slope determination constant of channel n, and v ref,n is the maximum slope position of the nth channel, and can be expressed as the equation below in the above <Mathematical Formula 3>, and v ABM,max,n is the maximum output of channel n of the ABM sensor, and v ABM,min,n is the minimum output of the nth channel of the ABM sensor, and c is the position determination constant of the maximum slope of the nth channel.

[0082] As shown in Fig. 4, as a result of applying the above <Mathematical Formula 3>, the nth output signal of the ABM sensor can be converted into the nth output signal of the AHC model, and it can be seen that the slope, which is the result of differentiation of the sigmoid function, represents a continuous distribution. According to one embodiment of the present invention, the signal processing model of the multi-channel artificial basilar membrane sensor system is the maximum output (v) of the artificial basilar membrane sensor electric signal according to the sound pressure size of the sound signal. ABM,max,n) and minimum output (v ABM,min,n ) can be used as a variable.

[0083] A signal processing model of a multi-channel artificial basilar membrane sensor system according to one embodiment of the present invention can determine a function modification of the signal processing model using an electric signal measured from a sensing unit of an artificial basilar membrane as a variable.

[0084] The signal processing model of the signal processing unit (120) can normalize and band-filter the electrical signal detected through nonlinear amplification in the multi-channel artificial basilar membrane sensor system. That is, the electrical signal of the sensor processed according to the signal processing model of the multi-channel artificial basilar membrane sensor system can be normalized by compensating for the difference in output according to the position of the artificial basilar membrane (114). In addition, the electrical signal of the sensor processed according to the signal processing model of the multi-channel artificial basilar membrane sensor system can be band-filtered so that only the output within the critical band can be amplified (e.g., nonlinearly amplified) and the other signals can be attenuated.

[0085] FIG. 5 is a graph comparing simulation results of a conventional artificial basilar membrane and an artificial basilar membrane modeled according to a design formula of an embodiment of the present invention, and FIG. 6 is a graph comparing vibration displacement measurement results of an artificial basilar membrane sensor manufactured according to a design formula of an embodiment of the present invention.

[0086] On the left side of FIG. 5, a simulation result measuring the characteristic frequency for each channel in a conventional artificial basilar membrane is shown, and on the right side of FIG. 5, a simulation result of an artificial basilar membrane modeled according to an embodiment of the present invention is shown.

[0087] The artificial basilar membrane illustrated in the upper part of Fig. 5 (a) has a shape in which the width (W) increases linearly from the base to the apex. Referring to the graph illustrated in the lower part of Fig. 5 (a), it can be seen that the distribution of characteristic frequencies for each channel is nonlinear. The artificial basilar membrane illustrated in the upper part of Fig. 5 (b) is designed according to the above <Mathematical Formula 2>, and has a shape in which the width (W) increases in a curved shape from the base to the apex. Referring to the graph illustrated in the lower part of Fig. 5 (b), it can be seen that the distribution of characteristic frequencies for each channel is linear.

[0088] FIG. 6 illustrates the frequency band measured for each channel in an artificial basilar membrane sensor manufactured according to the design formula of one embodiment of the present invention. FIG. 6 (a) and (b) illustrate the displacement magnitude of the frequency measured for each channel normalized by the signal processing unit (120). Referring to FIG. 6, it can be confirmed that the critical band frequency for each channel is properly separated according to the vibration of the artificial basilar membrane. In addition, it can be seen that a similar signal magnitude is formed for the difference in vibration displacement for each channel of the artificial basilar membrane sensor.

[0089] FIG. 7 is a graph showing a sigmoid function of each detection unit according to a signal processing model of one embodiment of the present invention and a graph showing a change in sensor output according to signal processing.

[0090] FIGS. 7 and 8 show signal analysis results measured in channel 1 (ch. 1), channel 3 (ch. 3), and channel 8 (ch. 8) in an artificial basement membrane sensor modeled according to a design formula according to one embodiment of the present invention.

[0091] FIG. 7 (a) shows a sigmoid function graph of an output signal (ABM+AHC signal) modeled for signal processing according to an embodiment of the present invention for each channel. As shown in FIG. 7 (a), it can be confirmed that the modeled signal is separated by forming peak values ​​of different sizes for each channel. FIG. 7 (b) shows a comparison of the characteristic frequency sizes of an output signal (ABM+AHC signal) processed for signal processing according to an embodiment of the present invention and an unmodeled output signal (ABM) for each channel. As shown in FIG. 7 (b), the characteristic frequency size of the unmodeled output signal (ABM) decreases as it goes to ch. 8, but the output signal (ABM+AHC) modeled according to an embodiment of the present invention can be seen to have the same or similar size due to nonlinear amplification of the characteristic frequency in each channel.

[0092] FIG. 8 shows the magnitude of a signal (ABM) output through each channel (e.g., channel 1, channel 3, channel 8) of an artificial basilar membrane sensor when an input sound signal (Input sound) is input to an artificial basilar membrane sensor system according to an embodiment of the present invention, and the magnitude of a signal (AHC) output by modeling these signals in a signal processing unit (120) are shown over time. As shown in FIG. 8, it can be seen that the magnitude of the modeled AHC is amplified and larger than the magnitude of the ABM signal measured in each channel.

[0093] Meanwhile, the signal processing unit (120) according to one embodiment of the present invention may use a neural network algorithm as a method for recognizing and distinguishing detected sound features. Various artificial intelligence techniques may be used in this process. For example, artificial intelligence techniques such as convolutional neural networks (CNNs), recurrent neural networks (RNNs), artificial neural networks (ANNs), deep neural networks (DNNs), Gaussian mixture models (GMMs), hidden Markov models (HMMs), and long short-term memory (LSTMs) may be used.

[0094] FIG. 9 is a real-time spectrogram that expresses the electrical signals of each sensing unit of an artificial basilar membrane sensor manufactured according to one embodiment of the present invention as a heat map image, thereby eliminating the need for a separate STFT (Short Time Fourier Transform) process.

[0095] Meanwhile, the signal processing unit (120) can integrate the signals of each channel as a method for detecting sound characteristics by directly using the signals of the multi-channel artificial basilar membrane sensor system according to one embodiment of the present invention.

[0096] FIG. 9 illustrates a state in which various information (four types of words: go, stop, up, down) spoken by a speaker is converted into a heat map image, which is a visual graphic in the form of a heat distribution on a certain image. In the heat map image illustrated in FIG. 9, the horizontal axis represents time information, the vertical axis represents the channel number of the sensing unit, and the magnitude of the electrical signal is represented by color distinction. The channel number of the vertical axis can provide frequency information. According to one embodiment of the present invention, the above-described heat map image can be utilized as data for artificial intelligence as a method for extracting features of a sound signal.

[0097] As shown in Fig. 9, it can be confirmed that there is a clear difference in the heatmap image between cases where the artificial hair cell model is applied and cases where it is not. In the case of the heatmap image of the artificial basilar membrane sensor signal (ABM signal) without applying the artificial hair cell model, due to the difference in sensitivity according to the basilar membrane location, only some of the low-frequency components among the sound features are included, and the noise component appears relatively large. On the other hand, when the artificial hair cell model is applied, the size of the electrical signal (AHC signal) of each channel can be normalized so that sound features can be effectively detected in all frequency bands. In addition, sound features with a high signal-to-noise ratio can be provided through the band-filter function through nonlinear amplification.

[0098] While the present invention has been described with reference to the embodiments illustrated in the accompanying drawings, these are merely exemplary, and those skilled in the art will appreciate that various modifications and equivalent embodiments are possible. Accordingly, the true scope of protection of the present invention should be determined solely by the appended claims.

Claims

1. An artificial basilar membrane having a constant thickness and a width designed to have a characteristic frequency uniformly distributed along its length in order to detect sound by separating it according to frequency; and A plurality of sensing units that convert vibrations installed on the artificial basement membrane into electrical signals; Multi-channel artificial basilar membrane sensor system.

2. In paragraph 1, A multi-channel artificial basilar membrane sensor system characterized in that the outer part of the artificial basilar membrane is fixed and the inner part is supported so as to allow free vibration.

3. In paragraph 1, A multi-channel artificial basement membrane sensor system characterized in that it has a width designed using the <mathematical formula> below according to the structural variables, material properties, and target frequency band of the artificial basement membrane. <Mathematical formula> Here, here, f L (or f low ) represents the lowest resonant frequency of the artificial basilar membrane, and f H (or f high ) represents the maximum resonant frequency of the artificial basilar membrane, L represents the length of the artificial basilar membrane, and f CF (x) represents the resonant frequency according to the distance of the artificial basilar membrane, and ω(x) represents the width according to the distance of the artificial basilar membrane.

4. In paragraph 1, A multi-channel artificial basement membrane sensor system, wherein the structural variable is the thickness of the artificial basement membrane.

5. In paragraph 1, A multi-channel artificial basement membrane sensor system characterized in that the above material properties are the density and Young's modulus of the artificial basement membrane.

6. In paragraph 1, A multi-channel artificial basilar membrane sensor system characterized in that the sensing part of the artificial basilar membrane is made of a piezoelectric material.

7. In paragraph 1, A multi-channel artificial basilar membrane sensor system characterized in that a signal processing model of artificial hair cells calculated by imitating biological hair cells is used to process electrical signals generated from each sensing unit of the artificial basilar membrane.

8. In paragraph 7, The above signal processing model is a multi-channel artificial basilar membrane sensor system characterized in that it normalizes and band-filters the electric signal of the artificial basilar membrane detection unit by applying a sigmoid function as in the <mathematical formula> below. <Mathematical formula> Here, V AHC,n is the output signal of channel n of the AHC model, and v ABM,max is the maximum output of the ABM sensor, and v ABM,n is the output signal of channel n of the ABM sensor, and α n and β n is the slope determination constant of channel n, and v ref,n is the maximum slope position of the nth channel, and can be expressed as the equation below in the above <Mathematical Formula 3>, and v ABM,max,n is the maximum output of channel n of the ABM sensor, and v ABM,min,n is the minimum output of channel n of the ABM sensor, and c is the position determination constant of the maximum slope of channel n.

9. A multi-channel artificial basilar membrane sensor system characterized by combining an artificial basilar membrane and artificial hair cells to detect sound signals as electrical signals by dividing them into frequency bands and processing them in real time without separate calculations.

10. In paragraph 9, A multi-channel artificial basilar membrane sensor system characterized in that an artificial intelligence algorithm part is added based on the signal-processed electrical signal obtained from the artificial hair cell.

11. In paragraph 10, A multi-channel artificial basilar membrane sensor system characterized in that at least one of a convolutional neural network (CNN), a recurrent neural network (RNN), an artificial neural network (ANN), a deep neural network (DNN), a Gaussian mixture model (GMM), a hidden Markov model (HMM), and a long short-term memory (LSTM) is used as the artificial intelligence algorithm.

12. In paragraph 9, A multi-channel artificial basilar membrane sensor system characterized in that a heat map image composed of a horizontal axis representing time information, a vertical axis representing the channel number of the sensing unit, and a color representing the size of the electric signal is used as one of the feature extraction methods of the above sound signal.

Images