Audio searching method and device, equipment, storage medium and program product

Abstract

The invention discloses an audio search method and device, equipment, a storage medium and a program product, and relates to the technical field of network search, and the method comprises the steps: obtaining current audio data and to-be-searched information in a preset database; performing data feature processing on the current audio data to obtain keyword information; performing parallel optical calculation on the keyword information and the information to be searched to obtain a level signal; matching the keyword information with the to-be-searched information on the basis of the level signal to obtain an audio search result, performing data feature processing on current audio data to obtain keyword information representing frequency domain features, and performing parallel optical information calculation on the keyword information and the to-be-searched information to obtain an audio search result. Optical calculation matching is achieved under the condition that CPU resources are not consumed basically, so that the audio search result is obtained, the query efficiency of audio data is improved, and calculation resources are reduced.

Description

Technical Field

[0001] The present application relates to the field of network search technology, and in particular to an audio search method, device, equipment, storage medium and program product. Background Art

[0002] In daily life, audio is an important multimedia data. Audio retrieval technology can perform real-time retrieval, review and monitoring of online media, thereby achieving the purpose of protecting copyright. Audio retrieval technology plays an important role in audio data classification and statistics. At present, the main search method for audio search is audio feature extraction, such as traditional audio feature extraction models based on Gaussian mixture model (GMM), hidden Markov model (HMM), recurrent neural network, long short-term memory (LSTM), encoder-code framework and attention mechanism.

[0003] The recognition rate of traditional audio feature extraction models based on Gaussian mixture model (GMM) and hidden Markov model (HMM) is not high. Audio feature extraction methods based on deep learning, such as recurrent neural network, long short-term memory (LSTM), code-code framework and attention mechanism, have high recognition rate, but consume large computing power. With the accumulation of various types of audio data such as songs, speech recognition, and language, the demand for audio recognition search technology is increasing, especially when a large number of users are concurrent in the future, the system will be overloaded, resulting in low search efficiency. Summary of the invention

[0004] The main purpose of this application is to provide an audio search method, device, equipment, storage medium and program product, aiming to solve the current technical problem of low audio search efficiency.

[0005] To achieve the above object, the present application proposes an audio search method, which includes:

[0006] Obtain current audio data and information to be searched in a preset database;

[0007] Performing data feature processing on the current audio data to obtain keyword information;

[0008] Performing parallel optical calculation on the keyword information and the information to be searched to obtain a level signal;

[0009] The keyword information is matched with the information to be searched based on the level signal to obtain an audio search result.

[0010] In one embodiment, the step of performing parallel optical calculation on the keyword information and the information to be searched to obtain a level signal comprises:

[0011] Inputting the information to be searched into a preset laser diode matrix for driving to obtain a light spot matrix;

[0012] Inputting the keyword information and the light spot matrix into a preset liquid crystal spatial light convolution matrix for driving, thereby obtaining a spatial light convolution matrix;

[0013] The light spot matrix and the spatial light convolution matrix are input into a convex lens in parallel for optical calculation to obtain a level signal.

[0014] In one embodiment, the step of inputting the light spot matrix and the spatial light convolution matrix in parallel into a convex lens for optical calculation to obtain a level signal comprises:

[0015] Inputting the light spot matrix and the spatial light convolution matrix into a convex lens in parallel for convergence to obtain total light intensity information;

[0016] The total light intensity information is irradiated onto a photodiode to obtain a level signal.

[0017] In one embodiment, the step of performing data feature processing on the current audio data to obtain keyword information includes:

[0018] Processing the current audio data at a multiple speed to obtain a current multiple speed broadband optical signal;

[0019] Performing Fourier transformation on the current double-speed broadband optical signal through a microlens and a grating array to obtain current spectrum data;

[0020] The spectrum features of the current spectrum data are extracted by an array image sensor to obtain keyword information.

[0021] In one embodiment, the step of matching the keyword information with the information to be searched based on the level signal to obtain an audio search result includes:

[0022] Obtaining a preset amplitude threshold and matching coefficient;

[0023] Obtaining a level signal amplitude according to the level signal;

[0024] Calculating an amplitude difference according to the level signal amplitude and the preset amplitude threshold;

[0025] When the amplitude difference is less than or equal to the matching coefficient, determining that the keyword information matches the information to be searched, and outputting a matching flag and a search matching degree;

[0026] The audio search result is obtained according to the matching mark and the search matching degree.

[0027] In one embodiment, before the step of obtaining the current audio data and the information to be searched in the preset database, the method further includes:

[0028] Process the original audio data on the Internet and in the database at double the speed to obtain the original double-speed broadband optical signal;

[0029] Performing Fourier transformation on the original double-speed broadband optical signal through a microlens and a grating array to obtain original spectral data;

[0030] Extracting spectrum characteristics from the original spectrum data by using an array image sensor to obtain original spectrum characteristic data;

[0031] The audio file name, the audio file and the original frequency spectrum feature data in the original audio data are stored correspondingly to obtain the information to be searched, and a preset database is constructed.

[0032] In addition, to achieve the above purpose, the present application also proposes an audio search device, the audio search device comprising:

[0033] An acquisition module is used to acquire current audio data and information to be searched in a preset database;

[0034] A processing module, used for performing data feature processing on the current audio data to obtain keyword information;

[0035] A calculation module, used for performing parallel optical calculation on the keyword information and the information to be searched to obtain a level signal;

[0036] A matching module is used to match the keyword information with the information to be searched based on the level signal to obtain audio search results.

[0037] In addition, to achieve the above purpose, the present application also proposes an audio search device, the audio search device comprising: a signal processing module, a microlens, a grating, a linear array image sensor, a convex lens and a photodiode connected in sequence;

[0038] The device further comprises: a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the computer program is configured to implement the steps of the audio search method described above.

[0039] In addition, to achieve the above-mentioned purpose, the present application also proposes a storage medium, which is a computer-readable storage medium, and a computer program is stored on the storage medium. When the computer program is executed by a processor, the steps of the audio search method described above are implemented.

[0040] In addition, to achieve the above-mentioned purpose, the present application also provides a computer program product, which includes a computer program, and when the computer program is executed by a processor, the steps of the audio search method described above are implemented.

[0041] One or more technical solutions proposed in the present application obtain current audio data and information to be searched in a preset database; perform data feature processing on the current audio data to obtain keyword information; perform parallel optical calculation on the keyword information and the information to be searched to obtain a level signal; match the keyword information with the information to be searched based on the level signal to obtain audio search results, perform data feature processing on the current audio data to obtain keyword information representing frequency domain characteristics, and perform parallel optical information calculation on the keyword information and the information to be searched to achieve optical calculation matching without consuming substantially any CPU resources, thereby obtaining audio search results, improving the query efficiency of audio data, and reducing computing resources. BRIEF DESCRIPTION OF THE DRAWINGS

[0042] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the present application and, together with the description, serve to explain the principles of the present application.

[0043] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, for ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative labor.

[0044] Figure 1 A flowchart of the first embodiment of the audio search method of the present application is provided;

[0045] Figure 2 This is a hardware schematic diagram of an audio processing device in an embodiment of the audio search method of the present application;

[0046] Figure 3 A flowchart of the second embodiment of the audio search method of the present application is provided;

[0047] Figure 4 A flowchart of the third embodiment of the audio search method of the present application is provided;

[0048] Figure 5 This is a brief flowchart of an embodiment of the audio search method of the present application;

[0049] Figure 6 This is a schematic diagram of the module structure of the audio search device according to an embodiment of the present application;

[0050] Figure 7 Schematic diagram of the device structure of the hardware operating environment involved in the audio search method in the embodiment of the present application.

[0051] The purpose, features and advantages of this application will be further described in conjunction with the embodiments and with reference to the accompanying drawings. DETAILED DESCRIPTION

[0052] It should be understood that the specific embodiments described herein are only used to explain the technical solutions of the present application and are not used to limit the present application.

[0053] In order to better understand the technical solution of the present application, a detailed description will be given below in conjunction with the accompanying drawings and specific implementation methods.

[0054] The main solution of the embodiment of the present application is: obtaining current audio data and information to be searched in a preset database; performing data feature processing on the current audio data to obtain keyword information; performing parallel optical calculation on the keyword information and the information to be searched to obtain a level signal; matching the keyword information with the information to be searched based on the level signal to obtain an audio search result.

[0055] The recognition rate of traditional audio feature extraction models based on Gaussian mixture models (GMM) and hidden Markov models (HMM) in existing technologies is not high. Audio feature extraction methods based on deep learning, such as recurrent neural networks, long short-term memory (LSTM), encoding-coding frameworks, and attention mechanisms, have high recognition rates, but consume a lot of computing power. With the accumulation of various types of audio data such as songs, speech recognition, and language, the demand for audio recognition search technology is increasing, especially in the future when a large number of users are concurrent, the system will be overloaded.

[0056] The present application provides a solution that breaks through the existing computing system and uses a simulated optical computing method to complete the convolution operation and Fourier transform that consume more computing resources at one time through an optical structure, which can significantly improve computing efficiency.

[0057] It should be noted that the execution subject of this embodiment can be a computing service device with data processing, network communication and program running functions, such as a tablet computer, a personal computer, a mobile phone, etc., or an electronic device capable of realizing the above functions, an audio search device, etc. The following takes the audio search device as an example to illustrate this embodiment and the following embodiments.

[0058] Based on this, the present application embodiment provides an audio search method, referring to Figure 1 , Figure 1 This is a flowchart diagram of the first embodiment of the audio search method of the present application.

[0059] In this embodiment, the audio search method includes steps S10 to S40:

[0060] Step S10: Acquire current audio data and information to be searched in a preset database.

[0061] It should be noted that the current audio data is the audio data that the user needs to search, and the information to be searched can be searched in a preset database based on the current audio data to obtain content that matches the current audio data.

[0062] The preset database is established in advance to store audio data collected from various channels, and the information to be searched is all the audio data in the preset database.

[0063] It is understandable that the current audio data or the audio data collected in advance can be processed by the audio search device, and the audio search device may include a general signal processing module, a microlens, a transmission grating, a linear array image sensor, a convex lens, and a photodiode. The general signal processing module (may be composed of a CPU or a GPU or an FPGA or a DSP)

[0064] In a feasible implementation manner, step S10 may include steps A01 to A04:

[0065] Step A01: Process the original audio data on the Internet and in the database at a double speed to obtain an original double-speed broadband optical signal.

[0066] It should be noted that the original audio data collected from the Internet and database can be processed at multiple speeds through the universal signal processing module. The universal signal processing module mainly completes the establishment of the audio data speed to be searched and the search keyword, feature extraction, database establishment and management of the storage of relevant information. The original audio data can be processed at multiple speeds by the universal signal processing module, and the multiple can be 10 3 ~10 12 times, and at the same time drives the broadband light source, thereby converting the original audio data into the original times-speed broadband optical signal, and the original times-speed broadband optical signal is a high-speed broadband optical signal.

[0067] Step A02: Perform Fourier transform on the original double-speed broadband optical signal through a microlens and a grating array to obtain original spectral data.

[0068] In a specific implementation, the original double-speed broadband signal can be Fourier transformed through a microlens and a grating array. Specifically, a diffraction grating can be formed by a microlens and a transmission grating to realize Fourier analysis of the optical signal and convert the original double-speed broadband optical information into a frequency domain signal, i.e., the original spectral data.

[0069] Step A03: extracting spectrum features from the original spectrum data by using an array image sensor to obtain original spectrum feature data.

[0070] In a specific implementation, the array image sensor can be a COMS, which is used to measure and convert the spectral data formed by the microlens and the transmission grating, and perform feature extraction to obtain the original spectrum feature data, and convert it into a one-dimensional data sequence of length n, representing the spectrum energy distribution. The data sequence length is n, and the data order of the data sequence represents the frequency domain order. Each data amplitude of the data sequence represents the energy of the frequency domain, where the sequence length n must be equal to m×m, and the spectrum feature data is output.

[0071] Step A04: The audio file name, the audio file and the original frequency spectrum feature data in the original audio data are stored correspondingly to obtain the information to be searched, and a preset database is constructed.

[0072] In a specific implementation, the audio file name, audio file, and original spectrum feature data in the original audio data may be stored correspondingly to form the information to be searched and construct a preset database.

[0073] Step S20: performing data feature processing on the current audio data to obtain keyword information.

[0074] In a specific implementation, after the user submits the audio data, the current audio data may be processed for data features, including data speed processing, Fourier transform processing, and feature extraction processing, so as to obtain keyword information. The keyword information is the frequency spectrum feature data of the current audio data.

[0075] In a feasible implementation, step S20 may include steps B11 to B13:

[0076] Step B11: Process the current audio data at a multiple speed to obtain a current multiple speed broadband optical signal.

[0077] It should be noted that the current audio data can be processed at a double speed by the above-mentioned processing method for the original audio data, thereby obtaining the current double-speed broadband optical signal.

[0078] Step B12: Perform Fourier transform on the current double-speed broadband optical signal through a microlens and a grating array to obtain current spectrum data.

[0079] In a specific implementation, the current double-speed broadband optical signal is subjected to Fourier transform through a microlens and a grating array. Specifically, a diffraction grating can be formed by a microlens and a transmission grating to realize Fourier analysis of the optical signal and convert the current double-speed broadband optical signal into a frequency domain signal, i.e., the current spectral data.

[0080] Step B13: extracting spectrum features of the current spectrum data through an array image sensor to obtain keyword information.

[0081] In a specific implementation, the spectrum features of the current spectral data are extracted through an array image sensor to obtain keyword information, which is converted into a one-dimensional data sequence of length n, representing the spectrum energy distribution. The data sequence length is n, and the data order of the data sequence represents the frequency domain order. Each data amplitude of the data sequence represents the energy of the frequency domain. The sequence length n must be equal to m×m, and the spectrum feature data is output, i.e., the keyword information.

[0082] Step S30: performing parallel optical calculation on the keyword information and the information to be searched to obtain a level signal.

[0083] In a specific implementation, the keyword information and the information to be searched can be optically calculated in parallel to realize high-speed parallel search of massive audio data in the database. The high-speed parallel scale is determined and adjusted by the scale of the multi-byte storage matrix, the laser diode matrix, and the ultra-high-speed liquid crystal spatial optical convolution matrix, thereby obtaining a level signal. The level signal can reflect the matching degree between the keyword information and the information to be searched, thereby quickly determining the audio data that matches the keyword information in the information to be searched.

[0084] Step S40: matching the keyword information with the information to be searched based on the level signal to obtain audio search results.

[0085] It can be understood that the matching between the keyword information and the information to be searched can be based on the level signal, and specifically the matching degree between the keyword information and the information to be searched can be calculated to obtain the audio search results. The audio search results may include the matching mark between the keyword information and the information to be searched and the search matching degree, so that the corresponding audio file name and audio file can be accurately and quickly queried from the preset database according to the matching mark and the search matching degree.

[0086] like Figure 2 As shown, Figure 2The hardware schematic diagram of the audio processing device includes a universal signal processing module, a microlens, a grating, an array COMS, a convex lens and a control module. The universal signal processing module processes the audio data at a double speed to obtain the doubled-speed audio data, and passes through a broadband light source to generate a high-speed optical signal. The spectrum is obtained through the microlens and the grating, and a 1-dimensional data sequence with a length of n is formed through the linear array COMS to the universal signal processing module. The universal signal processing module inputs the information to be searched into the m×m multi-byte storage matrix and the m×m laser diode matrix to form a parallel optical information transmitter for parallel optical calculation to obtain a spot matrix. The spot matrix is ​​input into the m×m ultra-high-speed liquid crystal spatial optical convolution matrix to perform convolution operation with the keyword information to obtain the spatial optical convolution matrix. The spatial optical convolution matrix and the spot matrix are used for optical calculation, that is, the two matrices are converged on the convex lens for optical calculation, so as to obtain the total light intensity, and the total light intensity is converted into a level signal through the photodiode.

[0087] The present embodiment provides an audio search method, which obtains current audio data and information to be searched in a preset database; performs data feature processing on the current audio data to obtain keyword information; performs parallel optical calculation on the keyword information and the information to be searched to obtain a level signal; matches the keyword information with the information to be searched based on the level signal to obtain an audio search result, by performing data feature processing on the current audio data to obtain keyword information representing frequency domain characteristics, and performs parallel optical information calculation on the keyword information and the information to be searched, to achieve optical calculation matching without consuming substantially any CPU resources, thereby obtaining an audio search result, improving the query efficiency of audio data, and reducing computing resources.

[0088] Based on the first embodiment of the present application, in the second embodiment of the present application, the same or similar contents as those in the above-mentioned embodiment 1 can be referred to the above introduction, and will not be repeated in the following. Figure 3 , step S30 includes steps S301 to S303:

[0089] Step S301: input the information to be searched into a preset laser diode matrix for driving to obtain a light spot matrix.

[0090] It should be noted that the information to be searched is stored in an m×m multi-byte storage matrix. Therefore, the m×m multi-byte storage matrix can output the information to be searched to drive a preset laser diode matrix. The preset laser diode matrix is ​​an m×m laser diode matrix, thereby outputting a light spot matrix.

[0091] The m×m multi-byte storage matrix is ​​mainly used to cache the information to be searched. The information to be searched is stored in the multi-byte storage matrix in a "Z"-shaped order, and the parallel output drives the m×m laser diode matrix. The m×m laser diode matrix has the same matrix scale as the multi-byte storage matrix, which is m×m. According to the high and low levels of the data in the corresponding bits of the m×m multi-byte storage matrix, the corresponding laser diodes are controlled to emit or extinguish, forming an m×m light spot matrix. The m×m multi-byte storage matrix and the m×m laser diode matrix together constitute a parallel optical information transmitter. The matrix data parallel output drives the laser diode matrix of the same scale. The m×m multi-byte matrix storage and the m×m laser diode matrix serve as parallel optical information transmitters. The light spot matrix is ​​a matrix in which the information to be searched is characterized by light intensity.

[0092] Step S302: inputting the keyword information and the light spot matrix into a preset liquid crystal spatial light convolution matrix for driving to obtain a spatial light convolution matrix.

[0093] In a specific implementation, the keyword information can be input into a preset liquid crystal spatial light convolution matrix for driving. The preset liquid crystal spatial light convolution matrix is ​​an m×m ultra-high-speed liquid crystal spatial light convolution matrix, and the matrix scale is the same as the multi-byte storage matrix and the laser diode matrix. By converting the matrix composed of the keyword information into the light-transmitting dot matrix of the liquid crystal dot matrix, a convolution matrix of the spatial light convolver, i.e., a spatial light convolution matrix, is generated, so that the light attenuation amplitude (or transmission amplitude) corresponds to the light spot matrix composed of the information to be searched. The spatial light convolution matrix is ​​a weighted coefficient vector matrix characterized by the light transmittance intensity of the keyword information. After the light intensity matrix of the information to be searched passes through the light transmittance intensity matrix of the keyword information, a convolution operation is performed to obtain the spatial light convolution matrix.

[0094] Step S303: inputting the light spot matrix and the spatial light convolution matrix into a convex lens in parallel for optical calculation to obtain a level signal.

[0095] In a specific implementation, a convex lens can converge the light spot matrix passing through the m×m ultra-high-speed liquid crystal spatial light convolution matrix to one point, so that the light spot matrix and the spatial light convolution matrix can be optically calculated to obtain a level signal.

[0096] In a feasible implementation, step S303 may include steps C11 to C12:

[0097] Step C11: Input the light spot matrix and the spatial light convolution matrix into a convex lens in parallel for convergence to obtain total light intensity information.

[0098] It should be noted that the spot matrix and the spatial light convolution matrix can be converged to one point through a convex lens to obtain the total light intensity information. The total light intensity of this point is equivalent to the inner product of the spot matrix as the input vector and the spatial light convolution matrix as the weighting coefficient vector.

[0099] Ideally, the total light intensity s(u) at this point is: s(u) = h(u1)*f(u2), where f(u2) is the input vector, that is, the light spot matrix, which is the encrypted optical signal expression matrix of the information to be searched, and h(u1) is the system pulse response function, that is, the spatial light convolution matrix, which is the optical signal expression matrix of the keyword information. The * sign represents the convolution operation.

[0100] In the actual system, the total light intensity at this point is linearly different from the total light intensity under ideal light conditions. This difference can be expressed by the difference coefficient M, that is, the total light intensity at this point in the actual system is: s M (u) = M(h(u1)*f(u2)), and the convolution process does not use interference technology.

[0101] Step C12: irradiating the total light intensity information onto a photodiode to obtain a level signal.

[0102] In a specific implementation, the photodiode can convert the light intensity at the spatial position into a level signal. The photoelectric conversion function is L(n), which represents the conversion capability from light intensity to power amplitude. Then the output level signal of the photodiode Vo=L(n)S M (u) = L(n)M(h(u1)*f(u2)).

[0103] In this embodiment, the information to be searched is input into a preset laser diode matrix for driving to obtain a light spot matrix; the keyword information and the light spot matrix are input into a preset liquid crystal spatial light convolution matrix for driving to obtain a spatial light convolution matrix; the light spot matrix and the spatial light convolution matrix are input into a convex lens in parallel for optical calculation to obtain a level signal. Through parallel optical information calculation, optical calculation matching is achieved without consuming CPU resources.

[0104] Based on the first embodiment of the present application, in the third embodiment of the present application, the same or similar contents as those in the above-mentioned embodiment 1 can be referred to the above introduction, and will not be repeated in the following. Figure 4 , step S40 includes steps S401 to S405:

[0105] Step S401: Obtain a preset amplitude threshold and a matching coefficient.

[0106] It should be noted that the preset amplitude threshold is a matching amplitude set in advance, and the matching coefficient is an inexact matching coefficient P, which can represent the degree of matching between the keyword audio and the audio to be searched.

[0107] Step S402: obtaining a level signal amplitude according to the level signal.

[0108] The level signal is calculated through the difference coefficient M, the optical signal expression matrix of the keyword information, and the optical signal expression matrix of the information to be searched, so as to obtain a specific level signal amplitude.

[0109] Step S403: calculating an amplitude difference according to the level signal amplitude and the preset amplitude threshold.

[0110] In a specific implementation, the amplitude difference Po may be calculated according to the level signal amplitude Vo and a preset amplitude threshold Vp, where the amplitude difference Po=(Vo-Vp).

[0111] Step S404: When the amplitude difference is less than or equal to the matching coefficient, determine that the keyword information matches the information to be searched, and output a matching flag and a search matching degree.

[0112] In a specific implementation, when the amplitude difference Po≤matching coefficient P, it is defined as search data matching. At this time, the output matching flag can be controlled, and the amplitude difference Po can be output as the search matching degree at the same time.

[0113] Step S405: obtaining audio search results according to the matching mark and the search matching degree.

[0114] In a specific implementation, the smaller the search matching degree Po is, the higher the matching degree of the search result is. When the matching mark is output, the data in the m×m multi-byte storage matrix is ​​read, which is the searched information. When the mismatch mark is output, the search continues until all the information data to be searched is calculated. The search results can be displayed in order from small to large according to the search matching degree. The display order is from large to small for the search matching degree to complete the audio search.

[0115] This embodiment obtains a preset amplitude threshold and a matching coefficient; obtains the amplitude of the level signal according to the level signal; calculates the amplitude difference according to the amplitude of the level signal and the preset amplitude threshold; determines that the keyword information matches the information to be searched when the amplitude difference is less than or equal to the matching coefficient, and outputs a matching flag and a search matching degree; obtains the audio search result according to the matching flag and the search matching degree. Whether the data to be searched matches the search keyword is determined by the total light intensity detected by the photodiode. This process does not require the participation of the CPU and is completed once by the photoelectric device. Only when the total light intensity reaches a certain value, the match is determined to be successful, and the current multi-byte matrix memory data is output as the data to be searched. The data to be searched that has not been successfully matched will not be read, identified, or output, thereby ensuring information security.

[0116] For example, to help understand the implementation process of the audio search method obtained by combining this embodiment with the above-mentioned embodiment 1, please refer to Figure 5 , Figure 5 A brief flow diagram of an audio search method is provided. Specifically, the original audio data is sequentially processed at a double speed, processed with a broadband light source, processed with a microlens and processed with a grating to form a spectrum, and then input into a linear array COMS to form a 1-dimensional data sequence, and the audio file and the name are used together to construct a matrix database to be searched. When audio search is required, the user inputs the current audio data, and the current audio data is processed at a double speed, processed with a broadband light source, processed with a microlens and processed with a grating to form keyword information. The information to be searched is input into a parallel optical information transmitter to output a light spot matrix, the keyword information and the light spot matrix are input into an m×m ultra-high-speed liquid crystal spatial light convolution matrix, and the spatial light convolution matrix is ​​output. At the same time, the spatial light convolution matrix and the light spot matrix are input into a convex lens for convergence, and a level signal is obtained through a photodiode, and matching and comparison are performed through the level signal. When matching, a matching mark and a search matching degree are output, and if not matching, a non-matching mark is output, and the initial step is returned to re-search until matching audio data is found, and the matching audio data is sorted and displayed from large to small according to the search matching degree.

[0117] It should be noted that the above examples are only used to understand the present application and do not constitute a limitation on the audio search method of the present application. More simple transformations based on this technical concept are all within the scope of protection of the present application.

[0118] This application also provides an audio search device, please refer to Figure 6 , the audio search device comprises:

[0119] The acquisition module 10 is used to acquire the current audio data and the information to be searched in the preset database.

[0120] The processing module 20 is used to perform data feature processing on the current audio data to obtain keyword information.

[0121] The calculation module 30 is used to perform parallel optical calculation on the keyword information and the information to be searched to obtain a level signal.

[0122] The matching module 40 is used to match the keyword information with the information to be searched based on the level signal to obtain audio search results.

[0123] The audio search device provided by the present application adopts the audio search method in the above embodiment, which can solve the technical problem of low audio search efficiency. Compared with the prior art, the beneficial effects of the audio search device provided by the present application are the same as the beneficial effects of the audio search method provided by the above embodiment, and other technical features in the audio search device are the same as the features disclosed in the above embodiment method, which will not be repeated here.

[0124] In one embodiment, the calculation module 30 is also used to input the information to be searched into a preset laser diode matrix for driving to obtain a light spot matrix; input the keyword information and the light spot matrix into a preset liquid crystal spatial light convolution matrix for driving to obtain a spatial light convolution matrix; input the light spot matrix and the spatial light convolution matrix in parallel into a convex lens for optical calculation to obtain a level signal.

[0125] In one embodiment, the calculation module 30 is further used to input the light spot matrix and the spatial light convolution matrix in parallel into a convex lens for convergence to obtain total light intensity information; and irradiate the total light intensity information onto a photodiode to obtain a level signal.

[0126] In one embodiment, the processing module 20 is further used to perform double-speed processing on the current audio data to obtain a current double-speed broadband light signal; perform Fourier transform on the current double-speed broadband light signal through a microlens and a grating array to obtain current spectral data; and perform spectral feature extraction on the current spectral data through an array image sensor to obtain keyword information.

[0127] In one embodiment, the matching module 40 is further used to obtain a preset amplitude threshold and a matching coefficient; obtain a level signal amplitude according to the level signal; calculate an amplitude difference according to the level signal amplitude and the preset amplitude threshold; when the amplitude difference is less than or equal to the matching coefficient, determine that the keyword information matches the information to be searched, and output a matching flag and a search matching degree; and obtain an audio search result according to the matching flag and the search matching degree.

[0128] In one embodiment, the acquisition module 10 is also used to process the original audio data in the Internet and the database at a double speed to obtain an original double-speed broadband optical signal; perform Fourier transform on the original double-speed broadband optical signal through a microlens and a grating array to obtain original spectral data; perform spectrum feature extraction on the original spectral data through an array image sensor to obtain original spectrum feature data; store the audio file name, audio file and the original spectrum feature data in the original audio data in a corresponding manner to obtain information to be searched, and build a preset database.

[0129] The present application provides an audio search device, which includes a signal processing module, a microlens, a grating, a linear array image sensor, a convex lens and a photodiode connected in sequence; the audio search device also includes: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions that can be executed by the at least one processor, and the instructions are executed by the at least one processor so that the at least one processor can execute the audio search method in the above-mentioned embodiment one.

[0130] Reference below Figure 7 , which shows a schematic diagram of the structure of an audio search device suitable for implementing the embodiment of the present application. The audio search device in the embodiment of the present application may include but is not limited to mobile terminals such as mobile phones, laptop computers, digital broadcast receivers, PDAs (Personal Digital Assistants), PADs (Portable Application Descriptions), PMPs (Portable Media Players), vehicle-mounted terminals (such as vehicle-mounted navigation terminals), etc., and fixed terminals such as digital TVs, desktop computers, etc. Figure 7 The audio search device shown is merely an example and should not bring any limitation to the functions and scope of use of the embodiments of the present application.

[0131] like Figure 7As shown, the audio search device may include a processing device 1001 (e.g., a central processing unit, a graphics processor, etc.), which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM: Read Only Memory) 1002 or a program loaded from a storage device 1003 to a random access memory (RAM: Random Access Memory) 1004. In RAM1004, various programs and data required for the operation of the audio search device are also stored. The processing device 1001, ROM1002, and RAM1004 are connected to each other through a bus 1005. An input / output (I / O) interface 1006 is also connected to the bus. Generally, the following systems can be connected to the I / O interface 1006: an input device 1007 including, for example, a touch screen, a touch pad, a keyboard, a mouse, an image sensor, a microphone, an accelerometer, a gyroscope, etc.; an output device 1008 including, for example, a liquid crystal display (LCD: Liquid Crystal Display), a speaker, a vibrator, etc.; a storage device 1003 including, for example, a magnetic tape, a hard disk, etc.; and a communication device 1009. The communication device 1009 can allow the audio search device to communicate wirelessly or wired with other devices to exchange data. Although the figure shows an audio search device with various systems, it should be understood that it is not required to implement or have all the systems shown. More or fewer systems can be implemented or provided alternatively.

[0132] In particular, according to the embodiments disclosed in the present application, the process described above with reference to the flowchart can be implemented as a computer software program. For example, the embodiments disclosed in the present application include a computer program product, which includes a computer program carried on a computer-readable medium, and the computer program includes a program code for executing the method shown in the flowchart. In such an embodiment, the computer program can be downloaded and installed from a network through a communication device, or installed from a storage device 1003, or installed from a ROM 1002. When the computer program is executed by the processing device 1001, the above-mentioned functions defined in the method of the embodiment disclosed in the present application are executed.

[0133] The audio search device provided by the present application adopts the audio search method in the above embodiment, which can solve the technical problem of low audio search efficiency. Compared with the prior art, the beneficial effects of the audio search device provided by the present application are the same as the beneficial effects of the audio search method provided by the above embodiment, and other technical features in the audio search device are the same as the features disclosed in the method of the previous embodiment, which will not be repeated here.

[0134] It should be understood that the various parts disclosed in this application can be implemented by hardware, software, firmware or a combination thereof. In the description of the above embodiments, specific features, structures, materials or characteristics can be combined in any one or more embodiments or examples in a suitable manner.

[0135] The above is only a specific implementation of the present application, but the protection scope of the present application is not limited thereto. Any person skilled in the art who is familiar with the present technical field can easily think of changes or substitutions within the technical scope disclosed in the present application, which should be included in the protection scope of the present application. Therefore, the protection scope of the present application should be based on the protection scope of the claims.

[0136] The present application provides a computer-readable storage medium having computer-readable program instructions (ie, computer programs) stored thereon, and the computer-readable program instructions are used to execute the audio search method in the above-mentioned embodiment.

[0137] The computer-readable storage medium provided in the present application may be, for example, a USB flash drive, but is not limited to electrical, magnetic, optical, electromagnetic, infrared, or semiconductor systems, systems or devices, or any combination of the above. More specific examples of computer-readable storage media may include, but are not limited to: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In this embodiment, the computer-readable storage medium may be any tangible medium containing or storing a program that can be used by or in combination with an instruction execution system, system or device. The program code contained on the computer-readable storage medium may be transmitted using any appropriate medium, including but not limited to: wires, optical cables, RF (Radio Frequency), etc., or any suitable combination of the above.

[0138] The computer-readable storage medium may be included in the audio search device; or may exist independently without being assembled into the audio search device.

[0139] The above-mentioned computer-readable storage medium carries one or more programs. When the above-mentioned one or more programs are executed by the audio search device, the audio search device: obtains current audio data and information to be searched in a preset database; performs data feature processing on the current audio data to obtain keyword information; performs parallel optical calculation on the keyword information and the information to be searched to obtain a level signal; matches the keyword information with the information to be searched based on the level signal to obtain audio search results.

[0140] Computer program code for performing the operations of the present application may be written in one or more programming languages ​​or a combination thereof, including object-oriented programming languages ​​such as Java, Smalltalk, C++, and conventional procedural programming languages ​​such as "C" or similar programming languages. The program code may be executed entirely on the user's computer, partially on the user's computer, as a separate software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In the case of a remote computer, the remote computer may be connected to the user's computer through any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computer (e.g., via the Internet using an Internet service provider).

[0141] The flow chart and block diagram in the accompanying drawings illustrate the possible architecture, function and operation of the system, method and computer program product according to various embodiments of the present application. In this regard, each square box in the flow chart or block diagram can represent a module, a program segment or a part of a code, and the module, the program segment or a part of the code contains one or more executable instructions for realizing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the square box can also occur in a sequence different from that marked in the accompanying drawings. For example, two square boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each square box in the block diagram and / or flow chart, and the combination of the square boxes in the block diagram and / or flow chart can be implemented with a dedicated hardware-based system that performs a specified function or operation, or can be implemented with a combination of dedicated hardware and computer instructions.

[0142] The modules involved in the embodiments described in this application may be implemented by software or hardware, wherein the name of the module does not constitute a limitation on the unit itself in some cases.

[0143] The readable storage medium provided by the present application is a computer-readable storage medium, which stores computer-readable program instructions (i.e., computer programs) for executing the above-mentioned audio search method, and can solve the technical problem of low audio search efficiency. Compared with the prior art, the beneficial effects of the computer-readable storage medium provided by the present application are the same as the beneficial effects of the audio search method provided by the above-mentioned embodiment, and will not be repeated here.

[0144] The present application also provides a computer program product, including a computer program, which implements the steps of the above-mentioned audio search method when executed by a processor.

[0145] The computer program product provided by this application can solve the current technical problem of low audio search efficiency. Compared with the prior art, the beneficial effects of the computer program product provided by this application are the same as the beneficial effects of the audio search method provided by the above embodiment, which will not be repeated here.

[0146] The above descriptions are only some embodiments of the present application, and are not intended to limit the patent scope of the present application. All equivalent structural changes made using the contents of the present application specification and drawings under the technical concept of the present application, or direct / indirect applications in other related technical fields are included in the patent protection scope of the present application.

Claims

1. An audio search method, characterized in that: The audio search method comprises: Obtain current audio data and information to be searched in a preset database; Performing data feature processing on the current audio data to obtain keyword information; Performing parallel optical calculation on the keyword information and the information to be searched to obtain a level signal; The keyword information is matched with the information to be searched based on the level signal to obtain an audio search result.

2. The method according to claim 1, characterized in that The step of performing parallel optical calculation on the keyword information and the information to be searched to obtain a level signal comprises: Inputting the information to be searched into a preset laser diode matrix for driving to obtain a light spot matrix; Inputting the keyword information and the light spot matrix into a preset liquid crystal spatial light convolution matrix for driving, thereby obtaining a spatial light convolution matrix; The light spot matrix and the spatial light convolution matrix are input into a convex lens in parallel for optical calculation to obtain a level signal.

3. The method according to claim 2, characterized in that The step of inputting the light spot matrix and the spatial light convolution matrix in parallel to a convex lens for optical calculation to obtain a level signal comprises: Inputting the light spot matrix and the spatial light convolution matrix into a convex lens in parallel for convergence to obtain total light intensity information; The total light intensity information is irradiated onto a photodiode to obtain a level signal.

4. The method according to claim 1, characterized in that The step of performing data feature processing on the current audio data to obtain keyword information comprises: Processing the current audio data at a multiple speed to obtain a current multiple speed broadband optical signal; Performing Fourier transformation on the current double-speed broadband optical signal through a microlens and a grating array to obtain current spectrum data; The spectrum features of the current spectrum data are extracted by an array image sensor to obtain keyword information.

5. The method according to claim 1, characterized in that The step of matching the keyword information with the information to be searched based on the level signal to obtain an audio search result comprises: Obtaining a preset amplitude threshold and matching coefficient; Obtaining a level signal amplitude according to the level signal; Calculating an amplitude difference according to the level signal amplitude and the preset amplitude threshold; When the amplitude difference is less than or equal to the matching coefficient, determining that the keyword information matches the information to be searched, and outputting a matching flag and a search matching degree; The audio search result is obtained according to the matching mark and the search matching degree.

6. The method according to any one of claims 1 to 5, characterized in that Before the step of obtaining the current audio data and the information to be searched in the preset database, the method further includes: Process the original audio data on the Internet and in the database at double the speed to obtain the original double-speed broadband optical signal; Performing Fourier transformation on the original double-speed broadband optical signal through a microlens and a grating array to obtain original spectral data; Extracting spectrum characteristics from the original spectrum data by using an array image sensor to obtain original spectrum characteristic data; The audio file name, the audio file and the original frequency spectrum feature data in the original audio data are stored correspondingly to obtain the information to be searched, and a preset database is constructed.

7. An audio search device, characterized in that: The device comprises: An acquisition module is used to acquire current audio data and information to be searched in a preset database; A processing module, used for performing data feature processing on the current audio data to obtain keyword information; A calculation module, used for performing parallel optical calculation on the keyword information and the information to be searched to obtain a level signal; A matching module is used to match the keyword information with the information to be searched based on the level signal to obtain audio search results.

8. An audio search device, characterized in that: The audio search device comprises: a signal processing module, a microlens, a grating, a linear array image sensor, a convex lens and a photodiode connected in sequence; The device further comprises: a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the computer program is configured to implement the steps of the audio search method according to any one of claims 1 to 6.

9. A storage medium, characterized in that: The storage medium is a computer-readable storage medium, and a computer program is stored on the storage medium. When the computer program is executed by a processor, the steps of the audio search method according to any one of claims 1 to 6 are implemented.

10. A computer program product, characterized in that The computer program product comprises a computer program, and when the computer program is executed by a processor, the steps of the audio search method according to any one of claims 1 to 6 are implemented.

Images