Method and device for analyzing sound signal, and method and device for designing chip

Abstract

The present application provides a sound signal analysis method, a sound signal analysis device, a chip design method and a chip design device. The sound signal analysis method comprises: in a first update gradient, using a plurality of sets of first sound training data in a task set to train a resolution restoration model under the same grouping condition; in a second update gradient, using second sound training data in the task set under different grouping conditions to train the resolution restoration model; iteratively performing the first update gradient and the second update gradient to set the initial model parameters of the resolution restoration model; and using the resolution restoration model to restore a low-resolution snoring sound signal to a high-resolution snoring sound signal, wherein the resolution of the low-resolution snoring sound signal is lower than that of the high-resolution snoring sound signal. In this way, the problems of reducing signal error and inaccuracy and reducing chip area can be improved.

Description

Technical Field

[0001] This invention relates to a sound signal analysis method, a sound signal analysis device, a chip design method, and a chip design device. Background Technology

[0002] In response to the development of smart headphones, the integration of physiological signal sensing is currently a very popular field. One of the newest applications is in snoring-related medical applications. However, prolonged snoring detection is quite power-intensive, hence the market's pursuit of low-power physiological sensing technology. However, when traditional super-resolution methods reduce power consumption, the reconstructed signal is prone to errors, leading to inaccurate physiological signal calculations. For example, when smart headphones detect snoring, traditionally, directly reducing the sampling frequency can reduce power consumption, but this requires the use of super-resolution methods to train a reconstruction model between low-sampling and high-sampling signals to reconstruct the high-sampling-frequency signal. However, traditional super-resolution methods can result in inaccurate reconstructions. Therefore, how to achieve low-power physiological sensing technology is one of the research topics that those skilled in the art are dedicated to.

[0003] Furthermore, the market also demands smaller electronic products that are easier to insert into the ear canal, conforming to ergonomics. However, traditional chip area reduction uses heuristic algorithms, which easily find local minima (i.e., the second smallest area). Therefore, how to design and reduce chip area is also one of the research topics that those skilled in the art are dedicated to. Summary of the Invention

[0004] This invention provides a sound signal analysis method, a sound signal analysis device, a chip design method, and a chip design device, which can improve and reduce power consumption signal errors and inaccuracies, and reduce chip area.

[0005] An embodiment of the present invention provides a sound signal analysis method, comprising: in a first update gradient, using multiple first sound training data from multiple task sets that meet the same grouping condition to train a resolution restoration model in an audio processing chip; in a second update gradient, using multiple second sound training data from the multiple task sets that meet different grouping conditions alternately to train the resolution restoration model; iteratively executing the first update gradient and the second update gradient to set initial model parameters of the resolution restoration model; and using the resolution restoration model to restore a low-resolution snoring signal to a high-resolution snoring signal, wherein the resolution of the low-resolution snoring signal is lower than the resolution of the high-resolution snoring signal.

[0006] An embodiment of the present invention further provides a sound signal analysis device, which includes an audio processing chip and a storage circuit. The audio processing chip has a built-in resolution restoration model. The storage circuit is connected to the audio processing chip and is used to store multiple sets of task data. The audio processing chip is used to: in a first update gradient, train the resolution restoration model using multiple first sound training data that meet the same grouping condition from the multiple sets of task data; in a second update gradient, train the resolution restoration model by interleaving multiple second sound training data that meet different grouping conditions from the multiple sets of task data; iteratively execute the first update gradient and the second update gradient to set the initial model parameters of the resolution restoration model; and use the resolution restoration model to restore a low-resolution snoring signal to a high-resolution snoring signal, wherein the resolution of the low-resolution snoring signal is lower than the resolution of the high-resolution snoring signal.

[0007] Embodiments of the present invention further provide a chip design method for a chip layout prediction model within a chip design apparatus. The chip design method includes: reconstructing based on multiple constrained execution models; and iteratively searching for a chip layout design corresponding to an audio processing chip via multiple decision paths to produce the audio processing chip, wherein the chip layout design satisfies the minimum layout area of ​​the audio processing chip.

[0008] Embodiments of the present invention further provide a chip design apparatus, comprising a storage circuit and a processor. The storage circuit stores a chip layout prediction model. The processor is connected to the storage circuit. The processor is configured to run the chip layout prediction model to: perform model reconstruction based on multiple constraints; and iteratively search for a chip layout design corresponding to an audio processing chip via multiple decision paths to produce the audio processing chip, wherein the chip layout design satisfies the minimum layout area of ​​the audio processing chip. Attached Figure Description

[0009] Figure 1A This is a schematic diagram of a sound signal analysis device according to an embodiment of the present invention;

[0010] Figure 1B This is a flowchart illustrating the initial model parameters for setting the resolution of the restored model according to an embodiment of the present invention;

[0011] Figure 1C This is a schematic diagram of the model architecture of the resolution restoration model according to an embodiment of the present invention;

[0012] Figure 2 This is a schematic diagram of a sound signal analysis device according to an embodiment of the present invention;

[0013] Figure 3This is a schematic diagram illustrating the process of restoring a low-resolution snoring signal to a high-resolution snoring signal according to an embodiment of the present invention;

[0014] Figure 4 This is a schematic diagram of a chip design apparatus according to an embodiment of the present invention;

[0015] Figure 5 This is a schematic diagram illustrating the reduction of chip size according to an embodiment of the present invention;

[0016] Figure 6 This is a flowchart illustrating the traditional process of using heuristic algorithms to find the minimum layout area of ​​a chip.

[0017] Figure 7 This is a schematic flowchart of the relaxation-based reconstruction branching method according to an embodiment of the present invention;

[0018] Figure 8 This is a flowchart illustrating a sound signal analysis method according to an embodiment of the present invention;

[0019] Figure 9 This is a flowchart illustrating a chip design method according to an embodiment of the present invention. Detailed Implementation

[0020] Reference will now be made in detail to exemplary embodiments of the invention, examples of which are illustrated in the accompanying drawings. Wherever possible, the same element symbols are used in the drawings and description to denote the same or similar parts.

[0021] In the super-resolution method, low-resolution snoring can be incorporated into randomly initialized model parameters and fed into the first layer of the neural network (i.e., learning small features, including calculating weights, biases, and activation functions) and the second layer of the neural network (i.e., learning large features, including calculating weights, biases, and activation functions) to generate high-resolution snoring. However, when reducing power consumption, the restored signal is prone to errors, leading to inaccurate calculation of physiological signals.

[0022] This invention utilizes a dual gradient learning generalization method to replace random initialization, thereby improving and reducing the problems of power consumption signal error and inaccuracy.

[0023] Figure 1A This is a schematic diagram of a sound signal analysis device according to an embodiment of the present invention. Please refer to... Figure 1A The sound signal analysis device 10 may include various electronic devices that can process sound signals, such as wired headphones, wireless headphones, smartphones, tablet computers, laptop computers, or voice recorders, and the type of sound signal analysis device 10 is not limited to these.

[0024] The audio signal analysis device 10 includes an audio processing chip 11 and a storage circuit 12. The audio processing chip 11 is used to perform audio signal processing. For example, the audio processing chip 11 may include a central processing unit (CPU), or other programmable general-purpose or special-purpose microprocessors, digital signal processors (DSPs), programmable controllers, application-specific integrated circuits (ASICs), programmable logic devices (PLDs), or other similar devices or combinations thereof. In one embodiment, the audio processing chip 11 may also be responsible for the overall or partial operation of the audio signal analysis device 10.

[0025] Storage circuit 12 is connected to audio processing chip 11. Storage circuit 12 is used to store data. For example, storage circuit 12 may include volatile storage circuitry and non-volatile storage circuitry. Volatile storage circuitry is used to volatilely store data. For example, volatile storage circuitry may include random access memory (RAM) or similar volatile storage media. Non-volatile storage circuitry is used to non-volatilely store data. For example, non-volatile storage circuitry may include read-only memory (ROM), solid-state disk (SSD), and / or hard disk drive (HDD) or similar non-volatile storage media.

[0026] The audio signal analysis device 10 may also include a power management circuit for managing the power supply of the audio signal analysis device 10, various sensors, and various input / output (I / O) interfaces. This invention does not limit the types of sensors and I / O interfaces included in the audio signal analysis device 10.

[0027] The audio processing chip 11 may include a resolution restoration model 13. For example, the resolution restoration model 13 may be built into the audio processing chip 11. The resolution restoration model 13 can be used to restore a sound sample signal (also called a first sound sample signal) having a certain resolution (also called a first resolution) to a sound sample signal (also called a second sound sample signal) having another resolution (also called a second resolution). The resolution of the second sound sample signal is higher than the resolution of the first sound sample signal. For example, after sampling a sound signal (also called the original sound signal) to obtain a first sound sample signal with a lower resolution, the resolution restoration model 13 can restore the first sound sample signal to a second sound sample signal with a higher resolution. For example, the resolution of the second sound sample signal may be close to or even the same as the resolution of the original sound signal (also called the original resolution). In an exemplary embodiment, the first sound sample signal includes a low-resolution snoring signal, and / or the second sound sample signal includes a high-resolution snoring signal. The resolution of the low-resolution snoring signal is lower than the resolution of the high-resolution snoring signal.

[0028] The resolution restoration model 13 may include one or more neural networks. The neural network may be implemented using various neural network models such as Convolutional Neural Network (CNN) and / or Feed Forward Deep Neural Network (FFDNN) and / or deep learning models, and may be adjusted according to practical needs.

[0029] The storage circuit 12 can be used to store multiple task sets 14(1) to 14(n). Each task set 14(1) to 14(n) may include at least one audio training data set to train the resolution reconstruction model 13. For example, the audio training data may include sampled data of the audio signal (also referred to as audio sampling data). Furthermore, the present invention does not limit the total number of task sets 14(1) to 14(n).

[0030] The voice training data in task sets 14(1) to 14(n) can be grouped according to specific grouping conditions (also known as screening conditions). For example, the grouping conditions may include the age, gender, and nationality of the voice sensing object, which can be used to distinguish different types of voice sensing objects, and the grouping conditions are not limited to these. Voice training data that meet the same grouping conditions can be stored in the same task set in task sets 14(1) to 14(n). For example, task set 14(1) can be used to store voice training data of voice sensing objects aged between 15 and 18 years old, task set 14(2) can be used to store voice training data of voice sensing objects aged between 19 and 22 years old, task set 14(3) can be used to store voice training data of voice sensing objects of male gender, and task set 14(4) can be used to store voice training data of voice sensing objects of female gender, etc., and the grouping conditions can be adjusted according to practical needs, which is not limited in this invention. In addition, different task sets in task sets 14(1) to 14(n) respectively store voice training data that meet different grouping conditions. In addition, the audio processing chip 11 can iteratively execute different types of update gradient operations according to the task set 14(1) to 14(n) to set the initial parameters (also known as the initial model parameters) of the resolution restoration model 13.

[0031] Figure 1B This is a flowchart illustrating the initial model parameters for setting the resolution of the restored model according to an embodiment of the present invention. Please refer to... Figure 1B In step 101, the audio processing chip 11 can set the initialization parameters of the resolution restoration model 13. In step 102, the iteration begins. In step 103, the audio processing chip 11 can randomly select from... Figure 1A Tasks are selected from task sets 14(1) to 14(n) for training.

[0032] In step 104, the audio processing chip 11 may perform an update gradient (also referred to as the first update gradient (learning individual tasks)) operation. In step 104 (i.e., the first update gradient), the audio processing chip 11 may use multiple sound training data (also referred to as the first sound training data) from task sets 14(1) to 14(n) that meet the same grouping condition to train the resolution restoration model 13 to learn the audio features of individual tasks. For example, in step 104, all or at least some of the sound training data in task set 14(i) may be sequentially input into the resolution restoration model 13, so that the resolution restoration model 13 continuously learns the audio features of individual tasks in task set 14(i). After training the resolution restoration model 13 using all or at least some of the sound training data in task set 14(i), all or at least some of the sound training data in task set 14(i) may be sequentially input into the resolution restoration model 13, so that the resolution restoration model 13 continuously learns the audio features of individual tasks in task set 14(j), where i is not equal to j.

[0033] In one embodiment, in step 104, the audio processing chip 11 can perform weight calculation, bias calculation, and activation function calculation related to the resolution restoration model 13 based on the audio features of the individual task learned by the resolution restoration model 13. Then, the audio processing chip 11 can calculate the cross-entropy and update the parameters of the resolution restoration model 13 based on the calculation result.

[0034] Figure 1C This is a schematic diagram of the model architecture of the resolution restoration model according to an embodiment of the present invention. Please refer to... Figure 1C The input signal 110 may include sound training data and be input to the resolution restoration model 13. The input signal 110 may include multiple input values ​​x1 to x2. N The resolution restoration model 13 can use computational models 120(1) to 120(L) to iteratively compute the sound training data 110 and generate an output signal 130. The output signal 130 includes multiple output values ​​y1 to y2. M It should be noted that N and M are both positive integers, and M is greater than N. That is, the output values ​​y1 to y2 are... M The total number is greater than the output values ​​x1 to x N The total number of signals. In this way, the resolution restoration model 13 can restore the lower-resolution input signal 110 to a higher-resolution output signal 130. Furthermore, during the training of the resolution restoration model 13, the operational parameters used by the resolution restoration model 13 can be continuously updated to find the optimal parameters.

[0035] The resolution restoration model 13 may include multiple operational models 120(1) to 120(L). For example, the number of layers in operational models 120(1) to 120(L) is L, and L can be any positive integer greater than 1. Operational model 120(k) in operational models 120(1) to 120(L) is used to calculate the excitation function based on a weight value W(k) and a deviation value b(k), where k is between 1 and L. For example, operational model 120(k) can calculate the excitation function based on the following equations (1.1) and (1.2).

[0036] σ(x) = max(0, x) (1.1)

[0037] x = W(k) × x i +b(k) (1.2)

[0038] Equation (1.1) represents the excitation function. In Equation (1.2), W(k) represents the weight value used in the operational model 120(k), b(k) represents the deviation value used in the operational model 120(k), and xi represents the input value of a certain operational node in the operational model 120(k). Furthermore, after generating the output signal 130, the audio processing chip 11 can calculate the cross-entropy according to the following equation (1.3).

[0039]

[0040] In equation (1.3), CE represents cross-entropy, and CE reflects the difference between the output signal 130 generated by the resolution restoration model 13 and the verification data. The audio processing chip 11 can continuously update (e.g., adjust) the operational parameters (e.g., the aforementioned weight values ​​and bias values) used by the resolution restoration model 13 based on the calculation results of equation (1.3) until the optimal parameters are found.

[0041] Please return Figure 1BFollowing step 104, in step 105, the audio processing chip 11 may perform another update gradient operation (also referred to as the second update gradient (learning individual tasks)). In step 105 (i.e., the second update gradient), the audio processing chip 11 may interleave multiple sound training data (also referred to as second sound training data) from task sets 14(1) to 14(n) that meet different grouping conditions to train the resolution restoration model 13. For example, in the second update gradient, a sound training data from task set 14(p) may first be input into the resolution restoration model 13 to train the resolution restoration model 13. Then, a sound training data from task set 14(q) may be input into the resolution restoration model 13 to train the resolution restoration model 13, where p is not equal to q. And so on, in the second update gradient, more sound training data from different task sets may be interleaved to train the resolution restoration model 13 in order to maximize the accuracy of the resolution restoration model 13 in restoring sound signals from different types of sound sensing objects.

[0042] For example, in step 105, the audio processing chip 11 can perform weight calculations, bias calculations, and activation function calculations related to the resolution restoration model 13 based on the audio features of the total individual tasks (i.e., tasks across the task set) learned by the resolution restoration model 13. Then, the audio processing chip 11 can calculate the cross-entropy and perform a total parameter update of the resolution restoration model 13 based on the calculation results. Related operational details can be found in steps 104 and... Figure 1C The explanation will not be elaborated upon here.

[0043] The audio processing chip 11 can iteratively execute steps 104 and 105 to set the initial parameters (i.e., initial model parameters) of the resolution restoration model 13. For example, after iteratively executing steps 104 and 105 to obtain optimal (or better) parameters using all the sound training data in task sets 14(1) to 14(n), the audio processing chip 11 can determine the end of the iteration in step 106 and generate the model parameters to be used (i.e., initial model parameters) in step 107. Then, the resolution restoration model 13 can perform the restoration of the sound sampled signal based on these initial model parameters. In one embodiment, the method for setting the initial parameters (i.e., initial model parameters) of the resolution restoration model 13 is also called the dual gradient learning generalization method.

[0044] In one embodiment, the task is often used randomly to train the neural network model, and the initialization parameters of the model are often set randomly. Therefore, even after training, the neural network model cannot effectively improve the accuracy of sound signal reconstruction. However, in the embodiments of the present invention, by forcibly adopting the above-mentioned customized and phased model training mechanism, after training is completed, the resolution reconstruction model 13 can accurately reconstruct the low-resolution snoring signal (i.e., the first sound sample signal) into a high-resolution snoring signal (i.e., the second sound sample signal) based on better initial model parameters, and the reconstruction error can be significantly reduced.

[0045] In one embodiment, by employing a low-power sampling mode to sample (i.e., sense) the original sound signal to obtain a low-resolution snoring signal (i.e., the first sound sample signal), the power consumption (e.g., device power consumption) of the sound signal analysis device 10 in performing sound signal sampling can be effectively reduced. Furthermore, by improving the restoration efficiency of the resolution restoration model 13 in restoring the low-resolution snoring signal (i.e., the first sample sound signal), the resolution restoration model 13 can output a high-resolution snoring signal (i.e., the second sound sample signal) that is close to or even almost identical to the original sound signal for subsequent analysis. Thus, throughout the entire sound signal sampling (i.e., sensing) and restoration process, the power consumption of the sound signal analysis device 10 can be significantly reduced, while the accuracy of sound signal analysis can still be maintained or even improved.

[0046] In one embodiment, during the iterative execution of the first update gradient (i.e., learning the individual task) and the second update gradient (i.e., learning the overall individual task), some parameters (e.g., weight values) used by the neural network in the resolution restoration model 13 can be continuously updated to reduce the prediction error of the resolution restoration model 13 and / or improve its prediction accuracy. Furthermore, the audio processing chip 11 can evaluate the prediction error of the resolution restoration model 13 in each iteration based on activation functions and cross-entropy, parameter updates (in the "learning the individual task") or total parameter updates (in the "learning the overall individual task") commonly used in neural networks, deep learning, and machine learning, until the iterative calculation is completed. The relevant operational details have been described above and will not be repeated here.

[0047] Figure 2 This is a schematic diagram of a sound signal analysis device according to an embodiment of the present invention. Please refer to... Figure 2 The sound signal analysis device 20 may include various electronic devices that can receive and process sound signals, such as wired headphones, wireless headphones, smartphones, tablet computers, laptop computers, or voice recorders, and the type of sound signal analysis device 20 is not limited to these.

[0048] The sound signal analysis device 20 may include an audio processing chip 21, a storage circuit 22, a microphone 23, and a resolution restoration model 24. The audio processing chip 21, the storage circuit 22, and the resolution restoration model 24 are identical or similar to... Figure 1A The audio processing chip 11, the storage circuit 12, and the resolution restoration model 13.

[0049] The microphone 23 is connected to the audio processing chip 21. The microphone 23 is used to receive sound signals (i.e., raw sound signals). For example, the microphone 23 may include an audio receiving device such as a microphone. The raw sound signal may include ambient sounds or other sound signals emitted by the target person. For example, the raw sound signal may reflect the breathing sounds emitted by the target person while sleeping or at other times.

[0050] The audio processing chip 21 can receive the original sound signal via the microphone 23. Then, the audio processing chip 21 can sample the original sound signal based on a preset sampling frequency (also referred to as the first sampling frequency) to obtain a low-resolution snoring signal. The audio processing chip 21 can then restore the low-resolution snoring signal to a high-resolution snoring signal using a resolution restoration model 24. For example, the low-resolution snoring signal corresponds to the first sampling frequency, and the high-resolution snoring signal corresponds to another sampling frequency (also referred to as the second sampling frequency), where the second sampling frequency is higher than the first sampling frequency.

[0051] In one embodiment, the first sampling frequency is directly related to the power consumption of the audio processing chip 21 when sampling (i.e., sensing) the original audio signal. That is, the higher the first sampling frequency, the greater the power consumption of the audio processing chip 21 when sampling the original audio signal. Conversely, the lower the first sampling frequency, the lower the power consumption of the audio processing chip 21 when sampling the original audio signal. Therefore, by reducing the first sampling frequency, the power consumption of the audio signal analysis device 20 when performing signal sampling can be effectively reduced.

[0052] Figure 3 This is a schematic diagram illustrating the process of restoring a low-resolution snoring signal to a high-resolution snoring signal according to an embodiment of the present invention. Please refer to... Figure 3 In step 301, a low-resolution snoring signal is obtained. In step 302, the audio processing chip 21 can set the initial parameters (i.e., initial model parameters) of the resolution restoration model 13 through the aforementioned dual gradient learning generalization method. Then, the audio processing chip 21 can restore the low-resolution snoring signal to a high-resolution snoring signal through the resolution restoration model 31. For example, the process of processing the low-resolution snoring signal by the resolution restoration model 31 may include increasing the resolution of the low-resolution snoring signal. For example, the resolution restoration model 31 may include... Figure 1C Resolution restoration model 13 or Figure 2The resolution restoration model 24. Figure 1C For example, input signal 110 may include a low-resolution snoring signal, and output signal 130 may include a high-resolution snoring signal. In step 303, audio processing chip 21 can obtain a high-resolution snoring signal based on the output of resolution restoration model 31. Subsequently, in step 304, the high-resolution snoring signal can be used in various applications.

[0053] In one embodiment, the resolution restoration model 31 may include a first-layer neural network 311 and a second-layer neural network 312. The first-layer neural network 311 and the second-layer neural network 312 can sequentially process the low-resolution snoring signal according to the initial model parameters set in step 302. For example, the first-layer neural network 311 can be used to perform small feature learning on the low-resolution snoring signal, while the second-layer neural network 312 can be used to perform large feature learning on the low-resolution snoring signal. The resolution restoration model 31 can output a high-resolution snoring signal based on the results of small feature learning and large feature learning. The operation process of small feature learning and large feature learning also includes weight calculation, bias calculation, and activation function calculation, etc., and the relevant operation details have been described above and will not be repeated here. In addition, the resolution restoration model 31 may also include more layers of neural networks, depending on the practical needs.

[0054] In one embodiment, after obtaining a high-resolution snoring signal, the audio processing chip 21 can assess the respiratory condition of the target person based on the signal characteristics of the high-resolution snoring signal. For example, the audio processing chip 21 can compare the waveform and other signal characteristics of the second sound sampling signal with signal feature samples in a database and assess the respiratory condition of the target person based on the comparison results. For example, when it is found that the signal characteristics of the second sound sampling signal match a specific signal feature sample in the database, the audio processing chip 21 can generate assessment information to reflect that the target person's respiratory tract has specific symptoms. For example, the assessment information may reflect information related to the target person's respiratory condition, such as whether the target person has sleep apnea, whether the target person's airway is blocked, and / or the location of the airway blockage. Furthermore, if the signal characteristics of the second sound sampling signal do not match any of the signal feature samples in the database, the audio processing chip 21 can generate assessment information to reflect that the target person's respiratory tract is healthy.

[0055] In particular, the foregoing embodiments have already mentioned how to train the resolution restoration model 31 to improve its restoration accuracy. Therefore, in Figure 3In this embodiment, the resolution restoration model 31 can output a high-resolution snoring signal that is close to or even almost identical to the original sound signal based on the low-resolution snoring signal. Subsequently, the signal characteristics of the high-resolution snoring signal are analyzed to assess the respiratory condition of the target person, which is almost equivalent to analyzing the original sound signal emitted by the target person. This achieves a better balance between reducing device power consumption and improving detection accuracy.

[0056] The steps to reduce chip area may include defining an objective function, minimizing the total area, using constraints (including component size, component pins, and component configuration), random initial parameters, heuristic algorithms, and easily obtaining the second smallest total area. However, how to further design and reduce the chip area is a problem.

[0057] This invention uses the relaxation reconstruction branch method to replace the heuristic algorithm in order to obtain the solution with the minimum total area.

[0058] Figure 4 This is a schematic diagram of a chip design apparatus according to an embodiment of the present invention. Please refer to... Figure 4 The chip design device 40 may include various electronic devices with data processing functions, such as smartphones, tablet computers, laptop computers, desktop computers or servers, and the type of chip design device 40 is not limited to these.

[0059] The chip design device 40 may include a processor 41 and a storage circuit 42. The processor 41 may be responsible for the overall or partial operation of the chip design device 40. For example, the processor 41 may include a CPU, or other programmable general-purpose or special-purpose microprocessors, DSPs, programmable controllers, ASICs, PLDs, or other similar devices or combinations thereof.

[0060] Storage circuitry 42 is connected to processor 41. Storage circuitry 42 can be used to store data. For example, storage circuitry 42 may include volatile storage circuitry and non-volatile storage circuitry. Volatile storage circuitry is used to volatilely store data. For example, volatile storage circuitry may include RAM or similar volatile storage media. Non-volatile storage circuitry is used to non-volatilely store data. For example, non-volatile storage circuitry may include ROM, SSD, HDD, or similar non-volatile storage media.

[0061] The storage circuit 42 can be used to store the chip layout prediction model 43. The chip layout prediction model 43 can be designed using customized algorithms. For example, the customized algorithm may include the relaxation-based reconstruction branch method. The chip layout prediction model 43 can be used during the chip design phase to minimize the impact of chip layout on the chip's basic design requirements (e.g., the size of components to be placed within the chip, component pin positions, and component configuration). Figure 1A Audio processing chip 11 or Figure 2 The size of the audio processing chip 21.

[0062] Figure 5 This is a schematic diagram illustrating a reduction in chip size according to an embodiment of the present invention. Please refer to... Figure 5 Assuming that a traditional heuristic algorithm is used to estimate the minimum size of a chip (such as an audio processing chip), the possible minimum layout area of ​​the chip can be region 51. For example, the width, height, area, and bus length of region 51 can be 35.9 mm, 37.7 mm, 1353.43 mm², and 3622.9 mm, respectively.

[0063] Figure 6 This is a flowchart illustrating a typical heuristic algorithm for finding the minimum layout area of ​​a chip. Please refer to it. Figure 6 In step 601, an objective function is defined. For example, solving the objective function can be used to find the minimum layout area of ​​the chip. In step 602, the objective function is attempted to be used to find the minimum layout area of ​​the chip. In step 603, constraints are introduced to set the basic design requirements of the chip (e.g., the size of components to be placed on the chip, component pinouts, and component configuration). In step 604, random initial parameters are set. In step 605, the minimum layout area of ​​the chip is sought through a heuristic algorithm. In step 606, based on the heuristic algorithm in step 605, the second smallest total chip area is easily obtained, rather than the minimum total chip area.

[0064] Please return Figure 5 Compared to Figure 6 The second smallest total chip area can be easily obtained by using a heuristic algorithm. Figure 4 The chip layout prediction model 43, which employs the relaxed reconstruction branch method, is used to find the minimum possible size of the same chip (e.g., an audio processing chip). This allows the minimum layout area of ​​the chip to be reduced from region 51 to region 52. For example, the width, height, area, and bus length of region 52 can be 34.4 mm, 35.8 mm, 1231.52 mm², and 3149.2 mm, respectively, but the invention is not limited to these dimensions.

[0065] Please return Figure 4 During the chip design phase, the processor 41 can run a chip layout prediction model 43 employing a relaxed reconstruction branching method and import the chip's basic design requirements (e.g., component dimensions, pinouts, and component configurations to be placed within the chip). After importing the chip's basic design requirements, the chip layout prediction model 43 can reconstruct the chip layout according to various constrained execution models and iteratively search for the corresponding target chip (e.g., ...) through multiple decision paths. Figure 1A Audio processing chip 11 or Figure 2The chip layout design of the audio processing chip 21) is provided, and the chip layout design can be used to manufacture the target chip. In particular, the chip layout design can meet the (close to) minimum layout area of ​​the target chip.

[0066] Figure 7 This is a schematic flowchart illustrating the relaxation-based reconstruction branching method according to an embodiment of the present invention. Please refer to... Figure 4 and Figure 7 In step 701, during the chip design phase, processor 41 can enable the relaxed reconstruction branching method (also known as the restricted relaxation method). In step 702, processor 41 can obtain a first type of restriction (also known as a hard restriction) and a second type of restriction (also known as a soft restriction) corresponding to the relaxed reconstruction branching method. In step 703, processor 41 can perform model reconstruction based on the hard and soft restrictions. In particular, the hard restrictions must be satisfied during the model reconstruction process, while the soft restrictions may not be satisfied during the model reconstruction process.

[0067] During model reconstruction, processor 41 can reconstruct the mathematical model preset by chip layout prediction model 43 based on hard and soft constraints. For example, processor 41 can adjust the mathematical model preset by chip layout prediction model 43 to a new mathematical model. Compared to the mathematical model preset by chip layout prediction model 43, the new mathematical model will more easily find the optimal solution for the objective function. For example, the optimal solution may include the minimum total area solution of the predicted target chip (e.g., the length and width of the target chip).

[0068] In one embodiment, the objective function can be represented by the following equation (2.1).

[0069] α(H+W)+β∑ e (R e -L e +U e -D e (2.1)

[0070] In equation (2.1), H represents the length of the substrate, W represents the width of the substrate, Re represents the right boundary position of the e-th netlist, Le represents the left boundary position of the e-th netlist, Ue represents the top boundary position of the e-th netlist, and De represents the bottom boundary position of the e-th netlist. Alternatively, from another perspective, α(H+W) can be used to represent the sum of the length and width of the substrate, and β(Re-Le+Ue-De) can be used to represent the line length of each netlist. By finding the minimum solution set of equation (2.1), the minimum total area solution of the target chip can be obtained.

[0071] In one embodiment, at least one of the following equations (3.1) to (3.8) can also be used in conjunction with the above equation (2.1).

[0072] X i +(1 - r i - r″ i )wid i / 2+(r i + r″ i )ht i / 2 ≤ W (3.1)

[0073] y i +(1 - r i - r” i )ht i / 2+(r i + r” i )wid i / 2 ≤ H (3.2)

[0074] r i + r′ i + r″ i ≤ 1 (3.3)

[0075] x i +(1 - r i - r″ i )wid i / 2+(r i + r″ i )ht i / 2 + spc i，j ≤ x j -(1 - r j - r″ j )wid j / 2-(r j + r″ j )ht j / 2 + M(z i，j + z′ i，j ) (3.4)

[0076] x j +(1 - r j - r″ j )wid j / 2+(r j + r″ j )ht j / 2 + spc i，j ≤ x i -(1 - r i - r″ i )wid i / 2 一(r i+r” i )ht j / 2+M(1-z i，j +z′ i，j ) (3.5)

[0077] y i +(1-r i -r” i )ht i / 2+(r i +r” i )wid i / 2+spc i，j ≤y j +(1-r j -r” j )ht i / 2+(r j +r” j )wid j / 2+M(1+z i，j +z′ i，j ) (3.6)

[0078]

[0079] L e ≤x i +(1-r i -r’ i -r” i )dx 0 +r i dx+r i ′dx’+r i ”dx”

[0080] R e ≥x i +(1-r i -r’ i -r” i )dx 0 +r i dx+r i ′dx’+r i ″dx″

[0081] D e ≤x i +(1-r i -r′ i -r″ i )dy 0 +r i dy+r i ′dy’+r i ”dy″

[0082] U e≥x i +(1-r i -r′ i -r″ i )dy 0 +r i dy+r i 'dy'+r i “dy” (3.8)

[0083] In equations (3.1) to (3.8), xi represents the X-axis position of the center point of element i, yi represents the Y-axis position of the center point of element i, zi,j and z'i,j represent the relative positions of element i and j, ri represents whether element i is rotated 90 degrees, r'i represents whether element i is rotated 180 degrees, and r”i represents whether element i is rotated 270 degrees.

[0084] After model reconstruction is completed, processor 41 can iteratively find the minimum area solution for the chip layout design corresponding to the target chip by executing the branch method in step 708 through the multiple decision paths corresponding to steps 704-707. In particular, the multiple decision paths corresponding to steps 704-707 each correspond to an alternative solution rule to satisfy the solution of the objective function. In step 709, processor 41 can obtain the minimum total area solution.

[0085] Specifically, in the decision path corresponding to step 704, processor 41 can replace some decision variables in chip layout prediction model 43, for example, by merging or splitting at least some unknown decision variables (i.e., replacing decision variables). Then, processor 41 can reduce the feasible solution or solution space of the mathematical model used by chip layout prediction model 43 in an attempt to speed up the solution of the objective function (i.e., reduce the feasible solution / smallest solution space).

[0086] In the decision path corresponding to step 705, processor 41 can use slack variables to change some inequalities in chip layout prediction model 43 into equalities with the same or similar meanings (i.e., generate effective inequalities). Then, processor 41 can relax the restriction that some variables can only be "0" or "1" to be between "0" and "1" in an attempt to speed up the solution of the objective function (i.e., obtain the enhanced reality relaxation solution).

[0087] In the decision path corresponding to step 706, the processor 41 can optimize some or all of the variables in the chip layout prediction model 43. For example, it can select some or all of the variables to solve or adjust the variable values ​​in an attempt to speed up the solution of the objective function (i.e., partial variable optimization / all variable optimization).

[0088] In the decision path corresponding to step 707, processor 41 can combine some independent constraints in the chip layout prediction model 43 with the objective function to form sub-problems (i.e., independent constraints). Then, processor 41 can decompose the mathematical model used by the chip layout prediction model 43 into multiple sub-problems and back-calculate the solution to the original problem (i.e., the objective function) based on the solutions to these sub-problems, in an attempt to accelerate the solution to the objective function (i.e., the solution to the sub-problems). The total number of decision paths can be more or less, depending on practical needs. In addition, the content of the decision paths can also be adjusted according to practical needs, and this invention does not impose any limitations.

[0089] In step 708, during the process of solving the objective function, the processor 41 can use a branching method to combine and optimize the multiple decision paths corresponding to steps 704-707. For example, the processor 41 can use a branching method to perform an iterative search to solve the problem through different branch variables and / or sub-problems in the multiple decision paths. The final optimal solution for the objective function is the minimum total area solution. The minimum total area solution reflects the minimum layout area of ​​the target chip (e.g., the length and width of the target chip) derived by the chip layout prediction model 43. From another perspective, Figure 7 The relaxation-based reconstruction branching method shown can be used to replace Figure 6 Step 605 uses a traditional heuristic algorithm to attempt to further reduce the chip layout area.

[0090] In one embodiment, after obtaining the (near) minimum layout area (or the chip layout design) of the target chip, the processor 41 can provide the chip layout design to the next stage chip manufacturing department or chip manufacturer to plan and produce the target chip. In particular, for device types such as wireless headphones that require minimizing device size as much as possible, effectively reducing the size of the internal audio processing chip can significantly increase the design flexibility of the wireless headphones and / or the user's comfort when wearing the wireless headphones.

[0091] Figure 8 This is a flowchart illustrating a sound signal analysis method according to an embodiment of the present invention. Please refer to... Figure 8In step S801, during the first update gradient, multiple first sound training data sets from multiple task sets that meet the same grouping condition are used to train the resolution restoration model in the audio processing chip. In step S802, during the second update gradient, multiple second sound training data sets from the multiple task sets that meet different grouping conditions are used alternately to train the resolution restoration model. In step S803, the first update gradient and the second update gradient are iteratively executed to set the initial model parameters of the resolution restoration model. In step S804, the resolution restoration model is used to restore the low-resolution snoring signal to a high-resolution snoring signal, wherein the resolution of the low-resolution snoring signal is lower than the resolution of the high-resolution snoring signal.

[0092] Figure 9 This is a flowchart illustrating a chip design method according to an embodiment of the present invention. Please refer to... Figure 9 In step S901, reconstruction is performed based on multiple restricted execution models. In step S902, a chip layout design corresponding to the audio processing chip is iteratively searched via multiple decision paths to produce the audio processing chip, wherein the chip layout design satisfies the minimum layout area of ​​the audio processing chip.

[0093] However, Figure 8 and Figure 9 Each step has been explained in detail above, so it will not be repeated here. Figure 8 and Figure 9 Each step can be implemented as multiple program codes or circuits, and this invention is not limited thereto. Furthermore, Figure 8 and Figure 9 The method can be used in conjunction with the above examples and embodiments, or it can be used alone. This invention does not impose any limitations.

[0094] In summary, by employing different update gradient mechanisms at different update gradients to set the initial parameters of the resolution restoration model in the audio processing chip, the accuracy of the resolution restoration model in restoring the resolution of the sound sampled signal can be effectively improved. Furthermore, through customized model reconstruction and relaxation reconstruction branching techniques for multiple decision paths, the chip layout prediction model can more easily find the minimum layout area of ​​the audio processing chip.

[0095] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and not to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some or all of the technical features; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of the present invention.

Claims

1. A method for analyzing sound signals, characterized in that, include: In the first update gradient operation, multiple first sound training data belonging to the same task set from multiple task sets are used to train the resolution restoration model in the audio processing chip. In the second update gradient operation, the resolution restoration model is trained by interleaving multiple second sound training data belonging to different task sets in the multiple task sets; Perform the first and second gradient update operations to set the initial parameters of the resolution restoration model; as well as The resolution restoration model is used to restore a low-resolution snoring signal to a high-resolution snoring signal, wherein the resolution of the low-resolution snoring signal is lower than that of the high-resolution snoring signal.

2. The sound signal analysis method according to claim 1, wherein the low-resolution snoring signal corresponds to a first sampling frequency, the high-resolution snoring signal corresponds to a second sampling frequency, and the second sampling frequency is higher than the first sampling frequency.

3. The sound signal analysis method according to claim 1, wherein the step of using the resolution restoration model to restore the low-resolution snoring signal to the high-resolution snoring signal includes: The first layer of the neural network in the resolution restoration model performs feature learning on the low-resolution snoring signal; as well as The low-resolution snoring signal is subjected to feature learning by the second layer neural network in the resolution restoration model; as well as Based on the feature learning results of the first layer neural network on the low-resolution snoring signal and the feature learning results of the second layer neural network on the low-resolution snoring signal, the low-resolution snoring signal is restored to the high-resolution snoring signal.

4. The sound signal analysis method according to claim 1, further comprising: During the chip design phase, a chip layout prediction model reconstructs and iteratively searches for the chip layout design corresponding to the audio processing chip based on multiple restricted execution models and through multiple decision paths, in order to produce the audio processing chip. The chip layout design meets the minimum layout area of ​​the audio processing chip.

5. The sound signal analysis method according to claim 4, wherein the multiple constraints include hard constraints and soft constraints, wherein the hard constraints must be satisfied during the model reconstruction process, and the soft constraints may not be satisfied during the model reconstruction process.

6. The sound signal analysis method according to claim 5, wherein each of the multiple decision paths corresponds to an alternative solution rule to satisfy the objective function solution.

7. A sound signal analysis device, characterized in that, include: Audio processing chip with built-in resolution restoration model; as well as A storage circuit, connected to the audio processing chip, is used to store multiple sets of tasks. The audio processing chip is used for: In the first update gradient operation, the resolution restoration model is trained using multiple first sound training data belonging to the same task set from the multiple task sets. In the second update gradient operation, the resolution restoration model is trained by interleaving multiple second sound training data belonging to different task sets in the multiple task sets; Perform the first and second gradient update operations to set the initial parameters of the resolution restoration model; as well as The resolution restoration model is used to restore a low-resolution snoring signal to a high-resolution snoring signal, wherein the resolution of the low-resolution snoring signal is lower than that of the high-resolution snoring signal.

8. The sound signal analysis device according to claim 7, wherein the low-resolution snoring signal corresponds to a first sampling frequency, the high-resolution snoring signal corresponds to a second sampling frequency, and the second sampling frequency is higher than the first sampling frequency.

9. The sound signal analysis apparatus according to claim 7, wherein the operation of the audio processing chip in restoring the low-resolution snoring signal to the high-resolution snoring signal includes: The first layer of the neural network in the resolution restoration model performs feature learning on the low-resolution snoring signal; The low-resolution snoring signal is subjected to feature learning by the second layer neural network in the resolution restoration model; as well as Based on the feature learning results of the first layer neural network on the low-resolution snoring signal and the feature learning results of the second layer neural network on the low-resolution snoring signal, the low-resolution snoring signal is restored to the high-resolution snoring signal.

Images