Z-transform-based convolution kernel optimization method, processing method, and related apparatus

By employing Z-transform decomposition and cascaded convolution kernel optimization methods, the computational complexity and memory requirements of convolutional neural networks on low-performance hardware devices were addressed, enabling normal operation and independent controllability on domestically produced hardware devices.

CN122491354BActive Publication Date: 2026-11-10IFLYTEK CO LTD
View PDF 1 Cites 0 Cited by

Patent Information

Application Number
CN202610931253.3
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2026-06-26
Publication Date
2026-11-10
Estimated Expiration
2046-06-26

AI Technical Summary

Technical Problem

On low-performance or domestically produced hardware, convolutional neural network models with high computational complexity and large memory requirements cannot run properly, resulting in a decline in hardware performance. Furthermore, the strict reliance on foreign high-performance hardware leads to insufficient self-control.

Method used

By using a convolution kernel optimization method based on Z-transform, the convolution kernel is decomposed into sub-convolution kernels that meet the deployment size requirements, and then cascaded to construct a cascaded convolutional neural network. This is to match the resources of domestic or low-performance hardware devices and achieve model compression and deployment.

Benefits of technology

Reduce hardware resource consumption, improve hardware performance, and enable the normal deployment and operation of complex neural network models on domestically produced hardware devices to ensure independent controllability.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN122491354B_ABST
    Figure CN122491354B_ABST
Patent Text Reader

Abstract

This application proposes a method, processing method, and related apparatus for convolution kernel decomposition and cascaded quantization optimization based on Z-transform. The method obtains the convolution kernels of the convolutional neural network in the model to be compressed; based on the kernel size and pre-set deployment size requirements, the kernels are decomposed to obtain at least two sub-convolution kernels; the sub-convolution kernels are cascaded to obtain a cascaded convolutional neural network; the model composed of the cascaded convolutional neural network is used as the target model after compression; the target model is deployed on a hardware device to perform speech processing and / or image processing and / or text processing. This scheme decomposes the convolution kernel into multiple sub-convolution kernels that meet the deployment size requirements, thereby compressing the model to be compressed. This ensures that the compressed target model matches the hardware resources of the hardware device deploying the model, reducing resource consumption and improving hardware performance after deployment.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] This application relates to the field of neural network technology, and in particular to a convolution kernel optimization method, processing method and related apparatus based on Z-transform. Background Technology

[0002] Convolutional Neural Networks (CNNs) are widely used in fields such as computer vision, speech recognition, natural language processing, and signal processing. The convolution operation, as the core computational unit of a CNN, has its computational complexity and memory usage primarily determined by the size of the convolution kernel.

[0003] In practical applications, certain network architectures (such as temporal convolutional networks in speech processing and large receptive field convolutional layers in image processing) use large convolutional kernels. This results in convolutional neural network models constructed using these architectures having high computational complexity and large memory requirements, often requiring high-performance foreign hardware to run. However, on low-performance hardware devices, such as domestically produced hardware, hardware resources and computing power are limited, making it impossible for these computationally complex and memory-intensive convolutional neural network models to run properly, or requiring almost all of the hardware resources, leading to a decline in hardware performance. Furthermore, strict reliance on foreign high-performance hardware can also cause neural networks or models based on these networks to lose their autonomy and controllability, leading to security issues. Summary of the Invention

[0004] Based on the aforementioned needs, this application proposes a convolution kernel optimization method, processing method, and related apparatus based on Z-transform. This method enables the decomposition and compression of the model to be compressed, ensuring that the decomposed and compressed target model is compatible with the hardware resources of domestically produced or low-performance hardware devices on which the model is deployed. Deploying the target model to such hardware devices reduces resource consumption and improves hardware performance, achieving computational capabilities consistent with high-performance hardware devices on domestically produced or low-performance hardware. Furthermore, by implementing the above scheme, the normal deployment and operation of complex neural network models on domestically produced hardware devices can be achieved, ensuring the autonomous and controllable operation of complex neural network models and avoiding model security issues.

[0005] To achieve the above objectives, this application proposes the following technical solution: According to a first aspect of the embodiments of this application, a convolution kernel optimization method based on Z-transform is provided, comprising: Obtain the convolution kernels of the convolutional neural network in the model to be compressed; wherein, the model to be compressed includes: a speech processing model and / or an image processing model and / or a text processing model; Based on the size of the convolution kernel and the pre-set deployment size requirements, the convolution kernel is decomposed to obtain at least two sub-convolution kernels; wherein the size of the sub-convolution kernels conforms to the deployment size requirements; the deployment size requirements match the hardware resources of the hardware device used to deploy the model to be compressed; The concatenation of the sub-convolutional kernels is performed to obtain a concatenated convolutional neural network. The model composed of the concatenated convolutional neural network is used as the target model after compression. The target model is used to perform speech processing and / or image processing and / or text processing after being deployed on a hardware device.

[0006] Optionally, based on the size of the convolutional kernel and pre-set deployment size requirements, the convolutional kernel is decomposed to obtain at least two sub-convolutional kernels, including: When the size of the convolution kernel is larger than the preset deployment size requirement, a Z-transform is performed on the convolution kernel to obtain the polynomial corresponding to the convolution kernel; Based on the deployment size requirements, the polynomial is factored to obtain the factors contained in the polynomial; Perform an inverse Z-transform on the factor to obtain the sub-convolution kernel corresponding to the factor.

[0007] Optionally, based on the deployment size requirements, the polynomial is factored to obtain the factors contained in the polynomial, including: Find the zeros of the polynomial to obtain all the zeros of the polynomial. Based on the deployment size requirements and all the zeros of the polynomial, the polynomial is factored to obtain the factors contained in the polynomial.

[0008] Optionally, based on the deployment size requirement and all zeros of the polynomial, the polynomial is factored to obtain the factors contained in the polynomial, including: Based on the deployment size requirement, all zeros of the polynomial are grouped to obtain at least two groups of zeros; wherein the number of zeros in each group of zeros is less than the deployment size requirement, and complex zeros that are conjugates of each other are in the same group of zeros. Based on the set of zeros, the polynomial is factored to obtain the factors corresponding to the set of zeros.

[0009] Optionally, based on the deployment size requirements, all zeros of the polynomial are grouped to obtain at least two groups of zeros, including: With the goal of minimizing the number of zeros in each zero group to less than the deployment size requirement and minimizing the dynamic range of the sub-kernel coefficients corresponding to each zero group, all zeros of the polynomial are grouped to obtain at least two zero groups; the dynamic range of the sub-kernel coefficients is determined based on the maximum and minimum zero moduli among the zero moduli of each zero in the zero group.

[0010] Optionally, cascading the sub-convolutional kernels to perform convolution operations yields a cascaded convolutional neural network, including: Determine the amplification capability parameter of the sub-convolution kernel for the input error; wherein the amplification capability parameter is the L1 norm of the sub-convolution kernel, or the product of the L1 norm of the sub-convolution kernel and the quantization error; Based on the amplification capability parameters, all sub-convolutional kernels are sorted to obtain the concatenated order of all sub-convolutional kernels; By cascading the sub-convolutional kernels according to the cascading order, a cascaded convolutional neural network is obtained.

[0011] Optionally, cascading the sub-convolutional kernels to perform convolution operations yields a cascaded convolutional neural network, including: According to the deployment scenario of the model to be compressed, the sub-kernel coefficients of the sub-convolution kernel are quantized to obtain the quantized sub-convolution kernel; By cascading the quantized sub-convolutional kernels through convolution operations, a cascaded convolutional neural network is obtained.

[0012] Optionally, if the convolution kernel is a two-dimensional convolution kernel, based on the size of the convolution kernel and a pre-set deployment size requirement, the convolution kernel is decomposed to obtain at least two sub-convolution kernels, including: The dimensions to be decomposed of the convolution kernel are determined based on the kernel size and the pre-set deployment size requirements. The convolution kernel is decomposed along the dimension to be decomposed to obtain at least two sub-convolution kernels corresponding to the dimension to be decomposed.

[0013] According to a second aspect of the embodiments of this application, a voice processing method is provided, including: The speech data is input into the speech processing model to obtain the speech processing result corresponding to the speech data; The speech processing model is obtained by optimizing the convolution kernel based on the Z-transform described above.

[0014] According to a third aspect of the embodiments of this application, an image processing method is provided, comprising: The image data is input into the image processing model to obtain the image processing result corresponding to the image data; The image processing model is obtained by optimizing the convolution kernel based on the Z-transform described above.

[0015] According to a fourth aspect of the embodiments of this application, a text processing method is provided, including: The text data is input into the text processing model to obtain the text processing result corresponding to the text data. The text processing model is obtained by optimizing the convolution kernel based on the Z-transform described above.

[0016] According to a fifth aspect of the embodiments of this application, an electronic device is provided, including: a memory and a processor; The memory is connected to the processor and is used to store programs; The processor is configured to implement the Z-transform-based convolution kernel optimization method, the speech processing method, the image processing method, or the text processing method by running the program in the memory.

[0017] According to a sixth aspect of the embodiments of this application, a computer program product is provided, including computer program instructions, which, when executed by a processor, cause the processor to implement the above-described Z-transform-based convolution kernel optimization method, the above-described speech processing method, the above-described image processing method, or the above-described text processing method.

[0018] The convolution kernel optimization method proposed in this application obtains the convolution kernel of the convolutional neural network in the model to be compressed. The model to be compressed includes a speech processing model and / or an image processing model and / or a text processing model. Based on the size of the convolution kernel and a pre-set deployment size requirement, the convolution kernel is decomposed to obtain at least two sub-convolution kernels. The size of the sub-convolution kernels meets the deployment size requirement, which matches the hardware resources of the hardware device used to deploy the model to be compressed. The sub-convolution kernels are concatenated to obtain a cascaded convolutional neural network. The model composed of the cascaded convolutional neural network is used as the target model after compression of the model to be compressed. The target model is used to perform speech processing and / or image processing and / or text processing after being deployed on the hardware device. The technical solution of this application can decompose the convolutional kernels in the model to be compressed according to the deployment size requirements, decomposing them into several sub-convolutional kernels that meet the deployment size requirements. This achieves the decomposition and compression of the model to be compressed, ensuring that the decomposed and compressed target model can match the hardware resources of domestic or low-performance hardware devices where the model is deployed. Deploying this target model on domestic or low-performance hardware devices reduces resource consumption and improves hardware performance, enabling the same computing power as high-performance hardware devices. Furthermore, by implementing the above solution, the normal deployment and operation of complex neural network models on domestic hardware devices can be achieved, ensuring the autonomous and controllable operation of complex neural network models and avoiding model security issues. Attached Figure Description

[0019] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only embodiments of this application. For those skilled in the art, other drawings can be obtained based on the provided drawings without creative effort.

[0020] Figure 1 This is a flowchart illustrating a Z-transform-based convolution kernel decomposition and cascaded quantization optimization method provided in an embodiment of this application.

[0021] Figure 2 This is a schematic diagram of the polynomial factorization process provided in the embodiments of this application.

[0022] Figure 3 This is a schematic diagram of a processing flow for cascading sub-convolution kernel convolution operations provided in an embodiment of this application.

[0023] Figure 4 This is a schematic diagram of another processing flow for cascading sub-convolution kernel convolution operations provided in an embodiment of this application.

[0024] Figure 5 This is a schematic diagram of the convolution kernel decomposition process provided in an embodiment of this application.

[0025] Figure 6 This is a flowchart illustrating a speech processing method provided in an embodiment of this application.

[0026] Figure 7 This is a flowchart illustrating an image processing method provided in an embodiment of this application.

[0027] Figure 8 This is a flowchart illustrating a text processing method provided in an embodiment of this application.

[0028] Figure 9 This is a schematic diagram of a Z-transform-based convolution kernel decomposition and cascaded quantization optimization device provided in an embodiment of this application.

[0029] Figure 10 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application. Detailed Implementation

[0030] The technical solution of this application is applicable to the application scenario of convolutional neural network compression. By adopting the technical solution of this application, the model to be compressed can be compressed so that the compressed target model can match the hardware resources of the hardware device on which the model is deployed. After the target model is deployed to the hardware device, the resource consumption of the hardware device can be reduced, and the performance of the hardware device can be improved.

[0031] In core areas of artificial intelligence such as computer vision, speech recognition, natural language processing, and signal processing, Convolutional Neural Networks (CNNs) are widely used due to their powerful feature extraction capabilities and end-to-end learning paradigm. Specifically, in computer vision, CNNs are widely used for tasks such as image classification, object detection, semantic segmentation, instance segmentation, and image generation; in speech processing, CNNs are used for acoustic feature modeling, speech enhancement, and speaker recognition; and in natural language processing, CNNs are applied to tasks such as text classification, sequence labeling, machine translation, and sentiment analysis. Convolutional Neural Networks have become one of the most fundamental and important model architectures in the deep learning technology system.

[0032] Convolution, as the core computational unit of CNNs, has computational complexity and memory consumption primarily determined by the kernel size. In practical applications, certain network architectures (such as temporal convolutional networks in speech processing and large receptive field convolutional layers in image processing) use large convolutional kernels, resulting in high computational complexity and large memory requirements for convolutional neural network models constructed from these architectures. This often necessitates running on high-performance foreign hardware. However, on low-performance hardware, such as domestically produced hardware, limited resources and computing power prevent these computationally complex and memory-intensive convolutional neural network models from running properly, or they may consume almost all of the hardware's resources, leading to performance degradation. Furthermore, strict reliance on high-performance foreign hardware can cause neural networks or models based on them to lose their autonomy and controllability, leading to security issues.

[0033] Based on this, this application proposes a convolution kernel optimization method based on Z-transform. This technical solution can decompose and compress the model to be compressed, so that the decomposed and compressed target model can be matched with the hardware resources of domestic or low-performance hardware devices that deploy the model. After the target model is deployed on such hardware devices, it can reduce the resource consumption of the hardware devices, improve the performance of the hardware devices, and achieve the same computing power as high-performance hardware devices on domestic or low-performance hardware devices. At the same time, by implementing the above solution, the normal deployment and operation of complex neural network models on domestic hardware devices can be achieved, making the operation of complex neural network models autonomous and controllable, and avoiding model security issues.

[0034] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0035] Exemplary methods See Figure 1 As shown in the figure, this application proposes a convolution kernel optimization method based on Z-transform. The method includes: S101. Obtain the convolution kernel of the convolutional neural network in the model to be compressed.

[0036] In this embodiment, the model containing a convolutional neural network that needs to be compressed is taken as the model to be compressed, and the convolutional kernel of the convolutional neural network in the model to be compressed is obtained. The model to be compressed includes a speech processing model and / or an image processing model and / or a text processing model, and the convolutional kernel of the convolutional neural network can be a one-dimensional convolutional kernel or a two-dimensional convolutional kernel.

[0037] For example, temporal convolutional networks are included in speech processing models in the field of speech processing; large receptive field convolutional layers are included in image processing models in the field of image processing; and convolutional neural networks are included in text processing models such as text translation and text reasoning prediction in the field of text processing.

[0038] S102. Based on the size of the convolution kernel and the pre-set deployment size requirements, decompose the convolution kernel to obtain at least two sub-convolution kernels.

[0039] In this embodiment, the deployment size requirement corresponding to the hardware device used to deploy the model is determined in advance based on the hardware resources of the hardware device required to deploy the model to be compressed. In other words, the pre-set deployment size requirement matches the hardware resources of the hardware device used to deploy the model to be compressed. In the embodiments of this application, the aforementioned hardware device used to deploy the model to be compressed includes low-performance hardware devices compared to advanced foreign hardware, such as domestically produced hardware devices.

[0040] For example, some DSP chips in a DSP processor only support convolution kernels of up to 16×16. Exceeding this size will cause a sharp drop in the performance of the DSP processor. Therefore, when the hardware device for deploying the model to be compressed is this DSP processor, the pre-set deployment size requirement is 16×16. As another example, the hardware convolution engine of some neural network processing units in an NPU accelerator limits the maximum dimension of a single convolution kernel to 32. Therefore, when the hardware device for deploying the model to be compressed is this NPU accelerator, the pre-set deployment size requirement is 32 dimensions.

[0041] This embodiment first requires obtaining the size of the convolutional kernels in the convolutional neural network of the model to be compressed, and then comparing and analyzing this kernel size with the preset deployment size requirement. If the kernel size is not greater than the preset deployment size requirement, it means that the kernel size matches the hardware resources of the hardware device deploying the model to be compressed, and will not affect the performance of the hardware device; therefore, there is no need to decompose the kernel. If the kernel size is greater than the preset deployment size requirement, it means that the kernel will affect the performance of the hardware device, and the kernel needs to be decomposed to obtain at least two sub-kernels. Furthermore, the kernel needs to be decomposed according to the preset deployment size requirement to ensure that the size of each sub-kernel obtained by decomposition is not greater than the deployment size requirement, i.e., it meets the deployment size requirement. Among these methods, convolution kernel decomposition can be performed using low-rank decomposition, which approximates the convolution kernel as the product of two low-rank matrices based on matrix decomposition theory, such as singular value decomposition (SVD), CP decomposition, and Tucker decomposition. Alternatively, separable convolution can be used, decomposing the two-dimensional convolution into depthwise separable convolution, i.e., first performing spatial convolution independently on each channel, then performing 1×1 pointwise convolution, or further decomposing the spatial convolution into a concatenation of row and column convolution kernels. Another method is to use Z-transform and polynomial factorization, performing Z-transform on the convolution kernel and factoring the polynomial obtained from the Z-transform, determining the corresponding sub-convolution kernels after decomposition based on the factors obtained from the factorization.

[0042] In one specific implementation, based on the size of the convolution kernel and pre-set deployment size requirements, the convolution kernel is decomposed to obtain at least two sub-convolution kernels, specifically including the following steps: First, when the size of the convolution kernel is larger than the pre-set deployment size requirement, a Z-transform is performed on the convolution kernel to obtain the polynomial corresponding to the convolution kernel.

[0043] When the size of the convolution kernel exceeds the pre-set deployment size requirement, a Z-transform is performed on the kernel to obtain the corresponding polynomial. The Z-transform converts a time-domain sequence into a complex frequency-domain expression. For example, if the convolution kernel is a one-dimensional kernel h[n] of length N (n=0,1,…,N-1), the polynomial obtained after performing a Z-transform on it is: in, It is about The N-1 order polynomial.

[0044] Second, based on deployment size requirements, the polynomial is factored to obtain the factors contained in the polynomial.

[0045] This embodiment requires polynomial factorization based on pre-set deployment size requirements to obtain the factors contained in the polynomial. Specifically: First, find the zeros of the polynomial to obtain all the zeros of the polynomial; Zeros are the input values ​​that make the Z-transform result zero, that is, the input values ​​that make the polynomial... The z-value. This embodiment requires finding the zeros of the polynomial after the Z-transformation. According to the fundamental theorem of algebra, an N-1 order polynomial must have N-1 zeros (including repeated roots). For the polynomial in the above steps... It has N-1 zeros (including repeated roots) in the complex field. .

[0046] Furthermore, in practical calculations, zero-point solving is achieved by solving the eigenvalues ​​of the companion matrix. For higher-order polynomials (N>50), zero-point solving may exhibit numerical instability. Therefore, this embodiment employs the following strategy to ensure numerical accuracy: First, use double-precision floating-point numbers (float64) to solve for the zero point; Second, the zeros obtained are verified. This involves substituting the zeros back into the original polynomial and checking if the residual is less than a threshold. If the residual is less than the threshold, the zero has passed verification; otherwise, it has failed verification. The zero must be re-evaluated or marked as a valid zero. The threshold is preferably set to 10. -10 ; Third, pairing corrections are performed on conjugate complex zeros. If a zero has an imaginary part, a conjugate zero is found, and a forced correction is applied to the two conjugate complex zeros. Determining whether two zeros are conjugate involves calculating whether the difference between the absolute values ​​of their imaginary parts is less than a threshold. If it is less than the threshold, the two complex zeros are conjugate. The forced correction of conjugate complex zeros adjusts the absolute values ​​of their imaginary parts to their average.

[0047] Then, based on the deployment size requirements and all the zeros of the polynomial, the polynomial is factored to obtain the factors contained in the polynomial.

[0048] This embodiment can factorize the polynomial based on all its zeros, and decompose the factors corresponding to each zero as factors included in the polynomial.

[0049] For example, for the polynomial in the above steps , and the N-1 zeros (including repeated roots) of the polynomial. It can be decomposed into N-1 factors, as follows: in, It is the first element of the convolution kernel (i.e., the first coefficient of the polynomial). This represents the k-th zero point.

[0050] This embodiment can also group all the zeros of the polynomial based on the deployment size requirements, obtaining at least two groups of zeros. Then, the polynomial is factored according to each group of zeros, and the factors corresponding to the zeros in each group are used as factors included in the polynomial. The number of zeros in each group must be less than the deployment size requirement to ensure that the size of the sub-convolution kernel after the inverse Z-transform of the factors corresponding to the zero group meets the deployment size requirement. Furthermore, two conjugate zeros need to be grouped into the same zero group to ensure that the sub-kernel coefficients of the sub-convolution kernel after the inverse Z-transform of the factors corresponding to the zero group are real numbers. This embodiment, by grouping the zeros and then determining the factors corresponding to each group, reduces the number of sub-convolution kernels while ensuring that the sub-convolution kernels meet the deployment size requirements, compared to factoring out the factors corresponding to each zero individually. This improves the computational efficiency of the convolutional neural network after kernel decomposition.

[0051] Third, perform an inverse Z-transform on the factor to obtain the sub-convolution kernel corresponding to the factor.

[0052] After determining the factors contained in the polynomial in this embodiment, it is necessary to perform an inverse Z-transform on each factor to transform each factor back to the time domain and obtain the sub-convolution kernels corresponding to each factor.

[0053] S103. Concatenate the sub-convolutional kernels to obtain a cascaded convolutional neural network, and use the model composed of the cascaded convolutional neural network as the target model after compression.

[0054] In this embodiment, after decomposing the convolutional kernel to obtain at least two sub-convolutional kernels, the sub-convolutional kernels are concatenated through convolution operations to obtain a cascaded convolutional neural network. This cascaded convolutional neural network is a compressed version of the convolutional neural network. Replacing the convolutional neural network in the model to be compressed with this cascaded convolutional neural network yields the compressed target model. Concatenating the sub-convolutional kernels involves sequentially chaining multiple sub-convolutional kernels together, achieving the effect of a single large convolutional kernel through multiple consecutive convolutions. In this embodiment, the concatenation order of the sub-convolutional kernels can be directly based on the order of the zeros used in the factorization of the inverse Z-transform, or it can be based on the amplification capability parameter of each sub-convolutional kernel for input error. That is, the larger the amplification capability parameter, the later the kernel is in the concatenation order. This avoids amplifying the error of the preceding stages, reduces the error of the cascaded convolutional neural network, improves the accuracy of the target model, and ultimately improves the performance of the hardware deployed with the target model.

[0055] In this embodiment, after obtaining the target model after compression of the model to be compressed, the target model is deployed to a hardware device so that the hardware device can use the target model to perform corresponding functions. For example, if the model to be compressed is a speech processing model, then the target model is the compressed speech processing model. The target model is deployed to the hardware device, and the hardware device uses the target model to perform speech processing. If the model to be compressed is an image processing model, then the target model is the compressed image processing model. The target model is deployed to the hardware device, and the hardware device uses the target model to perform image processing. If the model to be compressed is a text processing model, then the target model is the compressed text processing model. The target model is deployed to the hardware device, and the hardware device uses the target model to perform text processing.

[0056] For example, the convolution operation of the cascaded convolutional neural network in the target model is as follows: in, Indicates the input signal. Indicates the output signal. This represents the convolution operation, where K represents the number of sub-convolutional kernels in the cascaded convolutional neural network. express The sub-convolutional kernel (i.e., the coefficient sequence of the i-th sub-convolutional kernel) is obtained after the inverse Z-transform. In a cascaded convolutional neural network, the input signal... First, the input is fed into the first sub-convolutional kernel for convolution. The output of the first sub-convolutional kernel is... Then output The input is fed into the second sub-convolutional kernel for convolution, and the output of the second sub-convolutional kernel is... This is then used as the input to the third sub-convolutional kernel, and so on, until the output of the last sub-convolutional kernel of the cascaded convolutional neural network. The output is used as the output signal of the cascaded convolutional neural network. When the model to be compressed is a speech processing model, the input signal is speech data and the output data is the speech processing result. When the model to be compressed is an image processing model, the input signal is image data and the output data is the image processing result. When the model to be compressed is a text processing model, the input signal is text data and the output data is the text processing result.

[0057] As can be seen from the above description, the convolution kernel optimization method based on Z-transform proposed in this application obtains the convolution kernel of the convolutional neural network in the model to be compressed; wherein, the model to be compressed includes: a speech processing model and / or an image processing model and / or a text processing model; based on the size of the convolution kernel and a pre-set deployment size requirement, the convolution kernel is decomposed to obtain at least two sub-convolution kernels; wherein, the size of the sub-convolution kernels meets the deployment size requirement, and the deployment size requirement matches the hardware resources of the hardware device used to deploy the model to be compressed; the sub-convolution kernels are concatenated to obtain a cascaded convolutional neural network, and the model composed of the cascaded convolutional neural network is used as the target model after compression of the model to be compressed, and the target model is used to perform speech processing and / or image processing and / or text processing after being deployed on the hardware device. The technical solution of this embodiment can decompose the convolutional kernels in the model to be compressed according to the deployment size requirements, decomposing them into several sub-convolutional kernels that meet the deployment size requirements. This achieves the decomposition and compression of the model to be compressed, ensuring that the decomposed and compressed target model can match the hardware resources of the domestic or low-performance hardware devices on which the model is deployed. After the target model is deployed on domestic or low-performance hardware devices, the resource consumption of the hardware devices can be reduced, the performance of the hardware devices can be improved, and the same computing power as that on high-performance hardware devices can be achieved on domestic or low-performance hardware devices. At the same time, by implementing the above solution, the normal deployment and operation of complex neural network models on domestic hardware devices can be achieved, making the operation of complex neural network models autonomous and controllable, and avoiding model security issues.

[0058] As an optional implementation method, see [link to implementation details]. Figure 2 As shown, another embodiment of this application discloses a method for factoring a polynomial based on deployment size requirements and all zeros of the polynomial to obtain the factors contained in the polynomial. This method specifically includes the following steps: S201. Based on the deployment size requirements, group all the zeros of the polynomial to obtain at least two groups of zeros.

[0059] This embodiment requires grouping all zeros of the polynomial based on the deployment size requirement to obtain at least two groups of zeros. During the grouping process, all zeros are grouped under the constraints that the number of zeros in a zero group is less than the deployment size requirement and that conjugate complex zeros are in the same zero group.

[0060] Specifically, the convolution kernel coefficients h[n] are real numbers, therefore, It is a polynomial with real coefficients. By the properties of polynomials with real coefficients, its zeros must satisfy one of the following two conditions: Case 1: Zero of a real number. Corresponding to first-order factor After expansion, the coefficients Since the zeros are real numbers, they can be grouped individually or in the same group as other zeros.

[0061] Case 2: Conjugate complex number with zero. and (b≠0) appear in pairs, corresponding to second-order factors: coefficient All are real numbers. Therefore, the zeros of complex numbers must be grouped in terms of conjugate pairs as the smallest unit, and the zeros of conjugate complex pairs cannot be split into different groups.

[0062] Therefore, the constraints for grouping zeros are as follows: real zeros can be grouped individually or merged with other zeros; and complex zeros must participate in grouping in conjugate pairs as the smallest unit, and conjugate pairs cannot be split into different groups. If conjugate complex zeros are split into different zero groups, the sub-kernel coefficients of the sub-convolution kernel will have imaginary parts and cannot be quantized to int8.

[0063] Furthermore, if a zero-point group contains m zeros, the sub-convolution kernel obtained by the inverse Z-transform of the corresponding factor of the zero-point group, after Z-transformation, becomes an m-th order polynomial, and the corresponding time-domain convolution kernel length is m+1. Therefore, when the deployment size requirement is L, the upper limit of the number of zeros in each group is L-1. Thus, the constraint condition for zero-point grouping is that the number of zeros in the zero-point group is less than the deployment size requirement. For example, a set of conjugate pairs of complex zeros occupies 2 zero-point positions, and a real zero occupies 1 zero-point position. If the deployment size requirement is L=16, then each zero-point group can contain a maximum of 15 zeros, that is, a maximum of 7 sets of conjugate pairs of complex zeros plus 1 real zero, or 15 real zeros.

[0064] In one specific implementation, based on the deployment size requirement, grouping all the zeros of the polynomial to obtain at least two groups of zeros may include the following steps: With the goal of minimizing the number of zeros in each zero-point group to the required deployment size and minimizing the dynamic range of the sub-kernel coefficients corresponding to each zero-point group, all zeros of the polynomial are grouped to obtain at least two zero-point groups. The dynamic range of the sub-kernel coefficients is determined based on the maximum and minimum zero-point moduli among the zero-point moduli of each zero in the zero-point group. If the zero-point moduli of all zeros are similar, the dynamic range of the sub-kernel coefficients is small; if there are zeros with significantly different zero-point moduli, the dynamic range of the sub-kernel coefficients is large. In this embodiment, the difference between the maximum and minimum zero-point moduli of each zero in the zero-point group can be used as the dynamic range of the sub-kernel coefficients, or the ratio between the maximum and minimum zero-point moduli of each zero in the zero-point group can be used as the dynamic range of the sub-kernel coefficients.

[0065] Specifically, in this embodiment, each real zero in all zeros is treated as a real zero-point unit, and two conjugate complex zeros in all zeros are treated as a conjugate pair zero-point unit. All zero-point units are sorted in ascending order of zero-point modulus, and then assigned sequentially to existing zero-point groups that minimize the dynamic range of the corresponding sub-kernel coefficients. If there are no available existing zero-point groups (e.g., the number of zeros in an existing zero-point group is full, or the increment of the dynamic range of the corresponding sub-kernel coefficients exceeds a preset threshold when the zero-point unit is assigned to all existing zero-point groups), a new group is created, and the zero-point unit is assigned to the new group. After all zero-point units are assigned in this way, all existing zero-point groups obtained are the final zero-point groups.

[0066] This embodiment can use a greedy approximation algorithm to achieve zero-point grouping. The specific steps are as follows: First, each real zero in the zero-point system is treated as a real zero-point unit (occupying 1 zero-point position). Two conjugate complex zeros in the zero-point system are treated as a conjugate pair zero-point unit (occupying 2 zero-point positions). All zero-point units are then sorted according to their zero-point modulus (e.g., ...). Where M represents the number of zero-point elements in all zeros, and the zero-point modulus of the conjugate zero-point element is taken as... (where a is the real part and b is the imaginary part).

[0067] Then, initialize the empty group list. For each sorted zero-point unit The following operations are performed: First, for each existing group Check capacity constraints L represents the deployment size requirement. If the capacity constraint is met, calculate the zero-point unit. Join Group Dynamic range of the subsequent daughter kernel coefficients ; Second, choose to increase the dynamic range of the daughter kernel coefficients. Minimum group ; Third, if all groups are full or the minimum dynamic range increment of the sub-kernel coefficients mentioned above is reached. If the threshold is exceeded, a new group will be created. and zero-point unit Add to the newly created group Add the newly created group to the group list. ; Fourth, otherwise, use the zero-point unit. Add the group that minimizes the dynamic range increment of the daughter kernel coefficients. .

[0068] Finally, after all zero-point units have undergone the above operations, all groups in the grouping list are treated as zero-point groups.

[0069] S202. Based on the zero-point group, factorize the polynomial to obtain the factors corresponding to the zero-point group.

[0070] In this embodiment, after grouping all zeros to obtain at least two zero-point groups, it is also necessary to determine the scaling factor corresponding to each zero-point group. The product of the scaling factors corresponding to all zero-point groups is the first element of the convolution kernel (i.e., the first term coefficient of the polynomial). In this embodiment, the scaling factor corresponding to the first group of zeros can be directly determined as the first element of the convolution kernel (i.e., the first coefficient of the polynomial). The scaling factor for all other zero-point groups is set to 1; the scaling factor for each zero-point group can also be evenly distributed.

[0071] This embodiment factorizes the polynomial based on each zero-point group and its corresponding scaling factor to obtain the factors corresponding to each zero-point group. For example, N-1 zeros are divided into K groups: ,in Include One zero point, satisfying Therefore, the factor corresponding to each group of zeros is: in, Denotes the scaling factor, and satisfies .

[0072] As an optional implementation method, see [link to implementation details]. Figure 3 As shown, in another embodiment of this application, step S103, which involves cascading the sub-convolutional kernels to obtain a cascaded convolutional neural network, specifically includes the following steps: S301. Determine the amplification capability parameter of the sub-convolution kernel for input error.

[0073] In cascaded convolutions, the output of each stage needs to be quantized back to int8 before being passed to the next stage. Let the kernel of the i-th stage be... The input is The output is : Quantization process introduces errors : in, For quantitative operations, For quantization error, satisfy . The scaling factor for int8 quantization is calculated independently for each sub-convolutional kernel. , This indicates the number of sub-convolution kernels.

[0074] Quantization error of level i As the input error of level i+1, its influence on the output after passing through level i+1 convolution is as follows: The upper bound of this error can be estimated using the L1 norm (gain) of the sub-convolution kernel: in Let L1 norm be the L1 norm of the (i+1)th level sub-convolution kernel. This represents the maximum value of the quantization error at level i.

[0075] For a K-level cascaded convolution (i.e., a cascaded convolution containing K sub-convolutional kernels), the quantization error introduced by the i-th level... After subsequent Ki-level sub-kernel propagation, the contribution to the error of the final output is: The upper bound of the total error is the sum of the contributions of each level of error: The agreement (The error at the last stage is not amplified by subsequent stages).

[0076] As shown above, the error contribution of the i-th stage is proportional to the gain of all subsequent sub-convolutional kernels. Therefore, sub-convolutional kernels with higher gains should be placed in later stages to reduce the chance of them amplifying the error of the preceding stages.

[0077] In this embodiment, the L1 norm of the sub-convolution kernel can be directly used as a parameter of the sub-convolution kernel's amplification capability for input errors. Alternatively, the product of the L1 norm of the sub-convolution kernel and the quantization error can be used as a parameter of the sub-convolution kernel's amplification capability for input errors. The formulas for calculating the L1 norm of the sub-convolution kernel and the quantization error have been given above.

[0078] S302. Based on the amplification capability parameter, sort all sub-convolution kernels to obtain the cascade order of all sub-convolution kernels.

[0079] Based on the amplification capability parameter of each sub-convolutional kernel for the input error, all sub-convolutional kernels are sorted to obtain the concatenation order of all sub-convolutional kernels. That is, all sub-convolutional kernels are sorted in ascending order of amplification capability parameter, that is, the concatenation order of sub-convolutional kernels is that the sub-convolutional kernel with the smaller input error amplification capability comes first, and the sub-convolutional kernel with the larger input error amplification capability comes last.

[0080] S303. By cascading the sub-convolutional kernels in a cascaded order, a cascaded convolutional neural network is obtained.

[0081] A concatenated convolutional neural network is formed by concatenating all sub-convolutional kernels in the order they are concatenated.

[0082] As an optional implementation method, see [link to implementation details]. Figure 4 As shown, in another embodiment of this application, step S103, which involves cascading the sub-convolutional kernels to obtain a cascaded convolutional neural network, specifically includes the following steps: S401. According to the deployment scenario of the model to be compressed, the sub-kernel coefficients of the sub-convolution kernel are quantized to obtain the quantized sub-convolution kernel.

[0083] This embodiment first needs to determine the deployment scenario of the model to be compressed, and then quantize the sub-kernel coefficients of the sub-convolution kernel according to the deployment scenario of the model to be compressed, so as to obtain the quantized sub-convolution kernel. For example, when the deployment scenario is an NPU accelerator, DSP processor, etc., the sub-kernel coefficients of the sub-convolution kernel need to be quantized to int8 (8-bit signed integer, range [-128, 127]).

[0084] The coefficients of the sub-convolution kernel are floating-point numbers, and the quantization process to int8 is as follows: Where w represents the coefficients of the sub-convolution kernel, i.e., the floating-point coefficients. The quantized integer value. The scaling factor is used to calculate the quantization error. The quantization error is directly proportional to the maximum absolute value of the sub-kernel coefficients (i.e., the dynamic range). The smaller the dynamic range, the smaller the quantization step size, and the smaller the quantization error.

[0085] S402. Concatenate the quantized sub-convolutional kernels to obtain a cascaded convolutional neural network.

[0086] After quantizing each sub-convolutional kernel, convolution operations are performed on the quantized sub-convolutional kernels in a cascaded manner to obtain a cascaded convolutional neural network. Specifically, firstly, the amplification capability parameter of the sub-convolutional kernel for input error is determined; where the amplification capability parameter is the L1 norm of the sub-convolutional kernel, or the product of the L1 norm of the sub-convolutional kernel and the quantization error; then, based on the amplification capability parameter, all sub-convolutional kernels are sorted to obtain the cascaded order of all sub-convolutional kernels; finally, convolution operations are performed on the quantized sub-convolutional kernels in the cascaded order to obtain the cascaded convolutional neural network.

[0087] As an optional implementation method, see [link to implementation details]. Figure 5 As shown, in another embodiment of this application, if the convolution kernel is a two-dimensional convolution kernel, step S102 involves decomposing the convolution kernel based on its size and a pre-set deployment size requirement to obtain at least two sub-convolution kernels. This specifically includes the following steps: S501. Determine the dimensions to be decomposed for the convolution kernel based on the kernel size and the pre-set deployment size requirements.

[0088] If the convolution kernel is a two-dimensional convolution kernel, this embodiment needs to compare and analyze the row size and column size of the convolution kernel with the pre-set deployment size requirements. If the row size of the convolution kernel is larger than the deployment size requirement, it is determined that the dimension to be decomposed of the convolution kernel includes the row dimension. If the column size of the convolution kernel is larger than the deployment size requirement, it is determined that the dimension to be decomposed of the convolution kernel includes the column dimension. If the row size of the convolution kernel is larger than the deployment size requirement and the column size of the convolution kernel is also larger than the deployment size requirement, it is determined that the dimension to be decomposed of the convolution kernel includes both the row dimension and the column dimension.

[0089] S502. Perform convolution kernel decomposition on the dimension to be decomposed to obtain at least two sub-convolution kernels corresponding to the dimension to be decomposed.

[0090] This embodiment requires decomposing the convolution kernel along the dimension to be decomposed to obtain at least two sub-convolution kernels corresponding to the dimension to be decomposed. That is, if the dimension to be decomposed only contains the row dimension, then the convolution kernel needs to be decomposed along the row dimension to obtain at least two sub-convolution kernels corresponding to the dimension to be decomposed. If the dimension to be decomposed only contains the column dimension, then the convolution kernel needs to be decomposed along the column dimension to obtain at least two sub-convolution kernels corresponding to the dimension to be decomposed. If the dimension to be decomposed contains both the row and column dimensions, then the convolution kernel needs to be decomposed along the column dimension first to obtain at least two sub-convolution kernels corresponding to the column dimension, and then the convolution kernels corresponding to each sub-convolution kernel along the row dimension need to be decomposed to obtain at least two sub-convolution kernels corresponding to the dimension to be decomposed.

[0091] Specifically, if the dimension to be decomposed only contains column dimensions, the convolution kernel is decomposed along the column dimensions to obtain at least two sub-convolution kernels corresponding to the dimension to be decomposed. The specific steps are as follows: Perform a Z-transform on each row of the convolution kernel to obtain the polynomial corresponding to each row of the convolution kernel. Then, find the zeros of the polynomial corresponding to each row to obtain the zeros of the polynomial corresponding to each row. Based on the zeros of the polynomial corresponding to each row, factorize the polynomial corresponding to each row of the convolution kernel to obtain the factors corresponding to the column dimension. Perform an inverse Z-transform on the factors corresponding to the column dimension to obtain the sub-convolution kernel corresponding to the column dimension.

[0092] The process of factoring the polynomial corresponding to each row of the convolution kernel based on the zeros of the polynomial corresponding to each row to obtain the factors corresponding to the column dimensions includes: determining the zero-point grouping rule based on the zeros of the polynomial corresponding to each row; grouping the zeros of the polynomial corresponding to each row according to the zero-point grouping rule to obtain at least two groups of zeros corresponding to each row; and factoring the polynomial corresponding to each row according to the at least two groups of zeros corresponding to each row to obtain at least two groups of factors corresponding to the column dimensions, wherein the number of factors is the same as the number of groups of zeros corresponding to each row. In addition, based on the zeros of the polynomial corresponding to each row, a zero-point grouping rule is determined. The zeros of the polynomial corresponding to the first row can be directly grouped, and this grouping rule can be directly used as the zero-point grouping rule. The execution method of grouping the zeros of the polynomial corresponding to the first row is the same as the execution method of "grouping all zeros of the polynomial based on the deployment size requirements to obtain at least two groups of zeros" in the above embodiment, and will not be repeated in this embodiment. Alternatively, the zeros of the polynomial corresponding to all rows can be clustered, and the cluster center can be taken as the grouping basis zero. The grouping basis zero is then grouped according to the grouping execution method in the above embodiment to obtain the zero-point grouping rule.

[0093] If the dimension to be decomposed only contains row dimensions, perform convolution kernel decomposition on the row dimensions to obtain at least two convolution kernels corresponding to the dimension to be decomposed. Then, perform a Z-transform on each column of the convolution kernel to obtain the zeros of the polynomial corresponding to each column. Based on the zeros of the polynomial corresponding to each column, perform factorization on the polynomial corresponding to each column of the convolution kernel to obtain the factors corresponding to the row dimensions. Perform an inverse Z-transform on the factors corresponding to the row dimensions to obtain the sub-convolution kernels corresponding to the row dimensions. The specific execution method is similar to the method for obtaining the sub-convolution kernels corresponding to the column dimensions described above, and will not be repeated in this embodiment.

[0094] If the dimension to be decomposed includes both row and column dimensions, then first perform convolution kernel decomposition on the column dimension according to the above method for decomposing convolution kernels when the dimension to be decomposed only includes column dimensions, to obtain at least two sub-convolution kernels corresponding to the column dimension. Then, perform convolution kernel decomposition on the row dimension according to the above method for decomposing convolution kernels when the dimension to be decomposed only includes row dimensions, to obtain at least two sub-convolution kernels corresponding to the dimension to be decomposed.

[0095] Correspondingly, if the convolution kernel is a two-dimensional kernel, and the dimension to be decomposed includes both row and column dimensions, the concatenation of the sub-convolution kernels needs to be implemented according to a preset dimension concatenation order. The preferred preset dimension concatenation order is column dimensions first, then row dimensions. Therefore, when concatenating the convolution operations of the sub-convolution kernels corresponding to the dimension to be decomposed, the column-direction sub-convolution kernels are concatenated first, followed by the row-direction sub-convolution kernels.

[0096] See Figure 6 As shown in the figure, this application proposes a speech processing method. The method includes: S601. Input the speech data into the speech processing model to obtain the speech processing result corresponding to the speech data.

[0097] In this embodiment, the speech processing model is obtained by compression using the Z-transform-based convolution kernel decomposition and concatenated quantization optimization method described in the previous embodiment. The speech processing model is deployed on a hardware device to enable the hardware device to perform speech processing. Speech data is input into the speech processing model of the hardware device to obtain the speech processing result corresponding to the speech data.

[0098] In this embodiment, the speech processing model is obtained by decomposing the convolutional kernels in the speech processing model to be compressed into several sub-convolutional kernels that meet the deployment size requirements of the hardware device. This speech processing model can match the hardware resources of domestic or low-performance hardware devices where it is deployed. After being deployed on such hardware devices, the speech processing model can reduce the consumption of hardware resources, improve hardware performance, and achieve the same computing power as high-performance hardware devices on domestic or low-performance hardware devices. At the same time, by implementing the above scheme to achieve the normal deployment and operation of complex speech processing models on domestic hardware devices, the operation of complex speech processing models can be autonomously controlled, avoiding model security issues.

[0099] See Figure 7 As shown in the figure, this application provides an image processing method. The method includes: S701. Input the image data into the image processing model to obtain the image processing result corresponding to the image data.

[0100] In this embodiment, the image processing model is obtained by compression using the Z-transform-based convolution kernel decomposition and concatenated quantization optimization method described in the previous embodiment. The image processing model is deployed on a hardware device to enable the hardware device to perform image processing. Image data is input into the image processing model of the hardware device to obtain the image processing result corresponding to the image data.

[0101] In this embodiment, the image processing model is obtained by decomposing the convolutional kernels in the image processing model to be compressed into several sub-convolutional kernels that meet the deployment size requirements of the hardware device. This image processing model can match the hardware resources of domestic or low-performance hardware devices that deploy the model. After the image processing model is deployed on such hardware devices, it can reduce the consumption of hardware resources, improve the performance of the hardware devices, and achieve the same computing power as high-performance hardware devices on domestic or low-performance hardware devices. At the same time, by implementing the above scheme to achieve the normal deployment and operation of complex image processing models on domestic hardware devices, the operation of complex image processing models can be autonomously controlled, avoiding model security issues.

[0102] See Figure 8 As shown in the figure, this application proposes a text processing method. The method includes: S801. Input the text data into the text processing model to obtain the text processing result corresponding to the text data.

[0103] In this embodiment, the text processing model is obtained by compression using the Z-transform-based convolution kernel optimization method described in the previous embodiment. The text processing model is deployed on a hardware device to enable the device to perform text processing. Text data is input into the text processing model on the hardware device to obtain the corresponding text processing result.

[0104] In this embodiment, the text processing model is obtained by decomposing the convolutional kernels in the text processing model to be compressed into several sub-convolutional kernels that meet the deployment size requirements of the hardware device. This text processing model can match the hardware resources of domestic or low-performance hardware devices that deploy the model. After being deployed on such hardware devices, the text processing model can reduce the consumption of hardware resources, improve hardware performance, and achieve the same computing power as high-performance hardware devices on domestic or low-performance hardware devices. At the same time, by implementing the above scheme to achieve the normal deployment and operation of complex text processing models on domestic hardware devices, the operation of complex text processing models can be autonomously controlled, avoiding model security issues.

[0105] Exemplary device Accordingly, embodiments of this application also provide a convolution kernel optimization device based on Z-transform, see [link to relevant documentation]. Figure 9 As shown, the device includes: The acquisition module 100 is used to acquire the convolution kernels of the convolutional neural network in the model to be compressed; wherein, the model to be compressed includes: a speech processing model and / or an image processing model and / or a text processing model; The decomposition module 110 is used to decompose the convolution kernel based on the kernel size and a pre-set deployment size requirement to obtain at least two sub-convolution kernels; wherein the size of the sub-convolution kernels conforms to the deployment size requirement; and the deployment size requirement matches the hardware resources of the hardware device used to deploy the model to be compressed. The cascade module 120 is used to cascade the sub-convolutional kernels to obtain a cascaded convolutional neural network, and the model composed of the cascaded convolutional neural network is used as the target model after compression of the model to be compressed. The target model is used to perform speech processing and / or image processing and / or text processing after being deployed on hardware devices.

[0106] As described above, the Z-transform-based convolutional kernel optimization device proposed in this application can decompose the convolutional kernels in the model to be compressed according to the deployment size requirements, decomposing them into several sub-convolutional kernels that meet the deployment size requirements. This achieves the decomposition and compression of the model to be compressed, ensuring that the decomposed and compressed target model can match the hardware resources of domestic or low-performance hardware devices where the model is deployed. Deploying the target model on domestic or low-performance hardware devices reduces resource consumption and improves hardware performance, enabling the same computing power as high-performance hardware devices. Furthermore, the device can normally deploy and run complex neural network models on domestic hardware devices, ensuring the autonomous and controllable operation of complex neural network models and avoiding model security issues.

[0107] As an optional implementation, another embodiment of this application discloses that the decomposition module 110 includes: a Z-transform unit, a factorization unit, and an inverse Z-transform unit.

[0108] The Z-transform unit is used to perform a Z-transform on the convolution kernel when the size of the convolution kernel is larger than the preset deployment size requirement, so as to obtain the polynomial corresponding to the convolution kernel. The factorization unit is used to perform polynomial factorization on a polynomial based on deployment size requirements to obtain the factors contained in the polynomial. The inverse Z-transform unit is used to perform an inverse Z-transform on the factor to obtain the sub-convolution kernel corresponding to the factor.

[0109] As an optional implementation, another embodiment of this application discloses a factorization unit, specifically used for: Find the zeros of the polynomial to obtain all the zeros of the polynomial; Based on the deployment size requirements and all zeros of the polynomial, the polynomial is factored to obtain the factors contained in the polynomial.

[0110] As an optional implementation, another embodiment of this application discloses factoring the polynomial based on deployment size requirements and all zeros of the polynomial to obtain the factors contained in the polynomial, including: Based on the deployment size requirements, all zeros of the polynomial are grouped to obtain at least two groups of zeros; wherein the number of zeros in a zero group is less than the deployment size requirements, and the complex zeros that are conjugates of each other are in the same zero group. Based on the zero-point group, the polynomial is factored to obtain the factors corresponding to the zero-point group.

[0111] As an optional implementation, another embodiment of this application discloses grouping all zeros of the polynomial based on deployment size requirements to obtain at least two groups of zeros, including: With the goal of minimizing the number of zeros in each zero group to the required deployment size and minimizing the dynamic range of the sub-kernel coefficients corresponding to each zero group, all zeros of the polynomial are grouped to obtain at least two zero groups; the dynamic range of the sub-kernel coefficients is determined based on the maximum and minimum zero moduli among the zero moduli of each zero in the zero group.

[0112] As an optional implementation, another embodiment of this application discloses a cascade module 120, specifically used for: Determine the amplification capability parameter of the sub-convolution kernel for the input error; wherein the amplification capability parameter is the L1 norm of the sub-convolution kernel, or the product of the L1 norm of the sub-convolution kernel and the quantization error; Based on the amplification capability parameter, all sub-convolutional kernels are sorted to obtain the concatenation order of all sub-convolutional kernels; By cascading the sub-convolutional kernels in a cascaded order, a cascaded convolutional neural network is obtained.

[0113] As an optional implementation, another embodiment of this application discloses a cascade module 120, specifically used for: According to the deployment scenario of the model to be compressed, the sub-kernel coefficients of the sub-convolution kernel are quantized to obtain the quantized sub-convolution kernel; By cascading the quantized sub-convolutional kernels through convolution operations, a cascaded convolutional neural network is obtained.

[0114] As an optional implementation, another embodiment of this application discloses that, if the convolution kernel is a two-dimensional convolution kernel, the decomposition module 110 is specifically used for: Based on the size of the convolution kernel and the pre-set deployment size requirements, determine the dimensions to be decomposed for the convolution kernel; Decompose the convolution kernel on the dimension to be decomposed to obtain at least two sub-convolution kernels corresponding to the dimension to be decomposed.

[0115] The Z-transform-based convolution kernel optimization device provided in this embodiment belongs to the same concept as the Z-transform-based convolution kernel optimization method provided in the above embodiments of this application. It can execute the Z-transform-based convolution kernel optimization method provided in any of the above embodiments of this application, and has the corresponding functional modules and beneficial effects of executing the Z-transform-based convolution kernel optimization method. Technical details not described in detail in this embodiment can be found in the specific processing content of the Z-transform-based convolution kernel optimization method provided in the above embodiments of this application, and will not be repeated here.

[0116] Accordingly, embodiments of this application also provide a voice processing device, which includes: The voice input module is used to input voice data into the voice processing model and obtain the voice processing result corresponding to the voice data; The speech processing model is obtained by compression using the Z-transform-based convolution kernel optimization method described above.

[0117] The speech processing apparatus provided in this embodiment belongs to the same concept as the speech processing method provided in the above embodiments of this application. It can execute the speech processing method provided in any of the above embodiments of this application and has the corresponding functional modules and beneficial effects for executing the speech processing method. Technical details not described in detail in this embodiment can be found in the specific processing content of the speech processing method provided in the above embodiments of this application, and will not be repeated here.

[0118] Accordingly, embodiments of this application also provide an image processing apparatus, which includes: The image input module is used to input image data into the image processing model and obtain the image processing result corresponding to the image data; The image processing model is obtained by compressing the image using the Z-transform-based convolution kernel optimization method described above.

[0119] The image processing apparatus provided in this embodiment belongs to the same concept as the image processing method provided in the above embodiments of this application. It can execute the image processing method provided in any of the above embodiments of this application and has the corresponding functional modules and beneficial effects for executing the image processing method. Technical details not described in detail in this embodiment can be found in the specific processing content of the image processing method provided in the above embodiments of this application, and will not be repeated here.

[0120] Accordingly, embodiments of this application also provide a text processing apparatus, which includes: The text input module is used to input text data into the text processing model and obtain the text processing result corresponding to the text data; The text processing model is obtained by compressing the text using the Z-transform-based convolution kernel optimization method described above.

[0121] The text processing apparatus provided in this embodiment belongs to the same concept as the text processing method provided in the above embodiments of this application. It can execute the text processing method provided in any of the above embodiments of this application and has the corresponding functional modules and beneficial effects for executing the text processing method. Technical details not described in detail in this embodiment can be found in the specific processing content of the text processing method provided in the above embodiments of this application, and will not be repeated here.

[0122] Exemplary electronic devices Another embodiment of this application also provides an electronic device, see [link to relevant documentation] Figure 10 As shown, the device includes: Memory 200 and processor 210; The memory 200 is connected to the processor 210 and is used to store programs; The processor 210 is configured to implement the Z-transform-based convolution kernel optimization method, speech processing method, image processing method, or text processing method disclosed in any of the above embodiments by running the program stored in the memory 200.

[0123] Specifically, the aforementioned electronic device may also include: a bus, a communication interface 220, an input device 230, and an output device 240.

[0124] The processor 210, memory 200, communication interface 220, input device 230, and output device 240 are interconnected via a bus. Among them: A bus can include a pathway for transmitting information between various components of a computer system.

[0125] Processor 210 can be a general-purpose processor, such as a general-purpose central processing unit (CPU), a microprocessor, etc., or an application-specific integrated circuit (ASIC), or one or more integrated circuits used to control the execution of the program of the present invention. It can also be a digital signal processor (DSP), an application-specific integrated circuit (ASIC), an off-the-shelf programmable gate array (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components.

[0126] Processor 210 may include a main processor, as well as a baseband chip, modem, etc.

[0127] The memory 200 stores a program that executes the technical solution of this invention, and may also store an operating system and other key business functions. Specifically, the program may include program code, which includes computer operation instructions. More specifically, the memory 200 may include read-only memory (ROM), other types of static storage devices capable of storing static information and instructions, random access memory (RAM), other types of dynamic storage devices capable of storing information and instructions, disk storage, flash memory, etc.

[0128] Input device 230 may include a device for receiving user input data and information, such as a keyboard, mouse, camera, scanner, light pen, voice input device, touch screen, pedometer, or gravity sensor.

[0129] Output device 240 may include devices that allow information to be output to a user, such as a display screen, printer, speaker, etc.

[0130] The communication interface 220 may include a device that uses any transceiver to communicate with other devices or communication networks, such as Ethernet, Radio Access Network (RAN), Wireless Local Area Network (WLAN), etc.

[0131] The processor 210 executes the program stored in the memory 200 and calls other devices, which can be used to implement any step of any Z-transform-based convolution kernel optimization method, any step of any speech processing method, any step of any image processing method, or any step of any text processing method provided in the above embodiments of this application.

[0132] Exemplary computer program products and storage media In addition to the methods and devices described above, embodiments of this application may also be computer program products, which include computer program instructions that, when executed by a processor, cause the processor to perform steps in the Z-transform-based convolution kernel optimization method, the speech processing method, the image processing method, or the text processing method described in the "Exemplary Methods" section above.

[0133] The computer program product can be written in any combination of one or more programming languages ​​to perform the operations of the embodiments of this application. The programming languages ​​include object-oriented programming languages ​​such as Java and C++, as well as conventional procedural programming languages ​​such as C or similar languages. The program code can be executed entirely on the user's computing device, partially on the user's computing device, as a standalone software package, partially on the user's computing device and partially on a remote computing device, or entirely on a remote computing device or server.

[0134] Furthermore, embodiments of this application may also be storage media storing computer programs, which are executed by a processor from steps in the Z-transform-based convolution kernel optimization method, speech processing method, image processing method, or text processing method described in the "Exemplary Methods" section above.

[0135] For the foregoing method embodiments, in order to simplify the description, they are all described as a series of actions. However, those skilled in the art should understand that this application is not limited to the described order of actions, because according to this application, some steps can be performed in other orders or simultaneously. Furthermore, those skilled in the art should also understand that the embodiments described in the specification are all preferred embodiments, and the actions and modules involved are not necessarily essential to this application.

[0136] It should be noted that the various embodiments in this specification are described in a progressive manner, with each embodiment focusing on the differences from other embodiments. Similar or identical parts between embodiments can be referred to interchangeably. For apparatus embodiments, since they are basically similar to method embodiments, the description is relatively simple; relevant parts can be referred to the descriptions in the method embodiments.

[0137] The steps in the methods of the various embodiments of this application can be adjusted, merged, or deleted in order according to actual needs, and the technical features described in each embodiment can be replaced or combined.

[0138] The modules and sub-modules in the various embodiments of the present application's devices and terminals can be merged, divided, and deleted according to actual needs.

[0139] It should be understood that the disclosed terminals, devices, and methods can be implemented in other ways, given the several embodiments provided in this application. For example, the terminal embodiments described above are merely illustrative. For instance, the division of modules or sub-modules is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple sub-modules or modules may be combined or integrated into another module, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be indirect coupling or communication connection through some interfaces, devices, or modules, and may be electrical, mechanical, or other forms.

[0140] The modules or submodules described as separate components may or may not be physically separate. The components that constitute a module or submodule may or may not be physical modules or submodules; that is, they may be located in one place or distributed across multiple network modules or submodules. Some or all of the modules or submodules can be selected to achieve the purpose of this embodiment's solution, depending on actual needs.

[0141] Furthermore, the functional modules or sub-modules in the various embodiments of this application can be integrated into one processing module, or each module or sub-module can exist physically separately, or two or more modules or sub-modules can be integrated into one module. The integrated modules or sub-modules described above can be implemented in hardware or in the form of software functional modules or sub-modules.

[0142] Those skilled in the art will further recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, computer software, or a combination of both. To clearly illustrate the interchangeability of hardware and software, the components and steps of the various examples have been generally described in terms of functionality in the foregoing description. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0143] The steps of the methods or algorithms described in conjunction with the embodiments disclosed herein can be implemented directly by hardware, a software unit executed by a processor, or a combination of both. The software unit can be located in random access memory (RAM), main memory, read-only memory (ROM), electrically programmable ROM, electrically erasable programmable ROM, registers, hard disk, removable disk, CD-ROM, or any other form of storage medium known in the art.

[0144] Finally, it should be noted that in this document, relational terms such as "first" and "second" are used only to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.

[0145] The above description of the disclosed embodiments enables those skilled in the art to make or use this application. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of this application. Therefore, this application is not to be limited to the embodiments shown herein, but is to be accorded the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. A convolution kernel optimization method based on Z-transform, characterized in that, include: Obtain the convolution kernels of the convolutional neural network in the model to be compressed; wherein, the model to be compressed includes: a speech processing model and / or an image processing model and / or a text processing model; When the size of the convolution kernel is larger than the preset deployment size requirement, a Z-transform is performed on the convolution kernel to obtain the polynomial corresponding to the convolution kernel; Based on the deployment size requirements, the polynomial is factored to obtain the factors contained in the polynomial; The factor is subjected to an inverse Z-transform to obtain the sub-convolution kernel corresponding to the factor; wherein the size of the sub-convolution kernel conforms to the deployment size requirement; the deployment size requirement matches the hardware resources of the hardware device used to deploy the model to be compressed; The concatenation of the sub-convolutional kernels is performed to obtain a concatenated convolutional neural network. The model composed of the concatenated convolutional neural network is used as the target model after compression. The target model is used to perform speech processing and / or image processing and / or text processing after being deployed on a hardware device.

2. The convolution kernel optimization method based on Z-transform according to claim 1, characterized in that, Based on the deployment size requirements, the polynomial is factored to obtain the factors contained in the polynomial, including: Find the zeros of the polynomial to obtain all the zeros of the polynomial. Based on the deployment size requirements and all the zeros of the polynomial, the polynomial is factored to obtain the factors contained in the polynomial.

3. The convolution kernel optimization method based on Z-transform according to claim 2, characterized in that, Based on the deployment size requirements and all zeros of the polynomial, the polynomial is factored to obtain the factors contained in the polynomial, including: Based on the deployment size requirement, all zeros of the polynomial are grouped to obtain at least two groups of zeros; wherein the number of zeros in each group of zeros is less than the deployment size requirement, and complex zeros that are conjugates of each other are in the same group of zeros. Based on the set of zeros, the polynomial is factored to obtain the factors corresponding to the set of zeros.

4. The convolution kernel optimization method based on Z-transform according to claim 3, characterized in that, Based on the deployment size requirements, all zeros of the polynomial are grouped to obtain at least two groups of zeros, including: With the goal of minimizing the number of zeros in each zero group to less than the deployment size requirement and minimizing the dynamic range of the sub-kernel coefficients corresponding to each zero group, all zeros of the polynomial are grouped to obtain at least two zero groups; the dynamic range of the sub-kernel coefficients is determined based on the maximum and minimum zero moduli among the zero moduli of each zero in the zero group.

5. The convolution kernel optimization method based on Z-transform according to claim 1, characterized in that, The concatenation of the sub-convolutional kernels yields a cascaded convolutional neural network, comprising: Determine the amplification capability parameter of the sub-convolution kernel for the input error; wherein the amplification capability parameter is the L1 norm of the sub-convolution kernel, or the product of the L1 norm of the sub-convolution kernel and the quantization error; Based on the amplification capability parameters, all sub-convolutional kernels are sorted to obtain the concatenated order of all sub-convolutional kernels; By cascading the sub-convolutional kernels according to the cascading order, a cascaded convolutional neural network is obtained.

6. The convolution kernel optimization method based on Z-transform according to claim 1, characterized in that, The concatenation of the sub-convolutional kernels yields a cascaded convolutional neural network, comprising: According to the deployment scenario of the model to be compressed, the sub-kernel coefficients of the sub-convolution kernel are quantized to obtain the quantized sub-convolution kernel; By cascading the quantized sub-convolutional kernels through convolution operations, a cascaded convolutional neural network is obtained.

7. The convolution kernel optimization method based on Z-transform according to claim 1, characterized in that, If the convolution kernel is a two-dimensional convolution kernel, based on the size of the convolution kernel and the pre-set deployment size requirements, the convolution kernel is decomposed to obtain at least two sub-convolution kernels, including: The dimensions to be decomposed of the convolution kernel are determined based on the kernel size and the pre-set deployment size requirements. The convolution kernel is decomposed along the dimension to be decomposed to obtain at least two sub-convolution kernels corresponding to the dimension to be decomposed.

8. A speech processing method, characterized in that, include: The speech data is input into the speech processing model to obtain the speech processing result corresponding to the speech data; The speech processing model is obtained by optimizing the convolution kernel based on Z-transform as described in any one of claims 1-7.

9. An image processing method, characterized in that, include: The image data is input into the image processing model to obtain the image processing result corresponding to the image data; The image processing model is obtained by optimizing the convolution kernel based on Z-transform as described in any one of claims 1-7.

10. A text processing method, characterized in that, include: The text data is input into the text processing model to obtain the text processing result corresponding to the text data. The text processing model is obtained by optimizing the convolution kernel based on Z-transform as described in any one of claims 1-7.

11. An electronic device, characterized in that, include: Memory and processor; The memory is connected to the processor and is used to store programs; The processor is configured to implement, by running a program in the memory, the Z-transform-based convolution kernel optimization method as described in any one of claims 1 to 7, the speech processing method as described in claim 8, the image processing method as described in claim 9, or the text processing method as described in claim 10.

12. A computer program product, characterized in that, It includes computer program instructions that, when executed by a processor, cause the processor to implement the Z-transform-based convolution kernel optimization method as described in any one of claims 1 to 7, the speech processing method as described in claim 8, the image processing method as described in claim 9, or the text processing method as described in claim 10.

Citation Information

Patent Citations

  • Meter defect detection method based on efficient local attention and large separable nuclear attention

    CN119991545A