Data processing method and device based on plural neural network models

Abstract

The invention provides a data processing method and device based on a plurality of neural network models. The method comprises the following steps: acquiring input data; inputting the input data into a plurality of neural network models to obtain a task result output by the plurality of neural network models; wherein the complex neural network model comprises a generalized multi-linear layer; the generalized complex linear layer is used for performing complex linear mapping operation on an input complex input vector and performing inverse linear mapping operation on a conjugate vector of the complex input vector; the multi-linear mapping operation performs mapping operation through a trainable first complex number parameter, and the inverse linear mapping operation performs mapping operation through a trainable second complex number parameter. According to the method, a high-performance complex neural network model can be realized, the data processing capability of a computer is improved, and the accuracy of a task result is further improved.

Description

Technical Field

[0001] This invention relates to the field of computer technology, and in particular to a data processing method and apparatus based on a complex neural network model. Background Technology

[0002] In recent years, deep learning technology, especially large-scale neural network architectures represented by the Transformer model, has achieved remarkable success in fields such as natural language processing, speech recognition, and multimodal learning. Current mainstream deep learning systems are almost entirely built on real-valued vector spaces.

[0003] However, in many important scientific and engineering fields, signals or data themselves naturally possess complex structures. For example, the first-order representations of signals such as speech, radar, and wireless communications are generally derived from the time-frequency characteristics of the complex domain, such as the complex spectrum obtained by the Short-Time Fourier Transform (STFT), which naturally contains real and imaginary parts, i.e., amplitude and phase information. Furthermore, in fundamental scientific computing, such as the wave function in quantum mechanics and the wave equation of many-body systems, their mathematical descriptions are essentially defined in complex vector spaces. When mainstream real-number neural networks are used to process these tasks, the models cannot directly model structural features such as phase coupling and conjugate symmetry in the complex plane, often exhibiting insufficient expressive power, optimization difficulties, or numerical instability. To address the limitations of real-number neural network models, academia and industry have attempted to introduce complex-number neural networks; therefore, how to construct complex-number neural networks is a pressing technical need.

[0004] Currently, most complex neural networks employ naive complex linear layers, performing complex linear mappings only on the complex input vectors at the linear transformation level. However, models based on single-path complex linear mappings, which strictly adhere to the Cauchy–Riemann conditions, only cover a finite subset of affine transformations in the complex domain. When faced with physical signals or quantum states exhibiting widespread conjugate coupling, their expressive power is inherently limited, failing to fully capture the intrinsic structure of the data. This restricts the modeling performance of complex neural network models, resulting in lower accuracy of task results when computers process data using complex neural network models to perform related tasks. Summary of the Invention

[0005] This invention provides a data processing method and apparatus based on a complex neural network model to address the limitations of existing complex neural network models in terms of modeling performance and accuracy of task results. It enables high-performance complex neural network models, improves the data processing capabilities of computers, and thereby enhances the accuracy of task results.

[0006] This invention provides a data processing method based on a complex neural network model, comprising: Get the input data; The input data is fed into a complex neural network model to obtain the task result output by the complex neural network model; The complex neural network model includes a generalized complex linear layer; the generalized complex linear layer is used to perform complex linear mapping operations on the input complex input vector and to perform inverse linear mapping operations on the conjugate vector of the complex input vector; the complex linear mapping operation is performed through a trainable first complex parameter, and the inverse linear mapping operation is performed through a trainable second complex parameter.

[0007] According to the present invention, a data processing method based on a complex neural network model is provided, wherein the generalized complex linear layer outputs a real number output vector or a complex number output vector; The real number output vector is determined based on the following method: The real part vector and the imaginary part vector of the complex input vector are concatenated to obtain the real input vector, and the real parameters are determined based on the first complex parameter and the second complex parameter. Based on the real number input vector and the real number parameters, determine the real number output vector; The complex output vector is determined based on the following method: The real number output vector is split to obtain a first real number output component and a second real number output component; The complex output vector is determined based on the first real number output component and the second real number output component.

[0008] According to a data processing method based on a complex neural network model provided by the present invention, determining real parameters based on the first complex parameter and the second complex parameter includes: Based on the first complex parameter and the second complex parameter, four real number sub-parameters are generated; The four real number sub-parameters are combined to obtain the real number parameters; The real number parameters are as follows: ; In the formula, Represents the real number parameter, , , , Represents the four real number sub-parameters, This represents the real part of the first complex number parameter. This represents the imaginary part of the first complex number parameter. This represents the real part of the second complex number parameter. This represents the imaginary part of the second complex number parameter.

[0009] According to the data processing method based on a complex neural network model provided by the present invention, when the first complex parameter and the second complex parameter are generated based on the real parameters of a real linear layer, the first complex parameter and the second complex parameter are generated in the following manner: The real number parameters are divided into blocks to obtain four real number sub-parameters; Based on the four real number sub-parameters, the first complex number parameter and the second complex number parameter are generated; Wherein, the input dimension of the real linear layer is twice the input dimension of the generalized complex linear layer, the real part of the complex input vector is regarded as the first component, and the imaginary part of the complex input vector is regarded as the second component; the first component and the second component are obtained by splitting the real input vector input to the real linear layer.

[0010] According to the data processing method based on a complex neural network model provided by the present invention, the real parameters are as follows: ; In the formula, Represents the real number parameter, , , , Represents the four real number sub-parameters; The first complex parameter is generated based on the following formula: ; ; ; In the formula, Represents the first complex number parameter. This represents the real part of the first complex number parameter. The parameter represents the imaginary part of the first complex number parameter; The second complex parameter is generated based on the following formula: ; ; ; In the formula, This represents the second complex parameter. This represents the real part of the second complex number parameter. This represents the imaginary part of the second complex number parameter.

[0011] According to the present invention, a data processing method based on a complex neural network model is provided, wherein the complex neural network model includes an attention layer; the attention layer includes three generalized complex linear layers; The attention layer outputs attention results based on the following method: Based on the three generalized complex linear layers, the input complex vector of the attention layer is mapped to the query complex vector, the key complex vector, and the value complex vector, respectively. Calculate the attention score between the query complex vector and the key complex vector; Based on the attention score, the complex vector of values ​​is weighted to obtain the attention result.

[0012] According to a data processing method based on a complex neural network model provided by the present invention, the step of calculating the attention score between the query complex vector and the key complex vector includes: The attention score is determined based on the real part of the complex inner product of the conjugate transpose of the query complex vector and the key complex vector; or, The attention score is determined based on the real part of the complex inner product of the query complex vector and the conjugate transpose of the key complex vector.

[0013] According to the present invention, a data processing method based on a complex neural network model is provided, wherein the complex neural network model includes a complex feedforward network, and the complex feedforward network includes a generalized complex linear layer and an activation function layer connected in sequence; The activation function layer is used to apply real-valued nonlinear activation functions to the real and imaginary parts of the output complex vector of the generalized complex linear layer connected to it.

[0014] According to a data processing method based on a complex neural network model provided by the present invention, the step of inputting the input data into the complex neural network model and obtaining the task result output by the complex neural network model includes: When the input data is real number input data, the input data is input into the input layer of the complex neural network model to obtain the real number vector output by the input layer; The real number vector is split to obtain a third component and a fourth component; Based on the third component and the fourth component, a complex vector is determined; the real part of the complex vector is the third component, and the imaginary part of the complex vector is the fourth component. The complex vector is input into the next layer of the input layer in the complex neural network model to obtain the task result output by the complex neural network model; When the input data is complex, the complex input data is input into the complex neural network model to obtain the task result output by the complex neural network model.

[0015] According to the present invention, a data processing method based on a complex neural network model is provided, wherein the complex neural network model includes a complex detection head; the complex detection head is the generalized complex linear layer; The complex neural network model is used to determine the real part of the complex detection vector output by the complex detection head as a real detection vector; or, The complex neural network model is used to concatenate the real and imaginary parts of the complex detection vector output by the complex detection head to obtain a real detection vector.

[0016] The present invention also provides a data processing apparatus based on a complex neural network model, comprising: The acquisition module is used to acquire input data; The output module is used to input the input data into the complex neural network model and obtain the task result output by the complex neural network model; The complex neural network model includes a generalized complex linear layer; the generalized complex linear layer is used to perform complex linear mapping operations on the input complex input vector and to perform inverse linear mapping operations on the conjugate vector of the complex input vector; the complex linear mapping operation is performed through a trainable first complex parameter, and the inverse linear mapping operation is performed through a trainable second complex parameter.

[0017] The present invention also provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the program to implement the data processing method based on the complex neural network model as described above.

[0018] The present invention also provides a non-transitory computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the data processing method based on the complex neural network model as described above.

[0019] The present invention also provides a computer program product, including a computer program that, when executed by a processor, implements the data processing method based on the complex neural network model as described above.

[0020] The data processing method and apparatus based on a complex neural network model provided by this invention inputs input data into a complex neural network model to obtain the task results output by the complex neural network model. The complex neural network model includes a generalized complex linear layer, which performs complex linear mapping operations on the input complex input vector and inverse linear mapping operations on the conjugate vector of the complex input vector. The complex linear mapping operation is performed using a trainable first complex parameter, and the inverse linear mapping operation is performed using a trainable second complex parameter. This enables the complex neural network model to simultaneously model complex linear and inverse linear relationships, thereby more fully capturing the inherent structure of the data and improving the modeling performance of the complex neural network model, i.e., enhancing its expressive power. Based on this, when performing specific tasks, more accurate and robust task results can be obtained, thereby improving data processing capabilities and ultimately improving the accuracy of the task results, i.e., improving the accuracy of task processing. Attached Figure Description

[0021] To more clearly illustrate the technical solutions in this invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of this invention. For those skilled in the art, other drawings can be obtained from these drawings without creative effort.

[0022] Figure 1 This is a flowchart illustrating the data processing method based on a complex neural network model provided by the present invention.

[0023] Figure 2 This is a schematic diagram of the data processing device based on a complex neural network model provided by the present invention.

[0024] Figure 3 This is a schematic diagram of the structure of the electronic device provided by the present invention. Detailed Implementation

[0025] To make the objectives, technical solutions, and advantages of this invention clearer, the technical solutions of this invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of this invention. All other embodiments obtained by those skilled in the art based on the embodiments of this invention without creative effort are within the scope of protection of this invention.

[0026] The present invention proposes the following embodiments. The following is a detailed description in conjunction with... Figure 1This invention describes a data processing method based on a complex neural network model. The execution entity of this data processing method based on a complex neural network model can be an electronic device, which may include, but is not limited to, computers, servers, desktop computers, laptops, tablets, smartphones, etc.

[0027] Figure 1 This is a flowchart illustrating the data processing method based on a complex neural network model provided by the present invention, as shown below. Figure 1 As shown, the data processing method based on the complex neural network model includes the following steps 110 and 120.

[0028] Step 110: Obtain input data.

[0029] Here, the input data is the data used for data processing by the complex neural network model, that is, the data used for predictive reasoning to obtain the task results.

[0030] The input data can be either real or complex. That is, the input to a complex neural network model can come from two different sources: one is native real input (real input data), such as word embedding vectors in traditional natural language processing tasks; the other is native complex input (complex input data), such as frequency domain coefficients of speech signals after Fourier transform, complex amplitude vectors of quantum states, etc.

[0031] For example, in a voice data processing scenario, such as a voice recognition scenario, the input data is voice data, such as user voice data.

[0032] Step 120: Input the input data into the complex neural network model to obtain the task result output by the complex neural network model.

[0033] The complex neural network model includes a generalized complex linear layer; the generalized complex linear layer is used to perform complex linear mapping operations on the input complex input vector and to perform inverse linear mapping operations on the conjugate vector of the complex input vector; the complex linear mapping operation is performed through a trainable first complex parameter, and the inverse linear mapping operation is performed through a trainable second complex parameter.

[0034] Here, a complex neural network model refers to a neural network model whose internal parameters or activation values ​​are at least some of complex numbers, enabling data processing and feature extraction in the complex domain. In embodiments of the present invention, this complex neural network model may include, but is not limited to, convolutional layers, recurrent layers, attention layers, or fully connected layers, as long as it internally contains the generalized complex linear layer described in this invention. In one embodiment, the complex neural network model is a complex Transformer model. In one embodiment, the complex neural network model can be a large model (such as a large language model).

[0035] Here, the Widely Complex Linear Layer is a novel neural network layer proposed in this embodiment of the invention. Unlike the traditional complex linear layer (naive complex linear layer) which contains only a single mapping path, the Widely Complex Linear Layer sets up two mapping paths (complex linear mapping and antilinear mapping) in parallel.

[0036] Here, a complex input vector (i.e., a complex activation vector) refers to a vector whose components are all complex numbers. For example, a complex input vector is shown below: ; In the formula, Represents a complex input vector. This represents the real part of a complex input vector. The vector representing the imaginary part of a complex input vector. Indicates the imaginary part. Indicates dimension.

[0037] Here, complex linear mapping operation refers to a linear transformation performed on a complex input vector.

[0038] Here, the conjugate vector refers to the new vector obtained by taking the complex conjugate of each component of the complex input vector. If the complex input vector... Then its conjugate vector is .

[0039] Here, the inverse linear mapping operation refers to a linear transformation performed on the conjugate vector of a complex input vector. Based on this, support for inverse linear mappings of complex conjugate vectors is provided to fully represent complex linear-inverse linear combinations.

[0040] Here, the first complex parameter can be the complex weight matrix of the complex linear mapping operation, and its parameter values ​​are learned and optimized through the training process of the complex neural network model (such as backpropagation and gradient descent). The second complex parameter can be the complex weight matrix of the inverse linear mapping operation, which is also learned and optimized through the training process of the complex neural network model. Here, the task result is the output of a complex neural network model performing a specific task; for example, the speech recognition result output when performing a speech recognition task.

[0041] For example, the generalized complex linear layer performs both complex linear and antilinear mappings on the input complex input vector and combines the two mapping results; the generalized complex linear layer is shown below. ; In the formula, This represents the output of the generalized complex linear layer; Represents a complex input vector; Represents a complex input vector The conjugate vector; Indicates the first complex parameter; This represents the second complex parameter. , Representing dimension, , , The parameter represents the real part of the first complex number parameter. The parameter represents the imaginary part of the first complex number parameter. The parameter represents the real part of the second complex number parameter. The parameter represents the imaginary part of the second complex number parameter.

[0042] To facilitate understanding of the embodiments of the present invention, a specific application scenario is used below to illustrate the specific execution flow of the data processing method based on a complex neural network model provided by the embodiments of the present invention. This application scenario involves using a complex neural network model to process speech signals for speech recognition tasks.

[0043] Step 1: Acquire input data. The electronic device acquires a raw speech waveform signal through an audio acquisition device (such as a microphone). This raw speech waveform signal is a one-dimensional real number sequence defined in the time domain.

[0044] Step 2: Input the input data into a complex neural network model (such as a speech recognition model) to obtain the speech recognition result (task result).

[0045] Specifically, the processor of the electronic device performs a short-time Fourier transform on the acquired speech waveform signal, outputting a series of complex spectra. Each complex spectrum corresponds to a short-time frame of the speech and can be represented as a complex vector. These complex vectors constitute the initial input data of the complex neural network model.

[0046] Assume the first layer of the complex neural network model is a generalized complex linear layer. A complex input vector corresponding to a speech frame is input into this first layer. The electronic device performs a complex linear mapping operation and calculates the conjugate vector of the complex input vector, then performs an inverse linear mapping operation. The results of these two operations are added together to obtain the output of this layer. This output is used as the input to the next layer, and so on, sequentially passing through subsequent layers (which could be more generalized complex linear layers, nonlinear activation function layers, attention layers, etc.) in the complex neural network model for layer-by-layer feature extraction.

[0047] The final layer of a complex neural network model (e.g., a classification layer) outputs a probability distribution for each word. Based on this probability distribution, the electronic device uses a decoding algorithm to generate the final recognized text sequence, which is the speech recognition result.

[0048] During the training phase, the text output by the complex neural network model is compared with the actual labeled text, and the loss function is calculated. Then, the gradient of the loss function with respect to all trainable parameters (including the first and second complex parameters in all generalized complex linear layers) is calculated using the backpropagation algorithm, and these parameters are updated using the gradient descent optimizer. This process is repeated until the model converges.

[0049] It should be understood that the embodiments of the present invention aim to solve the problem of limited modeling performance caused by the insufficient expressive power of linear transformation units in existing complex neural network models. The embodiments of the present invention improve the ability of complex neural network models to process data with intrinsically complex structures by introducing a generalized complex linear layer with stronger expressive power into the complex neural network model, thereby improving data processing capabilities and ultimately increasing the accuracy of task results. In other words, this generalized complex linear layer can represent arbitrary complex affine transformations, enabling the complex neural network model to simultaneously model complex linear and antilinear relationships. This is crucial for processing signals that inherently contain conjugate symmetry or complex phase coupling (such as speech spectra, quantum states, etc.), thus enabling a more complete capture of the intrinsic structure of the data and improving the modeling performance of the complex neural network model, i.e., enhancing the expressive power of the complex neural network model.

[0050] Furthermore, due to the fundamentally enhanced expressive power of complex neural network models, they are able to more accurately fit the distribution of complex data. When the model can more realistically characterize the data, it naturally achieves more accurate and robust task results when performing specific tasks (such as speech recognition, signal classification, and physical simulation). Therefore, by processing data through the embodiments of this invention, the accuracy of the final output task results is effectively improved, thus enhancing the accuracy of task processing.

[0051] It should be understood that the generalized complex linear layer in the embodiments of the present invention formally covers all affine transformations in the complex domain, making up for the deficiency of traditional complex linear layers that can only express a single complex linear mapping. This ensures that the complex neural network model can fully handle phase, amplitude, and conjugate symmetric structures in complex space, giving it higher expressive power in physical scenarios, signal processing tasks, and multimodal modeling.

[0052] It should be understood that the generalized complex linear layer proposed in this embodiment of the invention is a basic computational unit that can be flexibly integrated into any type of complex neural network model (not limited to the Transformer model), has wide applicability, and ensures its engineering feasibility in industry and academia.

[0053] The data processing method based on a complex neural network model provided in this invention inputs input data into the complex neural network model to obtain the task results output by the complex neural network model. The complex neural network model includes a generalized complex linear layer, which performs complex linear mapping operations on the input complex input vector and inverse linear mapping operations on the conjugate vector of the complex input vector. The complex linear mapping operation is performed through a trainable first complex parameter, and the inverse linear mapping operation is performed through a trainable second complex parameter. This enables the complex neural network model to simultaneously model complex linear and inverse linear relationships, thereby more fully capturing the inherent structure of the data and improving the modeling performance of the complex neural network model, i.e., enhancing the expressive power of the complex neural network model. Based on this, when performing specific tasks, more accurate and robust task results can be obtained, thereby improving data processing capabilities and the accuracy of task results, i.e., improving the accuracy of task processing.

[0054] Based on any of the above embodiments, in this method, the generalized complex linear layer outputs a real number output vector or a complex number output vector.

[0055] Given that current technologies have not established a strict equivalence between complex neural network models and real neural network models, and lack a reversible closed-form transformation mechanism that can map any real linear layer to its complex counterpart, complex neural network models cannot reuse existing efficient matrix multiplication, attention acceleration operators, and inference frameworks in engineering applications. This directly leads to the difficulty in deploying complex neural network models in real-world industrial environments and fails to meet hardware optimization requirements. Furthermore, considering that current technologies have not established an implementation path that is engineering-equivalent to real neural network models—that is, they cannot strictly expand complex operations into structured real matrix operations—they cannot be directly implemented in mainstream deep learning frameworks, thus limiting the further expansion of model size. In other words, the operator design of complex neural networks differs inherently from that of real neural networks; there is no standardized way to map complex linear operations to structured real matrices, making it difficult to reuse existing deep learning frameworks, GPU-efficient kernels, and inference engines. This severely limits the training scale and engineering feasibility of complex neural network models. Based on this, embodiments of the present invention are proposed.

[0056] The real number output vector is determined based on the following method: The real part vector and the imaginary part vector of the complex input vector are concatenated to obtain the real input vector, and the real parameters are determined based on the first complex parameter and the second complex parameter. The real number output vector is determined based on the real number input vector and the real number parameters.

[0057] It should be noted that the embodiments of the present invention describe how to completely convert generalized complex linear operations in the complex field into matrix multiplication operations in the real field. If the next layer of the generalized complex linear layer needs to receive a real output vector, then it is not necessary to convert the real output vector into a complex output vector.

[0058] For example, a complex input vector is shown below: ; In the formula, Represents a complex input vector. This represents the real part of a complex input vector. The vector representing the imaginary part of a complex input vector. Represents complex dimension.

[0059] During the execution process of the electronic device, two real number vectors are concatenated to form a 2D real number input vector. This step normalizes the input vector in the complex field into a form that can be directly processed by the real number computation unit.

[0060] In one embodiment, the real part vector is... Concatenate first, then concatenate the imaginary part vector. The concatenation is then performed to obtain the real number input vector. ;in, , The first part of the real number input vector Each component and the following Each component can be considered as the real and imaginary parts of the complex input vector, respectively. Of course, in other embodiments, the real part vector can also be... Concatenate later, and the imaginary part vector The assembly process has already been described, so it will not be repeated here.

[0061] For example, the first complex parameter is The second complex parameter is To process a 2D real-valued input vector, we need to construct a matching 2D×2D real-valued parameter matrix. This invention provides a deterministic conversion rule from complex number parameters to real number parameters, which guarantees that subsequent real number operations are mathematically equivalent to the original complex number operations.

[0062] For example, after determining the real number input vector and real number parameters, the electronic device can perform a standard real number matrix multiplication operation to obtain a 2D real number output vector. Modern computing hardware (such as GPUs and TPUs) has been optimized for large-scale real matrix multiplication, converting complex matrix multiplication into real matrix multiplication operations, which improves computational efficiency, i.e., data processing efficiency. The real matrix multiplication operation (real linear layer, i.e., the linear layer in a traditional real neural network model) is shown below: ; In the formula, , Represents a real number output vector; , Represents a real number parameter; , This represents a real number input vector.

[0063] The complex output vector is determined based on the following method: The real number output vector is split to obtain a first real number output component and a second real number output component; The complex output vector is determined based on the first real number output component and the second real number output component.

[0064] It should be noted that in the intermediate layers of a complex neural network model, it is usually necessary to pass the real-valued output vector to the next layer in complex form. This embodiment of the invention describes how to recover an equivalent complex-valued output vector from a real-valued output vector.

[0065] For example, the calculated real-number output vector has a dimension of 2D. It is split into two real components with dimension D.

[0066] In one embodiment, the first D elements of the real number output vector are used as the first real number output component. The last D elements of the real output vector are used as the second real output component. That is, the real number output vector ,in, Of course, in other embodiments, the first D elements of the real number output vector can also be used as the second real number output component. The last D elements of the real output vector are taken as the first real output component. This will not be elaborated upon here.

[0067] For example, an equivalent complex output vector is constructed based on the two real components obtained from the split. The complex output vector It can be used as input for subsequent network layers.

[0068] It should be understood that this invention bridges the gap between designing complex neural network architectures and utilizing real-number hardware for efficient computation. Designers of complex neural network models can think about and build the network in the complex domain, while the underlying computational engine executes entirely in the real domain, thus balancing expressive power and computational efficiency. The output of a generalized complex linear layer can be a real-number output vector from intermediate steps or a final complex-number output vector, depending on the data format required by subsequent network layers or the task. In other words, this invention proposes a specific technical path for efficient execution on electronic device hardware. This path establishes a strict equivalence mapping between the mathematical representation in the complex domain and the computational implementation in the real domain, allowing complex complex operations to fully reuse computational operators highly optimized for real-number operations in existing deep learning frameworks. This improves data processing efficiency and performance, ultimately enhancing task processing efficiency and performance.

[0069] It should be understood that the embodiments of the present invention provide a technical path to completely transform abstract generalized complex linear operations into standard real matrix multiplication. Based on this, the complex neural network model of the embodiments of the present invention can be implemented without developing any dedicated complex operation hardware or underlying software libraries; it can directly and seamlessly utilize all the results optimized for real number operations in the existing deep learning ecosystem, including the hardware acceleration capabilities of GPUs and TPUs, efficient matrix multiplication operators, and mature training and inference frameworks. This greatly reduces the computational overhead of the complex model and significantly improves the computational efficiency of training and inference, giving it performance comparable to real number models of the same scale. That is, it greatly improves computational efficiency and engineering compatibility. In other words, the embodiments of the present invention not only mathematically guarantee strict equivalence between the complex and real number domains, but also ensure in engineering practice that all complex components can reuse existing infrastructure, thereby giving the complex neural network model significant advantages in training stability, convergence speed, numerical characteristics, and expressive power.

[0070] In other words, the embodiments of the present invention provide a strict equivalence relationship between generalized complex linear layers and real number block matrices. This extension not only does not destroy the original structured representation of deep networks, but also makes the complex neural network model more complete in form, and can communicate with traditional real number models through explicit mapping rules.

[0071] It should be understood that, by defining a bidirectional conversion between real and complex output vectors, this embodiment of the invention allows the generalized complex linear layer to be flexibly embedded, like a plug-and-play module, into complex network architectures that combine real and complex layers. Regardless of whether the output of the previous layer is real or complex, or whether the next layer requires real or complex numbers, this layer can act as a smooth bridge, improving the flexibility of network architecture design and enhancing the model's flexibility and modularity.

[0072] It should be understood that the embodiments of the present invention emphasize how to change the way data is organized and the computation steps within a computer to solve the technical problem of computer execution efficiency. That is, in model training, the embodiments of the present invention do not require additional computing power, the number of parameters and FLOPs can be fully aligned with real number models of the same size, and existing distributed training, tensor parallelism and memory optimization strategies can be reused, so that the implementation of complex number models in engineering systems is not restricted in any way.

[0073] The data processing method based on a complex neural network model provided in this invention allows designers of complex neural network models to think about and construct the network in the complex domain, while the underlying computation engine executes entirely in the real domain, thus balancing expressive power and computational efficiency. Specifically, by establishing a strict equivalence mapping between the mathematical representation in the complex domain and the computational implementation in the real domain, complex complex operations can fully reuse computational operators highly optimized for real number operations in existing deep learning frameworks, thereby improving data processing efficiency and performance, and ultimately enhancing task processing efficiency and performance.

[0074] Based on any of the above embodiments, in this method, determining the real number parameter based on the first complex number parameter and the second complex number parameter includes: Based on the first complex parameter and the second complex parameter, four real number sub-parameters are generated; The four real number sub-parameters are combined to obtain the real number parameters.

[0075] It should be noted that the embodiments of the present invention clearly define the positive mapping relationship between the two complex parameters of the generalized complex linear layer and their equivalent real parameters. This mapping relationship is the fundamental basis for the complete mathematical equivalence between real number operations and primitive complex number operations.

[0076] Here, the first complex parameter can be decomposed into its real part parameter during the calculation. and imaginary part parameter That is, the first complex parameter , and All are real number matrices.

[0077] Here, the second complex parameter can be decomposed into its real part parameter during the calculation. and imaginary part parameter That is, the second complex parameter , and All of them are real number matrices.

[0078] This invention generates four real sub-parameters for constructing the final real parameters by performing a specific linear combination of the four decomposed real matrices. The generation rules are as follows: ; ; ; ; In the formula, , , , Represents four real number sub-parameters, The parameter represents the real part of the first complex number parameter. The parameter represents the imaginary part of the first complex number parameter. The parameter represents the real part of the second complex number parameter. The parameter represents the imaginary part of the second complex number parameter.

[0079] It should be noted that complex numbers are input into vectors. and its conjugate vector Substitute achievable After unfolding and organizing, it can be The real and imaginary parts are written as pairs Real linear combinations: Based on this, the above generation rules are obtained.

[0080] After generating the four real-number sub-parameters mentioned above, they are merged or concatenated in the form of a block matrix to construct the final real-number parameter with a dimension of 2D×2D. The real number parameters are as follows: ; In the formula, Represents the real number parameter, , , , Represents the four real number sub-parameters, This represents the real part of the first complex number parameter. This represents the imaginary part of the first complex number parameter. This represents the real part of the second complex number parameter. This represents the imaginary part of the second complex number parameter.

[0081] As can be seen from the above derivation, the equivalent form of the generalized complex linear layer in the real number field is precisely a structure with a specific feature. Block matrix. , , , yes The real matrix block is strictly composed of and It consists of the real and imaginary parts. This form shows that for any given... and It is always possible to construct a real matrix (with real parameters). This allows for the real-number linear layer to be treated as having a real part and a virtual part under the convention of treating the first half of the dimension as the real part and the second half of the dimension as the imaginary part. With complex generalized linear layer Completely equivalent.

[0082] It should be understood that the specific construction formulas given in the embodiments of the present invention are generalized complex linear operations. It is a unique and definite form obtained by expanding the model in the real number field and reorganizing it according to its real and imaginary parts. Mathematically, it rigorously guarantees that the results calculated using real number parameters are completely consistent with the results of direct complex number calculations. This lossless equivalence is the foundation for achieving efficient and reliable computation, ensuring the consistency of the model's behavior across different implementations.

[0083] It should be understood that the specific construction formula described above provides a clear, executable, and standardized technical path for mapping a novel complex number operation unit (generalized complex linear layer) to standard, existing computational operations (real matrix multiplication). Any implementation that follows this rule is guaranteed to be correct. This facilitates reproduction and integration into different deep learning frameworks, lowering the barrier to technical implementation.

[0084] The data processing method based on a complex neural network model provided in this invention, through the above-mentioned method, mathematically guarantees that the result calculated using real number parameters is completely consistent with the result of directly performing complex number calculations, thereby achieving efficient and reliable calculations, improving data processing efficiency and performance, and ultimately improving task processing efficiency and performance.

[0085] Based on any of the above embodiments, in this method, when the first complex parameter and the second complex parameter are generated based on the real parameters of the real linear layer, the first complex parameter and the second complex parameter are generated in the following manner: The real number parameters are divided into blocks to obtain four real number sub-parameters; Based on the four real number sub-parameters, the first complex number parameter and the second complex number parameter are generated.

[0086] Given that existing technologies have not established a strict equivalence relationship between complex neural network models and real neural network models, and lack a reversible closed-form transformation mechanism that can map any real linear layer to its complex counterpart, complex neural network models cannot reuse the massive pre-trained parameter assets of real neural network models. This directly results in complex networks failing to meet the current demands for large-scale model training, distributed training, and hardware optimization. Based on this, embodiments of the present invention are proposed.

[0087] It should be noted that this embodiment of the invention establishes a parameter transformation channel from a standard real linear layer to a generalized complex linear layer. This channel is reversible, enabling complex neural network models to directly utilize and inherit the pre-training results of existing large-scale real neural network models.

[0088] It should be noted that the first complex parameter and the second complex parameter can be obtained directly through training, or the real parameters obtained by training the real linear layer can be reused. Therefore, the embodiment of the present invention is only used when the first complex parameter and the second complex parameter are generated based on the real parameters of the real linear layer.

[0089] Wherein, the input dimension of the real linear layer is twice the input dimension of the generalized complex linear layer, the real part of the complex input vector is regarded as the first component, and the imaginary part of the complex input vector is regarded as the second component; the first component and the second component are obtained by splitting the real input vector input to the real linear layer.

[0090] In order to establish an equivalence relationship between a real linear layer and a generalized complex linear layer, it is necessary to first define their input and output dimensions and data structures.

[0091] Dimensional convention: The input dimension of the real linear layer is set to twice the input dimension (complex dimension) of the generalized complex linear layer. For example, if the generalized complex linear layer processes complex vectors of dimension D, then the corresponding real linear layer processes real vectors of dimension 2D.

[0092] Data structure convention: There exists a structural correspondence between a real vector and its corresponding complex vector. A 2D real input vector can be decomposed into two D-dimensional components (the first component and the second component). When establishing an equivalent mapping, one component is treated as the real part of the corresponding complex input vector, and the other component is treated as the imaginary part. In one embodiment, the first half of the real input vector can be considered as the real part, and the second half as the imaginary part. This convention of "treating" provides a unified interpretive framework for linear transformations in two different domains.

[0093] It should be noted that the correspondence between the real and imaginary parts is described by treating them as such, and there is no limitation on which specific part of the real vector the first and second components are. This gives the embodiments of the present invention a high degree of flexibility. Developers can choose to use the first half or the second half of the real vector as the real part, depending on the specific implementation or the convenience of hardware memory layout, and this choice does not affect the validity of the mapping relationship. This flexibility of implementation makes the embodiments of the present invention more easily adaptable to different programming habits and framework designs, and has better versatility.

[0094] Under this premise, from a given real-valued linear layer of real parameters The first complex parameter for generating the generalized complex linear layer Second complex parameter The steps are as follows: The real-number parameters are divided into blocks; specifically, the given real-number parameters are... Divide along the midpoints of its rows and columns to obtain four real sub-parameters (or block matrices) of dimension D×D.

[0095] In one specific embodiment, the real number parameter is as follows: ; In the formula, Represents the real number parameter, , , , These represent the four real number sub-parameters.

[0096] Complex parameters are generated based on real sub-parameters. Specifically, using the four real sub-parameters obtained in the previous step, the first complex parameter is derived through reverse analysis using a specific linear combination rule. Second complex parameter The real and imaginary parameters.

[0097] In one specific embodiment, the first complex parameter is generated based on the following formula: ; ; ; In the formula, Represents the first complex number parameter. This represents the real part of the first complex number parameter. This represents the imaginary part of the first complex number parameter.

[0098] The second complex parameter is generated based on the following formula: ; ; ; In the formula, This represents the second complex parameter. This represents the real part of the second complex number parameter. This represents the imaginary part of the second complex number parameter.

[0099] This series of steps constitutes a unique and deterministic parameter mapping method from any real linear layer to a generalized complex linear layer.

[0100] It should be understood that the embodiments of the present invention achieve smooth migration and parameter reuse from real-number neural network models to complex-number neural network models. Specifically, the embodiments of the present invention provide a technical approach to directly and losslessly utilize existing massive real-number pre-trained model assets. Researchers or developers do not need to train a large-scale complex-number neural network model from scratch, but can directly upgrade a real-number neural network model that has already been trained with huge computational resources to the complex-number neural network model of the embodiments of the present invention through a one-time parameter conversion. This greatly reduces the threshold and cost of applying complex-number neural network models and accelerates their research and development and deployment cycle.

[0101] It should be understood that the embodiments of the present invention guarantee strict equivalence of the model across different representation domains. Specifically, a reverse mapping from real parameters to uniquely recover complex parameters is described, thus proving that this equivalence relation is bidirectional and unique. This means that a real linear layer can be uniquely mapped to a generalized complex linear layer, and vice versa. This strict mathematical equivalence guarantees that the transformed complex model is functionally and behaviorally identical to the original real model (when the input is an equivalent real / complex pair). This provides a solid theoretical foundation for model debugging, validation, and deployment switching across different hardware or frameworks.

[0102] In other words, in the embodiments of the present invention, "the real dimension is..." Complex dimension is Under the convention that "the first half of the dimension is the real part and the second half is the imaginary part", there is a one-to-one correspondence between any real linear layer and a pair of generalized complex linear layer parameters. This correspondence ensures both the strict equivalence between the complex model and the real model, and the feasibility and uniqueness of smoothly migrating from the existing real model to the complex model.

[0103] In other words, the embodiments of this invention offer significant advantages in terms of model transferability. Through a bidirectional, unique, and structured mapping between generalized complex linear layers and real linear layers, the embodiments of this invention can achieve a smooth transition from any real-valued model to a complex model. Existing large-scale real-valued pre-trained models can be directly transformed into complex models through a single mapping without retraining; conversely, when model deployment, computing power, or inference requirements necessitate a switch to real-valued form, the complex model can also be reverse-mapped back to the original real-valued structure. This reversibility has extremely high practical value in the training, fine-tuning, distillation, and cross-modal tasks of large-scale models, enabling models to switch freely between different representation domains, thereby improving reusability and reducing resource costs.

[0104] It should be understood that both forward and backward propagation during the training phase and efficient deployment during the inference phase can be achieved by simply rearranging and combining the weight matrix in the aforementioned block structure, thus realizing the goal of "designing the architecture in the complex domain and computing efficiently in the real domain".

[0105] The data processing method based on a complex neural network model provided in this invention achieves smooth migration and parameter reuse from a real neural network model to a complex neural network model through the above-described method. This provides a direct and lossless technical approach to utilizing existing massive amounts of pre-trained real neural network model assets. Instead of training a large-scale complex neural network model from scratch, a real neural network model that has already been trained with significant computational resources can be directly upgraded to the complex neural network model of this invention through a one-time parameter conversion. This greatly reduces the threshold and cost of applying complex neural network models, accelerates their research and development and deployment cycle, and ultimately reduces the training and deployment costs of complex neural network models.

[0106] Based on any of the above embodiments, in this method, the complex neural network model includes an attention layer; the attention layer includes three of the generalized complex linear layers.

[0107] Here, the attention layer contains three independent generalized complex linear layers. These three generalized complex linear layers structurally follow the definition in the above embodiment, but each has its own independent trainable first and second complex parameters. In one embodiment, the attention layer is a self-attention layer.

[0108] It should be noted that the generalized complex linear layer in the above embodiment is used as the basic building block to construct the attention layer in the complex domain. This gives the attention mechanism itself the advantage of processing complex signals and utilizing the expressive power of generalized complex linearity, thereby achieving structural uniformity of the entire complex neural network model in the complex domain.

[0109] The attention layer outputs attention results based on the following method: Based on the three generalized complex linear layers, the input complex vector of the attention layer is mapped to the query complex vector, the key complex vector, and the value complex vector, respectively. Calculate the attention score between the query complex vector and the key complex vector; Based on the attention score, the complex vector of values ​​is weighted to obtain the attention result.

[0110] For each input vector to the attention layer, this complex input vector... They are simultaneously fed into three generalized complex linear layers.

[0111] Let the complex input vector be... for ,in, The sequence length is given.

[0112] The first generalized complex linear layer outputs a query complex vector. The calculation method is as follows: ; The second generalized complex linear layer outputs a complex vector of bonds. The calculation method is as follows: ; The third generalized complex linear layer outputs a vector of values. The calculation method is as follows: ; In the formula, , , , Denotes the first complex parameter of each generalized complex linear layer. , , This represents the second complex parameter of each generalized complex linear layer.

[0113] It should be noted that, according to the aforementioned equivalence relation of the generalized complex linear layer, the three mappings mentioned above correspond to three specific structures in the real number field. The weight matrix can be directly implemented using real matrix multiplication, thus improving computational efficiency. Furthermore, the latent vector dimensions of the real and complex models are different; during the transformation, the attention normalization factor can be set according to actual needs. or .

[0114] Attention scores are used to calculate the relevance or attention between any two positions in a sequence (e.g., position t and position s). For a given query complex vector, its attention score needs to be calculated with respect to all key complex vectors in the sequence. Specifically, attention scores are calculated using inner products.

[0115] In one specific embodiment, a normalization function (such as the Softmax function) is applied to the calculated set of attention scores to obtain normalized attention weights. Then, these attention weights are used to perform a weighted summation of all complex vectors of values ​​in the sequence to obtain the final attention result. This attention result is a complex vector that incorporates information from the entire sequence and is passed as the output of this attention layer to subsequent network layers.

[0116] For example, the attention results are generated as follows: ; ; In the formula, Indicates the first The query complex vector and the first Attention weights between complex vectors of keys; Indicates the sequence length; Indicates the first The query complex vector and the first Attention weights between complex vectors of keys; Indicates the first Attention results corresponding to each query complex vector; Indicates the first A complex vector of values.

[0117] It should be understood that by employing a generalized complex linear layer to generate query (Q), key (K), and value (V) vectors, the attention mechanism itself also possesses the ability to handle complex linear and antilinear combinations. Compared to attention layers that only use simple complex linear mappings, the attention layer in this embodiment can more profoundly capture the complex phase relationships and conjugate structures existing in the input complex vector sequence, thereby improving the model's accuracy in modeling the internal dependencies of the sequence. In other words, it enhances the expressive power of the attention mechanism, thereby improving data processing capabilities, improving task processing performance, and ultimately improving the accuracy of task results.

[0118] It should be understood that the core component, the generalized complex linear layer, is seamlessly extended to the attention mechanism. This allows the entire complex neural network model (especially the attention-based Transformer model) to use a unified, mathematically complete linear transformation unit from the input layer, intermediate transformation layers to the attention layer. This architectural consistency is beneficial to model optimization and stability, avoiding problems such as poor gradient propagation or representation mismatch that may result from mixing components with different mathematical properties. In other words, it achieves the unity and consistency of the model architecture.

[0119] It should be understood that in the spectral or quantum state evolution sequence of a speech signal, the dependencies between different time steps or positions may be reflected in the cooperative changes in phase or the preservation of conjugate symmetry. The complex attention layer proposed in this embodiment of the invention has Q and K vectors that are themselves complex numbers, and its subsequent scoring mechanism can directly utilize these complex structures. This enables the model to learn dependency patterns specific to the complex domain that traditional real-number attention mechanisms cannot perceive, thereby gaining a performance advantage when handling such tasks.

[0120] It should be understood that, by combining the equivalence relationship between generalized complex linear layers and structured real matrices, this invention can implement the entire complex attention module in the real domain as a set of standard matrix multiplication and softmax operations, requiring only a structural rearrangement of the weight matrix according to the aforementioned formula. Therefore, the complex attention proposed in this invention not only maintains the ability to express complex phase structures mathematically, but is also highly compatible with real-valued attention modules in engineering implementation.

[0121] The data processing method based on a complex neural network model provided in this invention enables the attention mechanism itself to handle complex linear and antilinear combination relationships, thereby capturing more profound phase relationships and conjugate structures in the input complex vector sequence. This improves the modeling accuracy of the internal dependencies of the sequence, enhances the expressive power of the attention mechanism, and thus improves the data processing capability, thereby improving task processing performance and ultimately improving the accuracy of the task results.

[0122] Based on any of the above embodiments, in this method, calculating the attention score between the query complex vector and the key complex vector includes: The attention score is determined based on the real part of the complex inner product of the conjugate transpose of the query complex vector and the key complex vector; or, The attention score is determined based on the real part of the complex inner product of the query complex vector and the conjugate transpose of the key complex vector.

[0123] Here, the conjugate transpose is to first transpose the matrix or vector (interchange rows and columns), and then take the complex conjugate of each element (invert the sign of the imaginary part).

[0124] Here, the result of the complex inner product is a complex scalar. The result of the complex inner product contains the combined relationship between the two vectors in terms of magnitude and phase.

[0125] For example, in the convention of treating the query complex vector as a column vector, in order to calculate the inner product, the query complex vector needs to be transposed into a row vector via its conjugate. The attention score is calculated as follows: ; In the formula, Indicates the first The query complex vector and the first Attention score among complex vectors of keys; Represents the dimension of a complex number; Indicates taking the real part; Indicates the first One query complex vector; Indicates conjugate transpose; Indicates the first A complex vector of keys.

[0126] It should be understood that by taking the real part of the complex inner product, the embodiments of the present invention ensure that the final attention score is a real-valued scalar. This allows for seamless application of standard, real-number-based normalization functions for subsequent normalization without requiring any modification to the complex domain of the normalization function itself, thereby maximizing the reuse of engineering infrastructure. Furthermore, it is naturally compatible with complex phase structures, leveraging the phase difference between the query complex vector and the key complex vector in the complex plane to enhance the representational power of the attention mechanism and thus improve data processing capabilities. Moreover, in the entire attention calculation process, only the attention score calculation stage relies on the real part of the complex inner product; the remaining steps are completely consistent with traditional attention methods.

[0127] In other words, the result of the complex inner product depends on both the magnitude of the vectors and the phase difference between them. By using their real parts as attention scores, the model can naturally and implicitly utilize the phase difference information between the query vector and the key vector in the complex plane. For example, phase-aligned vectors will produce larger positive real parts, while vectors with opposite phases may produce negative real parts. This mechanism allows attention scores to reflect richer geometric relationships than real inner products, enabling the model to learn phase-based dependencies and enhancing the expressive power and modeling accuracy of the attention mechanism.

[0128] Furthermore, it ensures full compatibility with existing engineering infrastructure. Specifically, the most significant engineering advantage of this invention is that, although its mathematical meaning is complex, the final calculated attention score is a real-valued scalar, thus ensuring compatibility with the standard Softmax function. No modification or redesign of the Softmax function is required; highly optimized real-valued Softmax implementations within existing deep learning frameworks can be directly used. Moreover, the process of calculating the real part of the complex inner product can be completely decomposed into two standard real-valued matrix multiplications and one addition. This means that the entire attention score calculation can fully reuse the most optimized real-valued matrix operators on hardware such as GPUs, without requiring any special complex-valued computation support. This dual compatibility ensures that the complex attention mechanism of this invention maintains powerful expressive capabilities while fully aligning with the standard real-valued attention mechanism in terms of computational efficiency and engineering feasibility.

[0129] It should be understood that, compared to some other complex scoring mechanisms (such as directly using the complex modulus), the real part of the complex inner product in this embodiment of the invention is a signed value with a wider range and generally better numerical properties during gradient calculation. This helps to maintain gradient stability during model training, avoiding gradient vanishing or exploding problems that may be caused by the range limitations of activation functions or scoring mechanisms, thus ensuring numerical stability.

[0130] The data processing method based on a complex neural network model provided in this invention ensures that the final attention score is a real-valued scalar by taking the real part of the complex inner product. This allows for seamless application of standard real-number-based normalization functions for subsequent normalization without requiring any modification to the complex domain of the normalization function itself. This maximizes the reuse of engineering infrastructure and reduces the construction cost of the complex neural network model. Furthermore, it leverages the phase difference between the query complex vector and the key complex vector in the complex plane to enhance the representational ability of the attention mechanism, thereby improving data processing capabilities, task processing performance, and ultimately, the accuracy of the task results.

[0131] Based on any of the above embodiments, in this method, the complex neural network model includes a complex feedforward network, which includes a generalized complex linear layer and an activation function layer connected in sequence; the activation function layer is used to apply real-valued nonlinear activation functions to the real and imaginary parts of the output complex vector of the generalized complex linear layer connected to it.

[0132] This invention introduces nonlinear processing capabilities into complex neural network models, providing a simple, computationally efficient, and numerically stable technical approach. The powerful expressive power of complex neural networks stems not only from linear transformations but also from nonlinear activation functions. This invention addresses the challenge of effectively introducing nonlinearity into the complex domain while maintaining compatibility with generalized complex linear layers and real-number computation frameworks.

[0133] In one embodiment, in the feedforward network part, the two real linear layers in the traditional real neural network model are uniformly replaced with two generalized complex linear layers, and an activation function layer adapted to complex representation is inserted in the middle. That is, the complex feedforward network includes a generalized complex linear layer, an activation function layer and a generalized complex linear layer connected in sequence.

[0134] Let the complex hidden state of the input of the first generalized complex linear layer be... The first generalized complex linear layer typically expands the input dimension from D to intermediate dimensions. The second one is from Map back to D. The complex feedforward network is shown below: ; ; ; ; In the formula, This represents the output of the first generalized complex linear layer; This represents the first complex parameter of the first generalized complex linear layer; The second complex parameter represents the first generalized complex linear layer; This represents the output of the activation function layer; Represents a complex vector The conjugate vector; Represents the complex activation function; This represents the output of the second generalized complex linear layer; Denotes the first complex parameter of the second generalized complex linear layer; This represents the second complex parameter of the second generalized complex linear layer; Represents a complex vector The conjugate vector; This represents the output of a complex feedforward network.

[0135] For example, the activation function layer receives an output complex vector as input, let the output complex vector be... ,in, This indicates that the real part of the output complex vector is represented. This represents the imaginary part of the output complex vector. Instead of performing some complex function transformation on the output complex vector itself, the activation function decomposes it into two real vectors, one with a real part and the other with an imaginary part. Then, the same standard real-valued nonlinear activation function is applied to these two real vectors separately and independently, thereby recombinating the two real vectors after the activation function into a new complex vector.

[0136] For example, the activation function layer is shown below: ; In the formula, Represents the complex activation function. Represents a real-valued nonlinear activation function. This indicates that the real part of the output complex vector is represented. This indicates the imaginary part of the output complex vector.

[0137] Here, the real-valued nonlinear activation function can be any commonly used and well-performing real-domain activation function in deep learning, such as ReLU (Rectified Linear Unit), GELU (Gaussian Error Linear Unit), or SwiGLU, which are any differentiable or piecewise differentiable real-valued nonlinear functions.

[0138] It should be understood that this component-based activation method, while maintaining nonlinearity, performs its calculations entirely in the real number domain, structurally maintains the separation of the real and imaginary channels, and is compatible with existing activation functions.

[0139] It should be noted that complex activation functions such as zReLU and modReLU, which are specifically designed for complex numbers, can also be used. Without changing the generalized complex linear layer structure, a suitable activation function can be selected according to the specific task.

[0140] It should be understood that by expanding the above operations in the real number domain, this complex feedforward network can be completely equivalent to one with a real-valued hidden dimension of The feedforward network has two weight matrices with specific block structures. and For a real matrix, the activation function in the real domain is represented by applying the same real-valued activation independently to both the first and second halves of the matrix. Therefore, in practical implementations, it is only necessary to follow the aforementioned mapping relationship... By converting it into the corresponding real-valued block matrix, the complex feedforward network of this invention can be implemented using existing feedforward network operators, thereby ensuring alignment with traditional real-valued neural network models in terms of FLOPs, parameter size, and acceleration framework.

[0141] In other words, the component activation method proposed in this invention is computed entirely in the real number domain. This means that in the key step of introducing nonlinearity, highly optimized computational cores written for standard real number activation functions (such as ReLU and GELU) within deep learning frameworks can be directly utilized. This ensures that the computational efficiency of the complex feedforward network is almost identical to that of the standard real number feedforward network, greatly guaranteeing the efficiency and compatibility of engineering implementation. In other words, it achieves efficient and compatible nonlinearity introduction, thereby improving data processing efficiency and ultimately task processing efficiency.

[0142] It should be understood that the design of this complex feedforward network can be viewed as a real feedforward network with a specific block structure for the weight matrix and an activation function that acts independently on the two parts of the concatenated vector. This ensures that the complex neural network model proposed in this embodiment of the invention maintains structural and computational equivalence with the real neural network model in all core components (linear layers, attention layers, and feedforward networks), forming a highly unified architecture.

[0143] It should be understood that by applying bounded or well-behaved real-valued nonlinear activation functions to the real and imaginary parts respectively, the numerical range of the activated vector can be effectively controlled. Compared to some complex variable activation functions that may cause drastic changes in the magnitude, this component-based approach helps avoid gradient explosion or vanishing problems in deep networks, thereby enhancing the numerical stability and convergence of the training process of large-scale complex neural network models, that is, helping to maintain numerical stability.

[0144] The data processing method based on a complex neural network model provided in this invention, through the aforementioned component activation method, ensures that the computation occurs entirely in the real number domain. This guarantees that the computational efficiency of the complex feedforward network is almost identical to that of the standard real feedforward network, greatly ensuring the efficiency and compatibility of engineering implementation. In other words, it achieves efficient and compatible nonlinearity introduction, thereby improving data processing efficiency and, consequently, task processing efficiency.

[0145] Based on any of the above embodiments, in this method, step 120 includes: When the input data is real number input data, the input data is input into the input layer of the complex neural network model to obtain the real number vector output by the input layer; The real number vector is split to obtain a third component and a fourth component; Based on the third component and the fourth component, a complex vector is determined; the real part of the complex vector is the third component, and the imaginary part of the complex vector is the fourth component. The complex vector is input into the next layer of the input layer in the complex neural network model to obtain the task result output by the complex neural network model; When the input data is complex, the complex input data is input into the complex neural network model to obtain the task result output by the complex neural network model.

[0146] It should be noted that the embodiments of the present invention provide a unified input interface for complex neural network models, enabling them to seamlessly process inputs from different modalities and with different data structures (real or complex numbers). This greatly expands the applicability of the model, allowing it to be used as a general-purpose infrastructure in complex scenarios such as multimodal learning.

[0147] For example, when the input data is in native real number form (real number input data), such as natural language text, discrete category labels, or tabular data, the processing flow is as follows.

[0148] First, real-valued input data is fed into the input layer of the complex neural network model. This input layer is typically an embedding layer. Its function is to map the discrete input into a high-dimensional, continuous real-valued vector. For example, for a model with a vocabulary size of V and a hidden dimension of 2D, the input layer will map a word to a 2D real-valued vector.

[0149] Secondly, after obtaining a 2D real vector, it is split to obtain two D-dimensional components, namely the third and fourth components. Subsequently, a D-dimensional complex vector is constructed based on these two components. This construction rule is consistent with the real-complex correspondence between layers within the model, that is, one component is used as the real part of the complex vector, and the other component is used as the imaginary part.

[0150] Then, the complex vector constructed in the previous step is fed as input into the next layer of the input layer in the complex neural network model (e.g., the first generalized complex linear layer or the first attention layer).

[0151] Through this process, the original real-number input data is smoothly and structurally transformed into the complex domain, enabling all subsequent network layers to be processed under a unified complex representation.

[0152] For example, when the input data is in its original complex form (complex input data), such as the complex spectrum of a speech signal after STFT, the complex echo of a radar signal, or the complex amplitude vector of a quantum state, the processing flow is more direct, that is, the complex input data is directly input into the complex neural network model for processing.

[0153] Typically, this means using these complex vectors or tensors directly as input to the first computational layer of the model (such as a generalized complex linear layer). Alternatively, a complex positional encoding module can be used to inject positional information into these complex input data, depending on the requirements.

[0154] For example, in order to consistently process two types of input (complex input data and real input data) under a unified framework, embodiments of the present invention introduce a unified "real-complex correspondence convention" in the embedding layer and intermediate activation representation.

[0155] For the original real-number neural network model, let its vocabulary size be... The embedding vector of each token. for Its corresponding embedding matrix for In this embodiment of the invention, each embedding vector is split into two segments (the third component and the fourth component) in the last dimension, i.e. ,in, Indicates the third component, Represent the fourth component; and construct the corresponding complex vector according to the aforementioned convention. This process can be viewed as applying the same real-imaginary part concatenation rules as the intermediate activation to the embedding layer, thereby ensuring that the entire network uses a unified real-imaginary correspondence across all layers.

[0156] For native complex inputs, such as speech domain features or quantum state This invention directly treats it as the complex input data of the model. ,Right now ,or The data is then further mapped to the model's hidden space through a generalized complex linear layer or a complex positional encoding module. If compatibility with real number operators is required, the data can be converted to a real-valued representation according to the aforementioned equivalence rules, thus maintaining consistency with the aforementioned equivalence relation of real number block matrices.

[0157] For the activation of intermediate layers, the embodiments of the present invention uniformly adopt the following convention: complex hidden states of any layer Recorded as: ; In the formula, ; When alignment or transformation with a real-valued neural network model is required, the corresponding real-valued hidden state is... for: ; Thus, from the embedding layer to the output of each layer, the real-valued hiding dimension is always maintained in this embodiment of the invention. The hidden dimension of complex numbers is This unified correspondence provides a foundation for the equivalence of all subsequent linear layers, attention layers, and feedforward networks.

[0158] It should be understood that this embodiment of the invention defines a dual-channel input processing mechanism for the model, enabling it to natively and seamlessly process two completely different types of data: real numbers and complex numbers. This allows the same complex neural network model architecture to be applied to various tasks, including natural language processing (real number input), speech processing (complex number input), and physics simulation (complex number input), and even to multimodal tasks that require simultaneous processing of text and speech. This greatly enhances the versatility and application scope of the model architecture, achieving compatibility and versatility for multimodal inputs.

[0159] It should be understood that, for real number inputs, this embodiment of the invention employs the exact same real / imaginary part splitting and concatenation rules at the input layer as the intermediate layers of the model. This means that from the very first step of data entering the model, the transformation from the real domain to the complex domain follows a unified correspondence that runs throughout the entire network. This end-to-end representation consistency is the foundation for ensuring that the various components of the model can work collaboratively, that parameters can be smoothly transformed, and that the entire system remains mathematically rigorous and stable.

[0160] It should be understood that developers can flexibly configure input modules for the model according to specific task requirements. For a complex system that needs to handle multiple modal inputs, a corresponding input front-end (real number embedding layer or direct complex number interface) can be designed for each modality, while the main body of the model (composed of generalized complex linear layers, complex attention layers, etc.) remains unchanged. This modular design of the input end simplifies the construction process of multimodal models and improves development efficiency and model scalability.

[0161] The data processing method based on a complex neural network model provided in this invention enables the same complex neural network model to be applied to multiple tasks, including real-number input data and complex-number input data, thereby achieving the model's compatibility and versatility with multimodal inputs and improving the applicability of the complex neural network model.

[0162] Based on any of the above embodiments, in this method, the complex neural network model includes a complex detection head; the complex detection head is the generalized complex linear layer.

[0163] The complex neural network model is used to determine the real part of the complex detection vector output by the complex detection head as a real detection vector; or, The complex neural network model is used to concatenate the real and imaginary parts of the complex detection vector output by the complex detection head to obtain a real detection vector.

[0164] Here, the complex detector head is the last (or several) layers of the complex neural network model, responsible for mapping the hidden state representations of the model's highest layers to the final task space. For example, in a language model, it is responsible for mapping to a logit space the size of the vocabulary; in a regression task, it is responsible for mapping to the target real value.

[0165] In this embodiment of the invention, the complex detection head is implemented as a generalized complex linear layer. This means that it also follows the above definition of a generalized complex linear layer and has its own independent trainable first and second complex parameters.

[0166] For example, the complex detection head receives the complex hidden state of the last layer of the model. As input, it outputs a complex detection vector; the complex detection head is shown below: ; In the formula, Represents a complex detection vector; , The first complex parameter of the generalized complex linear layer; , The second complex parameter represents the generalized complex linear layer; Represents a complex vector The conjugate vector.

[0167] Since most standard loss functions (such as cross-entropy loss) and subsequent processing modules (such as sampling decoding) are based on real number inputs, it is necessary to convert the complex detection vector into a real number detection vector.

[0168] In one embodiment, the real part of the complex detection vector is determined as the real detection vector. Specifically, the real part of the complex detection vector is directly extracted as the final real detection vector. This method is the most direct and computationally efficient. This method can be used when the dimension of the task output space is the same as the output dimension of the model's complex detection head (e.g., the vocabulary size V of the language model is equal to the complex dimension of the detection head's output). For example, as shown below: ; In the formula, Represents a real-valued detection vector. Indicates taking the real part, This represents a complex detection vector.

[0169] In another embodiment, the real and imaginary parts of the complex detection vector are concatenated to obtain the real detection vector. Specifically, the real and imaginary parts of the complex detection vector are concatenated to form a longer-dimensional real vector, which serves as the real detection vector. For example, if the dimensions of the real and imaginary parts are V, the concatenation yields a real vector with dimension 2V. This method preserves all the information (real and imaginary parts) of the complex detection vector. In some cases, this 2V-dimensional vector can be further reduced back to the required V dimensions using an optional, fixed, or learnable real linear layer, thereby satisfying the dimensionality requirements of the task while retaining more information. For example, as shown below: ; In the formula, Represents a real-valued detection vector. Indicates taking the real part, This indicates taking the imaginary part. Represents a complex detection vector. This represents a real-valued reduced-dimensional matrix.

[0170] It should be understood that by implementing the output (detection head) as a generalized complex linear layer, this embodiment of the invention ensures that the entire model, from input, intermediate layers, attention to output, follows a unified mathematical structure. More importantly, since there is a strict equivalence mapping between the generalized complex linear layer and the real number block matrix, this means that there is also a clear and unique mapping between the complex number detector head and its equivalent real number detector head. This end-to-end consistency and equivalence is the foundation for the model's stability, interpretability, and ease of conversion, thus guaranteeing end-to-end architectural consistency and equivalence.

[0171] It should be understood that the two paths provided in this embodiment of the invention for converting complex detection vectors into real detection vectors enable the final output of the model to be directly accepted by all existing standard modules based on real numbers. This includes loss functions and decoding and sampling strategies, allowing the direct use of standard cross-entropy loss, mean squared error loss, etc., without any modifications. Furthermore, it allows the direct application of all standard algorithms designed for real number logits, such as greedy decoding, beam search, and kernel sampling. This seamless compatibility greatly reduces the complexity of applying complex neural network models to practical tasks and ensures their stable integration into existing training and inference processes.

[0172] It should be understood that the embodiments of the present invention provide two conversion methods from complex number output to real number output, giving developers the flexibility to choose according to specific task requirements. Method one is computationally simple and efficient; method two retains more information and may perform better in tasks that require richer output representation. This flexibility allows the model to better adapt to the characteristics of different tasks, thereby potentially achieving better performance in a wider range of applications.

[0173] The data processing method based on a complex neural network model provided in this invention converts complex detection vectors into real detection vectors in the above manner, so that the final output of the model can be directly accepted by all existing standard modules based on real numbers. This greatly reduces the complexity of applying the complex neural network model to practical tasks, ensures that it can be stably integrated into the existing training and inference process, and thus reduces the construction cost of the complex neural network model.

[0174] To facilitate understanding of the above embodiments, two specific embodiments are also provided to demonstrate their applications in end-to-end speech modeling and neural network-based quantum state simulation, further verifying the feasibility and versatility of the complex neural network model.

[0175] In one particular embodiment, a complex neural network architecture and training scheme for an end-to-end large speech model is described.

[0176] This invention demonstrates a complete implementation of the invention in large-scale speech model scenarios, including input format, embedding strategy, self-supervision and multi-task objectives, complexification of attention, and a training process that can be directly deployed in engineering. This embodiment showcases the advantages of the invention in processing signals naturally belonging to the complex domain (e.g., the short-time Fourier transform spectrum of speech) and demonstrates the characteristics of generalized complex linear layers in terms of representational structure and numerical stability.

[0177] In speech tasks, the input signal is typically subjected to a short-time Fourier transform (STFT) to obtain a complex spectrum, with each frame containing a time-frequency vector consisting of a real part and an imaginary part. This invention provides two input paths.

[0178] The first approach is to directly use native complex embedding (complex input data) to convert the complex spectrum of each frame. Concatenate into a complex vector This serves as the input to the complex neural network model; at the lexical or text input end, two sets of weight matrices can also be learned. (i.e., the first complex parameter and the second complex parameter), forming a complex embedding. This approach requires no additional processing, allowing the model to flow in a complex domain across the entire network depth, maximizing the geometric properties of the amplitude-phase structure.

[0179] The second approach is used in scenarios with existing real-valued acoustic front-ends, such as mel-spectrograms, convolutional front-ends, or other real-domain signal processing modules. In this case, embodiments of the invention apply a real-complex equivalence mapping to the first input linear layer (or a 1×1 convolutional layer), transforming it into a generalized complex linear layer, thus enabling real-domain input... Equivalently mapped to a complex vector From this point onward, all layers are recursively processed in the complex domain. Since the complexification transition involves only one weight transformation, and its form is consistent with that given in this invention... The mapping is strictly consistent, thus ensuring that existing real-valued acoustic models are smoothly complexified within the overall framework of the speech model.

[0180] In the attention mechanism, this embodiment of the invention fully employs generalized complex linear layers to define the query, key, and value projections. To ensure comprehensive representation of the complex domain and compatibility with conjugate symmetric structures, the query, key, and value are generated by three generalized complex linear layers respectively. Furthermore, the attention score uses the real part of the complex inner product as the attention score. This design guarantees that the score is real, thus maintaining numerical stability, and also enables the model to capture the phase relationship between complex vectors. In engineering implementation, the real part of the complex inner product can be equivalently implemented as two real matrix multiplications, thus directly reusing existing real operators without introducing new low-level implementations.

[0181] Attention convergence and output projection are also achieved through complex-real equivalent expansion. For example, the final output can be written as: ; In the formula, This indicates the final output. Indicates the result of attention. Denotes the first complex parameter of the generalized complex linear layer. This represents the second complex parameter of the generalized complex linear layer. The FLOPs and parameter size of the entire attention process are completely consistent with the real-number model of equal width.

[0182] In the feedforward network, the embodiments of the present invention employ a complex two-layer structure, namely: ; ; ; In the formula, This represents the output of the first generalized complex linear layer; This represents the first complex parameter of the first generalized complex linear layer; The second complex parameter represents the first generalized complex linear layer; This represents the complex hidden state of the input to the first generalized complex linear layer. Represents a complex vector The conjugate vector; , This is a preset vector; This represents the output of the activation function layer; Represents the complex activation function; This represents the output of the second generalized complex linear layer; Denotes the first complex parameter of the second generalized complex linear layer; This represents the second complex parameter of the second generalized complex linear layer; Represents a complex vector The conjugate vector.

[0183] Among them, activation function Component-based nonlinearity is employed, for example, applying GELU or SwiGLU to the real and imaginary parts respectively. This feedforward network module is equivalent in the real domain to a standard feedforward structure that applies the same activation function to each of the real-imaginary concatenation vectors; therefore, its computational requirements are completely consistent with those of real-valued feedforward networks. Embodiments of this invention are also compatible with activation forms specifically designed for complex domains, such as modReLU and zReLU, which can be freely selected according to different acoustic tasks.

[0184] To achieve a large end-to-end speech model, this invention constructs a unified multi-task training framework, including tasks such as self-supervised acoustic modeling, ASR, TTS, and speech translation. Regardless of whether the input is speech or text, the output head can be uniformly implemented through a generalized complex linear layer to ensure that the entire model maintains consistency in the complex domain from input to output.

[0185] In the pre-training phase from scratch, this embodiment of the invention employs a complex-domain-friendly target design. First, for the complex spectral mask recovery task, the model needs to predict the complex frequency domain vector of the masked time-frequency block, which can be achieved using L2 loss, SI-SDR loss, or complex-domain reconstruction loss in the amplitude-phase domain. Second, by imposing phase smoothing or geometric consistency constraints, the predicted vector is ensured to have a reasonable envelope structure in the complex plane. Furthermore, this embodiment of the invention utilizes the complex-domain semantic representation of the attention output for cross-block contrastive learning or autoregressive prediction, thereby establishing the ability to represent speech in the time-frequency space.

[0186] In terms of engineering implementation, the training strategy of this invention remains completely consistent with that of large real-domain models, including the use of the AdamW optimizer, direct application of LayerNorm to the real-to-virtual concatenation vectors, gradient pruning, and cosine annealing learning rate scheduling. Since the complex structure provided by this invention can be fully expanded into standard operators in the real domain, there is no need to design special normalization or initialization methods for complex-domain networks, nor is it necessary to change the existing training process. Through complex-to-real equivalence mapping, this invention can train a large speech model with stronger frequency structure expressiveness and phase robustness without increasing computational power.

[0187] In summary, the embodiments of this invention demonstrate a complete implementation of the complex neural network model of this invention in the field of speech, reflecting its natural compatibility with complex spectra, structural stability, cross-task versatility, and complete compatibility with the existing hardware ecosystem.

[0188] In another specific embodiment, a complex neural network model based on neural network quantum states is presented. This embodiment of the invention demonstrates a second core application in the AI4Science field: neural network quantum state modeling for quantum physics systems. Quantum states are essentially complex vectors or complex wave functions, thus exhibiting a high degree of compatibility with the generalized complex linear layer and complex neural network architecture proposed in this invention.

[0189] In the quantum many-body problem, the quantum state of a system is usually represented as: Its representation in high-dimensional space requires simultaneous encoding of amplitude. and phase Traditional real-valued neural networks require decomposing complex numbers into two real channels for modeling, which not only disrupts the geometry of the complex state space but also makes it difficult to capture conjugation relations, symmetries, and phase interference effects in quantum systems. In contrast, the generalized complex linear layer form proposed in this invention is naturally suitable for modeling quantum state behavior that includes both linear evolution and conjugation coupling, and can cover the general linear operator structure in quantum systems.

[0190] Specifically, in quantum state modeling tasks, the input space is typically a set of particle configurations. or spin configuration The learned complex embeddings can be mapped to a sequence of complex vectors. The complex neural network model of this invention, through multi-layer complex self-attention, can encode characteristics such as long-range correlations, entanglement structures, and superposition interference between particles into the hidden vectors. Since the scoring of self-attention is based on the real part of the complex inner product, the embodiments of this invention can simulate properties similar to Hermitian operators, making the attention weights physically reasonable; while the context representation of complex values ​​(attention results) naturally characterizes the complex amplitude propagation process of quantum states.

[0191] In quantum state generation or energy prediction tasks, the final output of a complex neural network model can be converted into a wave function through a complex language head (or a complex-valued energy head). Or its logarithmic magnitude and phase representation. For example, for a given input configuration Model output ; In the formula, In response to several amplitude values, The corresponding phase perfectly matches the mathematical form of the quantum state. Through the complex-real equivalence mapping, this output can also be converted into a real scalar energy or used in variational Monte Carlo (VMC) gradient estimation.

[0192] During training, embodiments of this invention can employ log-likelihood, energy expectation minimization, or consistency constraints on the amplitude-phase of the wavefunction. Since the generalized complex linear layer of this invention can completely cover the complex linear-antilinear combination space, its expressive power is far superior to that of naive complex networks. Therefore, it can significantly improve accuracy and stability when learning difficult aspects such as phase structure, symmetry breaking, and parity constraints in quantum states. Furthermore, because this invention provides a strict equivalence relationship between complex and real number models, the entire quantum state modeling process can still run in a highly efficient, fully real-valued acceleration library without additional modifications to the computation graph, thus possessing extremely high scalability in large-scale training.

[0193] In summary, the embodiments of this invention demonstrate typical applications of the invention in quantum physics, electronic structure calculation, and many-body system simulation, showing that complex neural network models have a natural fit and significant advantages in this field.

[0194] Based on the above embodiments, considering that the structure of existing complex neural networks often lacks uniformity, only introducing complexification processing in local modules (such as single-layer linear projection or certain convolutional kernels), lacking a systematic complex architecture that spans embedding, attention mechanisms, feedforward networks, and output layers. This localized complexification is difficult to maintain optimization stability at large model scales and is also difficult to transfer to complex multimodal tasks. Furthermore, considering that existing technologies cannot simultaneously meet multiple requirements such as completeness of complex domain representation, strict equivalence with real number models, engineering feasibility, and cross-modal scalability, it is difficult to support the training and application of future large-scale complex models. Therefore, this invention proposes a complex neural network architecture based on generalized complex linear layers, achieving a comprehensive breakthrough from mathematical structure and network design to engineering deployment. Specifically, the purpose of this invention is to propose a complex domain modeling method for large-scale neural network models. The core includes a generalized complex linear layer with full mathematical expression capabilities, strictly aligned with real-number neural network architectures, and capable of stable expansion in existing mainstream engineering systems, as well as a complete complex neural network architecture built based on this generalized complex linear layer. This invention aims to solve the problems of existing complex models being difficult to train stably, lacking invertible mappings with real-number models, being unable to be integrated with real-number models, and lacking unified component design, so that the modeling capabilities of complex space can be fully utilized on a large model scale.

[0195] More specifically, this invention proposes constructing a complex Transformer model based on a generalized complex linear layer as the fundamental unit. This generalized complex linear layer integrates complex linear and antilinear terms into the same operator, enabling any affine transformation in the complex domain to be represented by the same structure. Mathematically, this operator is equivalent to a 2×2 block matrix with a specific structure in the real domain, thus allowing for efficient implementation within a real-valued framework. At the engineering level, this means that the complex model of this invention can be seamlessly mapped to the existing real-valued deep learning ecosystem, including matrix multiplication kernels, attention acceleration operators, training and inference frameworks, thereby achieving high scalability while maintaining expressive power.

[0196] Furthermore, based on this generalized complex linear layer, this invention constructs a unified complex neural network architecture, including all core components such as complex embedding, complex positional encoding, complex attention mechanism, complex feedforward network, and complex activation function. These components are designed to be fully aligned with traditional real neural network models, maintaining consistency in dimensionality, parameter size, and computational complexity, enabling complex neural network models to possess both stronger expressive power and the ability to directly utilize the pre-training results of real neural network models.

[0197] Furthermore, this invention proposes a series of engineering practices adapted to the complex domain, including a complex activation function system compatible with complex linear structures and attention functions constructed in the complex domain. These collectively constitute a complex neural network architecture that can be used for large-scale training, extended across modalities, reused from existing real-number models, and achieves performance improvements across various physical and signal domain tasks.

[0198] In summary, this invention solves the long-standing problems of complex neural networks in mathematics, engineering and application through the generalized complex linear layer and its derived complete set of complex neural network components, making large complex models a structured, standardized and scalable model architecture, thus providing a unified solution for large speech models, quantum state modeling, multimodal learning and complex scientific computing.

[0199] In other words, this invention proposes a generalized complex linear layer capable of establishing a strictly invertible mapping between the complex and real domains, and uses this as the basis for constructing a complete complex neural network model architecture. This generalized complex linear layer, by simultaneously introducing complex linear and antilinear paths, achieves a complete expression of general complex affine transformations, fundamentally breaking through the limitation of traditional complex neural networks that can only represent single-path structures. More importantly, this invention formally provides a one-to-one correspondence between arbitrary real linear layers and generalized complex linear layers. Through an explicit real-complex mapping formula, the complex weight matrix and the real block matrix are precisely interchanged, thereby ensuring complete equivalence of the complex neural network architecture between the complex and real domains.

[0200] Building upon this structure, this invention further derives the self-attention module, feedforward network, embedding layer, and output layer all into the complex domain, constructing a complex neural network framework with a fully aligned topology and consistent parameter count and computational cost with real-number models. Furthermore, by using the real part of the complex inner product as the attention score, this invention ensures numerical stability while enabling the model to naturally handle phase relationships, conjugate coupling, and frequency domain structures. Since all complex operations can be rigorously expanded into structured real matrix multiplications, this invention can seamlessly reuse existing GPU and NPU acceleration operators, attention optimization operators, and inference frameworks in engineering, thereby achieving trainability and deployability of large-scale complex models.

[0201] Furthermore, this invention provides, for the first time, a unified embedding strategy, normalization method, and training process for complex Transformers, enabling the model to simultaneously handle native complex inputs (such as STFT complex spectra and quantum state wave functions), traditional real inputs (such as text embeddings), and multimodal mixed inputs. Building upon this, the invention presents typical implementations suitable for large speech models and neural network quantum states (NQS), further demonstrating the significant advantages of this architecture in frequency domain structure modeling, phase representation capabilities, and quantum state complex amplitude modeling.

[0202] In summary, this invention proposes a generalized complex linear layer with both complex linear and antilinear pathways, and constructs a complete and unified complex neural network architecture based on this layer. This invention achieves significant advantages in terms of mathematical expression, numerical stability, engineering feasibility, and cross-modal scalability.

[0203] Furthermore, this invention possesses inherent advantages in handling tasks with strong phase structures or frequency domain characteristics. For example, in inputs such as speech signals, radar signals, and telecommunication signals, which are natively represented in complex form, traditional real-number models require additional phase encoding or complex data augmentation to compensate for phase instability. In contrast, this invention directly performs projection, attention calculation, and feedforward transformation in the complex domain, allowing phase relationships to be represented and propagated within the model in the most natural way, significantly improving the model's robustness to phase perturbations, phase envelope changes, and frequency domain mode migration. In physics simulations and quantum computing tasks, quantum states and wave functions naturally reside in complex vector spaces. This invention can directly model the amplitude and phase structures of these objects, ensuring that the model's representational power aligns with the mathematical structure of the physical system itself, thus significantly outperforming traditional real-number neural networks in terms of accuracy, stability, and interpretability.

[0204] In terms of overall architecture design, this invention makes full use of the natural separation characteristics of amplitude and phase in the complex domain, and maintains structural stability in attention mechanism, activation function and normalization strategy, so that the complex neural network model performs more robustly when facing practical tasks such as frequency disorder, signal aliasing and noise disturbance.

[0205] The design of using the real part of the complex inner product as the attention score enables the attention mechanism to directly capture phase differences in the complex domain, while ensuring that the computation process maintains the real value form, thus combining expressiveness and engineering stability.

[0206] Furthermore, by using component activation and a complexity-controlled complex feedforward network, this invention avoids the gradient explosion and numerical instability problems that are traditionally prone to occur in complex networks while maintaining high expressive power, enabling the model to converge reliably in large-scale training environments.

[0207] This invention offers greater flexibility in multimodal learning scenarios. Because the same model can accommodate native complex inputs, real inputs, and mixtures of both, text, speech, images, and scientific computing data can be modeled within a unified complex neural network framework. In multimodal alignment tasks, this invention provides a richer representation space through the complex domain, enabling the model to more naturally align modalities with different geometries and encode more cross-modal relationships through combinations of amplitude and phase. This capability has broad application potential in future complex system modeling, communication system optimization, and large-scale multimodal models.

[0208] Therefore, this invention has significant advantages in terms of expressiveness, engineering feasibility, transferability, stability, and cross-modal scalability. It can serve as a unified infrastructure for large-scale complex neural networks and effectively promote the widespread application of complex large models in multiple fields.

[0209] The data processing apparatus based on a complex neural network model provided by the present invention is described below. The data processing apparatus based on a complex neural network model described below and the data processing method based on a complex neural network model described above can be referred to in correspondence with each other.

[0210] Figure 2 This is a schematic diagram of the data processing device based on a complex neural network model provided by the present invention, as shown below. Figure 2 As shown, the data processing device based on the complex neural network model includes: an acquisition module 210 and an output module 220.

[0211] The acquisition module 210 is used to acquire input data.

[0212] The output module 220 is used to input the input data into the complex neural network model to obtain the task result output by the complex neural network model.

[0213] The complex neural network model includes a generalized complex linear layer; the generalized complex linear layer is used to perform complex linear mapping operations on the input complex input vector and to perform inverse linear mapping operations on the conjugate vector of the complex input vector; the complex linear mapping operation is performed through a trainable first complex parameter, and the inverse linear mapping operation is performed through a trainable second complex parameter.

[0214] Figure 3 An example is a schematic diagram of the physical structure of an electronic device, such as... Figure 3As shown, the electronic device may include a processor 310, a communications interface 320, a memory 330, and a communication bus 340, wherein the processor 310, communications interface 320, and memory 330 communicate with each other via the communication bus 340. The processor 310 can call logical instructions in the memory 330 to execute a data processing method based on a complex neural network model. This method includes: acquiring input data; inputting the input data into the complex neural network model to obtain the task result output by the complex neural network model; wherein the complex neural network model includes a generalized complex linear layer; the generalized complex linear layer is used to perform complex linear mapping operations on the input complex input vector and inverse linear mapping operations on the conjugate vector of the complex input vector; the complex linear mapping operation is performed using trainable first complex parameters, and the inverse linear mapping operation is performed using trainable second complex parameters.

[0215] Furthermore, the logical instructions in the aforementioned memory 330 can be implemented as software functional units and, when sold or used as independent products, can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, essentially, or the part that contributes to the prior art, or a part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of the present invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0216] On the other hand, the present invention also provides a computer program product, which includes a computer program that can be stored on a non-transitory computer-readable storage medium. When the computer program is executed by a processor, the computer can execute the data processing method based on a complex neural network model provided by the above methods. The method includes: acquiring input data; inputting the input data into a complex neural network model to obtain the task result output by the complex neural network model; wherein the complex neural network model includes a generalized complex linear layer; the generalized complex linear layer is used to perform complex linear mapping operations on the input complex input vector and to perform inverse linear mapping operations on the conjugate vector of the complex input vector; the complex linear mapping operation is performed through a trainable first complex parameter, and the inverse linear mapping operation is performed through a trainable second complex parameter.

[0217] In another aspect, the present invention also provides a non-transitory computer-readable storage medium storing a computer program thereon, which, when executed by a processor, implements the data processing method based on a complex neural network model provided by the above methods. The method includes: acquiring input data; inputting the input data into a complex neural network model to obtain a task result output by the complex neural network model; wherein the complex neural network model includes a generalized complex linear layer; the generalized complex linear layer is used to perform complex linear mapping operations on the input complex input vector and inverse linear mapping operations on the conjugate vector of the complex input vector; the complex linear mapping operation is performed using trainable first complex parameters, and the inverse linear mapping operation is performed using trainable second complex parameters.

[0218] The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs. Those skilled in the art can understand and implement this without any creative effort.

[0219] Through the above description of the embodiments, those skilled in the art can clearly understand that each embodiment can be implemented by means of software plus necessary general-purpose hardware platforms, and of course, it can also be implemented by hardware. Based on this understanding, the above technical solutions, in essence or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, magnetic disk, optical disk, etc., and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute the methods described in the various embodiments or some parts of the embodiments.

[0220] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and not to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A data processing method based on a complex neural network model, characterized in that, include: Get the input data; The input data is fed into a complex neural network model to obtain the task result output by the complex neural network model; The complex neural network model includes a generalized complex linear layer; the generalized complex linear layer is used to perform complex linear mapping operations on the input complex input vector and to perform inverse linear mapping operations on the conjugate vector of the complex input vector; the complex linear mapping operation is performed through a trainable first complex parameter, and the inverse linear mapping operation is performed through a trainable second complex parameter.

2. The data processing method based on a complex neural network model according to claim 1, characterized in that, The generalized complex linear layer outputs a real number output vector or a complex number output vector; The real number output vector is determined based on the following method: The real part vector and the imaginary part vector of the complex input vector are concatenated to obtain the real input vector, and the real parameters are determined based on the first complex parameter and the second complex parameter. Based on the real number input vector and the real number parameters, determine the real number output vector; The complex output vector is determined based on the following method: The real number output vector is split to obtain a first real number output component and a second real number output component; The complex output vector is determined based on the first real number output component and the second real number output component.

3. The data processing method based on a complex neural network model according to claim 2, characterized in that, The step of determining the real number parameter based on the first complex number parameter and the second complex number parameter includes: Based on the first complex parameter and the second complex parameter, four real number sub-parameters are generated; The four real number sub-parameters are combined to obtain the real number parameters; The real number parameters are as follows: ； In the formula, Represents the real number parameter, , , , Represents the four real number sub-parameters, This represents the real part of the first complex number parameter. This represents the imaginary part of the first complex number parameter. This represents the real part of the second complex number parameter. This represents the imaginary part of the second complex number parameter.

4. The data processing method based on a complex neural network model according to claim 1, characterized in that, When the first complex parameter and the second complex parameter are generated based on the real parameters of the real linear layer, the first complex parameter and the second complex parameter are generated in the following manner: The real number parameters are divided into blocks to obtain four real number sub-parameters; Based on the four real number sub-parameters, the first complex number parameter and the second complex number parameter are generated; Wherein, the input dimension of the real linear layer is twice the input dimension of the generalized complex linear layer, the real part of the complex input vector is regarded as the first component, and the imaginary part of the complex input vector is regarded as the second component; the first component and the second component are obtained by splitting the real input vector input to the real linear layer.

5. The data processing method based on a complex neural network model according to claim 4, characterized in that, The real number parameters are as follows: ； In the formula, Represents the real number parameter, , , , Represents the four real number sub-parameters; The first complex parameter is generated based on the following formula: ； ； ； In the formula, Represents the first complex number parameter. This represents the real part of the first complex number parameter. The parameter represents the imaginary part of the first complex number parameter; The second complex parameter is generated based on the following formula: ； ； ； In the formula, This represents the second complex parameter. This represents the real part of the second complex number parameter. This represents the imaginary part of the second complex number parameter.

6. The data processing method based on a complex neural network model according to claim 1, characterized in that, The complex neural network model includes an attention layer; the attention layer includes three generalized complex linear layers. The attention layer outputs attention results based on the following method: Based on the three generalized complex linear layers, the input complex vector of the attention layer is mapped to the query complex vector, the key complex vector, and the value complex vector, respectively. Calculate the attention score between the query complex vector and the key complex vector; Based on the attention score, the complex vector of values ​​is weighted to obtain the attention result.

7. The data processing method based on a complex neural network model according to claim 6, characterized in that, The calculation of the attention score between the query complex vector and the key complex vector includes: The attention score is determined based on the real part of the complex inner product of the conjugate transpose of the query complex vector and the key complex vector; or, The attention score is determined based on the real part of the complex inner product of the query complex vector and the conjugate transpose of the key complex vector.

8. The data processing method based on a complex neural network model according to claim 1, characterized in that, The complex neural network model includes a complex feedforward network, which includes a generalized complex linear layer and an activation function layer connected in sequence. The activation function layer is used to apply real-valued nonlinear activation functions to the real and imaginary parts of the output complex vector of the generalized complex linear layer connected to it.

9. The data processing method based on a complex neural network model according to any one of claims 1 to 8, characterized in that, The step of inputting the input data into a complex neural network model to obtain the task result output by the complex neural network model includes: When the input data is real number input data, the input data is input into the input layer of the complex neural network model to obtain the real number vector output by the input layer; The real number vector is split to obtain a third component and a fourth component; Based on the third component and the fourth component, a complex vector is determined; the real part of the complex vector is the third component, and the imaginary part of the complex vector is the fourth component. The complex vector is input into the next layer of the input layer in the complex neural network model to obtain the task result output by the complex neural network model; When the input data is complex, the complex input data is input into the complex neural network model to obtain the task result output by the complex neural network model.

10. The data processing method based on a complex neural network model according to any one of claims 1 to 8, characterized in that, The complex neural network model includes a complex detection head; the complex detection head is the generalized complex linear layer; The complex neural network model is used to determine the real part of the complex detection vector output by the complex detection head as a real detection vector; or, The complex neural network model is used to concatenate the real and imaginary parts of the complex detection vector output by the complex detection head to obtain a real detection vector.

11. A data processing device based on a complex neural network model, characterized in that, include: The acquisition module is used to acquire input data; The output module is used to input the input data into the complex neural network model and obtain the task result output by the complex neural network model; The complex neural network model includes a generalized complex linear layer; the generalized complex linear layer is used to perform complex linear mapping operations on the input complex input vector and to perform inverse linear mapping operations on the conjugate vector of the complex input vector; the complex linear mapping operation is performed through a trainable first complex parameter, and the inverse linear mapping operation is performed through a trainable second complex parameter.

12. An electronic device comprising a memory, a processor, and a computer program stored in the memory and running on the processor, characterized in that, When the processor executes the computer program, it implements the data processing method based on a complex neural network model as described in any one of claims 1 to 10.

13. A non-transitory computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the data processing method based on a complex neural network model as described in any one of claims 1 to 10.

14. A computer program product, comprising a computer program, characterized in that, When the computer program is executed by a processor, it implements the data processing method based on a complex neural network model as described in any one of claims 1 to 10.

Images