Crifd multi-vector semantic processing method and device based on dynamic mixed curvature product manifold, electronic equipment and storage medium

Abstract

The invention provides a dynamic mixed curvature product manifold-based Crifd multi-vector semantic processing method and device, electronic equipment and a storage medium, and relates to the fields of natural language processing, deep learning and neural network calculation. The method comprises the following steps: mapping an input sequence into a Crifd algebra multi-vector representation; generating a mixed curvature weight of the spherical / hyperbolic / Euclidean three-subspace and continuously normalizing the mixed curvature weight; projecting multiple vectors to each subspace according to the weights; carrying out continuous nonlinear function transformation on coefficient vector amplitudes of a scalar part and a bidirectional vector part based on geometric products in each subspace to form attention scores, carrying out Softmax normalization to obtain attention weights, and carrying out weighted aggregation on multiple vectors to obtain subspace output; outputting the subspace, cutting back the space through logarithm mapping, and outputting a semantic processing result according to weight fusion; the mapping and stabilization mechanism is in a continuous differentiable form and does not contain conditional branches based on threshold comparison.

Description

Technical Field

[0001] This invention relates to the fields of natural language processing, deep learning and neural network computing, and in particular to a Clifford multi-vector semantic processing method, apparatus, electronic device and storage medium based on dynamic hybrid curvature product manifold. Background Technology

[0002] In recent years, attention-based neural network models have been widely used in natural language processing and multimodal learning. To enhance the model's ability to model hierarchical structures, long-range dependencies, and knowledge relationships, some techniques attempt to embed semantic vectors into non-Euclidean spaces (such as hyperbolic or spherical spaces). However, existing solutions typically suffer from the following problems:

[0003] (1) Topological prior is singular: using only a single type of curvature space (e.g., only negative curvature or only positive curvature) makes it difficult to simultaneously characterize the hierarchical structure, cyclic structure and parallel / tiled structure that are common in languages; (2) Numerical stability depends on hard threshold branch: Some schemes introduce threshold-based conditional branches when calculating distance, normalization or mapping (e.g., when the modulus is less than the threshold, an alternative formula is adopted), which leads to discontinuous computation graph, reduced parallel execution efficiency, and instability during training. (3) Limited operator expressive power: Attention scoring often uses scalar operators such as dot product or distance, which are difficult to express higher-order geometric relationships (such as directed area spanned by two semantic vectors, subspace relationship or asymmetric relationship), thus limiting the ability to make complex reasoning.

[0004] Therefore, a new semantic processing method is needed that can adaptively select multiple geometric topologies and utilize stronger geometric operators to express relations while maintaining differentiability and hardware friendliness. Summary of the Invention

[0005] Purpose of the invention: To address the shortcomings of the prior art, this invention proposes a Clifford multi-vector semantic processing method, apparatus, electronic device, and storage medium based on dynamic hybrid curvature product manifolds. This addresses the problems of existing technologies, such as the inability to simultaneously model hierarchical, periodic, and parallel structures due to the single topology of the semantic embedding space; the dependence of numerical stability on threshold branches leading to discontinuous computation graphs, low hardware efficiency, and unstable training; and the difficulty in expressing high-order geometric relationships and asymmetric semantic interactions due to the scalar similarity-based attention operators.

[0006] To achieve the above-mentioned technical objectives, a first aspect of the present invention proposes a Clifford multi-vector semantic processing method based on a dynamically hybrid curvature product manifold, comprising the following steps: Step S1: Obtain the feature representation of the input sequence and map the feature representation to a multi-vector feature representation in the Clifford algebra space. The multi-vector feature representation includes at least a scalar part, a vector part, and a bi-vector part. Step S2: Based on the multi-vector feature representation or its contextual features, a hybrid curvature distribution parameter is generated through a curvature prediction network. The hybrid curvature distribution parameter includes at least positive curvature weights, negative curvature weights, and zero curvature weights. The weight vector is obtained through Softmax normalization, temperature Softmax normalization, or LogSumExp normalization. The weight vector satisfies w i ≥0 and Σw i =1; where the positive curvature weight corresponds to the spherical subspace, the negative curvature weight corresponds to the hyperbolic subspace, and the zero curvature weight corresponds to the Euclidean subspace; Step S3: Based on the hybrid curvature distribution parameters, project the multi-vector feature representation onto the spherical subspace, hyperbolic subspace, and Euclidean subspace respectively to obtain the triplet subspace feature representation (X). - ,X0,X + The projection is used as a product manifold representation for subsequent attention computation, wherein the product manifold representation is a parallel triplet representation; wherein the projection imposes an exponential mapping / logarithmic mapping constraint matching the curvature parameter on at least the vector part (or its learnable linear combination) of the multi-vector, and the scalar part and the two-vector part maintain Euclidean coordinates or adopt a continuous mapping of the same shape; Step S4: Construct query multi-vectors, key multi-vectors, and value multi-vectors in the spherical subspace, hyperbolic subspace, and Euclidean subspace, respectively, and calculate attention scores and subspace attention outputs using the geometric product operator, respectively; wherein, the attention score includes the scalar component of the geometric product result and the outer product scalar term obtained by calculating the magnitude of the coefficient vector based on the coefficient vector of the two vector parts of the geometric product result and transforming it through a continuous nonlinear function; the scalar component and the outer product scalar term are linearly combined to form an attention score matrix, which is then used for Softmax normalization to generate attention weights.

[0007] Step S5: The attention outputs of the spherical subspace, hyperbolic subspace, and Euclidean subspace are respectively tangent back to the logarithmic map space and weighted and fused according to the hybrid curvature distribution parameters to obtain the fusion context representation; Step S6: Input the fusion context representation into the task head to output semantic processing results; the semantic processing results include: retrieval score, re-ranking score or generation conditional context for retrieval augmented generation (RAG) system; reasoning result, loop consistency label or confidence for knowledge graph reasoning or loop closure consistency determination; category probability for text classification; probability distribution of the next token for sequence generation; ranking score for ranking or recommendation, at least one of the following.

[0008] As a preferred embodiment of the first aspect, the curvature prediction network includes a feature transformation subnetwork and a weight output layer; the weight output layer outputs an unnormalized curvature score vector u=[u - ,u0,u + The curvature score vector is then subjected to Softmax normalization, temperature Softmax normalization, or LogSumExp normalization to obtain the weight vector w=[w - ,w0,w + The curvature prediction network can be generated at the following granularities: by token, by attention, or by network layer.

[0009] As a preferred embodiment of the first aspect, the projection process and the attention calculation process employ a branchless soft stabilization mechanism, which includes at least: The curvature parameter is constrained to a preset range using a continuously differentiable function. Using z / sqrt(∥z∥) during normalization or mapping 2 The continuously differentiable normalized form of z / (∥z∥+ε) suppresses singularities; Continuous saturation scaling is used for norm, radius, or mapping inputs instead of threshold clipping. The continuous saturation scaling includes continuous monotonically saturated forms of u←u·tanh(γ·u) / (u+ε) or u←u / (1+γ·u) to ensure that the inputs of exponential / logarithmic mappings are in the effective domain and to suppress overflow. In attention scoring, the masked item is implemented using an additive bias (e.g., adding a bias to the masked position). The additive bias of M (where M is a preset large positive constant) is used to avoid conditional branching based on threshold comparison; wherein, none of the branchless soft stability mechanisms include conditional branching based on threshold comparison for numerical stability protection.

[0010] As a preferred embodiment of the first aspect, the attention scoring includes: calculating the geometric product of the query multivector Q and the key multivector K to obtain the multivector result G=geo. prod (Q,K); Extract the scalar part of the multi-vector result. G 0 is used as the similarity term; the bi-vector part of the multi-vector result is extracted. G 2. And obtain its two-vector part coefficient vector b=coeff( G 2) The amplitude of b is calculated using L2 amplitude based on the coefficient vector, and the amplitude is calculated as ∥b∥=sqrt(Σ k bk 2 The magnitude is equivalent to the Frobenius norm; the magnitude is transformed by a continuous nonlinear function to obtain an outer product scalar term, wherein the continuous nonlinear function is at least one of the Softplus function and the log1p function; the magnitude is further transformed according to the bi-vector dimension d. b Perform scale normalization (e.g., using ∥b∥ / sqrt(d)). b ), and / or apply SoftNorm to b before taking the amplitude value; linearly combine the similarity term with the outer product scalar term to obtain the attention score.

[0011] As a preferred embodiment of the first aspect, the mapping projected onto the spherical or hyperbolic subspace comprises a constant curvature exponential mapping Exp0 that matches the curvature parameter κ. κ (·), where Exp0 κ (·) represents the exponential mapping of the κ-stereographic constant curvature model or its numerical equivalent; where the vector to be mapped is v, r = ∥v∥2, and ε is a positive small constant: when κ < 0, Exp0 κ (v)=tanh(√|κ|·r)·v / (√|κ|·r+ε); when κ>0, Exp0 κ (v) = tan(√κ·r)·v / (√κ·r+ε); when κ=0, Exp0 0 (v) = v; and the mapping at least includes continuous normalization to avoid singularities, the continuous normalization including z / sqrt(∥z∥) 2 +ε), and may further include continuous saturation scaling to continuously compress the input of the exponential map to the effective domain; in one embodiment, when κ>0, let =(π / 2 δ) / √κ·tanh(√κ·r / (π / 2 δ)) and with Substitute r into tan(·), where δ is a small positive constant; the above mapping process does not include conditional branches based on threshold comparison.

[0012] As a preferred embodiment of the first aspect, when constructing the query multivector and the key multivector, a rotor transformation is applied to the query multivector and the key multivector: Q ’ =R·Q·R rev K ’ =R·K·R rev ; Where R is the rotor multivector, R revR represents the multivector obtained by applying a reversion operation to the rotor multivector; when the rotor multivector satisfies the normalization constraint, R rev R, as the inverse of the rotor multivector -1 .

[0013] As a preferred embodiment of the first aspect, the attention outputs of each subspace are weighted and fused according to the hybrid curvature distribution parameters after being logarithmically mapped back to the tangent space, and the fusion context representation satisfies: H=w - ·Log o κ- (H - )+w0·H0+w + ·Log o κ+ (H + ); Among them, κ - <0, κ0=0, κ + >0, and Log o κ- (·), Log o κ+ (·) represent the logarithmic mappings corresponding to the hyperbolic and spherical subspaces, respectively; when κ0 = 0, the output H0 of the Euclidean subspace remains unchanged in the tangent space and Log o 0 (·)=Id(·); In one embodiment, the reference point is the origin o=0, at which point Log o κ (·) can be implemented as a constant curvature logarithmic mapping Log0 κ (·)(κ≠0), the Log0 κ (·) represents the logarithmic mapping of the κ-stereographic constant curvature model or its numerical equivalent implementation; where the coordinates of the subspace to be mapped are x, ρ = ∥x∥2, and ε is a positive small constant: when κ < 0, Log0 κ (x)=artanh(√|κ|·ρ)·x / (√|κ|·ρ+ε); when κ>0, Log0 κ (x) = arctan(√κ·ρ)·x / (√κ·ρ+ε); and the logarithmic mapping includes at least one of continuous saturation scaling and continuous normalization to ensure that the input is within the valid domain and to suppress singularities; in one embodiment, for the case κ<0, let =(1 δ) / √|κ|·tanh(√|κ|·ρ / (1 δ)) and with Substitute ρ into artanh(·), where δ is a small positive constant; the logarithmic mapping does not contain conditional branches based on threshold comparisons.

[0014] As a preferred embodiment of the first aspect, the different order components of the multi-vector are stored in a multi-channel real tensor in a contiguous memory block manner, and the scalar component and bi-vector part of the geometric product are computed in a vectorized parallel manner in one or more fusion operators, including at least attention scoring and weighted aggregation operations, wherein the fusion operators run on a GPU, NPU, or TPU; wherein the fusion operators cache intermediate multi-vector results in on-chip shared memory and / or registers and perform intra-block reduction to reduce global memory round trips; the fusion operators perform geometric product calculation, grade projection, and bi-vector magnitude calculation in the same kernel, and further perform continuous nonlinear transformation of the outer product scalar term, linear combination of scalar components and outer product scalar term, temperature scaling, and mask additive bias in the same kernel to generate attention scores; and at least one of the intermediate multi-vector results and intermediate attention scores is not written back to global memory or external memory.

[0015] As a preferred embodiment of the first aspect, when training the model corresponding to the method, a regularization term is further added in addition to the task loss, the regularization term including: Entropy regularization term of mixed curvature distribution parameters (e.g., L) ent =-Σ i w i ·log(w i +ε)) and at least one of the following: sparse regularization terms (e.g., L1 regularization of w or KL distance to the target distribution); and For the order regularization term of the magnitude of the two vector components, the order regularization term includes: L1 norm penalty and L2 norm penalty for the coefficient vector b of the two vector components (e.g., ∥b∥1, ∥b∥2). 2 At least one of the following, and continuous constraints on the energy proportions of different grade components (e.g., applying a squared penalty to ∥b∥2 / (∥v∥2+ε) for deviations from the target proportion); wherein the regularization terms are all continuously differentiable and do not contain conditional branches based on threshold comparisons.

[0016] A second aspect of the present invention provides a multi-vector semantic processing apparatus capable of automatically executing the method flow disclosed in the first aspect and its preferred embodiments. The apparatus includes: A multi-vector embedding module is used to receive an input sequence and map the input sequence to a multi-vector feature representation in the Clifford algebra space, wherein the multi-vector includes a scalar part, a vector part, and a bi-vector part. The curvature prediction module is used to generate mixed curvature distribution parameters through a curvature prediction network. The mixed curvature distribution parameters include positive curvature weights, negative curvature weights, and zero curvature weights. The positive curvature weights correspond to a spherical subspace, the negative curvature weights correspond to a hyperbolic subspace, and the zero curvature weights correspond to a Euclidean subspace. The product manifold mapping module is used to project the multi-vector feature representation onto a spherical subspace, a hyperbolic subspace, and a Euclidean subspace, respectively, to obtain a triplet subspace feature representation, which is then used as a product manifold representation for subsequent attention calculation. The product manifold representation is a parallel triplet representation. The geometric algebraic attention module is used to construct query multivectors, key multivectors, and value multivectors in spherical subspaces, hyperbolic subspaces, and Euclidean subspaces, respectively, and to calculate attention scores and subspace attention outputs using geometric integration operators. The tangent space fusion module is used to map the attention outputs of the spherical subspace, hyperbolic subspace, and Euclidean subspace back to the tangent space through logarithmic mapping, and then weight and fuse them according to the hybrid curvature distribution parameters to obtain the fusion context representation. The output module is used to input the fusion context representation into the task header to output the semantic processing result.

[0017] A third aspect of the present invention provides an electronic device comprising a processor and a memory, the memory storing a computer program, which, when executed on the processor, causes the electronic device to perform the Clifford multi-vector semantic processing method based on a dynamic hybrid curvature product manifold as described in the first aspect.

[0018] In a fourth aspect, the present invention provides a computer-readable storage medium having a computer program stored thereon, which, when executed on a processor, implements the Clifford multi-vector semantic processing method based on a dynamically hybrid curvature product manifold as described in the first aspect.

[0019] Compared with the prior art, the present invention has at least the following beneficial effects: (1) By dynamically mixing curvature distribution and cumulative manifold mapping, hierarchical structure (negative curvature), periodic / cyclic structure (positive curvature) and parallel / tiled structure (zero curvature) can be represented in the same framework for modeling and reasoning of complex semantic structures.

[0020] (2) Curvature, modulus and mapping adopt continuous function constraints and smooth normalization to avoid introducing threshold condition branches, keep the computation graph continuous, and help improve training stability and parallel execution efficiency, which is convenient for vectorized parallel computing on hardware such as GPU / NPU / TPU.

[0021] (3) Geometric algebraic attention introduces an outer product scalar term (obtained by scalarizing the magnitude of the coefficient vector of the two vector part) in addition to scalar similarity. This term can characterize higher-order geometric information such as directed area / subspace relations spanned by semantic representations, so as to support the modeling of asymmetric semantic interactions and inclusion / implication relations.

[0022] (4) Compared with attention schemes that only use mixed curvature or product manifold but still rely mainly on scalar similarity, this invention introduces a scoring structure of “geometric product scalar part + outer product scalar term (scalarized by the magnitude of the coefficients of the two vector part and transformed by a continuous function)” in each subspace; compared with schemes that only use geometric algebra multivectors but calculate in a single geometric space or do not perform three-topology probabilistic route fusion, this invention performs probabilistic route and tangent space fusion on the spherical / hyperbolic / Euclidean three subspaces by mixing curvature distribution parameters, and combines branchless soft stability and fusion operators to achieve parallel hardware-friendly collaborative technical effects. Attached Figure Description

[0023] Figure 1 This is a schematic diagram of the overall process of the method of the present invention.

[0024] Figure 2 This is a schematic diagram of the device module structure of the present invention.

[0025] Figure 3 This is a schematic diagram of the dynamic mixing curvature distribution and the cumulative manifold mapping.

[0026] Figure 4 A diagram illustrating attention scoring in geometric algebra.

[0027] Figure 5 This is a schematic diagram of a branchless soft stability mechanism.

[0028] Figure 6 This is a visualization diagram of the task and curvature weights for an example implementation. Detailed Implementation

[0029] In the following description, numerous specific details are set forth in order to provide a more thorough understanding of the invention. However, it will be apparent to those skilled in the art that the invention can be practiced without one or more of these details. In other instances, certain technical features well-known in the art have not been described in order to avoid obscuring the invention.

[0030] Before describing the embodiments, the proprietary terms, symbols and definitions used below will be explained.

[0031] The input sequence has a length of L and a batch size of B.

[0032] The multi-vector channel dimension is denoted as C. mvIn one example, the multivector consists of three channels: a scalar part α (1 channel), a vector part v (d...). v (one channel), two-vector unit b (d) b (one channel), namely C mv =1+d v +d b For example, when the dimension of the vector part is d v When, d can be taken b =d v (d v -1) / 2 corresponds to the number of bi-vector bases.

[0033] The mixed curvature distribution is w=[w - ,w0,w + ], where w - The corresponding hyperbolic subspace, w0 corresponds to the Euclidean subspace, w + Corresponding to a spherical subspace, and satisfying w i ≥0 and Σw i =1.

[0034] Terminology for “Exponential / Logarithmic Mapping (Exp / Log)”: In this specification, Exp0... κ (·) , Log0 κ (·) indicates the use of "tangent space" to implement The mapping operator for the "constant curvature subspace". In one embodiment, it is implemented in the closed form of the given κ-stereographic constant curvature model; other known closed or numerically equivalent implementations of constant curvature models (e.g., Poincaré or Lorentz) may also be used as alternatives without changing the principle of the invention.

[0035] Geometric Product (Clifford / Geometric Product) and Grade Projection: Calculating the geometric product G=geo over multiple vectors Q and K. prod (Q,K) yields a multi-vector result, and the scalar part is extracted using grade projection. G 0 and the two-vector part G 2 (and its coefficient vector coeff( G 2) As a component of attention scoring; when the components involved in the calculation are pure vectors q and k, their geometric product can be decomposed into the sum of the inner and outer product terms, i.e., geo prod (q,k)=q·k+q∧k.

[0036] The coefficient extraction operator `coeff(·)` expands a multi-vector component of a given grade into a coefficient vector representation on a set of bases. For example, for a two-vector component... G 2. Denote coeff( G 2) = [b1, b2, ..., b db ], where b k These are the coefficients corresponding to the two-vector basis, which can be used for amplitude / magnitude calculation and template correlation calculation.

[0037] Product Manifold Representation: In this invention, "product manifold representation" refers to the parallel-obtained triplet subspace representation (X). - ,X0,X + ), which belong to the hyperbolic subspace M respectively. - Euclidean subspace M0 and spherical subspace M + Therefore, the whole belongs to the Cartesian product M. - ×M0×M + The subsequent fusion is the result of a logarithmic mapping in a unified tangent space. - , 0, + Weighted combination according to weight w.

[0038] Bivector-to-Scalar quantization refers to the process of converting a bivector (exterior product / bivector part) into a scalar scoring term that can participate in Softmax. In one embodiment, the scalar is obtained by calculating the magnitude / modulus of the bivector part coefficient vector; in another embodiment, the scalar is obtained by calculating the correlation between the bivector part coefficient vector and the coefficient vector of a predefined or learnable template. Both embodiments are continuously differentiable and do not contain threshold conditional branches.

[0039] Amplitude / Magnitude (implementation definition): Let the multi-vector M be carried in the form of real-valued channel coefficients as M=[α,v,b], where α is the scalar part, v is the coefficient vector of the vector part, and b is the coefficient vector of the two-vector part. Then we can take ∥M∥=sqrt(α) 2 +∥v∥2 2 +∥b∥2 2 () as the module length at the implementation level.

[0040] Two-vector magnitude (Frobenius / L2): Let two vectors B lie on a basis {e pq} expands to B=Σ_{p <q}b pq e pq Let b = coeff(B) = (bpq If ), then we can take ∥b∥=sqrt(Σ). p<q b pq 2 As an amplitude, it is equivalent to the Frobenius norm of the two-vector part B, so as to be implemented efficiently in the hardware kernel.

[0041] Rotor: Refers to a multivector used to represent rotation / conjugate transformation. In one embodiment, the rotor is an even-grade multivector; in another embodiment, the rotor R satisfies the normalization constraint R· =1 (where) (for the inversion of R), thus it can be expressed in the conjugate form R·X· Perform rotations or equivalent linear transformations on a multivector X.

[0042] Rotor reversal (reversion / reverse): The reversal of rotor R is denoted as R. rev (or denoted as) ), used to construct R·X·R rev ;R rev This represents the multivector obtained by applying a reversion operation to R; when R satisfies the normalization constraint, R... rev It can be used as the inverse of R. -1 .

[0043] Clifford conjugation: In another embodiment, Clifford conjugation can be applied to the rotor R to obtain R. conj and R·X·R conj Construct conjugate transformations.

[0044] The present invention proposes a Clifford multi-vector semantic processing method based on a dynamic hybrid curvature product manifold, the specific implementation steps of which are as follows: S1, Multi-vector Embedding In one embodiment, the input is a discrete token sequence x 1:L First, the Euclidean feature e is obtained through embedding lookup tables or a previous-level network. t ∈R d Then e t Mapping to a multi-vector representation X t : Scalar part: α t =f s (e t ); Vector part: v t =f v (e t ); Two vector parts: b t =f b (e t ).

[0045] Where f s f v f b It can be a linear transformation, a two-layer perceptron, or a low-rank decomposition network. In implementation, multiple vectors can be carried by a multi-channel real tensor: X∈R B×L×Cmv .

[0046] In another embodiment, the multi-vector consists only of a scalar part and a vector part (omitting the two-vector part), and the two-vector part is generated on demand within the attention module to reduce overhead.

[0047] S2, Curvature Distribution Generation In one embodiment, the curvature prediction module predicts a mixed curvature distribution for each token: w t =Softmax(W c ·g(v t )+b c ), Where g(·) is the normalization or gating function, W c With b c The curvature parameter can be a learnable parameter. It can be fixed as a constant or learned. In one embodiment, let c... - c + These represent the magnitudes of negative curvature and positive curvature, respectively (c). - >0, c + >0), and let κ - =-c - κ + =c + κ0=0, where: c - =c min +(c max -c min )·sigmoid(θ - ), c + =c min +(c max -c min )·sigmoid(θ + ).

[0048] In another embodiment, w outputs by attention point or by layer to strike a trade-off between accuracy and computation.

[0049] S3, Manifold Mapping The multi-vector features are projected into three subspaces respectively: X- =Π κ- (X), X0=Π κ0 (X), X + =Π κ+ (X). Where κ - <0, κ0=0, κ + >0.

[0050] In one possible embodiment, the Π κ- With Π κ+ All of these can be achieved by combining the Exp mapping from "tangent space to subspace" with continuous soft constraints (e.g., Π). κ (X)=Exp0 κ (f(X)), where κ∈{κ - ,κ +}, f(·) is a learnable mapping), and can be returned to the unified tangent space for fusion through the corresponding Log mapping.

[0051] In one embodiment, for the coefficient tensor X=[α,v,b] carrying multiple vectors, Exp / Log is applied only to the coefficient vector v (or its learnable linear combination) of the vector part to satisfy the constant curvature constraint of the subspace, while the scalar part α and the bivector part b maintain Euclidean coordinates or adopt the same form of continuous mapping. In another embodiment, the vector part and the bivector part are simultaneously subjected to the same form of Exp / Log mapping or equivalent continuous normalization, thereby enhancing the constraint strength on the geometry of the subspace while maintaining implementation simplicity. All the above implementations are continuously differentiable and do not contain threshold conditional branches.

[0052] Among them Π κ0 It can be an identity mapping; Π κ- With Π κ+ It can consist of learnable mappings and continuous normalized / conformal mappings. To ensure numerical stability and continuous differentiability, the mapping process uses one of the following soft constraint forms: SoftNorm: softnorm(z)=z / sqrt(∥z∥ 2 +ε); log1p / softplus suppresses excessive amplitude: z←z·tanh(γ·∥z∥) / (∥z∥+ε).

[0053] None of the above operations include a threshold if-else branch.

[0054] To improve feasibility, an executable mapping implementation is given below using the κ-stereographic constant curvature model as an example. Let the curvature of the subspace be κ (hyperbolic: κ < 0; Euclidean: κ = 0; spherical: κ > 0), and the reference point be the origin o = 0. For any Euclidean tangent space vector v (which can be the vector part of a multi-vector or the Euclidean coefficient vector output by the network), define r = ∥v∥2, and let a = a(κ) = sqrt(|κ|) (a = 0 when κ = 0).

[0055] (1) Exp mapping (from tangent space to subspace coordinates): When |κ| < 0 (hyperbolic), let a = sqrt(||κ||), then Exp0 < 0. κ (v)=tanh(a·r)·v / (a·r+ε).

[0056] When |κ|>0 (spherical), let a=sqrt(κ), then Exp0 κ (v)=tan(a·r)·v / (a·r+ε).

[0057] When |κ|=0 (Euclidean), Exp0 0 (v)=v.

[0058] (2) Log mapping (cutting back from subspace coordinates to space): For any subspace coordinate x, let ρ = ∥x∥2.

[0059] When |κ| < 0 (hyperbolic), let a = sqrt(|κ|), then Log0 < 0. κ (x)=artanh(a·ρ)·x / (a·ρ+ε).

[0060] When |κ|>0 (spherical), let a=sqrt(κ), then Log0 κ (x)=arctan(a·ρ)·x / (a·ρ+ε).

[0061] When |κ|=0 (Euclidean), Log0 0 (x)=x.

[0062] To avoid ambiguity in terminology, "exponential mapping / logarithmic mapping (Exp / Log)" in this specification refers to the mapping operator used to implement the above coordinates / constraints, which can be implemented in the given closed form or its equivalent substitution.

[0063] (3) Numerical domain constraints: To ensure that the inputs of artanh(·) in the hyperbolic subspace and tan(·) / arctan(·) in the spherical subspace are within their valid domains, continuous saturation scaling of r or ρ can be used instead of thresholding. For example: When κ < 0, we can let =(1-δ) / a·tanh(a·ρ / (1-δ)), so there is always a· <1-δ<1, ensuring artanh(a· It is defined; When κ>0, we can let =(π / 2-δ) / a·tanh(a·r / (π / 2-δ)), so there is always a· <π / 2-δ, to avoid the singularity of tan(a·r) near π / 2. Here, δ is a small positive constant (e.g., 1e^(-δ / π / 2)). -3 The above scaling is in a continuously differentiable form, independent of if-else or threshold branches. When using the exponential / logarithmic map for computation, r is replaced with... (Exp mapping for κ>0) and / or replacing ρ with Substitute (Log mapping for κ<0) into the above formula to ensure that the function input is within the valid domain.

[0064] S4, Geometric Algebraic Attention In each subspace i∈{-,0,+}, construct query, key, and value multivectors: Q i =f Q (X i ), K i =f K (X i V i =f V (X i ).

[0065] In one embodiment, f Q f K f V It is a multi-vector linear layer (parameterized by grade block).

[0066] (1) Optional rotor transformation To characterize the relationship between direction and rotation, a rotor transformation can be applied to Q / K: Q i ’ =R i ·Q i ·R i,rev K i ’ =R i ·K i ·R i,rev .

[0067] Where R i For learnable rotor multivectors, R i,revR represents the multivector obtained by applying a reversion operation to the rotor multivector; when the rotor multivector satisfies the normalization constraint, R i,rev It can be taken as the inverse R of the rotor multi-vector. i -1 .

[0068] In another embodiment, R can be obtained by applying Clifford conjugate operation to the rotor multivectors instead. i,conj To construct R i ·X·R i,conj The conjugate form is used as an equivalent substitution.

[0069] (2) Geometric affinity (attention score) The attention scoring of this invention includes at least scalar components and an outer product scalar term; when using a rotor transformation, Q is used in subsequent attention scoring. i ’ K i ’ Replace Q i K i For example, for any token pair (t,j), let G... i (t,j)=geo prod (Q i (t),K i (j)) represents the geometric product of the t-th query multivector and the j-th key multivector in the i-th subspace. Then, the definition is: S i (t,j)= G i (t,j) 0+λ·φ(∥b i (t,j)∥ / sqrt(d b )), Where b i (t,j)=coeff( G i (t,j) 2) is the coefficient vector of the two-vector part, d_b is the dimension of the coefficient vector of the two-vector part, ∥b i (t,j)∥ represents the L2 / Frobenius magnitude of the coefficient vector; by adjusting ∥b i (t,j)∥According to sqrt(d b Scale normalization can reduce the scale difference between the outer product term and the scalar term under different dimensions / grades to improve trainability; in one embodiment, λ can be learned by attention heads or network layers.

[0070] in · 0 indicates the extraction of the scalar part of the geometric product. · 2 represents the two-vector part of the extracted geometric product, φ is a continuous nonlinear function (e.g., Softplus or log1p), and λ is a learnable or preset weight; when using a rotor transformation, Q is used in the above formula. i ’ (t), K i ’ (j) Substitute Q i (t), K i (j) Calculate G i (t,j).

[0071] Amplitude calculation: To enable the outer product scalar term to participate in Softmax, this invention uses the bivector part B= G_i(t,j) The coefficient vector of 2 is denoted as b = coeff(B), and the magnitude / modulus of the coefficient vector b is calculated to obtain a scalar; where |b| is equivalent to the Frobenius / L2 magnitude of the two-vector part B. In one embodiment, the magnitude is calculated according to the L2 / Frobenius norm of the coefficient vector of the two-vector part: if B = Σ p<q b pq e pq And b = coeff(B) = (b pq If ), then ∥b∥=sqrt(Σ p<q b pq 2 This is equivalent to the Frobenius norm. For example: - In the two-dimensional example, the bivector basis contains only e 12 If B=b 12 e 12 Then |b| = |b 12 |;- In the three-dimensional example, if B=b 12 e 12 +b 23 e 23 +b 31 e 31 Then ||b|| = sqrt(b 12 2 +b 23 2 +b 31 2 The above magnitude calculation only involves the addition, subtraction, multiplication, and square root of coefficients, which is easy to vectorize and implement in the fusion operator, and is continuously differentiable.

[0072] In another embodiment, the outer product scalar term can also be obtained by calculating the correlation between the coefficient vector of the two-vector part and the coefficient vector of the learnable template, and is also continuously differentiable and does not contain a threshold conditional branch.

[0073] (3) Attention weights and aggregation A i (t,:)=Softmax(S i (t,:) / τ), H i (t)=Σ j A i (t,j)·V i (j), where τ is the temperature coefficient.

[0074] S5, Space Blending To unify and merge the outputs of different curvature subspaces, the outputs of each subspace are mapped back to the tangent space: - =Log o κ- (H - ), 0=H0, + =Log o κ+ (H + ), When κ0=0 0 = H0.

[0075] Where o is the reference point (which can be the origin or a learnable reference point), κ - <0, κ0=0, κ + >0; Log o κ- (·) and Log o κ+ (·) corresponds one-to-one with the hyperbolic and spherical subspaces, respectively, and the Euclidean subspace corresponds to the identity mapping Id(·). In one possible embodiment, Log0 above can be used. κ (·) (or its equivalent constant curvature model's Log mapping) is used as the Log o κ The implementation of (·) ensures that the input is in the valid domain and does not introduce threshold condition branches through at least one of continuous saturation scaling and SoftNorm.

[0076] In one embodiment, the reference point is taken as the origin o=0, at which point Log o κ (·) is equivalent to Log0 κ (·). In another embodiment, the reference point is parameterized as the tangent space vector o. tan and with i =Log0 κi (H i ) o tanThe reference point is translated in a continuous form; or equivalently, with H i =Exp0 κi ( i +o tan ) as the inverse mapping, where κ i ∈{κ - ,κ0,κ + The translation of the reference point is a continuously differentiable implementation and does not include conditional branches based on threshold comparisons.

[0077] Then, they are blended according to curvature weights: H=w - · - +w0· 0+w + · + .

[0078] In one embodiment, curvature entropy regularization can be added to prevent w from degenerating into a single subspace: L ent =-Σ i w i ·log(w i +ε).

[0079] S6, Output The fused representation H is input to the task header, which outputs the semantic processing result. For example, the semantic processing result includes: RAG Search / Re-ranking: Search relevance score, supportability / implication score, or re-ranking score; RAG generation: as a conditional context representation of the generative model or a score used to assign document fusion weights; Knowledge graph reasoning / closed-loop consistency: reasoning results, closed-loop consistency judgment labels or confidence levels; Classification / Generation: The probability distribution of the category or the next token; Sorting / Recommendation: Sorting score or recommendation score.

[0080] To ensure continuous differentiability of the computation and adaptability to parallel hardware, the curvature, modulus, mapping, and mask in the embodiment all adopt continuous function constraints and do not include threshold-based conditional branches.

[0081] The term "branchless" in this specification means: not employing threshold if-else branches for numerical stability protection (e.g., switching piecewise formulas for radius / modulus); and not excluding comparison operations included within general operators such as sorting, Top-K, max, or reduction for selection / reduction purposes. Examples include: Curvature constraint: c=c min+(c max -c min The curvature parameter can be constrained to a feasible range using sigmoid(θ) or by using Softplus / Logistic. SoftNorm: softnorm(z)=z / sqrt(∥z∥ 2 +ε); Continuous saturation scaling (alternative to threshold clipping): For u=∥z∥ or radius r, ρ, use u←u·tanh(γ·u) / (u+ε) (or its equivalent continuous monotonic saturation function) to suppress excessive amplitude values ​​and ensure that the Exp / Log input is in the valid domain; External product amplitude suppression: First extract the coefficient vector b=coeff(B) of the two vector part B, and apply a continuous nonlinear function φ(·) to ∥b∥, such as φ(∥b∥)=log(1+∥b∥) or Softplus, to suppress numerical instability caused by excessive amplitude. Mask continuation: Using an additive bias mask in attention scoring (e.g., adding a bias to the masked location). The additive bias of M (where M is a preset large positive constant) avoids if-else or threshold comparisons.

[0082] The above mechanism ensures that both forward and backward propagation remain continuously differentiable and reduces the hardware efficiency loss caused by branch divergence, thereby reducing warp divergence in SIMD / warp execution on GPUs and reducing branch prediction overhead on general-purpose processors.

[0083] The derivative of Softplus is sigmoid(x), whose range is within (0,1), which helps to suppress gradient explosion; SoftNormz / sqrt(∥z∥ 2 +ε) is differentiable everywhere when ε>0, thereby avoiding the singular gradient when ∥z∥→0, so that the forward and backward propagation of this invention have branchless stability.

[0084] At the implementation level, multi-vectors are carried by multi-channel real tensors and stored in blocks according to grade. The geometric integration operator is implemented using predefined channel combination rules, and a grade projection operator is provided. geo prod (a,b)→c (multi-vector); grade0(c) → scalar part; grade2(c) → bi-vector part.

[0085] In one embodiment, to improve parallel efficiency on GPU / NPU / TPU and reduce memory round trips, this invention fuses multiple operators in attention computation into one or more fused operators (Fused Operator / Fused Kernel), and enables them to perform at least two of the following operations without including threshold condition branches (example: "attention scoring + weighted aggregation", optionally further fused with Softmax normalization): (1) Block loading and memory layout: Multi-vectors are stored in blocks according to grade as contiguous memory areas (e.g., scalar part, vector part, and dual vector part are contiguous channel blocks respectively), and block loading is performed according to attention head / sequence position; on GPU, on-chip shared cache / register cache can be used, and on NPU / TPU, it can be mapped to on-chip SRAM or systolic array input buffer to reduce memory access overhead.

[0086] (2) Geometric product and grade projection fusion: Perform geometric product (geo) on the query multivector and the key multivector in the same fusion operator. prod (Q,K), and then perform the projection of grade0 and grade2 to obtain the scalar component s0= geo prod (Q,K) 0 and the two-vector coefficient vector b=coeff( geo prod (Q,K) 2) Avoid writing back to external memory for intermediate multi-vector c, thereby avoiding writing the intermediate multi-vector results back to global memory / external memory; intermediate results are preferably kept in registers or on-chip cache.

[0087] (3) Fusion of amplitude calculation and nonlinear transformation: In the same fusion operator, the amplitude ∥b∥(L2 / Frobenius) based on the coefficient vector b is calculated, and a continuous nonlinear function φ(·) (e.g., Softplus / log1p) is applied to it to obtain the outer product scalar term s2=φ(∥b∥ / sqrt(d b )).

[0088] (4) Linear combination and scoring matrix generation: The score Score = s0 + λ·s2 is calculated in the same fusion operator, and temperature scaling (e.g., dividing by τ) and mask additive bias are superimposed in the same fusion operator to form the attention score for Softmax; the mask is preferably implemented in the form of additive bias (e.g., adding...). An additive bias of M (where M is a preset large positive constant) is used to avoid threshold branching; in one implementation, the intermediate score can directly enter the Softmax / aggregation calculation without being written back to global memory.

[0089] (5) Softmax and weighted aggregation fusion (optional): In one implementation, the row-wise normalization of the score (Softmax) is combined with the weighted aggregation Σ of the value multivectors. j A(t,j)·V(j) is fused into the same kernel, and the maximum value / exponential sum is calculated through block reduction. The weighted accumulation of V is completed in the same kernel, thereby avoiding the external memory write-back of attention weights or intermediate scoring matrices. In another implementation, Softmax is used as an independent fusion operator, while “scoring (including temperature scaling and mask bias) + weighted aggregation” is used as a fusion operator. This can still significantly improve throughput by reducing intermediate tensor write-back.

[0090] (6) Mixed precision and vectorization: Different precisions can be used for different grade components (e.g., scalar part INT8 / FP16, dual vector part FP16 / BF16), and vectorization instructions or tensor core / pulse array are used to perform multiplication, addition and reduction operations; under the premise of ensuring numerical stability, the fusion operator can use continuous functions such as SoftNorm, log1p / Softplus to suppress overflow and gradient instability.

[0091] Through the above-mentioned fusion operator design, this invention can reduce the number of memory accesses, reduce kernel startup overhead and branch divergence without introducing if-else branches, thereby improving inference throughput and training stability on both edge and cloud hardware.

[0092] In another embodiment, it can be extended to include trivector components to express higher-order relations.

[0093] The following provides several other implementation methods and application scenarios to further illustrate the possible forms and alternative solutions of the present invention. Those skilled in the art should understand that the following implementation methods can be combined with, substituted for, or implemented in parallel with one or more steps S1–S6 described above, and should not be considered as limiting the scope of protection of the present invention.

[0094] Alternative Implementation Method A: Pure Clifford Multi-Vector Attention Independent of Dynamic Curvature In an alternative embodiment, steps S2 (curvature distribution generation) and S3 (integral manifold mapping) can be omitted, and multivectors can be constructed directly on Euclidean features or single subspace features and geometric algebraic attention can be performed.

[0095] Alternative Implementation Method B: Search Enhancement Generation (RAG) Process In another alternative embodiment, the method of the present invention can be used in a retrieval enhancement generation (RAG) system. The system may include a retrieval module, a reordering module, and a generation module.

[0096] In one implementation, this RAG implementation can be used in conjunction with the aforementioned steps S2–S5 to enhance the joint characterization of “relevance + supportability / implication” by utilizing hybrid curvature routing and tangent space fusion; in another implementation, geometric algebraic attention and outer product scalar term scoring can also be performed only in the Euclidean subspace (κ=0) as an equivalent replacement and to reduce deployment complexity.

[0097] (1) Query and document multi-vectorization: The query and candidate document fragments are mapped to Clifford multi-vector representations respectively, and optionally, mixed curvature distribution parameters are generated according to the context and mapped to the cumulative flow shape subspace.

[0098] (2) Entailment-aware Scoring: A retrieval relevance score is calculated for the query multivector and candidate document multivectors. This score includes a scalar similarity term as well as an outer product scalar term, used to characterize the inclusion / entailment relationship between the query and candidate documents. For example, the score can be written as: Score= G 0+λ·φ(∥coeff( G 2)∥ / sqrt(d b )), Where G=geo prod (Q,K).

[0099] in G 0 represents the similarity term obtained from the scalar part of the geometric product, coeff( G 2) is the coefficient vector of the two-vector part of the geometric product, ∥·∥ is the L2 / Frobenius magnitude calculation based on the coefficient vector, and φ is a continuous nonlinear function; in one implementation, φ can be chosen to be monotonically increasing (e.g., Softplus or log1p); in another implementation, to enhance the separability of the "supported / included" discrimination, it can be taken as... (u)= φ(u) is an equivalent transformation that is monotonically decreasing, where u = ||coeff( G 2)∥.

[0100] (3) Reordering and selection: The candidate set is reordered according to the score and the Top-K results are selected. In one implementation, the selection process is completed by sorting / Top-K operation without relying on threshold condition branches. In another implementation, soft selection based on Softmax weights can be used to assign continuous weights to each candidate to replace or supplement Top-K, thereby maintaining continuous differentiability in end-to-end training or joint optimization scenarios.

[0101] (4) Generate fusion: The Top-K results are used as context input to the generation module to output the generation results; in one implementation, the Score is also used to assign fusion weights to each document fragment to control the reference bias during generation.

[0102] Through the above process, the outer product scalar term can provide additional criteria for "supportability / implication", reducing false detections of "relevant but unsupported" results caused by relying solely on cosine similarity.

[0103] Alternative Implementation Method C: Knowledge Graph Reasoning and Closed-Loop Consistency In another alternative embodiment, the method of the present invention can be used for knowledge graph reasoning. The feature representations of entities and relations can be multi-vectorized and mapped to spherical / hyperbolic / Euclidean subspaces. Geometric algebraic attention is performed within each subspace to obtain edge / path representations, which are then fused and constrained by a logarithmic mapping back to the tangent space.

[0104] In one implementation, positive curvature components can be used to enhance the consistent representation of the closed-loop structure, and the inference path or closed-loop constraint can be mapped back to the tangent space to compute consistent constraint terms (e.g., closure residuals) to suppress contradictions in closed-loop inference. In another implementation, only negative curvature components can be used to emphasize hierarchical paths, or only Euclidean components can be used as equivalent replacements.

[0105] Alternative Implementation Method D: Edge / End-to-Cloud Collaborative Deployment In another alternative embodiment, the method of the present invention can be deployed on an edge intelligent terminal (e.g., smartphone, edge computing box, in-vehicle computing platform, wearable device, IoT gateway, etc.). The terminal may include a processor, on-chip memory, and a neural network accelerator (NPU / TPU or tensor computing accelerator containing a systolic array). Different order components of the multi-vector are stored in contiguous memory blocks, and the scalar component and bivector part of the geometric product are computed in a vectorized parallel manner in the fusion operator; a hierarchical precision strategy (e.g., scalar part INT8, bivector part FP16 / BF16) may be adopted.

[0106] Alternative Implementation Method E: Model / Parameter Carrier and Delivery Form In another alternative embodiment, the present invention can be delivered as a model file / parameter set / inference engine.

[0107] Alternative Implementation Method F: Training Method and Regularization Terms In another alternative embodiment, the present invention also provides a method for training a semantic processing model, which may incorporate curvature entropy regularization and bi-vector order regularization in addition to the task loss to improve training stability and prevent curvature routing and multi-vector component degradation.

[0108] (1) Curvature distribution regularization (example): For mixed curvature weights w=[w - ,w0,w + Apply at least one of entropy regularization, sparsity regularization, or target distribution regularization, for example: Entropy regularization: L ent =-Σ i w i ·log(w i +ε) is used to prevent w from collapsing into a single subspace too early; Sparse / Target Distribution Regularization: L sp =∥w∥1 or L KL =KL(w∥u), where u is the preset target distribution (e.g., uniform distribution or task prior distribution).

[0109] (2) Grade regularization (example): For a two-vector part coefficient vector b = coeff( · 2) Apply at least one of amplitude constraints and energy ratio constraints, for example: Amplitude penalty: L b2 =∥b∥2 2 L b1 = at least one of ∥b∥1; Energy proportion constraint: Let the vector coefficient be v, and define ρ = ∥b∥2 / (∥v∥2+ε), then L can be used. ratio =(ρ-ρ0) 2 This is to encourage the energy ratios of different grades to approach the target ρ0.

[0110] All of the above regularization terms can be aggregated into a total regularization loss by token, attention head, or network layer, and implemented in a continuously differentiable form, without relying on conditional branches based on threshold comparison.

[0111] Example 1: Periodic semantic tasks (spherical component advantage) Choose a task dataset D1 with a clear periodic structure (this can be replaced with any sequence prediction or classification dataset with a periodic structure, such as a periodic label task like weekday / month / season). During training / inference, the positive curvature weights w output by the curvature prediction module... +With a significant increase in the periodic token, the model outperforms Euclidean or negative curvature-only schemes in terms of periodicity prediction metrics.

[0112] Example 2: Hierarchical Semantic Task (Hyperbolic Component Advantage) Choose the task dataset D2 with a clear hierarchical structure (it can be replaced with any hierarchical classification, hyperspeech-hypospeech inference, or ontology hierarchical inference dataset). The curvature prediction module outputs negative curvature weights w. - Increasing the number of tokens related to hierarchical inference can improve the model's performance in hierarchical consistency and long-chain inference metrics.

[0113] Example 3: Reasoning with a knowledge graph containing rings (multiple topological advantages, including specific examples) Choose the knowledge graph reasoning dataset D3 containing closed-loop relationships (it can be replaced with any knowledge graph dataset containing loop / closed-loop consistency judgments). For example: The knowledge graph contains the relationship assertions "A is the father of B", "B is the father of C", and "C is the father of A". Under normal logic, these three edges constitute an illogical closed loop (error loop).

[0114] In this invention, the curvature prediction module can increase the negative curvature weight w in a context related to "family relationship / generation". - To emphasize hierarchy, and at the same time increase the positive curvature weight w when detecting closed-loop consistency. + This emphasizes angular / closure constraints. Furthermore, the extra-product magnitude term in geometric algebraic attention is used to characterize the changes in the subspace spanned by entity representations in the path; when closure leads to accumulated contradictions in semantic directions, the extra-product magnitude term and the closure residual exhibit anomalies, thereby improving the discriminability of "error loops".

[0115] Specifically, the subspace output of the candidate path A→B→C→A can be mapped back to the shear space, and a closed-loop consistency constraint term can be constructed (example): L cycle =∥Log o κ+ (h A→B,+ )+Log o κ+ (h B→C,+ )+Log o κ+ (h C→A,+ )∥.

[0116] Where h u→v,+ This represents the edge / path representation obtained by geometric algebraic attention aggregation within a spherical subspace. In another embodiment, the edge / path representations of each subspace can be first mapped back to the tangent space and then fused according to the hybrid curvature weights to obtain the representation. u→v =w - ·Logo κ- (h u→v,-})+w0·h u→v,0 +w + ·Log o κ+ (h u→v,+ ), then calculate L cycle =∥ A→B + B→C + C→A ∥.

[0117] Where h u→v This represents the edge / path representation obtained by geometric algebraic attention aggregation within the flow-form subspace. When this loop is an "error loop," the angle accumulation and closure residual in the spherical subspace will increase significantly, thereby triggering consistency judgments (e.g., outputting "loop contradiction / low confidence") or suppressing the weights of related edges during the training or inference phase. The above process is continuously differentiable and does not depend on hard thresholding branches.

[0118] Example 4: Hardware Efficiency and Stability (Branchless Advantage) Compared with the numerical protection scheme with threshold conditional branches and the soft stabilization scheme of the present invention, under the same hardware platform, the present invention avoids pipeline bubbles caused by branch divergence during parallel execution, the training loss curve can be more stable, and the inference throughput can be improved or the end-to-end latency can be reduced; wherein, the implementation method of the "fusion operator / fusion kernel" can be found in the fusion operator embodiment of this specification.

[0119] In one evaluation example, under the same model size, input length, and precision settings, the "traditional non-Euclidean implementation with threshold branching (including branching)" and the "branchless implementation based on SoftNorm / Softplus of this invention (which can be combined with fusion operators)" are compared, and metrics such as kernel execution time, end-to-end latency, and training convergence curves are recorded. It can be observed that the branchless implementation is more conducive to parallel execution and training stability; its kernel execution time and end-to-end latency show a decreasing trend, and the convergence curve is more stable. Specific changes are related to sequence length, hidden dimension, fusion degree, and hardware platform.

[0120] Example 5: Inclusion / Implication Matching in RAG (Outer Product Advantage) In the Retrieval Enhancement Generation (RAG) scenario, the relationship between a query and a document includes not only similarity (relevance) but also inclusion relationships such as "whether the query is supported / implied by the document." To characterize this type of relationship, this invention introduces an outer product scalar term (obtained by scalarizing the magnitude of the coefficient vector of the two vector part) into the retrieval relevance score, and linearly combines it with the scalar similarity term to construct a supportability score, which is used to reorder or filter candidate documents that are "relevant but not supported."

[0121] For example, for any candidate document fragment d, the following can be defined: SupportScore(d) = Σ t∈Query ω t ·max j∈d { G t,j 0+λ·φ(∥coeff( G t,j 2)∥ / sqrt(d b ))}.

[0122] Where G{t,j}=geo prod (Q t ,K j ).

[0123] Continuous implementation of end-to-end joint optimization (optional): In a joint training embodiment, to maintain end-to-end continuous differentiability, the max_{j∈d}{·} can be replaced with the smooth maximum operator smoothmax. β ({s j})=(1 / β)·log(Σ j exp(β·s j (i.e., in the form of LogSumExp, where β is a temperature parameter and β>0), and / or replace the Top-K candidate selections with soft selection based on Softmax weights to assign continuous weights to candidate documents; in the inference deployment embodiment, precise max and sorting / Top-K operations can be used. None of the above continuous replacements introduce conditional branches based on threshold comparisons.

[0124] Where ω t To query the token weight, · 0 and · 2 represents the scalar and bivector parts of the geometric product, respectively; ∥·∥ represents the L2 / Frobenius magnitude calculation based on the coefficient vector; and φ is a continuous nonlinear function. Candidate documents are sorted by SupportScore(d) and the Top-K are selected before being input into the generation module. Alternatively, document fusion can be performed using the continuous weights assigned by the soft selection method. This can improve the "supportability" related indicators and reduce the risk of phantom citations without introducing threshold branches.

[0125] Example 6: Exemplary Comparison Results To illustrate the technical effects, a set of exemplary comparison results are provided in Table 1. The comparison objects include: Euclidean Transformer (Baseline), Hyperbolic-only Transformer, Clifford-only geometric algebraic attention, and our invention.

[0126] For example, D1 can be a sequence prediction or classification task dataset with a periodic structure, D2 can be an inference or classification task dataset with a hierarchical structure, and D3 can be a knowledge graph inference dataset containing loops / closed-loop consistency determination. Evaluation can be conducted using metrics such as accuracy, F1 score, consistency determination index, number of iterations required to reach a preset training threshold, and edge inference throughput / latency trends. The report should clearly specify the hardware platform, batch size, sequence length, and fusion strategy used in the reproducibility experiment.

[0127] Table 1. Exemplary Comparison Results

[0128] The above qualitative results illustrate that: dynamic hybrid curvature enables the model to adapt its routing on different structures; the magnitude term of the outer product coefficient enhances the expression of asymmetric / inclusion relationships; and the branchless and fusion operators are beneficial to parallel efficiency and training stability.

[0129] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and not to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A Clifford multi-vector semantic processing method based on dynamic hybrid curvature product manifold, characterized in that, Includes the following steps: S1. Receive the input sequence and map the input sequence into a multi-vector feature representation in the Clifford algebra space, wherein the multi-vector includes a scalar part, a vector part and a bi-vector part; S2. Based on the multi-vector feature representation or its contextual features. A curvature prediction network generates hybrid curvature distribution parameters, which include positive curvature weights, negative curvature weights, and zero curvature weights; wherein, the positive curvature weights correspond to a spherical subspace, the negative curvature weights correspond to a hyperbolic subspace, and the zero curvature weights correspond to a Euclidean subspace. S3. Based on the hybrid curvature distribution parameters, the multi-vector feature representation is projected onto the spherical subspace, hyperbolic subspace, and Euclidean subspace respectively to obtain the triplet subspace feature representation, which is used as the product manifold representation for subsequent attention calculation. The product manifold representation is a parallel triplet representation. S4. Construct query multi-vectors, key multi-vectors, and value multi-vectors in the spherical subspace, hyperbolic subspace, and Euclidean subspace, respectively, and calculate the attention score and subspace attention output using the geometric integration operator. S5. The attention outputs of the spherical subspace, hyperbolic subspace, and Euclidean subspace are respectively tangent back to the logarithmic map space and then weighted and fused according to the hybrid curvature distribution parameters to obtain the fusion context representation. S6. Input the fusion context representation into the task header to output the semantic processing result.

2. The Clifford multi-vector semantic processing method based on dynamic hybrid curvature product manifold according to claim 1, characterized in that, Step S2 further includes: processing the positive curvature weight, negative curvature weight, and zero curvature weight through Softmax normalization, temperature Softmax normalization, or LogSumExp normalization to obtain a weight vector, wherein the weight vector satisfies w i ≥0 and Σw i =1.

3. The Clifford multi-vector semantic processing method based on dynamic hybrid curvature product manifold according to claim 1, characterized in that, The curvature prediction network includes a feature transformation subnetwork and a weight output layer; the weight output layer outputs an unnormalized curvature score vector u=[u - ,u0,u + The curvature score vector is then subjected to Softmax normalization, temperature Softmax normalization, or LogSumExp normalization to obtain the weight vector w=[w - ,w0,w + The curvature prediction network can be generated at the following granularities: by token, by attention, or by network layer.

4. The Clifford multi-vector semantic processing method based on dynamic hybrid curvature product manifold according to claim 1, characterized in that, In step S3, the projection process includes applying an exponential mapping / logarithmic mapping constraint that matches the curvature parameter to the vector part of the multi-vector or its learnable linear combination, wherein the scalar part and the bi-vector part maintain Euclidean coordinates or adopt a continuous mapping of the same shape.

5. The Clifford multi-vector semantic processing method based on dynamic hybrid curvature product manifold according to claim 1, characterized in that, In step S4, the attention scoring includes the scalar component of the geometric product result and the outer product scalar term obtained by performing magnitude calculation based on the coefficient vector of the two-vector part of the geometric product result and transforming it through a continuous nonlinear function. The scalar components and the outer product scalar terms are linearly combined to form an attention scoring matrix, which is then used for Softmax normalization to generate attention weights.

6. The Clifford multi-vector semantic processing method based on dynamic hybrid curvature product manifold according to claim 1, characterized in that, In step S6, the semantic processing result includes: Used to retrieve search scores, reordering scores, or generation conditional contexts from the enhanced RAG system; The reasoning results, closed-loop consistency labels, or confidence scores used for knowledge graph reasoning or closed-loop consistency determination; Category probabilities used for text classification; The probability distribution for the next token used in sequence generation; Ranking score used for sorting or recommendation.

7. The Clifford multi-vector semantic processing method based on dynamic hybrid curvature product manifold according to claim 1, characterized in that, The different order components of the multi-vector are stored in a multi-channel real tensor in a contiguous memory block manner, and the scalar component and the bi-vector part of the geometric product are computed in a vectorized parallel manner in one or more fusion operators, including at least attention scoring and weighted aggregation operations, wherein the fusion operators run on a GPU, NPU or TPU. The fusion operator caches intermediate multi-vector results in on-chip shared memory and / or registers and performs intra-block reduction to reduce global memory round trips. The fusion operator performs geometric product calculation, grade projection, and bi-vector magnitude calculation in the same kernel, and further performs continuous nonlinear transformation of the outer product scalar term, linear combination of scalar components and outer product scalar term, temperature scaling, and mask additive bias in the same kernel to generate attention scores. At least one of the intermediate multi-vector results and intermediate attention scores is not written back to global memory or external memory.

8. A multi-vector semantic processing device, characterized in that, include: A multi-vector embedding module is used to receive an input sequence and map the input sequence to a multi-vector feature representation in the Clifford algebra space, wherein the multi-vector includes a scalar part, a vector part, and a bi-vector part. The curvature prediction module is used to generate mixed curvature distribution parameters through a curvature prediction network. The mixed curvature distribution parameters include positive curvature weights, negative curvature weights, and zero curvature weights. The positive curvature weights correspond to a spherical subspace, the negative curvature weights correspond to a hyperbolic subspace, and the zero curvature weights correspond to a Euclidean subspace. The product manifold mapping module is used to project the multi-vector feature representation onto a spherical subspace, a hyperbolic subspace, and a Euclidean subspace, respectively, to obtain a triplet subspace feature representation, which is then used as a product manifold representation for subsequent attention calculation. The product manifold representation is a parallel triplet representation. The geometric algebraic attention module is used to construct query multivectors, key multivectors, and value multivectors in spherical subspaces, hyperbolic subspaces, and Euclidean subspaces, respectively, and to calculate attention scores and subspace attention outputs using geometric integration operators. The tangent space fusion module is used to map the attention outputs of the spherical subspace, hyperbolic subspace, and Euclidean subspace back to the tangent space through logarithmic mapping, and then weight and fuse them according to the hybrid curvature distribution parameters to obtain the fusion context representation. The output module is used to input the fusion context representation into the task header to output the semantic processing result.

9. An electronic device comprising a processor and a memory, wherein the memory stores a computer program, characterized in that, When the computer program is executed on the processor, it causes the electronic device to perform the Clifford multi-vector semantic processing method based on dynamic hybrid curvature product manifold as described in any one of claims 1 to 7.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed on a processor, it implements the Clifford multi-vector semantic processing method based on dynamic hybrid curvature product manifold as described in any one of claims 1 to 7.

Images