Unified visual word segmentation model construction method and system based on topological orthogonal decoupling

Abstract

The application discloses a unified visual word segmentation model construction method and system based on topological orthogonal decoupling, and belongs to the cross field of artificial intelligence and computer vision, and comprises the following steps: building the overall framework of the network, the network is entirely composed of neural network layers, and comprises a visual encoder, a collaborative connector module and a pixel decoder; setting the network structure of the collaborative block, setting the orthogonal isolation mechanism of the topological flow and the semantic flow in the collaborative connector module, and corresponding structure topological alignment and active semantic anchoring algorithms; training the network in three stages, obtaining a unified visual word segmentation model, which can better solve the manifold dislocation conflict between high-fidelity pixel reconstruction and low-frequency semantic abstraction, simultaneously consider extracting high-frequency structural details and low-frequency abstract concepts, design a collaborative connector model suitable for cross-modal tasks, not only maintain high image generation fidelity, but also completely eliminate destructive gradient interference in multi-task learning.

Description

Technical Field

[0001] This invention belongs to the field of interdisciplinary technology of artificial intelligence and computer vision, specifically involving a visual representation learning technology based on multimodal large models and image generation, and particularly involving a unified visual word segmentation model construction method and system based on topological orthogonal decoupling. Background Technology

[0002] With the success of large language models (LLMs), unified modeling in multimodal domains has become a research hotspot, aiming to bridge the architectural gap between image understanding and image generation. Traditionally, these two tasks rely on completely different paradigms: understanding models typically align semantic encoders like CLIPs with large language models, while generative models employ diffuse latent variables or discrete VQ-VAE lexical units. To establish a unified interface, recent research has focused on developing unified visual segmenters, attempting to integrate generation and understanding capabilities within a single codebook or latent space. However, existing unified visual segmentation methods are caught in a zero-sum game between reconstruction and understanding. The root cause lies in manifold misalignment: there is an inherent conflict between optimizing high-fidelity pixel reconstruction (pursuing spatial equivariance and focusing on high-frequency details) and optimizing semantic abstraction (pursuing conceptual invariance and filtering spatial details). Simple joint optimization leads to destructive interference of gradients—pixel gradients attempt to unfold the manifold to preserve details, while semantic gradients attempt to fold the manifold to extract concepts. This results in severe polarization in the model's internal perception, either becoming a fragmented high-frequency view that ignores semantics or a blurry, abstract view that loses details. Existing multi-task weighted summation methods cannot fully capture and decouple these two objectives, and often sacrifice semantic alignment capabilities for generation fidelity, failing to achieve true mutual promotion. Summary of the Invention

[0003] To address the problems existing in the prior art, this invention provides a method for constructing a unified visual tokenizer based on topological orthogonal decoupling. By utilizing the physical isolation of the parameter subspace within the Transformer architecture, the MUSE framework is proposed. Through active semantic anchoring and structural topological alignment, destructive gradient interference is transformed into mutually reinforcing synergistic effects, achieving end-to-end high-fidelity reconstruction and high-precision understanding.

[0004] To achieve the above objectives, in a first aspect, the present invention provides a method for constructing a unified visual word segmentation model based on topological orthogonal decoupling, comprising the following steps: A unified visual word segmentation model is constructed using an overall network framework, which includes a visual encoder, a cooperative connector module, and a pixel decoder connected in sequence. The visual encoder is used to extract high-dimensional visual features from the input image, the cooperative connector module is used to perform topological orthogonal decoupling processing on the geometric structure features and semantic content features in the high-dimensional visual features, and the pixel decoder is used to decode the processed visual words and restore them to the pixel space to complete image reconstruction. The internal network structure of the collaborative connector module is constructed, which consists of several sequentially stacked collaborative blocks. Each collaborative block sets up a dual-path orthogonal isolation mechanism for topological flow and semantic flow, where the topological flow is used to encode the geometric adjacency relationship between tokens, determined by the query matrix. Bond matrix Parameterization is performed and updated based on structural objectives; semantic flow is used to update features based on the topological graph constructed by topological flow, consisting of independent value matrices. The data is updated based on semantic target optimization. Based on the network framework, a three-stage training strategy is adopted to jointly train the network, resulting in a unified visual word segmentation model based on topological orthogonal decoupling. The three-stage training strategy includes a topology warm-up stage, a semantic injection stage, and a collaborative fine-tuning stage.

[0005] Furthermore, the specific computation process of the topological flow and semantic flow of the collaborative block includes: The adjacency matrix A of the geometric relationships between encoded tokens is calculated using topological flow.

[0006] For topology flow query vectors, For topological flow key vectors, For vector dimensions; Features are updated based on the established topological graph A using semantic flow, employing independent value projection matrices. Calculate the semantic flow vector value The aggregated output is .

[0007] Furthermore, the topology warm-up stage includes: freezing the network parameters of the visual encoder, introducing a pre-trained self-supervised visual model as the structural teacher model, aligning the spatial dimensions through a 4D interpolation function, calculating the KL divergence between the student network attention map and the teacher model attention map as the structural topology alignment loss, and completing the warm-up training of the topology flow parameters. The collaborative fine-tuning phase specifically involves: unfreezing all backbone parameters of the network framework, introducing pixel-level reconstruction loss, including generative adversarial loss and MSE loss, and performing end-to-end joint optimization of the network framework; during the optimization process, topology flow parameters are updated only by structural loss, and semantic flow parameters are updated only by semantic loss. The semantic injection phase specifically involves maintaining the query matrix corresponding to the topology flow. Bond matrix With the parameters fixed, a projector is introduced to map the pooled visual lexical units to the visual-language alignment space. The active semantic anchoring loss is calculated using a noise contrast estimation algorithm. The text embedding corresponding to the input image is used as a positive sample for constraint, thus completing the optimization training of the semantic flow parameters.

[0008] Furthermore, structural topology alignment loss L topo :

[0009] in, L The number of layers in the collaborative block. H For the number of attention heads, Here is the KL divergence calculation function. (·) is the 4D interpolation function. For the teacher model l Layer h Attention map of individual attention heads, For student network l Layer h Attention map of individual attention heads.

[0010] Furthermore, the active semantic anchoring loss The calculation is as follows: Calculating active semantic anchoring loss using noise contrast estimation Embedded with corresponding text t As a positive sample constraint:

[0011] in, Let g(·) be the noise contrast estimation loss function, g(·) be the projector mapping function, and z be the pooled visual words. t This is the text embedding vector corresponding to the input image.

[0012] Furthermore, the visual encoder adopts a visual Transformer architecture, and the pixel decoder adopts a high compression ratio autoencoder network structure. The number of stacked layers of the collaborative blocks within the collaborative connector module is 6, and the collaborative blocks are built based on the Transformer architecture.

[0013] Secondly, the present invention provides a unified visual word segmentation model construction system based on topological orthogonal decoupling, including a network structure construction module, a cooperative connector configuration module, and a model training module; The network structure construction module is used to build the overall network framework of the unified visual word segmentation model. The network framework includes a visual encoder, a collaborative connector module, and a pixel decoder connected in sequence. The visual encoder is used to extract high-dimensional visual features of the input image. The collaborative connector module is used to perform topological orthogonal decoupling processing on the geometric structure features and semantic content features in the high-dimensional visual features. The pixel decoder is used to decode the processed visual words and restore them to the pixel space to complete image reconstruction. A collaborative connector configuration module is used to construct the internal network structure of the collaborative connector module, which consists of several sequentially stacked collaborative blocks. Each collaborative block sets up a dual-path orthogonal isolation mechanism for topological flow and semantic flow, wherein the topological flow is used to encode the geometric adjacency relationship between tokens, determined by the query matrix. Bond matrix Parameterization is performed and updated based on structural objectives; semantic flow is used to update feature content based on the topological graph constructed by topological flow, consisting of independent value matrices. Parameterization, optimized and updated by semantic objectives, achieves orthogonal isolation between structural and semantic objectives in the parameter space; The model training module is used to jointly train the network based on the network framework using a three-stage training strategy, and finally obtain a unified visual word segmentation model based on topological orthogonal decoupling; the three-stage training strategy includes a topology warm-up stage, a semantic injection stage, and a collaborative fine-tuning stage.

[0014] Thirdly, this invention provides a unified visual word segmentation method based on topological orthogonal decoupling, comprising the following steps: acquiring the original input image of high-dimensional pixels, inputting it into an end-to-end unified visual word segmentation model, and outputting the continuous or discrete visual words corresponding to the image; the end-to-end unified visual word segmentation model is obtained by the above-mentioned unified visual word segmentation model construction method based on topological orthogonal decoupling; and using the visual words as a unified interface to input into a multimodal large language model for image semantic understanding tasks, or to input into a diffusion model / autoregressive model for high-fidelity image generation and editing tasks.

[0015] Fourthly, the present invention also provides a computer device, including a processor and a memory, wherein the memory is used to store a computer executable program, the processor reads part or all of the computer executable program from the memory and executes it, and when the processor executes part or all of the computer executable program, it can realize the above-mentioned unified visual word segmentation model construction method based on topological orthogonal decoupling or the unified visual word segmentation method based on topological orthogonal decoupling.

[0016] Simultaneously, a computer-readable storage medium can be provided, in which a computer program is stored. When the computer program is executed by a processor, it can implement the above-mentioned unified visual word segmentation model construction method based on topological orthogonal decoupling or the unified visual word segmentation method based on topological orthogonal decoupling.

[0017] Compared with the prior art, the present invention has at least the following beneficial effects: This invention reveals and solves for the first time the problem of reconstruction and understanding conflict caused by manifold misalignment in visual word segmentation tasks. By physically isolating semantic and structural update paths, the model significantly improves zero-shot semantic understanding accuracy without sacrificing generation fidelity (gFID reaches 3.08). The proposed Synergistic Block utilizes the inherent functional specialization of Transformer layers to precisely route structural gradients to the attention topology. Semantic gradient routing to feature values ​​( This completely eliminates destructive gradient interference in traditional multi-task learning; Thanks to the geometric foundation established by structural topology alignment, this invention not only avoids semantic drift due to pixel-level reconstruction, but also actively refines semantic perception by utilizing the preserved structural information, making the linear detection accuracy of the word segmentation model even surpass that of the expert-level teacher backbone network it uses. Attached Figure Description

[0018] Figure 1 This is a schematic diagram showing the geometric principle and gradient conflict elimination for solving the manifold misalignment problem in this invention.

[0019] Figure 2 This is a schematic diagram of the overall architecture and three-stage training of the unified visual word segmentation model MUSE constructed in this invention.

[0020] Figure 3 This is a schematic diagram illustrating gradient conflict analysis for the popular misalignment problem studied in this invention.

[0021] Figure 4 This is a schematic diagram illustrating the image generation and editing effects after combining the unified word segmentation model MUSE constructed in this invention with LLM. Detailed Implementation

[0022] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of the present invention. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0023] The problem this invention aims to solve is: how to construct a unified visual tokenizer that can retain high-frequency pixel-level structural information for high-fidelity image generation, while extracting low-frequency abstract concepts for visual understanding in multimodal language models, whether continuous or discrete visual tokens. This invention, from an information theory perspective, guides network architecture design by decomposing the mutual information objective into two orthogonal parts: geometric structure and semantic content.

[0024] Let the input observation space be X, the semantic space be Y, and the latent variable be Z. To maximize the mutual information I(Z;X,Y), this invention introduces a latent variable S representing the "structural state," and decomposes the optimization objective according to the chain rule: I(Z;X,Y)≈I(Z;S)+I(Z;Y|S)+I(Z;X|S,Y). The above formula shows that the optimization of the latent variable should follow the priority: first, align the geometric basis I(Z;S); second, inject semantic I(Z;Y|S) into the given topology; and finally, pass the remaining high-frequency texture features I(Z;X|S,Y) to the decoder for processing. Based on the above theory, this invention provides a method for constructing a unified visual word segmentation model based on topological orthogonal decoupling, comprising the following steps: constructing a unified visual word segmentation network model, wherein the network model includes a visual encoder, a cooperative connector module, and a pixel decoder; the visual encoder is used to extract visual features of the input image, the cooperative connector module is used to orthogonally decouple and fuse high-level semantic features and structural geometric features, and the pixel decoder is used to reconstruct the image based on visual lexical units; setting the network structure inside the cooperative connector module, which adopts a cooperative attention mechanism to map semantic features to a value subspace and structural geometric features to an attention topological subspace, forming isolation on the computational graph; setting a structural topological alignment algorithm and an active semantic anchoring algorithm to construct a multi-task loss function; using a training dataset, optimizing the weight connection parameters inside the network model in stages to solve the manifold misalignment problem between reconstruction and understanding tasks, and obtaining a trained unified visual word segmentation model.

[0025] refer to Figure 1 and Figure 2 This invention employs an advanced visual Transformer architecture (such as InternViT) as the visual encoder to initially extract high-dimensional features from the image. Simultaneously, a high-compression-ratio autoencoder is used as the decoding network to decode the processed visual terms back into pixel space.

[0026] The connector module consists of multiple stacked Synergistic Blocks (six layers in this embodiment). While conventional Transformer layers couple semantic content and geometry in a single hybrid process, the Synergistic Blocks of this invention process these elements separately through branched functional paths. Let the input to the... The characteristics of the layer are The specific calculation process includes: First, the Topology Stream calculates the adjacency matrix A, which represents the geometric relationships between encoded terms. This Topology Stream is composed solely of the query matrix. Bond matrix Parameterized and optimized only by structural objectives: The second step, the Semantic Stream, updates the feature content based on the established adjacency matrix A. This is achieved using independent value projection matrices. Calculate semantic flow vector values Attention aggregation output features are During this process, the semantic stream applies a stop-gradient operation to the gradient reconstruction of the underlying pixels, achieving the semantic objective within the parameter space (only updating the gradient). ) and structural objectives (updated only) , The orthogonal isolation of the topology blocks avoids destructive interference from gradients. The cooperative blocks utilize topology parameters... and With semantic parameters The separation of functions and the isolation and control of gradient propagation paths enable structural modeling and semantic modeling to be independent and complementary in the optimization process, thus forming an orthogonal decoupled design.

[0027] refer to Figure 3 This invention further analyzes and explains the gradient conflict problem in the traditional unified visual word segmentation model. Figure 3 (a) in the baseline method shows the gradient conflict between structural and semantic objectives during joint training. That is, the structural gradient guided by pixel reconstruction and the semantic gradient guided by semantic alignment restrain each other in the shared parameter space, resulting in unstable model training and difficulty in simultaneously achieving image generation fidelity and semantic understanding accuracy. Figure 3 (b) illustrates the collaborative optimization process of the method of the present invention under the three-stage training strategy. By dividing the training into three stages in sequence—Structure, Semantic and Refinement—the learning of structural topology, the injection of semantic information and the joint refinement of overall performance are carried out in sequence, thereby reducing the direct interference between different optimization objectives. Figure 3 (c) further illustrates the inherent tilt angle difference between structural gradient and semantic gradient in traditional methods, indicating that there is a natural inconsistency in their optimization directions. Figure 3 (d) in the figure illustrates that the present invention limits the topology flow parameters to , The corresponding topological space restricts the semantic flow parameters to The corresponding value space achieves gradient conflict resolution at the subspace level, enabling structural modeling and semantic modeling to be stably optimized along their respective parameter subspaces.

[0028] The training process of this invention is divided into three stages: Stage 1: Topology Warmup - Freeze the visual encoder and introduce a self-supervised visual model as the structural teacher (the self-supervised visual model can be DINOv3). Utilize the 4D interpolation function Ψ( Align the spatial dimensions and calculate the KL divergence between the student attention map AS and the teacher attention map AT as the structural topological alignment loss. L topo : Phase Two: Semantic Injection – Preserving Learned Topological Parameters ), introducing the projector g( The pooled visual lexicon z is mapped to the visual-linguistic space. Here, the pooled visual lexicon z is obtained by global average pooling of the lexicon sequence output by the visual encoder or co-connector, and is used to represent the global visual semantics of the entire image. The active semantic anchoring loss L is calculated using Noise Contrast Estimation (NCE). anchor The corresponding text embedding t is used as the positive sample constraint: This constraint firmly anchors the feature values ​​to the semantic manifold, preventing subsequent "semantic drift." Stage 3: Synergistic Tuning unfreezes the network backbone and introduces pixel-level reconstruction losses (such as generative adversarial loss and MSE loss) for end-to-end optimization. Thanks to the orthogonal decoupling design of the synergistic blocks, high-frequency pixel gradients and low-frequency semantic gradients no longer conflict, achieving true mutual promotion.

[0029] Furthermore, combined Figure 3 (b) It can be seen that in the topology warm-up stage, the model prioritizes establishing a stable geometric structure representation; in the semantic injection stage, the model injects text-guided abstract semantic information while maintaining the existing topology; in the collaborative fine-tuning stage, the model further refines the overall network by combining pixel-level reconstruction targets. This training sequence is consistent with... Figure 3 The optimization path shown corresponds to the transformation of the originally competing structural and semantic goals into a collaborative optimization relationship that is interconnected and mutually reinforcing.

[0030] Based on the model obtained by the above method, this invention also provides a unified visual word segmentation application method based on topological orthogonal decoupling. Using the unified visual word segmentation model trained above, the method includes: inputting the original input image into the unified visual word segmentation model; the visual encoder and the co-connector module outputting decoupled unified visual lexical units; using the unified visual lexical units as a standard interface, on the one hand, inputting them into a multimodal large language model to perform image semantic understanding tasks, and on the other hand, inputting them into an image generation model to perform high-fidelity image generation and editing tasks. (Reference) Figure 4 , Figure 4 The diagram illustrates the application effect of combining the unified visual word segmentation model MUSE constructed in this invention with a multimodal large language model. By using the unified visual lexical units output by this invention as a cross-modal interface, it can simultaneously support two types of tasks: image understanding and image generation / editing. In image understanding scenarios, the unified visual lexical units can provide the multimodal large language model with visual representations that combine spatial structural information and abstract semantic information, thereby improving the model's ability to understand image content, target relationships, and scene semantics. In image generation and editing scenarios, the unified visual lexical units can retain relatively complete pixel structural details and geometric layout information, enabling images generated by diffusion models or autoregressive models to have higher fidelity in terms of local texture, overall composition, and semantic consistency. Figure 4 The results show that the unified visual word segmentation model proposed in this invention can serve as a general visual interface connecting the understanding model and the generation model, meeting the needs of various downstream tasks for high-quality visual word representation.

[0031] To verify the effectiveness of the proposed topological orthogonal decoupling model (MUSE) in this invention, it was rigorously compared with existing mainstream benchmark models on a unified dataset (BLIP3-o, ImageNet-1K, etc.). The evaluation metrics covered the generation task (rFID, the lower the better), the understanding task (Zero-Shot accuracy, Linear Probe accuracy, the higher the better), and the structure awareness capability (mIoU, the higher the better).

[0032] Table 1. Performance Comparison of Different Visual Word Segmentation Models

[0033] As shown in Table 1, traditional unified models often involve compromises. For example, the VTP model sacrifices semantic alignment (zero-shot accuracy drops to 71.2%) for generation fidelity; while UniLIP's generation quality (rFID 0.79) and structure awareness (mIoU 15.4) are mediocre. Our invention's model (MUSE) achieves optimal generation fidelity (rFID 0.62) through topological orthogonal decoupling. Furthermore, due to the active refinement of visual perception by structure alignment, its linear detection accuracy reaches 85.2%, even surpassing the InternViT teacher model (82.5%), which serves as its backbone network and is specifically designed for understanding tasks. Figure 3 and Figure 4 To further explain, this invention does not sacrifice the performance of one task to gain the performance of another. Instead, it resolves gradient conflicts in subspaces and constructs a unified visual lexical interface, enabling the model to exhibit good overall performance in multiple tasks such as image understanding, image generation, and image editing.

[0034] This invention provides a unified visual word segmentation model construction system based on topological orthogonal decoupling. This system is a virtual software functional module architecture, with each module corresponding one-to-one with the steps of the aforementioned model construction method. Specifically, it includes: a network structure construction module, a network parameter pre-setting module, and a training module. The network structure construction module is used to construct the overall framework of the network model, which consists of neural network layers, including a visual encoder, a co-connector module, and a pixel decoder. The visual encoder is used to extract high-dimensional features, the co-connector module is used to orthogonally decouple semantic features and structural geometric features, and the pixel decoder is used for image reconstruction. The network parameter pre-setting module is used to set the internal network structure of the co-connector. Internally, it uses a synergistic block to separate the topological flow and semantic flow through physical or parameter space isolation; it also sets a structural constraint algorithm, using KL divergence of self-supervised features for topological alignment; and a semantic constraint algorithm, using noise contrast estimation for dynamic semantic anchoring. The training module uses an image dataset and follows a three-stage strategy of topological warm-up, semantic injection, and co-connector fine-tuning to optimize the internal weight connection parameters of the network model, ultimately obtaining a unified visual word segmentation model that solves the manifold misalignment problem.

[0035] This invention also provides a computer device, including a bus, a processor, a memory, and a communication interface. The processor can be connected to the memory and the communication interface via the bus. The memory is used to store computer-executable programs (instructions). The processor reads part or all of the computer-executable programs from the memory and executes them. When the processor executes these programs, it can implement the steps of the unified visual word segmentation model construction method based on topological orthogonal decoupling described in the above embodiments of this invention, or implement the application steps of the unified visual word segmentation method based on topological orthogonal decoupling. This device can be a server, workstation, cloud computing node, or a local computer equipped with a high-performance GPU (such as a graphics processing unit).

[0036] The present invention also provides a computer-readable storage medium (such as ROM, RAM, magnetic disk, USB flash drive, SSD solid-state drive, etc.) storing a computer program thereon. When the computer program is executed by a processor, it can implement the steps of the model construction method or application method in the above embodiments of the present invention.

[0037] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the protection scope of the present invention.

Claims

1. A method for constructing a unified visual word segmentation model based on topological orthogonal decoupling, characterized in that, Includes the following steps: A unified visual word segmentation model is constructed using an overall network framework, which includes a visual encoder, a collaborative connector module, and a pixel decoder connected in sequence. The visual encoder is used to extract high-dimensional visual features from the input image, the collaborative connector module is used to perform topological orthogonal decoupling processing on the geometric structure features and semantic content features in the high-dimensional visual features, and the pixel decoder is used to decode the processed visual words and restore them to the pixel space to complete image reconstruction. The internal network structure of the collaborative connector module is constructed, which consists of several sequentially stacked collaborative blocks. Each collaborative block sets up a dual-path orthogonal isolation mechanism for topological flow and semantic flow, where the topological flow is used to encode the geometric adjacency relationship between tokens, determined by the query matrix. Bond matrix Parameterization is performed and updated based on structural objectives; semantic flow is used to update features based on the topological graph constructed by topological flow, consisting of independent value matrices. The data is updated based on semantic target optimization. Based on the network framework, a three-stage training strategy is adopted to jointly train the network, resulting in a unified visual word segmentation model based on topological orthogonal decoupling. The three-stage training strategy includes a topology warm-up stage, a semantic injection stage, and a collaborative fine-tuning stage.

2. The unified visual word segmentation model construction method based on topological orthogonal decoupling according to claim 1, characterized in that, The specific computation process of the topological flow and semantic flow of the collaborative block includes: The adjacency matrix A of the geometric relationships between encoded tokens is calculated using topological flow. For topology flow query vectors, For topological flow key vectors, For vector dimensions; Features are updated based on the established topological graph A using semantic flow, employing independent value projection matrices. Calculate semantic flow vector values The aggregated output is .

3. The unified visual word segmentation model construction method based on topological orthogonal decoupling according to claim 1, characterized in that, The topology warm-up stage includes: freezing the network parameters of the visual encoder, introducing a pre-trained self-supervised visual model as the structural teacher model, aligning the spatial dimensions through a 4D interpolation function, calculating the KL divergence between the student network attention map and the teacher model attention map as the structural topology alignment loss, and completing the warm-up training of the topology flow parameters. The collaborative fine-tuning phase specifically involves: unfreezing all backbone parameters of the network framework, introducing pixel-level reconstruction loss, including generative adversarial loss and MSE loss, and performing end-to-end joint optimization of the network framework; during the optimization process, topology flow parameters are updated only by structural loss, and semantic flow parameters are updated only by semantic loss. The semantic injection phase specifically involves maintaining the query matrix corresponding to the topology flow. Bond matrix With the parameters fixed, a projector is introduced to map the pooled visual lexical units to the visual-language alignment space. The active semantic anchoring loss is calculated using a noise contrast estimation algorithm. The text embedding corresponding to the input image is used as a positive sample for constraint, thus completing the optimization training of the semantic flow parameters.

4. The unified visual word segmentation model construction method based on topological orthogonal decoupling according to claim 3, characterized in that, Structural topology alignment loss L topo : in, L The number of layers in the collaborative block. H For the number of attention heads, Here is the KL divergence calculation function. (·) is the 4D interpolation function. For the teacher model l Layer h Attention map of individual attention heads, For student network l Layer h Attention map of individual attention heads.

5. The unified visual word segmentation model construction method based on topological orthogonal decoupling according to claim 3, characterized in that, Active semantic anchoring loss The calculation is as follows: Calculating active semantic anchoring loss using noise contrast estimation Embedded with corresponding text t As a positive sample constraint: in, Let g(·) be the noise contrast estimation loss function, g(·) be the projector mapping function, and z be the pooled visual words. t This is the text embedding vector corresponding to the input image.

6. The unified visual word segmentation model construction method based on topological orthogonal decoupling according to claim 1, characterized in that, The visual encoder adopts a visual Transformer architecture, and the pixel decoder adopts a high compression ratio autoencoder network structure. The number of stacked layers of the collaborative blocks within the collaborative connector module is 6, and the collaborative blocks are built based on the Transformer architecture.

7. A unified visual word segmentation model construction system based on topological orthogonal decoupling, characterized in that, It includes a network structure building module, a co-connector configuration module, and a model training module; The network structure construction module is used to build the overall network framework of the unified visual word segmentation model. The network framework includes a visual encoder, a collaborative connector module, and a pixel decoder connected in sequence. The visual encoder is used to extract high-dimensional visual features of the input image. The collaborative connector module is used to perform topological orthogonal decoupling processing on the geometric structure features and semantic content features in the high-dimensional visual features. The pixel decoder is used to decode the processed visual words and restore them to the pixel space to complete image reconstruction. A collaborative connector configuration module is used to construct the internal network structure of the collaborative connector module, which consists of several sequentially stacked collaborative blocks. Each collaborative block sets up a dual-path orthogonal isolation mechanism for topological flow and semantic flow, wherein the topological flow is used to encode the geometric adjacency relationship between tokens, determined by the query matrix. Bond matrix Parameterization is performed and updated based on structural objectives; semantic flow is used to update feature content based on the topological graph constructed by topological flow, consisting of independent value matrices. Parameterization, optimized and updated according to semantic goals, achieves orthogonal isolation between structural and semantic goals in the parameter space; The model training module is used to jointly train the network based on the network framework using a three-stage training strategy, and finally obtain a unified visual word segmentation model based on topological orthogonal decoupling; the three-stage training strategy includes a topology warm-up stage, a semantic injection stage, and a collaborative fine-tuning stage.

8. A unified visual word segmentation method based on topological orthogonal decoupling, characterized in that, Includes the following steps: The original input image with high-dimensional pixels is obtained and fed into an end-to-end unified visual word segmentation model, which outputs the continuous or discrete visual words corresponding to the image. The end-to-end unified visual word segmentation model is obtained by the unified visual word segmentation model construction method based on topological orthogonal decoupling as described in any one of claims 1-6. The visual words are used as a unified interface and input into a multimodal large language model for image semantic understanding tasks, or into a diffusion model / autoregressive model for high-fidelity image generation and editing tasks.

9. A computer device, characterized in that, It includes a processor and a memory, the memory being used to store a computer-executable program, the processor reading part or all of the computer-executable program from the memory and executing it, and the processor executing part or all of the computer-executable program enabling the implementation of claim 1. The unified visual word segmentation model construction method based on topological orthogonal decoupling as described in any one of claims 6, or the unified visual word segmentation method based on topological orthogonal decoupling as described in claim 8.

10. A computer-readable storage medium, characterized in that, A computer-readable storage medium stores a computer program that, when executed by a processor, enables the implementation of claim 1. The unified visual word segmentation model construction method based on topological orthogonal decoupling as described in any one of claims 6, or the unified visual word segmentation method based on topological orthogonal decoupling as described in claim 8.

Images