A deep learning method for visual or text tasks based on adaptive topology

Abstract

A deep learning method for visual or textual tasks based on an adaptive topological structure significantly improves the performance of the state-space model. In terms of vision, by receiving an image sequence and constructing an undirected m-connected graph, a tree topology scanning algorithm is used to dynamically generate a minimum spanning tree (MST) to effectively encode spatial and semantic information. The feature data undergoes multi-stage processing and ultimately generates the visual task results through the output layer. For textual tasks, the model receives sentences or documents, constructs an undirected m-connected graph to represent the text structure, and also applies a tree topology scanning algorithm for feature aggregation and state updates. Multi-stage processing extracts and converts language features. The final state is processed by the output layer to generate the textual task results. The present invention effectively captures long-distance dependencies, reduces computational costs, and improves the processing accuracy of visual and textual tasks. It is suitable for scenarios such as security monitoring and industrial quality inspection.

Description

Technical Field

[0001] The present invention relates to the application of deep learning technology to vision or text tasks, and in particular to a deep learning method for vision or text tasks based on an adaptive topological structure. Background Art

[0002] The development of deep neural networks has been a crucial catalyst for the exploration of machine visual perception. CNN-based models were the first to achieve this milestone, with ResNet, known for its innovative residual connection module, gaining widespread adoption across various areas of visual recognition. Furthermore, more efficient convolutional operations emerged, such as the deep convolution introduced by MobileNet, paving the way for lightweight models. Subsequently, the Visual Transformer (ViT) redefined the paradigm for visual recognition. By incorporating the Transformer architecture from natural language processing, it redefined the architecture design and training mechanism, aiming to improve computational efficiency and broaden its application. Subsequent research has focused on hierarchical structure design, which involves gradually reducing the feature resolution of the backbone network. Furthermore, recent research based on convolutional neural networks has re-emphasized the power of convolutional networks. For example, InternImage proposed a large-scale model based on deformable convolutions, while UniRepLKNet demonstrated significant performance through large kernel convolutions.

[0003] In the field of deep learning, state-space models (SSMs) draw inspiration from control theory and have attracted attention for their advantages in sequence modeling. Structured state-space models utilize structural parameter initialization to achieve stable optimization and superior computational performance. They are typically formulated as linear time-invariant systems that map input signals to output signals, transforming them through a state vector.

[0004] In addition, state space models (SSMs) have become a novel class of models in the deep learning paradigm, showing great potential in sequence transformation. These methods have attracted widespread attention due to their linear scalability with sequence length. The early method LSSL drew inspiration from continuous state space models in control systems and attempted to solve the long-distance dependency problem by combining it with HIPPO[8] initialization. The S4 model proposed normalizing the parameters into a diagonal matrix, which triggered a series of studies on structured SSMs. Recently, the selective state space model, known as Mamba, achieved a balance between effectiveness and efficiency by designing an input-dependent parameter initialization strategy, which has become a strong competitor to Transformer and CNN structures. In addition to demonstrating excellent results in sequence modeling, Mamba has also been seamlessly integrated into the visual field. These studies usually rely on hand-crafted fixed scanning mechanisms to alleviate the performance bias of selective state space models on 2D non-causal images.

[0005] The main drawback of existing technologies is that they cannot fully capture the inherent spatial structure and long-range dependencies of the input data. In particular, in visual tasks, traditional SSMs often adopt fixed-pattern scanning strategies, such as multi-directional raster scanning or local window scanning. These methods cannot effectively preserve the two-dimensional structural information of the image, limiting the model's ability to capture spatial relationships. In addition, although the Mamba model performs well in sequence modeling, its fixed memory size limits the expansion of the state space, resulting in reduced extrapolation ability during the rolling process. In addition, the fixed propagation trajectory of existing technologies cannot dynamically adjust the topology structure and cannot adapt to changes in the input data.

[0006] It should be noted that the information disclosed in the above background technology section is only used to understand the background of this application, and therefore may include information that does not constitute prior art known to ordinary technicians in this field. Summary of the Invention

[0007] The main purpose of the present invention is to overcome the defects of the above-mentioned background technology and provide a deep learning method for visual or text tasks based on adaptive topological structure.

[0008] To achieve the above object, the present invention adopts the following technical solutions:

[0009] In a first aspect of the present invention, a deep learning method for visual tasks based on an adaptive topology structure comprises:

[0010] Receive input data of a visual task and convert it into an embedded feature representation; the received input data of the visual task includes an image sequence, wherein the input feature representation is a pixel value of an image in the image sequence;

[0011] Based on the embedded features, an undirected m-connected graph is constructed, where nodes represent pixels in the image, edges connect adjacent pixels, and each pixel is connected to its m nearest neighbors. The number of vertices corresponds to the length of the input sequence or the number of input pixels, and the weights of the edges are calculated based on the differences in the features of adjacent pixels.

[0012] By utilizing feature dissimilarity, a minimum spanning tree (MST) is dynamically generated on the m-connected graph through a tree topology scanning algorithm. The sum of the edge weights of the tree is minimized, and spatial and semantic information is encoded into the tree structure.

[0013] Through the tree topology scanning algorithm, each node in the tree structure is iteratively traversed, each node in the tree is used as the root node, the features from the child nodes are aggregated, and the state of the root node is updated using the state transfer function;

[0014] The feature data passes through multiple processing stages, each of which consists of multiple basic blocks to perform feature processing and spatial dimension transformation;

[0015] At the root node of the tree, the aggregated features are fused and further feature transformation and processing is performed;

[0016] The final state passes through the output layer and is finally processed according to the requirements of the visual task;

[0017] Based on the calculation of the output layer, the final visual task processing result is generated.

[0018] In a second aspect of the present invention, a text task deep learning method based on an adaptive topology structure comprises:

[0019] Receive input data for text tasks, such as sentences or documents, and convert them into numerical feature representations that the model can understand through the embedding layer;

[0020] Using the embedded features, an undirected m-connected graph is constructed, where nodes represent tokens in the text, such as words or characters, and edges connect adjacent tokens, representing the relationship between tokens to overcome the geometric constraints of text sequences; the weights of the edges are calculated based on the feature differences between adjacent tokens;

[0021] By analyzing the dissimilarity between marker features, the minimum spanning tree (MST) is constructed using the tree topology scanning algorithm, providing a structural basis for state propagation and feature aggregation.

[0022] Through the tree topology scanning algorithm, each node of the tree topology structure is iteratively traversed, each node in the tree is used as the root node, the features from the child nodes are aggregated, and the state of the root node is updated using the state transfer function;

[0023] The feature data passes through multiple processing stages, each consisting of multiple basic blocks. Each stage further extracts and transforms language features to capture deep patterns in the text data.

[0024] At the root node of the tree, the aggregated features are fused and further feature transformation and processing are performed to enhance the language representation;

[0025] The final state passes through the output layer and is finally processed according to the requirements of the text task;

[0026] Based on the calculation of the output layer, the final text task processing result is generated.

[0027] In a third aspect of the present invention, a computer program product comprises a computer program, wherein the computer program implements the method when executed by a processor.

[0028] Compared with the prior art, the present invention has the following beneficial effects:

[0029] The present invention effectively addresses the shortcomings of state-space models (SSMs) in modeling long-distance dependencies. Since SSMs usually adopt a unidirectional cyclic propagation mechanism, although they are more efficient than Transformer models, they are limited in their representational ability, that is, in capturing long-distance dependencies in sequence data. In addition, there are also great challenges in extending SSMs from one-dimensional sequence modeling to the field of two-dimensional visual space recognition. In response to these shortcomings, the present invention proposes a new method to enhance the long-distance modeling capability of SSMs by introducing an input-aware tree topology network. The present invention breaks the traditional sequence constraints and achieves more effective feature propagation and interaction through a dynamically generated tree topology structure, thereby improving the performance of the model in visual and text (language) tasks.

[0030] The method of the embodiment of the present invention is used to perform visual recognition tasks and natural language processing tasks, such as image classification, target detection, natural language understanding and text generation and classification. By introducing an adaptive tree topology structure and a dynamic programming algorithm with linear complexity, the performance of the deep learning model in visual and language tasks is improved, especially in capturing long-distance dependencies and spatial relationships. This adaptive tree topology enables the model to dynamically generate optimal topological connections based on input features, effectively capturing long-distance dependencies in sequence data. At the same time, the dynamic programming algorithm with linear complexity solves the quadratic complexity problem in traditional tree structure propagation, realizes an efficient feature propagation mechanism, allows the model to capture important long-distance information while maintaining computational efficiency, enhances the model's ability to capture spatial relationships, and improves the adaptability of the state-space model in visual and language tasks. These advantages make the present invention have significant application value in intelligent monitoring equipment in the security industry and in intelligent quality inspection algorithms in the industrial industry, and can achieve higher-precision target detection and improve the accuracy and efficiency of industrial quality inspection.

[0031] Other beneficial effects of the embodiments of the present invention will be further described below. BRIEF DESCRIPTION OF THE DRAWINGS

[0032] Figure 1 Scanning strategy graphs for existing visual state spaces.

[0033] Figure 2 Graph of the existing text state space propagation strategy.

[0034] Figure 3 Schematic diagram of a tree state space model according to an embodiment of the present invention.

[0035] Figure 4 Schematic diagram of the GrootV architecture according to an embodiment of the present invention. DETAILED DESCRIPTION

[0036] The following is a detailed description of the embodiments of the present invention. It should be emphasized that the following description is only exemplary and is not intended to limit the scope of the present invention and its application.

[0037] The embodiment of the present invention provides a deep learning method for visual tasks based on an adaptive topological structure, comprising: receiving input data of a visual task and converting it into an embedded feature representation; the received input data of the visual task includes an image sequence, wherein the input feature representation is a pixel value of an image in the image sequence; the visual task can be a task such as image classification or target detection; based on the embedded features, constructing an undirected m-connected graph, wherein nodes represent pixels in the image, edges connect adjacent pixels, and each pixel is connected to its m nearest neighbor pixels; the number of vertices corresponds to the length of the input sequence or the number of input pixels, and the weight of the edge is calculated based on the difference in features of adjacent pixels; m is preferably 4; utilizing feature dissimilarity, dynamically generating on the m-connected graph through a tree topology scanning algorithm The minimum spanning tree (MST) has the smallest sum of edge weights and encodes spatial and semantic information into the tree structure. Through the tree topology scanning algorithm, each node in the tree structure is iteratively traversed, and each node in the tree is used as the root node. The features from the child nodes are aggregated, and the state of the root node is updated using the state transfer function. The feature data passes through multiple processing stages, each stage consists of multiple basic blocks, performing feature processing and spatial dimension transformation, such as feature extraction, conversion, and downsampling, to reduce the spatial dimension and increase the number of feature channels. At the root node of the tree, the aggregated features are fused for further feature transformation and processing. The final state passes through the output layer for final processing according to the requirements of the visual task. Based on the calculation of the output layer, the final visual task processing result is generated.

[0038] See Figure 3 In a preferred embodiment, the deep learning method for visual tasks adopts a tree state space model, which includes the following submodules: an embedding module, which converts the input visual data into an embedded feature representation; a graph construction module, which constructs an undirected m-connected graph, where vertices represent pixel embeddings and edges represent spatial relationships between pixels; a pruning module, which uses the shrinking Boruvka algorithm to prune the edges in the graph, generates a minimum spanning tree MST, and optimizes the sum of edge weights; a state space model initialization module, which initializes the state for each node in the MST and prepares for dynamic state updates; a tree topology scanning module, which iteratively traverses each vertex, regards it as a root node, aggregates features from other vertices, and updates the state; a parameter generator, which generates a data-related transformation matrix for state propagation; a feature propagation and interaction module, which realizes dynamic propagation and interaction of features through a tree topology structure; an activation function module, which uses SiLU and SoftPlus activation functions to introduce nonlinearity and enhance the expressive power of the model; a linear transformation module, which transforms features through a series of linear layers; and an output layer, which performs final processing and generates prediction results according to the requirements of the visual task.

[0039] In a preferred embodiment, in the tree topology scanning algorithm: cosine distance is used as a measure of the distance between vertices; the shrinking Boruvka algorithm is applied to prune the edges in the graph to generate a minimum spanning tree (MST); for each node, the aggregation state is calculated based on the transition matrix and path weight of its parent node; the derivative of the output hidden state with respect to the input variable is derived to achieve end-to-end training of the model; and the output features are calculated based on the aggregated state and the original input features for subsequent visual task processing.

[0040] In a preferred embodiment, the tree topology scanning algorithm implements dynamic programming, which specifically includes (see the algorithm shown in Table 1 below): receiving input features, input matrix and state matrix of the visual task, and loss function gradient; performing breadth-first search BFS on the spanning tree to obtain a topological sorting order; updating the state from the leaf node to the root node in the topological sorting order through forward propagation; for each node, updating the state of the current node according to the state and related parameters of its child nodes; if the current node is the root node, directly using the input features and state matrix to determine its state; if it is not the root node, then updating the state of the current node according to the state and related parameters of the parent node; The input features of the current node are used to calculate the state of the current node; through backpropagation, the gradient is updated from the root node to the leaf node again in the topological sorting order; for each node, the gradient of the current node is updated according to the gradient of the child node and the state of the current node. For the root node, the gradient update is directly associated with the state matrix and the input matrix. For non-root nodes, the gradient update needs to consider the gradient of the parent node and the state of the current node; thus, the state propagation process is optimized through dynamic programming to achieve linear complexity propagation; based on the updated state matrix and input features, the output features are generated for subsequent visual task processing.

[0041] The embodiments of the present invention significantly improve the performance of the state-space model in processing long-distance dependencies and two-dimensional visual space recognition by introducing an adaptive tree topology network and a dynamic programming algorithm with linear complexity. The adaptive tree topology structure can dynamically generate optimal connections based on the input features, effectively capturing long-distance dependencies in sequence data, while the linear complexity algorithm solves the high computational cost problem in traditional tree structure propagation and achieves efficient feature propagation. As a result, the model performs well in visual recognition tasks such as image classification and target detection, especially in capturing long-distance dependencies and spatial relationships. The present invention can be used for intelligent monitoring equipment in the security industry and intelligent quality inspection in the industrial industry, and can significantly improve target detection accuracy and industrial quality inspection efficiency.

[0042] In another embodiment, the present invention also provides a text task deep learning method based on an adaptive topological structure, comprising: receiving input data of a text task, such as a sentence or a document, and converting it into a numerical feature representation that can be understood by the model through an embedding layer; the text task can be a task such as text generation or text classification; using the embedded features, constructing an undirected m-connected graph, in which nodes represent tags in the text, such as words or characters, and edges connect adjacent tags, representing the relationship between tags, so as to overcome the geometric constraints of the text sequence; the weight of the edge is calculated based on the feature difference between adjacent tags; m is preferably 3; by analyzing the dissimilarity between the tag features, a minimum tree topology scanning algorithm is used to construct a minimum tree topology scanning algorithm. The spanning tree MST provides a structural basis for state propagation and feature aggregation; through the tree topology scanning algorithm, each node of the tree topology structure is iteratively traversed, each node in the tree is used as the root node, the features from the child nodes are aggregated, and the state of the root node is updated using the state transfer function; the feature data passes through multiple processing stages, each stage consists of multiple basic blocks, and each stage further extracts and transforms language features to capture deep patterns in the text data; at the root node of the tree, the aggregated features are fused, and further feature transformation and processing are performed to enhance the language representation; the final state passes through the output layer and is finally processed according to the requirements of the text task; based on the calculation of the output layer, the final text task processing result is generated.

[0043] In some embodiments, in the tree topology scanning algorithm: cosine distance is used as a measure of the distance between vertices; the shrinking Boruvka algorithm is applied to prune the edges in the graph to generate a minimum spanning tree (MST); based on the causal prediction method in the text task, the last tag of the sentence or document is selected as the root node; the derivative of the output hidden state with respect to the input variable is derived to achieve end-to-end training of the model; the output features are calculated based on the aggregated state and the original input features for subsequent text task processing.

[0044] In a preferred embodiment, the tree topology scanning algorithm implements dynamic programming, which specifically includes (see the algorithm shown in Table 2 below): receiving input features, an input matrix, a state matrix, and a loss function gradient of a text task. Performing a breadth-first search (BFS) on the spanning tree to obtain a topological sorting order; updating the state from leaf nodes to root nodes in the topological sorting order through forward propagation; calculating and updating the gradient from root nodes to leaf nodes in the topological sorting order through backward propagation; for root nodes, the gradient update is directly related to the state matrix and the input matrix, while for non-root nodes, the gradient update needs to consider the gradient of the parent node and the state of the current node; thus, the state propagation process is optimized through dynamic programming to achieve linear complexity propagation; and output features are generated based on the updated state matrix and input features for subsequent text task processing.

[0045] The embodiment of the present invention significantly improves the performance of the state-space model in processing long-distance dependencies and two-dimensional visual space recognition by introducing an adaptive tree topology network and a linear complexity dynamic programming algorithm. The adaptive tree topology structure can dynamically generate optimal connections based on the input features, effectively capturing long-distance dependencies in sequence data, while the linear complexity algorithm solves the high computational cost problem in traditional tree structure propagation and achieves efficient feature propagation. As a result, the present invention performs well in natural language processing tasks such as natural language understanding, text generation, and classification, especially in capturing long-distance dependencies and spatial relationships.

[0046] Specific embodiments of the present invention are further described below.

[0047] In the following specific embodiment, a tree state space model is constructed and a network named GrootVL is proposed. This is an innovative network framework that can include two sub-networks GrootV and GrootL, designed for vision and language tasks respectively. The core idea of ​​GrootVL is to dynamically generate a tree topology based on input features and spatial relationships, and perform feature propagation on this structure. A schematic diagram of a tree state space model is shown below. Figure 3 shown.

[0048] Tree topology scanning algorithm

[0049] Given an input feature Where L is the sequence length (or the number of input pixels), an undirected m-connected graph G = (V, E) is constructed for the feature. Among them, m is a hyperparameter representing the number of adjacent tags. In visual tasks, m = 4, which means that each pixel is connected to its four neighboring pixels. For language tasks, the default setting is m = 3, which means that each tag is connected to the previous three tags. In addition, the vertex V represents the pixel (or tag) embedding, and E represents the edge of the graph. The edge weight is calculated by the feature dissimilarity between adjacent vertices. In addition, the distance metric between vertices uses the cosine distance by default. The shrinking Boruvka algorithm is used to prune edges with significant differences, thereby generating a minimum spanning tree (MST) whose sum of difference weights is the smallest among all spanning trees. During the propagation process, each vertex is iterated, regarded as the root, and the features of the remaining vertices are aggregated. Intuitively, applying state propagation in such a geometric configuration will preferentially interact between vertices with smaller spatial and feature distances. Referring to Mamba, state propagation is performed using a data-dependent transformation matrix. For vertex k, the transformation matrix for the parent node is expressed as Therefore, the state aggregation process of the i-th vertex can be expressed as:

[0050]

[0051] Where Ω represents the index set of all vertices in the tree. ij ) represents the hyperedge E ij The path weight from the jth vertex to the ith vertex in the MST, N ij Represents the index set of all vertices on this hyperedge. For visual tasks, iterate over each vertex, treat it as the root of the spanning tree, and aggregate the states from other vertices to obtain the transformed state For text tasks, due to the causal prediction method in large language models, only the last token is taken as the root and aggregated from other tokens. In order to achieve end-to-end training, the output hidden state hi is derived for the input variable and x j The derivative of is as follows:

[0052]

[0053]

[0054] in represents the child node of vertex k with vertex i as the root, and n is the parent node of vertex k. Finally, the output feature Y can be expressed as:

[0055] Y=C⊙Norm(H)+D⊙X,

[0056] Where Y, H and X represent and ⊙ represents element-wise multiplication.

[0057] To handle the resulting quadratic complexity, a dynamic programming algorithm is proposed that implements linear complexity propagation to enhance long-distance interactions without increasing computational cost. This algorithm optimizes the state propagation process, enabling GrootVL to achieve more effective long-distance dependency modeling while maintaining computational efficiency. The pseudo code for the visual tree topology scanning algorithm that implements dynamic programming is shown in Table 1:

[0058] Table 1

[0059]

[0060] The pseudo code for implementing the text tree topology scanning algorithm for dynamic programming is shown in Table 2 below:

[0061] Table 2

[0062]

[0063] Based on these basic operators, networks can be constructed for various vision and text tasks. For vision tasks, GrootV first leverages the dissimilarity between adjacent features to construct a minimum spanning tree (MST) on a four-connected planar graph. This process adaptively encodes spatial and semantic information into the tree. It then iterates through each pixel, treating it as a root node and aggregating features from other pixels using Mamba's state transition function. Figure 4 A preferred GrootV architecture is shown.

[0064] like Figure 4 As shown, based on the GrootV architecture, the visual task processing method of a specific embodiment includes the following steps:

[0065] 1. Input processing: Receive input data for a vision task, such as an image sequence, and convert the pixel values ​​of each image into an embedded feature representation.

[0066] 2. Graph construction: Based on the embedded features, a minimum spanning tree (MST) is constructed on a four-connected planar graph, where each pixel is regarded as a node in the graph and is connected to its four nearest neighbor pixels.

[0067] 3. Feature dissimilarity analysis: Exploiting the dissimilarity between adjacent features, we adaptively encode spatial and semantic information into the tree graph.

[0068] 4. Tree topology scanning: It iteratively traverses each pixel node and regards it as the root node, using Mamba's state transfer function to aggregate the features of other pixels.

[0069] 5. Multi-stage feature processing: Feature data passes through multiple processing stages, each consisting of multiple basic blocks, performing feature extraction, transformation, and downsampling. Within each stage, a tree state space model and downsampling techniques are applied to reduce spatial dimensions and increase the number of feature channels.

[0070] 6. Feature fusion and transformation: At the root node of the tree, the aggregated features are fused and further feature transformation and processing is performed through linear layers, SiLU activation functions, SoftPlus functions, etc.

[0071] 7. Output layer processing: The final state passes through the output layer (Head) and is finally processed according to the requirements of the visual task (such as classification Cls., detection Det., segmentation Seg.).

[0072] 8. Result generation: Based on the calculation of the output layer, the final visual task processing results are generated, such as category labels, object detection boxes, segmentation maps, etc.

[0073] pass Figure 4 The GrootV network shown in this embodiment can effectively process visual tasks, capture key information in images, and provide accurate visual recognition results.

[0074] For language tasks, the GrootL network overcomes the geometric constraints of text sequences by constructing a tree topology based on the dissimilarity of labeled features. The GrootL of the embodiment uses a similar aggregation process to GrootV (not repeated here), which can significantly enhance the language representation capabilities of pre-trained large language models.

[0075] In summary, the method of the embodiment of the present invention is used to perform visual recognition tasks and natural language processing tasks, such as image classification, target detection, natural language understanding and text generation and classification. By introducing an adaptive tree topology structure and a dynamic programming algorithm with linear complexity, the performance of the deep learning model in visual and language tasks is improved, especially in capturing long-distance dependencies and spatial relationships. This adaptive tree topology structure enables the model to dynamically generate optimal topological connections based on input features, effectively capturing long-distance dependencies in sequence data. At the same time, the dynamic programming algorithm with linear complexity solves the quadratic complexity problem in traditional tree structure propagation, realizes an efficient feature propagation mechanism, allows the model to capture important long-distance information while maintaining computational efficiency, enhances the model's ability to capture spatial relationships, and improves the adaptability of the state-space model in visual and language tasks. These advantages make the present invention have significant application value in intelligent monitoring equipment in the security industry and in intelligent quality inspection algorithms in the industrial industry, and can achieve higher-precision target detection and improve the accuracy and efficiency of industrial quality inspection.

[0076] Compared with conventional technologies, the innovations and advantages of the embodiments of the present invention are mainly reflected in the following aspects:

[0077] 1. Adaptive tree topology: The model can dynamically generate a tree topology based on the input features, which enables the model to adaptively capture spatial and semantic information, thereby enhancing the ability to model long-range dependencies.

[0078] 2. Linear complexity feature propagation: The linear complexity dynamic programming algorithm effectively solves the high complexity problem in traditional tree structure propagation. This mechanism effectively reduces the computational cost while maintaining the model's ability to capture long-range dependencies.

[0079] 3. By introducing an adaptive tree topology structure, the limitations of traditional state-space models in visual tasks are improved, and the model's ability to capture spatial relationships is enhanced.

[0080] 4. Compared with other visual recognition methods based on state-space models, the method of the present invention can more comprehensively and systematically improve the two-dimensional spatial perception ability of the model, thereby achieving comprehensive and systematic visual recognition improvement.

[0081] Deep learning methods for visual tasks based on adaptive topology can be used in security surveillance tasks, enabling real-time monitoring and detection of various objects, with higher-precision target detection. They can also be used in industrial quality inspection tasks, improving accuracy and efficiency, enhancing inspection results and product quality, and reducing labor costs.

[0082] An embodiment of the present invention further provides a storage medium for storing a computer program, which at least performs the above method when executed.

[0083] An embodiment of the present invention further provides a control device, comprising a processor and a storage medium for storing a computer program; wherein the processor is configured to execute at least the method described above when executing the computer program.

[0084] An embodiment of the present invention further provides a processor, which executes a computer program and at least performs the method described above.

[0085] The storage medium can be implemented by any type of non-volatile storage device, or a combination thereof. Among them, the non-volatile memory can be a read-only memory (ROM), a programmable read-only memory (PROM), an erasable programmable read-only memory (EPROM), an electrically erasable programmable read-only memory (EEPROM), a magnetic random access memory (FRAM), a flash memory, a magnetic surface memory, an optical disc, or a compact disc read-only memory (CD-ROM); the magnetic surface memory can be a magnetic disk memory or a magnetic tape memory. The storage medium described in the embodiments of the present invention is intended to include, but is not limited to, these and any other suitable types of memory.

[0086] In the several embodiments provided by the present invention, it should be understood that the disclosed systems and methods can be implemented in other ways. The device embodiments described above are merely schematic. For example, the division of the units is merely a logical function division. In actual implementation, there may be other division methods, such as: multiple units or components can be combined, or can be integrated into another system, or some features can be ignored or not executed. In addition, the coupling, direct coupling, or communication connection between the components shown or discussed can be through some interfaces, and the indirect coupling or communication connection of the devices or units can be electrical, mechanical or other forms.

[0087] The units described above as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units; some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0088] In addition, all functional units in the embodiments of the present invention may be integrated into one processing unit, or each unit may be separately used as a unit, or two or more units may be integrated into one unit; the above-mentioned integrated units may be implemented in the form of hardware or in the form of hardware plus software functional units.

[0089] Those skilled in the art will appreciate that all or part of the steps of the above-mentioned method embodiments may be implemented by hardware associated with program instructions, and the aforementioned program may be stored in a computer-readable storage medium. When the program is executed, the program executes the steps of the above-mentioned method embodiments. The aforementioned storage medium includes various media that can store program codes, such as mobile storage devices, read-only memories (ROMs), random access memories (RAMs), magnetic disks, or optical disks.

[0090] Alternatively, if the above-mentioned integrated unit of the present invention is implemented in the form of a software functional module and sold or used as an independent product, it can also be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the embodiment of the present invention, or the part that contributes to the prior art, can be embodied in the form of a software product. The computer software product is stored in a storage medium and includes a number of instructions for enabling a computer device (which can be a personal computer, server, or network device, etc.) to execute all or part of the methods described in each embodiment of the present invention. The aforementioned storage medium includes: various media that can store program codes, such as mobile storage devices, ROM, RAM, magnetic disks or optical disks.

[0091] The methods disclosed in the several method embodiments provided by the present invention can be arbitrarily combined without conflict to obtain new method embodiments.

[0092] The features disclosed in several product embodiments provided by the present invention can be arbitrarily combined without conflict to obtain new product embodiments.

[0093] The features disclosed in several method or device embodiments provided by the present invention can be arbitrarily combined without conflict to obtain new method embodiments or device embodiments.

[0094] The above is a further detailed description of the present invention in conjunction with specific preferred embodiments, and the specific implementation of the present invention should not be considered to be limited to these descriptions. Those skilled in the art will recognize that, without departing from the scope of the present invention, several equivalent substitutions or obvious variations can be made, and the performance or use of the same should be considered to fall within the scope of protection of the present invention.

Claims

1. A deep learning method for visual tasks based on adaptive topology structure, characterized by: include: Receive input data for a visual task and convert it into an embedded feature representation; The received visual task input data includes an image sequence, wherein the input features are represented as pixel values ​​of images in the image sequence; Based on the embedded features, an undirected m-connected graph is constructed, where nodes represent pixels in the image, edges connect adjacent pixels, and each pixel is connected to its m nearest neighbors. The number of vertices corresponds to the length of the input sequence or the number of input pixels, and the weights of the edges are calculated based on the differences in the features of adjacent pixels. By utilizing feature dissimilarity, a minimum spanning tree (MST) is dynamically generated on the m-connected graph through a tree topology scanning algorithm. The sum of the edge weights of the tree is minimized, and spatial and semantic information is encoded into the tree structure. Through the tree topology scanning algorithm, each node in the tree structure is iteratively traversed, each node in the tree is used as the root node, the features from the child nodes are aggregated, and the state of the root node is updated using the state transfer function; The feature data passes through multiple processing stages, each of which consists of multiple basic blocks to perform feature processing and spatial dimension transformation; At the root node of the tree, the aggregated features are fused and further feature transformation and processing is performed; The final state passes through the output layer and is finally processed according to the requirements of the visual task; Generate the final visual task processing results based on the calculation of the output layer; The dynamic programming is realized by the tree topology scanning algorithm, specifically including: Receive the input features, input matrix and state matrix of the visual task, as well as the loss function gradient; Perform a breadth-first search (BFS) on the spanning tree to obtain a topological sort order; Through forward propagation, the state is updated from the leaf node to the root node in the topological sorting order; For each node, update the status of the current node according to the status of its child nodes and related parameters, If the current node is the root node, its state is determined directly using the input features and state matrix. If it is not the root node, the state of the current node is calculated based on the state of the parent node and the input features of the current node. Through back propagation, from the root node to the leaf node, the gradient is updated again according to the topological sorting order; For each node, the gradient of the current node is updated according to the gradient of the child node and the state of the current node. For the root node, the gradient update is directly associated with the state matrix and the input matrix. For non-root nodes, the gradient update needs to consider the gradient of the parent node and the state of the current node; Therefore, the state propagation process is optimized through dynamic programming to achieve linear complexity propagation; Based on the updated state matrix and input features, output features are generated for subsequent visual task processing.

2. The method for deep learning of visual tasks based on adaptive topology structure according to claim 1, characterized in that: A tree state space model is used, which includes the following submodules: An embedding module that converts input visual data into embedded feature representations; Graph construction module: It constructs an undirected m-connected graph, where vertices represent pixel embeddings and edges represent spatial relationships between pixels; Pruning module: It uses the contraction Boruvka algorithm to prune the edges in the graph, generate the minimum spanning tree MST, and optimize the sum of edge weights; State space model initialization module: It initializes the state of each node in the MST and prepares for dynamic state updates; Tree topology scanning module: It iteratively traverses each vertex, treating it as a root node, aggregates features from other vertices, and updates the state; Parameter generator: It generates the data-dependent transition matrix for state propagation; Feature propagation and interaction module: It realizes dynamic propagation and interaction of features through tree topology structure; Activation function module: It uses SiLU and SoftPlus activation functions to introduce nonlinearity and enhance the expressiveness of the model; Linear transformation module: transforms features through a series of linear layers; Output layer: Performs final processing according to the requirements of the visual task and generates prediction results.

3. The method for deep learning of visual tasks based on adaptive topology structure according to claim 1 or 2, characterized in that: In the tree topology scanning algorithm: using cosine distance as a measure of distance between vertices; Apply the contraction Boruvka algorithm to prune the edges in the graph to generate the minimum spanning tree MST; For each node, the aggregation state is calculated based on the transition matrix and path weight of its parent node; Deriving the derivatives of the output hidden state with respect to the input variables to enable end-to-end training of the model; Based on the aggregated state and the original input features, the output features are calculated for subsequent visual task processing.

4. The method for deep learning of visual tasks based on adaptive topology structure according to any one of claims 1 to 2, characterized in that: The vision tasks include image classification or object detection.

5. The method for deep learning of visual tasks based on adaptive topology structure according to claim 4, characterized in that: m is 4.

6. A text task deep learning method based on adaptive topology structure, characterized by: include: Receive input data for text tasks and convert it into numerical feature representations that the model can understand through the embedding layer; Using the embedded features, an undirected m-connected graph is constructed, where nodes represent tokens in the text and edges connect adjacent tokens, representing the relationship between tokens, to overcome the geometric constraints of text sequences; the weight of the edge is calculated based on the feature difference between adjacent tokens; By analyzing the dissimilarity between marker features, the minimum spanning tree (MST) is constructed using the tree topology scanning algorithm, providing a structural basis for state propagation and feature aggregation. Through the tree topology scanning algorithm, each node of the tree topology structure is iteratively traversed, each node in the tree is used as the root node, the features from the child nodes are aggregated, and the state of the root node is updated using the state transfer function; The feature data passes through multiple processing stages, each consisting of multiple basic blocks. Each stage further extracts and transforms language features to capture deep patterns in the text data. At the root node of the tree, the aggregated features are fused and further feature transformation and processing are performed to enhance the language representation; The final state passes through the output layer and is finally processed according to the requirements of the text task; Generate the final text task processing results based on the calculation of the output layer; The dynamic programming is realized by the tree topology scanning algorithm, specifically including: Receive the input features, input matrix and state matrix of the text task, as well as the loss function gradient; Perform a breadth-first search (BFS) on the spanning tree to obtain a topological sort order; Through forward propagation, the state is updated from the leaf node to the root node in the topological sorting order; Through back propagation, the gradient is calculated and updated from the root node to the leaf node in the topological sorting order; For the root node, the gradient update is directly related to the state matrix and the input matrix. For non-root nodes, the gradient update needs to consider the gradient of the parent node and the state of the current node. Therefore, the state propagation process is optimized through dynamic programming to achieve linear complexity propagation; Based on the updated state matrix and input features, output features are generated for subsequent text task processing.

7. The text task deep learning method based on adaptive topology structure according to claim 6, characterized in that: In the tree topology scanning algorithm: using cosine distance as a measure of distance between vertices; Apply the contraction Boruvka algorithm to prune the edges in the graph to generate the minimum spanning tree MST; Based on the causal prediction method in text tasks, the last token of a sentence or document is selected as the root node; Deriving the derivatives of the output hidden state with respect to the input variables to enable end-to-end training of the model; Based on the aggregated state and the original input features, the output features are calculated for subsequent text task processing.

8. The text task deep learning method based on adaptive topology structure according to any one of claims 6 to 7, characterized in that: The text task includes text generation or text classification.

9. The text task deep learning method based on adaptive topology structure according to claim 8, characterized in that: m is 3.

10. A computer program product comprising a computer program, characterized in that When the computer program is executed by a processor, the method according to any one of claims 1 to 9 is implemented.

Images