Neural network image-text analysis and cross-framework code generation method and system

Abstract

The invention discloses a neural network image-text analysis and cross-framework code generation method and system, and the method comprises the steps: analyzing a neural network architecture picture through a visual large model, and extracting a layer type, a connection relation and a topological structure feature; performing semantic understanding on text description by combining a large language model, and extracting layer parameters and configuration information in a standardized manner; utilizing a multi-modal alignment mechanism to fuse vision and text features, and generating unified model representation; and finally, directly generating an executable code supporting a mainstream framework based on a large language model and grammar check. According to the method, the limitation of traditional manual coding is broken through, end-to-end generation from a complex framework to a multi-framework code is achieved, the problems of cross-framework adaptation and semantic understanding are solved, the development efficiency of deep learning is improved, and the method is suitable for scientific research verification and industrial deployment scenes.

Description

Technical Field

[0001] The present invention belongs to the technical fields of artificial intelligence and software development, and relates to an automatic generation method and system for generating multi-frame deep learning code based on neural network architecture pictures and text descriptions. Background Art

[0002] In recent years, with the rapid development of deep learning technology, neural networks have demonstrated powerful capabilities in multiple fields such as image recognition, natural language processing, and autonomous driving. The design and implementation of neural networks have become the core of artificial intelligence research and applications. However, the current development process of deep learning models still highly relies on manual operations. Developers need to manually write code according to design documents or architecture diagrams. This process is not only inefficient but also prone to deviations from the expected results due to human errors.

[0003] In deep learning development, the widespread use of multiple frameworks further exacerbates the development complexity. For example, TensorFlow is known for its static computational graph and is suitable for large-scale distributed training; PyTorch is favored by researchers due to its dynamic computational graph and flexibility; Keras simplifies model construction as a high-level encapsulation tool but has limited functional depth. These frameworks have significant differences in syntax structure, API design, and implementation logic, resulting in developers having to rewrite a large amount of code when migrating across frameworks or supporting multiple frameworks. In addition, neural network architectures are usually presented in the form of pictures (such as flowcharts, topology diagrams) supplemented by text descriptions to explain layer structures, parameter configurations, etc. Developers need to extract key information from them and convert them into executable code one by one. This manual transcoding process is highly repetitive and seriously restricts development efficiency, especially during rapid prototyping verification or model optimization iterations.

[0004] In the prior art, there have been some attempts to reduce the development burden through automated means. For example, in the field of front-end development, researchers use convolutional neural networks (CNNs) combined with image processing technology to achieve pix2code, which can extract element layouts from web design drafts and generate HTML / CSS code. In addition, there are also studies that use natural language processing (NLP) technology to parse programming logic from text descriptions and generate simple scripts or pseudocode. Some scholars have also proposed Sketch2Code based on optical character recognition (OCR) and object detection, attempting to extract structural information from hand-drawn sketches and generate corresponding code. However, these methods mostly focus on specific scenarios (such as user interface generation) or single programming languages and are difficult to be directly applied to generating code that supports multiple deep learning frameworks from complex neural network architecture pictures.

[0005] In the field of deep learning, some research attempts to introduce automation technology into model development. For example, some research uses generative adversarial networks (GANs) or reinforcement learning to generate neural network architectures. However, these methods focus more on architecture search (NAS, Neural Architecture Search) rather than directly generating code from existing design diagrams. In addition, some tools (such as ONNX) provide model format conversion functions, enabling the transfer of pre-trained models between different frameworks. However, they rely on existing code or model files and cannot handle raw architecture pictures and text descriptions as inputs. Another limitation of the existing technology is that the accuracy of image parsing is limited by the difficulty of recognizing complex structures (such as nested layers, multi-branch networks), and the semantic understanding of text descriptions is also difficult to accurately map to code logic due to the lack of a standardized format.

[0006] With the increasing demand for deep learning applications in enterprises, shortening the development cycle and cross-platform compatibility have become key requirements. For example, in scientific research, researchers may need to quickly verify multiple architecture designs; in industrial scenarios, engineers need to switch between different frameworks to adapt to hardware environments (such as GPUs, TPUs, NPUs). However, the existing tools and methods lack flexibility in supporting multi-framework code generation and have limited comprehensive processing capabilities for input forms (pictures and text). This results in developers still having to spend a lot of time on code adaptation and debugging, making it difficult to achieve seamless connection from design to implementation.

[0007] In summary, how to efficiently and accurately convert neural network architecture pictures and text descriptions into executable code that supports multiple deep learning frameworks not only requires solving the technical problems of image parsing and semantic understanding but also needs to overcome the challenges of multi-framework compatibility and insufficient automation. The deficiencies of the existing technology in terms of generality, accuracy, and efficiency provide an urgent need for innovation in this field. Therefore, developing an automated method and system that can generate cross-framework code from multi-modal inputs has become an urgent need for the current technological development. Summary of the Invention

[0008] To solve the problem of generating code that supports multiple deep learning frameworks from complex neural network design diagrams and non-standard text descriptions, reduce the workload of developers writing code manually, improve development efficiency, and enhance the flexibility of cross-framework adaptation, the present invention provides a method and system for neural network graphic and text parsing and cross-framework code generation. The present invention breaks through the limitations of traditional manual coding, realizes end-to-end generation from complex architectures to multi-framework code, solves the problems of cross-framework adaptation and semantic understanding, improves the development efficiency of deep learning, and is applicable to scientific research verification and industrial deployment scenarios.

[0009] The object of the present invention is achieved by the following technical solutions:

[0010] A method for neural network image and text analysis and cross-frame code generation, comprising the following steps:

[0011] Step S1: Use a vision large model to perform visual analysis on the input neural network architecture image, extract the network structure features in the image, including the type of each layer, the connection mode, and the overall topological layout, and generate a structured visual feature vector. The specific steps are as follows:

[0012] Step S11: First, preprocess the input neural network architecture image, including denoising and edge enhancement, to highlight the network layer units and connection line features;

[0013] Step S12: The vision large model extracts features from the neural network architecture image through its vision encoder. The specific steps are as follows: The vision large model divides the image into multiple regions, identifies the network layer units in each region, and infers the inter-layer connection relationship through the pre-trained image-text contrast learning ability, and outputs a feature vector containing position information and semantic labels;

[0014] Step S2: Use a large language model to perform semantic understanding on the input text description, extract the key information in the description, and convert the non-standardized expression into a standardized semantic representation. The specific steps are as follows:

[0015] Step S21: Optimize the input form through prompt engineering: Design a predefined prompt template to guide the large language model to accurately understand the text content;

[0016] Step S22: The large language model decomposes the description and identifies the key components: The large language model uses its natural language understanding ability, inputs the description text through prompt engineering (Prompt Engineering), and extracts the layer type, parameter configuration, and layer order;

[0017] Step S23: To deal with non-standardized expressions, use a semantic normalization mechanism to unify different expressions into a standard format through pre-trained language knowledge. If necessary information is missing in the description, complete the default value according to the conventional practice in the field of deep learning;

[0018] Step S24: The large language model outputs a structured semantic feature vector, including the layer configuration and parameter details of the network;

[0019] Step S3: Through a multimodal alignment mechanism, fuse the visual feature vector extracted by the vision large model with the semantic representation generated by the large language model to generate a unified neural network model feature representation. The specific steps are as follows:

[0020] Step S31: Use projection to align the visual feature vector of the vision large model with the semantic representation of the large language model, and use the multi-head self-attention mechanism to calculate the cross-modal correlation;

[0021] Step S32: To preserve the spatial information of the sequence, introduce learnable position encoding to record the spatial order of the feature sequence.

[0022] Step S33: To improve the fusion effect, introduce residual connections to retain part of the original feature sequence and avoid information loss due to compression; at the same time, use layer normalization technology to adjust the feature distribution and enhance stability.

[0023] Step S34: Output a unified neural network model feature representation, which is stored in the form of a high-dimensional vector and contains the complete topological structure and parameter configuration of the network.

[0024] Step S4: Utilize the code generation ability of the large language model to directly generate executable code supporting the target framework from the fused model feature representation according to the user-specified target framework. The specific steps are as follows:

[0025] Step S41: The large language model receives the fused features and generates target framework code through prompt engineering.

[0026] Step S42: To optimize the generation result, use syntax checking to verify the syntax consistency of the code. If logical errors are detected, adjust and regenerate to ensure the executability of the code.

[0027] A neural network graphic and text parsing and cross-frame code generation system for implementing the above method, including an image parsing module, a text processing module, a multimodal fusion module, a code generation module, and a user interaction module, where:

[0028] The image parsing module receives a neural network architecture picture based on a vision large model and extracts visual features.

[0029] The text processing module parses the text description based on the large language model and generates a standardized semantic representation.

[0030] The multimodal fusion module fuses image and text information through an attention mechanism to form a unified neural network model feature representation.

[0031] The code generation module directly generates deep learning code for the user-specified framework from the fused features using the large language model.

[0032] The user interaction module provides a graphical interface and natural language input function, supporting users to modify and verify the code in real time.

[0033] Compared with the prior art, the present invention has the following advantages:

[0034] 1) The present invention combines the image-text joint learning ability of the vision large model and the semantic understanding and code generation expertise of the large language model, realizing efficient multi-modal processing of neural network architecture pictures and text descriptions, overcoming the limitations of single-input parsing in the prior art, and being able to accurately identify complex network structures and parameter details.

[0035] 2) The present invention directly generates code for multiple deep learning frameworks from the fused features through the large language model, avoiding the intermediate mapping step, significantly improving the generation efficiency, and enabling users to seamlessly switch between frameworks such as TensorFlow, PyTorch, and MindSpore through simple instructions.

[0036] 3) In the present invention, the user interaction module combines the context understanding ability of the large language model, supports natural language modification and real-time optimization, and enhances the practicality and flexibility of the code.

[0037] 4) Through the direct generation mechanism of the large model, the present invention greatly improves the automation level and efficiency of deep learning development, provides an efficient and convenient solution for scientific research verification, model iteration, and industrial deployment, and promotes the rapid transformation from design to implementation. BRIEF DESCRIPTION OF THE DRAWINGS

[0038] Figure 1 It is a flowchart of the method for neural network graphic and text parsing and cross-framework code generation;

[0039] Figure 2 It is a comparison chart of image segmentation with different sizes;

[0040] Figure 3 It is a flowchart of the text understanding stage

[0041] Figure 4 It is a schematic diagram of the feature fusion process;

[0042] Figure 5 It is a schematic diagram of the code generation process;

[0043] Figure 6 It is an architecture diagram of the neural network graphic and text parsing and cross-framework code generation system. DETAILED DESCRIPTION OF THE INVENTION

[0044] The technical solutions of the present invention will be further described below with reference to the accompanying drawings, but are not limited thereto. Any modification or equivalent replacement of the technical solutions of the present invention without departing from the spirit and scope of the technical solutions of the present invention shall be covered by the protection scope of the present invention.

[0045] The present invention provides a method for neural network graphic parsing and cross-frame code generation. The method uses a vision large model to parse neural network architecture pictures, extracting layer types, connection relationships, and topological structure features; combines a large language model to perform semantic understanding on text descriptions, and standardizes the extraction of layer parameters and configuration information; uses a multimodal alignment mechanism to fuse visual and text features to generate a unified model representation; and finally, based on the large language model and grammar checking, directly generates executable code that supports mainstream frameworks. As Figure 1 shown, the specific steps are as follows:

[0046] Step S1: Use a vision large model to perform visual parsing on the input neural network architecture picture, extracting the network structure features in the picture, including the types of each layer (such as convolutional layer, fully connected layer), connection patterns (such as series, branch), and overall topological layout, to generate a structured visual feature vector.

[0047] In this step, a vision large model is used to extract structured features from the input neural network architecture picture. The picture needs to be preprocessed first, such as a hand-drawn network structure sketch or a topological graph generated by software. The picture size may vary from a few hundred pixels to thousands of pixels. The preprocessing process is as follows: Remove background noise interference through a Gaussian blur filter, and apply edge enhancement technology to highlight the contours of network layer units and connection lines to improve the clarity of subsequent feature extraction. The feature extraction of the vision large model starts from this step. To avoid information loss caused by fixed scaling, the system dynamically divides the picture into variable-length sequences according to the original picture size. Specifically, a grid division strategy is adopted, setting a fixed grid size, and dividing the image into multiple regions according to the aspect ratio and resolution of the picture. The number of regions is positively correlated with the size, as Figure 2 shown. Subsequently, the vision encoder of the vision large model independently extracts features from each region and identifies the network layer units therein. Through the multimodal learning ability fine-tuned based on the labeled dataset, the model infers the topological relationship between regions, such as identifying skip connections or parallel paths through the arrow direction. Finally, the system integrates the features of each region into a variable-length sequence of visual feature vectors, retaining the spatial information and details of the original picture, providing a flexible and comprehensive visual representation for multimodal fusion.

[0048] Step S2: Use a large language model to perform semantic understanding on the input text description, extract key information in the description, such as layer parameters (convolution kernel size, stride), activation function type (such as ReLU, Sigmoid), etc., and convert non-standard expressions into standardized semantic representations.

[0049] In this step, the present invention uses a large language model to perform semantic parsing, structured processing, and extract text features from the input text description, as Figure 3As shown, this process starts with receiving the description provided by the user. During implementation, first, the input form is optimized through prompting engineering: a predefined prompt template is designed to guide the large language model to accurately understand the text content. The model then decomposes the description and identifies key components: information such as layer types, parameter configurations, and the order between layers is extracted through natural language processing techniques. To handle non-standard expressions, the method provides a semantic normalization mechanism to unify different expressions into a standard format through pre-trained language knowledge. If necessary information is missing from the description, the model fills in default values according to common practices in the field of deep learning. Finally, the large language model outputs a structured semantic feature vector, which contains the layer configuration and parameter details of the network. This processing method converts diverse text inputs into consistent semantic representations, ensuring alignment with the image feature sequence and providing reliable semantic support for subsequent fusion.

[0050] Step S3: Through a multimodal alignment mechanism, fuse the visual feature vector extracted by the visual large model with the semantic representation generated by the large language model to generate a unified neural network model feature representation.

[0051] In this step, the present invention integrates the visual feature sequence extracted by the visual large model and the semantic features generated by the large language model through an attention mechanism, as Figure 4 shown. This process starts with feature mapping: projecting the variable-length visual feature vector sequence and semantic feature vector into a unified feature space to ensure that the data of the two modalities are semantically comparable.

[0052] During implementation, a multi-head self-attention mechanism is used to calculate cross-modal correlations: for each regional feature in the visual feature sequence, calculate its attention weight with the corresponding description in the semantic features, and generate a comprehensive representation through weighted fusion. To retain the spatial information of the sequence, position encoding is introduced during the fusion process to record the relative positions and layer sequence relationships between regions. To improve the fusion effect, residual connections are introduced to retain part of the original feature sequence and avoid information compression and loss; at the same time, layer normalization technology is used to adjust the feature distribution and enhance stability. The final output is a unified neural network model feature representation, which is stored in the form of a high-dimensional vector and contains the complete topological structure of the network (such as layer sequence, connection pattern) and parameter configuration (such as convolution kernel size, activation function). This fusion method makes full use of the flexibility of variable-length sequences and the semantic complement of text to ensure that the model features fully reflect the design intent.

[0053] Step S4: Utilize the code generation ability of the large language model to directly generate executable code supporting the target framework from the fused model feature representation according to the user-specified target framework.

[0054] In this step, the present invention provides the fused model feature representation to the large language model, which directly generates deep learning framework code that meets the user-specified requirements, such as Figure 5As shown. During implementation, the system takes the fusion feature as input and guides the instructions through prompt engineering design. The large language model utilizes its pre-trained code generation ability to parse the structural information in the fusion feature, maps the layer type to the function of the corresponding framework, and fills the parameter configuration as function parameters. To ensure that the generated code meets the requirements and improves the result quality, a built-in syntax template check is performed to conduct a preliminary syntax verification on the code after generation. For example, it checks whether the function parameters match the framework specifications. If a syntax error is found, it regenerates through context adjustment, and finally obtains executable code that conforms to the specified framework specifications.

[0055] The present invention also provides a multi-framework code automatic generation system based on a neural network architecture picture and text description, as Figure 6 shown. The system includes an image parsing module, a text processing module, a multi-modal fusion module, a code generation module, and a user interaction module, where:

[0056] The image parsing module is implemented based on a vision large model and is equipped with a picture preprocessing and dynamic segmentation unit to convert the picture into a variable-length feature sequence, providing flexible visual support for multi-modal processing;

[0057] The text processing module is deployed based on a large language model and integrates prompt engineering and semantic normalization functions to ensure the accurate parsing and consistency of the text description;

[0058] The multi-modal fusion module adopts an attention mechanism and residual connection to integrate the image sequence and text information to generate a unified model feature representation. Among them, the residual connection adds the original visual feature vector sequence to the intermediate feature vector after attention fusion to retain visual details;

[0059] The code generation module utilizes the generation ability of the large language model to generate target framework code with the fusion feature as input and optimizes the result through syntax checking;

[0060] The user interaction module provides natural language and graphical interfaces to support real-time adjustment and optimization of the generated code by the user.

[0061] In the present invention, system modules are interconnected through a standardized interface RESTAPI, support distributed deployment or local operation, and have good scalability. The user interaction module integrates a code verification function, checks the syntax correctness through static analysis, and detects the tensor dimension matching through logical reasoning. Specifically, the user can modify instructions through natural language input or edit the network structure through a graphical interface, and the system will feedback the modification information to the fusion features and regenerate the code; at the same time, the verification function is integrated to check the code correctness through static analysis and logical reasoning, and provide optimization suggestions. This modular-designed system realizes the complete process from input parsing to code generation through collaborative work, and enhances the practicality and flexibility of the system through user interaction.

[0062] To further illustrate the actual application effect of the present invention, a specific application example based on the above system is provided below to generate an object detection model (YOLO variant) in an autonomous driving system.

[0063] Step S1, Image Parsing:

[0064] The input architecture picture is first denoised and edge-enhanced through a preprocessing module, and the Canny edge detection algorithm is used to highlight the contours of network layer units and connection lines, reducing the interference of background textures in hand-drawn sketches. The vision large model (based on the CLIP-ViT architecture) dynamically divides the picture into a 14×14 grid, and each grid area independently extracts features. Through the pre-trained image-text contrast learning ability, the model identifies the layer type within the area (such as "3×3Conv" is parsed as a convolutional layer) and its connection relationship (such as an arrow represents a residual connection), generating a sequence of visual feature vectors containing position information (grid coordinates) and semantic labels (layer type, parameters). For example, the ResNet-50 structure of the backbone network is parsed as a topological sequence of 50 convolutional layers and residual connections, and the fusion node of FPN is identified as a multi-input addition operation.

[0065] Step S2, Text Parsing:

[0066] The text description optimizes the input through prompt engineering, using the following prompt template: "Extract the layer types, parameters, and training requirements of the neural network from the following description, unify the terms, and complete the missing information: {user description}." The large language model (based on the CodeLLaMA architecture) decomposes the description and identifies the key information: The backbone network is ResNet-50, the convolutional kernel size is 3×3, and the activation function is ReLU; FPN includes multi-scale feature fusion; the detection head outputs three scales. For the problem of inconsistent terms (such as "convolution" and "Conv"), the model standardizes the expression to "Convolution" through semantic normalization. For missing parameters (such as the stride of the pooling layer), the model completes the default value according to the common practice of the YOLO model (the stride is 2). The final output is a structured semantic feature vector, including the layer sequence, parameters (convolutional kernel size, stride, number of filters), and training requirements (PyTorch or MindSpore deployment).

[0067] Step S3, Multimodal Fusion:

[0068] The visual feature vector (representing the topological structure) and the semantic feature vector (representing the layer parameters and training requirements) are fused through a multimodal alignment mechanism. The system first projects the two features into a unified 512-dimensional feature space and uses the multi-head self-attention mechanism to calculate the cross-modal correlation. For example, the FPN fusion node in the visual feature is associated with the "multi-scale feature fusion" description in the semantic feature through the attention weight. Learnable position encoding is introduced to record the spatial order of the grid regions to ensure that the topological information of the branch structure is not lost. Residual connections preserve the details of the original visual features (such as the number of filters in the handwritten annotations), and layer normalization adjusts the feature distribution to improve the fusion stability. The final output is a high-dimensional model feature representation that fully describes the topology and parameter configuration of the ResNet-50 backbone, FPN structure, and detection head, as well as the hardware adaptation requirements.

[0069] Step S4, Code Generation:

[0070] The large language model receives the fused features and the user-specified framework and generates code through prompt engineering. The prompt template is: "Generate {framework} code based on the following model features, adapted to {hardware}, including the complete network structure and parameters: {fused features}." After the code is generated, the system verifies the code consistency through syntax checking and detects the tensor dimension matching through logical reasoning. If an error is found (such as the output dimension of the detection head does not match), the model adjusts and regenerates according to the context.

[0071] Step S5, User Interaction and Optimization:

[0072] The user views the generated code through a graphical interface and submits modification requirements in the interaction module, such as: "Increase the number of detection heads from 3 to 4 to support finer-grained detection." The user interaction module parses the natural language instructions, updates the detection head configuration in the fused features (changing from 3 scales to 4), and triggers the code generation module to regenerate the code. The system also provides visual feedback, displays the modified network topology diagram, and verifies the syntactic correctness (checking function parameters) and logical consistency (verifying tensor dimensions) of the new code through static analysis.

Claims

1. A method for neural network graphic and text parsing and cross-frame code generation, characterized in that The method includes the following steps: Step S1: Use a vision large model to perform visual parsing on the input neural network architecture image, extract the network structure features in the image, including the type of each layer, connection mode, and overall topological layout, and generate a structured visual feature vector. Step S2: Use a large language model to perform semantic understanding on the input text description, extract the key information in the description, and convert non-standard expressions into a standardized semantic representation. Step S3: Through a multimodal alignment mechanism, fuse the visual feature vector extracted by the vision large model with the semantic representation generated by the large language model to generate a unified neural network model feature representation. Step S4: Utilize the code generation ability of the large language model to directly generate executable code that supports the target framework from the fused model feature representation according to the target framework specified by the user.

2. The neural network graphic and text parsing and cross-frame code generation method according to claim 1, characterized in that The specific steps of step S1 are as follows: Step S11: First, preprocess the input neural network architecture image, including denoising and edge enhancement, to highlight the network layer units and connection line features. Step S12: The vision large model extracts features from the neural network architecture image through its vision encoder. The specific steps are as follows: The vision large model divides the image into multiple regions, identifies the network layer units in each region, and infers the inter-layer connection relationship through the pre-trained image-text contrast learning ability, and outputs a feature vector containing position information and semantic labels.

3. The neural network graphic and text parsing and cross-frame code generation method according to claim 1, characterized in that The specific steps of step S2 are as follows: Step S21: Optimize the input form through prompt engineering: Design a predefined prompt template to guide the large language model to accurately understand the text content. Step S22: The large language model decomposes the description and identifies key components: The large language model uses its natural language understanding ability to input the description text through prompt engineering, and extracts the layer type, parameter configuration, and inter-layer order. Step S23: To deal with non-standard expressions, use a semantic normalization mechanism to unify different expressions into a standard format through pre-trained language knowledge. If necessary information is missing in the description, complete the default value according to the common practice in the field of deep learning. Step S24: The large language model outputs a structured semantic feature vector, including the layer configuration and parameter details of the network.

4. The method for neural network graphic and text parsing and cross-frame code generation according to claim 1, wherein The specific steps of step S3 are as follows: Step S31: Use projection to align the visual feature vector of the vision large model with the semantic representation of the large language model, and calculate the cross-modal correlation using the multi-head self-attention mechanism. Step S32: To retain the spatial information of the sequence, introduce learnable position encoding to record the spatial order of the feature sequence. Step S33: To improve the fusion effect, introduce residual connections to retain part of the original feature sequence and avoid information compression loss; at the same time, use layer normalization technology to adjust the feature distribution and enhance stability. Step S34: Output a unified neural network model feature representation, stored in the form of a high-dimensional vector, including the complete topological structure and parameter configuration of the network.

5. The neural network graphic and text parsing and cross-frame code generation method according to claim 1, characterized in that The specific steps of step S4 are as follows: Step S41: The large language model receives the fused features and generates target framework code through prompt engineering. Step S42: To optimize the generation result, syntax checking is used to verify the code syntax consistency. If a logical error is detected, it is adjusted and regenerated to ensure the code's executability.

6. A neural network graphic and text parsing and cross-frame code generation system for implementing the method according to any one of claims 1-5, characterized in that The system includes an image parsing module, a text processing module, a multimodal fusion module, a code generation module, and a user interaction module, where: The image parsing module receives neural network architecture pictures based on a vision large model and extracts visual features. The text processing module parses text descriptions based on a large language model and generates a standardized semantic representation. The multimodal fusion module fuses image and text information through an attention mechanism to form a unified neural network model feature representation. The code generation module directly generates deep learning code in the user-specified framework from the fused features using a large language model. The user interaction module provides a graphical interface and natural language input function, supporting users to modify and verify the code in real time.

7. The neural network graphic and text parsing and cross-frame code generation system according to claim 6, wherein The image parsing module, text processing module, multimodal fusion module, code generation module, and user interaction module are interconnected through a standardized interface RESTAPI.

8. The neural network graphic and text parsing and cross-frame code generation system according to claim 6, wherein The user interaction module integrates a code verification function, checks the syntax correctness through static analysis, and detects the tensor dimension matching through logical reasoning.

Images