Code file processing method and device based on large language model and electronic equipment

Abstract

This application provides a method, apparatus, and electronic device for processing code files based on a large language model. The method includes: acquiring multiple code files to be processed; analyzing and processing the code files according to a preset first tag encoding system to generate a first tag encoding corresponding to the code file; extracting first semantic element information from the code files according to a preset first semantic element extraction specification; generating a first index entry corresponding to the code file based on the first tag encoding and the first semantic element information; and inputting the first index entry into a large language model for analysis and processing to obtain auxiliary content of the code file. This method aims to extract and compress the key features of the code content of the code file, enabling the large language model to recognize the key features of the full code content in the code file.

Description

Technical Field

[0001] This application relates to the fields of artificial intelligence technology and software engineering technology, and in particular to a method, apparatus and electronic device for processing code files based on a large language model. Background Technology

[0002] With the rapid development of artificial intelligence technology, large language models are being widely used in software development assistance. Large language models can understand natural language and code, assisting developers in tasks such as code generation, code review, and problem localization.

[0003] In some technologies, due to the large amount of code in the code files during the software development process, large language models are unable to analyze and process the full amount of code content in the code files during auxiliary tasks.

[0004] Therefore, there is an urgent need for a solution that can extract and compress the key features of the code content in a code file. Summary of the Invention

[0005] The method, apparatus, and electronic device for processing code files based on a large language model provided in this application are used to extract and compress the key features of the code content in the code file, so that the large language model can recognize the key features of the full code content in the code file.

[0006] In a first aspect, embodiments of this application provide a method for processing code files based on a large language model, including:

[0007] Retrieve multiple code files to be processed;

[0008] According to the preset first tag encoding system, the code file is analyzed and processed to generate the first tag encoding corresponding to the code file; wherein, the first tag encoding system is used to indicate the value of the first tag encoding corresponding to different features of the code file;

[0009] According to the preset first semantic element extraction specification, the first semantic element information of the code file is extracted; the first semantic element information is used to indicate at least one content element of the code file.

[0010] Based on the first tag code and the first semantic element information, a first index entry corresponding to the code file is generated; wherein, the first index entry includes: the file identifier of the code file, the first tag code corresponding to the code file, and the first semantic element information corresponding to the code file;

[0011] The first index entry is input into the large language model for analysis and processing to obtain auxiliary content for the code file.

[0012] In one possible implementation, the code file is analyzed and processed according to a preset first tag encoding system to generate the first tag encoding corresponding to the code file, including:

[0013] The first tag encoding system is input into the large language model; and the code files are input into the large language model in batches; wherein each batch includes a preset number of code files;

[0014] Based on the large language model, and according to the first tag encoding system, the code files in each batch are analyzed and processed sequentially to obtain the first tag encoding corresponding to the code file.

[0015] In one possible implementation, the first semantic element information of the code file is extracted according to a preset first semantic element extraction specification, including:

[0016] The first semantic element extraction specification is input into the large language model;

[0017] Based on the large language model, and according to the first semantic element extraction specification, semantic elements are extracted from the code files in each batch in sequence to obtain the first semantic element information of the code files.

[0018] In one possible implementation, the first label encoding includes label encoding in at least one of the following dimensions: hierarchy dimension, module dimension, first importance dimension, feature dimension, and scale dimension.

[0019] Among them, the hierarchy dimension is used to indicate the technical architecture level to which the code file belongs; the module dimension is used to indicate the functional module to which the code file belongs; the first importance dimension is used to indicate the importance of the code file; the feature dimension is used to indicate the technical implementation features of the code file; and the scale dimension is used to indicate the code size of the code file.

[0020] In one possible implementation, when the importance of the code file is at the core level, the first tag encoding of the code file takes a value of 9 in the first importance dimension;

[0021] When the importance level of the code file is high frequency, the first tag encoding of the code file takes a value of 8 in the first importance dimension;

[0022] When the importance of the code file is at the business level, the first tag of the code file is encoded with a value of 7 in the first importance dimension;

[0023] When the importance level of the code file is normal, the first tag encoding of the code file takes a value of 5 in the first importance dimension;

[0024] When the importance level of the code file is auxiliary, the first tag encoding of the code file takes a value of 3 in the first importance dimension;

[0025] When the importance level of a code file is marginal, the first tag encoding of the code file takes a value of 1 in the first importance dimension.

[0026] In one possible implementation, at least one content element includes: a functional element, a dependency element, an interface element, and a descriptive element.

[0027] Among them, the functional element is used to indicate the functional responsibilities of the code file; the dependency element is used to indicate other modules or other libraries that the code file depends on; the interface element is used to indicate the interface path of the code file; and the description element is used to indicate the technical characteristics of the code file.

[0028] In one possible implementation, the first index entry is input into a large language model for analysis and processing to obtain auxiliary content for the code file, including:

[0029] An index document is created by combining the first index entries of multiple code files.

[0030] The indexed documents are input into a large language model for analysis and processing to obtain auxiliary content for the code files.

[0031] In one possible implementation, the method further includes:

[0032] In response to code file modification operations, a new index entry corresponding to the modified code file is generated based on the large language model.

[0033] The indexed documents are updated based on the new index entries.

[0034] In one possible implementation, the code file is stored in the codebase to be processed; the method further includes:

[0035] If the codebase to be processed includes data tables, retrieve multiple data tables from the codebase to be processed;

[0036] Based on the preset second label coding system, the data table is analyzed and processed to generate the second label code corresponding to the data table; wherein, the second label coding system is used to indicate the value of the second label code corresponding to different features of the data table;

[0037] According to the preset second semantic element extraction specification, extract the second semantic element information of the data table; the second semantic element information is used to indicate at least one table structure element of the data table;

[0038] Based on the second tag code and the second semantic element information, a second index entry corresponding to the data table is generated; wherein, the second index entry includes: the data table identifier of the data table, the second tag code corresponding to the data table, and the second semantic element information corresponding to the data table;

[0039] Integrate the second index entries from multiple data tables into the index document.

[0040] In one possible implementation, the second tag encoding includes tag encoding of at least one of the following dimensions: table type dimension, table characteristic dimension, and second importance dimension;

[0041] Among them, the table type dimension is used to indicate the business type of the data table; the table characteristic dimension is used to indicate the technical characteristics of the data table; and the second importance dimension is used to indicate the importance of the data table.

[0042] In one possible implementation, the method further includes:

[0043] The first tag is encoded and embedded in the filename of the code file to generate a semantic filename for the code file.

[0044] Secondly, embodiments of this application provide a processing apparatus for code files based on a large language model, comprising:

[0045] The acquisition module is used to acquire multiple code files to be processed.

[0046] The processing module is used to analyze and process the code file according to the preset first tag encoding system, and generate the first tag encoding corresponding to the code file; wherein, the first tag encoding system is used to indicate the value of the first tag encoding corresponding to different features of the code file;

[0047] The processing module is also used to extract the first semantic element information of the code file according to the preset first semantic element extraction specification; the first semantic element information is used to indicate at least one content element of the code file;

[0048] The processing module is further configured to generate a first index entry corresponding to the code file based on the first tag code and the first semantic element information; wherein the first index entry includes: the file identifier of the code file, the first tag code corresponding to the code file, and the first semantic element information corresponding to the code file;

[0049] The processing module is also used to input the first index entry into the large language model for analysis and processing, and to obtain auxiliary content for the code file.

[0050] In one possible implementation, the code file is analyzed and processed according to a preset first tag encoding system to generate the first tag encoding corresponding to the code file. The processing module is used for:

[0051] The first tag encoding system is input into the large language model; and the code files are input into the large language model in batches; wherein each batch includes a preset number of code files;

[0052] Based on the large language model, and according to the first tag encoding system, the code files in each batch are analyzed and processed sequentially to obtain the first tag encoding corresponding to the code file.

[0053] In one possible implementation, the first semantic element information of the code file is extracted according to a preset first semantic element extraction specification, and the processing module is used for:

[0054] The first semantic element extraction specification is input into the large language model;

[0055] Based on the large language model, and according to the first semantic element extraction specification, semantic elements are extracted from the code files in each batch in sequence to obtain the first semantic element information of the code files.

[0056] In one possible implementation, the first label encoding includes label encoding in at least one of the following dimensions: hierarchy dimension, module dimension, first importance dimension, feature dimension, and scale dimension.

[0057] Among them, the hierarchy dimension is used to indicate the technical architecture level to which the code file belongs; the module dimension is used to indicate the functional module to which the code file belongs; the first importance dimension is used to indicate the importance of the code file; the feature dimension is used to indicate the technical implementation features of the code file; and the scale dimension is used to indicate the code size of the code file.

[0058] In one possible implementation, when the importance of the code file is at the core level, the first tag encoding of the code file takes a value of 9 in the first importance dimension;

[0059] When the importance level of the code file is high frequency, the first tag encoding of the code file takes a value of 8 in the first importance dimension;

[0060] When the importance of the code file is at the business level, the first tag of the code file is encoded with a value of 7 in the first importance dimension;

[0061] When the importance level of the code file is normal, the first tag encoding of the code file takes a value of 5 in the first importance dimension;

[0062] When the importance level of the code file is auxiliary, the first tag encoding of the code file takes a value of 3 in the first importance dimension;

[0063] When the importance level of a code file is marginal, the first tag encoding of the code file takes a value of 1 in the first importance dimension.

[0064] In one possible implementation, at least one content element includes: a functional element, a dependency element, an interface element, and a descriptive element.

[0065] Among them, the functional element is used to indicate the functional responsibilities of the code file; the dependency element is used to indicate other modules or other libraries that the code file depends on; the interface element is used to indicate the interface path of the code file; and the description element is used to indicate the technical characteristics of the code file.

[0066] In one possible implementation, the first index entry is input into a large language model for analysis and processing to obtain auxiliary content for the code file. The processing module is used for:

[0067] An index document is created by combining the first index entries of multiple code files.

[0068] The indexed documents are input into a large language model for analysis and processing to obtain auxiliary content for the code files.

[0069] In one possible implementation, the processing module is further configured to:

[0070] In response to code file modification operations, a new index entry corresponding to the modified code file is generated based on the large language model.

[0071] The indexed documents are updated based on the new index entries.

[0072] In one possible implementation, the code file is stored in the code library to be processed; the acquisition module is further configured to acquire multiple data tables in the code library to be processed if the code library to be processed includes data tables.

[0073] The processing module is also used to analyze and process the data table according to the preset second label coding system, and generate the second label code corresponding to the data table; wherein, the second label coding system is used to indicate the value of the second label code corresponding to different features of the data table;

[0074] The processing module is also used to extract the second semantic element information of the data table according to the preset second semantic element extraction specification; the second semantic element information is used to indicate at least one table structure element of the data table;

[0075] The processing module is also used to generate a second index entry corresponding to the data table based on the second tag code and the second semantic element information; wherein, the second index entry includes: the data table identifier of the data table, the second tag code corresponding to the data table, and the second semantic element information corresponding to the data table;

[0076] The processing module is also used to integrate second index entries from multiple data tables into an index document.

[0077] In one possible implementation, the second tag encoding includes tag encoding of at least one of the following dimensions: table type dimension, table characteristic dimension, and second importance dimension;

[0078] Among them, the table type dimension is used to indicate the business type of the data table; the table characteristic dimension is used to indicate the technical characteristics of the data table; and the second importance dimension is used to indicate the importance of the data table.

[0079] In one possible implementation, the processing module is further configured to:

[0080] The first tag is encoded and embedded in the filename of the code file to generate a semantic filename for the code file.

[0081] Thirdly, embodiments of this application provide an electronic device, including: a memory and a processor;

[0082] The memory stores instructions that the computer executes;

[0083] The processor executes computer execution instructions stored in memory, causing the processor to perform the first aspect and / or various possible implementations of the first aspect as described above.

[0084] Fourthly, embodiments of this application provide a computer-readable storage medium storing computer-executable instructions, which, when executed by a processor, are used to implement the first aspect and / or various possible implementations of the first aspect.

[0085] Fifthly, embodiments of this application provide a computer program product, including a computer program that, when executed by a processor, implements the first aspect and / or various possible implementations of the first aspect.

[0086] The code file processing method, apparatus, and electronic device based on a large language model provided in this application process the code file to be processed according to a preset encoding system and semantic element extraction specifications to obtain the tag encoding and semantic elements of the code file. Based on the tag encoding and semantic elements, index entries are constructed. The index entries are then analyzed and processed based on the large language model to obtain auxiliary content of the code file. This allows for the extraction and compression of intermediate features of the code content in the code file, enabling the large language model to identify the key features of the entire code content and output more accurate auxiliary content. Attached Figure Description

[0087] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments consistent with this application and, together with the description, serve to explain the principles of this application.

[0088] Figure 1 A flowchart illustrating the code file processing method based on a large language model provided in this application. Figure 1 ;

[0089] Figure 2 A flowchart illustrating the code file processing method based on a large language model provided in this application. Figure 2 ;

[0090] Figure 3 A flowchart illustrating the code file processing method based on a large language model provided in this application. Figure 3 ;

[0091] Figure 4 A schematic diagram illustrating the encoding of the first tag in an exemplary authentication controller file;

[0092] Figure 5 A flowchart illustrating the code file processing method based on a large language model provided in this application. Figure 4 ;

[0093] Figure 6 This diagram illustrates the relationship between indexed documents and the attention mechanism of the large model.

[0094] Figure 7 Here is a flowchart illustrating the processing of an example code file;

[0095] Figure 8 Here are flowcharts illustrating different processing modes;

[0096] Figure 9 This is an example application hierarchy diagram;

[0097] Figure 10 A schematic diagram of the structure of the code file processing device based on a large language model provided in this application;

[0098] Figure 11 A schematic diagram of the processing flow of the code file processing device based on a large language model provided in this application;

[0099] Figure 12 A schematic diagram of the structure of the electronic device provided in this application.

[0100] The accompanying drawings have illustrated specific embodiments of this application, which will be described in more detail below. These drawings and descriptions are not intended to limit the scope of the concept in any way, but rather to illustrate the concept of this application to those skilled in the art through reference to specific embodiments. Detailed Implementation

[0101] Exemplary embodiments will now be described in detail, examples of which are illustrated in the accompanying drawings. When the following description relates to the drawings, unless otherwise indicated, the same numbers in different drawings denote the same or similar elements. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with this application. Rather, they are merely examples of apparatuses and methods consistent with some aspects of this application as detailed in the appended claims.

[0102] First, let me explain the terms used in this application:

[0103] Large Language Model (LLM): This refers to an artificial intelligence model based on the Transformer architecture, which achieves parallel processing of input sequences through a self-attention mechanism. The computational complexity of the Transformer architecture is O(n log n). , where n is the length of the input sequence, a characteristic that determines the hard technical constraint of the context window capacity of the model.

[0104] Context Window: This refers to the maximum length of a text sequence that a large language model can receive and process during a single inference iteration, measured in the number of tokens. This is a technical constraint determined by the computational complexity and hardware resources of the Transformer architecture.

[0105] Attention mechanism: This refers to the core computational component of the Transformer architecture, which determines the association weights between positions in a sequence by calculating the product of the Query, Key, and Value matrices. Multi-head attention allows the model to focus on input information from different dimensions simultaneously.

[0106] Vector embedding is a technique that converts text or code into high-dimensional numerical vectors. Embedded vectors capture semantic information but are not readable by humans.

[0107] Retrieval-Augmented Generation (RAG) is a technical architecture that combines information retrieval and text generation, where relevant content is first retrieved and then input into a large language model to generate the text.

[0108] Codebase: refers to the collection of all source code files and related technical assets in a software project. It is a data collection with specific technical meaning in the field of software engineering technology.

[0109] Index document: refers to the structured text file generated by this application, which contains multi-dimensional label encoding and semantic element description of the code file, and can be directly used as the input context of a large language model.

[0110] AOCI (AI-Oriented Code Indexing): refers to the code indexing method for artificial intelligence proposed in this application. It achieves semantic compression of the code library by generating independent index documents, with an information retention rate of about 65% and a compression ratio of about 500:1.

[0111] SEFS (Self-Explanatory File System): refers to the self-explanatory file system method proposed in this application, which achieves semantic naming by embedding tag encoding into filenames, with an information retention rate of approximately 15% to 20% and a compression ratio higher than AOCI.

[0112] Single file independence: This refers to the core feature of index generation in this application. The index entries for each code file can be generated independently. The large language model only needs to read the contents of this file to complete the tag encoding determination and semantic element information extraction without reading other files, supporting parallel processing and incremental updates.

[0113] With the rapid development of artificial intelligence technology, large language models are being applied more and more widely in various fields. For example, in the field of software development assistance, large language models, based on the Transformer architecture, can understand the code content of code files. Specifically, they can assist developers in completing tasks such as code generation, code review, and problem localization.

[0114] Large language models are built on the Transformer architecture, and the computational complexity of their self-attention mechanism is O(n log n). Where n is the length of the input sequence. This technical characteristic determines that large language models have a hard constraint on the context window capacity, that is, the maximum length of the text sequence that can be received and processed in a single inference is limited. In modern software development projects, the number of code files is huge, and the size of the code content in the code files is also large, which may exceed the context window capacity of large language models.

[0115] For example, a typical medium-sized project may contain hundreds of code files and tens of thousands to hundreds of thousands of lines of code, and its tokens may number in the hundreds of thousands or even millions, far exceeding the context window capacity of most large language models.

[0116] Due to the limitations of the context window in large language models, they are unable to analyze and process the full content of the code file during auxiliary tasks.

[0117] In some embodiments, a vector embedding and retrieval-enhanced generation (RAG) technical architecture is employed to address this issue. Specifically, the code file is segmented and converted into high-dimensional numerical vectors, which are then stored in a vector database. When a user queries, relevant code blocks are found through similarity retrieval, and the retrieved original code fragments are used as context input to a large language model.

[0118] However, the above embodiments also have the following technical drawbacks: First, vector embeddings are uninterpretable numerical values, and developers cannot verify what information the large language model "sees"; second, the retrieval results are local code fragments, making it difficult for the large language model to obtain a global view of the system; third, all code is treated equally in the vector space, and the large language model cannot distinguish between core modules and peripheral functions; fourth, the code index and the database index are independent of each other, making it difficult to establish a cognitive association between code and data models; fifth, modifications to a single code file may affect the global vector representation, leading to the need to rebuild the index.

[0119] The code file processing method based on a large language model provided in this application processes the code file to be processed according to a preset encoding system and semantic element extraction specifications to obtain the tag encoding and semantic elements of the code file. Based on the tag encoding and semantic elements, index entries are constructed. The large language model then analyzes and processes the index entries to obtain auxiliary content for the code file. On the one hand, this method can extract and compress intermediate features of the code content in the code file, enabling the large language model to identify the key features of the entire code content and output more accurate auxiliary content. On the other hand, by using index entries composed of tag encoding and semantic elements, which have better user readability, the key features of the code file are extracted and compressed, avoiding the problem of poor readability of code file features caused by vector embedding.

[0120] The technical solution of this application and how it solves the above-mentioned technical problems will be described in detail below with specific embodiments. These specific embodiments can be combined with each other, and the same or similar concepts or processes may not be described again in some embodiments. The embodiments of this application will be described below with reference to the accompanying drawings.

[0121] Figure 1 A flowchart illustrating the code file processing method based on a large language model provided in this application. Figure 1 ,like Figure 1 As shown, the method includes:

[0122] Step 101. Obtain multiple code files to be processed.

[0123] For example, the code files are stored in the code repository to be processed. Multiple code files to be processed in the code repository are retrieved via system commands. Optionally, the directory structure of each code file can be obtained.

[0124] Specifically, for code files in the code repository to be processed, the complete directory structure of the code repository can be obtained through system commands (such as tree, find, etc.), including the path, file name and file type of each code file.

[0125] For example, a typical web application project might contain the following structure: the backend code directory contains subdirectories such as controllers, services, models, and middleware; the frontend code directory contains subdirectories such as components, views, and store. This directory information allows us to identify multiple code files within the codebase to be processed.

[0126] Step 102. Analyze and process the code file according to the preset first tag encoding system to generate the first tag encoding corresponding to the code file.

[0127] The first tag encoding system is used to indicate the value of the first tag encoding corresponding to different features of the code file.

[0128] For example, the first-label encoding system is pre-designed for the parallel processing characteristics of the Transformer multi-head attention mechanism in large language models. Its design principles include: multiple independent dimensions can be simultaneously attended to by different attention heads of the Transformer; the first-label encoding uses short encoding to minimize token consumption; digit importance can participate in attention weight calculation; a unified encoding format facilitates pattern matching by the attention mechanism; and the number and content of dimensions can be expanded according to needs.

[0129] The first tag encoding system can represent the values ​​of the first tag codes corresponding to different characteristics of a code file. For example, the first tag encoding system can represent the characteristics of a code file in different dimensions. For code files of different levels, different letters are used to represent the first character of the first tag code; for code files of different importance, different numbers are used to represent the second character of the first tag code. Based on the characteristics of the code file itself (such as level and importance), the first tag code corresponding to the code file is determined according to this first tag encoding system.

[0130] It should be noted that the above explanation of the first tag encoding system is not intended to limit its dimensions. On the contrary, the first tag encoding system can have first tag encoding matching rules with multiple dimensions, and the dimensions of the first tag encoding can be expanded.

[0131] Step 103. Extract the first semantic element information of the code file according to the preset first semantic element extraction specification.

[0132] The first semantic element information is used to indicate at least one content element of the code file.

[0133] For example, the first semantic element information is used to indicate elements of at least one of the following aspects of the code file: functional elements (F), dependency elements (R), interface elements (A), and descriptive elements (S).

[0134] Based on the content of the code file and the different results corresponding to the above aspects, the first semantic element information is determined. For example, the content corresponding to the interface element of a code file is " / api / chat / stream", which is used to represent the interface path corresponding to the code file. As another example, the content corresponding to the function element of a code file is "streaming dialogue service", which is used to represent the code content in the code file to implement the function of streaming dialogue service.

[0135] Step 104. Generate the first index entry corresponding to the code file based on the first tag encoding and the first semantic element information.

[0136] The first index entry includes: the file identifier of the code file, the first tag encoding corresponding to the code file, and the first semantic element information corresponding to the code file.

[0137] For example, a first index entry is generated for each code file based on its first tag code and first semantic element information. The first index entry includes not only the first tag code and first semantic element information for that code file, but also the file identifier of that code file. The file identifier is used to uniquely represent the code file.

[0138] Specifically, the format of the first index entry can be: "Filename [First Tag Encoding]: F: Function Description | R: Dependency | A: Interface Path | S: Feature Summary". Here, different content elements in the first semantic element information use "|" as a separator to facilitate pattern recognition by the Transformer attention mechanism. The filename is the file identifier for the code file.

[0139] For example, the first index entry of an authentication controller might be: "AuthController.js[CCA8RL]:F:JWT authentication control| R:authService,userModel,bcrypt | A: / api / auth / | S: Login / Register, Token Refresh, Permission Verification.

[0140] Step 105. Input the first index entry into the large language model for analysis and processing to obtain auxiliary content for the code file.

[0141] For example, the first index entry of the code file is input into a large language model for analysis and processing to obtain auxiliary content for the code file. This auxiliary content can be the processing result corresponding to an auxiliary task for that code file.

[0142] For example, if the auxiliary task for this code file is impact analysis, then the auxiliary content of the code file output by the large language model is the impact analysis report of this code file.

[0143] For example, if the auxiliary task for this code file is code review, then the auxiliary content output by the large language model is the code review report for that code file.

[0144] Optionally, when inputting the first index entry into the large language model for analysis and processing, prompt words can also be input. These prompt words indicate the type of auxiliary task for the code file. By inputting prompt words, the auxiliary content generated by the large language model can be constrained to match the task objective indicated by the auxiliary task.

[0145] Optionally, auxiliary tasks may also include one or more of the following: problem identification, feature development, data modeling, and architecture understanding.

[0146] The code file processing method based on a large language model provided in this application processes the code file to be processed according to a preset first tag encoding system and a first semantic element extraction specification to obtain the first tag encoding and first semantic element information of the code file. Based on the first tag encoding and first semantic element information, a first index entry is constructed. The first index entry is then analyzed and processed based on the large language model to obtain auxiliary content of the code file. On the one hand, this method can extract and compress intermediate features of the code content in the code file, enabling the large language model to identify the key features of the entire code content and output more accurate auxiliary content. On the other hand, by using the first index entry composed of the first tag encoding and first semantic element information, which has better user readability, the key features of the code file are extracted and compressed, avoiding the problem of poor readability of code file features caused by vector embedding.

[0147] Figure 2 A flowchart illustrating the code file processing method based on a large language model provided in this application. Figure 2 ,like Figure 2 As shown, in this embodiment... Figure 1 Based on the embodiments, the process of generating the first tag encoding corresponding to the code file in step 102 above is described in detail. The method includes:

[0148] Step 201. Input the first tag encoding system into the large language model; and input the code files into the large language model in batches.

[0149] Each batch includes a preset number of code files.

[0150] For example, firstly, the definition of the first tag encoding system is input into the large language model. Secondly, multiple code files in the code library to be processed are divided into different batches, with each batch containing a preset number of code files. The preset number can be set to 5 to 10.

[0151] The purpose of batch processing is to control the context consumption of a single large language model call, ensuring that the large language model can completely read and analyze the contents of each batch of code files.

[0152] Specifically, the following are examples of instructions for reading code files in batches:

[0153] "==Batch file viewing command - Batch 1=="

[0154] cat / var / www / project / backend / src / controllers / AuthController.js

[0155] cat / var / www / project / backend / src / controllers / ChatController.js

[0156] cat / var / www / project / backend / src / services / authService.js

[0157] cat / var / www / project / backend / src / services / chatService.js

[0158] cat / var / www / project / backend / src / models / userModel.js

[0159] ==First batch of files viewed successfully=="

[0160] Step 202. Based on the large language model, and according to the first tag encoding system, the code files in each batch are analyzed and processed sequentially to obtain the first tag encoding corresponding to the code file.

[0161] For example, for each batch of code files, the large language model analyzes and processes the code files in each batch according to the input first tag encoding system to obtain the first tag encoding of the code files contained in each batch.

[0162] The analysis of each code file is performed independently. The large language model only needs to read the contents of the code file to complete the determination of the first tag encoding, without having to read other files.

[0163] In the above embodiments, by using a batch input mechanism for code files, the large language model can efficiently generate the first tag encoding of the code files within a limited context window. This reduces the computational processing pressure on the large language model and also enables parallel processing, thereby improving overall generation efficiency. It also improves the adaptability of the first tag encoding system.

[0164] Furthermore, based on the above embodiments, the first semantic element information of the code file is also obtained by processing the code file in batches according to the extraction specifications based on the large language model.

[0165] Figure 3 A flowchart illustrating the code file processing method based on a large language model provided in this application. Figure 3 ,like Figure 3 As shown, the process of generating the first semantic element information corresponding to the code file in step 103 above is explained in detail. The method includes:

[0166] Step 301. Input the first semantic element extraction specification into the large language model.

[0167] For example, the first semantic element extraction specification is input into the large language model. In conjunction with the aforementioned embodiments, the first tag encoding and the first semantic element extraction specification can be jointly input into the large language model as the first index specification.

[0168] For example, the format of the first index specification is as follows:

[0169] Index format: Filename [Label]: F: Function | R: Associated File | A: API Path | S: Description

[0170] Tagging system [ABCDE]:

[0171] A-Tier: F - Frontend; B - Backend; S - Service; C - Controller; M - Middleware; D - Database; U - Utilities; R - Router

[0172] Module B: Core A Authentication T Conversation R Points K Knowledge S Storage P Payment N Notification O Other

[0173] C-Importance: 9 cores, 8 high-frequency cores, 7 service cores, 5 routine cores, 3 auxiliary cores, 1 edge core.

[0174] D-Characteristics: P - Billing; A - Asynchronous; T - Transactions; E - External API; R - Permissions; C - Configuration; B - Bugs; H - Caching

[0175] File Size: L Large (>600 lines) M Medium (200-600 lines) S Small (<200 lines)

[0176] Note: The description should not exceed 50 Chinese characters.

[0177] Step 302. Based on the large language model, according to the first semantic element extraction specification, semantic elements are extracted from the code files in each batch in sequence to obtain the first semantic element information of the code files.

[0178] For example, as can be seen from the foregoing embodiments, multiple code files in the code library to be processed have been divided into different batches and input into the large language model. Then, for the code files in each batch, semantic elements are extracted based on the first semantic element specification, thereby obtaining the first semantic element information of the code files contained in each batch.

[0179] As can be seen from the foregoing embodiments, the large language model performs the following analysis on each code file:

[0180] (1) Hierarchical analysis: Determine the architecture hierarchy based on file paths, import statements, class or function definitions;

[0181] (2) Module analysis: Based on file function, namespace, and business logic, determine the functional modules;

[0182] (3) Importance assessment: The importance level is assessed based on the number of times the code is depended on, the criticality of the function, and the complexity of the code.

[0183] (4) Feature recognition: Identify technical features such as asynchronous processing, transaction control, and external calls;

[0184] (5) Scale statistics: The number of lines of code is used to determine the scale label;

[0185] (6) Extracting elements: extract the four elements of function, dependency, interface and characteristics.

[0186] Among them, the processes (1) to (5) above belong to the process of obtaining the first tag encoding of the code file in steps 201 to 202; the process (6) above belongs to the process of obtaining the first semantic element information of the code file in steps 301 to 302.

[0187] Optionally, based on the above analysis results, the large language model outputs the first tag encoding and first semantic element information corresponding to each code file. An example output of the large language model is as follows:

[0188] "==Batch 1 completed, output as follows=="

[0189] ===Directory / var / www / project / backend / src / controllers===

[0190] AuthController.js[CCA8RL]: F: JWT authentication control | R: authService, userModel, bcrypt | A: / api / auth / S: Login / Register, Token Refresh, Permission Verification

[0191] ChatController.js[CCT9APL]: F: Dialogue controller core | R: chatService,aiService | A: / api / chat / | S: Dual-mode streaming / non-streaming, pre-deducted points, message caching

[0192] ===Directory / var / www / project / backend / src / services===

[0193] authService.js [SAC9TM]: F: Authentication service core | R: JWT, bcrypt, redis | A: None | S: JWT generation and verification, password encryption, token refresh

[0194] chatService.js[SCT9APEL]: F: Dialogue service core | R: OpenAI, Anthropic, Redis | A: None | S: Multi-model adaptation, streaming response, context management

[0195] ===Directory / var / www / project / backend / src / models===

[0196] userModel.js[DCA8M]: F: User Data Model | R: Sequelize, bcrypt | A: None | S: User CRUD, Password Encryption, Relationship Definition

[0197] Progress: Of the total 85 core files, 5 have been completed.

[0198] ==Batch 1 document summary output completed=="

[0199] In the above embodiments, by using a batch input mechanism for code files, the large language model can efficiently generate the first semantic element information of the code files within a limited context window. Combined with the aforementioned exemplary embodiments, this enables the generation of the first tag encoding and the first semantic element information of the code files. This lays the foundation for subsequently constructing the first index entries representing the key characteristics of the code files.

[0200] In one possible implementation, the first tag encoding system enables the specification of multi-dimensional tag encoding for code files. Furthermore, the first tag encoding of the code file is also a multi-dimensional tag encoding.

[0201] In one example, the first label encoding includes label encoding in at least one of the following dimensions: hierarchy dimension, module dimension, first importance dimension, feature dimension, and scale dimension.

[0202] Among them, the hierarchy dimension is used to indicate the technical architecture level to which the code file belongs; the module dimension is used to indicate the functional module to which the code file belongs; the first importance dimension is used to indicate the importance of the code file; the feature dimension is used to indicate the technical implementation features of the code file; and the scale dimension is used to indicate the code size of the code file.

[0203] For example, the dimensions in the first tag encoding are independent of each other.

[0204] Among them, at least one dimension includes: hierarchy dimension (A dimension), module dimension (B dimension), first importance dimension (C dimension), feature dimension (D dimension), and scale dimension (E dimension).

[0205] The hierarchical dimension is used to identify the technical architecture layer to which the code file belongs. Specifically, B represents the backend layer, F represents the frontend layer, S represents the service layer, C represents the controller layer, M represents the middleware layer, D represents the database / model layer, U represents the utility layer, R represents the router layer, T represents the test layer, and I represents the infrastructure layer.

[0206] The module dimension is used to identify the functional module to which a code file belongs. Specifically, C represents the core module, A represents the authentication module, T represents the talk / chat module, R represents the credits module, K represents the knowledge module, P represents the payment module, N represents the notification module, and O represents the other module. Module tags can be customized according to specific business domains.

[0207] The first importance dimension is used to identify the importance of code files and is represented by a numerical label. See the example below for specific values.

[0208] Feature dimensions are used to identify the technical implementation characteristics of code files, and multiple combinations can be selected. Specifically, P indicates that it involves billing processing (Payment), A indicates that it involves asynchronous processing (Async), T indicates that it involves transaction control (Transaction), E indicates that it involves external API calls (External), R indicates that it involves access control (Rights), C indicates that it involves configuration management (Configuration), B indicates that there is a known technical defect (Bug), H indicates that it involves caching processing (Cache), and Q indicates that it involves message queues (Queue).

[0209] The size dimension is used to identify the size of a code file. Specifically, L represents large-scale, with more than 600 lines of code; M represents medium-scale, with 200 to 600 lines of code; and S represents small-scale, with less than 200 lines of code.

[0210] In practical applications, the first label coding system includes at least three dimensions.

[0211] More specifically, the first label encoding can have different values ​​for different first importance dimensions.

[0212] For example, when the importance level of a code file is core, the first tag encoding of the code file takes a value of 9 in the first importance dimension.

[0213] For example, when the importance level of the code file is high frequency, the first tag encoding of the code file takes a value of 8 in the first importance dimension.

[0214] For example, when the importance of a code file is at the business level, the first tag encoding of the code file takes a value of 7 in the first importance dimension.

[0215] For example, when the importance level of a code file is normal, the first tag encoding of the code file takes a value of 5 in the first importance dimension.

[0216] For example, when the importance level of a code file is auxiliary, the first tag encoding of the code file takes a value of 3 in the first importance dimension.

[0217] For example, when the importance level of a code file is marginal, the first tag encoding of the code file takes a value of 1 in the first importance dimension.

[0218] 9 indicates the core level, which is a necessary dependency for system operation; 8 indicates the high-frequency level, which is a key component of the main business process; 7 indicates the business level, which is the implementation of important functions; 5 indicates the regular level, which is a standard functional module; 3 indicates the auxiliary level, which is the support for edge functions; 1 indicates the edge level, which is the implementation of optional functions.

[0219] For example, the tag code of an authentication controller file may be CCA8RL, indicating that the file belongs to the controller layer (C), the authentication function of the core module (C) (A), the importance level is high frequency (8), it involves access control (R), and the code size is large (L).

[0220] Figure 4 This is a schematic diagram illustrating the encoding of the first tag in an exemplary authentication controller file. Figure 4 As shown, taking the label encoding CCA8RL as an example, this label can be parsed simultaneously by multiple attention heads of the Transformer: Attention head 1 focuses on C in dimension A, identifying that the file belongs to the controller layer; Attention head 2 focuses on the combination of CA in dimension B, identifying that the file belongs to the core authentication module; Attention head 3 focuses on 8 in dimension C, which can directly affect the attention weight allocation, causing the model to give more attention to high-importance items; Attention head 4 focuses on R in dimension D, identifying that the file involves access control; Attention head 5 focuses on L in dimension E, identifying that the file is a large-scale file.

[0221] Optionally, the first tag coding system can be extended to six or more tag dimensions. The extended tag dimensions include: security level dimension (F dimension), used to identify the security level involved in the code file, 1 for public, 2 for internal, and 3 for confidential; and version status dimension (G dimension), used to identify the version status of the code file, S for stable, B for testing, D for obsolete, and E for experimental.

[0222] In the above embodiments, the preset first label encoding system includes label encodings with multiple dimensions. This allows for the marking of different dimensions of the code file, comprehensively reflecting the characteristics of each dimension. Furthermore, using different numbers to encode the first importance dimension in the first label encoding of the code file facilitates the participation of the attention mechanism of the large language model in weight calculation.

[0223] In one example, at least one content element includes: a functional element, a dependency element, an interface element, and a descriptive element.

[0224] Among them, the functional element is used to indicate the functional responsibilities of the code file; the dependency element is used to indicate other modules or other libraries that the code file depends on; the interface element is used to indicate the interface path of the code file; and the description element is used to indicate the technical characteristics of the code file.

[0225] For example, as can be seen from the foregoing embodiments, the first semantic element information is used to indicate elements of at least one of the following aspects of the code file: functional elements (F), dependency elements (R), interface elements (A), and descriptive elements (S).

[0226] The following section provides a detailed explanation of the different characteristics of the code files indicated by each content element.

[0227] Functional elements are used to describe the core functional responsibilities of the code file, such as "JWT authentication control", "streaming dialogue service", "user data model", etc.

[0228] Dependency elements describe other modules or libraries that the code file depends on, and are presented as a comma-separated list of modules, such as "authService,userModel,bcrypt".

[0229] The interface element describes the path to the interface exposed or processed by the code file, such as " / api / auth / ". If there is no API, mark it as " / api / chat / stream"; otherwise, mark it as "none".

[0230] The brief description element is used to describe the key technical features of the code file. It should not exceed 50 characters in length, such as "login / registration, token refresh, permission verification".

[0231] In the example above, various primary content elements comprehensively represent the different content elements of the code file. Using diverse content elements to represent the key characteristics of the code file, such as its functional or technical features, allows the large language model to understand the characteristics of the code file more completely and clearly, without needing to read and process the entire code content.

[0232] Based on any of the foregoing embodiments or examples, an index document can be obtained by integrating the first index entries.

[0233] Specifically, in one example, the first index entry is input into a large language model for analysis and processing to obtain auxiliary content for the code file, including:

[0234] An index document is created by combining the first index entries from multiple code files. This index document is then input into a large language model for analysis and processing to obtain auxiliary content for the code files.

[0235] For example, the first index entries of each code file are organized into an index document. This index document is used as input to a large language model to assist in code understanding; that is, the large language model generates auxiliary content for the code files based on the first index entries of each code file contained in the index document.

[0236] Specifically, the structure of the index document includes: an index specification section, which defines the meaning of the first tag code and the meaning of the first semantic element information; and a code index section, which organizes the index entries of each code file according to business modules.

[0237] Among them, the adaptation features of indexed documents and Transformer attention mechanism include: a one-line-one-purpose format to support parallel processing; multi-dimensional labels to support multi-head attention from different dimensions; digital importance labels to participate in attention weight calculation; and a unified format to facilitate pattern matching.

[0238] In the example above, the first index entry is constructed based on the first tag encoding and the first semantic element information of the code file, and then combined into an index document. This enables the compression of code files in the codebase to be processed into a structured index document, achieving a compression ratio of approximately 500:1. This allows a large language model to obtain global information about the code files in the codebase within a limited context window, solving the technical problem in existing technologies where large language models cannot fully understand the code files of large codebases.

[0239] Based on the above example, the method further includes:

[0240] In response to code file modifications, a new index entry is generated for the modified code file based on the large language model. The indexed documents are then updated based on this new index entry.

[0241] For example, the first index entry of the code file in this application embodiment has single-file independence. Specifically, the first index entry of each code file is generated independently. It can be understood that the large language model only needs to read the content of the code file to complete the determination of the value of the first tag encoding and the extraction of the first semantic element information, without reading other code files.

[0242] Based on the above characteristics, when a single code file is modified, only the first index entry corresponding to the modified code file needs to be regenerated. Specifically, the modified code file is input into the large language model. Based on the large language model, and according to the first tag encoding system and the first semantic element extraction specification, the first tag encoding and first semantic element information of the modified code file are determined. Then, a new index entry for the modified code file is constructed.

[0243] Based on the large language model, the index document is updated according to the new index entries. Specifically, the file identifiers of the code files in the new index entries are matched with the file identifiers of the first index entries for each code file in the index document. The first index entry with the matching file identifier is replaced with the new index entry. This completes the update process for the index document.

[0244] Optionally, based on the above characteristics, the generation of indexes for multiple code files can be performed in parallel, with no dependencies between the code files; when adding a new code file, only the corresponding index entry needs to be appended, and when deleting a code file, only the corresponding index entry needs to be removed.

[0245] In the above example, when a code file needs to be modified, the index entries for that code file can be modified individually without regenerating the entire index document. This improves the efficiency of updating the index document when modifying a code file and shortens the time consumed in generating auxiliary content for the code file. Furthermore, the index document constructed based on index entries in the method provided in this application achieves single-file independence. In some embodiments, vector embedding typically requires calculating the embedding space based on a global corpus; modifications to a single file may affect the global vector representation, necessitating index reconstruction. However, the index specification design of this application ensures that each dimension (ABCDE) and each semantic element (F / R / A / S) can be independently determined by reading a single file, thereby achieving true incremental update capability.

[0246] In one possible implementation, the code files are stored in a codebase to be processed. The codebase may also include data tables.

[0247] For example, the codebase to be processed stores code files, and each code file has a directory structure.

[0248] In some cases, the codebase to be processed also contains data tables, which have a data table structure.

[0249] Based on this, Figure 5 A flowchart illustrating the code file processing method based on a large language model provided in this application. Figure 4 ,like Figure 5 As shown, the method also includes:

[0250] Step 501. If the codebase to be processed includes data tables, obtain multiple data tables from the codebase to be processed.

[0251] For example, determining whether the codebase to be processed contains a data table can be understood as determining whether the codebase to be processed contains a data table structure. If so, it is determined that the codebase to be processed contains a data table. Further, multiple data tables from the codebase to be processed are obtained.

[0252] For example, the codebase to be processed contains tables such as users, orders, and products. Optionally, multiple tables in the codebase can be retrieved using query commands (such as SHOW TABLES, DESCRIBE, etc.). Specifically, this retrieves a list of tables and their structure definitions.

[0253] Step 502. Analyze and process the data table according to the preset second tag encoding system to generate the second tag encoding corresponding to the data table.

[0254] The second label coding system is used to indicate the values ​​of the second label codes corresponding to different features of the data table.

[0255] For example, the second label coding system can represent the values ​​of the second label codes corresponding to different characteristics of a data table. For instance, the second label coding system can represent the characteristics of a data table in different dimensions. Different letters are used to represent the first position of the second label code for different types of data tables; different numbers are used to represent the second position of the second label code for data tables of different importance. Based on the characteristics of the data table itself (such as type and importance), the second label code corresponding to the data table is determined according to this second label coding system.

[0256] It should be noted that the above explanation of the second tag encoding system is not intended to limit its dimensions.

[0257] Step 503. Extract the second semantic element information from the data table according to the preset second semantic element extraction specification.

[0258] The second semantic element information is used to indicate at least one table structure element of the data table.

[0259] For example, the second semantic element information is used to indicate at least one of the following table structure elements of the data table: primary key, unique key, foreign key, index, and special field.

[0260] Based on the content of the data table and the different results corresponding to the above aspects, the second semantic element information is determined. Specifically, the content of the primary key in the data table corresponds to the content corresponding to the primary key element mentioned above, the content of the unique key in the data table corresponds to the content corresponding to the unique key element mentioned above, and so on.

[0261] Step 504. Generate the second index entry corresponding to the data table based on the second tag encoding and the second semantic element information.

[0262] The second index entry includes: the data table identifier of the data table, the second tag code corresponding to the data table, and the second semantic element information corresponding to the data table.

[0263] For example, based on the second tag code and second semantic element information of the data table, a second index entry corresponding to each data table is generated. The second index entry includes, in addition to the second tag code and second semantic element information corresponding to the data table, the table identifier of the data table. The table identifier is used to uniquely represent the data table.

[0264] Specifically, the format of a second index entry can be: "Table Name [Second Tag Code] | Primary Key | Unique Key | Foreign Key | Index | Special Field". Here, the table name is the table identifier for the data table.

[0265] For example, a database index entry for a user table might be:

[0266] "users[U9PST] | id[PK] | uuid,email,username[UK] | group_id→user_groups[FK] | role,status[IX] | credits_quota[NUM],deleted_at[TS]".

[0267] Step 505. Integrate the second index entries from multiple data tables into the index document.

[0268] For example, the second index entries corresponding to multiple data tables are integrated with the first index entries corresponding to the code files into a unified index document. This unified index document enables the large language model to establish a cognitive connection between the code and the data model.

[0269] It should be noted that steps 502 to 505 above can be processed based on a large language model. Specifically, the second tag encoding system and the second semantic element extraction specification are input into the large language model, and multiple data tables are input into the large language model in batches. Based on the large language model, the data tables in each batch are analyzed and processed to obtain the second index entry for each data table in each batch. Then, combined with the first index entry of the code file, the second index entries corresponding to each data table are integrated to obtain the index document.

[0270] In one example, an index document containing a first index entry and a second index entry is shown below:

[0271] =====Complete AOCI Index Documentation for the Project=====

[0272] ===Index Specifications===

[0273] Code index format: filename [ABCDE tags]: F: function | R: dependency | A: interface | S: description

[0274] Database index format: Table name [ABC tags] | Primary key | Unique key | Foreign key | Index | Special field

[0275] Code tagging system [ABCDE]:

[0276] A-level: B-backend, F-frontend, S-service, C-controller, M-middleware, D-database, U-tools, R-routing

[0277] Module B: Core A Authentication T Conversation R Points K Knowledge S Storage P Payment N Notification O Other

[0278] Importance of C: 9 cores, 8 high-frequency cores, 7 service cores, 5 routine cores, 3 auxiliary cores, 1 edge core.

[0279] D. Characteristics: P. Billing; A. Asynchronous; T. Transactions; E. External API; R. Permissions; C. Configuration; B. Bugs; H. Caching.

[0280] E size: L large (>600 rows) M medium (200-600 rows) S small (<200 rows)

[0281] Database Tag System [ABC]:

[0282] Table A is of type: U (User), T (Transaction), C (Content), S (System), L (Log), R (Relationship).

[0283] B-table characteristics: P-points, S-soft deletion, U=UUID, J=JSON, F-multiple foreign keys, T-timestamp.

[0284] Importance of C: 9 cores, 7 business functions, 5 configuration functions, 3 auxiliary functions

[0285] ===Index Specification Explanation Complete===

[0286] ===Code Index===

[0287] Module 1: Core System

[0288] server.js[BCC9S]: F: Server startup entry point | R: app, database, redis | A: Port 4000 | S: PM2 management, graceful shutdown

[0289] app.js[BCC9M]: F: Express application initialization | R: routes, middleware | A: / health | S: route registration, rate limiting

[0290] Module Two: Authentication and Authorization

[0291] AuthController.js[CCA8RL]: F: JWT authentication control | R: authService, userModel | A: / api / auth / S: Login / Register, Permission Verification

[0292] authService.js [SAC9TM]: F: Authentication service core | R: jwt, bcrypt, redis | A: None | S: JWT generation, password encryption

[0293] Module 3: Dialogue System

[0294] ChatController.js[CCT9APL]: F: Dialogue Controller | R: chatService,aiService | A: / api / chat / S: Flow response, integral pre-deduction

[0295] chatService.js[SCT9APEL]: F: Dialogue Service Core | R: OpenAI, Anthropic | A: None | S: Multi-model Adaptation, Streaming Response

[0296] ===Code index complete===

[0297] ===Database Index===

[0298] [User-related tables]

[0299] users[U9PST] | id[PK] | uuid,email[UK] | group_id→user_groups[FK] |role,status[IX] | credits[NUM],deleted_at[TS]

[0300] user_groups[U5ST] | id[PK] | name[UK] | - | status[IX] | permissions[JSON]

[0301] [Dialogue Related Form]

[0302] conversations[C9UTF] | id(uuid)[PK] | - | user_id→users,model_id→ai_models[FK] | created_at[IX] | messages[JSON]

[0303] ===Database indexing complete===

[0304] Statistics: 85 code files, 83 database tables, and approximately 350 lines of indexed documents.

[0305] =====AOCI Index Documentation Complete=====”

[0306] In the above embodiments, when a database table exists in the codebase to be processed, the database table can be used to obtain a second index entry based on the second tag encoding system and the second semantic element extraction specification. This second index entry is then integrated with the first index entry of the code file, enabling the large language model to establish a cognitive relationship between the code logic represented in the code file and the data model embodied in the data table. This solves the problem of code and data separation in the prior art and significantly improves the large language model's overall cognitive ability of the system.

[0307] In one example, the second label encoding system for a data table can be multi-dimensional label encoding.

[0308] Specifically, the second tag encoding includes tag encoding in at least one of the following dimensions: table type dimension, table characteristic dimension, and second importance dimension.

[0309] Among them, the table type dimension is used to indicate the business type of the data table; the table characteristic dimension is used to indicate the technical characteristics of the data table; and the second importance dimension is used to indicate the importance of the data table.

[0310] For example, the second label encoding includes three dimensions: table type dimension (dimensional A), table characteristic dimension (dimensional B), and second importance dimension (dimensional C).

[0311] The table type dimension (A dimension) is used to identify the business type of the database table. U represents user-related, T represents transaction-related, C represents content-related, S represents system configuration, L represents log recording, and R represents association.

[0312] The table feature dimension (B dimension) is used to identify the technical features of a database table. P indicates that it involves points, S indicates that it supports soft delete, U indicates that it is a UUID primary key, J indicates that it contains JSON fields, F indicates that it has multiple foreign key relationships, and T indicates that it is a timestamp tracking.

[0313] The importance dimension (C dimension) is used to identify the importance of database tables. 9 represents core tables, 7 represents business tables, 5 represents configuration tables, and 3 represents auxiliary tables.

[0314] In the example above, the preset second label encoding system has multiple dimensions of label encoding. This allows for the labeling of different dimensions of the data table, comprehensively reflecting the characteristics of these different dimensions.

[0315] Figure 6 This diagram illustrates the relationship between indexed documents and the attention mechanism of a large model. For example... Figure 6 As shown, the format design of the indexed documents is technically related to the Transformer attention mechanism.

[0316] Specifically, parallel processing adaptation: the one-line-one-target format design allows the Transformer's self-attention mechanism to calculate the association weights between all index entries simultaneously, rather than processing them sequentially one by one.

[0317] Multi-head attention adaptation: The multi-dimensional label design supports multiple attention heads of the Transformer to simultaneously focus on information from different dimensions (level, module, importance, feature, scale).

[0318] Attention weight adaptation: The importance dimension uses digital labels (9 / 8 / 7 / 5 / 3 / 1), which can be directly used in the calculation of attention weight, enabling the large language model to automatically allocate more attention resources to high-importance items.

[0319] Pattern matching adaptation: A unified format specification ("|" separating the four elements) facilitates the Transformer's attention mechanism in recognizing and extracting structured information.

[0320] Code and database association and adaptation: The unified index document includes code indexes and database indexes. The large language model can establish an association and understanding between code and data models through dependency elements and foreign key definitions.

[0321] In the solutions formed by the foregoing embodiments or examples, the limitation of the context window of a large language model is addressed by outputting independent index documents (AOCI format). In one possible implementation, semantic file naming (SEFS format) can also be used. It should be noted that these two formats have different compression levels and information retention rates, and can be used individually or in combination.

[0322] Based on any of the foregoing embodiments or examples, in one example, the method further includes:

[0323] The first tag is encoded and embedded in the filename of the code file to generate a semantic filename for the code file.

[0324] For example, the first tag of the code file is encoded and embedded in the filename of the code file to obtain the semantic filename of the code file.

[0325] Specifically, append the first tag code of the code file to the original filename. The format is: original filename of the code file - first tag code of the code file.extension.

[0326] For example, AuthController.js is renamed to AuthController-CCA8RL.js.

[0327] Figure 7 This is a flowchart illustrating the processing of an example code file. Figure 7 As shown, the code library structure information is first obtained, which is equivalent to obtaining the code files to be processed in the code library in the previous embodiment. A preset index specification is input into the large language model, which is equivalent to inputting the first tag encoding system and the first semantic element extraction specification into the large language model in the previous embodiment. The code files are input into the large language model in batches of 5-10 files each. The large language model reads the code content, performs multi-dimensional analysis according to the index specification, and outputs tag encoding and semantic element information. This is equivalent to analyzing and processing the code files based on the large language model in the previous embodiment to obtain the first tag encoding and the first semantic element information of the code files.

[0328] If no database table structure exists (i.e., the codebase to be processed does not contain any data tables), then the complete index document is directly organized based on the first index entry of the code file (including the first tag encoding and the first semantic element information). If a database table structure exists (i.e., the codebase to be processed includes data tables), the table structure is input into the large language model to generate database index entries. The code index and the database index are then integrated, which is equivalent to generating second index entries for each data table based on the large language model and integrating them with the first index entries to form the index document.

[0329] If semantic filenames are not required, the indexed document is directly input into the large language model to obtain auxiliary content for the code file. If semantic filenames are required, the tag encoding is embedded in the filename to generate a semantic filename, which is to append the first tag encoding of the code file to the original filename.

[0330] It is understood that three usage modes can be provided in the embodiments of this application:

[0331] Mode 1, semantic filenames combined with indexed documents (preferred mode): semantic filenames provide a macro overview, while indexed documents provide micro details, suitable for medium to large-scale projects.

[0332] Mode 2, using the index document alone: ​​without modifying the file name, only the index document is generated, with zero intrusion, suitable for legacy project transformation.

[0333] Mode 3, Semantic filenames used alone: ​​Information is encoded solely through the filename, eliminating the need to maintain additional documentation, making it suitable for lightweight new projects.

[0334] Of the three modes mentioned above, Mode 2, using only the index document, retains tag encoding and semantic element information, with an information retention rate of approximately 65% ​​and a compression ratio of approximately 500:1. Mode 3, using only semantic filenames, retains approximately 15% to 20% of the information, with a higher compression ratio than Mode 2. Mode 1 is used in combination; semantic filenames provide a quick overview of the system, while the index document provides a deeper understanding of the system details.

[0335] Figure 8 Here are flowcharts illustrating different processing modes. Figure 8 As shown, the first level is the index document (AOCI format): retaining tag encoding and four semantic elements (function, dependency, interface, and description), with an information retention rate of approximately 65% ​​and a compression ratio of approximately 500:1, suitable for in-depth understanding of system details. The second level is the semantic filename (SEFS format): retaining only tag encoding, with an information retention rate of approximately 15% to 20% and a higher compression ratio than the index document, suitable for quickly gaining a comprehensive overview of the system.

[0336] When the two compression levels are used together, semantic filenames provide a macro perspective through directory traversal commands (such as tree), while indexed documents provide micro-details through complete semantic elements, forming a complete hierarchical cognitive scheme.

[0337] Furthermore, comparing traditional filenames and semantic filenames reveals that traditional filenames (such as AuthController.js) can only contain one information dimension when used as file identifiers, namely the file identifier of the code file; while semantic filenames (such as AuthController-CCA8RL.js) can contain five information dimensions, increasing the amount of information by 300%-500%.

[0338] The following is an example of a traditional directory structure:

[0339] project /

[0340] ├── controllers /

[0341] │ ├── AuthController.js

[0342] │ ├── ChatController.js

[0343] │ └── UserController.js”

[0344] Large language models can only obtain one dimension of information from traditional naming, namely the semantics of filenames, and can only know that "these are three controller files, which are related to authentication, dialogue and user respectively".

[0345] Furthermore, the following is an example of a directory structure with semantic filenames:

[0346] project /

[0347] ├── controllers /

[0348] │ ├── AuthController-CCA8RL.js

[0349] │ ├── ChatController-CCT9APL.js

[0350] │ └── UserController-CCA7RM.js”

[0351] The large language model can extract five information dimensions from semantic filenames: architectural level (C controller), functional modules (A authentication / T conversation), importance (9 / 8 / 7), technical features (P billing / A asynchronous / R permission), and code size (L large / M medium). The large language model immediately recognizes that ChatController (9) is more core than AuthController (8).

[0352] Compared to traditional naming, semantic filenames increase the amount of architectural information that large language models can obtain through directory traversal commands by approximately 300% to 500%.

[0353] Optionally, after generating the semantic filename for the code file, the method also includes:

[0354] Iterate through the semantic filenames of each code file to generate a directory structure. The directory structure includes the semantic filenames of each code file.

[0355] The directory structure is input into the large language model to obtain the architecture information of the code library to be processed, which stores multiple code files.

[0356] Among them, the architecture information is used in the process of analyzing and processing the large language model, combined with the first index entry, to obtain auxiliary understanding content of the code file.

[0357] For example, if semantic filenames are generated for each code file, traversing each code file can yield the directory structure. Based on the large language model, the architecture information of the codebase to be processed can be obtained from the directory structure. This architecture information can be combined with the first index entry to assist the large language model in generating auxiliary content for the code files.

[0358] Optionally, the large language model generates the corresponding index entry and semantic file name simultaneously while generating the code file content; the index entry, semantic file name, and code file content are output together.

[0359] In the example above, by embedding the first tag of the code file into the filename, the filename can directly reflect the technical characteristics of the code file. Furthermore, the design of semantic filenames enables large language models to obtain the architectural information of the code library to be processed through traversal operations. This allows large language models to quickly understand the overall system without reading the file contents of the code files, significantly improving their ability to oversee the code files in the code library.

[0360] The code file processing method based on a large language model provided in this application involves processing the code file to be processed according to a preset first tag encoding system and a first semantic element extraction specification to obtain the first tag encoding and first semantic element information of the code file. Based on the first tag encoding and first semantic element information, a first index entry is constructed. The first index entry is then analyzed and processed based on the large language model to obtain auxiliary content of the code file. On the one hand, this method can extract and compress intermediate features of the code content in the code file, enabling the large language model to identify the key features of the entire code content and output more accurate auxiliary content. On the other hand, by using the first index entry composed of the first tag encoding and first semantic element information, which has better user readability, the key features of the code file are extracted and compressed, avoiding the problem of poor readability of code file features caused by vector embedding.

[0361] Specifically, by inputting pre-defined indexing specifications (such as tag encoding systems and semantic element extraction specifications) into a large language model, code files are input into the model in batches. The large language model then analyzes the data according to the indexing specifications and generates structured index entries. The index generation in this application features single-file independence; index entries for each code file can be generated independently. The large language model only needs to read the contents of that file to complete tag encoding determination and semantic element information extraction, without needing to read other files. The index document format is specifically designed for the Transformer attention mechanism, allowing for complete input into the large language model, enabling the model to obtain a global architectural view of the codebase to be processed.

[0362] Figure 9 Here is an example application hierarchy diagram, such as Figure 9 As shown in the embodiments of this application, the method for processing code files based on a large language model can include an input layer, a processing layer, an output layer, and an application layer in practical applications.

[0363] In one possible application project, the project's technical parameters are as follows: the technology stack is Vue 3 frontend plus Node.js / Express backend plus MySQL database; it includes 17 functional modules and 83 database tables; the backend code is approximately 55,000 lines and 181 files, the frontend code is approximately 66,000 lines; the total code (including database migration scripts) is approximately 140,000 lines, averaging approximately 611 lines per file.

[0364] Project size conversion: Based on an estimated 4 characters per token, 140,000 lines of code would amount to approximately 3.5 to 4 million characters, or approximately 870,000 to 1 million tokens, far exceeding the context window capacity of mainstream large language models.

[0365] Application effect verification: The index document generated by the method of this invention has approximately 300 to 400 lines, with a compression ratio of approximately 400:1 to 500:1. The large language model can achieve approximately 65% ​​system understanding within minutes using the index document, accurately answering questions about system architecture and module relationships, understanding the association between code and database tables, quickly locating the module where the problem lies, and accurately analyzing the impact of code modifications on other modules.

[0366] Verification of Single-File Independence: During project iteration, when a single code file is modified, only the corresponding index entry (approximately 1 line) needs to be regenerated, without regenerating the entire index document (approximately 350 lines). Adding a new code file only requires appending the corresponding index entry, and deleting a code file only requires removing the corresponding index entry. This feature reduces the index maintenance cost from O(n) to O(1), significantly improving maintainability in practical applications.

[0367] Comparative Results: In related technologies, large language models cannot fully input the project code due to context window limitations; if related RAG retrieval schemes are used, only local code fragments can be obtained, failing to establish a global architectural understanding, and modifications to a single file may require rebuilding the global vector index. The method provided in this application, through a preset tag encoding system and semantic element extraction specifications, extracts key features from code files and / or data tables and compresses semantics, enabling large language models to obtain all code content information within limited context window constraints. It also provides efficient incremental update capabilities.

[0368] Figure 10 A schematic diagram of the structure of the code file processing device based on a large language model provided in this application is shown below. Figure 10 As shown, the code file processing device 100 based on a large language model provided in this embodiment includes:

[0369] Module 1001 is used to acquire multiple code files to be processed.

[0370] The processing module 1002 is used to analyze and process the code file according to the preset first tag encoding system to generate the first tag encoding corresponding to the code file; wherein, the first tag encoding system is used to indicate the value of the first tag encoding corresponding to different features of the code file;

[0371] The processing module 1002 is further configured to extract first semantic element information of the code file according to a preset first semantic element extraction specification; the first semantic element information is used to indicate at least one content element of the code file.

[0372] The processing module 1002 is further configured to generate a first index entry corresponding to the code file based on the first tag code and the first semantic element information; wherein, the first index entry includes: the file identifier of the code file, the first tag code corresponding to the code file, and the first semantic element information corresponding to the code file;

[0373] The processing module 1002 is also used to input the first index entry into the large language model for analysis and processing to obtain auxiliary content of the code file.

[0374] In one possible implementation, the code file is analyzed and processed according to a preset first tag encoding system to generate the first tag encoding corresponding to the code file. The processing module 1002 is used for:

[0375] The first tag encoding system is input into the large language model; and the code files are input into the large language model in batches; wherein each batch includes a preset number of code files;

[0376] Based on the large language model, and according to the first tag encoding system, the code files in each batch are analyzed and processed sequentially to obtain the first tag encoding corresponding to the code file.

[0377] In one possible implementation, the processing module 1002 extracts the first semantic element information of the code file according to a preset first semantic element extraction specification, and is used for:

[0378] The first semantic element extraction specification is input into the large language model;

[0379] Based on the large language model, and according to the first semantic element extraction specification, semantic elements are extracted from the code files in each batch in sequence to obtain the first semantic element information of the code files.

[0380] In one possible implementation, the first label encoding includes label encoding in at least one of the following dimensions: hierarchy dimension, module dimension, first importance dimension, feature dimension, and scale dimension.

[0381] Among them, the hierarchy dimension is used to indicate the technical architecture level to which the code file belongs; the module dimension is used to indicate the functional module to which the code file belongs; the first importance dimension is used to indicate the importance of the code file; the feature dimension is used to indicate the technical implementation features of the code file; and the scale dimension is used to indicate the code size of the code file.

[0382] In one possible implementation, when the importance of the code file is at the core level, the first tag encoding of the code file takes a value of 9 in the first importance dimension;

[0383] When the importance level of the code file is high frequency, the first tag encoding of the code file takes a value of 8 in the first importance dimension;

[0384] When the importance of the code file is at the business level, the first tag of the code file is encoded with a value of 7 in the first importance dimension;

[0385] When the importance level of the code file is normal, the first tag encoding of the code file takes a value of 5 in the first importance dimension;

[0386] When the importance level of the code file is auxiliary, the first tag encoding of the code file takes a value of 3 in the first importance dimension;

[0387] When the importance level of a code file is marginal, the first tag encoding of the code file takes a value of 1 in the first importance dimension.

[0388] In one possible implementation, at least one content element includes: a functional element, a dependency element, an interface element, and a descriptive element.

[0389] Among them, the functional element is used to indicate the functional responsibilities of the code file; the dependency element is used to indicate other modules or other libraries that the code file depends on; the interface element is used to indicate the interface path of the code file; and the description element is used to indicate the technical characteristics of the code file.

[0390] In one possible implementation, the first index entry is input into a large language model for analysis and processing to obtain auxiliary content for the code file. The processing module 1002 is used for:

[0391] An index document is created by combining the first index entries of multiple code files.

[0392] The indexed documents are input into a large language model for analysis and processing to obtain auxiliary content for the code files.

[0393] In one possible implementation, the processing module 1002 is further configured to:

[0394] In response to code file modification operations, a new index entry corresponding to the modified code file is generated based on the large language model.

[0395] The indexed documents are updated based on the new index entries.

[0396] In one possible implementation, the code file is stored in the code library to be processed; the acquisition module 1001 is further configured to acquire multiple data tables in the code library to be processed if the code library to be processed includes data tables.

[0397] The processing module 1002 is also used to analyze and process the data table according to the preset second label coding system, and generate the second label code corresponding to the data table; wherein, the second label coding system is used to indicate the value of the second label code corresponding to different features of the data table;

[0398] The processing module 1002 is further configured to extract second semantic element information of the data table according to a preset second semantic element extraction specification; the second semantic element information is used to indicate at least one table structure element of the data table.

[0399] The processing module 1002 is further configured to generate a second index entry corresponding to the data table based on the second tag code and the second semantic element information; wherein the second index entry includes: the data table identifier of the data table, the second tag code corresponding to the data table, and the second semantic element information corresponding to the data table;

[0400] The processing module 1002 is also used to integrate the second index entries of multiple data tables into an index document.

[0401] In one possible implementation, the second tag encoding includes tag encoding of at least one of the following dimensions: table type dimension, table characteristic dimension, and second importance dimension;

[0402] Among them, the table type dimension is used to indicate the business type of the data table; the table characteristic dimension is used to indicate the technical characteristics of the data table; and the second importance dimension is used to indicate the importance of the data table.

[0403] In one possible implementation, the processing module 1002 is further configured to:

[0404] The first tag is encoded and embedded in the filename of the code file to generate a semantic filename for the code file.

[0405] The code file processing device based on a large language model provided in this embodiment can execute the method provided in the above method embodiment. Its implementation principle and technical effect are similar, and will not be described in detail here.

[0406] It should be noted that the above module division does not imply a limitation on the actual module division for processing and transposing code files based on a large language model. It can be understood that the above module division is only based on functional logic. In practical applications, the module division of this device can take other forms.

[0407] In one possible implementation, Figure 11 A schematic diagram of the processing flow of the code file processing device based on a large language model provided in this application is shown below. Figure 11As shown, the acquisition module is used to acquire the structural information of the code library to be processed, which includes the directory structure of the code files and / or the table structure of the database.

[0408] The tag generation module is used to perform multi-dimensional analysis on the code files in the code library according to a preset multi-dimensional tag encoding system, and generate tag codes corresponding to each code file.

[0409] The element extraction module is used to extract semantic element information from each code file according to the preset semantic element extraction specifications.

[0410] The index generation module generates index entries for each code file based on tag encoding and semantic feature information. Index entry generation is file-independent; each code file's index entries can be generated independently, supporting parallel processing and incremental updates.

[0411] The document organization module is used to organize index entries into index documents.

[0412] In this embodiment, the functions of the tag generation module, feature extraction module, and index generation module are implemented through a large language model. Specifically, a preset index specification (including a multi-dimensional tag encoding system and semantic feature extraction specification) is input into the large language model, and then the code file content is input in batches. The large language model analyzes the code according to the index specification and outputs the corresponding tag encoding and semantic feature information.

[0413] Optionally, the tag generation module is specifically used to perform multi-dimensional analysis of code files based on a five-dimensional tag coding system that includes hierarchical, module, importance, feature, and scale dimensions.

[0414] Optionally, the label generation module is also used to: divide the importance into multiple levels, each level corresponding to a different numerical label, which is used to participate in weight calculation in the attention mechanism of the large language model.

[0415] Optionally, a database indexing module is also included, which performs multi-dimensional analysis of tables in the database according to a preset database tag encoding system, generates database index entries, and integrates the database index entries with the index entries of the code files into a unified index document. The functionality of the database indexing module is also implemented through a large language model.

[0416] Optionally, a naming module is also included to embed tag encoding into the filename of the code file, generating semantic filenames, which serve as a representation with a higher compression ratio than the indexed documents.

[0417] Figure 12 A schematic diagram of the structure of the electronic device provided in this application. Figure 12As shown, the electronic device 120 provided in this embodiment includes at least one processor 1201 and a memory 1202. Optionally, the electronic device 120 further includes a communication component 1203. The processor 1201, the memory 1202, and the communication component 1203 are connected via a bus 1204.

[0418] In a specific implementation, at least one processor 1201 executes computer execution instructions stored in memory 1202, causing at least one processor 1201 to perform the above-described method.

[0419] Optionally, the processor 1201 can run the large language model directly, or call the API service of the large language model through the communication component 1203.

[0420] The specific implementation process of processor 1201 can be found in the above method embodiments, and its implementation principle and technical effect are similar. It will not be repeated here.

[0421] In the above embodiments, it should be understood that the processor can be a Central Processing Unit (CPU), or other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), etc. The general-purpose processor can be a microprocessor or any conventional processor. The steps of the method disclosed in this invention can be directly implemented by a hardware processor, or implemented by a combination of hardware and software modules within the processor.

[0422] The memory may include random access memory (RAM) and may also include non-volatile memory (NVM), such as at least one disk storage device.

[0423] The bus can be an Industry Standard Architecture (ISA) bus, a Peripheral Component Interconnect (PCI) bus, or an Extended Industry Standard Architecture (EISA) bus, etc. Buses can be categorized as address buses, data buses, control buses, etc. For ease of illustration, the buses shown in the accompanying drawings are not limited to a single bus or a single type of bus.

[0424] This application also provides a computer program product, including a computer program that, when executed by a processor, implements the above-described method.

[0425] This application also provides a computer-readable storage medium storing computer-executable instructions, which, when executed by a processor, implement the above-described method.

[0426] The aforementioned readable storage medium can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic storage, flash memory, magnetic disk, or optical disk. The readable storage medium can be any available medium accessible to a general-purpose or special-purpose computer.

[0427] An exemplary readable storage medium is coupled to a processor, enabling the processor to read information from and write information to the readable storage medium. Of course, the readable storage medium can also be a component of the processor. The processor and the readable storage medium can reside in an Application Specific Integrated Circuit (ASIC). Alternatively, the processor and the readable storage medium can exist as discrete components in the device.

[0428] The division of units is merely a logical functional division; in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be indirect coupling or communication connection through some interfaces, devices, or units, and may be electrical, mechanical, or other forms.

[0429] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0430] In addition, the functional units in the various embodiments of the present invention can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit.

[0431] If a function is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this invention, or the part that contributes to the prior art, or a part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods of the various embodiments of this invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0432] Those skilled in the art will understand that all or part of the steps of the above-described method embodiments can be implemented by hardware related to program instructions. The aforementioned program can be stored in a computer-readable storage medium. When executed, the program performs the steps of the above-described method embodiments; and the aforementioned storage medium includes various media capable of storing program code, such as ROM, RAM, magnetic disks, or optical disks.

[0433] Finally, it should be noted that other embodiments of the invention will readily occur to those skilled in the art upon consideration of the specification and practice of the invention disclosed herein. This invention is intended to cover any variations, uses, or adaptations of the invention that follow the general principles of the invention and include common knowledge or customary techniques in the art not disclosed herein, and is not limited to the precise structures described above and shown in the accompanying drawings, and various modifications and changes can be made without departing from its scope. The scope of the invention is limited only by the appended claims.

Claims

1. A method for processing code files based on a large language model, characterized in that, include: Retrieve multiple code files to be processed; Input the preset first tag encoding system into the large language model; The code files are input into the large language model in batches; wherein, the first tag encoding system is used to indicate the value of the first tag encoding corresponding to different features of the code files; each batch includes a preset number of code files; Based on the large language model, the code files in each batch are analyzed and processed sequentially according to the first tag encoding system to obtain the first tag encoding corresponding to the code file; The preset first semantic element extraction specification is input into the large language model; Based on the large language model, and according to the first semantic element extraction specification, semantic elements are extracted sequentially from the code files in each batch to obtain the first semantic element information of the code files; the first semantic element information is used to indicate at least one content element of the code files; and a first index entry corresponding to the code files is generated according to the first tag code and the first semantic element information; wherein, the first index entry includes: the file identifier of the code files, the first tag code corresponding to the code files, and the first semantic element information corresponding to the code files; The first index entry is input into a large language model for analysis and processing to obtain the auxiliary content of the code file; The first label encoding includes label encoding in at least three dimensions: hierarchy dimension, module dimension, first importance dimension, feature dimension, and scale dimension; The hierarchy dimension indicates the technical architecture level to which the code file belongs; the module dimension indicates the functional module to which the code file belongs; the first importance dimension indicates the importance of the code file; the feature dimension indicates the technical implementation features of the code file; and the scale dimension indicates the code size of the code file.

2. The method according to claim 1, characterized in that, When the importance level of the code file is core level, the first tag encoding of the code file takes the value of 9 in the first importance dimension; When the importance level of the code file is high frequency, the value of the first tag encoding of the code file in the first importance dimension is 8; When the importance of the code file is at the business level, the first tag encoding of the code file takes a value of 7 in the first importance dimension; When the importance level of the code file is normal, the first tag code of the code file takes a value of 5 in the first importance dimension; When the importance level of the code file is auxiliary, the first tag encoding of the code file takes a value of 3 in the first importance dimension; When the importance level of the code file is marginal, the first tag encoding of the code file takes a value of 1 in the first importance dimension.

3. The method according to claim 1, characterized in that, The at least one content element includes: functional elements, dependency elements, interface elements, and description elements; The functional elements are used to indicate the functional responsibilities of the code file; the dependency elements are used to indicate other modules or libraries that the code file depends on; the interface elements are used to indicate the interface path of the code file; and the description elements are used to indicate the technical characteristics of the code file.

4. The method according to any one of claims 1-3, characterized in that, The first index entry is input into a large language model for analysis and processing to obtain auxiliary content of the code file, including: An index document is created by combining the first index entries of multiple code files. The index document is input into a large language model for analysis and processing to obtain auxiliary content for the code file.

5. The method according to claim 4, characterized in that, The method further includes: In response to a modification operation on a code file, a new index entry corresponding to the modified code file is generated based on the large language model. The index document is updated based on the new index entry.

6. The method according to claim 4, characterized in that, The code file is stored in the code library to be processed; the method further includes: If the code library to be processed includes data tables, obtain multiple data tables from the code library to be processed; According to a preset second tag encoding system, the data table is analyzed and processed to generate a second tag code corresponding to the data table; wherein, the second tag encoding system is used to indicate the value of the second tag code corresponding to different features of the data table; According to the preset second semantic element extraction specification, the second semantic element information of the data table is extracted; the second semantic element information is used to indicate at least one table structure element of the data table; Based on the second tag code and the second semantic element information, a second index entry corresponding to the data table is generated; wherein, the second index entry includes: the data table identifier of the data table, the second tag code corresponding to the data table, and the second semantic element information corresponding to the data table; The second index entries from multiple data tables are integrated into the index document.

7. The method according to claim 6, characterized in that, The second tag encoding includes tag encoding of at least one of the following dimensions: table type dimension, table characteristic dimension, and second importance dimension; The table type dimension indicates the business type of the data table; the table characteristic dimension indicates the technical characteristics of the data table; and the second importance dimension indicates the importance of the data table.

8. The method according to any one of claims 1-3, characterized in that, The method further includes: The first tag is encoded and embedded into the filename of the code file to generate a semantic filename for the code file.

9. A processing device for code files based on a large language model, characterized in that, include: The acquisition module is used to acquire multiple code files to be processed. The processing module is used to input the preset first tag encoding system into the large language model; The code files are input into a large language model in batches. Based on the large language model, the code files in each batch are analyzed and processed sequentially according to the first tag encoding system to obtain the first tag encoding corresponding to the code file. The first tag encoding system is used to indicate the value of the first tag encoding corresponding to different features of the code file. Each batch includes a preset number of code files. The processing module is further configured to input a preset first semantic element extraction specification into the large language model; based on the large language model, and according to the first semantic element extraction specification, sequentially extract semantic elements from the code files in each batch to obtain the first semantic element information of the code files; the first semantic element information is used to indicate at least one content element of the code file. The processing module is further configured to generate a first index entry corresponding to the code file based on the first tag code and the first semantic element information; wherein, the first index entry includes: the file identifier of the code file, the first tag code corresponding to the code file, and the first semantic element information corresponding to the code file; The processing module is also used to input the first index entry into a large language model for analysis and processing to obtain auxiliary content of the code file; The first label encoding includes label encoding in at least three dimensions: hierarchy dimension, module dimension, first importance dimension, feature dimension, and scale dimension; The hierarchy dimension indicates the technical architecture level to which the code file belongs; the module dimension indicates the functional module to which the code file belongs; the first importance dimension indicates the importance of the code file; the feature dimension indicates the technical implementation features of the code file; and the scale dimension indicates the code size of the code file.

10. An electronic device, characterized in that, include: Memory, processor; The memory stores computer-executed instructions; The processor executes computer execution instructions stored in the memory, causing the processor to perform the method as described in any one of claims 1-8.

11. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores computer-executable instructions, which, when executed by a processor, are used to implement the method as described in any one of claims 1-8.

12. A computer program product, characterized in that, Includes a computer program that, when executed by a processor, implements the method of any one of claims 1-8.

Images