Database code extraction method and system based on large language model rapid screening

Abstract

The invention relates to the technical field of software development data management, and provides a database code extraction method and system based on large language model rapid screening, a database code extraction method based on an AI large language model, a user can automatically extract a database code from a software file code library more accurately and efficiently, and the user experience is improved. The method is applied to database code auditing, publishing and database type code conversion scenes. The invention provides a method for rapidly screening and filtering database code files by a database grammar supplementary format capable of enabling a large language model to better understand and an automatic database type and version accurate identification scheme, and can effectively solve the problems of low accuracy and efficiency of large language model database code extraction.

Description

Technical Field

[0001] The present invention relates to the technical field of software development data management, and in particular to a database code extraction method based on rapid screening of a large language model, a database code extraction system based on rapid screening of a large language model, an electronic device and a computer-readable storage medium. Background Art

[0002] In the process of software development, code review is often required. There are many review tools and methods for language codes such as Java and C++ on the market, while database codes are usually written in SQL language. The written SQL codes are not independent files, but are usually scattered in other language code files such as Java, C++, and Python. Database codes usually need to be submitted to database administrators for review and operation separately, so it is necessary to extract the SQL codes scattered in different code files. Traditional database code extraction work used to be done by engineers manually one file at a time, which takes a lot of time, resulting in low efficiency in database code review and release.

[0003] In addition, although the current AI technology can send the software code file to the big language model after the AI ​​big language model comes out, and the big language model automatically extracts the database code, there are still several problems: There are many types of databases. If the AI ​​language model is not clear about the database type, the code recognition rate is not high and the wrong content may be extracted. The AI ​​large language model lacks the latest database version corpus, and the success rate of extracting new databases or new version codes is not high; The database SQL codes are scattered in different program code files. A software may have hundreds or thousands of files. If all program code files are sent to the large language model, a large number of tokens of the large language model will be consumed, and the generation efficiency will be low. In addition, the large file may exceed the length limit of the large language model token, and accurate results cannot be obtained. Summary of the invention

[0004] In order to solve the technical problems existing in the prior art, the present invention provides the following technical solutions: On the one hand, a method for extracting database codes based on rapid screening of a large language model is provided, the method being implemented by an electronic device, the method comprising: S1. Obtain the database type and version in the user code library file; S2. According to the database type and version, the corresponding database keywords are searched from the preset database code keyword list and word segmentation enhancement is performed; S3, based on the enhanced database code keywords, filter out files containing keywords from the user code library files and mark them as program files containing database codes; S4, constructing basic prompt words for the large language model based on the database code keywords, adding new database query syntax to the basic prompt words, generating new large model prompt word information and inputting it into the preset large language model; S5. activating the large language model, and having the large language model extract corresponding database codes from the program file containing the database codes based on the large model prompt word information, and outputting the codes.

[0005] Further, S1, obtaining the database type and version in the user code library file, including: Input the user code library file; Parse the user code library file to obtain all program files; Traverse each program file to see if it contains a database configuration file: If not, traverse the next one until all program files are checked; If yes, the database configuration file is read and parsed to obtain basic data corresponding to the database type and version, thereby obtaining the database type and version.

[0006] Further, S2, according to the database type and version, the corresponding database keywords are searched from the preset database code keyword list and word segmentation enhancement processing is performed, including: Pre-building the database code keyword list constructed by different versions of databases and corresponding keywords; According to the database type and version, the corresponding database code keyword is searched from the preset database code keyword list and input into the preset Bloom filter to generate a Bloom filter data structure for data retrieval; The word segmentation processing technology is used to identify and extract the word segments in the code file content of the user code library file, and the word segments are written into the Bloom filter data structure and input into the Bloom filter.

[0007] Further, S3, based on the enhanced database code keywords, filtering out files containing keywords from the user code library files and marking them as program files containing database codes, including: Traverse all program files; Perform word segmentation analysis on each program file to obtain the word segmentation of the corresponding program file; Use Bloom filters to match database keywords for each program file: If it is found that the program file matches the keyword, the program file matching the keyword is identified as a program file containing the database code; If no matching keywords are found in the program file, proceed to the next step; Determine whether all the program files have been traversed: If so, all traversals are completed and proceed to the next step; If there are other program files that have not been traversed, continue to traverse the program files; The program file containing the database code is saved in a software file code library.

[0008] Further, S4, constructing basic prompt words for the large language model based on the database code keywords, adding new database query syntax to the basic prompt words, generating new large model prompt word information and inputting it into the preset large language model, including: Preparing basic prompt words of a large language model, wherein the basic prompt words are constructed based on the database type and version in the database code keywords; Based on the basic prompt words, construct corresponding database query grammar; Organizing the database query grammar, generating the large model prompt word information and inputting it into the large language model; When the large language model is used to traverse the program files later, it is checked in real time whether a new database type and version appears in the dynamically input user code library file: If so, based on the new database type and version, a new database query grammar is constructed and added to the large language model; If it doesn't exist, give up.

[0009] Furthermore, before S5, activating the large language model, and extracting and outputting the corresponding database code from the program file containing the database code based on the large model prompt word information by the large language model, the method further includes: Traverse all program files identified as containing database code; Calculate the file size of each program file containing the database code: token length: , B is the token length, n is the number of text characters in the program file, Lsi is the word length of each text; Determine whether the file size of the program file containing the database code: token length exceeds the token limit of the large language model: If yes, the program file containing the database code is fragmented to obtain a plurality of program fragment files and input them into the large language model; If not, the program file containing the database code is directly input into the large language model.

[0010] Further, S5, activating the large language model, the large language model extracting and outputting the corresponding database code from the program file containing the database code based on the large model prompt word information, including: issuing an extraction instruction to the large language model to activate the large language model; Extracting and returning corresponding database codes from the program file containing database codes based on the large model prompt word information through the large language model; The database code is parsed and saved as a database code in a corresponding structured format.

[0011] On the other hand, a database code extraction system based on rapid screening of a large language model is provided, and the database code extraction system based on rapid screening of a large language model is used to implement the database code extraction method based on rapid screening of a large language model, and is characterized in that the system comprises: The file acquisition module is used to obtain the database type and version in the user code library file; The keyword search module is used to search for corresponding database keywords from a preset database code keyword list and perform word segmentation enhancement processing according to the database type and version; A file screening module, for screening out files containing keywords from the user code library files based on the enhanced database code keywords and marking them as program files containing database codes; An intelligent screening module, used to construct basic prompt words for a large language model based on the database code keywords, and to add new database query syntax to the basic prompt words, generate new large model prompt word information and input it into a preset large language model; The code extraction module is used to activate the large language model, and the large language model extracts and outputs the corresponding database code from the program file containing the database code based on the large model prompt word information.

[0012] On the other hand, an electronic device is provided, comprising: a processor; a memory, wherein the memory stores computer-readable instructions, and when the computer-readable instructions are executed by the processor, any one of the above-mentioned database code extraction methods based on rapid screening of a large language model is implemented.

[0013] On the other hand, a computer-readable storage medium is provided, wherein at least one instruction is stored in the storage medium, and the at least one instruction is loaded and executed by a processor to implement any one of the above-mentioned database code extraction methods based on fast screening of a large language model.

[0014] The beneficial effects brought about by the technical solution provided by the embodiment of the present invention include at least: Based on the implementation of the present invention, the present invention proposes a database code extraction method based on an AI large language model, so that users can automatically extract database codes from software file code libraries more accurately and efficiently, and apply it to database code review, release, and database type code conversion scenarios.

[0015] The present invention allows a large language model to better understand the database syntax supplement format, automates the database type and version accurate identification scheme, and provides a method for quickly screening and filtering database code files, which can effectively solve the problem of low accuracy and efficiency in large language model database code extraction. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the accompanying drawings required for use in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings in the following description are only some embodiments of the present invention. For ordinary technicians in this field, other accompanying drawings can be obtained based on these accompanying drawings without paying creative work.

[0017] Figure 1 It is an operation flow chart of a database code extraction method based on rapid screening of a large language model provided by an embodiment of the present invention; Figure 2 It is a schematic diagram of a process of obtaining database configuration information from a user code library file provided by an embodiment of the present invention; Figure 3 is a database code keyword list provided by an embodiment of the present invention; Figure 4 It is a schematic diagram of a flow chart of screening out program files containing keywords provided by an embodiment of the present invention; Figure 5 It is a schematic diagram of a process of extracting database code from a large language model provided by an embodiment of the present invention; Figure 6 It is a block diagram of a database code extraction system based on fast screening of a large language model provided by an embodiment of the present invention; Figure 7 It is a structural schematic diagram of an electronic device provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0018] The technical solution of the present invention is described below in conjunction with the accompanying drawings.

[0019] In the embodiments of the present invention, words such as "exemplarily" and "for example" are used to indicate examples, illustrations or explanations. Any embodiment or design described as "example" in the present invention should not be interpreted as being more preferred or more advantageous than other embodiments or designs. Specifically, the use of the word "example" is intended to present the concept in a specific way. In addition, in the embodiments of the present invention, the meaning expressed by "and / or" can be both, or it can be either of the two.

[0020] In the embodiments of the present invention, "image" and "picture" can sometimes be used interchangeably. It should be noted that when the difference between them is not emphasized, the meanings they intend to express are the same. "of", "corresponding, relevant" and "corresponding" can sometimes be used interchangeably. It should be noted that when the difference between them is not emphasized, the meanings they intend to express are the same.

[0021] In the embodiments of the present invention, sometimes a subscript such as W1 may be mistakenly written as a non-subscript such as W1. When the difference is not emphasized, the meanings to be expressed are consistent.

[0022] In order to make the technical problems, technical solutions and advantages to be solved by the present invention more clear, a detailed description will be given below with reference to the accompanying drawings and specific embodiments.

[0023] An embodiment of the present invention provides a database code extraction method based on rapid screening of a large language model. The method can be implemented by an electronic device, which can be a terminal or a server.

[0024] The present invention is applicable to the mainstream large language models on the market, such as ChatGPT\Llama\Baichuan Intelligence\Wenxin Yiyan\Tongyi Qianwen and other large language models, or AI large language models. Large Language Model (LLM) is a deep learning model trained based on massive text data. It not only has the ability to generate natural language text, but also can deeply understand the meaning of the text and handle various natural language tasks, such as text summarization, question answering, translation, etc.

[0025] The large language model uses deep learning technology and multi-layer neural networks to model the statistical laws and potential semantic information of language. During the training process, the large language model will learn and abstract a large amount of text data, so that it can generate logical and coherent language output. At the same time, in order to ensure that the model has good generalization ability, it is also necessary to collect and organize large-scale data sets for training.

[0026] The big language model has demonstrated strong capabilities and potential in natural language processing, machine translation, dialogue systems, text generation, and other fields. It can understand human natural language input and generate semantically relevant output based on the input content. By learning a large amount of text data, the big language model can gain a deep understanding of language structure, grammar, semantics, and other aspects.

[0027] In this embodiment, the AI ​​large language model can be selected by the user, such as Chat GPT\Llama\Wenxin Yiyan\Tongyi Qianwen.

[0028] Regarding the semantic interaction of the AI ​​big language model, please understand it in conjunction with the logical architecture of the AI ​​big language model. This embodiment will not go into details.

[0029] The present invention proposes a flowchart of a database code extraction method based on rapid screening of a large language model. The processing flow of the method may include the following steps: S1. Obtain the database type and version in the user code library file; S2. According to the database type and version, the corresponding database keywords are searched from the preset database code keyword list and word segmentation enhancement is performed; S3, based on the enhanced database code keywords, filter out files containing keywords from the user code library files and mark them as program files containing database codes; S4, constructing basic prompt words for the large language model based on the database code keywords, adding new database query syntax to the basic prompt words, generating new large model prompt word information and inputting it into the preset large language model; S5. activating the large language model, and having the large language model extract corresponding database codes from the program file containing the database codes based on the large model prompt word information, and outputting the codes.

[0030] like Figure 1 As shown in the operation flow chart, the present invention mainly inputs the user code library file into the system, trains the large language model through the constructed large model prompt word information, and the large language model automatically searches the library based on the prompt words. The large model returns the database code (file) obtained by searching the library and outputs it.

[0031] Specifically: The present invention obtains the user code library file, extracts the database configuration information, connects to the database to obtain the database type and version. Then, according to the database type and version, extract the database keywords and supplementary grammar definitions. Use the large language model to traverse all program files, and quickly filter out program files containing database codes through database keywords. Build the corresponding large model prompt word information through keywords and input it into the large language model, let the large language model traverse all files containing database codes, send the database type, version and supplementary grammar to the large model, and extract the database code content. Therefore, based on the database code extraction method based on the AI ​​large language model, users can automatically extract database codes from the software file code library more accurately and efficiently, and apply it to database code review, release, and database type code conversion scenarios.

[0032] The implementation steps of the present invention will be described in detail below.

[0033] Further, S1, obtaining the database type and version in the user code library file, including: Input the user code library file; Parse the user code library file to obtain all program files; Traverse each program file to see if it contains a database configuration file: If not, traverse the next one until all program files are checked; If yes, the database configuration file is read and parsed to obtain basic data corresponding to the database type and version, thereby obtaining the database type and version.

[0034] like Figure 2 As shown, according to the program code package uploaded by the user or the code library address provided, all program code files are extracted. The configuration files are obtained by parsing the files.

[0035] Traverse each code package to find the database configuration file (the information in the configuration file includes: type, connection protocol, connection address, login name, password), extract the database configuration file information, parse the configuration file format, extract the database type and connection information of the database link address, and then write code based on the information, connect to the database, and obtain the database type and version used by the program code (that is, obtain the basic data corresponding to the database type and version, including the corresponding keywords and feature syntax).

[0036] The format of the user code library file is determined by the user. If it is a code library address, the program code file is obtained by linking based on the address.

[0037] Further, S2, according to the database type and version, the corresponding database keywords are searched from the preset database code keyword list and word segmentation enhancement processing is performed, including: Pre-building the database code keyword list constructed by different versions of databases and corresponding keywords; According to the database type and version, the corresponding database code keyword is searched from the preset database code keyword list and input into the preset Bloom filter to generate a Bloom filter data structure for data retrieval; The word segmentation processing technology is used to identify and extract the word segments in the code file content of the user code library file, and the word segments are written into the Bloom filter data structure and input into the Bloom filter.

[0038] Here, we need to combine keywords to search for target files. In order to search accurately and in a wider range, we use Bloom filter to search and quickly identify files with database codes.

[0039] like Figure 3 As shown, in order to quickly match the keywords corresponding to different database types and versions, a database code keyword list is created here (consisting of database type, version, and keyword. Several keywords are set for different database types and versions, which can be customized by users) to quickly find the keywords corresponding to the database version. Therefore, you can find the corresponding database code keyword from the preset database code keyword list according to the database type and version.

[0040] Word segmentation processing technology, such as the lex lexical parsing algorithm, has the following steps for word segmentation extraction: Lexical Analysis Input Buffer; Token Initialization Character Reading; Token Recognition Token Conversion Output token (Token Output); Error Handling.

[0041] Therefore, the lex lexical parsing algorithm can be used here to identify and extract the word segments in the code file content of the user code library file, including: Traverse the user code library file and scan each character; Match the longest word according to the dictionary; Add the matched words to the word list; Remove the matched words and continue to segment the remaining text; Repeat steps 3-5 until the entire text is segmented.

[0042] The Bloom filter data structure can be automatically generated by the Bloom filter data format and template.

[0043] Further, S3, based on the enhanced database code keywords, filtering out files containing keywords from the user code library files and marking them as program files containing database codes, including: Traverse all program files; Perform word segmentation analysis on each program file to obtain the word segmentation of the corresponding program file; Use Bloom filters to match database keywords for each program file: If a match is found, the matched file is represented as a program file containing the database code; determine whether all traversals are completed: if so, traverse all program files marked as containing the database code and proceed to the next step; if not, continue traversing; If there is no match, determine whether all traversals are completed: if so, traverse all program files marked as containing database codes and proceed to the next step; if there are other program files that have not been traversed, continue traversing; The program file containing the database code is saved in a software file code library.

[0044] When doing specific operations: Find the database code keyword list prepared in advance by database type and version. For reference of various database keyword lists Figure 3 Example: Generate a Bloom filter data structure with all keywords corresponding to the database version.

[0045] like Figure 4 As shown, first start traversing all program files, perform word segmentation analysis on each traversed program file, obtain the word segmentation of the corresponding program file (word segmentation is performed on the code file content, and a lex lexical analysis algorithm or tool can be used here to obtain all word segments), and each word segmentation is quickly matched with the database code keyword through the Bloom filter; if any database keyword is found to match, the file is identified as a file containing database code and proceeds to the next step; if no match is found, continue traversing until all traversals are completed; If there is no match, determine whether all traversals are completed: if so, traverse all program files marked as containing database codes and proceed to the next step; if not, continue traversing; The program file containing the database code is saved in a software file code library.

[0046] Next, we will combine keywords to build a large model prompt word.

[0047] Further, S4, constructing basic prompt words for the large language model based on the database code keywords, adding new database query syntax to the basic prompt words, generating new large model prompt word information and inputting it into the preset large language model, including: Preparing basic prompt words of a large language model, wherein the basic prompt words are constructed based on the database type and version in the database code keywords; Based on the basic prompt words, construct corresponding database query grammar; The database query grammar is organized to generate the large model prompt word information and input it into the large language model.

[0048] This application needs to use a large language model (or large model), traverse all files marked as containing database codes, organize the large model prompt word information and send it to the large language model to extract the database code. Specifically: Prepare basic prompt words to be sent to the large language model. The prompt words contain database type and version information, which can enable the large language model to extract content more accurately (for specific information that can be included in the prompt words, please refer to Figure 5 "Database type, version, new syntax, program files, extraction instructions, return format requirements, etc." as shown in .

[0049] For new database types or versions, the new database supplementary syntax is added to the prompt word to tell the large language model the new corpus information. The supplementary syntax of various databases is prepared in advance. For details, see the following description of adding new database query syntax to the large language model.

[0050] The large language model can be based on the database query syntax constructed from keywords such as database type and version in the prompt word, and filter files that meet the prompt word characteristics from all program files, and use them as target files (program files containing database code). Therefore, the model can be used to filter files to replace manual work and improve work efficiency.

[0051] In the process of software development, database types and versions change rapidly. Therefore, the present invention proposes a database grammar supplement format that allows the large language model to better understand, automatically updating prompt words according to the database type and version, so as to accurately identify. Specifically, when the large language model is used to traverse the program file in the future, it is checked in real time whether a new database type and version appears in the dynamically input user code library file: If so, based on the new database type and version, a new database query grammar is constructed and added to the large language model; If it doesn't exist, give up.

[0052] Example of the database syntax definition supplement format: [ { "COMMAND_KEY":"CREATE EVENT", "COMMAND_DATABASE_TYPE":"MYSQL", "COMMAND_DATABASE_VERSION":"5.1" "COMMAND_GRAMMAR":"CREATE EVENT {event name} ON SCHEDULE {schedule} Do {event body}" },{ "COMMAND_KEY":"CREATE_PLUGGIN_DATABASE", "COMMAND_DATABASE_TYPE":"ORACLE", "COMMAND_DATABASE_VERSION":"12" "COMMAND_GRAMMAR":"CREATE_PLUGGABLE_DATABASE{pdb_database_name} FILES {file_ path1,file path2,...}"} ].

[0053] In the process of database code extraction by the large language model, the system can, through step S1, dynamically monitor and automatically identify the database type and version in real time, search for corresponding database keywords through the database version and type, and efficiently identify program files containing database codes from all program files. Through the combined large language model prompt words, the database type version and additional database syntax information are added, so that the large language model can extract the database code content more accurately.

[0054] The large language model has a token length limit. For very large program files, the present invention calculates the token length, performs lexical segmentation, and sends the file segments to the large language model, thereby achieving the ability to completely extract very large program files.

[0055] Furthermore, before S5, activating the large language model, and extracting and outputting the corresponding database code from the program file containing the database code based on the large model prompt word information by the large language model, the method further includes: Traverse all program files identified as containing database code; Calculate the file size of each program file containing the database code: token length: , B is the token length, n is the number of text characters in the program file, Lsi is the word length of each text; Determine whether the file size of the program file containing the database code: token length exceeds the token limit of the large language model: If yes, the program file containing the database code is fragmented to obtain a plurality of program fragment files and input them into the large language model; If not, the program file containing the database code is directly input into the large language model.

[0056] like Figure 5 As shown, when sending each program file to the large language model, it is necessary to perform smooth transmission of the program file according to the token limit of the large language model. If the token size of the program file exceeds the limit of the large language model, the program file that exceeds the limit needs to be fragmented. Several program fragment files of the program file are input into the large language model (each program fragment file needs to be marked with the attributes and numbers of the original program file, so as to facilitate subsequent collection based on the data of the large language model).

[0057] The present invention provides for the calculation of the program file size: token length. Due to the different types of large language models, the algorithms may differ. Please determine the definition based on the selected large language model, for example: For ChatGPT\Llama and other English large model algorithms are as follows: 1 English word = 1, 1 Chinese character = 1.5, 1 punctuation mark = 1, 1 number = 0.5; The algorithms for the Chinese large models such as Wenxin Yiyan\Tongyi Qianwen are as follows: 1 English word = 1.5, 1 Chinese character = 1, 1 punctuation mark = 1, 1 number = 0.5.

[0058] The steps to send a file in pieces are as follows: File sharding algorithm steps reference: 1. Read the token length limit for each interaction of the large language model. The limit for each large model is different, usually 4K~128K; 2. Read the file contents; 3. Calculate the token length of each word in the file according to the previous token length calculation method, and then cut it into several shards according to the token length limit of each interaction of the large model. The size of each shard does not exceed the token length limit of each interaction of the large model. 4. The large language model performs extraction operations on each shard in turn until all shards are completed.

[0059] The present invention calculates the token length, performs lexical segmentation, and sends file segments to a large language model, thereby achieving the ability to completely extract very large program files.

[0060] Further, S5, activating the large language model, the large language model extracting and outputting the corresponding database code from the program file containing the database code based on the large model prompt word information, including: issuing an extraction instruction to the large language model to activate the large language model; Extracting and returning corresponding database codes from the program file containing database codes based on the large model prompt word information through the large language model; The database code is parsed and saved as a database code in a corresponding structured format.

[0061] like Figure 5As shown, an extraction instruction is sent to the large language model, requiring the large language model to return the database code content in the program file according to the large model prompt word information prepared above (the organized large language model prompt words, such as "database type, version, new syntax, program file, extraction instruction, return format requirements"). After the large language model retrieves the corresponding extraction result, it parses the content returned by the large language model according to the requirements in the prompt words, generates json-like structured content, and each database SQL code is a record, which is saved in the result.sql file. This is the final file content of the extracted database code. It can be used for database code review, release, or code rewriting and replacement.

[0062] The following is a general Java language code example: import java.sql.*; public class demo{ public static void updateproduct(connection connection,int id,stringname,string type){String sql="update product p set name=?,type=?"; if(type!=null){ sql=sql+",type=?" } sql=sql+" where id=?" PrepareStatementstatement =connection.preparestatement(sql); statement.setstring(1,name); if(type!=nu1l){ statement.setstring(2,type); statement.setInt(3,id); }else{ statement.setInt(2,id); } statement.execute(sql); statement.close(); } public static void deleteProduct(connection connection,int id){ String sql="delete product p"; if(id!=nul1){ sql=sql+" where id.?" }else{ throw new Exception("error,id is empty!") } Preparestatement statement = connection.preparestatement(sql); statement.setInt(1,id); statement.execute(sql); statement.close(); }. An example of a database code result quickly screened by the large language model of the present invention: [ {"sql":"update product p set name=?,type=? where id=?","line":5}, {"sql":"delete product p where id=?","line":22} ].

[0063] Therefore, in contrast, the present invention enables the large language model to better understand the database syntax supplement format, automates the database type and version accurate identification solution, and provides a method for quickly screening and filtering database code files, which can effectively solve the problem of low accuracy and efficiency in large language model database code extraction.

[0064] Figure 6 The block diagram of a database code extraction system based on a large language model for rapid screening according to an exemplary embodiment is shown. The system is used for a database code extraction method based on a large language model for rapid screening. Figure 6 The system includes a file acquisition module 610, a keyword search module 620, a file screening module 630, an intelligent screening module 640 and a code extraction module 650. Among them: The file acquisition module is used to obtain the database type and version in the user code library file; The keyword search module is used to search for corresponding database keywords from a preset database code keyword list and perform word segmentation enhancement processing according to the database type and version; A file screening module, for screening out files containing keywords from the user code library files based on the enhanced database code keywords and marking them as program files containing database codes; An intelligent screening module, used to construct basic prompt words for a large language model based on the database code keywords, and to add new database query syntax to the basic prompt words, generate new large model prompt word information and input it into a preset large language model; The code extraction module is used to activate the large language model, and the large language model extracts and outputs the corresponding database code from the program file containing the database code based on the large model prompt word information.

[0065] The functional services and interaction processes of the above-mentioned modules should be understood in conjunction with the corresponding steps and principles of the above-mentioned method, and will not be described in detail in this embodiment.

[0066] Figure 7 is a schematic diagram of the structure of an electronic device provided by an embodiment of the present invention, such as Figure 7 As shown, the electronic device may include the above Figure 6 The database code extraction system based on large language model rapid screening is shown. Optionally, the electronic device 410 may include a first processor 2001.

[0067] Optionally, the electronic device 410 may further include a memory 2002 and a transceiver 2003 .

[0068] The first processor 2001, the memory 2002 and the transceiver 2003 may be connected via a communication bus.

[0069] Combine the following Figure 7 The components of the electronic device 410 are described in detail: The first processor 2001 is the control center of the electronic device 410, and may be a processor or a general term for multiple processing elements. For example, the first processor 2001 is one or more central processing units (CPUs), or may be application specific integrated circuits (ASICs), or may be one or more integrated circuits configured to implement the embodiments of the present invention, such as one or more microprocessors (digital signal processors, DSPs), or one or more field programmable gate arrays (field programmable gate arrays, FPGAs).

[0070] Optionally, the first processor 2001 can perform various functions of the electronic device 410 by running or executing a software program stored in the memory 2002 and calling data stored in the memory 2002 .

[0071] In a specific implementation, as an embodiment, the first processor 2001 may include one or more CPUs, such as Figure 7 CPU0 and CPU1 are shown in FIG.

[0072] In a specific implementation, as an embodiment, the electronic device 410 may also include multiple processors, such as Figure 7 The first processor 2001 and the second processor 2004 are shown in FIG. Each of these processors may be a single-core processor (single-CPU) or a multi-core processor (multi-CPU). The processor here may refer to one or more devices, circuits, and / or processing cores for processing data (e.g., computer program instructions).

[0073] The memory 2002 is used to store the software program for executing the solution of the present invention, and is controlled to be executed by the first processor 2001. The specific implementation method can refer to the above method embodiment, which will not be repeated here.

[0074] Optionally, the memory 2002 may be a read-only memory (ROM) or other types of static storage devices that can store static information and instructions, a random access memory (RAM) or other types of dynamic storage devices that can store information and instructions, or an electrically erasable programmable read-only memory (EEPROM), a compact disc read-only memory (CD-ROM) or other optical disc storage, optical disc storage (including compressed optical disc, laser disc, optical disc, digital versatile disc, Blu-ray disc, etc.), a magnetic disk storage medium or other magnetic storage device, or any other medium that can be used to carry or store the desired program code in the form of instructions or data structures and can be accessed by a computer, but is not limited thereto. The memory 2002 may be integrated with the first processor 2001, or may exist independently and access the first processor 2001 through the interface circuit ( Figure 7 (not shown) is coupled to the first processor 2001, which is not specifically limited in this embodiment of the present invention.

[0075] The transceiver 2003 is used to communicate with a network device or a terminal device.

[0076] Optionally, the transceiver 2003 may include a receiver and a transmitter ( Figure 7 The receiver is used to implement a receiving function, and the transmitter is used to implement a sending function.

[0077] Optionally, the transceiver 2003 may be integrated with the first processor 2001, or may exist independently and communicate with the first processor 2001 through the interface circuit ( Figure 7 (not shown) is coupled to the first processor 2001, which is not specifically limited in this embodiment of the present invention.

[0078] It should be noted that Figure 7 The structure of the electronic device 410 shown in the figure does not constitute a limitation on the router, and the actual knowledge structure recognition device may include more or fewer components than those shown in the figure, or combine certain components, or arrange the components differently.

[0079] In addition, the technical effects of the electronic device 410 can refer to the technical effects of the database code extraction method based on rapid screening of a large language model described in the above method embodiment, which will not be repeated here.

[0080] It should be understood that the first processor 2001 in the embodiment of the present invention may be a central processing unit (CPU), and the processor may also be other general-purpose processors, digital signal processors (DSP), application specific integrated circuits (ASIC), field programmable gate arrays (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. A general-purpose processor may be a microprocessor or the processor may also be any conventional processor, etc.

[0081] It should also be understood that the memory in the embodiments of the present invention may be a volatile memory or a non-volatile memory, or may include both volatile and non-volatile memories. Among them, the non-volatile memory may be a read-only memory (ROM), a programmable read-only memory (PROM), an erasable programmable read-only memory (EPROM), an electrically erasable programmable read-only memory (EEPROM), or a flash memory. The volatile memory may be a random access memory (RAM), which is used as an external cache. By way of example and not limitation, many forms of random access memory (RAM) are available, such as static RAM (SRAM), dynamic random access memory (DRAM), synchronous DRAM (SDRAM), double data rate synchronous dynamic random access memory (DDR SDRAM), enhanced synchronous dynamic random access memory (ESDRAM), synchronous link DRAM (SLDRAM), and direct rambus RAM (DR RAM).

[0082] The above embodiments can be implemented in whole or in part by software, hardware (such as circuits), firmware or any other combination. When implemented by software, the above embodiments can be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer instructions or computer programs. When the computer instructions or computer programs are loaded or executed on a computer, the process or function described in the embodiment of the present invention is generated in whole or in part. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable system. The computer instructions can be stored in a computer-readable storage medium, or transmitted from one computer-readable storage medium to another computer-readable storage medium. For example, the computer instructions can be transmitted from one website, computer, server or data center to another website, computer, server or data center by wired (such as infrared, wireless, microwave, etc.). The computer-readable storage medium can be any available medium that can be accessed by a computer or a data storage device such as a server or data center that contains one or more available media sets. The available medium can be a magnetic medium (for example, a floppy disk, a hard disk, a tape), an optical medium (for example, a DVD), or a semiconductor medium. The semiconductor medium can be a solid-state hard disk.

[0083] It should be understood that the term "and / or" in this article is only a description of the association relationship of associated objects, indicating that there can be three relationships. For example, A and / or B can represent: A exists alone, A and B exist at the same time, and B exists alone. A and B can be singular or plural. In addition, the character " / " in this article generally indicates that the associated objects before and after are in an "or" relationship, but it may also indicate an "and / or" relationship. Please refer to the context for specific understanding.

[0084] In the present invention, "at least one" means one or more, and "plurality" means two or more. "At least one of the following" or similar expressions refers to any combination of these items, including any combination of single items or plural items. For example, at least one of a, b, or c can mean: a, b, c, ab, ac, bc, or abc, where a, b, and c can be single or multiple.

[0085] It should be understood that in various embodiments of the present invention, the size of the serial numbers of the above-mentioned processes does not mean the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of the present invention.

[0086] Those skilled in the art will appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of the present invention.

[0087] Those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working processes of the above-described devices, systems and units can refer to the corresponding processes in the aforementioned method embodiments and will not be repeated here.

[0088] In the several embodiments provided by the present invention, it should be understood that the disclosed devices, systems and methods can be implemented in other ways. For example, the system embodiments described above are only schematic. For example, the division of the units is only a logical function division. There may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another device, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of the system or unit, which can be electrical, mechanical or other forms.

[0089] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0090] In addition, each functional unit in each embodiment of the present invention may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit.

[0091] If the functions are implemented in the form of software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, or the part that contributes to the prior art or the part of the technical solution, can be embodied in the form of a software product. The computer software product is stored in a storage medium, including several instructions for enabling a computer device (which can be a personal computer, server, or network device, etc.) to perform all or part of the steps of the methods described in various embodiments of the present invention. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM), random access memory (RAM), disk or optical disk, and other media that can store program codes.

[0092] The above is only a specific embodiment of the present invention, but the protection scope of the present invention is not limited thereto. Any person skilled in the art can easily think of changes or substitutions within the technical scope disclosed by the present invention, which should be included in the protection scope of the present invention. Therefore, the protection scope of the present invention should be based on the protection scope of the claims.

Claims

1. A database code extraction method based on rapid screening of a large language model, characterized in that: The method comprises: S1. Obtain the database type and version in the user code library file; S2. According to the database type and version, the corresponding database keywords are searched from the preset database code keyword list and word segmentation enhancement is performed; S3. Based on the enhanced database code keywords, files containing keywords are screened out from the user code library files and marked as program files containing database codes; S4, constructing basic prompt words for the large language model based on the database code keywords, adding new database query syntax to the basic prompt words, generating new large model prompt word information and inputting it into the preset large language model; S5. activating the large language model, and having the large language model extract corresponding database codes from the program file containing the database codes based on the large model prompt word information, and outputting the codes.

2. The database code extraction method based on large language model rapid screening according to claim 1 is characterized in that: S1. Get the database type and version in the user code library file, including: Input the user code library file; Parse the user code library file to obtain all program files; Traverse each program file to see if it contains a database configuration file: If not, traverse the next one until all program files are checked; If yes, the database configuration file is read and parsed to obtain basic data corresponding to the database type and version, thereby obtaining the database type and version.

3. The database code extraction method based on large language model rapid screening according to claim 1 is characterized in that: S2. According to the database type and version, the corresponding database keywords are searched from the preset database code keyword list and word segmentation enhancement processing is performed, including: Pre-building the database code keyword list constructed by different versions of databases and corresponding keywords; According to the database type and version, the corresponding database code keyword is searched from the preset database code keyword list and input into the preset Bloom filter to generate a Bloom filter data structure for data retrieval; The word segmentation processing technology is used to identify and extract the word segments in the code file content of the user code library file, and the word segments are written into the Bloom filter data structure and input into the Bloom filter.

4. The database code extraction method based on large language model rapid screening according to claim 3 is characterized in that: S3. Based on the enhanced database code keywords, files containing keywords are screened out from the user code library files and marked as program files containing database codes, including: Traverse all program files; Perform word segmentation analysis on each program file to obtain the word segmentation of the corresponding program file; Use Bloom filters to match database keywords for each program file: If it is found that the program file matches the keyword, the program file matching the keyword is identified as a program file containing the database code; If no matching keywords are found in the program file, proceed to the next step; Determine whether all the program files have been traversed: If so, all traversals are completed and proceed to the next step; If there are other program files that have not been traversed, continue to traverse the program files; The program file containing the database code is saved in a software file code library.

5. The method for extracting database codes based on rapid screening of a large language model according to claim 4 is characterized in that: S4, constructing basic prompt words for the large language model based on the database code keywords, adding new database query syntax to the basic prompt words, generating new large model prompt word information and inputting it into the preset large language model, including: Preparing basic prompt words of a large language model, wherein the basic prompt words are constructed based on the database type and version in the database code keywords; Based on the basic prompt words, construct corresponding database query grammar; Organizing the database query grammar, generating the large model prompt word information and inputting it into the large language model; When the large language model is used to traverse the program files later, it is checked in real time whether a new database type and version appears in the dynamically input user code library file: If so, based on the new database type and version, a new database query grammar is constructed and added to the large language model; If it doesn't exist, give up.

6. The method for extracting database codes based on rapid screening of a large language model according to claim 4, characterized in that: Before S5, activating the large language model, and extracting and outputting the corresponding database code from the program file containing the database code based on the large model prompt word information by the large language model, the method further includes: Traverse all program files identified as containing database code; Calculate the file size of each program file containing the database code: token length: ， B is the token length, n is the number of text characters in the program file, Lsi is the word length of each text; Determine whether the file size of the program file containing the database code: token length exceeds the token limit of the large language model: If yes, the program file containing the database code is fragmented to obtain a plurality of program fragment files and input them into the large language model; If not, the program file containing the database code is directly input into the large language model.

7. The method for extracting database codes based on rapid screening of a large language model according to claim 6, characterized in that: S5, activating the large language model, and having the large language model extract corresponding database codes from the program file containing the database codes based on the large model prompt word information and outputting the codes, including: issuing an extraction instruction to the large language model to activate the large language model; Extracting and returning the corresponding database code from the program file containing the database code based on the large model prompt word information through the large language model; The database code is parsed and saved as a database code in a corresponding structured format.

8. A database code extraction system based on rapid screening of a large language model, the database code extraction system based on rapid screening of a large language model is used to implement the database code extraction method based on rapid screening of a large language model as described in any one of claims 1 to 7, characterized in that: The system comprises: The file acquisition module is used to obtain the database type and version in the user code library file; The keyword search module is used to search for corresponding database keywords from a preset database code keyword list and perform word segmentation enhancement processing according to the database type and version; A file screening module, for screening out files containing keywords from the user code library files based on the enhanced database code keywords and marking them as program files containing database codes; An intelligent screening module, used to construct basic prompt words for a large language model based on the database code keywords, and to add new database query syntax to the basic prompt words, generate new large model prompt word information and input it into a preset large language model; The code extraction module is used to activate the large language model, and the large language model extracts and outputs the corresponding database code from the program file containing the database code based on the large model prompt word information.

9. An electronic device, characterized in that: The electronic device comprises: processor; A memory having computer-readable instructions stored thereon, wherein when the computer-readable instructions are executed by the processor, the method according to any one of claims 1 to 7 is implemented.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores program codes, which can be called by a processor to execute the method according to any one of claims 1 to 7.

Images