ChatGLM model-based national capital supervision knowledge base optimization system and method

Abstract

The invention is suitable for the technical field of the Internet, and provides a ChatGLM model-based national capital supervision knowledge base optimization system and a ChatGLM model-based national capital supervision knowledge base optimization method. In the embodiment, after the first national capital supervision file is obtained, the first national capital supervision file is converted into the target data through the intermediate scanner and the data converter, the target data are filed into the memory, and then the national capital supervision knowledge base is created through the target data. And after a second national capital supervision file used for updating the national capital supervision knowledge base is obtained regularly or in real time, updating the second national capital supervision file. When the query request is received, the target data most matched with the query request is determined through the national capital supervision knowledge base, and the target data is input into the ChatGLM model, so that the summary information is determined, and the processing efficiency and accuracy of the ChatGLM model in the field of national capital supervision are improved.

Description

Technical Field

[0001] The present application relates to the field of Internet technology, and in particular to a state-owned assets supervision knowledge base optimization system and method based on the ChatGLM model. Background Art

[0002] With the rapid development of financial technology, the application of intelligent conversational big models in the fields of state-owned assets and financial supervision has been put forward in richer scenarios and higher requirements. As an advanced intelligent conversational big model, ChatGLM has powerful language processing capabilities, but it still has shortcomings in the processing of state-owned assets and financial supervision knowledge. Summary of the invention

[0003] In view of this, the present application provides a state-owned assets supervision knowledge base optimization system and method based on the ChatGLM model to improve the processing efficiency and accuracy of the ChatGLM model in the field of state-owned assets supervision, so as to meet the compliance management needs of modern state-owned financial enterprises and solve current technical problems.

[0004] The first aspect of the present application provides a state-owned assets supervision knowledge base optimization system based on the ChatGLM model, the system comprising a state-owned assets supervision knowledge base construction module, a state-owned assets supervision knowledge base update module and a data call module;

[0005] The state-owned assets supervision knowledge base construction module is used to obtain the first state-owned assets supervision file, convert the first state-owned assets supervision file into a target file in a preset format, and save the target file and corresponding file information to the data collection space, wherein the file information includes the file source and the update channel source;

[0006] Scan the data acquisition space by an intermediate scanner, obtain the target file and send it to the data converter;

[0007] The data converter performs processing and conversion according to the file type and file content of the target file, generates target data and sends it to the archive storage;

[0008] The archiving storage creates the state-owned assets supervision knowledge base through the target data;

[0009] The state-owned assets supervision knowledge base update module is used to obtain a second state-owned assets supervision file for updating, and through the intermediate scanner, the data converter generates second target data for updating the state-owned assets supervision knowledge base and sends it to the archive memory, and the archive memory updates the state-owned assets supervision knowledge base through the second target data;

[0010] The data calling module is used to, when receiving a query request, query the state-owned assets supervision knowledge base according to the query information in the query request, determine the target data with the highest matching degree with the query information and input it into the ChatGLM model, and output summary information through the ChatGLM model.

[0011] Optionally, the state-owned assets supervision knowledge base update module also includes a timer, which is used to set the collection time of the second state-owned assets supervision document. When the collection time is reached, the second state-owned assets supervision document is collected from the target website or target channel, and the collection time of the timer is updated after the collection.

[0012] Optionally, the state-owned assets supervision knowledge base update module detects in real time whether there is an increase in the state-owned assets supervision documents on the target website or target channel. If an increase is found, the increased state-owned assets supervision documents are downloaded from the target website or target channel and determined as the second state-owned assets supervision documents.

[0013] Optionally, the processing and conversion according to the file type and file content of the target file in the state-owned assets supervision knowledge base construction module includes:

[0014] When it is determined that the target file is a pure data file, the target file is stored in groups and batches;

[0015] When it is determined that the target file is not a pure data file, the target file is divided into a word group and a pdf group according to the file type;

[0016] The target files in the word group are grouped into a word document set, each document in the word document set is traversed, and after being read into the memory to obtain a word document object, the data structure is split according to a specific format or paragraph, the split data content is assembled into data rows, and the data rows are grouped and stored in batches;

[0017] Convert each target file in the PDF group into word format, and group the converted target files into a word document set, traverse each document in the word document set, read it into memory to obtain a word document object, split the data structure according to a specific format or paragraph, assemble the split data content into data rows, and store the data rows in groups and batches.

[0018] Optionally, the determining of the target data with the highest matching degree with the query information in the data calling module includes:

[0019] The target data that hit the query information are determined as initial target data, each initial target data is scored and sorted according to the number of times it hits the query information, and the initial target data with the highest score is determined as the target data with the highest matching degree with the query information.

[0020] The second aspect of the present application provides a method for optimizing a state-owned assets supervision knowledge base based on a ChatGLM model, the method comprising:

[0021] Obtaining a first state-owned assets supervision file, converting the first state-owned assets supervision file into a target file in a preset format, and saving the target file and corresponding file information to a data collection space, wherein the file information includes a file source and an update channel source;

[0022] Scan the data collection space to obtain the target file, process and convert the target file according to its file type and content, generate target data, and create the state-owned assets supervision knowledge base with the target data;

[0023] Acquire a second state-owned assets supervision file for updating, use the second state-owned assets supervision file to update the second target data of the state-owned assets supervision knowledge base, and update the state-owned assets supervision knowledge base through the second target data;

[0024] When a query request is received, a query is performed in the state-owned assets supervision knowledge base according to the query information in the query request, the target data with the highest matching degree with the query information is determined and input into the ChatGLM model, and summary information is output through the ChatGLM model.

[0025] Optionally, obtaining the second state-owned assets supervision document for updating includes:

[0026] The collection time of the second state-owned assets supervision document is set by a timer. When the collection time is reached, the second state-owned assets supervision document is collected from a target website or a target channel, and the collection time of the timer is updated after the collection.

[0027] Optionally, obtaining the second state-owned assets supervision document for updating includes:

[0028] Real-time detection is performed to determine whether the state-owned assets supervision documents of the target website or target channel have been increased. If an increase is found, the increased state-owned assets supervision documents are downloaded from the target website or target channel and determined as the second state-owned assets supervision document.

[0029] Optionally, processing and converting according to the file type and file content of the target file includes:

[0030] When it is determined that the target file is a pure data file, the target file is stored in groups and batches;

[0031] When it is determined that the target file is not a pure data file, the target file is divided into a word group and a pdf group according to the file type;

[0032] The target files in the word group are grouped into a word document set, each document in the word document set is traversed, and after being read into the memory to obtain a word document object, the data structure is split according to a specific format or paragraph, the split data content is assembled into data rows, and the data rows are grouped and stored in batches;

[0033] Convert each target file in the PDF group into word format, and group the converted target files into a word document set, traverse each document in the word document set, read it into memory to obtain a word document object, split the data structure according to a specific format or paragraph, assemble the split data content into data rows, and store the data rows in groups and batches.

[0034] Optionally, the determining the target data having the highest matching degree with the query information includes:

[0035] The target data that hit the query information are determined as initial target data, each initial target data is scored and sorted according to the number of times it hits the query information, and the initial target data with the highest score is determined as the target data with the highest matching degree with the query information.

[0036] In the embodiment provided in the present application, after the first state-owned assets supervision document is obtained, it is converted into target data through an intermediate scanner and a data converter, and then the archive memory creates a state-owned assets supervision knowledge base through the target data. After the second state-owned assets supervision document for updating the state-owned assets supervision knowledge base is obtained in a scheduled or real-time manner, it is updated. When a query request is received, the target data that best matches the query request is determined through the state-owned assets supervision knowledge base, and the target data is input into the ChatGLM model to determine the summary information, which improves the processing efficiency and accuracy of the ChatGLM model in the field of state-owned assets supervision. BRIEF DESCRIPTION OF THE DRAWINGS

[0037] Figure 1 A system module diagram provided for an embodiment of the present application;

[0038] Figure 2 A flowchart of file processing provided for an embodiment of the present application;

[0039] Figure 3 A flow chart of the method provided in the embodiment of the present application;

[0040] Figure 4 A schematic diagram of the internal structure of a computer device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0041] Exemplary embodiments will be described in detail herein, examples of which are shown in the accompanying drawings. When the following description refers to the drawings, the same numbers in different drawings represent the same or similar elements unless otherwise indicated. The implementations described in the following exemplary embodiments do not represent all implementations consistent with the present application. Instead, they are merely examples of devices and methods consistent with some aspects of the present application as detailed in the appended claims.

[0042] The terms used in this application are for the purpose of describing specific embodiments only and are not intended to limit this application. The singular forms of "a", "said" and "the" used in this application and the appended claims are also intended to include plural forms unless the context clearly indicates other meanings. It should also be understood that the term "and / or" used herein refers to and includes any or all possible combinations of one or more associated listed items.

[0043] It should be understood that although the terms first, second, third, etc. may be used in the present application to describe various information, these information should not be limited to these terms. These terms are only used to distinguish the same type of information from each other. For example, without departing from the scope of the present application, the first information may also be referred to as the second information, and similarly, the second information may also be referred to as the first information. Depending on the context, the word "if" as used herein may be interpreted as "at the time of" or "when" or "in response to determining".

[0044] This application provides a state-owned assets supervision knowledge base optimization system based on the ChatGLM model to improve the processing efficiency and accuracy of the ChatGLM model in the field of state-owned assets supervision.

[0045] The technical solution of the present application is described in detail with specific embodiments below. The following specific embodiments can be combined with each other, and the same or similar concepts or processes may not be described in detail in some embodiments.

[0046] like Figure 1 As shown, this is a module diagram of a state-owned assets supervision knowledge base optimization system based on the ChatGLM model provided in this application. The system includes a state-owned assets supervision knowledge base construction module, a state-owned assets supervision knowledge base update module and a data call module.

[0047] The functions of each module are described below:

[0048] 1. State-owned assets supervision knowledge base construction module. This module is used to build the latest state-owned assets supervision knowledge base. The specific construction method is as follows:

[0049] First, download the first state-owned assets supervision file through authoritative websites and related channels. The first state-owned assets supervision file can download all the laws and regulations in the field of state-owned assets supervision before the time. After the download is completed, the first state-owned assets supervision file is a preset format, such as a target file in PDF / WORD format, and the file information of each target file is recorded at the same time. The file information includes the file source and update channel source of the first state-owned assets supervision file corresponding to the target file. And save the target file and the corresponding file information to the data collection space.

[0050] In the second step, after the data collection is completed, the data collection space is scanned by the intermediate scanner, the target file is obtained and sent to the data converter.

[0051] In the third step, the data converter processes and converts the target file according to its file type and content to generate target data. After the target data is saved in the database, the creation of the state-owned assets supervision knowledge base can be completed.

[0052] Exemplarily, after determining the file type of the target file, a tool for reading the target file is determined according to the file type and the file is read, such as reading a file in word format through office software or wps, and reading a file in PDF format through PyPDF2;

[0053] After reading the file content, determine the paragraph separator based on the language conventions of the file content. In English, paragraphs are usually separated by two line breaks \n\n; Chinese may use one line break \n.

[0054] Use string splitting methods, such as str.split() in Python, to split the text based on the delimiter, and determine the maximum number of characters per paragraph during the splitting process. If the paragraph exceeds this limit, further splitting is required. At the same time, identify and replace or remove unnecessary line breaks, such as replacing \r\n with \n. For extra whitespace characters, you can use str.strip() to remove whitespace at the beginning and end of the string, and use str.replace(",") to remove multiple consecutive whitespace in a string.

[0055] Through the above processing, the target file is converted into target data for storage, thereby improving the retrieval efficiency.

[0056] In another embodiment, processing and converting according to the file type and file content of the target file includes:

[0057] When it is determined that the target file is a pure data file, the target file is stored in groups and batches;

[0058] When it is determined that the target file is not a pure data file, the target file is divided into a word group and a pdf group according to the file type;

[0059] The target files in the word group are grouped into a word document set, each document in the word document set is traversed, and after being read into the memory to obtain a word document object, the data structure is split according to a specific format or paragraph, the split data content is assembled into data rows, and the data rows are grouped and stored in batches;

[0060] Convert each target file in the PDF group into word format, and group the converted target files into a word document set. Iterate through each document in the word document set, read it into memory to obtain a word document object, split the data structure according to a specific format or paragraph, assemble the split data content into data rows, and group and store the data rows in batches, such as Figure 2 shown.

[0061] 2. State-owned assets supervision knowledge base update module. This module is used to update the above-mentioned state-owned assets supervision knowledge base. The specific update method is as follows:

[0062] Download the second state-owned assets supervision document through authoritative websites and related channels. The second state-owned assets supervision document is the laws and regulations in the field of state-owned assets supervision released by authoritative websites and related channels after the creation of the above-mentioned state-owned assets supervision knowledge base. After the download is completed, according to the method of processing the first state-owned assets supervision document during the creation of the state-owned assets supervision knowledge base, the second state-owned assets supervision document is converted into the second target data, and the second target data is stored in the above-mentioned state-owned assets supervision knowledge base, so as to complete the update of the above-mentioned state-owned assets supervision knowledge base.

[0063] It should be noted that when the second target data conflicts with the target data, the target data is updated by the second target data.

[0064] In another embodiment, the state-owned assets supervision knowledge base update module also includes a timer, which is used to set the collection time of the second state-owned assets supervision document. When the collection time is reached, the second state-owned assets supervision document is collected from the target website or target channel, and the collection time of the timer is updated after the collection.

[0065] This embodiment can realize the regular update of the state-owned assets supervision knowledge base.

[0066] In another embodiment, the state-owned assets supervision knowledge base update module detects in real time whether the state-owned assets supervision documents of the target website or target channel have been increased. If an increase is found, the increased state-owned assets supervision documents are downloaded from the target website or target channel and determined as the second state-owned assets supervision documents.

[0067] This embodiment can realize real-time updating of the state-owned assets supervision knowledge base, so that the state-owned assets supervision knowledge base can meet the latest needs of the state-owned assets supervision market.

[0068] 3. Data call module. This module is used to query the state-owned assets supervision knowledge base according to the query information in the query request when receiving a query request, and output the target data that matches the query information most. For example, when the query information received is a keyword, a keyword search is performed in the above-mentioned state-owned assets supervision knowledge base to obtain a list of data such as keywords, paragraphs, and files after the search, and weighted according to the number of keyword hits, and the 5-10 items with the highest scores are selected as the basis for the promote context data for the ChatGLM model to answer, so that the ChatGLM model can professionally summarize the information based on the promote data list and questions provided by the state-owned assets supervision knowledge base.

[0069] So far, completed Figure 1 System module description shown.

[0070] In an embodiment of the present application, after obtaining the first state-owned assets supervision document, it is converted into target data through an intermediate scanner and a data converter, and then the archive memory creates a state-owned assets supervision knowledge base through the target data. After the second state-owned assets supervision document used to update the state-owned assets supervision knowledge base is obtained in a regular or real-time manner, it is updated. When a query request is received, the target data that best matches the query request is determined through the state-owned assets supervision knowledge base, and the target data is input into the ChatGLM model to determine the summary information, which improves the processing efficiency and accuracy of the ChatGLM model in the field of state-owned assets supervision.

[0071] like Figure 3 As shown, the present application also provides a method for optimizing a state-owned assets supervision knowledge base based on the ChatGLM model, the method comprising:

[0072] S301, obtaining a first state-owned assets supervision file, converting the first state-owned assets supervision file into a target file in a preset format, and saving the target file and corresponding file information to a data collection space, wherein the file information includes a file source and an update channel source;

[0073] S302, scanning the data collection space, acquiring the target file, processing and converting the target file according to the file type and the file content, generating target data, and creating the state-owned assets supervision knowledge base with the target data;

[0074] S303, obtaining a second state-owned assets supervision file for updating, using the second state-owned assets supervision file to update second target data of the state-owned assets supervision knowledge base, and updating the state-owned assets supervision knowledge base through the second target data;

[0075] S304, when a query request is received, a query is performed in the state-owned assets supervision knowledge base according to the query information in the query request, the target data with the highest matching degree with the query information is determined and input into the ChatGLM model, and summary information is output through the ChatGLM model.

[0076] In another embodiment, obtaining the second state-owned assets supervision file for updating includes:

[0077] The collection time of the second state-owned assets supervision document is set by a timer. When the collection time is reached, the second state-owned assets supervision document is collected from a target website or a target channel, and the collection time of the timer is updated after the collection.

[0078] In another embodiment, obtaining the second state-owned assets supervision file for updating includes:

[0079] Real-time detection is performed to determine whether the state-owned assets supervision documents of the target website or target channel have been increased. If an increase is found, the increased state-owned assets supervision documents are downloaded from the target website or target channel and determined as the second state-owned assets supervision document.

[0080] In another embodiment, processing and converting according to the file type and file content of the target file includes:

[0081] When it is determined that the target file is a pure data file, the target file is stored in groups and batches;

[0082] When it is determined that the target file is not a pure data file, the target file is divided into a word group and a pdf group according to the file type;

[0083] The target files in the word group are grouped into a word document set, each document in the word document set is traversed, and after being read into the memory to obtain a word document object, the data structure is split according to a specific format or paragraph, the split data content is assembled into data rows, and the data rows are grouped and stored in batches;

[0084] Convert each target file in the PDF group into word format, and group the converted target files into a word document set, traverse each document in the word document set, read it into memory to obtain a word document object, split the data structure according to a specific format or paragraph, assemble the split data content into data rows, and store the data rows in groups and batches.

[0085] In another embodiment, the determining the target data with the highest matching degree with the query information includes:

[0086] The target data that hit the query information are determined as initial target data, each initial target data is scored and sorted according to the number of times it hits the query information, and the initial target data with the highest score is determined as the target data with the highest matching degree with the query information.

[0087] The above embodiment of the present invention provides a state-owned assets supervision knowledge base optimization system based on the ChatGLM model, and based on the system provides a state-owned assets supervision knowledge base optimization method based on the ChatGLM model. Through the above system and method, the processing efficiency and accuracy of the ChatGLM model in the field of state-owned assets supervision are improved.

[0088] This embodiment also discloses a computer device, such as Figure 4 As shown, the computer device includes a processor and a memory, wherein the memory stores at least one instruction, and the at least one instruction is loaded and executed by the processor to implement any of the above-mentioned state-owned assets supervision knowledge base optimization methods based on the ChatGLM model.

[0089] The above description is only a preferred embodiment of the present application and is not intended to limit the present application. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present application shall be included in the scope of protection of the present application.

Claims

1. A state-owned assets supervision knowledge base optimization system based on the ChatGLM model, characterized in that: The system includes a state-owned assets supervision knowledge base construction module, a state-owned assets supervision knowledge base update module and a data calling module; The state-owned assets supervision knowledge base construction module is used to obtain the first state-owned assets supervision file, convert the first state-owned assets supervision file into a target file in a preset format, and save the target file and corresponding file information to the data collection space, wherein the file information includes the file source and the update channel source; Scan the data acquisition space by an intermediate scanner, obtain the target file and send it to the data converter; The data converter performs processing and conversion according to the file type, file content and language habits of the target file, generates target data and sends it to the archive storage; The archiving storage creates the state-owned assets supervision knowledge base through the target data; The state-owned assets supervision knowledge base update module is used to obtain a second state-owned assets supervision file for updating, and through the intermediate scanner, the data converter generates second target data for updating the state-owned assets supervision knowledge base and sends it to the archive memory, and the archive memory updates the state-owned assets supervision knowledge base through the second target data; The data calling module is used to, when receiving a query request, query the state-owned assets supervision knowledge base according to the query information in the query request, determine the target data with the highest matching degree with the query information and input it into the ChatGLM model, and output summary information through the ChatGLM model.

2. The system according to claim 1, characterized in that The state-owned assets supervision knowledge base update module also includes a timer, which is used to set the collection time of the second state-owned assets supervision document. When the collection time is reached, the second state-owned assets supervision document is collected from the target website or target channel, and the collection time of the timer is updated after the collection.

3. The system according to claim 1, characterized in that The state-owned assets supervision knowledge base update module detects in real time whether the state-owned assets supervision documents of the target website or target channel have been increased. If an increase is found, the increased state-owned assets supervision documents are downloaded from the target website or target channel and determined as the second state-owned assets supervision documents.

4. The system according to claim 1, characterized in that The processing and conversion according to the file type and file content of the target file in the state-owned assets supervision knowledge base construction module includes: When it is determined that the target file is a pure data file, the target file is stored in groups and batches; When it is determined that the target file is not a pure data file, the target file is divided into a word group and a pdf group according to the file type; The target files in the word group are grouped into a word document set, each document in the word document set is traversed, and after being read into the memory to obtain a word document object, the data structure is split according to a specific format or paragraph, the split data content is assembled into data rows, and the data rows are grouped and stored in batches; Convert each target file in the PDF group into word format, and group the converted target files into a word document set, traverse each document in the word document set, read it into memory to obtain a word document object, split the data structure according to a specific format or paragraph, assemble the split data content into data rows, and store the data rows in groups and batches.

5. The system according to claim 1, characterized in that The target data that is determined to have the highest matching degree with the query information in the data calling module includes: The target data that hit the query information are determined as initial target data, each initial target data is scored and sorted according to the number of times it hits the query information, and the initial target data with the highest score is determined as the target data with the highest matching degree with the query information.

6. A method for optimizing the state-owned assets supervision knowledge base based on the ChatGLM model, characterized in that: The method comprises: Obtaining a first state-owned assets supervision file, converting the first state-owned assets supervision file into a target file in a preset format, and saving the target file and corresponding file information to a data collection space, wherein the file information includes a file source and an update channel source; Scan the data collection space to obtain the target file, process and convert the target file according to the file type, file content and language habits of the file content to generate target data, and the target data creates the state-owned assets supervision knowledge base; Acquire a second state-owned assets supervision file for updating, use the second state-owned assets supervision file to update the second target data of the state-owned assets supervision knowledge base, and update the state-owned assets supervision knowledge base through the second target data; When a query request is received, a query is performed in the state-owned assets supervision knowledge base according to the query information in the query request, the target data with the highest matching degree with the query information is determined and input into the ChatGLM model, and summary information is output through the ChatGLM model.

7. The method according to claim 6, characterized in that The obtaining of the second state-owned assets supervision document for updating includes: The collection time of the second state-owned assets supervision document is set by a timer. When the collection time is reached, the second state-owned assets supervision document is collected from a target website or a target channel, and the collection time of the timer is updated after the collection.

8. The method according to claim 6, characterized in that The obtaining of the second state-owned assets supervision document for updating includes: Real-time detection is performed to determine whether the state-owned assets supervision documents of the target website or target channel have been increased. If an increase is found, the increased state-owned assets supervision documents are downloaded from the target website or target channel and determined as the second state-owned assets supervision document.

9. The method according to claim 6, characterized in that The processing and conversion according to the file type and file content of the target file includes: When it is determined that the target file is a pure data file, the target file is stored in groups and batches; When it is determined that the target file is not a pure data file, the target file is divided into a word group and a pdf group according to the file type; The target files in the word group are grouped into a word document set, each document in the word document set is traversed, and after being read into the memory to obtain a word document object, the data structure is split according to a specific format or paragraph, the split data content is assembled into data rows, and the data rows are grouped and stored in batches; Convert each target file in the PDF group into word format, and group the converted target files into a word document set, traverse each document in the word document set, read it into memory to obtain a word document object, split the data structure according to a specific format or paragraph, assemble the split data content into data rows, and store the data rows in groups and batches.

10. The method according to claim 6, characterized in that The determining of the target data having the highest matching degree with the query information includes: The target data that hit the query information are determined as initial target data, each initial target data is scored and sorted according to the number of times it hits the query information, and the initial target data with the highest score is determined as the target data with the highest matching degree with the query information.

Images