Enterprise large model construction method and device, storage medium and electronic equipment

Abstract

The invention provides an enterprise large model construction method and device, a storage medium and electronic equipment, and the method comprises the steps: collecting a plurality of pieces of enterprise document data, and extracting corresponding keywords; combining the keywords, counting the occurrence frequency of each keyword, determining the keyword with the highest frequency as an enterprise data subject term, and determining the subject correlation of each piece of enterprise document data according to the matching degree between the subject term and the keyword; collecting a plurality of external document data, extracting corresponding keywords, and determining theme correlation of the external document data according to the matching degree between the subject terms and the keywords of the data; and training the artificial intelligence model by adopting the enterprise document data and theme correlation and the external document data and theme correlation to obtain a target model. According to the method, internal and external data of an enterprise are collected, quality filtering is carried out, and meanwhile, a data matching optimization method is put forward, so that when the model continues to be pre-trained, the universality and the field professional ability can be balanced.

Description

Technical Field

[0001] The present application relates to the field of financial big models, and more specifically, to a method for constructing a big enterprise model, a device for constructing a big enterprise model, a computer-readable storage medium, and an electronic device. Background Art

[0002] The artificial intelligence model product ChatGPT can understand and generate natural language, handle complex conversation tasks, and even create and program. However, it is costly to develop a large model directly from 0 to 1. The mainstream method in the industry is to build an enterprise model based on the open source large model. In order to better adapt to specific tasks or scenarios, you can use new data sets to conduct further training based on the open source pre-trained model, so that the model can better understand the task and generate enterprise-related content.

[0003] In order to better adapt to specific tasks or scenarios, new data sets can be used for further training based on open source pre-trained models, so that the model can better understand the task and generate content in the field. At present, the existing pre-training methods only use industry-related open source data, which uses too little data and does not consider the use of enterprise data. It also does not focus on the balance between general capabilities and domain expertise, which can easily lead to insufficient general capabilities. Summary of the invention

[0004] The main purpose of this application is to provide a method for constructing an enterprise big model, an enterprise big model construction device, a computer-readable storage medium and an electronic device, so as to at least solve the problem that the method in the prior art that continues to pre-train the open source big model only uses industry-related open source data, the data used is too little, and does not pay attention to the balance between general capabilities and domain professional capabilities, which easily leads to insufficient general capabilities.

[0005] In order to achieve the above-mentioned purpose, according to one aspect of the present application, a method for constructing a large enterprise model is provided, including: collecting multiple enterprise document data, and extracting keywords corresponding to each of the enterprise document data, the enterprise document data being internal data of a target enterprise with a text length less than a preset length and in a text format; merging the keywords of all the enterprise document data, counting the occurrence frequency of each of the keywords, determining the keyword with the highest occurrence frequency as the enterprise data subject word, and determining the subject relevance of each of the enterprise document data based on the matching degree between the enterprise data subject word and the keywords of each of the enterprise document data; collecting multiple external document data, extracting keywords corresponding to each of the external document data, and determining the subject relevance of each of the external document data based on the matching degree between the enterprise data subject word and the keywords of each of the external document data; using the enterprise document data, the subject relevance of the enterprise document data, the external document data, and the subject relevance of the external document data to train an artificial intelligence model to obtain a target model, and using the target model to process target tasks related to the target enterprise.

[0006] Optionally, extracting keywords corresponding to each of the enterprise document data includes: using separators to segment the enterprise document data and filtering out invalid auxiliary words to obtain initial words; determining one of the initial words in the enterprise document data as a preparatory word, and determining the quotient between the number of occurrences of the preparatory word and the number of all non-preparatory words as the frequency score of the preparatory word; determining the part-of-speech score weight corresponding to the preparatory word according to the part-of-speech of the preparatory word and a part-of-speech weight table, wherein the part-of-speech weight table represents the correspondence between words and part-of-speech weights; determining the part-of-speech score weight corresponding to the preparatory word according to the position and position of the preparatory word. A score mapping relationship is used to determine the position score corresponding to the prepared word, wherein the position score mapping relationship represents the mapping relationship between the position of a word and the position score of the word; based on the score calculation formula score=c×(a+s), the total score of the prepared word is obtained, where score is the total score of the prepared word, c is the part-of-speech score weight of the prepared word, a is the frequency score of the prepared word, and s is the position score of the prepared word; all the initial words are ranked according to the total score of each of the initial words, and the initial words ranked in the top k are determined as keywords of the enterprise document data.

[0007] Optionally, collecting multiple external document data includes: obtaining an initial external document and determining a document quality standard, wherein the document quality standard is at least used to limit document compliance, document security, document privacy and document malicious attack level, and the initial external document includes a first number of first documents and a second number of second documents; judging whether the first document meets the document quality standard, and if the first document meets the document quality standard, determining that the label of the first document is a high-quality sample label, and if the first document does not meet the document quality standard, determining that the label of the first document is a low-quality sample label; training a FastText algorithm model based on the first document and the corresponding label to obtain a text quality classifier; using the text quality classifier to classify the second document to obtain a predicted quality score, filtering the second document whose predicted quality score is less than a quality score limit, and determining the second document whose predicted quality score is greater than or equal to the quality score limit as the external document data.

[0008] Optionally, before filtering the second document whose predicted quality score is less than the quality score limit, the method further includes: using the equal interval method to determine a candidate threshold set, the candidate threshold set including multiple candidate score limits, and the difference between two adjacent candidate score limits in the candidate threshold set is a preset difference; using each of the candidate score limits in the candidate threshold set to screen multiple training document data in turn, and recording the predicted case quality of each of the training document data corresponding to the candidate score limit; using the true interest rate calculation formula TPR=TP / (TP+FN) to calculate the true interest rate of the candidate score limit, and using the false positive rate calculation formula FPR=FP / (FP+TN) to calculate the false positive rate of the candidate score limit. , wherein TPR is the true positive rate of the candidate score limit, TP is the number of training document data whose predicted case quality is high and whose true case quality is also high, FN is the number of training document data whose predicted case quality is low and whose true case quality is high, FPR is the false positive rate of the candidate score limit, FP is the number of training document data whose predicted case quality is high and whose true case quality is low, and TN is the number of training document data whose predicted case quality is low and whose true case quality is low; the difference between the true positive rate and the false positive rate of the candidate score limit is determined as the score of the candidate score limit, and the candidate score limit with the largest score is determined as the quality score limit.

[0009] Optionally, the enterprise document data, the subject relevance of the enterprise document data, the external document data and the subject relevance of the external document data are used to train the artificial intelligence model to obtain a target model, including: sorting the enterprise document data in ascending order of relevance and then performing equal frequency bucketing to obtain a plurality of enterprise internal data buckets, and sorting the external document data in ascending order of relevance and then performing equal frequency bucketing to obtain a plurality of enterprise external data buckets, the number of document data in each bucket is the same, and the number of the enterprise internal data buckets is the same as the number of the enterprise external data buckets; determining the document number array of each of the enterprise external data buckets at different training stages and the number of each of the enterprise internal data buckets at different training stages An array of the number of documents, each data in the array of the number of documents respectively represents the number of training samples extracted from the data bucket in different training stages, and the number of training stages is an integer multiple of the number of internal data buckets of the enterprise; according to the array of the number of documents of the internal data bucket of the enterprise, a first preset number of the enterprise document data corresponding to the current training stage is extracted from the internal data bucket of the enterprise, and according to the array of the number of documents of the external data bucket of the enterprise, a second preset number of the external document data corresponding to the current training stage is extracted from the external data bucket of the enterprise to constitute the training samples of the current training stage; the training samples corresponding to each training stage are randomly scattered and trained in sequence to obtain the target model.

[0010] Optionally, there are F levels of training stages, the serial number of the enterprise external data bucket is proportional to the correlation of the document data in the enterprise external data bucket, and the document quantity array of each enterprise external data bucket in different training stages is determined, including: determining that the number of training samples of the first enterprise external data bucket in the Nth level training stage is F-N+1; determining that the number of training samples of the Fth enterprise external data bucket in the Nth level training stage is N; determining that the number of training samples of the pth enterprise external data bucket in the Nth level training stage is F-h+1-N, where h=(p-1)×F / p.

[0011] Optionally, the subject relevance of each of the enterprise document data is determined based on the degree of match between the enterprise data subject words and the keywords of each of the enterprise document data, including: determining the number of identical keywords of the enterprise document data and the enterprise data subject words as the number of identical words; determining the subject relevance of the enterprise document data based on the correlation calculation formula T=t / k, wherein T is the subject relevance of the enterprise document data, t is the number of identical words, and k is the total number of keywords of the enterprise document data.

[0012] According to another aspect of the present application, a device for constructing a large enterprise model is provided, comprising: a first processing unit, for collecting a plurality of enterprise document data, and extracting keywords corresponding to each of the enterprise document data, wherein the enterprise document data is internal data of a target enterprise with a text length less than a preset length and in a text format; a second processing unit, for merging the keywords of all the enterprise document data, counting the occurrence frequency of each of the keywords, determining the keyword with the highest occurrence frequency as the enterprise data subject word, and determining the subject relevance of each of the enterprise document data according to the matching degree between the enterprise data subject word and the keywords of each of the enterprise document data; a third processing unit, for collecting a plurality of external document data, extracting keywords corresponding to each of the external document data, and determining the subject relevance of each of the external document data according to the matching degree between the enterprise data subject word and the keywords of each of the external document data; and a fourth processing unit, for training an artificial intelligence model using the enterprise document data, the subject relevance of the enterprise document data, the external document data, and the subject relevance of the external document data to obtain a target model, and using the target model to process target tasks related to the target enterprise.

[0013] According to another aspect of the present application, a computer-readable storage medium is provided, wherein the computer-readable storage medium includes a stored program, wherein when the program is executed, the device where the computer-readable storage medium is located is controlled to execute any one of the methods for constructing a large enterprise model.

[0014] According to another aspect of the present application, an electronic device is provided, comprising: one or more processors, a memory, and one or more programs, wherein the one or more programs are stored in the memory and are configured to be executed by the one or more processors, and the one or more programs include a method for executing any one of the above-mentioned enterprise big model construction methods.

[0015] Applying the technical solution of the present application, the method for constructing the above-mentioned enterprise big model first collects multiple enterprise document data and extracts the keywords corresponding to each enterprise document data, where the enterprise document data is the internal data of the target enterprise with a text length less than a preset length and a text format; then, the keywords of all the enterprise document data are merged, the frequency of occurrence of each keyword is counted, the keyword with the highest frequency of occurrence is determined as the enterprise data subject word, and the subject relevance of each enterprise document data is determined based on the matching degree between the enterprise data subject word and the keywords of each enterprise document data; then, multiple external document data are collected, the keywords corresponding to each external document data are extracted, and the subject relevance of each external document data is determined based on the matching degree between the enterprise data subject word and the keywords of each external document data; finally, the enterprise document data, the subject relevance of the enterprise document data, the external document data, and the subject relevance of the external document data are used to train the artificial intelligence model to obtain the target model, and the target model is used to process the target tasks related to the target enterprise. This method filters the data inside and outside the enterprise for quality, and proposes a data ratio optimization method, so that when the model continues to be pre-trained, it can balance the general capabilities and domain professional capabilities. This solves the problem that the existing method of continuing to pre-train large open source models only uses industry-related open source data, the data used is too little, and does not pay attention to the balance between general capabilities and domain professional capabilities, which easily leads to insufficient general capabilities. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] The drawings constituting part of the present application are used to provide a further understanding of the present application. The exemplary embodiments and descriptions of the present application are used to explain the present application and do not constitute an improper limitation on the present application. In the drawings:

[0017] Figure 1 A hardware structure block diagram of a mobile terminal for executing a method for constructing a large enterprise model provided in an embodiment of the present application is shown;

[0018] Figure 2 A schematic diagram of a process for constructing a large enterprise model according to an embodiment of the present application is shown;

[0019] Figure 3 A schematic diagram showing the number of training samples of a data bucket at each training stage according to an embodiment of the present application is shown;

[0020] Figure 4 A schematic diagram showing the number of training samples of the first data bucket at each training stage according to an embodiment of the present application is shown;

[0021] Figure 5 A schematic diagram showing the number of training samples of a second data bucket at each training stage according to an embodiment of the present application is shown;

[0022] Figure 6 A schematic diagram showing the number of training samples for the p-th data bucket at each training stage according to an embodiment of the present application is shown;

[0023] Figure 7 A schematic diagram of a process of constructing another enterprise macro model provided according to an embodiment of the present application is shown;

[0024] Figure 8 A structural block diagram of a device for constructing a large enterprise model provided according to an embodiment of the present application is shown.

[0025] The above drawings include the following reference numerals:

[0026] 102, processor; 104, memory; 106, transmission device; 108, input and output devices. DETAILED DESCRIPTION

[0027] It should be noted that, in the absence of conflict, the embodiments and features in the embodiments of the present application can be combined with each other. The present application will be described in detail below with reference to the accompanying drawings and in combination with the embodiments.

[0028] In order to enable those skilled in the art to better understand the solution of the present application, the technical solution in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work should fall within the scope of protection of the present application.

[0029] It should be noted that the terms "first", "second", etc. in the specification and claims of the present application and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequential order. It should be understood that the data used in this way can be interchanged where appropriate, so that the embodiments of the present application described here. In addition, the terms "including" and "having" and any of their variations are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device comprising a series of steps or units is not necessarily limited to those steps or units clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0030] For the convenience of description, some nouns or terms involved in the embodiments of the present application are explained below:

[0031] Continue pre-training: refers to further training based on the pre-trained model using a new dataset to better adapt to specific tasks or scenarios.

[0032] As introduced in the background technology, the prior art method for continuing pre-training only uses industry-related open source data, the data used is too little, and the use of enterprise data is not considered, and the balance between general capabilities and domain professional capabilities is not emphasized, which easily leads to insufficient general capabilities. In order to solve the problem that the prior art method for continuing pre-training of open source big models only uses industry-related open source data, the data used is too little, and the balance between general capabilities and domain professional capabilities is not emphasized, which easily leads to insufficient general capabilities, the embodiments of the present application provide a method for constructing an enterprise big model, a device for constructing an enterprise big model, a computer-readable storage medium and an electronic device.

[0033] The technical solutions in the embodiments of the present invention will be described clearly and completely below in conjunction with the accompanying drawings in the embodiments of the present invention.

[0034] The method embodiments provided in the embodiments of the present application can be executed in a mobile terminal, a computer terminal or a similar computing device. Taking running on a mobile terminal as an example, Figure 1 FIG. 1 is a hardware structure block diagram of a mobile terminal of a method for constructing a large enterprise model according to an embodiment of the present invention. Figure 1 As shown, the mobile terminal may include one or more ( Figure 1 Only one is shown in the figure) a processor 102 (the processor 102 may include but is not limited to a processing device such as a microprocessor MCU or a programmable logic device FPGA) and a memory 104 for storing data, wherein the mobile terminal may also include a transmission device 106 and an input / output device 108 for communication functions. It can be understood by those skilled in the art that Figure 1 The structure shown is for illustration only and does not limit the structure of the mobile terminal. Figure 1 More or fewer components as shown, or with Figure 1 Different configurations are shown.

[0035] The memory 104 can be used to store computer programs, for example, software programs and modules of application software, such as the computer program corresponding to the method for constructing the enterprise macro model in the embodiment of the present invention. The processor 102 executes various functional applications and data processing by running the computer program stored in the memory 104, that is, the above method is implemented. The memory 104 may include a high-speed random access memory, and may also include a non-volatile memory, such as one or more magnetic storage devices, flash memory, or other non-volatile solid-state memory. In some examples, the memory 104 may further include a memory remotely arranged relative to the processor 102, and these remote memories may be connected to the mobile terminal via a network. Examples of the above-mentioned network include but are not limited to the Internet, an intranet, a local area network, a mobile communication network, and a combination thereof. The transmission device 106 is used to receive or send data via a network. The above-mentioned specific examples of the network may include a wireless network provided by a communication provider of the mobile terminal. In one example, the transmission device 106 includes a network adapter (Network Interface Controller, referred to as NIC), which can be connected to other network devices through a base station so as to communicate with the Internet. In one example, the transmission device 106 may be a radio frequency (RF) module, which is used to communicate with the Internet wirelessly.

[0036] In this embodiment, a method for constructing an enterprise big model running on a mobile terminal, a computer terminal or a similar computing device is provided. It should be noted that the steps shown in the flowchart of the accompanying drawings can be executed in a computer system such as a set of computer executable instructions, and although a logical order is shown in the flowchart, in some cases, the steps shown or described can be executed in an order different from that shown here.

[0037] Figure 2 1 is a flow chart of a method for constructing a large enterprise model according to an embodiment of the present application. Figure 2 As shown, the method comprises the following steps:

[0038] Step S201, collecting a plurality of enterprise document data and extracting keywords corresponding to each of the enterprise document data, wherein the enterprise document data is internal data of the target enterprise whose text length is less than a preset length and whose format is text format;

[0039] Specifically, all kinds of documents in the enterprise (pdf, word, etc.) are collected and summarized, converted into text (txt) files for storage, and documents with a text length of less than 100 are filtered out to reduce invalid document interference.

[0040] The steps of extracting keywords corresponding to the enterprise document data include:

[0041] Step S2011, using separators to segment the above-mentioned enterprise document data, and filtering out invalid auxiliary words to obtain initial words;

[0042] Specifically, the above steps are document preprocessing steps, using Chinese stop words (such as: de, this, that, you, me, he, etc.) and punctuation marks as separators to segment the document and filter invalid auxiliary words. Stop words can use open source word lists from institutions such as Baidu and Harbin Institute of Technology.

[0043] Step S2012, determining one of the initial words in the enterprise document data as a preliminary word, and determining the quotient between the number of occurrences of the preliminary word and the number of all non-prepared words as the frequency score of the preliminary word;

[0044] Step S2013, determining the part-of-speech score weight corresponding to the prepared word according to the part-of-speech and part-of-speech weight table of the prepared word, wherein the part-of-speech weight table represents the correspondence between words and part-of-speech weights;

[0045] Specifically, nouns and verbs are generally more important, so the weights of nouns and verbs are set to 1, the weights of adjectives and adverbs are set to 0.5, the weights of other parts of speech are set to 0.2, and the weight of the part of speech score is represented by c. It should be noted that the weights of nouns and verbs, the weights of adjectives and adverbs, and the weights of other parts of speech can be set differently according to actual needs, and are not limited to the above values.

[0046] Step S2014, determining the position score corresponding to the prepared word according to the position and position score mapping relationship of the prepared word, wherein the position score mapping relationship represents the mapping relationship between the position of the word and the position score of the word;

[0047] Specifically, it is generally believed that the words at the beginning and end of a document are more important, and higher scores can be assigned to words at these positions. Therefore, the score changes with the position, first decreasing and then increasing. That is, the position score decreases from the beginning to the middle, and increases from the middle to the end.

[0048] The word in the middle of the document has a score of 0, and the word at the beginning and end of the document has a score of 1, that is, the function passes through the points (0, 1) and (N / 2, 0), and is simplified to be modeled with a one-dimensional linear function, where N is the number of words in the document, and x is the position of the word in the document, starting from 0. It should be noted that the above function is not limited to modeling with a one-dimensional linear function, and can also be set to other decreasing functions according to actual needs.

[0049] It decreases monotonically from the beginning to the middle, and the score function can be derived as: y = -2 / N×x+1;

[0050] The positions from the middle to the end are monotonically increasing, and the score function can be derived as: s = 2 / (N-1) × x-N / (N-1);

[0051] These two functions can be used to calculate the score of a word in a document. A word may appear multiple times, so there will be multiple scores, and the maximum score is taken as the final position score of the word.

[0052] Step S2015, based on the score calculation formula score=c×(a+s), the total score of the prepared word is obtained, where score is the total score of the prepared word, c is the part-of-speech score weight of the prepared word, a is the frequency score of the prepared word, and s is the position score of the prepared word;

[0053] Step S2016: rank all the initial words according to the total scores of the initial words, and determine the initial words ranked in the top k as keywords of the enterprise document data.

[0054] Specifically, by determining the keywords of the document, combining the keyword extraction algorithm with the scoring of frequency, part of speech, and position, and applying it to relevance calculations, the relevance of the document can be controlled, thereby balancing general capabilities and domain expertise.

[0055] Among them, the total score of the initial word describes the importance of the word, which is affected by its frequency, part of speech, and position. Generally speaking, the higher the frequency, the higher the importance; generally nouns are more important than adjectives; and generally, words that appear at the end of a document are of high importance.

[0056] In addition, the above keyword algorithm can also adopt LDA, textrank and other algorithms.

[0057] Step S202, merging the keywords of all the enterprise document data, counting the occurrence frequency of each keyword, determining the keyword with the highest occurrence frequency as the enterprise data subject word, and determining the subject relevance of each enterprise document data according to the matching degree between the enterprise data subject word and the keywords of each enterprise document data;

[0058] Specifically, through the keyword extraction process, we get the keywords of each document in the enterprise document, merge the keywords of all documents, count the frequency of each keyword, and select the N words with the highest frequency as the enterprise data subject words, which represent the main content types of enterprise documents. For example, the high-frequency words of financial companies may be: loans, assets, interest rates, etc.

[0059] The process of determining the subject relevance of each of the enterprise document data according to the matching degree between the enterprise data subject words and the keywords of each of the enterprise document data comprises the following steps:

[0060] Step S2021, determining the number of identical keywords of the enterprise document data and the enterprise data subject words as the number of identical words;

[0061] Step S2022, according to the correlation calculation formula T=t / k, determine the subject correlation of the above-mentioned enterprise document data, wherein T is the subject correlation of the above-mentioned enterprise document data, t is the number of the above-mentioned identical words, and k is the total number of keywords of the above-mentioned enterprise document data.

[0062] Specifically, enterprise documents can be screened based on the calculated relevance to balance general capabilities and domain-specific capabilities.

[0063] Step S203, collecting a plurality of external document data, extracting keywords corresponding to each of the external document data, and determining the subject relevance of each of the external document data according to the matching degree between the enterprise data subject words and the keywords of each of the external document data;

[0064] Specifically, Common Crawl is an open source web crawler data that stores a large amount of data on the Internet. It saves web page snapshots once a month. Considering timeliness, it only downloads snapshot data from the past year. The data of each web page snapshot is parsed through HTML to obtain text data as a text document, which constitutes external document data.

[0065] Collecting multiple external document data includes the following steps:

[0066] Step S301, obtaining initial external documents and determining document quality standards, wherein the document quality standards are at least used to limit document compliance, document security, document privacy, and document malicious attack levels, wherein the initial external documents include a first number of first documents and a second number of second documents;

[0067] Specifically, the requirements for document quality are as follows:

[0068] Compliance: The text content must comply with relevant laws and regulations and social morality, and must not contain illegal, illegal or immoral content.

[0069] Security: The text content cannot contain any information that may cause harm to users, such as fraudulent information, malicious links, viruses, etc.

[0070] Privacy protection: The text content cannot disclose the user's personal information, such as name, address, telephone number, bank account number, etc.

[0071] Prevent malicious attacks: Text content cannot contain any information that may be used for malicious attacks, such as inflammatory remarks, personal attacks, defamation, etc.

[0072] Step S302, determining whether the first document meets the document quality standard. If the first document meets the document quality standard, determining the label of the first document to be a high-quality sample label; if the first document does not meet the document quality standard, determining the label of the first document to be a low-quality sample label;

[0073] Specifically, whether the first document meets the document quality standards can be judged manually. If one item is not met, it is a low-quality sample with a label of 0; otherwise, it is a high-quality text with a label of 1.

[0074] Step S303, training the FastText algorithm model based on the first document and the corresponding label to obtain a text quality classifier;

[0075] Specifically, the FastText algorithm model is a small model, and the model training process includes the following stages:

[0076] Manual labeling stage: Manually labeled data is used as training data for the model (preparing learning samples): the first number of documents need to be labeled as training data. Labeling needs to be done manually, that is, the quality of each document is high, medium, or low (of course, it can also be divided into more categories), and then classified and labeled; (the process of manually sorting out questions and answers for model learning);

[0077] Model training phase: Feed the model with the above-organized data, that is, train the model to let the model learn the classification rules;

[0078] Model prediction stage: Input untrained data into the trained model, let the model make quality classification predictions for documents it has never seen, and give a prediction score for each prediction result.

[0079] Specific examples are shown in Table 1:

[0080] Table 1. Document classification example table

[0081] Document Name Category 1 (high quality) Category 2 Category 3 (low quality) Document 10.7 0.7 0.2 0.1

[0082] The FastText algorithm is a classification model, and for each classification, a corresponding probability value, i.e., a quality score, is obtained, such as: the classifications are high, medium, and low. In some instances, after inputting the data into the FastText algorithm model, the following is obtained: the probability of high quality = 0.7, the probability of medium quality = 0.2, and the probability of low quality = 0.1. Therefore, a threshold value needs to be set. When it is greater than this threshold, it is considered to be of high quality.

[0083] Step S304: Use the text quality classifier to classify the second document to obtain a predicted quality score, filter the second documents whose predicted quality score is less than a quality score limit, and determine the second documents whose predicted quality score is greater than or equal to the quality score limit as the external document data.

[0084] Specifically, a data quality standard is proposed, and a quality filtering algorithm is designed to filter low-quality external data. A quality filtering threshold determination method is proposed to improve filtering accuracy.

[0085] Before filtering the second documents whose predicted quality scores are less than the quality score limit, the method further includes the following steps:

[0086] Step S401, determining a candidate threshold set by using an equal interval method, wherein the candidate threshold set includes a plurality of candidate score limits, and a difference between two adjacent candidate score limits in the candidate threshold set is a preset difference;

[0087] Specifically, the quality score is a value of [0,1], with an interval of 0.05, and the candidate threshold set is determined by the equal interval method, that is, [0, 0.05, 0.1, ..., 0.95]. Assuming that the threshold value is 0.05 at this time, the accuracy is calculated once, and then assuming that it is 0.1, the accuracy is calculated again. The threshold is increased by 0.05 each time, and the accuracy is calculated respectively. The threshold value with the highest accuracy is selected as the final threshold value.

[0088] Step S402, sequentially using each of the candidate score limits in the candidate threshold set to screen a plurality of training document data, and recording the predicted case quality of each of the training document data corresponding to the candidate score limit;

[0089] Step S403, using the true interest rate calculation formula TPR=TP / (TP+FN) to calculate the true interest rate of the above-mentioned candidate score limit, and using the false positive rate calculation formula FPR=FP / (FP+TN) to calculate the false positive rate of the above-mentioned candidate score limit, wherein TPR is the true interest rate of the above-mentioned candidate score limit, TP is the number of training document data whose predicted case quality is high and whose true case quality is also high, FN is the number of training document data whose predicted case quality is low and whose true case quality is high, FPR is the false positive rate of the above-mentioned candidate score limit, FP is the number of training document data whose predicted case quality is high and whose true case quality is low, and TN is the number of training document data whose predicted case quality is low and whose true case quality is low;

[0090] Step S404, determining the difference between the true interest rate and the false positive interest rate of the above candidate score limits as the score of the above candidate score limits, and determining the above candidate score limit with the largest score as the above quality score limit.

[0091] Specifically, this can accurately determine the quality score limit, thereby accurately filtering the documents and leaving only the documents that best meet the training standards.

[0092] Among them, the significance of the difference between the true positive rate and the false positive rate of the candidate score limit is to maximize the ability to find true positive examples, that is, to maximize the number of true positive examples.

[0093] The key of the external document data is calculated by the same calculation method as that of the enterprise document data, and the relevance of the external document data is calculated by the same calculation method as that of the enterprise document data.

[0094] Step S204, using the above-mentioned enterprise document data, the subject relevance of the above-mentioned enterprise document data, the above-mentioned external document data and the subject relevance of the above-mentioned external document data to train the artificial intelligence model to obtain a target model, and using the above-mentioned target model to process target tasks related to the above-mentioned target enterprise.

[0095] Specifically, the general and professional capabilities of the model during the training process should be considered in a balanced way. The higher the correlation, the higher the relevance to the company's professional field. In any case, the lower the correlation, the more in line with the general capabilities. That is, use correlation to measure the professionalism of the enterprise field. In order to prevent the model from losing its general capabilities by being overly biased towards the professional field during the continued pre-training process, it is necessary to ensure that in a certain training stage, data from the general field is mixed for training, and attention is paid to the mixing of data; in addition, it is hoped that the model will be better at the professional field in the end, so there should be more professional data (more relevant data) at the end of the model training stage. Compared with external data, internal enterprise data is more professional, higher quality, and more controllable in content, and needs to be considered separately from external data.

[0096] The step of training the artificial intelligence model using the enterprise document data, the subject relevance of the enterprise document data, the external document data, and the subject relevance of the external document data to obtain a target model includes the following steps:

[0097] Step S501, sorting the enterprise document data in ascending order of relevance and then performing equal frequency bucketing to obtain a plurality of enterprise internal data buckets, and sorting the external document data in ascending order of relevance and then performing equal frequency bucketing to obtain a plurality of enterprise external data buckets, wherein the number of document data in each bucket is the same, and the number of the enterprise internal data buckets is the same as the number of the enterprise external data buckets;

[0098] Specifically, each training stage requires a mixture of professional and general data. As training progresses, there is more professional data and less general data.

[0099] In some examples, the internal data and external data of the enterprise are sorted from small to large according to the relevance, and then divided into P buckets with equal frequency. The number of data documents in each bucket is equal. The bucket of external data is WD = [wd_1, wd_2, .., wd_P], and the bucket of internal data is ND = [nd_1, nd_2, …, nd_P]. The larger the number, the higher the relevance and the stronger the professionalism. Note that internal and external data need to be considered separately.

[0100] Among them, it is necessary to sort first and then divide the data into equal frequency buckets. The relevance sorting is: the relevance score of each document may be inconsistent. After bucketing, in the subsequent document distribution link, the relevance of the data in the same bucket is considered to be in the same echelon. Equal frequency bucketing is: to ensure that the number of each bucket is consistent.

[0101] Step S502, determining the document quantity array of each of the above-mentioned enterprise external data buckets at different training stages and the document quantity array of each of the above-mentioned enterprise internal data buckets at different training stages, each data in the above-mentioned document quantity array represents the number of training samples extracted from the data bucket in different training stages, and the number of the above-mentioned training stages is an integer multiple of the number of the above-mentioned enterprise internal data buckets;

[0102] The training stage has F levels, the serial number of the enterprise external data bucket is proportional to the relevance of the document data in the enterprise external data bucket, and determining the document quantity array of each enterprise external data bucket in different training stages includes the following steps:

[0103] Step S5021, determining that the number of the training samples of the first enterprise external data bucket at the Nth level training stage is F-N+1;

[0104] Step S5022, determining that the number of the training samples of the Fth enterprise external data bucket at the Nth level training stage is N;

[0105] Step S5023, determining that the number of the training samples of the p-th enterprise external data bucket at the N-th level training stage is F-h+1-N, where h=(p-1)×F / p.

[0106] Specifically, this can prevent the model from becoming overly biased towards professional fields during the pre-training process and losing its general capabilities. Having more professional data (more relevant data) at the end of the model training phase can make the final model more proficient in professional fields.

[0107] In some instances, a discrete unit method is used to set the number of data shares in each stage. The wd1 bucket trains F units of data in the first stage, and then gradually reduces one unit in each subsequent stage until only 1 unit of data is trained in the F stage. This situation can be represented by the array Fg_wd_1 = [F, F-1, ..., 3, 2, 1], where each index position of the array represents the number of data shares in the corresponding training stage, and this array is called the data share array. On the other hand, the wd_P bucket trains 1 unit of data in the first stage, but increases one unit in each subsequent stage until F units of data are reached in the F stage. The data share array is Fg_wd_P = [1, 2, 3, ..., F].

[0108] like Figure 3 As shown in the figure, taking two rounds of training, that is, F=2P, as an example, the distribution of data in each stage of training is described. The data in bucket wd_1 reaches the maximum value in the first stage of training and takes the minimum value in the F stage. The data in bucket wd_P reaches the maximum value in the F stage of training and takes the minimum value in the first stage. That is, the distribution of data in bucket wd_n in each training stage is a curve that increases first and then decreases, and reaches the maximum value in the nth stage of training.

[0109] Among them, Figure 4 , Figure 5 and Figure 6 As shown in the figure, the data scores of the first bucket and the last wd_P bucket at different training stages are easier to determine. For the mth bucket, assuming that the number of data copies in the 1st to mth training stages is gradually increasing, and the number of data copies in the mth to Fth training stages is gradually decreasing, we can consider using a simplified function to model, but the cost is high. The array sliding window method is proposed to determine the data score array of the pth bucket.

[0110] First, construct the original number array, FG=[1,2,3,..,F-1,F,F-1,…,3,2,1], and the number array of data in each bucket is the continuous sub-data of the array. It can be seen that the number array of the first bucket is the second half of the array Fg_wd_1=[F,F-1,…,3,2,1], and the number array of the last bucket is the first half of the data Fg_wd_f=[1,2,3,..,F-1,F]. It can be found that in FG, sliding F indexes to the left with an array of length F, Fg_wd_1 can get Fg_wd_n.

[0111] Considering that F>=p and is divisible, for the pth bucket, the index number of Fg_wd_1 sliding to the left is h=(p-1)×F / p, that is, the number of shares of the pth bucket is: [Fh, F-h+1,…F,…,h+1].

[0112] Based on the above, we can get the number array of all buckets. Each element in the number array of each bucket (the number array of each bucket: the array composed of each sliding window) is divided by the element sum (element sum: the sum of the arrays in the sliding window) to get the corresponding data ratio, and then multiply it by the total number of documents in the current bucket to get the number of training documents for each bucket in different training stages, that is: the number of documents in the bucket can be represented by the array Ng_wd_p = [d_1, d_2, ..., d_F], and this array becomes the stage document number array.

[0113] Step S503, according to the document quantity array of the above-mentioned enterprise internal data bucket, extract the above-mentioned enterprise document data of a first preset number corresponding to the current training stage from the above-mentioned enterprise internal data bucket, and according to the document quantity array of the above-mentioned enterprise external data bucket, extract the above-mentioned external document data of a second preset number corresponding to the current training stage from the above-mentioned enterprise external data bucket to form the training samples of the current training stage;

[0114] Step S504, randomly breaking up the training samples corresponding to the above training stages and then training them in sequence to obtain the above target model.

[0115] Specifically, this can prevent overfitting.

[0116] In some examples, the array of the number of documents in each bucket of external document data and internal enterprise document data is calculated. Then, for the Fth training stage, the number at the Fth position is obtained from the array of the number of documents in the bucket, that is, the number of documents used for training in the bucket in this training stage, and then the corresponding number of documents are extracted from the data buckets to form the training samples for this stage. After obtaining the samples of each training stage, in order to prevent overfitting, the samples in each stage are randomly scattered, and then trained in sequence, and finally a trained model is obtained.

[0117] The method for constructing the above-mentioned enterprise big model of the present application first collects multiple enterprise document data and extracts the keywords corresponding to each enterprise document data, where the enterprise document data is the internal data of the target enterprise with a text length less than a preset length and a text format; then, the keywords of all the enterprise document data are merged, the occurrence frequency of each keyword is counted, and the keyword with the highest occurrence frequency is determined as the enterprise data subject word, and the subject relevance of each enterprise document data is determined based on the matching degree between the enterprise data subject word and the keywords of each enterprise document data; then, multiple external document data are collected, the keywords corresponding to each external document data are extracted, and the subject relevance of each external document data is determined based on the matching degree between the enterprise data subject word and the keywords of each external document data; finally, the enterprise document data, the subject relevance of the enterprise document data, the external document data and the subject relevance of the external document data are used to train the artificial intelligence model to obtain the target model, and the target model is used to process the target tasks related to the target enterprise. This method collects data and performs quality filtering, so that when the model continues to be pre-trained, it can balance general capabilities and domain expertise. It solves the problem that the existing method of continuing to pre-train large open source models only uses industry-related open source data, the data used is too little, and does not focus on the balance between general capabilities and domain expertise, which easily leads to insufficient general capabilities.

[0118] In order to enable those skilled in the art to more clearly understand the technical solution of the present application, the implementation process of the method for constructing the enterprise big model of the present application will be described in detail below in combination with specific embodiments.

[0119] This embodiment relates to a specific method for constructing a large enterprise model. Figure 7 As shown, the following steps are included:

[0120] Step S1: Collect enterprise document data and extract enterprise data keywords;

[0121] Step S2: Using the extracted keywords, extract the words with the highest frequency of occurrence as enterprise subject words;

[0122] Step S3: external document data collection, external document quality filtering, and keyword extraction;

[0123] Step S4: Calculate the topic relevance between enterprise documents and external documents;

[0124] Step S5: Data mixing and training order determination.

[0125] It should be noted that the steps shown in the flowcharts of the accompanying drawings can be executed in a computer system such as a set of computer executable instructions, and that, although a logical order is shown in the flowcharts, in some cases, the steps shown or described can be executed in an order different from that shown here.

[0126] The embodiment of the present application also provides a device for constructing a large model of an enterprise. It should be noted that the device for constructing a large model of an enterprise in the embodiment of the present application can be used to execute the method for constructing a large model of an enterprise provided in the embodiment of the present application. The device is used to implement the above-mentioned embodiments and preferred implementation modes, and the descriptions that have been made will not be repeated. As used below, the term "module" can implement a combination of software and / or hardware for a predetermined function. Although the devices described in the following embodiments are preferably implemented in software, the implementation of hardware, or a combination of software and hardware, is also possible and conceived.

[0127] The following is an introduction to the device for constructing a large enterprise model provided in an embodiment of the present application.

[0128] Figure 8 Schematic diagram of a device for constructing a large enterprise model according to an embodiment of the present application. Figure 6 As shown, the device includes: a first processing unit 10, a second processing unit 20, a third processing unit 30 and a fourth processing unit 40, the first processing unit 10 is used to collect a plurality of enterprise document data, and extract keywords corresponding to each of the above enterprise document data, the above enterprise document data is the internal data of the target enterprise with a text length less than a preset length and a text format; the second processing unit 20 is used to merge the above keywords of all the above enterprise document data, count the occurrence frequency of each of the above keywords, determine the above keywords with the highest occurrence frequency as the enterprise data subject words, and determine the subject relevance of each of the above enterprise document data according to the matching degree between the above enterprise data subject words and the keywords of each of the above enterprise document data; the third processing unit 30 is used to collect a plurality of external document data, extract keywords corresponding to each of the above external document data, and determine the subject relevance of each of the above external document data according to the matching degree between the above enterprise data subject words and the keywords of each of the above external document data; the fourth processing unit 40 is used to train the artificial intelligence model using the above enterprise document data, the subject relevance of the above enterprise document data, the above external document data and the subject relevance of the above external document data to obtain a target model, and use the above target model to process the target task related to the above target enterprise.

[0129] The above-mentioned enterprise large model construction device of the present application includes: a first processing unit, a second processing unit, a third processing unit and a fourth processing unit. The first processing unit is used to collect multiple enterprise document data and extract keywords corresponding to each enterprise document data. The enterprise document data is the internal data of the target enterprise with a text length less than a preset length and a text format; the second processing unit is used to merge the keywords of all enterprise document data, count the occurrence frequency of each keyword, determine the keyword with the highest occurrence frequency as the enterprise data subject word, and determine the subject relevance of each enterprise document data based on the matching degree between the enterprise data subject word and the keywords of each enterprise document data; the third processing unit is used to collect multiple external document data, extract keywords corresponding to each external document data, and determine the subject relevance of each external document data based on the matching degree between the enterprise data subject word and the keywords of each external document data; the fourth processing unit is used to use the enterprise document data, the subject relevance of the enterprise document data, the external document data and the subject relevance of the external document data to train the artificial intelligence model to obtain the target model, and use the target model to process the target task related to the target enterprise. The device collects data and performs quality filtering, so that when the model continues to be pre-trained, it can balance general capabilities and domain expertise. This solves the problem that the existing method of continuing to pre-train large open source models only uses industry-related open source data, the data used is too little, and does not focus on the balance between general capabilities and domain expertise, which easily leads to insufficient general capabilities.

[0130] In some examples, the first processing unit includes a first processing module, a first determining module, a second determining module, a third determining module, a second processing module and a fourth determining module, wherein the first processing module is used to segment the enterprise document data using a separator and filter out invalid auxiliary words to obtain initial words; the first determining module is used to determine one of the initial words in the enterprise document data as a preparatory word, and determine the quotient between the number of occurrences of the preparatory word and the number of all non-preparatory words as the frequency score of the preparatory word; the second determining module is used to determine the part-of-speech score weight corresponding to the preparatory word according to the part-of-speech of the preparatory word and the part-of-speech weight table, wherein the part-of-speech weight table represents the corresponding relationship between words and part-of-speech weights. system; the third determination module is used to determine the position score corresponding to the above-mentioned prepared words according to the mapping relationship between the position of the above-mentioned prepared words and the position score, and the above-mentioned position score mapping relationship represents the mapping relationship between the position of the word and the position score of the word; the second processing module is used to obtain the total score of the above-mentioned prepared words based on the score calculation formula score=c×(a+s), where score is the total score of the above-mentioned prepared words, c is the part-of-speech score weight of the above-mentioned prepared words, a is the frequency score of the above-mentioned prepared words, and s is the position score of the above-mentioned prepared words; the fourth determination module is used to rank all the above-mentioned initial words according to the total score of each of the above-mentioned initial words, and determine the above-mentioned initial words ranked in the top k as the keywords of the above-mentioned enterprise document data. By determining the keywords of the document, combining the keyword extraction algorithm with the scoring of the three dimensions of frequency, part of speech, and position, and applying it to the correlation calculation, the relevance of the document can be controlled, thereby balancing the general ability and the professional ability in the field.

[0131] In some instances, the third processing unit includes a first acquisition module, a first judgment module, a first training module and a first classification module. The first acquisition module is used to acquire an initial external document and determine a document quality standard. The document quality standard is at least used to limit document compliance, document security, document privacy and document malicious attack level. The initial external document includes a first number of first documents and a second number of second documents. The first judgment module is used to determine whether the first document meets the document quality standard. If the first document meets the document quality standard, the label of the first document is determined to be a high-quality sample label. If the first document does not meet the document quality standard, the label of the first document is determined to be a low-quality sample label. The first training module is used to train the FastText algorithm model based on the first document and the corresponding label to obtain a text quality classifier. The first classification module is used to classify the second document using the text quality classifier to obtain a predicted quality score, filter the second document whose predicted quality score is less than the quality score limit, and determine the second document whose predicted quality score is greater than or equal to the quality score limit as the external document data. Data quality standards are proposed, and a quality filtering algorithm is designed to filter low-quality external data. A quality filtering threshold determination method is proposed to improve filtering accuracy.

[0132] In some instances, before filtering the above-mentioned second document whose predicted quality score is less than the quality score limit, the above-mentioned device also includes: a first adoption module, a second adoption module, a third adoption module and a fifth determination module, the first adoption module is used to determine the candidate threshold set by using the equal interval method, the above-mentioned candidate threshold set includes multiple candidate score limits, and the difference between two adjacent candidate score limits in the above-mentioned candidate threshold set is a preset difference; the second adoption module is used to sequentially use each of the above-mentioned candidate score limits in the above-mentioned candidate threshold set to screen multiple training document data, and record the predicted case quality of each of the above-mentioned training document data corresponding to the above-mentioned candidate score limit; the third adoption module is used to use the true interest rate calculation formula TPR=TP / (TP+FN) to calculate the true interest rate of the above-mentioned candidate score limit, and use the false positive interest rate calculation formula FPR=FP / (FP+TN), calculate the false positive rate of the above candidate score limit, where TPR is the true rate of the above candidate score limit, TP is the number of training document data with high quality of the above predicted case and high quality of the real case, FN is the number of training document data with low quality of the above predicted case and high quality of the real case, FPR is the false positive rate of the above candidate score limit, FP is the number of training document data with high quality of the above predicted case and low quality of the real case, TN is the number of training document data with low quality of the above predicted case and low quality of the real case; the fifth determination module is used to determine the difference between the true rate and the false positive rate of the above candidate score limit as the score of the above candidate score limit, and determine the above candidate score limit with the largest score as the above quality score limit. In this way, the quality score limit can be accurately determined, so as to accurately filter the documents and leave the documents that best meet the training standards.

[0133] In some instances, the fourth processing unit includes a third processing module, a sixth determination module, a fourth processing module and a fifth processing module, the third processing module is used to sort the above-mentioned enterprise document data according to relevance from small to large and then perform equal frequency bucketing to obtain multiple enterprise internal data buckets, and sort the above-mentioned external document data according to relevance from small to large and then perform equal frequency bucketing to obtain multiple enterprise external data buckets, the number of document data in each bucket is the same, and the number of the above-mentioned enterprise internal data buckets is the same as the number of the above-mentioned enterprise external data buckets; the sixth determination module is used to determine the document number array of each of the above-mentioned enterprise external data buckets at different training stages and the document number array of each of the above-mentioned enterprise internal data buckets at different training stages, and the document number array in the above-mentioned document number array Each data represents the number of training samples extracted from the data bucket in different training stages, and the number of training stages is an integer multiple of the number of internal data buckets of the enterprise; the fourth processing module is used to extract the first preset number of enterprise document data corresponding to the current training stage from the internal data bucket of the enterprise according to the document number array of the internal data bucket of the enterprise, and extract the second preset number of external document data corresponding to the current training stage from the external data bucket of the enterprise according to the document number array of the external data bucket of the enterprise, so as to form the training samples of the current training stage; the fifth processing module is used to randomly scatter the training samples corresponding to each training stage and train them in sequence to obtain the target model. This can prevent overfitting.

[0134] In some examples, the training stage has F levels, the serial number of the enterprise external data bucket is proportional to the relevance of the document data in the enterprise external data bucket, and the sixth determination module includes a first determination submodule, a second determination submodule and a third determination submodule, the first determination submodule is used to determine that the number of the training samples of the first enterprise external data bucket at the Nth level training stage is F-N+1; the second determination submodule is used to determine that the number of the training samples of the Fth enterprise external data bucket at the Nth level training stage is N; the third determination submodule is used to determine that the number of the training samples of the pth enterprise external data bucket at the Nth level training stage is F-h+1-N, where h = (p-1) × F / p. This can prevent the model from being overly biased towards professional fields and losing general capabilities during the continued pre-training process. Having more professional data (higher relevance data) at the end of the model training stage can make the final model more proficient in professional fields.

[0135] In some examples, the second processing unit includes a fourth determination submodule and a fifth determination submodule, the fourth determination submodule is used to determine the number of identical words between the keywords of the enterprise document data and the subject words of the enterprise data as the number of identical words; the fifth determination submodule is used to determine the subject relevance of the enterprise document data according to the relevance calculation formula T=t / k, where T is the subject relevance of the enterprise document data, t is the number of identical words, and k is the total number of keywords of the enterprise document data. Enterprise documents can be screened according to the calculated relevance, thereby balancing general capabilities and domain professional capabilities.

[0136] The above-mentioned enterprise macro model construction device includes a processor and a memory, and the above-mentioned first processing unit and the like are stored in the memory as program units, and the processor executes the above-mentioned program units stored in the memory to implement corresponding functions. The above-mentioned modules are all located in the same processor; or, the above-mentioned modules are located in different processors in any combination.

[0137] The processor contains a kernel, which retrieves the corresponding program unit from the memory. One or more kernels can be set, and the kernel parameters can be adjusted to solve the problem that the existing method of pre-training large open source models only uses industry-related open source data, the data used is too small, and the balance between general capabilities and field professional capabilities is not emphasized, which easily leads to insufficient general capabilities.

[0138] The memory may include non-permanent memory in a computer-readable medium, random access memory (RAM) and / or non-volatile memory, such as read-only memory (ROM) or flash RAM, and the memory includes at least one memory chip.

[0139] An embodiment of the present invention provides a computer-readable storage medium, which includes a stored program, wherein when the program is executed, the device where the computer-readable storage medium is located is controlled to execute the method for constructing the enterprise big model.

[0140] Specifically, the method of constructing a large enterprise model includes:

[0141] Step S201, collecting a plurality of enterprise document data and extracting keywords corresponding to each of the enterprise document data, wherein the enterprise document data is internal data of the target enterprise whose text length is less than a preset length and whose format is text format;

[0142] Specifically, all kinds of documents in the enterprise (pdf, word, etc.) are collected and summarized, converted into text (txt) files for storage, and documents with a text length of less than 100 are filtered out to reduce invalid document interference.

[0143] Step S202, merging the keywords of all the enterprise document data, counting the occurrence frequency of each keyword, determining the keyword with the highest occurrence frequency as the enterprise data subject word, and determining the subject relevance of each enterprise document data according to the matching degree between the enterprise data subject word and the keywords of each enterprise document data;

[0144] Specifically, through the keyword extraction process, we get the keywords of each document in the enterprise document, merge the keywords of all documents, count the frequency of each keyword, and select the N words with the highest frequency as the enterprise data subject words, which represent the main content types of enterprise documents. For example, the high-frequency words of financial companies may be: loans, assets, interest rates, etc.

[0145] Step S203, collecting a plurality of external document data, extracting keywords corresponding to each of the external document data, and determining the subject relevance of each of the external document data according to the matching degree between the enterprise data subject words and the keywords of each of the external document data;

[0146] Specifically, Common Crawl is an open source web crawler data that stores a large amount of data on the Internet. It saves web page snapshots once a month. Considering timeliness, it only downloads snapshot data from the past year. The data of each web page snapshot is parsed through HTML to obtain text data as a text document, which constitutes external document data.

[0147] Step S204, using the above-mentioned enterprise document data, the subject relevance of the above-mentioned enterprise document data, the above-mentioned external document data and the subject relevance of the above-mentioned external document data to train the artificial intelligence model to obtain a target model, and using the above-mentioned target model to process target tasks related to the above-mentioned target enterprise.

[0148] Specifically, the general and professional capabilities of the model during the training process should be considered in a balanced way. The higher the correlation, the higher the relevance to the company's professional field. In any case, the lower the correlation, the more in line with the general capabilities. That is, use correlation to measure the professionalism of the enterprise field. In order to prevent the model from losing its general capabilities by being overly biased towards the professional field during the continued pre-training process, it is necessary to ensure that in a certain training stage, data from the general field is mixed for training, and attention is paid to the mixing of data; in addition, it is hoped that the model will be better at the professional field in the end, so there should be more professional data (more relevant data) at the end of the model training stage. Compared with external data, internal enterprise data is more professional, higher quality, and more controllable in content, and needs to be considered separately from external data.

[0149] An embodiment of the present invention provides a processor, and the processor is used to run a program, wherein the method for constructing the enterprise big model is executed when the program is run.

[0150] Specifically, the method of constructing a large enterprise model includes:

[0151] Step S201, collecting a plurality of enterprise document data and extracting keywords corresponding to each of the enterprise document data, wherein the enterprise document data is internal data of the target enterprise whose text length is less than a preset length and whose format is text format;

[0152] Specifically, all kinds of documents in the enterprise (pdf, word, etc.) are collected and summarized, converted into text (txt) files for storage, and documents with a text length of less than 100 are filtered out to reduce invalid document interference.

[0153] Step S202, merging the keywords of all the enterprise document data, counting the occurrence frequency of each keyword, determining the keyword with the highest occurrence frequency as the enterprise data subject word, and determining the subject relevance of each enterprise document data according to the matching degree between the enterprise data subject word and the keywords of each enterprise document data;

[0154] Specifically, through the keyword extraction process, we get the keywords of each document in the enterprise document, merge the keywords of all documents, count the frequency of each keyword, and select the N words with the highest frequency as the enterprise data subject words, which represent the main content types of enterprise documents. For example, the high-frequency words of financial companies may be: loans, assets, interest rates, etc.

[0155] Step S203, collecting a plurality of external document data, extracting keywords corresponding to each of the external document data, and determining the subject relevance of each of the external document data according to the matching degree between the enterprise data subject words and the keywords of each of the external document data;

[0156] Specifically, Common Crawl is an open source web crawler data that stores a large amount of data on the Internet. It saves web page snapshots once a month. Considering timeliness, it only downloads snapshot data from the past year. The data of each web page snapshot is parsed through HTML to obtain text data as a text document, which constitutes external document data.

[0157] Step S204, using the above-mentioned enterprise document data, the subject relevance of the above-mentioned enterprise document data, the above-mentioned external document data and the subject relevance of the above-mentioned external document data to train the artificial intelligence model to obtain a target model, and using the above-mentioned target model to process target tasks related to the above-mentioned target enterprise.

[0158] Specifically, the general and professional capabilities of the model during the training process should be considered in a balanced way. The higher the correlation, the higher the relevance to the company's professional field. In any case, the lower the correlation, the more in line with the general capabilities. That is, use correlation to measure the professionalism of the enterprise field. In order to prevent the model from losing its general capabilities by being overly biased towards the professional field during the continued pre-training process, it is necessary to ensure that in a certain training stage, data from the general field is mixed for training, and attention is paid to the mixing of data; in addition, it is hoped that the model will be better at the professional field in the end, so there should be more professional data (more relevant data) at the end of the model training stage. Compared with external data, internal enterprise data is more professional, higher quality, and more controllable in content, and needs to be considered separately from external data.

[0159] An embodiment of the present invention provides a device, the device comprising a processor, a memory, and a program stored in the memory and executable on the processor, and when the processor executes the program, at least the following steps are implemented:

[0160] Step S201, collecting a plurality of enterprise document data and extracting keywords corresponding to each of the enterprise document data, wherein the enterprise document data is internal data of the target enterprise whose text length is less than a preset length and whose format is text format;

[0161] Step S202, merging the keywords of all the enterprise document data, counting the occurrence frequency of each keyword, determining the keyword with the highest occurrence frequency as the enterprise data subject word, and determining the subject relevance of each enterprise document data according to the matching degree between the enterprise data subject word and the keywords of each enterprise document data;

[0162] Step S203, collecting a plurality of external document data, extracting keywords corresponding to each of the external document data, and determining the subject relevance of each of the external document data according to the matching degree between the enterprise data subject words and the keywords of each of the external document data;

[0163] Step S204, using the above-mentioned enterprise document data, the subject relevance of the above-mentioned enterprise document data, the above-mentioned external document data and the subject relevance of the above-mentioned external document data to train the artificial intelligence model to obtain a target model, and using the above-mentioned target model to process target tasks related to the above-mentioned target enterprise.

[0164] The devices in this article can be servers, PCs, PADs, mobile phones, etc.

[0165] The present application also provides a computer program product, which, when executed on a data processing device, is suitable for executing a program for initializing at least the following method steps:

[0166] Step S201, collecting a plurality of enterprise document data and extracting keywords corresponding to each of the enterprise document data, wherein the enterprise document data is internal data of the target enterprise whose text length is less than a preset length and whose format is text format;

[0167] Step S202, merging the keywords of all the enterprise document data, counting the occurrence frequency of each keyword, determining the keyword with the highest occurrence frequency as the enterprise data subject word, and determining the subject relevance of each enterprise document data according to the matching degree between the enterprise data subject word and the keywords of each enterprise document data;

[0168] Step S203, collecting a plurality of external document data, extracting keywords corresponding to each of the external document data, and determining the subject relevance of each of the external document data according to the matching degree between the enterprise data subject words and the keywords of each of the external document data;

[0169] Step S204, using the above-mentioned enterprise document data, the subject relevance of the above-mentioned enterprise document data, the above-mentioned external document data and the subject relevance of the above-mentioned external document data to train the artificial intelligence model to obtain a target model, and using the above-mentioned target model to process target tasks related to the above-mentioned target enterprise.

[0170] Obviously, those skilled in the art should understand that the above modules or steps of the present invention can be implemented by a general computing device, they can be concentrated on a single computing device, or distributed on a network composed of multiple computing devices, they can be implemented by a program code executable by a computing device, so that they can be stored in a storage device and executed by the computing device, and in some cases, the steps shown or described can be executed in a different order than here, or they can be made into individual integrated circuit modules, or multiple modules or steps therein can be made into a single integrated circuit module for implementation. Thus, the present invention is not limited to any specific combination of hardware and software.

[0171] Those skilled in the art will appreciate that the embodiments of the present application may be provided as methods, systems, or computer program products. Therefore, the present application may adopt the form of a complete hardware embodiment, a complete software embodiment, or an embodiment in combination with software and hardware. Moreover, the present application may adopt the form of a computer program product implemented in one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) that include computer-usable program code.

[0172] The present application is described with reference to the flowcharts and / or block diagrams of the methods, devices (systems), and computer program products according to the embodiments of the present application. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of the processes and / or boxes in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0173] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0174] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process in the computer or other programmable device. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0175] In a typical configuration, a computing device includes one or more processors (CPU), input / output interfaces, network interfaces, and memory.

[0176] The memory may include non-permanent memory in a computer-readable medium, random access memory (RAM) and / or non-volatile memory in the form of read-only memory (ROM) or flash RAM. The memory is an example of a computer-readable medium.

[0177] Computer readable media include permanent and non-permanent, removable and non-removable media that can be implemented by any method or technology to store information. Information can be computer readable instructions, data structures, program modules or other data. Examples of computer storage media include, but are not limited to, phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technology, compact disk read-only memory (CD-ROM), digital versatile disk (DVD) or other optical storage, magnetic cassettes, magnetic disk storage or other magnetic storage devices or any other non-transmission media that can be used to store information that can be accessed by a computing device. As defined herein, computer readable media does not include temporary computer readable media (transitory media), such as modulated data signals and carrier waves.

[0178] It should also be noted that the terms "include", "comprises" or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, method, commodity or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, commodity or device. In the absence of more restrictions, the elements defined by the sentence "comprises a ..." do not exclude the existence of other identical elements in the process, method, commodity or device including the elements.

[0179] From the above description, it can be seen that the above embodiments of the present application achieve the following technical effects:

[0180] 1) The method for constructing the above-mentioned enterprise big model of the present application first collects multiple enterprise document data and extracts the keywords corresponding to each enterprise document data. The enterprise document data is the internal data of the target enterprise with a text length less than a preset length and a text format; then the keywords of all the enterprise document data are merged, the occurrence frequency of each keyword is counted, and the keyword with the highest occurrence frequency is determined as the enterprise data subject word, and the subject relevance of each enterprise document data is determined based on the matching degree between the enterprise data subject word and the keywords of each enterprise document data; then multiple external document data are collected, the keywords corresponding to each external document data are extracted, and the subject relevance of each external document data is determined based on the matching degree between the enterprise data subject word and the keywords of each external document data; finally, the enterprise document data, the subject relevance of the enterprise document data, the external document data and the subject relevance of the external document data are used to train the artificial intelligence model to obtain the target model, and the target model is used to process the target tasks related to the target enterprise. This method collects data and performs quality filtering, so that when the model continues to be pre-trained, it can balance general capabilities and domain expertise. It solves the problem that the existing method of continuing to pre-train large open source models only uses industry-related open source data, the data used is too little, and does not focus on the balance between general capabilities and domain expertise, which easily leads to insufficient general capabilities.

[0181] 2) The device for constructing the above-mentioned enterprise large model of the present application includes: a first processing unit, a second processing unit, a third processing unit and a fourth processing unit. The first processing unit is used to collect multiple enterprise document data and extract keywords corresponding to each enterprise document data. The enterprise document data is the internal data of the target enterprise with a text length less than a preset length and a text format; the second processing unit is used to merge the keywords of all enterprise document data, count the frequency of occurrence of each keyword, determine the keyword with the highest frequency of occurrence as the enterprise data subject word, and determine the subject relevance of each enterprise document data based on the matching degree between the enterprise data subject word and the keywords of each enterprise document data; the third processing unit is used to collect multiple external document data, extract keywords corresponding to each external document data, and determine the subject relevance of each external document data based on the matching degree between the enterprise data subject word and the keywords of each external document data; the fourth processing unit is used to use the enterprise document data, the subject relevance of the enterprise document data, the external document data and the subject relevance of the external document data to train the artificial intelligence model to obtain the target model, and use the target model to process the target task related to the target enterprise. The device collects data and performs quality filtering, so that when the model continues to be pre-trained, it can balance general capabilities and domain expertise. This solves the problem that the existing method of continuing to pre-train large open source models only uses industry-related open source data, the data used is too little, and does not focus on the balance between general capabilities and domain expertise, which easily leads to insufficient general capabilities.

[0182] The above description is only the preferred embodiment of the present application and is not intended to limit the present application. For those skilled in the art, the present application may have various modifications and variations. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present application shall be included in the protection scope of the present application.

Claims

1. A method for constructing a large enterprise model, characterized in that: include: Collecting a plurality of enterprise document data and extracting keywords corresponding to each of the enterprise document data, wherein the enterprise document data is internal data of the target enterprise whose text length is less than a preset length and whose format is a text format; Merging the keywords of all the enterprise document data, counting the frequency of occurrence of each keyword, determining the keyword with the highest frequency of occurrence as the enterprise data subject word, and determining the subject relevance of each enterprise document data according to the matching degree between the enterprise data subject word and the keywords of each enterprise document data; Collecting a plurality of external document data, extracting keywords corresponding to each of the external document data, and determining the subject relevance of each of the external document data according to the matching degree between the enterprise data subject words and the keywords of each of the external document data; The enterprise document data, the subject relevance of the enterprise document data, the external document data and the subject relevance of the external document data are used to train an artificial intelligence model to obtain a target model, and the target model is used to process target tasks related to the target enterprise.

2. The method according to claim 1, characterized in that Extracting keywords corresponding to each of the enterprise document data includes: Using separators to segment the enterprise document data, and filtering out invalid auxiliary words to obtain initial words; Determine one of the initial words in the enterprise document data as a preliminary word, and determine the quotient between the number of occurrences of the preliminary word and the number of all non-prepared words as the frequency score of the preliminary word; Determining a part-of-speech score weight corresponding to the prepared word according to a part-of-speech weight table of the prepared word, wherein the part-of-speech weight table represents a correspondence between words and part-of-speech weights; Determine a position score corresponding to the prepared word according to a position and position score mapping relationship of the prepared word, wherein the position score mapping relationship represents a mapping relationship between a position of a word and a position score of the word; Based on the score calculation formula score=c×(a+s), the total score of the prepared word is obtained, where score is the total score of the prepared word, c is the part-of-speech score weight of the prepared word, and a is the frequency score of the prepared word. s is the position score of the preparatory word; All the initial words are ranked according to the total scores of the initial words, and the initial words ranked in the top k are determined as keywords of the enterprise document data.

3. The method according to claim 1, characterized in that Collects data from multiple external documents, including: Acquire initial external documents and determine document quality standards, where the document quality standards are at least used to limit document compliance, document security, document privacy, and document malicious attack levels, wherein the initial external documents include a first number of first documents and a second number of second documents; Determining whether the first document meets the document quality standard, and if the first document meets the document quality standard, determining that the label of the first document is a high-quality sample label; if the first document does not meet the document quality standard, determining that the label of the first document is a low-quality sample label; Training the FastText algorithm model based on the first document and the corresponding label to obtain a text quality classifier; The second document is classified using the text quality classifier to obtain a predicted quality score, the second document whose predicted quality score is less than a quality score limit is filtered, and the second document whose predicted quality score is greater than or equal to the quality score limit is determined as the external document data.

4. The method according to claim 3, characterized in that Before filtering the second document whose predicted quality score is less than a quality score limit, the method further includes: A candidate threshold set is determined by using an equal interval method, wherein the candidate threshold set includes a plurality of candidate score limits, and a difference between two adjacent candidate score limits in the candidate threshold set is a preset difference; Using each of the candidate score limits in the candidate threshold set to screen a plurality of training document data in turn, and recording the predicted case quality of each of the training document data corresponding to the candidate score limit; The true interest rate of the candidate score limit is calculated by using the true interest rate calculation formula TPR=TP / (TP+FN), and the false positive rate of the candidate score limit is calculated by using the false positive rate calculation formula FPR=FP / (FP+TN), wherein TPR is the true interest rate of the candidate score limit, TP is the number of training document data whose predicted case quality is high and whose true case quality is also high, FN is the number of training document data whose predicted case quality is low and whose true case quality is high, FPR is the false positive rate of the candidate score limit, FP is the number of training document data whose predicted case quality is high and whose true case quality is low, and TN is the number of training document data whose predicted case quality is low and whose true case quality is low; The difference between the true interest rate and the false positive interest rate of the candidate score limit is determined as the score of the candidate score limit, and the candidate score limit with the largest score is determined as the quality score limit.

5. The method according to claim 1, characterized in that The artificial intelligence model is trained using the enterprise document data, the subject relevance of the enterprise document data, the external document data, and the subject relevance of the external document data to obtain a target model, including: The enterprise document data is sorted in ascending order according to relevance and then divided into equal-frequency buckets to obtain a plurality of enterprise internal data buckets, and the external document data is sorted in ascending order according to relevance and then divided into equal-frequency buckets to obtain a plurality of enterprise external data buckets, the number of document data in each bucket is the same, and the number of the enterprise internal data buckets is the same as the number of the enterprise external data buckets; Determine an array of the number of documents of each of the enterprise's external data buckets at different training stages and an array of the number of documents of each of the enterprise's internal data buckets at different training stages, wherein each data in the array of the number of documents respectively represents the number of training samples extracted from the data bucket at different training stages, and the number of the training stages is an integer multiple of the number of the enterprise's internal data buckets; According to the document quantity array of the enterprise internal data bucket, extract the first preset number of enterprise document data corresponding to the current training stage from the enterprise internal data bucket, and according to the document quantity array of the enterprise external data bucket, extract the second preset number of external document data corresponding to the current training stage from the enterprise external data bucket to form training samples for the current training stage; The training samples corresponding to each training stage are randomly scattered and then trained in sequence to obtain the target model.

6. The method according to claim 5, characterized in that The training stage has F levels, the sequence number of the enterprise external data bucket is proportional to the relevance of the document data in the enterprise external data bucket, and the document quantity array of each enterprise external data bucket in different training stages is determined, including: Determine that the number of training samples of the first enterprise external data bucket at the Nth level training stage is F-N+1; Determine that the number of training samples of the F-th enterprise external data bucket at the N-th level training stage is N; Determine the number of training samples of the p-th enterprise external data bucket at the N-th level training stage as F-h+1-N, where h=(p-1)×F / p.

7. The method according to claim 1, characterized in that Determining the subject relevance of each of the enterprise document data according to the matching degree between the enterprise data subject words and the keywords of each of the enterprise document data includes: Determine the number of identical keywords of the enterprise document data and the enterprise data subject words as the number of identical words; The subject relevance of the enterprise document data is determined according to the relevance calculation formula T=t / k, wherein T is the subject relevance of the enterprise document data, t is the number of identical words, and k is the total number of keywords in the enterprise document data.

8. A device for constructing a large enterprise model, characterized in that: include: A first processing unit is used to collect a plurality of enterprise document data and extract keywords corresponding to each of the enterprise document data, wherein the enterprise document data is internal data of the target enterprise whose text length is less than a preset length and whose format is a text format; A second processing unit is used to merge the keywords of all the enterprise document data, count the occurrence frequency of each keyword, determine the keyword with the highest occurrence frequency as the enterprise data subject word, and determine the subject relevance of each enterprise document data according to the matching degree between the enterprise data subject word and the keywords of each enterprise document data; A third processing unit is used to collect multiple external document data, extract keywords corresponding to each of the external document data, and determine the subject relevance of each of the external document data according to the matching degree between the enterprise data subject words and the keywords of each of the external document data; The fourth processing unit is used to train the artificial intelligence model using the enterprise document data, the subject relevance of the enterprise document data, the external document data and the subject relevance of the external document data to obtain a target model, and use the target model to process target tasks related to the target enterprise.

9. A computer-readable storage medium, characterized in that: The computer-readable storage medium includes a stored program, wherein when the program is executed, the device where the computer-readable storage medium is located is controlled to execute the method for constructing a large enterprise model as described in any one of claims 1 to 7.

10. An electronic device, characterized in that: include: One or more processors, a memory, and one or more programs, wherein the one or more programs are stored in the memory and are configured to be executed by the one or more processors, and the one or more programs include a method for executing the construction method of the enterprise big model described in any one of claims 1 to 7.

Images