Data processing method and device, nonvolatile storage medium and electronic equipment

Abstract

The invention discloses a data processing method and device, a nonvolatile storage medium and electronic equipment. The method comprises the following steps: acquiring work order data of a public service hotline; inputting the work order data into a target large language model to obtain hot event information corresponding to the work order data output by the target large language model, the target large language model is a large language model obtained by performing low-rank adaptation fine tuning on the pre-trained large language model according to historical work order data of the target business field, hot event information corresponding to the historical work order data and professional knowledge of the target business field; and sending the hot event information corresponding to the work order data to the target object. According to the method and the device, the technical problem of relatively high training and maintenance cost of the large language model caused by relatively high requirement on the number of parameters participating in training in a related large language model training method is solved.

Description

Technical Field

[0001] The present application relates to the field of data processing technology, and in particular, to a data processing method and device, a non-volatile storage medium, and an electronic device. Background Art

[0002] As an artificial intelligence method based on large-scale neural network models, large language model technology has made breakthrough progress in academia and industry in recent years. The technical fields involved include deep learning, self-attention mechanism, pre-training and fine-tuning, which have injected strong impetus into the development of artificial intelligence. In the field of natural language processing, large language model technology has achieved remarkable results. For example, models such as GPT, chatGLM, baichuan, and Tongyi Qianwen have demonstrated excellent performance in text generation, dialogue systems, and machine translation. These achievements have not only attracted widespread attention in academia, but also brought huge commercial value to the industry.

[0003] Behind these high-performance large language models are huge parameter scales and extremely high hardware requirements. Taking GPT-4 as an example, its parameter volume has reached trillions, which not only requires massive amounts of data for training, but also requires strong computing power as support. In order to maintain the operation of such a model, it is necessary to equip specialized high-performance computing equipment, such as GPU or TPU clusters. The cost and energy consumption of these hardware facilities are huge. In addition, the storage and maintenance of large language models also require a lot of hard disk space and professional technical support. Therefore, although the large language model technology has shown unparalleled application potential in many fields, its huge demand for hardware resources has also become a threshold for the promotion and popularization of technology.

[0004] With the acceleration of urbanization and the improvement of public service quality requirements, the number of work orders handled by the citizen hotline has exploded, and these work orders contain the concerns and needs of citizens. However, faced with a large amount of work order data, the traditional manual processing method can no longer meet the requirements of timeliness and accuracy. According to the actual situation of the on-site operators of the Shanghai 12345 citizen hotline, the average production time of each special report is as high as 3 man-days, which not only creates huge work pressure on the operators, but also affects the timeliness and accuracy of the report.

[0005] Therefore, it is urgent to propose a large language model that can automatically process massive work order data and identify hot events. However, in the relevant large language model training methods, the number of parameters involved in the training is high, resulting in high training and maintenance costs for the large language model.

[0006] To address the above-mentioned problems, no effective solution has been proposed yet. Summary of the invention

[0007] The embodiments of the present application provide a data processing method and device, a non-volatile storage medium, and an electronic device to at least solve the technical problem of high training and maintenance costs for large language models due to the high requirements on the number of parameters involved in training in related large language model training methods.

[0008] According to one aspect of an embodiment of the present application, a data processing method is provided, including: obtaining work order data of a public service hotline; inputting the work order data into a target large language model to obtain hot event information corresponding to the work order data output by the target large language model, wherein the target large language model is a large language model obtained by performing low-rank adaptive fine-tuning on a pre-trained large language model based on historical work order data of a target business field, hot event information corresponding to the historical work order data, and professional knowledge of the target business field; and sending the hot event information corresponding to the work order data to a target object.

[0009] Optionally, the target large language model is obtained by training in the following method: using professional knowledge of the target business field to train a first large language model to obtain parameters of a second target large language model, wherein, in the first large language model, all parameters of the large language model and parameters of the first large language model are fixed parameters, the parameters of the second large language model are non-fixed parameters, and all parameters of the large language model are parameters in a pre-trained large language model; using historical work order data and hot event information corresponding to the historical work order data to train a second large language model to obtain parameters of a first target large language model, wherein, in the second large language model, all parameters of the large language model and parameters of the second target large language model are fixed parameters, and parameters of the first large language model are non-fixed parameters; superimposing the second target large language model parameters and the first target large language model parameters onto the full parameters of the large language model to obtain model parameters of the target large language model; generating a target large language model according to the model parameters of the target large language model.

[0010] Optionally, the first largest language model parameter is the product of a first matrix and a second matrix, wherein the dimension of the first matrix is ​​smaller than that of the second matrix; and the second largest language model parameter is a third matrix of rank 3.

[0011] Optionally, before using the expertise of the target business field to train the first language model, the method also includes: obtaining the expertise of the target business field, historical work order data of the target business field, and hot event information corresponding to the historical work order data, wherein the historical work order data includes at least one of the following: the timestamp of the historical work order, the request location of the historical work order, and the work order description information of the historical work order; deleting invalid characters, spaces, punctuation marks, and non-standard encodings in the historical work order data; identifying and removing duplicate historical work order data based on a text similarity algorithm; converting the historical work order data into standard text, wherein the conversion process at least includes: removing HTML tags, adjusting the text encoding to a first preset format, converting the time field to a second preset format, etc.

[0012] Optionally, the work order data is input into the target large language model to obtain hot event information corresponding to the work order data output by the target large language model, including: dividing the work order data into n groups of sub-data according to the number of words in the work order data, wherein n is a positive integer greater than 1, and n is proportional to the number of words in the work order data; by respectively calling the model interface of the target large language model, each group of sub-data is input into the target large language model to obtain the hot event information corresponding to each of the n groups of sub-data; the hot event information corresponding to each of the n groups of sub-data is summarized and deduplicated to obtain the hot event information of all the work order data.

[0013] Optionally, the hot event information corresponding to the work order data is sent to the target object, including: summarizing the hot event information corresponding to n groups of sub-data, and deduplicating the summarized results; sorting the deduplicated hot event information according to preset sorting rules, wherein the preset sorting rules include: the influence of the hot event and the urgency of the hot event; classifying the deduplicated hot event information according to preset classification rules, wherein the preset classification rules include: the type and subject of the hot event; combining the sorted and classified hot event information with the work order description information corresponding to each group of sub-data, and filling the combined results into a predefined report template to obtain a hot event report, wherein the hot event report includes at least one of the following: analysis content of the hot event, work order summary, event keywords and suggested measures; sending the hot event report to the hotline operating agency.

[0014] Optionally, after sending the hot event information corresponding to the work order data to the target object, the method also includes: obtaining feedback results from the target object on the hot event information corresponding to the work order data; analyzing the feedback results, if the analysis results indicate that the low recognition rate of hot events is due to insufficient knowledge understanding of the target business field, adjusting the parameters of the second largest language model; if the analysis results indicate that the low recognition rate of hot events is due to insufficient scenario-based capabilities, adjusting the parameters of the first largest language model.

[0015] According to another aspect of the embodiment of the present application, a data processing device is also provided, including: an acquisition module for acquiring work order data of a public service hotline; an input module for inputting the work order data into a target large language model to obtain hot event information corresponding to the work order data output by the target large language model, wherein the target large language model is a large language model obtained by performing low-rank adaptive fine-tuning on a pre-trained large language model based on historical work order data of a target business field, hot event information corresponding to the historical work order data, and professional knowledge of the target business field; and a sending module for sending the hot event information corresponding to the work order data to a target object.

[0016] According to another aspect of the embodiments of the present application, a non-volatile storage medium is provided, the storage medium includes a stored program, wherein when the program is executed, the device where the storage medium is located is controlled to execute the above data processing method.

[0017] According to another aspect of the embodiments of the present application, there is further provided an electronic device, including: a memory and a processor, wherein the processor is used to run a program stored in the memory, wherein the above data processing method is executed when the program is run.

[0018] According to yet another aspect of the embodiments of the present application, a computer program is further provided, wherein the above data processing method is implemented when the computer program is executed by a processor.

[0019] According to another aspect of the embodiments of the present application, a computer program product is provided, which includes a non-volatile computer-readable storage medium, wherein the non-volatile computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the above data processing method is implemented.

[0020] In an embodiment of the present application, work order data of a public service hotline is obtained; the work order data is input into a target large language model to obtain hot event information corresponding to the work order data output by the target large language model, wherein the target large language model is a large language model obtained by low-rank adaptive fine-tuning of a pre-trained large language model based on historical work order data of the target business field, hot event information corresponding to the historical work order data, and professional knowledge of the target business field; the hot event information corresponding to the work order data is sent to the target object, thereby achieving the purpose of improving the output accuracy of the large language model in the target business field while reducing the number of parameters, thereby achieving the technical effect of reducing the high training and maintenance costs of the large language model, and further solving the technical problem of high training and maintenance costs of the large language model caused by the high requirements on the number of parameters involved in the training in the relevant large language model training method. BRIEF DESCRIPTION OF THE DRAWINGS

[0021] The drawings described herein are used to provide a further understanding of the present application and constitute a part of the present application. The illustrative embodiments of the present application and their descriptions are used to explain the present application and do not constitute an improper limitation on the present application. In the drawings:

[0022] Figure 1 is a flow chart of a data processing method according to an embodiment of the present application;

[0023] Figure 2 is a flow chart of another data processing method according to an embodiment of the present application;

[0024] Figure 3 is a structural diagram of a data processing device according to an embodiment of the present application;

[0025] Figure 4 It is a hardware structure block diagram of a computer terminal according to a data processing method of an embodiment of the present application. DETAILED DESCRIPTION

[0026] In order to enable those skilled in the art to better understand the solution of the present application, the technical solution in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work should fall within the scope of protection of the present application.

[0027] It should be noted that the terms "first", "second", etc. in the specification and claims of the present application and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchangeable where appropriate, so that the embodiments of the present application described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "including" and "having" and any of their variations are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device comprising a series of steps or units is not necessarily limited to those steps or units clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0028] According to an embodiment of the present application, a method embodiment of a data processing method is provided. It should be noted that the steps shown in the flowchart of the accompanying drawings can be executed in a computer system such as a set of computer executable instructions, and although a logical order is shown in the flowchart, in some cases, the steps shown or described can be executed in an order different from that shown here.

[0029] Figure 1is a flow chart of a data processing method according to an embodiment of the present application, such as Figure 1 As shown, the method comprises the following steps:

[0030] Step S102, obtaining work order data of the public service hotline.

[0031] Extract work order data from the public service hotline database. This data includes but is not limited to information such as complaints, suggestions, and consultations submitted by citizens, as well as metadata such as the creation time, processing status, and associated departments or institutions of the work order. Data acquisition can be achieved through API interfaces, database queries, or other data capture technologies.

[0032] Step S104, input the work order data into the target large language model to obtain the hot event information corresponding to the work order data output by the target large language model, wherein the target large language model is a large language model obtained by low-rank adaptive fine-tuning of the pre-trained large language model based on the historical work order data of the target business field, the hot event information corresponding to the historical work order data, and the professional knowledge of the target business field.

[0033] The ticket data is fed into the target large language model, which is fine-tuned specifically for processing data from the target business domain. The fine-tuning process uses historical ticket data from the domain, known hot events corresponding to the historical ticket data, and expertise related to the target business domain.

[0034] By learning this information, the target large language model can more accurately understand and analyze the newly input work order data, and output the hot event information related to these work orders. This process can be regarded as the model's high-level semantic understanding and event recognition of the input data, and the results can include details such as the type of event, scope of impact, and urgency.

[0035] Step S106, sending the hot event information corresponding to the work order data to the target object.

[0036] After analyzing and identifying hot events in the work order data, this information is sent to the designated target object, which can be a hotline operator, decision maker, or other entity that needs this information. The transmission of information can be achieved through email, SMS, internal messaging system, or through API interface. After receiving this hot event information, the target object can take corresponding measures, such as allocating resources, adjusting policies, issuing announcements, etc., to respond to the needs and concerns of citizens and improve service quality.

[0037] According to the above steps, work order data of a public service hotline is obtained; the work order data is input into a target large language model to obtain hot event information corresponding to the work order data output by the target large language model, wherein the target large language model is a large language model obtained by performing low-rank adaptive fine-tuning on a pre-trained large language model based on historical work order data in the target business field, hot event information corresponding to the historical work order data, and professional knowledge in the target business field; the hot event information corresponding to the work order data is sent to the target object, thereby achieving the purpose of improving the output accuracy of the large language model in the target business field while reducing the number of parameters, thereby achieving the technical effect of reducing the high training and maintenance costs of the large language model.

[0038] The following Figure 1 The steps shown are exemplary and explanatory.

[0039] According to some optional embodiments of the present application, the target large language model is obtained by training in the following method: using professional knowledge of the target business field to train a first large language model to obtain parameters of a second target large language model, wherein, in the first large language model, all parameters of the large language model and parameters of the first large language model are fixed parameters, and parameters of the second large language model are non-fixed parameters, and all parameters of the large language model are parameters in the pre-trained large language model; using historical work order data and hot event information corresponding to the historical work order data to train a second large language model to obtain parameters of a first target large language model, wherein, in the second large language model, all parameters of the large language model and parameters of the second target large language model are fixed parameters, and parameters of the first large language model are non-fixed parameters; superimposing the second target large language model parameters and the first target large language model parameters onto the full parameters of the large language model to obtain model parameters of the target large language model; generating a target large language model according to the model parameters of the target large language model.

[0040] A pre-trained large language model is used as the base, and the parameters of this large language model (denoted as Wbase) remain fixed during the training process. The professional knowledge of the target business field, such as industry reports, literature, etc., are used as training data. These data are designed to fine-tune the model so that it can understand and adapt to the language and concepts of a specific field. Based on the pre-trained model, two low-rank matrices are introduced: the first large language model parameter Wadapter1 and the second large language model parameter Wadapter2. Wadapter2 is an ultra-low-rank matrix, and its purpose is to learn domain knowledge without changing the Wbase parameter. Wadapter1 is used in the second stage to learn the scenario-based capabilities of specific tasks. In the first stage (in the first large language model), Wbase and Wadapter1 remain fixed, and only Wadapter2 is trained to enable it to capture and understand short texts and specific semantic features in the target field. Wadapter2 is trained using the professional knowledge dataset, and the parameters of Wadapter2 are adjusted through machine learning techniques such as forward propagation and backpropagation so that it can better reflect the knowledge and context of the target field. The training process may involve strategies such as supervised learning and transfer learning to ensure that the model can accurately grasp the professional vocabulary and common expressions in the field.

[0041] Based on the model obtained in the first stage, the second largest language model also keeps the Wbase parameter fixed. Use historical work order data and the identified hot event information corresponding to these work orders as training data. This time, Wbase and Wadapter2 are fixed, and Wadapter1 is trained so that it can learn scenario-based capabilities in specific tasks (such as hot event identification). Through historical work order data and corresponding hot event information, Wadapter1 parameters are trained so that it can accurately identify and extract hot events based on the content of the work order. The training process involves strategies such as multi-task learning and contrastive learning to enhance the performance of the model on specific tasks.

[0042] The parameters of the second target large language model obtained from the first stage of training and the parameters of the first target large language model obtained from the second stage of training are superimposed on the full parameters of the large language model to obtain the final parameter combination of the target large language model. Use the integrated parameters to rebuild or generate the target large language model. At this point, the model includes both the general language understanding capabilities of the pre-trained model and the specific capabilities of understanding the professional knowledge of the target business field and identifying hot events.

[0043] Through the above two-stage training and parameter superposition, the target large language model can more accurately identify hot events in newly received work order data, provide more in-depth analysis and more timely responses, and thus achieve more efficient and intelligent information processing and event management in the public service field. This model training method not only reduces the demand for a large amount of computing resources, but also improves the pertinence and practicality of the model. It is an innovative application of natural language processing technology in the public service hotline scenario.

[0044] According to some other optional embodiments of the present application, the first largest language model parameter is the product of a first matrix and a second matrix, wherein the dimension of the first matrix is ​​smaller than that of the second matrix; and the second largest language model parameter is a third matrix of rank 3.

[0045] In this scheme, the design of the first and second language model parameters adopts a special matrix structure, the purpose is to reduce the number of parameters and computing resources required for training while maintaining the model performance. Specifically, the first language model parameter (Wadapter1) is decomposed into the product of two matrices, that is, Wadapter1 = A*B, where the dimensions of the first matrix A and the second matrix B satisfy that the output dimension of A is equal to the input dimension of B, and the output dimension of A (which is also the input dimension of B) is much smaller than the input and output dimensions of the initial model (pre-trained large model). This design enables Wadapter1 to capture task-specific features with fewer parameters without significantly increasing the complexity of the overall model, thereby saving computing resources. The dimensional design of A and B is based on the principle of low-rank approximation. By multiplying two smaller matrices, effective fine-tuning of the large model parameters is achieved while maintaining the generalization ability of the model. The second language model parameter (Wadapter2) is a third matrix of rank 3. In mathematics, the rank of a matrix represents the maximum number of linearly independent rows or columns in the matrix. A rank 3 matrix means that it has three linearly independent vectors. This ultra-low rank design is to further reduce the number of parameters, especially when dealing with short texts and domain-focused semantics, Wadapter2 can quickly capture key information without requiring a lot of training data and computing resources. By fixing the parameters of Wbase and Wadapter1 and only training Wadapter2, the system can efficiently adapt to the knowledge of specific fields while maintaining the ability to process large-scale data sets.

[0046] In some optional embodiments of the present application, before using the professional knowledge of the target business field to train the first language model, the following steps can also be performed: obtaining the professional knowledge of the target business field, historical work order data of the target business field, and hot event information corresponding to the historical work order data, wherein the historical work order data includes at least one of the following: the timestamp of the historical work order, the request location of the historical work order, and the work order description information of the historical work order; deleting invalid characters, spaces, punctuation marks, and non-standard encodings in the historical work order data; identifying and removing duplicate historical work order data based on a text similarity algorithm; converting the historical work order data into standard text, wherein the conversion process at least includes: removing HTML tags, adjusting the text encoding to a first preset format, converting the time field to a second preset format, etc.

[0047] Collect professional knowledge from the target business field, including industry reports, expert articles, and literature, to help the model understand the language and concepts of a specific field. At the same time, collect historical work order data from the target business field. This data includes at least the timestamp, location of the request, and work order description information of the work order, providing the model with specific service scenarios and event backgrounds. In addition, it is also necessary to obtain the hot event information corresponding to the historical work order data to guide the model to learn the ability to identify similar events. The work order data may contain various invalid characters, blanks, punctuation marks, and non-standard encodings. These elements are not helpful for the model to understand the meaning of the text, but may introduce noise and affect the performance of the model. Therefore, it is necessary to identify and delete these invalid elements through programming methods (such as regular expressions) to ensure the purity and consistency of the data. There may be duplicates in the work order data, that is, multiple work orders describe the same event or request. In order to avoid data deviation during training, text similarity algorithms (such as cosine similarity, Jaccard similarity, etc.) need to be used to identify and remove duplicate work order data to ensure that the model can learn from diverse data and improve the accuracy of identifying hot events. Historical work order data may come from different systems or platforms, and there are problems with inconsistent formats. In order to ensure that the data can be correctly read and understood by the model, the data needs to be converted into a standard text format. The specific steps include: The work order data may contain HTML tags for formatting. These tags are not helpful for the model to understand the text content and need to be removed using an HTML parsing library or regular expressions. The original text encoding of the work order data may not be uniform and needs to be uniformly adjusted to the first preset format (such as UTF-8) to prevent data reading problems caused by encoding errors. The timestamp of the work order may exist in different formats, such as YYYY-MM-DD or DD / MM / YYYY. It needs to be uniformly converted to the second preset format, such as the ISO 8601 standard (YYYY-MM-DDTHH:MM:SSZ), so that the model can understand and process time information.

[0048] Through the above steps, the quality of data can be effectively improved, ensuring that the data input in the model training phase is accurate, thereby improving the accuracy and efficiency of hot event identification and analysis. Standardized data processing processes help reduce uncertainty in the model training process, ensuring that the model can learn and optimize based on a comprehensive and consistent data set, and ultimately achieve more accurate event identification and more effective report generation.

[0049] As some optional embodiments of the present application, inputting work order data into a target large language model to obtain hot event information corresponding to the work order data output by the target large language model can be achieved by the following method: dividing the work order data into n groups of sub-data according to the number of words in the work order data, wherein n is a positive integer greater than 1, and n is proportional to the number of words in the work order data; inputting each group of sub-data into the target large language model by respectively calling the model interface of the target large language model to obtain the hot event information corresponding to each of the n groups of sub-data; summarizing and deduplicating the hot event information corresponding to each of the n groups of sub-data to obtain the hot event information of all the work order data.

[0050] First, count the number of words in all collected work order data. Based on the total number of words in the work order data, divide it into n groups of sub-data, where n is a positive integer greater than 1, and the value of n is proportional to the total number of words in the work order data. For example, if the total number of words in the work order data is large, it may be divided into more groups to improve processing efficiency; conversely, if the amount of data is small, the number of groups divided will be reduced accordingly. The number of words in each group of sub-data should be controlled within a certain range to ensure that the model can effectively process each group of data. The upper limit of the number of words in each group of sub-data is set to 100,000 words, which means that each group of work order data will not exceed this number of words to avoid affecting the response speed and performance of the model due to excessive data volume. Next, call the model interface of the target large language model to process each group of sub-data. Specifically, each group of sub-data will be input into the target large language model that has been fine-tuned with professional knowledge and historical work order data. The model will output the hot event information corresponding to each group of sub-data based on its trained parameters and language understanding ability. After completing the processing of all sub-data, the system will summarize the obtained n groups of hot event information. On this basis, duplicate hot event information is identified and removed through a data deduplication algorithm to ensure that the final output hot event information set is refined and unique, avoiding multiple reports on the same event and improving the practicality and readability of the report.

[0051] In some optional embodiments of the present application, sending the hot event information corresponding to the work order data to the target object can be achieved by the following method: aggregating the hot event information corresponding to n groups of sub-data, and deduplicating the aggregated results; sorting the deduplicated hot event information according to preset sorting rules, wherein the preset sorting rules include: the influence of the hot event and the urgency of the hot event; classifying the deduplicated hot event information according to preset classification rules, wherein the preset classification rules include: the type and subject of the hot event; combining the sorted and classified hot event information with the work order description information corresponding to each group of sub-data, and filling the combined results into a predefined report template to obtain a hot event report, wherein the hot event report includes at least one of the following: analysis content of the hot event, work order summary, event keywords and suggested measures; sending the hot event report to the hotline operating agency.

[0052] Summarize the hot event information corresponding to n groups of sub-data to form a preliminary list of all potential hot events. De-duplicate this preliminary list to ensure that each hot event is reported only once. De-duplication can be achieved by comparing the title, description or keywords of the event, avoiding repeated mentions of the same event, making the report more refined and valuable. Sort the de-duplicated hot event information according to the preset sorting rules. The sorting rules include at least the influence and urgency of the hot event, which can be based on factors such as the frequency of event mentions, sentiment analysis of the work order, the scope of involvement or the need for emergency response. Sorting can ensure that the most important events receive priority attention in the report, which helps the hotline operator to quickly locate matters that need to be handled urgently. Classify the de-duplicated hot event information according to the preset classification rules, such as the type and subject of the event.

[0053] Combine the sorted and classified hot event information with the work order description information corresponding to each group of sub-data to generate a detailed description of the hot event. Then, fill this information into the predefined report template to generate a complete hot event report. The report template contains at least the analysis content of the hot event, the work order summary, the event keywords and the recommended measures, which provide the operating agency with a comprehensive overview of the event and handling suggestions. Finally, the system sends the generated hot event report to the hotline operating agency, and the report can be distributed via email, internal messaging system or a dedicated report distribution platform. After receiving the report, the operating agency can quickly respond to the needs of citizens based on the content of the report, take necessary measures to solve hot issues, and improve service quality and public satisfaction.

[0054] As other optional embodiments of the present application, after sending the hot event information corresponding to the work order data to the target object, the following steps can also be performed: obtaining the feedback results of the target object on the hot event information corresponding to the work order data; analyzing the feedback results, if the analysis results indicate that the low recognition rate of hot events is due to insufficient knowledge understanding of the target business field, adjusting the parameters of the second largest language model; if the analysis results indicate that the low recognition rate of hot events is due to insufficient scenario-based capabilities, adjusting the parameters of the first largest language model.

[0055] Receive feedback from target objects on hot event reports on a regular or real-time basis. This feedback may include corrections to false positives and omissions of specific hot events, as well as evaluations of the accuracy and efficiency of hot event recognition. Feedback results can be collected through dedicated feedback channels, such as email, online forms, or direct system interfaces. Analyze the collected feedback results to determine the reasons for the low recognition rate of hot events. The analysis may include methods such as data statistics, model performance evaluation, and user comment analysis. If the low recognition rate is caused by the model's insufficient understanding of the professional knowledge of the target business field, this may be manifested as the model's inability to correctly recognize certain professional terms or domain-specific expressions; if the low recognition rate is caused by the model's insufficient scenario-based capabilities, this is usually because the model cannot accurately judge the importance and urgency of the event in the specific event context.

[0056] If the feedback analysis shows that the low recognition rate of hot events is due to insufficient understanding of the target business domain, the system will adjust the parameters of the second largest language model (Wadapter2). Wadapter2 is an ultra-low rank matrix designed to capture domain-specific short text and semantic features. Adjusting the parameters of Wadapter2 can further fine-tune the model by adding or updating training data related to the target domain, so that it can better understand and recognize professional terms and expressions in the domain. In addition, the model's ability to capture domain knowledge can be optimized by adjusting the specific parameter values ​​in Wadapter2.

[0057] If the feedback analysis shows that the low recognition rate of hot events is due to insufficient scenario-based capabilities, the system will adjust the first language model parameter (Wadapter1). Wadapter1 is designed to enhance the performance of the model in specific task scenarios (such as hot event recognition). Adjusting the Wadapter1 parameters can enhance the model's scenario perception capabilities through further training in specific scenarios, such as using a more focused hot event dataset. Adjusting Wadapter1 is intended to optimize the model's judgment and generalization capabilities when dealing with specific types of events, ensuring that it can more accurately identify the key information and urgency of the event.

[0058] After adjusting the parameters of Wadapter1 or Wadapter2, the adjusted parameters are added to the large language model parameters (Wbase) to form the updated target large language model parameters. The updated model parameters will be used to retrain or adjust the model to improve its accuracy and efficiency in hot event identification and analysis tasks.

[0059] Figure 2 is a flow chart of another data processing method according to an embodiment of the present application. Figure 2 As shown, the method comprises the following steps:

[0060] Step S201, data collection and processing: Collect historical work order data and corresponding hot event information in a specific field, including the time of the work order, the location of the request, the event type, the detailed description of the work order, etc. Pre-process the collected work order data, including data cleaning, deduplication, format conversion, etc., for subsequent analysis.

[0061] For example, based on the currently open source big model chatglm3, we train and fine-tune the private data to form a dedicated big model. The specific method is: collect data in the field of people's livelihood, such as hot events (100) in news reports and historical weekly reports and corresponding work order data. Each hot event corresponds to 50-200 work order data, and clean the work order data.

[0062] Step S202, large model training: using the preprocessed data, using the open source general large model chatglm as the base, train a large model specific to this field, which can identify and mine hot events. The training process uses a variant of LoRA (Low-Rank Adaptation) to fine-tune the large model.

[0063] It should be noted that the traditional LoRA decomposes the large model parameters into two matrices: one contains the original parameters Wbase, which contains the weights of the large model and is frozen (i.e. not updated) during fine-tuning; the other is the low-rank matrix Wadapter, which contains a small number of trainable parameters.

[0064] In view of the short text of citizen hotline work order data and the content focusing on people's livelihood, this embodiment designs a variant of the LoRA fine-tuning architecture, adopts two low-rank matrices, and the decomposed parameter W' can be expressed as: W'=Wbase+Wadapter1+Wadapter2.

[0065] Among them, Wadapter1 is a low-rank matrix consistent with the original lora structure, expressed as the product of two smaller matrices A and B, that is: Wadapter1 = A*B, the dimensions of A and B are much smaller than W', so they contain a small number of trainable parameters. Its role is to adapt to specific changes in specific tasks in the current field when fine-tuning the large model using scenario data. Wadapter2 is an ultra-low-cost matrix with a rank of only 3. Its role is to collect short texts and domain-focused semantics in the field when using the large model in the initial stage to do domain fine-tuning. In this way, the superposition of Wadapter2 and Wadapter1 not only ensures the effect of the original LoRA without increasing too much calculation, but also grasps the characteristics of hotline data in the people's livelihood field, thereby greatly reducing the demand for hardware resources. The specific steps are as follows:

[0066] (1) First, fix Wbase and Waadapter1, and use the common data in the field where the task is located to fine-tune the large model. Because the rank of Waadapter2 is only 3, it can quickly adapt to the short text and domain-focused semantic features of the field. (2) Then fix Wbase and Waadapter2, and use the scenario data of a specific task to fine-tune the large model. During the fine-tuning process, Waadapter1 learns the scenario capabilities of the specific task. In this way, Wbase can remain unchanged during the entire fine-tuning process, greatly reducing the parameters of model fine-tuning. At the same time, Waadapter1 and Waadapter2 can learn domain knowledge from different perspectives, increasing the effect and efficiency of fine-tuning.

[0067] For example, according to the requirements of the large model training format, prompt, input and output are sorted, and the variant LoRA fine-tuning method proposed in this embodiment is used. First, the data in the people's livelihood field is used to fix the full parameter Wbase of the large model and the first part of the parameter Waadapter1 of LoRa, and the model of the first stage is trained to obtain the Waadapter2 parameters of the variant LoRA; then the hot events and work order data of the weekly report are used to train the second stage, the full parameter Wbase of the large model and the second part of the parameter Waadapter2 of LoRa are fixed, and the model of the second stage is trained to obtain the Waadapter1 parameters of the variant LoRA, thereby obtaining an exclusive large model for this field, which not only has the knowledge attributes of the people's livelihood field, but also has the ability to understand work order information, extract hot events and summarize hot titles.

[0068] It should be noted that this embodiment proposes to add a new ultra-low rank matrix, that is, the training fine-tuning is divided into two stages. In the first stage, the ultra-low rank matrix is ​​fixed, and the model is trained using the traditional LoRa method to adapt it to the domain knowledge. In the second stage, the original LoRa bypass is fixed and the newly added ultra-low rank matrix is ​​trained to allow the ultra-low rank matrix to learn refined industry knowledge, so that the model can achieve the purpose of learning industry domain knowledge from different perspectives, thereby better improving the model effect.

[0069] In this way, the variant LoRa can not only maintain the performance of the chatglm model, but also greatly reduce the number of parameters that need to be trained. Through the dual-channel low-rank matrix, LoRa can be adapted from the perspectives of both the domain and the task, which can make the pre-trained model chatglm better adapt to the task of hot event extraction and analysis. The fine-tuned large model has the ability to discover hot events, can receive a batch of work order data (up to 100,000 words), and output the hot events mentioned in this batch of work orders.

[0070] Step S203, hot event discovery and analysis: the fine-tuned and trained large model is deployed for production, and the work order data within a certain period is received through the interface. The work order data is truncated and grouped according to the number of words. The number of work orders in each group is up to 100,000 words, and divided into n groups (the size of n is proportional to the number of work orders and the number of words). The n groups of work order data call the interface of the large model respectively. The fine-tuned large model gives n groups of hot events, and then the n groups of events are summarized and deduplicated to obtain the hot events of all work orders. For the discovered hot events, in-depth mining and analysis are carried out manually, such as the cause of the event, the scope of impact, and the people involved.

[0071] The above steps take advantage of the big model's ability to understand large-scale data, and fine-tune the variant LoRA for people's livelihood data, so that it can better extract hot events from work order data, fill the gap in automatic extraction of hot events, and improve the accuracy of hot event summarization.

[0072] Step S204, report generation and push: According to the hot events extracted above (assuming a total of k hot events), they are sorted into k groups again, each group is an event and all the work order data corresponding to the event, and each hot event and the corresponding work order information are sorted out. Finally, a complete hot event report is compiled, which describes the hot events and work order information at the current stage, and is pushed to the hotline operating organization to help them accurately and timely grasp the hot event situation and improve service quality and user satisfaction.

[0073] In actual application, for work orders in a certain period, the above-mentioned exclusive large model is called, and this batch of work orders is used as input. The exclusive large model gives k hot events and corresponding work order information, thereby completing the process of extracting hot events in this certain period. The operation personnel select some of the hot events according to needs, generate hot spot analysis, and finally integrate them into a complete weekly report.

[0074] Through the above steps, the number of parameters that need to be trained is significantly reduced without sacrificing the performance of the large model. This variant LoRA technology not only maintains the advantages of large language models in understanding complex contexts and extracting key information, but also uses variant LoRA technology to fine-tune very few parameters during the training process, thereby greatly reducing the demand for hardware resources and reducing the cost of training and maintenance, making large model technology more feasible and efficient in actual application scenarios such as citizen hotline hotspot analysis. This technology can automatically process massive work order data, identify hot events, and provide comprehensive and in-depth data support for hotline managers, thereby improving service quality and user satisfaction.

[0075] Figure 3 is a structural diagram of a data processing device according to an embodiment of the present application, such as Figure 3 As shown, the device comprises:

[0076] The acquisition module 32 is used to acquire the work order data of the public service hotline.

[0077] The input module 34 is used to input the work order data into the target large language model to obtain the hot event information corresponding to the work order data output by the target large language model, wherein the target large language model is a large language model obtained by low-rank adaptive fine-tuning of the pre-trained large language model based on the historical work order data of the target business field, the hot event information corresponding to the historical work order data, and the professional knowledge of the target business field.

[0078] The sending module 36 is used to send the hot event information corresponding to the work order data to the target object.

[0079] Optionally, the target large language model is obtained by training in the following method: using professional knowledge of the target business field to train a first large language model to obtain parameters of a second target large language model, wherein, in the first large language model, all parameters of the large language model and parameters of the first large language model are fixed parameters, the parameters of the second large language model are non-fixed parameters, and all parameters of the large language model are parameters in a pre-trained large language model; using historical work order data and hot event information corresponding to the historical work order data to train a second large language model to obtain parameters of a first target large language model, wherein, in the second large language model, all parameters of the large language model and parameters of the second target large language model are fixed parameters, and parameters of the first large language model are non-fixed parameters; superimposing the second target large language model parameters and the first target large language model parameters onto the full parameters of the large language model to obtain model parameters of the target large language model; generating a target large language model according to the model parameters of the target large language model.

[0080] Optionally, the first largest language model parameter is the product of a first matrix and a second matrix, wherein the dimension of the first matrix is ​​smaller than that of the second matrix; and the second largest language model parameter is a third matrix of rank 3.

[0081] Optionally, the data processing device is also used to perform the following steps before using the professional knowledge of the target business field to train the first language model: obtaining the professional knowledge of the target business field, historical work order data of the target business field, and hot event information corresponding to the historical work order data, wherein the historical work order data includes at least one of the following: the timestamp of the historical work order, the request location of the historical work order, and the work order description information of the historical work order; deleting invalid characters, spaces, punctuation marks, and non-standard encodings in the historical work order data; identifying and removing duplicate historical work order data based on a text similarity algorithm; converting the historical work order data into standard text, wherein the conversion process at least includes: removing HTML tags, adjusting the text encoding to a first preset format, converting the time field to a second preset format, etc.

[0082] Optionally, the input module 34 is also used to perform the following steps: dividing the work order data into n groups of sub-data according to the number of words in the work order data, wherein n is a positive integer greater than 1, and n is proportional to the number of words in the work order data; inputting each group of sub-data into the target large language model by respectively calling the model interface of the target large language model to obtain hot event information corresponding to each of the n groups of sub-data; summarizing and deduplicating the hot event information corresponding to each of the n groups of sub-data to obtain the hot event information of all the work order data.

[0083] Optionally, the sending module 36 is also used to perform the following steps: summarize the hot event information corresponding to n groups of sub-data, and deduplicate the summary results; sort the deduplicated hot event information according to preset sorting rules, wherein the preset sorting rules include: the influence of the hot event and the urgency of the hot event; classify the deduplicated hot event information according to preset classification rules, wherein the preset classification rules include: the type and theme of the hot event; combine the sorted and classified hot event information with the work order description information corresponding to each group of sub-data, and fill the combination result into a predefined report template to obtain a hot event report, wherein the hot event report includes at least one of the following: analysis content of the hot event, work order summary, event keywords and recommended measures; send the hot event report to the hotline operating agency.

[0084] Optionally, the data processing device is also used to perform the following steps after sending the hot event information corresponding to the work order data to the target object: obtaining the feedback results of the target object on the hot event information corresponding to the work order data; analyzing the feedback results, if the analysis results indicate that the low recognition rate of hot events is due to insufficient knowledge understanding of the target business field, adjusting the parameters of the second largest language model; if the analysis results indicate that the low recognition rate of hot events is due to insufficient scenario-based capabilities, adjusting the parameters of the first largest language model.

[0085] It should be noted that the above Figure 3 The modules in the embodiment may be program modules (e.g., a set of program instructions for implementing a specific function) or hardware modules. For the latter, they may be expressed in the following forms, but are not limited thereto: the expression form of the above modules is a processor, or the functions of the above modules are implemented by a processor.

[0086] It should be noted that Figure 3 The preferred implementation of the illustrated embodiment can be found in Figure 1 The relevant description of the illustrated embodiment will not be repeated here.

[0087] Figure 4 FIG. 1 shows a hardware structure block diagram of a computer terminal for implementing a data processing method. Figure 4As shown, the computer terminal 40 may include one or more (402a, 402b, ..., 402n are used to illustrate) processors 402 (the processor 402 may include but is not limited to a processing device such as a microprocessor MCU or a programmable logic device FPGA), a memory 404 for storing data, and a transmission module 406 for communication functions. In addition, it may also include: a display, an input / output interface (I / O interface), a universal serial bus (USB) port (which may be included as one of the ports of the BUS bus), a network interface, a power supply and / or a camera. It can be understood by those skilled in the art that Figure 4 The structure shown is only for illustration and does not limit the structure of the above electronic device. Figure 4 More or fewer components as shown, or with Figure 4 Different configurations are shown.

[0088] It should be noted that the one or more processors 402 and / or other data processing circuits described above may generally be referred to herein as "data processing circuits". The data processing circuits may be embodied in whole or in part as software, hardware, firmware, or any other combination thereof. In addition, the data processing circuit may be a single independent processing module, or may be incorporated in whole or in part into any of the other components in the computer terminal 40. As involved in the embodiments of the present application, the data processing circuit acts as a processor control (e.g., selection of a variable resistor terminal path connected to an interface).

[0089] The memory 404 can be used to store software programs and modules of application software, such as program instructions / data storage devices corresponding to the data processing method in the embodiment of the present application. The processor 402 executes various functional applications and data processing by running the software programs and modules stored in the memory 404, that is, realizing the above-mentioned data processing method. The memory 404 may include a high-speed random access memory, and may also include a non-volatile memory, such as one or more magnetic storage devices, flash memory, or other non-volatile solid-state memory. In some examples, the memory 404 may further include a memory remotely arranged relative to the processor 402, and these remote memories may be connected to the computer terminal 40 via a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.

[0090] The transmission module 406 is used to receive or send data via a network. The specific example of the above network may include a wireless network provided by a communication provider of the computer terminal 40. In one example, the transmission module 406 includes a network adapter (Network Interface Controller, NIC), which can be connected to other network devices through a base station so as to communicate with the Internet. In one example, the transmission module 406 can be a radio frequency (RF) module, which is used to communicate with the Internet wirelessly.

[0091] The display may be, for example, a touch screen liquid crystal display (LCD) that enables a user to interact with a user interface of the computer terminal 40 .

[0092] It should be noted that, in some optional embodiments, the above Figure 4 The computer terminal shown may include hardware elements (including circuits), software elements (including computer code stored on a computer-readable medium), or a combination of hardware elements and software elements. It should be noted that Figure 4 This is merely one example of a particular embodiment and is intended to illustrate the types of components that may be present in the computer terminal described above.

[0093] It should be noted that Figure 4 The computer terminal shown is used to execute Figure 1 The data processing method shown, therefore the relevant explanations in the execution method of the above command are also applicable to the electronic device and will not be repeated here.

[0094] An embodiment of the present application further provides a non-volatile storage medium, which includes a stored program, wherein when the program is executed, the device where the storage medium is located is controlled to execute the above data processing method.

[0095] A program for a non-volatile storage medium to perform the following functions: obtaining work order data of a public service hotline; inputting the work order data into a target large language model to obtain hot event information corresponding to the work order data output by the target large language model, wherein the target large language model is a large language model obtained by performing low-rank adaptive fine-tuning on a pre-trained large language model based on historical work order data of a target business field, hot event information corresponding to the historical work order data, and professional knowledge of the target business field; and sending the hot event information corresponding to the work order data to a target object.

[0096] An embodiment of the present application further provides an electronic device, including: a memory and a processor, wherein the processor is used to run a program stored in the memory, wherein the above data processing method is executed when the program is run.

[0097] The processor is used to run a program that performs the following functions: obtaining work order data of a public service hotline; inputting the work order data into a target large language model to obtain hot event information corresponding to the work order data output by the target large language model, wherein the target large language model is a large language model obtained by performing low-rank adaptive fine-tuning on a pre-trained large language model based on historical work order data of a target business field, hot event information corresponding to the historical work order data, and professional knowledge of the target business field; and sending the hot event information corresponding to the work order data to a target object.

[0098] The serial numbers of the above-mentioned embodiments of the present application are for description only and do not represent the advantages or disadvantages of the embodiments.

[0099] In the above embodiments of the present application, the description of each embodiment has its own emphasis. For parts that are not described in detail in a certain embodiment, please refer to the relevant description of other embodiments.

[0100] In the above-mentioned embodiments of the present application, the collected information is information and data authorized by the user or fully authorized by all parties, and the collection, storage, use, processing, transmission, provision, disclosure and application of the relevant data comply with relevant laws, regulations and standards, take necessary protection measures, do not violate public order and good morals, and provide corresponding operation entrances for users to choose to authorize or refuse.

[0101] In the several embodiments provided in this application, it should be understood that the disclosed technical content can be implemented in other ways. Among them, the device embodiments described above are only schematic. For example, the division of the units can be a logical function division. There may be other division methods in actual implementation. For example, multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of units or modules, which can be electrical or other forms.

[0102] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple units. Some or all of the units may be selected according to actual needs to achieve the purpose of the present embodiment.

[0103] In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of software functional units.

[0104] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application, or the part that contributes to the relevant technology or all or part of the technical solution, can be embodied in the form of a software product, which is stored in a storage medium and includes several instructions for a computer device (which can be a personal computer, a server or a network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present application. The aforementioned storage medium includes: U disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), mobile hard disk, disk or optical disk, etc. Various media that can store program codes.

[0105] The above is only a preferred implementation of the present application. It should be pointed out that for ordinary technicians in this technical field, several improvements and modifications can be made without departing from the principles of the present application. These improvements and modifications should also be regarded as the scope of protection of the present application.

Claims

1. A data processing method, characterized in that: include: Get work order data for the public service hotline; Input the work order data into a target large language model to obtain hot event information corresponding to the work order data output by the target large language model, wherein the target large language model is a large language model obtained by performing low-rank adaptive fine-tuning on a pre-trained large language model based on historical work order data of a target business field, hot event information corresponding to the historical work order data, and professional knowledge of the target business field; The hot event information corresponding to the work order data is sent to the target object.

2. The method according to claim 1, characterized in that The target large language model is obtained by training using the following method: Using the professional knowledge of the target business field, training a first large language model to obtain parameters of a second target large language model, wherein in the first large language model, all large language model parameters and the first large language model parameters are fixed parameters, and the second large language model parameters are non-fixed parameters, and the all large language model parameters are parameters in the pre-trained large language model; Using the historical work order data and the hot event information corresponding to the historical work order data, training a second large language model to obtain first target large language model parameters, wherein, in the second large language model, all large language model parameters and the second target large language model parameters are fixed parameters, and the first large language model parameters are non-fixed parameters; Adding the second target large language model parameters and the first target large language model parameters to the large language model full parameters to obtain model parameters of the target large language model; The target large language model is generated according to the model parameters of the target large language model.

3. The method according to claim 2, characterized in that The first largest language model parameter is the product of a first matrix and a second matrix, wherein the dimension of the first matrix is ​​smaller than that of the second matrix; and the second largest language model parameter is a third matrix of rank 3.

4. The method according to claim 2, characterized in that: Before training the first language model using the expertise in the target business field, the method further includes: Acquire professional knowledge of the target business field, historical work order data of the target business field, and hot event information corresponding to the historical work order data, wherein the historical work order data includes at least one of the following: the timestamp of the historical work order, the location of the historical work order, and the work order description information of the historical work order; Delete invalid characters, spaces, punctuation marks and non-standard codes in the historical work order data; identify and remove duplicate historical work order data based on a text similarity algorithm; convert the historical work order data into standard text, wherein the conversion process at least includes: removing HTML tags, adjusting text encoding to a first preset format, converting time fields to a second preset format, etc.

5. The method according to claim 1, characterized in that Input the work order data into the target large language model, Obtaining hot event information corresponding to the work order data output by the target large language model, including: According to the number of words in the work order data, the work order data is divided into n groups of sub-data, wherein n is a positive integer greater than 1, and n is proportional to the number of words in the work order data; By respectively calling the model interface of the target large language model, each group of the sub-data is input into the target large language model to obtain the hot event information corresponding to each of the n groups of the sub-data; The hot event information corresponding to each of the n groups of sub-data is aggregated and deduplicated to obtain the hot event information of all the work order data.

6. The method according to claim 5, characterized in that Sending the hot event information corresponding to the work order data to the target object includes: Summarize the hot event information corresponding to the n groups of sub-data, and perform deduplication processing on the summary results; Sorting the hot event information after deduplication according to a preset sorting rule, wherein the preset sorting rule includes: the influence of the hot event and the urgency of the hot event; Classify the hot event information after deduplication according to preset classification rules, wherein the preset classification rules include: the type and theme of the hot event; Combining the sorted and classified hot event information with the work order description information corresponding to each group of the sub-data, and filling the combination result into a predefined report template to obtain a hot event report, wherein the hot event report includes at least one of the following: analysis content of the hot event, work order summary, event keywords and recommended measures; The hot event report is sent to the hotline operating agency.

7. The method according to claim 2, characterized in that After sending the hot event information corresponding to the work order data to the target object, the method further includes: Obtaining feedback results from the target object on the hot event information corresponding to the work order data; Analyze the feedback results. If the analysis results indicate that the low recognition rate of hot events is due to insufficient knowledge understanding of the target business field, adjust the parameters of the second language model; if the analysis results indicate that the low recognition rate of hot events is due to insufficient scenario-based capabilities, adjust the parameters of the first language model.

8. A data processing device, characterized in that: include: The acquisition module is used to obtain the work order data of the public service hotline; An input module, used to input the work order data into a target large language model, and obtain hot event information corresponding to the work order data output by the target large language model, wherein the target large language model is a large language model obtained by performing low-rank adaptive fine-tuning on a pre-trained large language model based on historical work order data of a target business field, hot event information corresponding to the historical work order data, and professional knowledge of the target business field; The sending module is used to send the hot event information corresponding to the work order data to the target object.

9. A non-volatile storage medium, characterized in that: The non-volatile storage medium includes a stored program, wherein when the program is executed, the device where the non-volatile storage medium is located is controlled to execute the data processing method according to any one of claims 1 to 7.

10. An electronic device, characterized in that: include: A memory and a processor, wherein the processor is used to run a program stored in the memory, wherein the program executes the data processing method according to any one of claims 1 to 7 when running.

11. A computer program product, comprising a computer program, characterized in that When the computer program is executed by a processor, the data processing method according to any one of claims 1 to 7 is implemented.

Images