Text intention analysis method and device, equipment and medium

Abstract

The invention relates to the technical field of natural language processing, can be applied to business system platforms of financial science and technology, medical treatment and health and the like, and discloses a text intention analysis method, device and equipment and a medium. Dividing the original data according to the obtained target division strategy to obtain a plurality of target sub-data sets, screening a plurality of obtained federal parameter efficient fine-tuning algorithms according to the obtained client resource condition and fine-tuning task requirements to obtain a target fine-tuning algorithm, and adjusting a preset large language model to obtain a fine-tuning result. And carrying out resource consumption analysis on the adjustment process, carrying out performance optimization on the preliminary adjustment model to obtain a target large language model, and carrying out text language analysis on the obtained target text data by utilizing the target large language model to determine a target text intention. According to the method, the stability of the analysis performance of the large language model and the interpretability of the text intention are effectively improved.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of natural language processing, and in particular to a text intent analysis method, device, equipment and medium. BACKGROUND

[0002] With the wide application of large language models (LLM) in the field of natural language processing, the demand for fine-tuning in specific fields is increasing. In the prior art, although the large language model is fine-tuned, and the text intent analysis using the fine-tuned large language model has excellent semantic understanding ability and transfer learning effect, there are still some significant shortcomings.

[0003] In the field of medical health, text intent analysis can achieve patient intent recognition, symptom consultation classification, medical question and answer understanding, etc. through fine-tuning of a large language model, and is widely used in intelligent guidance, online consultation and electronic medical record analysis scenes. For example, the system can accurately identify the patient's intent for "appointment registration" or "symptom consultation", realize automatic answering and guidance, and effectively improve the efficiency of medical services. However, due to the problems of data privacy, model stability and model interpretability, this technology still needs to be carefully weighed in actual deployment.

[0004] In the field of financial technology business, text intent analysis can be used in customer service automation, risk prompt identification, complaint classification and transaction behavior analysis scenes. Through the fine-tuned large language model, the system can accurately understand the specific intent of the user when consulting loans, reporting lost bank cards or querying bills, so as to provide personalized services or automatically transfer to the corresponding business module. Although this technology significantly improves the response speed and accuracy of financial services, it still faces challenges in risk control, model stability and model interpretability, especially the controllability and interpretability of the model.

[0005] In summary, the fine-tuning process usually requires high computing resources and GPU support, and the training cost is high; the model is sensitive to data distribution, and is prone to overfitting or unstable performance in label imbalance or low resource scenarios; the large model lacks interpretability, making it difficult to understand and trace the intent judgment result, limiting its application in some high-risk scenarios (such as medical and financial).

[0006] Therefore, in the prior art, there are problems of poor stability of large language model analysis performance and poor interpretability of text intent. SUMMARY

[0007] The present application provides a text intent analysis method, device, equipment and medium, which mainly aims to solve the problems of poor stability of large language model analysis performance and poor interpretability of text intent.

[0008] In a first aspect, to achieve the above object, the present application provides a text intention analysis method, comprising: obtaining original data of a target field and fine-tuning task requirements, selecting a division strategy for the original data according to the fine-tuning task requirements, and obtaining a target division strategy; dividing the original data according to the target division strategy, and obtaining a plurality of target sub-data sets; obtaining a client resource status, screening a plurality of obtained federated parameter efficient fine-tuning algorithms according to the client resource status and the fine-tuning task requirements, and obtaining a target fine-tuning algorithm; adjusting a preset large language model using the target sub-data set and the target fine-tuning algorithm, and obtaining a preliminary adjustment model; analyzing resource consumption of the adjustment process of the large language model, obtaining resource consumption, and optimizing performance of the preliminary adjustment model according to the resource consumption, and obtaining a target large language model; obtaining target text data of the target field, performing text language analysis on the target text data using the target large language model, and determining a target text intention.

[0009] In a second aspect, the present application further provides a text intention analysis device, comprising: a division strategy selection module configured to obtain original data of a target field and fine-tuning task requirements, select a division strategy for the original data according to the fine-tuning task requirements, and obtain a target division strategy; an original data division module configured to divide the original data according to the target division strategy, and obtain a plurality of target sub-data sets; a fine-tuning algorithm screening module configured to obtain a client resource status, screen a plurality of obtained federated parameter efficient fine-tuning algorithms according to the client resource status and the fine-tuning task requirements, and obtain a target fine-tuning algorithm; a large language model adjustment module configured to adjust a preset large language model using the target sub-data set and the target fine-tuning algorithm, and obtain a preliminary adjustment model; a large language model optimization module configured to analyze resource consumption of the adjustment process of the large language model, obtain resource consumption, and optimize performance of the preliminary adjustment model according to the resource consumption, and obtain a target large language model; a text intention analysis module configured to obtain target text data of the target field, perform text language analysis on the target text data using the target large language model, and determine a target text intention.

[0010] In a third aspect, the present application further provides an electronic device, comprising: at least one processor; and a memory connected with the at least one processor in communication; wherein the memory stores a computer program executable by the at least one processor, and the computer program is executed by the at least one processor to enable the at least one processor to perform the text intent analysis method described above.

[0011] In a fourth aspect, the present application further provides a computer readable storage medium, wherein the computer readable storage medium stores at least one computer program, and the at least one computer program is executed by a processor in an electronic device to implement the text intent analysis method described above.

[0012] The application obtains original data of a target field and fine-tuning task requirements, selects a division strategy for the original data according to the fine-tuning task requirements, and obtains a target division strategy. This adaptive strategy selection not only improves the representativeness and diversity of the fine-tuning data, but also optimizes the model training effect and generalization ability, which helps to build a more robust and efficient large language model. According to the target division strategy, the original data is divided to obtain a plurality of target sub-data sets. The client resource status is obtained, and the plurality of obtained federal parameter efficient fine-tuning algorithms are screened according to the client resource status and the fine-tuning task requirements to obtain a target fine-tuning algorithm. The screening process flexibly matches the most suitable federal parameter efficient fine-tuning algorithm according to the model access permission, resource condition and task requirement of the client, has high adaptability and actual usability. The target sub-data set and the target fine-tuning algorithm are used to adjust a preset large language model to obtain a preliminary adjustment model. The initialization strategy is flexibly configured according to the type of the target fine-tuning algorithm, and the model initialization quality is effectively improved. The tokenization and encoding steps ensure that the data structure and model input depth are matched, and the training efficiency is improved. Through the cooperative optimization of the acceleration operator and the resource efficient operator, the model still obtains better performance under limited computing resources. The resource consumption of the adjustment process of the large language model is analyzed to obtain the resource consumption situation, and the performance of the preliminary adjustment model is optimized according to the resource consumption situation to obtain a target large language model. By combining the resource consumption analysis result and the process parameter diagram, the to-be-optimized parameters and modules in the large language model are adjusted in a targeted manner. The techniques such as compression, pruning and adapter simplification are used to dynamically reduce the resource occupation of inefficient or redundant parts, while retaining the core modules that contribute more to performance. Therefore, on the basis of ensuring the model accuracy, the running efficiency and deployment flexibility of the model are significantly improved. The target text data of the target field is obtained, and the target text data is analyzed by using the target large language model to determine the target text intention. The understanding ability of the system to user demand can be significantly improved to realize high-precision and high-robustness intention recognition. Compared with the traditional method, the use of the fine-tuning large language model has stronger semantic understanding and context modeling capability, can adapt to complex and diverse language expressions, and can effectively improve the stability of the large language model analysis performance and the explainability of the text intention. BRIEF DESCRIPTION OF DRAWINGS

[0013] In order to more clearly illustrate the technical solutions of the embodiments of the application, the following will briefly introduce the drawings needed to be used in the description of the embodiments of the application. Obviously, the drawings in the following description are only some embodiments of the application, and other drawings can be obtained by those skilled in the art without creating laboriousness.

[0014] Figure 1An application environment schematic diagram of a text intention analysis method in an embodiment of the present application; Figure 2 A flow schematic diagram of a text intention analysis method provided in an embodiment of the present application; Figure 3 A flow schematic diagram of a resource consumption analysis process in a text intention analysis method provided in an embodiment of the present application; Figure 4 A module schematic diagram of a text intention analysis device provided in an embodiment of the present application; Figure 5 A structure schematic diagram of an electronic device for implementing a text intention analysis method provided in an embodiment of the present application; Figure 6 Another structure schematic diagram of an electronic device for implementing a text intention analysis method provided in an embodiment of the present application.

[0015] The object, function characteristics and advantages of the present application will be further described with reference to the embodiments and the accompanying drawings. DETAILED DESCRIPTION

[0016] In order to make the person in the art better understand the technical solutions of the present disclosure, and to fully understand and implement the implementation process of the present disclosure how to apply technical means to solve technical problems and achieve corresponding technical effects, the technical solutions in the embodiments of the present disclosure will be clearly and completely described below with reference to the drawings in the embodiments of the present disclosure. Obviously, the described embodiments are only a part of the embodiments of the present disclosure, not all. The embodiments of the present disclosure and each feature in the embodiments can be combined with each other without conflict, and the technical solutions formed thereby are within the protection scope of the present disclosure. Based on the embodiments in the present disclosure, all other embodiments obtained by those of ordinary skill in the art without creative labor should be within the protection scope of the present disclosure.

[0017] It should be noted that the terms "first", "second", etc. in the specification and claims of the present disclosure and the above-mentioned drawings are used to distinguish similar objects, and do not necessarily describe a specific order or sequence. It should be understood that the data thus used can be interchanged under appropriate circumstances, so that the embodiments of the present disclosure described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "include" and "have" and any variations thereof are intended to cover non-exclusive inclusion, for example, a process, method, device, product or equipment including a series of steps or units does not have to be limited to those steps or units clearly listed, but can include other steps or units not clearly listed or inherent to these processes, methods, products or equipment.

[0018] The embodiment of the present application provides a text intention analysis method, and the execution subject of the text intention analysis method includes but is not limited to at least one of electronic devices such as a server, a terminal and the like which can be configured to execute the device provided by the embodiment of the present application. In other words, the text intention analysis method can be executed by software or hardware installed in a terminal device or a server device. The server includes but is not limited to a single server, a server cluster, a cloud server or a cloud server cluster and the like. The server can be a stand-alone server, or a cloud server providing cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communication, middleware services, domain name services, security services, content distribution networks (CDN), and basic cloud computing services such as big data and artificial intelligence platforms.

[0019] The text intention analysis method provided by the embodiment of the present application can be applied to, for example, Figure 1In the application environment, the client communicates with the server through the network. The server can obtain the original data of the target field and the fine-tuning task demand through the client, select the division strategy according to the fine-tuning task demand, obtain the target division strategy, the adaptive strategy selection not only improves the representativeness and diversity of the fine-tuning data, but also optimizes the effect and generalization ability of the model training, which is helpful to build more robust and efficient large language model, divide the original data according to the target division strategy to obtain a plurality of target sub-data sets, obtain the client resource status, and select the plurality of obtained federal parameter efficient fine-tuning algorithms according to the client resource status and the fine-tuning task demand to obtain the target fine-tuning algorithm, the selection process flexibly matches the most suitable federal parameter efficient fine-tuning algorithm according to the model access permission, resource condition and task demand of the client, which has high adaptability and actual usability, adjusts the preset large language model using the target sub-data set and the target fine-tuning algorithm to obtain a preliminary adjustment model, flexibly configures the initialization strategy according to the type of the target fine-tuning algorithm, effectively improves the model initialization quality, and ensures that the data structure and model input depth are matched to improve the training efficiency. Through the cooperative optimization of the acceleration operator and the resource efficient operator, the model still obtains better performance under limited computing resources. The resource consumption analysis of the adjustment process of the large language model is carried out to obtain the resource consumption situation, and the performance of the preliminary adjustment model is optimized according to the resource consumption situation to obtain the target large language model. By combining the resource consumption analysis result and the process parameter diagram, the to-be-optimized parameters and modules in the large language model are adjusted, and technical means such as compression, pruning and adapter simplification are adopted to dynamically reduce the resource occupation of inefficient or redundant parts, while retaining the core modules with greater performance contribution, thereby significantly improving the running efficiency and deployment flexibility of the model on the basis of ensuring the model accuracy. The target text data of the target field is obtained, and the target text data is analyzed by using the target large language model to determine the target text intention, which can significantly improve the understanding ability of the system to user demand and realize high-precision and high-robustness intention recognition. Compared with the traditional method, the fine-tuning large language model has stronger semantic understanding and context modeling ability, can adapt to complex and diverse language expressions, and can effectively improve the stability of the large language model analysis performance and the explainability of the text intention. Finally, the target text intention is output and fed back to the user client. The client can be, but is not limited to, various personal computers, notebook computers, smart phones, tablet computers and portable wearable devices. The server can be realized by an independent server or a server cluster composed of multiple servers. The application will be described in detail below through specific embodiments.

[0020] The following is explained in the description of the present application, the present application adjusts the to-be-optimized parameters and modules in the large language model by combining resource consumption analysis results and process parameter diagrams, dynamically reduces the resource occupation of inefficient or redundant parts by using compression, pruning and adapter simplification technical means, and retains the core modules which contribute more to performance, thereby significantly improving the running efficiency and stability of the model on the basis of ensuring the accuracy of the model.

[0021] Referring to Figure 2 Fig. 1 is a flowchart of a text intent analysis method provided by an embodiment of the present application. In this embodiment, the text intent analysis method comprises the following steps: S1, obtaining original data of a target field and fine-tuning task requirements, selecting a division strategy for the original data according to the fine-tuning task requirements, and obtaining a target division strategy.

[0022] In the embodiment of the present application, original data related to the target field is collected from multiple channels such as public data sets, field expert annotated data, and network crawled data, to ensure data diversity, representativeness and sufficiency. The original data is de-duplicated to obtain de-duplicated data after removing duplicate samples, irrelevant samples to the target field are identified and filtered to obtain more relevant filtered data. The filtered data is annotated according to the task requirements to form an annotated data set, the distribution difference value of the annotated data among different clients is calculated, and whether the data distribution is uniform is determined according to the distribution difference value, and the most suitable target division strategy is determined according to the data distribution.

[0023] In the specific scenario of medical health, medical data usually has great heterogeneity, and health records from different hospitals, clinics or patients may have distribution differences. By de-duplicating and filtering data irrelevant to disease prediction, the model can focus on relevant diseases and symptoms. Through annotation and difference value calculation, if the sample distribution of a certain type of disease is significantly different, Dirichlet distribution strategy can be used to handle it, which helps to balance the training of different types of patient data and enhances the generalization ability of the model. If the data is relatively balanced, uniform division strategy is used, which helps to share knowledge among hospitals and improve the accuracy and robustness of the model.

[0024] In the specific scenario of financial technology, financial data usually comes from different regions and customer groups, and the data distribution may have large differences. By de-duplicating and filtering data irrelevant to financial risk prediction, the model can only learn based on core financial features. In the difference value calculation process, if the data distribution of different regions or customer groups is significantly different, Dirichlet distribution strategy can better handle this uneven distribution and improve the accuracy and adaptability of the model. When the data sources are consistent, uniform division strategy is used to ensure the balance of the model in different markets and customer groups, and to optimize the prediction performance.

[0025] In the embodiments of the present application, the dividing strategy selection of the original data according to the fine-tuning task requirement obtains a target dividing strategy, which includes: The original data is de-duplicated to obtain de-duplicated data; Irrelevant data irrelevant to the target field in the de-duplicated data is identified, and the irrelevant data in the de-duplicated data is filtered to obtain filtered data; The filtered data is labeled according to the fine-tuning task requirement to obtain labeled data; The distribution difference value of the labeled data is determined, and it is judged whether the distribution difference value is greater than a preset difference threshold; If the distribution difference value is greater than the difference threshold, Dirichlet distribution dividing strategy is used as the target dividing strategy; If the distribution difference value is less than or equal to the difference threshold, uniform dividing strategy is used as the target dividing strategy.

[0026] In detail, the samples in the original data set are uniquely identified, usually based on specific fields (such as text content, ID, timestamp, etc.) for comparison, and data entries with exactly the same or highly similar content are identified. Remove these duplicate or redundant records and only keep one representative data sample. Finally, a non-duplicate and cleaner de-duplicated data set is obtained, laying a foundation for subsequent data screening and model training.

[0027] Through domain-related rules or using domain classification model, each de-duplicated data is matched with the domain and judged for relevance, and those data samples that do not meet the characteristics or theme of the target domain are screened out. These irrelevant data are excluded or filtered from the data set. The exclusion process can be based on keyword matching, semantic similarity or classification label, etc. Finally, only the filtered data highly related to the target domain is obtained, providing an accurate and focused data basis for subsequent labeling and model training.

[0028] According to the specific task type (such as classification, entity recognition, question and answer, etc.), the corresponding labeling specification and standard are formulated, and each sample in the filtered data is labeled by manual annotators or automatic labeling tools to ensure that the labeling content meets the task target and quality requirements. When necessary, multiple rounds of review and correction are carried out to improve the labeling accuracy, and finally high-quality labeled data meeting the fine-tuning requirements are generated to provide accurate supervision signals for model training.

[0029] The number of samples of each category or feature in the labeled data is counted to obtain the actual distribution of the data, a reference distribution is determined, which can be an ideal uniform distribution or a distribution mode expected by the task, and a difference measurement index between the actual distribution and the reference distribution, such as KL divergence, JS divergence or total variation distance, is calculated to quantify the difference between the two, that is, the distribution difference value, which is used to reflect the deviation degree of the data distribution, and then guide the selection of the division strategy.

[0030] If the difference is large, it means that the data distribution is biased towards certain categories or features, and the Dirichlet distribution division strategy that can reflect the diversity of data is suitable; if the difference is small, it means that the data is relatively balanced, and the uniform division strategy is used to ensure the fairness and representativeness of data division, and then provide a reasonable data basis for subsequent tasks.

[0031] Through systematic data preprocessing and intelligent division strategy selection, the quality and adaptability of the data used for fine-tuning are ensured. The redundancy and irrelevant data are effectively eliminated through the de-duplication and filtering steps, improving the purity and relevance of the data; the labeling process generates high-quality supervision information according to the specific task requirements. By calculating the distribution difference value of the labeled data, the balance of the data can be dynamically judged, and then a more suitable division strategy can be selected. This adaptive strategy selection not only improves the representativeness and diversity of the fine-tuning data, but also optimizes the effect of model training and generalization ability, which helps to build more robust and efficient large language models.

[0032] S2, dividing the original data according to the target division strategy to obtain a plurality of target sub-data sets.

[0033] In the embodiment of the application, if the target division strategy is a uniform division strategy, a uniform division ratio is obtained, and the labeled data is evenly distributed to each sub-data set according to the ratio, ensuring the consistency of each sub-set in data quantity and category distribution. If the target division strategy is a Dirichlet distribution division strategy, the category label information in the labeled data and the division heterogeneity parameter are obtained, the allocation ratio of each category label in each sub-set is randomly generated according to the Dirichlet distribution, and the heterogeneity parameter is combined to non-uniformly divide the labeled data, thereby generating a plurality of target sub-data sets with different distribution characteristics, to more realistically simulate the heterogeneity distribution of the client data in reality.

[0034] In the medical health specific scenario, for disease prediction or medical image analysis, patient data usually has different categories and distribution characteristics, such as medical record data of different regions and different hospitals. When the uniform division strategy is adopted, the disease sample of each category can be evenly distributed to different sub-data sets, so that the data of a certain disease category is not too much or too little. In the case of large regional differences or unbalanced cases, the Dirichlet distribution division strategy can dynamically allocate the data proportion based on the heterogeneity of different categories. For example, according to the distribution of a specific disease and the resource limit, the allocation proportion is randomly generated, so that the training data can better fit the data heterogeneity in the real clinical scene.

[0035] In the financial technology specific scenario, when processing customer data from different financial institutions and different regions, the problem of uneven data distribution often arises. When the uniform division strategy is adopted, the proportion of different types of customers (such as high-risk and low-risk customers) can be reasonably ensured. In complex financial scenarios, such as training customer data for different risk levels, the Dirichlet distribution division strategy can randomly generate the corresponding allocation proportion according to the distribution of different risk labels. Especially in the face of diversified data across regions and industries, the robustness of the model in processing highly heterogeneous data sets is further improved.

[0036] In the embodiment of the present application, the original data is divided according to the target division strategy to obtain a plurality of target sub-data sets, comprising: When the target division strategy is a uniform division strategy, the division proportion is obtained according to the uniform division strategy; The labeled data is divided according to the division proportion to obtain a plurality of target sub-data sets; When the target division strategy is a Dirichlet distribution division strategy, the category label and the division heterogeneity parameter of the labeled data are obtained; The allocation proportion of each category label is randomly generated according to the Dirichlet distribution division strategy; The labeled data is divided according to the allocation proportion and the division heterogeneity parameter to obtain a plurality of target sub-data sets.

[0037] In detail, when the target division strategy is a uniform division strategy, the proportion of each sub-data set is determined according to the uniform division rule, and the labeled data is usually divided into a plurality of target sub-data sets in order or randomly according to the division proportion, so that each subset is as balanced as possible in quantity and category distribution to meet the needs of subsequent model training or evaluation.

[0038] When the target partitioning strategy is the Dirichlet distribution partitioning strategy, the category labels and partitioning heterogeneity parameters in the labeled data are obtained to reflect the distribution differences of the data between categories. According to the partitioning heterogeneity parameter (usually the concentration parameter of the Dirichlet distribution), a Dirichlet distribution model is initialized. The parameters control the sparsity of the distribution ratio and the imbalance between categories. Sampling is done from the Dirichlet distribution to obtain the specific distribution ratio vector of each category label in each sub-dataset. Since the Dirichlet distribution itself has randomness and controllable diversity, the sampling process can generate different proportion distributions that meet the set heterogeneity level, ensuring that the data partitioning is both random and reflects the heterogeneity characteristics of the real data. Combining these allocation ratios and heterogeneity parameters, the labeled data is distributed into multiple target sub-datasets by category and ratio to simulate the heterogeneous distribution of data in real scenarios and support more representative and robust model training.

[0039] This approach combines uniform and Dirichlet distribution strategies to flexibly adapt to varying data distribution characteristics. The uniform partitioning strategy ensures a balanced distribution of data across subsets, contributing to model stability and fairness during training. The Dirichlet distribution strategy, by introducing heterogeneous parameters and randomized proportional sampling, realistically simulates data imbalance and diversity, improving the model's adaptability to complex real-world scenarios. This partitioning scheme ensures both the scientific and rational nature of data partitioning and the generalization and robustness of model training.

[0040] S3. Obtain client resource status, and screen multiple obtained federated parameter efficient fine-tuning algorithms according to the client resource status and the fine-tuning task requirements to obtain a target fine-tuning algorithm.

[0041] In this embodiment of the present invention, the model access status of the target client is obtained, and a determination is made as to whether the model access status grants access to the model weights. Further determination is then made based on resource availability. If the client can access the model, a specific algorithm from the efficient federated parameter fine-tuning algorithm is selected based on the task type. If the client cannot access the model, a federated optimization algorithm based on optimal transmission is employed. By modeling distribution differences, effective collaborative optimization of the model in inaccessible environments is achieved, ensuring a balance between privacy and training efficiency.

[0042] In the specific scenario of medical health, if a hospital has limited computing resources, a prompt fine-tuning algorithm can be used to reduce resource consumption and ensure that disease prediction or medical image analysis can still be performed under limited computing power. If the hospital has sufficient computing resources and the task requirement is a fine-tuning task (such as refining early diagnosis of a certain disease), a low-rank adaptive algorithm (LoRA) can be selected to make the fine-tuning process more efficient and accurate, and adapt to the characteristics of specific diseases; if the task focuses on text coherence optimization, such as optimizing the generation of electronic medical records or automatic writing of medical articles, a prompt-based fine-tuning algorithm can be used to make the generated text more coherent and readable. For remote hospitals that cannot access certain resources, a federated optimization algorithm based on optimal transport can be used to model and optimize data distribution, improve the collaboration efficiency of the model, and protect patient privacy.

[0043] In the specific scenario of financial technology, if a financial institution has limited computing resources, a prompt fine-tuning algorithm can be selected to fine-tune credit scoring or fraud detection models, ensuring that tasks are completed under limited computing resources; when the financial institution has strong computing power and the task requirement is to fine-tune the existing credit scoring model, a low-rank adaptive algorithm (LoRA) can be selected to efficiently optimize the financial behavior of specific customer groups and improve risk prediction accuracy. If the task focuses on text coherence optimization, such as generating accurate financial reports or investment analysis reports, a federated fine-tuning algorithm based on prompts can be selected to enhance the fluency and context consistency of text generation. For financial platforms that cannot directly access customer data, a federated optimization algorithm based on optimal transport can be used to intelligently adjust heterogeneous data distribution, enabling efficient cross-institution collaboration and data privacy protection.

[0044] In an embodiment of the present application, the filtering of the plurality of obtained federated parameter efficient fine-tuning algorithms according to the client resource status and the fine-tuning task requirement to obtain a target fine-tuning algorithm comprises: obtaining the model access state of the target client and determining whether the model access state is accessible or inaccessible; If the model access state is accessible, when the client resource status is resource-constrained, a prompt fine-tuning algorithm in the obtained federated parameter efficient fine-tuning algorithm is selected as the target fine-tuning algorithm; When the client resource status is resource-sufficient, determine whether the fine-tuning task requirement is a fine-tuning task or a text coherence optimization task; If the fine-tuning task requirement is a fine-tuning task, a low-rank adaptive algorithm in the federated parameter efficient fine-tuning algorithm is selected as the target fine-tuning algorithm; If the fine-tuning task requirement is a text coherence optimization task, the prompt word-based federated parameter efficient fine-tuning algorithm is selected as the target fine-tuning algorithm. If the model access state is inaccessible, the federated optimization algorithm based on optimal transport theory in the federated parameter efficient fine-tuning algorithm is selected as the target fine-tuning algorithm.

[0045] In detail, the access permission of the target client to the large language model is evaluated to determine whether the model access state is "accessible" or "inaccessible". If the client can access the model ontology and the computing resources are tight, the prompt tuning algorithm (Prompt Tuning) is selected as the target fine-tuning algorithm. The prompt tuning algorithm significantly reduces the computing and storage overhead by optimizing only a small number of prompt vectors without modifying the model main body parameters, and is suitable for lightweight terminal devices.

[0046] If the client can access the model ontology and the resources are sufficient, the specific requirement type of the fine-tuning task will be further determined. If the task belongs to a fine adjustment task (such as medical question answering, financial proofreading, etc. which require high accuracy of output), the low-rank adaptive algorithm (LoRA) is selected as the target fine-tuning algorithm. The LoRA algorithm introduces a low-rank matrix into the weight matrix of the model for parameter fine-tuning, which not only maintains the original model structure but also efficiently improves the performance, and is suitable for scenarios with sufficient resources and high requirements for model performance. If the task belongs to a text coherence optimization task (such as dialogue generation, report polishing, etc.), the prompt word-based federated parameter efficient fine-tuning algorithm (such as Prompt Tuning or P-Tuning) is selected as the target algorithm. The prompt word-based federated parameter efficient fine-tuning algorithm learns a set of continuous or discrete prompt vectors to optimize the context understanding ability of the language model, and improves the text coherence and generation quality without modifying the main model parameters. The selection strategy reasonably matches the algorithm type according to the different task objectives, fully utilizes the resource advantages, and improves the pertinence and practicality of the model effect.

[0047] When the client cannot access the model parameters (such as due to privacy, license restrictions, etc.), the FedOT algorithm is a federated learning optimization algorithm based on optimal transport theory, which aims to effectively align the distribution differences between clients without sharing the original data and model parameters. FedOT maps the local model output (such as feature distribution or prediction distribution) of the client to the global reference distribution through optimal transport, constructs a consistency constraint, and improves the generalization ability and convergence stability of the global model in a non-independent and identically distributed (Non-IID) data environment.

[0048] The screening process flexibly matches the most suitable federated parameter efficient fine-tuning algorithm according to the model access permission of the client, resource conditions and task requirements, and has high adaptability and actual availability. When the model is accessible, the system can select schemes such as prompt tuning (efficient and light) or low-rank adaptive algorithm (high-performance fine-tuning) according to the resource status and task type, so as to realize the optimization of resources and the consideration of task effect; when the model is not accessible, the federated optimization algorithm based on optimal transmission is used to effectively improve the performance of the model in the non-independent and identically distributed (Non-IID) environment while protecting privacy, and the strategy optimizes the path through "on-demand matching", thereby improving the practicability, efficiency and generalization ability of federated learning.

[0049] S4, adjusting a preset large language model by using the target sub-data set and the target fine-tuning algorithm to obtain a preliminary adjustment model.

[0050] In the embodiment of the application, according to whether the selected fine-tuning algorithm is a federated optimization algorithm based on optimal transmission, the original large language model is parameterized, and an initial language model is constructed. The target sub-data set is segmented and encoded to obtain a corresponding input vector, and a real data label is generated. The prediction label is obtained by model inference, and the loss value between the prediction and the real label is calculated by combining the loss function. Then, the model is iteratively optimized by combining the acceleration operator and the resource efficient operator to generate an adjusted language model of each client. The global model parameters are obtained by integrating the model parameters of each client through weighted averaging, and the model is further fine-tuned based on the global model parameters to finally form a preliminary fine-tuned model, providing a basis for subsequent optimization.

[0051] In the medical health specific scenario, patient data in different hospitals often have heterogeneity, and by adjusting the large language model based on different fine-tuning algorithms, the model can be efficiently customized for each hospital. When the federated optimization algorithm based on optimal transmission is used, the data cannot be directly shared between different hospitals, but by configuring the compression simulator and the initial adapter to generate an initialization scheme, the model can be effectively optimized while maintaining data privacy in combination with the distribution of patient data. For disease diagnosis tasks, the target sub-data set is segmented and encoded, and the prediction label generated by the model is compared with the real label to optimize the accuracy and robustness of disease prediction. Finally, the knowledge of each hospital is integrated through weighted averaging of global model parameters to realize an accurate and adaptive preliminary adjustment model that can cope with complex situations in different medical scenarios.

[0052] In the specific scenario of financial technology, when the sharing of data between financial institutions is strictly limited, the federated optimization algorithm based on optimal transmission can enable different banks to jointly train a model through an encrypted optimization protocol, generate an initial configuration through a compression simulator and other tools, and ensure that each bank optimizes the model while maintaining data privacy. For credit evaluation tasks, the financial data in the target sub-data set is used for tokenization and encoding, and the corresponding financial labels are generated. By comparing the predicted labels, the performance of the model in different customer groups is optimized. Finally, the model is fine-tuned through global parameter weighted averaging and resource-efficient operators to obtain a preliminary adjustment model that can integrate information from various banks and respond to market changes, thereby improving the risk prediction capabilities across institutions.

[0053] In the embodiments of the present application, the adjustment of the preset large language model using the target sub-data set and the target fine-tuning algorithm to obtain a preliminary adjustment model comprises: determining whether the target fine-tuning algorithm is a federated optimization algorithm based on optimal transmission; if the target fine-tuning algorithm is a federated optimization algorithm based on optimal transmission, obtaining a compression simulator and an initial adapter, and generating an initialization configuration according to the compression simulator and the initial adapter; if the target fine-tuning algorithm is not a federated optimization algorithm based on optimal transmission, obtaining initialization learnable parameters, and generating an initialization configuration according to the initialization learnable parameters; performing parameter initialization on the preset large language model using the initialization configuration to obtain an initial language model; tokenizing and encoding the target sub-data set to obtain a plurality of target tokenized encodings; generating real data labels according to the target tokenized encodings; obtaining predicted labels for the target tokenized encodings, and determining a loss value between the predicted labels and the real data labels using a preset loss function; obtaining an acceleration operator and a resource-efficient operator, and adjusting the initial language model using the loss value, the acceleration operator and the resource-efficient operator to obtain an adjusted language model; performing weighted averaging on the model parameters of the adjusted language model of all the target clients to obtain global model parameters; fine-tuning the adjusted language model using the global model parameters to obtain a preliminary fine-tuned model.

[0054] In detail, before fine-tuning begins, it is determined whether the selected target fine-tuning algorithm is a federated optimization algorithm based on optimal transmission. If so, the client cannot directly access the model parameters, and the system obtains a compression simulator and an initial adapter to build an initial configuration for cross-client transmission and adaptation, thereby achieving efficient coordination and alignment of the model update process. If the target fine-tuning algorithm is not based on the optimal transmission method (such as prompt tuning or low-rank adaptive algorithm), the system obtains the corresponding initial learnable parameters (such as prompt vectors and LoRA weights), and generates the initial configuration required for fine-tuning to support subsequent efficient parameter training, ensuring that the initialization method matches the algorithm characteristics and guarantees the effectiveness of the fine-tuning process and resource utilization efficiency.

[0055] If prompt tuning or low-rank adaptive algorithm is used, the initialized learnable parameters (such as prompt vectors or low-rank matrices) are injected into specific locations of the model. If a federated optimization algorithm based on optimal transmission is used, the model is subjected to lightweight structural adaptation and alignment by loading the parameters of the compression simulator and the initial adapter. The entire process does not change the original model structure, but loads adjustable components based on it to build an initial language model with fine-tuning capabilities, laying the foundation for subsequent training and optimization.

[0056] The text data in each target sub-dataset is segmented and encoded to convert natural language into a vector form recognizable by the model, obtaining multiple target segmented encodings. According to the task requirements (such as text classification, question answering, and summary) of each segment of encoding, its corresponding real data label is generated as the target of supervised learning. The initial language model performs forward propagation on these encoded inputs and outputs the corresponding predicted labels. Using a loss function (such as cross-entropy and mean square error), the loss value between the predicted label and the real label is calculated to measure the output deviation of the current model and provide a basis for subsequent optimization of model parameters.

[0057] After obtaining the loss value, an acceleration operator and a resource-efficient operator are introduced to participate in the optimization and adjustment of the initial language model. The acceleration operator aims to speed up the model training and convergence process, such as using gradient clipping, mixed precision training, etc. The resource-efficient operator is used to reduce computational resource consumption, such as parameter sharing, low-rank update, and weight compression. Combined with these operators and the loss value, the learnable parameters are updated through the backpropagation mechanism, thereby improving training efficiency and resource utilization while ensuring performance, and ultimately obtaining an adjusted language model to lay the foundation for subsequent federated aggregation or further fine-tuning.

[0058] After completing the model adjustment of each target client locally, the adjusted language model parameters of all clients are weighted and averaged, and the weight can be set according to factors such as sample quantity, data quality or resource contribution, so as to fuse the unified global model parameters. The global parameters are used to update the adjusted language model of each client, that is, to fine-tune the global consistency, so as to obtain a preliminary fine-tuned model with better generalization ability and task adaptability, providing an optimized basis for the next round of training or final deployment.

[0059] According to the flexible configuration of the initialization strategy according to the target fine-tuning algorithm type, the model initialization quality is effectively improved, the segmentation and coding steps ensure that the data structure and model input depth are matched, and the training efficiency is improved; the loss function is used to quantify the error, and the acceleration operator and resource efficient operator are optimized in cooperation, so that the model still obtains better performance under limited computing resources; by weighting and aggregating the parameters of each client model and globally fine-tuning, the rapid convergence and generalization ability of the model are realized, and the efficient fine-tuning demand in the multi-task and multi-terminal environment is met.

[0060] S5, resource consumption analysis is performed on the adjustment process of the large language model, resource consumption conditions are obtained, and performance optimization is performed on the preliminary adjustment model according to the resource consumption conditions, to obtain a target large language model.

[0061] In the embodiment of the application, the fine-tuning task type is determined and the corresponding analysis parameters are generated, the model fine-tuning process is valued and the fine-tuning effect is analyzed, and the computing resource consumption is recorded in real time and a resource consumption graph is generated; the fine-tuning effect value is mapped into the resource consumption graph to form a process parameter graph, and the resource consumption condition is extracted accordingly. Based on the resource consumption condition, the parameters and modules to be optimized are identified, and the model structure is optimized through parameter adjustment, compression and module pruning, and finally a target large language model with better performance and resource proportion is obtained.

[0062] In the specific scenario of medical health, the resource consumption perception and optimization process can be applied to the model fine-tuning process of an intelligent inquiry system. By analyzing the resource consumption and performance of the model in identifying patient intentions (such as registration, symptom consultation, and medication advice) in real time, visual management of computing resource consumption and fine-tuning efficiency evaluation are realized. The system can automatically optimize the model parameters and structure according to the task complexity, such as pruning modules that have little effect on diagnosis and judgment, so as to reduce resource consumption while ensuring the accuracy of diagnosis and treatment, which is suitable for edge deployment or resource-constrained environments in hospital information systems.

[0063] In the specific scenario of financial technology, the method can be applied to customer service robots or risk control systems to analyze and visually evaluate the resource occupation of the model when processing intent recognition tasks (such as loan consultation, account anomaly reporting, and transaction behavior analysis). According to the relationship between fine-tuning effect and resource consumption, the system can intelligently compress or optimize the modules in the model that contribute less to business response, minimize the deployment cost of the model, and ensure the real-time performance and accuracy of the service, which is suitable for financial business systems in high-concurrency scenarios.

[0064] Figure 3 A flowchart of a resource consumption analysis process in a text intent analysis method according to an embodiment of the present application is provided.

[0065] In an embodiment of the present application, the resource consumption analysis of the adjustment process of the large language model obtains the resource consumption situation, which includes: Obtaining the task type of the fine-tuning task requirement, and generating task analysis parameters according to the task type; Determining the parameter value of the task analysis parameter according to the adjustment process of the large language model; Analyzing the fine-tuning effect of the large language model according to the parameter value, and obtaining a fine-tuning effect value; Real-time recording the resource consumption of the adjustment process of the large language model, and visualizing the real-time recording data to obtain a resource consumption graph; Mapping the fine-tuning effect value to the resource consumption graph to obtain a process parameter graph; Generating the resource consumption situation according to the process parameter graph.

[0066] In detail, the specific type of the fine-tuning task (such as classification, generation or optimization task) is identified, and corresponding task analysis parameters are generated accordingly. These parameters cover model performance indicators, computing resource requirements and training strategies, and the specific values of these parameters are dynamically determined according to the actual adjustment process of the large language model, so as to accurately reflect the characteristics and resource consumption of the current fine-tuning task and provide data support for subsequent performance evaluation and optimization.

[0067] According to the determined task analysis parameter value, the performance indicators of the large language model at each stage in the fine-tuning process are collected, including but not limited to accuracy, precision, recall, F1 score, loss function value and convergence speed, etc. These indicators are used to evaluate the performance of the model in the specified fine-tuning task in multiple dimensions, analyze the performance difference of the model on the training set and the validation set, identify whether there is overfitting or underfitting problem, and combine the specific requirements of the task (such as text generation quality, classification accuracy, etc.) to calculate a value representing the overall fine-tuning effect, i.e. the fine-tuning effect value, which can be used to guide the adjustment of the subsequent optimization strategy.

[0068] In the adjustment process of the large language model, the system monitors and records key resource consumption indicators in real time, such as calculation time, memory usage, GPU / CPU load, and energy consumption, and visualizes these data in the form of a chart to form a resource consumption chart.

[0069] In the resource consumption chart, the resource usage indicators (such as calculation time, memory occupation, and computing power consumption) corresponding to different time points or stages are quantified as specific values, the fine-tuning effect value is taken as a performance indicator, and the same measurement scale is converted through normalization or standardization processing. In the mapping process, the fine-tuning effect value of each time point is associated with the resource consumption value corresponding to the time point to form a two-dimensional coordinate point, with the horizontal axis representing resource consumption and the vertical axis representing fine-tuning effect, or to intuitively display the trend of fine-tuning effect changing with resource consumption. Based on the process parameter chart, the relationship between resource utilization efficiency and fine-tuning effect is analyzed, and the overall resource consumption is refined and output, providing a basis for subsequent optimization and decision-making.

[0070] Through comprehensive analysis and real-time visualization of resource consumption in the adjustment process of the large language model, combined with fine-tuning effect value mapping, the relationship between model performance and resource usage can be intuitively reflected, helping to accurately evaluate the efficiency and effect of fine-tuning tasks. Not only is resource allocation optimized and computing utilization improved, but subsequent adjustment strategies are also guided to ensure the best fine-tuning effect under limited resource conditions, improving the overall performance and practical value of the system.

[0071] In the embodiments of the present application, the performance optimization of the preliminary adjustment model according to the resource consumption situation to obtain the target large language model comprises: generating to-be-optimized parameters and to-be-optimized modules according to the resource consumption situation; adjusting the to-be-optimized parameters according to the process parameter chart to obtain adjustment parameters; compressing the adjustment parameters to obtain compressed parameters; determining the contribution parameter value of the to-be-optimized module and judging whether the contribution parameter value is greater than a preset contribution threshold value; if the contribution parameter value is less than or equal to the contribution threshold value, the to-be-optimized module corresponding to the contribution parameter value less than or equal to the contribution threshold value is pruned; reducing the number of adapter parameters in the to-be-optimized module remaining after pruning to obtain an adjustment module; if the contribution parameter value is greater than the contribution threshold value, the number of adapter parameters in the to-be-optimized module is reduced to obtain an adjustment module; optimizing the preliminary adjustment model using the compressed parameters and the adjustment module to obtain the target large language model.

[0072] In detail, according to the resource consumption, the parameters and modules that need to be optimized in the model are identified, the process parameter diagram is combined to reasonably adjust these to-be-optimized parameters, so as to improve the performance and reduce the resource occupation, the adjusted parameters are compressed to reduce the model size and the calculation complexity, thereby laying a foundation for subsequent model optimization and acceleration.

[0073] The importance scores of the parameters in the to-be-optimized module are obtained based on indicators such as the product of the gradient and the weight, and the scores are weighted and summed or averaged to obtain the overall contribution parameter value. The contribution parameter value is compared with the contribution threshold value. If the contribution parameter value is less than or equal to the contribution threshold value, it is determined that the to-be-optimized module has a lower contribution to the model performance, and a pruning operation is performed, specifically, part or all of the parameters in the module are deleted or compressed to reduce the model size and the calculation resource consumption. For the remaining to-be-optimized module after pruning, the number of adapter parameters of the remaining to-be-optimized module is further reduced in detail, usually through pruning or parameter sharing and other technical means, the adapter parameters that have a smaller contribution to the model performance are selected and deleted or merged, thereby reducing the parameter size of the module. If the contribution parameter value is greater than the contribution threshold value, the same operation is performed on the to-be-optimized module corresponding to the contribution parameter value greater than the contribution threshold value, and finally an adjusted module with more simplified parameters and more efficient calculation is obtained to improve the performance and resource utilization efficiency of the overall model.

[0074] The obtained compressed parameters and the adjusted module are integrated to optimize the preliminary adjustment model, the corresponding parameters and module structures are replaced or fused to reduce the model redundancy and the calculation burden, while maintaining or improving the model performance, thereby generating a more efficient and simplified target large language model to meet the balance demand of resources and performance.

[0075] By combining the resource consumption analysis result and the process parameter diagram, the to-be-optimized parameters and modules in the large language model are adjusted, and technical means such as compression, pruning and adapter reduction are used to dynamically reduce the resource occupation of inefficient or redundant parts, while retaining the core modules that have a larger contribution to the performance, thereby significantly improving the running efficiency and deployment flexibility of the model on the basis of ensuring the model accuracy, and meeting the practical demand of the large model in the algorithm power limited environment.

[0076] S6, obtaining target text data of the target field, performing text language analysis on the target text data by using the target large language model, and determining a target text intention.

[0077] In an embodiment of the present application, representative target text data is obtained from a target field (such as medical health or financial technology), for example, patient consultation dialogue, financial user consultation record, etc., and the target text data is preprocessed, including word segmentation, denoising, entity recognition and format standardization, etc. The cleaned text is input into the optimized target large language model, and the powerful semantic understanding and context modeling capability is used to perform deep language analysis on the input text, extract semantic features and context relationships. The model combines the domain intent label system learned in the previous fine-tuning process to identify and classify the intent of each piece of text data, and finally outputs accurate target text intent, which is used to support downstream applications such as automatic question answering, business distribution or intelligent recommendation.

[0078] In the medical health specific scenario, it can be applied to intelligent guide diagnosis and online consultation system. By obtaining the natural language input of the patient (such as "recently coughing has been getting worse, want to ask if it is pneumonia"), the optimized medical field large language model is used to perform semantic analysis, accurately identify the intent as "symptom consultation" or "disease self-diagnosis", and further extract relevant symptom keywords to assist in triage, recommend departments or generate preliminary consultation suggestions, thereby improving the efficiency of the diagnosis and treatment process and the patient service experience.

[0079] In the financial technology specific scenario, it is suitable for intelligent customer service and risk control warning system. By collecting text input of users in mobile banking, loan platform and other scenarios (such as "unauthorized transactions have occurred in my account recently"), the optimized financial field large language model is used to perform in-depth semantic analysis, identify the user intent as "account abnormality complaint" or "risk alarm", and quickly classify and process in combination with historical behavior, thereby realizing automatic shunting, risk warning or manual response, improving the intelligence and security of financial services.

[0080] By obtaining text data from a target field and using a target large language model to perform language analysis to extract text intent, the understanding ability of the system for user demand can be significantly improved, and high-precision and high-robustness intent recognition can be realized. Compared with traditional methods, the use of fine-tuned large language models has stronger semantic understanding and context modeling capabilities, and can adapt to complex and diverse language expressions, thereby providing more accurate and intelligent service responses, improving user experience, and improving model stability and text intent interpretability.

[0081] It should be understood that the size of the serial number of each step in the above embodiment does not mean the order of execution, and the execution order of each process should be determined according to its function and internal logic, and should not constitute any limitation on the implementation process of the embodiment of the present application.

[0082] As shown in Figure 4 , it is a functional module diagram of a text intent analysis device provided by an embodiment of the present application.

[0083] In the embodiments of the present disclosure, a text intention analysis device is provided, which corresponds to the text intention analysis method in the above embodiments. As shown in the following Figure 4 The text intention analysis device 100 can be installed in an electronic device, and according to the implemented functions, the text intention analysis device 100 includes a division strategy selection module 101, an original data division module 102, a fine-tuning algorithm screening module 103, a large language model adjustment module 104, a large language model optimization module 105, and a text intention analysis module 106. The detailed description of each functional module is as follows: The division strategy selection module 101 is configured to obtain original data of a target field and fine-tuning task requirements, select a division strategy for the original data according to the fine-tuning task requirements, and obtain a target division strategy. The original data division module 102 is configured to divide the original data according to the target division strategy to obtain a plurality of target sub-data sets. The fine-tuning algorithm screening module 103 is configured to obtain client resource conditions, screen a plurality of obtained federated parameter efficient fine-tuning algorithms according to the client resource conditions and the fine-tuning task requirements, and obtain a target fine-tuning algorithm. The large language model adjustment module 104 is configured to adjust a preset large language model using the target sub-data set and the target fine-tuning algorithm to obtain a preliminary adjustment model. The large language model optimization module 105 is configured to analyze resource consumption of the adjustment process of the large language model to obtain resource consumption conditions, and perform performance optimization on the preliminary adjustment model according to the resource consumption conditions to obtain a target large language model. The text intention analysis module 106 is configured to obtain target text data of the target field, perform text language analysis on the target text data using the target large language model, and determine a target text intention.

[0084] In an embodiment, when the division strategy selection module 101 performs division strategy selection on the original data according to the fine-tuning task requirements to obtain a target division strategy, it is configured to: de-duplicate the original data to obtain de-duplicated data; identify irrelevant data in the de-duplicated data that is irrelevant to the target field, filter the irrelevant data in the de-duplicated data, and obtain filtered data; label the filtered data according to the fine-tuning task requirements to obtain labeled data; determine a distribution difference value of the labeled data, and determine whether the distribution difference value is greater than a preset difference threshold; if the distribution difference value is greater than the difference threshold value, a Dirichlet distribution division strategy is taken as the target division strategy; if the distribution difference value is less than or equal to the difference threshold value, a uniform division strategy is taken as the target division strategy.

[0085] In an embodiment, the original data division module 102 is configured to, when performing division of the original data according to the target division strategy to obtain a plurality of target sub-data sets: when the target division strategy is a uniform division strategy, a division ratio is obtained according to the uniform division strategy; the labeled data is divided according to the division ratio to obtain a plurality of target sub-data sets; when the target division strategy is a Dirichlet distribution division strategy, a class label and a division heterogeneity parameter of the labeled data are obtained; an allocation ratio of each of the class labels is randomly generated according to the Dirichlet distribution division strategy; the labeled data is divided according to the allocation ratio and the division heterogeneity parameter to obtain a plurality of target sub-data sets.

[0086] In an embodiment, the fine-tuning algorithm screening module 103 is configured to, when performing screening of a plurality of obtained federated parameter efficient fine-tuning algorithms according to the client resource status and the fine-tuning task demand to obtain a target fine-tuning algorithm: a model access state of the target client is obtained to determine whether the model access state is accessible or inaccessible; if the model access state is accessible, when the client resource status is resource-constrained, a prompt tuning algorithm in the obtained federated parameter efficient fine-tuning algorithm is taken as the target fine-tuning algorithm; when the client resource status is resource-sufficient, it is determined whether the fine-tuning task demand is a fine adjustment task or a text coherence optimization task; if the fine-tuning task demand is a fine adjustment task, a low-rank adaptive algorithm in the federated parameter efficient fine-tuning algorithm is taken as the target fine-tuning algorithm; if the fine-tuning task demand is a text coherence optimization task, a federated parameter efficient fine-tuning algorithm based on a prompt word is taken as the target fine-tuning algorithm; if the model access state is inaccessible, a federated optimization algorithm based on optimal transport theory in the federated parameter efficient fine-tuning algorithm is taken as the target fine-tuning algorithm.

[0087] In an embodiment, the large language model adjustment module 104 is configured to, when performing adjustment of a preset large language model using the target sub-data set and the target fine-tuning algorithm to obtain a preliminary adjustment model: determining whether the target fine-tuning algorithm is a federated optimization algorithm based on optimal transport; if the target fine-tuning algorithm is a federated optimization algorithm based on optimal transport, obtaining a compression simulator and an initial adapter, and generating an initialization configuration according to the compression simulator and the initial adapter; if the target fine-tuning algorithm is not a federated optimization algorithm based on optimal transport, obtaining an initial learnable parameter, and generating an initialization configuration according to the initial learnable parameter; performing parameter initialization on a preset large language model using the initialization configuration to obtain an initial language model; performing word segmentation and encoding on the target sub-data set to obtain a plurality of target segmented encodings; generating a real data label according to the target segmented encoding; obtaining a predicted label of the target segmented encoding, and determining a loss value between the predicted label and the real data label using a preset loss function; obtaining an acceleration operator and a resource-efficient operator, and adjusting the initial language model using the loss value, the acceleration operator, and the resource-efficient operator to obtain an adjusted language model; performing weighted averaging on model parameters of the adjusted language model of all target clients to obtain global model parameters; fine-tuning the adjusted language model using the global model parameters to obtain a preliminary fine-tuned model.

[0088] In an embodiment, the large language model optimization module 105, when performing resource consumption analysis on the adjustment process of the large language model to obtain the resource consumption situation, is configured to: obtain a task type of the fine-tuning task requirement, and generate a task analysis parameter according to the task type; determine a parameter value of the task analysis parameter according to the adjustment process of the large language model; analyze the fine-tuning effect of the large language model according to the parameter value to obtain a fine-tuning effect value; real-time record the resource consumption of the adjustment process of the large language model, and visualize the real-time record data to obtain a resource consumption graph; map the fine-tuning effect value to the resource consumption graph to obtain a process parameter graph; generate a resource consumption situation according to the process parameter graph.

[0089] In an embodiment, the large language model optimization module 105, when performing performance optimization on the preliminary adjustment model according to the resource consumption situation to obtain a target large language model, is configured to: generate a to-be-optimized parameter and a to-be-optimized module according to the resource consumption; adjust the to-be-optimized parameter according to the process parameter map to obtain an adjusted parameter; compress the adjusted parameter to obtain a compressed parameter; determine a contribution parameter value of the to-be-optimized module, and judge whether the contribution parameter value is greater than a preset contribution threshold value; if the contribution parameter value is less than or equal to the contribution threshold value, then the to-be-optimized module corresponding to the contribution parameter value less than or equal to the contribution threshold value is pruned; reduce the number of adapter parameters in the to-be-optimized module remaining after pruning to obtain an adjusted module; if the contribution parameter value is greater than the contribution threshold value, then the number of adapter parameters in the to-be-optimized module is reduced to obtain an adjusted module; optimize the preliminary adjustment model using the compressed parameter and the adjusted module to obtain a target large language model.

[0090] In the present application, for a text intention analysis device, first, the present application obtains the original data of the target field and the fine-tuning task demand, selects a division strategy for the original data according to the fine-tuning task demand, and obtains a target division strategy. This adaptive strategy selection not only improves the representativeness and diversity of the fine-tuning data, but also optimizes the effect and generalization ability of model training, which helps to build more robust and efficient large language models. According to the target division strategy, the original data is divided to obtain a plurality of target sub-data sets, the client resource status is obtained, and the plurality of obtained federal parameter efficient fine-tuning algorithms are screened according to the client resource status and the fine-tuning task demand to obtain a target fine-tuning algorithm. The screening process flexibly matches the most suitable federal parameter efficient fine-tuning algorithm according to the model access permission, resource condition and task demand of the client, and has high adaptability and actual usability. Then, the target sub-data set and the target fine-tuning algorithm are used to adjust the preset large language model to obtain a preliminary adjustment model. The initialization strategy is flexibly configured according to the target fine-tuning algorithm type, the model initialization quality is effectively improved, the tokenization and encoding steps ensure that the data structure and model input depth are matched, the training efficiency is improved, the model still obtains better performance under limited computing resources through the cooperative optimization of the acceleration operator and the resource efficient operator, the resource consumption of the adjustment process of the large language model is analyzed to obtain the resource consumption situation, the performance of the preliminary adjustment model is optimized according to the resource consumption situation, and a target large language model is obtained. By combining the resource consumption analysis result and the process parameter diagram, the to-be-optimized parameters and modules in the large language model are adjusted, and technical means such as compression, pruning and adapter simplification are adopted to dynamically reduce the resource occupation of inefficient or redundant parts, while retaining the core modules that contribute more to performance. Thus, on the basis of ensuring the model accuracy, the running efficiency and deployment flexibility of the model are significantly improved, the target text data of the target field is obtained, the target text data is analyzed by using the target large language model, the target text intention is determined, the understanding ability of the system to user demand can be significantly improved, and high-precision and high-robustness intention recognition is realized. Compared with the traditional method, the use of fine-tuning large language model has stronger semantic understanding and context modeling capability, can adapt to complex and diverse language expressions, and can effectively improve the stability of large language model analysis performance and the explainability of text intention. For specific limitations of a text intention analysis device, refer to the limitations of a text intention analysis method in the foregoing, which will not be repeated here. Each module in the above text intention analysis device can be realized by software, hardware and their combination. The above modules can be embedded in or independent of the processor in the computer device in hardware form, or stored in the memory in the computer device in software form, so that the processor calls and executes the operations of the above modules.

[0091] In an embodiment, a computer device is provided, which can be a server, and an internal structure diagram thereof can be as shown in Figure 5 The computer device includes a processor, a memory, a network interface and a database connected through a system bus. The processor of the computer device is configured to provide computing and control capabilities. The memory of the computer device includes a non-volatile and / or volatile storage medium, an internal memory. The non-volatile storage medium stores an operating system, a computer program and a database. The internal memory provides an environment for the operating system and the computer program in the non-volatile storage medium to run. The network interface of the computer device is configured to communicate with an external client through a network connection. The computer program is executed by the processor to implement the functions or steps of the server side of the text intent analysis method.

[0092] In an embodiment, a computer device is provided, which can be a client, and an internal structure diagram thereof can be as shown in Figure 6 The computer device includes a processor, a memory, a network interface, a display screen and an input device connected through a system bus. The processor of the computer device is configured to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system and a computer program. The internal memory provides an environment for the operating system and the computer program in the non-volatile storage medium to run. The network interface of the computer device is configured to communicate with an external server through a network connection. The computer program is executed by the processor to implement the functions or steps of the client side of the text intent analysis method.

[0093] In an embodiment, a computer device is provided, which includes a memory, a processor and a computer program stored in the memory and executable on the processor, and the processor implements the following steps when executing the computer program: Obtain original data of a target field and fine-tuning task requirements, select a division strategy for the original data according to the fine-tuning task requirements, and obtain a target division strategy; Divide the original data according to the target division strategy to obtain a plurality of target sub-data sets; Obtain client resource conditions, and screen a plurality of obtained federated parameter fine-tuning algorithms according to the client resource conditions and the fine-tuning task requirements to obtain a target fine-tuning algorithm; Adjust a preset large language model using the target sub-data set and the target fine-tuning algorithm to obtain a preliminary adjustment model; Perform resource consumption analysis on the adjustment process of the large language model to obtain resource consumption conditions, and perform performance optimization on the preliminary adjustment model according to the resource consumption conditions to obtain a target large language model; Obtaining target text data of the target field, performing text language analysis on the target text data by using the target large language model, and determining a target text intention.

[0094] In several embodiments provided by the present application, it should be understood that the disclosed devices and apparatuses can be implemented in other manners. For example, the above-described system embodiments are merely illustrative. For example, the division of the modules is merely logical function division. In actual implementation, another division manner can be used.

[0095] In addition, each function module in each embodiment of the present application can be integrated in a processing unit, or each unit can exist physically, or two or more units can be integrated in one unit. The integrated unit can be implemented in the form of hardware or in the form of hardware plus software function modules.

[0096] Therefore, no matter from which point of view, the embodiments should be regarded as exemplary and non-limiting, and the scope of the present application is defined by the appended claims rather than the above description, and therefore all changes falling within the meaning and scope of the equivalent elements of the claims are intended to be included in the present application. Any reference signs in the claims should not be regarded as limiting the claims to which they relate.

[0097] It is obvious for those skilled in the art that the present application is not limited to the details of the above exemplary embodiments, and the present application can be implemented in other specific forms without departing from the spirit or essential characteristics of the present application.

[0098] In some embodiments of the present embodiment, a computer readable storage medium is provided, and the computer readable storage medium stores a computer program. The computer program is executed by a processor to implement the steps of the method described in the above embodiments.

[0099] The readable storage medium of the present application stores a computer program, and the computer program can implement the following when executed by a processor of an electronic device: Obtaining original data of a target field and fine-tuning task requirements, selecting a division strategy for the original data according to the fine-tuning task requirements, and obtaining a target division strategy; Dividing the original data according to the target division strategy, and obtaining a plurality of target sub-data sets; Obtaining a client resource status, and screening a plurality of obtained federated parameter fine-tuning algorithms according to the client resource status and the fine-tuning task requirements, and obtaining a target fine-tuning algorithm; Adjusting a preset large language model by using the target sub-data set and the target fine-tuning algorithm, and obtaining a preliminary adjustment model; The resource consumption analysis is performed on the adjustment process of the large language model to obtain resource consumption, and the performance of the preliminary adjustment model is optimized according to the resource consumption to obtain a target large language model. Target text data of the target field is obtained, and the target text data is analyzed by using the target large language model to determine a target text intent.

[0100] It should be noted that the functions or steps described above with respect to the computer-readable storage medium or the computer device can correspond to the related descriptions of the server side and the client side in the foregoing method embodiments. To avoid repetition, they will not be described one by one here.

[0101] The computer-readable storage medium can also store at least one computer executable program / instruction, such as computer readable instructions. The computer-readable storage medium includes, but is not limited to, for example, volatile memory and / or non-volatile memory. The volatile memory may, for example, include random access memory (RAM) and / or cache memory, etc. The computer-readable storage medium may, for example, include read-only memory (ROM), hard disk, flash memory, etc. For example, the non-transitory computer-readable storage medium can be connected to a computing device such as a computer, and then when the computing device runs the computer readable instructions stored on the computer readable storage medium, the various methods described above can be performed.

[0102] In addition, the computer device can also include (but not limited to) a data bus, an input / output (I / O) bus, a display, and an input / output device (such as a keyboard, a mouse, a speaker, etc.), etc.

[0103] The processor can communicate with external devices through the I / O bus via wired or wireless networks.

[0104] In one embodiment, the at least one computer executable instruction can also be compiled into or constitute a software product / computer program product, wherein one or more computer executable instructions are executed by the processor to perform the steps of the functions and / or methods described in the embodiments of the present technology.

[0105] Those skilled in the art can understand that all or part of the processes in the above-mentioned embodiment methods can be completed by instructing the relevant hardware through a computer program. The computer program can be stored in a non-volatile computer readable storage medium, and when the computer program is executed, the processes of the above-mentioned embodiments of the methods can be included. Any reference to memory, storage, database or other medium used in each embodiment provided by the present application can include non-volatile and / or volatile memory. Non-volatile memory can include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM) or flash memory. Volatile memory can include random access memory (RAM) or external cache memory. As an illustration but not limitation, RAM is available in various forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate SDRAM (DDR SDRAM), enhanced SDRAM (ESDRAM), synchronous link (Synchlink) DRAM (SLDRAM), Rambus direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM), etc.

[0106] Those skilled in the art can clearly understand that, for the convenience and brevity of description, only the above-mentioned division of functional units and modules is exemplified, and in actual application, the above-mentioned functions can be completed by different functional units and modules according to needs, that is, the internal structure of the device is divided into different functional units or modules to complete all or part of the functions described above.

[0107] In the embodiments provided by the present disclosure, it should be understood that the disclosed apparatus and method can also be implemented in other manners. The embodiments described above are merely exemplary for describing the present disclosure. For example, the flowcharts and block diagrams in the accompanying drawings show the possible implementation architectures, functions and operation of the apparatus, method and computer program product according to the embodiments of the present disclosure. In this regard, each block in the flowcharts and block diagrams can represent a module, a program segment or a part of code, which contains one or more executable instructions for implementing the specified logic function. It should also be noted that, in some alternative implementations, the functions noted in the blocks can occur in different orders from those noted in the accompanying drawings. For example, two consecutive blocks can actually be executed substantially in parallel, and sometimes they can be executed in reverse order, depending on the functions involved. It should also be noted that each block in the block diagrams and / or flowcharts, and the combination of blocks in the block diagrams and / or flowcharts, can be implemented by a special-purpose hardware-based system for implementing the specified functions or actions, or can be implemented by a combination of special-purpose hardware and computer instructions.

[0108] It should be noted that, in the present disclosure, the term "comprising" or "including" or any other variant thereof is intended to cover non-exclusive inclusion, so that the process, method, article or device including a series of elements not only includes those elements, but also includes other elements not explicitly listed, or further includes elements inherent to such process, method, article or device. Without more limitations, the element limited by the statement "including a" does not exclude the presence of additional same elements in the process, method, article or device including the element.

[0109] The above-described embodiments are merely used to illustrate the technical solutions of the present disclosure, rather than limit them; although the present disclosure has been described in detail with reference to the foregoing embodiments, those of ordinary skill in the art should understand that: the technical solutions recorded in the foregoing embodiments can be modified, or some technical features can be replaced by equivalent replacements; and these modifications or replacements do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present disclosure, and should be included in the protection scope of the present disclosure.

[0110] It should be noted that, in the embodiments of the present disclosure, if non-company software tools or components appear, they are only used for example introduction, and do not represent actual use.

Claims

1. A text intention analysis method, characterized in that: The method comprises: Obtaining original data and fine-tuning task requirements of the target domain, selecting a partitioning strategy for the original data according to the fine-tuning task requirements, and obtaining a target partitioning strategy; Dividing the original data according to the target partitioning strategy to obtain multiple target sub-datasets; Acquiring client resource status, and screening multiple acquired federated parameter efficient fine-tuning algorithms according to the client resource status and the fine-tuning task requirements to obtain a target fine-tuning algorithm; Adjusting a preset large language model using the target sub-dataset and the target fine-tuning algorithm to obtain a preliminary adjusted model; Performing a resource consumption analysis on the adjustment process of the large language model to obtain a resource consumption situation, and optimizing the performance of the preliminary adjustment model according to the resource consumption situation to obtain a target large language model; Target text data of the target domain is acquired, and text language analysis is performed on the target text data using the target large language model to determine the target text intent.

2. The text intention analysis method according to claim 1, wherein: The selecting a partitioning strategy for the original data according to the fine-tuning task requirements to obtain a target partitioning strategy includes: Deduplication is performed on the original data to obtain deduplication data; Identifying irrelevant data in the deduplicated data that is irrelevant to the target domain, and filtering the irrelevant data in the deduplicated data to obtain filtered data; Annotating the filtered data according to the fine-tuning task requirements to obtain annotated data; Determining a distribution difference value of the labeled data, and judging whether the distribution difference value is greater than a preset difference threshold; If the distribution difference value is greater than the difference threshold, the Dirichlet distribution partitioning strategy is used as the target partitioning strategy; If the distribution difference value is less than or equal to the difference threshold, the uniform partitioning strategy is used as the target partitioning strategy.

3. The text intention analysis method according to claim 2, wherein: The original data is divided according to the target division strategy to obtain multiple target sub-datasets, including: When the target partitioning strategy is a uniform partitioning strategy, obtaining a partitioning ratio according to the uniform partitioning strategy; Dividing the labeled data according to the division ratio to obtain multiple target sub-datasets; When the target partitioning strategy is the Dirichlet distribution partitioning strategy, obtaining the category label and partition heterogeneity parameter of the labeled data; Randomly generating a distribution ratio for each of the category labels according to the Dirichlet distribution partitioning strategy; The labeled data is divided according to the allocation ratio and the division heterogeneity parameter to obtain multiple target sub-datasets.

4. The text intention analysis method according to claim 1, wherein: The method of screening multiple obtained federated parameter efficient fine-tuning algorithms according to the client resource status and the fine-tuning task requirements to obtain a target fine-tuning algorithm includes: Obtaining the model access status of the target client, and determining whether the model access status is accessible or inaccessible; If the model access status is accessible, when the client resource status is resource-constrained, the obtained prompt tuning algorithm in the federated parameter efficient fine-tuning algorithm is used as the target fine-tuning algorithm; When the client resource status indicates that resources are sufficient, determining whether the fine-tuning task requirement is a fine-tuning task or a text coherence optimization task; If the fine-tuning task requirement is a fine-tuning task, the low-rank adaptive algorithm in the federated parameter efficient fine-tuning algorithm is used as the target fine-tuning algorithm; If the fine-tuning task requires text coherence optimization, the efficient fine-tuning algorithm for federated parameters based on prompt words is used as the target fine-tuning algorithm; If the model access status is inaccessible, the federated optimization algorithm based on optimal transmission theory in the federated parameter efficient fine-tuning algorithm is used as the target fine-tuning algorithm.

5. The text intention analysis method according to claim 4, wherein: The step of adjusting the preset large language model using the target sub-dataset and the target fine-tuning algorithm to obtain a preliminary adjusted model includes: Determining whether the target fine-tuning algorithm is a federated optimization algorithm based on optimal transmission; If the target fine-tuning algorithm is a federated optimization algorithm based on optimal transmission, obtaining a compression simulator and an initial adapter, and generating an initialization configuration according to the compression simulator and the initial adapter; If the target fine-tuning algorithm is not a federated optimization algorithm based on optimal transmission, obtaining initialization learnable parameters, and generating an initialization configuration according to the initialization learnable parameters; Initializing parameters of a preset large language model using the initialization configuration to obtain an initial language model; Performing word segmentation and encoding on the target sub-dataset to obtain multiple target word segmentation codes; Generate real data labels according to the target word segmentation encoding; Obtaining a predicted label of the target word segmentation encoding, and determining a loss value between the predicted label and the real data label using a preset loss function; Obtaining an acceleration operator and a resource-efficient operator, and adjusting the initial language model using the loss value, the acceleration operator, and the resource-efficient operator to obtain an adjusted language model; Performing weighted averaging on the model parameters of the adjusted language models of all the target clients to obtain global model parameters; The adjusted language model is fine-tuned using the global model parameters to obtain a preliminary fine-tuned model.

6. The text intention analysis method according to claim 5, characterized in that: The resource consumption analysis of the adjustment process of the large language model to obtain resource consumption information includes: Obtaining a task type required for the fine-tuning task, and generating task analysis parameters according to the task type; Determining parameter values ​​of the task analysis parameters according to the adjustment process of the large language model; Analyzing the fine-tuning effect of the large language model according to the parameter value to obtain a fine-tuning effect value; Recording resource consumption during the adjustment process of the large language model in real time, and visualizing the real-time recorded data to obtain a resource consumption graph; Mapping the fine-tuning effect value to the resource consumption graph to obtain a process parameter graph; A resource consumption profile is generated based on the process parameter graph.

7. The text intention analysis method according to claim 6, characterized in that: The performing performance optimization on the preliminary adjustment model according to the resource consumption to obtain a target large language model includes: Generate parameters and modules to be optimized according to the resource consumption situation; Adjusting the parameters to be optimized according to the process parameter diagram to obtain adjusted parameters; compressing the adjustment parameter to obtain a compression parameter; Determining a contribution parameter value of the module to be optimized, and judging whether the contribution parameter value is greater than a preset contribution threshold; If the contribution parameter value is less than or equal to the contribution threshold, then pruning the module to be optimized corresponding to the contribution parameter value less than or equal to the contribution threshold; Reducing the number of adapter parameters in the remaining modules to be optimized to obtain an adjustment module; If the contribution parameter value is greater than the contribution threshold, reducing the number of adapter parameters in the module to be optimized to obtain an adjustment module; The preliminary adjustment model is optimized using the compression parameters and the adjustment module to obtain a target large language model.

8. A text intention analysis device, characterized in that: The device comprises: A partitioning strategy selection module is used to obtain the original data of the target domain and the fine-tuning task requirements, and select a partitioning strategy for the original data according to the fine-tuning task requirements to obtain a target partitioning strategy; An original data partitioning module is used to partition the original data according to the target partitioning strategy to obtain multiple target sub-data sets; A fine-tuning algorithm screening module is used to obtain client resource status, and screen multiple obtained federated parameter efficient fine-tuning algorithms according to the client resource status and the fine-tuning task requirements to obtain a target fine-tuning algorithm; A large language model adjustment module, configured to adjust a preset large language model using the target sub-dataset and the target fine-tuning algorithm to obtain a preliminary adjusted model; A large language model optimization module is used to analyze resource consumption during the adjustment process of the large language model to obtain resource consumption information, and to optimize the performance of the preliminary adjustment model based on the resource consumption information to obtain a target large language model; The text intention analysis module is used to obtain target text data in the target field, perform text language analysis on the target text data using the target large language model, and determine the target text intention.

9. An electronic device, characterized in that: The electronic device comprises: at least one processor; and, a memory communicatively connected to the at least one processor; wherein, The memory stores a computer program that can be executed by the at least one processor, and the computer program is executed by the at least one processor so that the at least one processor can perform a text intent analysis method as described in any one of claims 1 to 7.

10. A computer-readable storage medium storing a computer program, characterized in that: When the computer program is executed by a processor, the method for analyzing text intent as described in any one of claims 1 to 7 is implemented.

Images