Text intent analysis method, apparatus, device, and medium

Abstract

The present application relates to the technical field of natural language processing, which can be applied to financial technology, medical health and other business system platforms, and discloses a text intention analysis method, device, equipment and medium. The method comprises: selecting a division strategy for original data according to obtained fine-tuning task requirements, dividing the original data according to the obtained target division strategy to obtain a plurality of target sub-data sets, screening a plurality of obtained federal parameter efficient fine-tuning algorithms according to obtained client resource conditions and fine-tuning task requirements to obtain a target fine-tuning algorithm, adjusting a preset large language model, analyzing resource consumption during the adjustment process, optimizing the performance of the preliminary adjustment model, obtaining a target large language model, and using the target large language model to analyze the text language of obtained target text data to determine a target text intention. The present application effectively improves the stability of the large language model analysis performance and the explainability of the text intention.

Description

Technical Field

[0001] This invention relates to the field of natural language processing technology, and in particular to a text intent analysis method, apparatus, device, and medium. Background Technology

[0002] With the widespread application of Large Language Models (LLMs) in the field of natural language processing, the demand for domain-specific fine-tuning is increasing. In existing technologies, fine-tuning of large language models and using the fine-tuned models for text intent analysis has excellent semantic understanding capabilities and transfer learning effects, but it still has some significant drawbacks.

[0003] In the healthcare field, text intent analysis, through fine-tuning of large language models, can achieve functions such as patient intent recognition, symptom consultation classification, and medical question-and-answer understanding, and is widely used in scenarios such as intelligent triage, online consultation, and electronic medical record analysis. For example, the system can accurately identify a patient's intent regarding "appointment registration" or "symptom consultation," enabling automatic response and guidance, effectively improving the efficiency of medical services. However, due to limitations in data privacy, model stability, and model interpretability, this technology still requires careful consideration in practical deployment.

[0004] In the fintech sector, text intent analysis can be applied to scenarios such as customer service automation, risk alert identification, complaint classification, and transaction behavior analysis. Through fine-tuning of a large language model, the system can accurately understand a user's specific intent when inquiring about loans, reporting lost bank cards, or checking bills, thereby providing personalized services or automatically transferring the user to the appropriate business module. While this technology significantly improves the response speed and accuracy of financial services, it still faces challenges in risk control, model stability, and model interpretability, especially requiring enhanced model controllability and interpretability.

[0005] In summary, fine-tuning typically requires significant computing resources and GPU support, resulting in high training costs. Models are highly sensitive to data distribution, especially prone to overfitting or performance instability in scenarios with imbalanced labels or low resources. Large models lack interpretability, making it difficult to understand and trace the intent judgment results, thus limiting their application in certain high-risk scenarios (such as healthcare and finance).

[0006] Therefore, current technologies suffer from poor performance stability in large language model analysis and poor interpretability of textual intent. Summary of the Invention

[0007] This invention provides a text intent analysis method, apparatus, device, and medium, the main purpose of which is to solve the problems of poor performance stability of large language model analysis and poor interpretability of text intent.

[0008] Firstly, to achieve the above objectives, the present invention provides a text intent analysis method, comprising: Obtain the original data and fine-tuning task requirements of the target domain, and select a partitioning strategy for the original data according to the fine-tuning task requirements to obtain the target partitioning strategy; The original data is divided according to the target partitioning strategy to obtain multiple target subsets; Obtain the client resource status, and based on the client resource status and the fine-tuning task requirements, filter multiple obtained federated parameters for efficient fine-tuning algorithms to obtain the target fine-tuning algorithm; The target subset dataset and the target fine-tuning algorithm are used to adjust the preset large language model to obtain a preliminary adjusted model; Resource consumption analysis is performed on the adjustment process of the large language model to obtain the resource consumption situation, and the performance of the preliminary adjusted model is optimized based on the resource consumption situation to obtain the target large language model; Obtain target text data in the target domain, and use the target large language model to perform text language analysis on the target text data to determine the target text intent.

[0009] Secondly, the present invention also provides a text intent analysis device, comprising: The partitioning strategy selection module is used to acquire the original data of the target domain and the fine-tuning task requirements, and select a partitioning strategy for the original data according to the fine-tuning task requirements to obtain the target partitioning strategy. The original data partitioning module is used to partition the original data according to the target partitioning strategy to obtain multiple target subsets. The fine-tuning algorithm filtering module is used to obtain the client resource status and filter multiple efficient fine-tuning algorithms based on the client resource status and the fine-tuning task requirements to obtain the target fine-tuning algorithm. The large language model adjustment module is used to adjust the preset large language model using the target subset and the target fine-tuning algorithm to obtain a preliminary adjusted model; The large language model optimization module is used to analyze the resource consumption of the adjustment process of the large language model, obtain the resource consumption situation, and optimize the performance of the initially adjusted model based on the resource consumption situation to obtain the target large language model. The text intent analysis module is used to acquire target text data in the target domain, perform text language analysis on the target text data using the target large language model, and determine the target text intent.

[0010] Thirdly, the present invention also provides an electronic device, the electronic device comprising: At least one processor; and, A memory communicatively connected to the at least one processor; wherein, The memory stores a computer program that can be executed by the at least one processor, the computer program being executed by the at least one processor to enable the at least one processor to perform the text intent analysis method described above.

[0011] Fourthly, the present invention also provides a computer-readable storage medium storing at least one computer program, which is executed by a processor in an electronic device to implement the text intent analysis method described above.

[0012] This invention acquires raw data from the target domain and fine-tuning task requirements. Based on these requirements, a partitioning strategy is selected for the raw data to obtain a target partitioning strategy. This adaptive strategy selection not only improves the representativeness and diversity of the fine-tuning data but also optimizes the model training effect and generalization ability, contributing to the construction of a more robust and efficient large language model. The raw data is partitioned according to the target partitioning strategy to obtain multiple target subsets. Client resource conditions are acquired, and multiple acquired efficient federated parameter fine-tuning algorithms are filtered based on these conditions and the fine-tuning task requirements to obtain a target fine-tuning algorithm. The filtering process flexibly matches the most suitable efficient federated parameter fine-tuning algorithm based on the client's model access permissions, resource conditions, and task requirements, exhibiting high adaptability and practical usability. The target subsets and the target fine-tuning algorithm are used to adjust a preset large language model to obtain a preliminary adjusted model. An initialization strategy is flexibly configured according to the target fine-tuning algorithm type, effectively improving the model initialization quality. The word and encoding steps ensure a deep match between the data structure and the model input, improving training efficiency. Through collaborative optimization of acceleration operators and resource-efficient operators, the model achieves superior performance even with limited computing resources. Resource consumption analysis is performed on the adjustment process of the large language model to obtain resource consumption data. Based on this data, the performance of the initially adjusted model is optimized to obtain the target large language model. By combining the resource consumption analysis results and process parameter graphs, targeted adjustments are made to the parameters and modules to be optimized in the large language model. Techniques such as compression, pruning, and adapter simplification are used to dynamically reduce the resource consumption of inefficient or redundant parts while retaining core modules that contribute significantly to performance. This significantly improves the model's operating efficiency and deployment flexibility while maintaining model accuracy. Target text data from the target domain is acquired, and the target large language model is used to perform text language analysis on the target text data to determine the target text intent. This significantly improves the system's ability to understand user needs, achieving high-precision and highly robust intent recognition. Compared to traditional methods, the approach of fine-tuning large language models has stronger semantic understanding and context modeling capabilities, can adapt to complex and diverse language expressions, and can effectively improve the stability of large language model analysis performance and the interpretability of text intent. Attached Figure Description

[0013] To more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings used in the description of the embodiments of the present invention will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0014] Figure 1This is a schematic diagram of an application environment for a text intent analysis method according to an embodiment of the present invention; Figure 2 This is a flowchart illustrating a text intent analysis method according to an embodiment of the present invention. Figure 3 This is a flowchart illustrating the resource consumption analysis process in a text intent analysis method according to an embodiment of the present invention. Figure 4 This is a schematic diagram of a text intent analysis device according to an embodiment of the present invention; Figure 5 A schematic diagram of an electronic device implementing a text intent analysis method according to an embodiment of the present invention; Figure 6 This is another structural schematic diagram of an electronic device that implements a text intent analysis method according to an embodiment of the present invention.

[0015] The objectives, features, and advantages of this invention will be further explained in conjunction with the embodiments and with reference to the accompanying drawings. Detailed Implementation

[0016] To enable those skilled in the art to better understand the technical solutions of this disclosure, and to fully understand and implement the process of how this disclosure applies technical means to solve technical problems and achieve corresponding technical effects, the technical solutions in the embodiments of this disclosure will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this disclosure, not all embodiments. The embodiments of this disclosure and the various features within them can be combined with each other without conflict, and the resulting technical solutions are all within the protection scope of this disclosure. All other embodiments obtained by those skilled in the art based on the embodiments of this disclosure without creative effort should fall within the protection scope of this disclosure.

[0017] It should be noted that the terms "first," "second," etc., in the specification, claims, and accompanying drawings of this disclosure are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments of this disclosure described herein can be implemented in orders other than those illustrated or described herein. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover non-exclusive inclusion; for example, a process, method, apparatus, product, or device that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or devices.

[0018] This application provides a text intent analysis method. The execution subject of this method includes, but is not limited to, at least one of the following electronic devices that can be configured to execute the device provided in this application: a server, a terminal, or other electronic devices. In other words, the text intent analysis method can be executed by software or hardware installed on a terminal device or a server device. The server includes, but is not limited to, a single server, a server cluster, a cloud server, or a cloud server cluster. The server can be an independent server or a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communication, middleware services, domain name services, security services, content delivery networks (CDNs), and big data and artificial intelligence platforms.

[0019] This invention provides a text intent analysis method, which can be applied to applications such as... Figure 1In this application environment, the client communicates with the server via a network. The server can obtain the original data of the target domain and the fine-tuning task requirements from the client. Based on the fine-tuning task requirements, the server selects a partitioning strategy for the original data to obtain a target partitioning strategy. This adaptive strategy selection not only improves the representativeness and diversity of the fine-tuning data but also optimizes the model training effect and generalization ability, helping to build a more robust and efficient large language model. The server partitions the original data according to the target partitioning strategy to obtain multiple target subsets. The server also obtains the client's resource status and filters multiple obtained federated parameter efficient fine-tuning algorithms based on the client's resource status and the fine-tuning task requirements to obtain a target fine-tuning algorithm. The filtering process flexibly matches the most suitable federated parameter efficient fine-tuning algorithm based on the client's model access permissions, resource conditions, and task requirements, exhibiting high adaptability and practical usability. The server uses the target subsets and the target fine-tuning algorithm to adjust the preset large language model to obtain a preliminary adjusted model. The server flexibly configures the initialization strategy according to the target fine-tuning algorithm type, effectively improving the model initialization quality. The word segmentation and encoding steps ensure a deep match between the data structure and the model input, improving training efficiency. Through collaborative optimization of acceleration operators and resource-efficient operators, the model achieves superior performance even with limited computing resources. Resource consumption analysis is performed on the adjustment process of the large language model to obtain resource consumption data. Based on this data, the performance of the initially adjusted model is optimized to obtain the target large language model. By combining the resource consumption analysis results and process parameter graphs, targeted adjustments are made to the parameters and modules to be optimized in the large language model. Techniques such as compression, pruning, and adapter simplification are used to dynamically reduce the resource consumption of inefficient or redundant parts while retaining core modules that contribute significantly to performance. This significantly improves the model's operating efficiency and deployment flexibility while maintaining model accuracy. Target text data in the target domain is obtained, and the target large language model is used to perform text language analysis on the target text data to determine the target text intent. This significantly improves the system's ability to understand user needs, achieving high-precision and highly robust intent recognition. Compared to traditional methods, the approach of fine-tuning a large language model possesses stronger semantic understanding and contextual modeling capabilities, can adapt to complex and diverse language expressions, and effectively improves the stability of large language model analysis performance and the interpretability of textual intent. Finally, the target textual intent is output and fed back to the user client. The client can be, but is not limited to, various personal computers, laptops, smartphones, tablets, and portable wearable devices. The server can be implemented using a standalone server or a server cluster consisting of multiple servers. The invention will be described in detail below through specific embodiments.

[0020] The following is an explanation of the present invention. The present invention combines resource consumption analysis results and process parameter diagrams to selectively adjust the parameters and modules to be optimized in the large language model. It uses techniques such as compression, pruning, and adapter simplification to dynamically reduce the resource consumption of inefficient or redundant parts, while retaining the core modules that contribute significantly to performance. Thus, while ensuring the accuracy of the model, it significantly improves the running efficiency and stability of the model.

[0021] Reference Figure 2 The diagram shown is a flowchart illustrating a text intent analysis method according to an embodiment of the present invention. In this embodiment, the text intent analysis method includes: S1. Obtain the original data and fine-tuning task requirements of the target domain, and select a partitioning strategy for the original data according to the fine-tuning task requirements to obtain the target partitioning strategy.

[0022] In this embodiment of the invention, raw data related to the target domain is collected from multiple channels, including public datasets, domain expert-annotated data, and web crawling data, to ensure data diversity, representativeness, and sufficiency. The raw data is deduplicated to remove duplicate samples, resulting in deduplicated data. Samples irrelevant to the target domain are identified and filtered to obtain more relevant filtered data. The filtered data is then labeled according to task requirements to form a labeled dataset. The distribution difference value of the labeled data across different clients is calculated, and the uniformity of the data distribution is determined based on the distribution difference value. Finally, the most suitable target partitioning strategy is determined based on the data distribution.

[0023] In specific healthcare scenarios, medical data often exhibits significant heterogeneity, with health records from different hospitals, clinics, or patients potentially showing distributional differences. By deduplicating and filtering data irrelevant to disease prediction, the model can ensure it focuses on relevant conditions and symptoms. Through labeling and dissimilarity calculations, if the sample distribution for a particular disease is significantly different, a Dirichlet distribution strategy can be used. This helps balance the training of data from different patient types, enhancing the model's generalization ability. If the data is relatively balanced, a uniform partitioning strategy can be used, facilitating knowledge sharing among hospitals and improving the model's accuracy and robustness.

[0024] In specific fintech scenarios, financial data typically originates from different regions and customer groups, and the data distribution can vary significantly. By deduplicating and filtering data irrelevant to financial risk prediction, the model ensures that it learns solely based on core financial characteristics. During the discrepancy calculation, if the data distribution varies considerably across different regions or customer groups, a Dirichlet distribution strategy can better handle this uneven distribution, improving the model's accuracy and adaptability. Conversely, when the data sources are relatively consistent, a uniform partitioning strategy ensures the model's balance across different markets and customer groups, optimizing predictive performance.

[0025] In this embodiment of the invention, the step of selecting a partitioning strategy for the original data based on the fine-tuning task requirements to obtain a target partitioning strategy includes: The original data is deduplicated to obtain deduplicated data; Identify irrelevant data in the deduplicated data that is unrelated to the target domain, and filter the irrelevant data in the deduplicated data to obtain filtered data; The filtered data is labeled according to the fine-tuning task requirements to obtain labeled data; Determine the distribution difference value of the labeled data, and determine whether the distribution difference value is greater than a preset difference threshold; If the distribution difference value is greater than the difference threshold, then the Dirichlet distribution partitioning strategy is used as the target partitioning strategy. If the distribution difference value is less than or equal to the difference threshold, then the uniform partitioning strategy will be used as the target partitioning strategy.

[0026] In detail, the uniqueness of samples in the original dataset is identified, usually based on comparison of specific fields (such as text content, ID, timestamp, etc.). Data entries with completely identical or highly similar content are identified, and these duplicate or redundant records are removed, leaving only a representative data sample. Finally, a clean deduplicated dataset without duplicates is obtained, laying the foundation for subsequent data filtering and model training.

[0027] By using domain-related rules or domain classification models, each deduplicated data point is matched for domain and relevance. Data samples that do not conform to the characteristics or themes of the target domain are screened out, and these irrelevant data are removed or filtered from the dataset. The removal process can be based on methods such as keyword matching, semantic similarity, or classification labels. Finally, filtered data containing only data highly relevant to the target domain is obtained, providing an accurate and focused data foundation for subsequent annotation and model training.

[0028] Based on the specific task type (such as classification, entity recognition, question answering, etc.), corresponding annotation specifications and standards are formulated. Each sample in the filtered data is labeled by human annotators or automatic annotation tools to ensure that the annotation content meets the task objectives and quality requirements. When necessary, multiple rounds of review and correction are carried out to improve the accuracy of annotation. Finally, high-quality labeled data that meets the fine-tuning needs are generated to provide accurate supervision signals for model training.

[0029] The sample size of each category or feature in the labeled data is counted to obtain the actual distribution of the data. A reference distribution is determined, which can be an ideal uniform distribution or the distribution pattern expected by the task. The degree of difference between the actual distribution and the reference distribution is quantified by calculating the difference measure index between the two, such as KL divergence, JS divergence or total variation distance. This is the distribution difference value, which is used to reflect the degree of deviation of the data distribution and thus guide the selection of partitioning strategy.

[0030] If the difference is large, it indicates that the data distribution is biased towards certain categories or features, and the Dirichlet distribution strategy, which can reflect the diversity of the data, is suitable. If the difference is small, it indicates that the data is relatively balanced, and the uniform partitioning strategy is adopted to ensure the fairness and representativeness of the data partitioning, thereby providing a reasonable data foundation for subsequent tasks.

[0031] Through systematic data preprocessing and intelligent partitioning strategy selection, the quality and adaptability of the data used for fine-tuning are ensured. Deduplication and filtering steps effectively remove redundant and irrelevant data, improving data purity and relevance. The annotation process generates high-quality supervisory information based on specific task requirements. By calculating the distribution difference value of the labeled data, the data balance can be dynamically determined, thereby selecting a more suitable partitioning strategy. This adaptive strategy selection not only improves the representativeness and diversity of the fine-tuning data but also optimizes the model training effect and generalization ability, contributing to the construction of more robust and efficient large language models.

[0032] S2. Divide the original data according to the target partitioning strategy to obtain multiple target subsets.

[0033] In this embodiment of the invention, if the target partitioning strategy is a uniform partitioning strategy, a uniform partitioning ratio is obtained, and the labeled data is evenly distributed to each subset according to the ratio, ensuring consistency in data volume and category distribution for each subset. If the target partitioning strategy is a Dirichlet distribution partitioning strategy, the category label information and partitioning heterogeneity parameters in the labeled data are obtained, and the distribution ratio of each category label in each subset is randomly generated according to the Dirichlet distribution. Combined with the heterogeneity parameters, the labeled data is non-uniformly partitioned, thereby generating multiple target subsets with different distribution characteristics to more realistically simulate the heterogeneous distribution of client data in reality.

[0034] In specific healthcare scenarios, for disease prediction or medical image analysis, patient data often exhibits diverse categories and distribution characteristics, such as medical records from different regions and hospitals. A uniform partitioning strategy ensures that samples from various disease categories are evenly distributed across different subsets, preventing an overabundance or underabundance of data for any particular disease category. When faced with significant regional differences or imbalanced case distributions, a Dirichlet distribution partitioning strategy can dynamically allocate data proportions based on the heterogeneity of different categories. For example, allocation proportions can be randomly generated based on the distribution of specific diseases and resource constraints, making the training data more closely reflect the data heterogeneity in real clinical scenarios.

[0035] In specific fintech scenarios, uneven data distribution is a common challenge when processing customer data from different financial institutions and regions. A uniform partitioning strategy ensures a reasonable ratio between different customer types (e.g., high-risk and low-risk customers). In complex financial scenarios, such as training on customer data with varying risk levels, the Dirichlet distribution partitioning strategy can randomly generate appropriate allocation ratios based on the distribution of different risk labels. This further enhances the model's robustness when dealing with highly heterogeneous datasets, especially when facing diverse data across regions and industries.

[0036] In this embodiment of the invention, the step of dividing the original data according to the target partitioning strategy to obtain multiple target subsets includes: When the target partitioning strategy is a uniform partitioning strategy, the partitioning ratio is obtained according to the uniform partitioning strategy. The labeled data is divided according to the specified division ratio to obtain multiple target subsets; When the target partitioning strategy is a Dirichlet distribution partitioning strategy, obtain the category labels and partitioning heterogeneity parameters of the labeled data; The allocation ratio of each category label is randomly generated according to the Dirichlet distribution partitioning strategy; The labeled data is divided according to the allocation ratio and the partitioning heterogeneity parameter to obtain multiple target subsets.

[0037] In detail, when the target partitioning strategy is a uniform partitioning strategy, the proportion of each subset should be determined according to the uniform partitioning rule. Usually, the data volume of each subset is kept roughly equal or distributed according to a fixed ratio. The labeled data is divided into multiple target subsets in a sequential or random manner according to the partitioning ratio, ensuring that each subset is as balanced as possible in terms of quantity and category distribution, so as to meet the needs of subsequent model training or evaluation.

[0038] When the target partitioning strategy is a Dirichlet distribution, the class labels and partitioning heterogeneity parameters from the labeled data are obtained to reflect the distributional differences among the classes. Based on the partitioning heterogeneity parameters (usually the concentration parameters of the Dirichlet distribution), a Dirichlet distribution model is initialized. These parameters control the sparsity of the allocation ratios and the imbalance between classes. Sampling is performed from this Dirichlet distribution to obtain the specific allocation ratio vector of each class label in each subset. Because the Dirichlet distribution itself has randomness and controllable diversity, the sampling process can generate different proportional distributions that conform to the set heterogeneity level, ensuring that the data partitioning is both random and reflects the heterogeneity characteristics of the real data. By combining these allocation ratios and heterogeneity parameters, labeled data is distributed into multiple target subsets according to categories and proportions to simulate the heterogeneous distribution of data in real-world scenarios, supporting more representative and robust model training.

[0039] Combining uniform and Dirichlet distribution strategies, this approach flexibly adapts to the needs of different data distribution characteristics. The uniform distribution strategy ensures a balanced distribution of data among subsets, contributing to the stability and fairness of the model during training. Meanwhile, the Dirichlet distribution strategy, by introducing heterogeneity parameters and random proportional sampling, realistically simulates the imbalance and diversity of data, enhancing the model's adaptability to complex real-world scenarios. This partitioning scheme ensures both the scientific and rational nature of data partitioning and strengthens the generalization and robustness of model training.

[0040] S3. Obtain the client resource status, and based on the client resource status and the fine-tuning task requirements, filter the multiple obtained federated parameters for efficient fine-tuning algorithms to obtain the target fine-tuning algorithm.

[0041] In this embodiment of the invention, the model access status of the target client is obtained, and it is determined whether the model access status has access permissions to the model weights. Further determination is then made based on resource availability. If the client can access the model, a specific algorithm from the efficient fine-tuning algorithm for federated parameters is selected according to the task type. If the client cannot access the model, a federated optimization algorithm based on optimal transmission is adopted. This algorithm effectively coordinates the model in inaccessible environments by modeling distribution differences, ensuring a balance between privacy and training efficiency.

[0042] In specific healthcare scenarios, if a hospital has limited computing resources, it can employ prompt-based optimization algorithms to reduce resource consumption while still ensuring disease prediction or medical image analysis can be performed within its limited computing power. If the hospital has sufficient computing resources and the task requires fine-tuning (e.g., refining the early diagnosis of a certain disease), a low-rank adaptive algorithm (LoRA) can be chosen, making the fine-tuning process more efficient and accurate, adapting to the characteristics of specific diseases. If the task focuses on optimizing text coherence, such as optimizing the generation of electronic medical records or the automatic writing of medical articles, a prompt-based fine-tuning algorithm can be used to make the generated text more coherent and readable. For remote hospitals that cannot access certain resources, a federated optimization algorithm based on optimal transmission can be used to improve the collaborative efficiency of the model and protect patient privacy by modeling and optimizing the data distribution.

[0043] In specific fintech scenarios, if a financial institution has limited computing resources, it can choose a prompt-based optimization algorithm to fine-tune credit scoring or fraud detection models, ensuring task completion within limited resources. When a financial institution has strong computing power and the task requires fine-tuning an existing credit scoring model, a low-rank adaptive algorithm (LoRA) can be chosen. This algorithm can efficiently optimize for the financial behavior of specific customer groups, improving the accuracy of risk prediction. If the task focuses on optimizing text coherence, such as generating accurate financial reports or investment analysis reports, a prompt-based federated fine-tuning algorithm can be selected to enhance the fluency and contextual consistency of text generation. For financial platforms that cannot directly access customer data, a federated optimization algorithm based on optimal transmission can be used to achieve efficient cross-institutional collaboration and data privacy protection by intelligently adjusting the distribution of heterogeneous data.

[0044] In this embodiment of the invention, the step of filtering multiple acquired federated parameters for efficient fine-tuning algorithms based on the client resource status and the fine-tuning task requirements to obtain the target fine-tuning algorithm includes: Obtain the model access status of the target client and determine whether the model access status is accessible or inaccessible; If the model access status is accessible, then when the client resource status is resource-scarce, the hint tuning algorithm in the efficient fine-tuning algorithm of the obtained federated parameters will be used as the target fine-tuning algorithm. When the client resource status is sufficient, determine whether the fine-tuning task requirement is a fine-tuning task or a text coherence optimization task. If the fine-tuning task requires precise adjustment, then the low-rank adaptive algorithm in the efficient fine-tuning algorithm for federated parameters will be used as the target fine-tuning algorithm. If the fine-tuning task requirement is a text coherence optimization task, then the efficient fine-tuning algorithm based on federated parameters of prompt words will be used as the target fine-tuning algorithm. If the model access status is inaccessible, then the federated optimization algorithm based on optimal transport theory in the federated parameter efficient fine-tuning algorithm will be used as the target fine-tuning algorithm.

[0045] In detail, the target client's access permissions to the large language model are evaluated to determine whether the model access status is "accessible" or "inaccessible". If the client can access the model ontology and computational resources are limited, the prompt tuning algorithm is selected as the target fine-tuning algorithm. Prompt tuning significantly reduces computational and storage overhead by optimizing only a small number of prompt vectors without modifying the main model parameters, making it suitable for lightweight terminal devices.

[0046] If the client has access to the model ontology and sufficient resources, the specific requirements of the fine-tuning task will be further determined. If the task is a fine-tuning task (such as medical question answering, financial proofreading, etc., where the accuracy of output is extremely important), then the Low-Rank Adaptive Algorithm (LoRA) is selected as the target fine-tuning algorithm. The LoRA algorithm fine-tunes parameters by introducing a low-rank matrix into the model's weight matrix, maintaining the original model structure while efficiently improving performance. It is suitable for scenarios with sufficient resources and high requirements for model performance accuracy. If the task is a text coherence optimization task (such as dialogue generation, report polishing, etc.), then a prompt-based federated parameter efficient fine-tuning algorithm (such as Prompt Tuning or P-Tuning) is selected as the target algorithm. The prompt-based federated parameter efficient fine-tuning algorithm learns a set of continuous or discrete prompt vectors to optimize the contextual understanding ability of the language model, improving text coherence and generation quality without modifying the main model parameters. The selection strategy reasonably matches the algorithm type according to the different task objectives, giving full play to resource advantages while improving the relevance and practicality of the model's performance.

[0047] When clients cannot access model parameters (e.g., due to privacy, licensing restrictions), the FedOT algorithm is a federated learning optimization algorithm based on optimal transport theory. It aims to effectively align distributional differences between clients without sharing the original data and model parameters. FedOT constructs consistency constraints by optimally transporting the client's local model output (e.g., feature distribution or prediction distribution) to the global reference distribution, thereby improving the generalization ability and convergence stability of the global model in non-independent and identically distributed (Non-IID) data environments.

[0048] The selection process flexibly matches the most suitable federated parameters and efficient fine-tuning algorithms based on the client's model access permissions, resource conditions, and task requirements, exhibiting high adaptability and practical usability. When the model is accessible, the system can select solutions such as hint-based tuning (efficient and lightweight) or low-rank adaptive algorithms (high-performance fine-tuning) based on resource status and task type, achieving a balance between resource optimization and task performance. When the model is inaccessible, a federated optimization algorithm based on optimal transmission is adopted, effectively improving the model's performance in non-independent identically distributed (Non-IID) environments while protecting privacy. The strategy optimizes the path through "on-demand matching," enhancing the practicality, efficiency, and generalization ability of federated learning.

[0049] S4. The preset large language model is adjusted using the target subset and the target fine-tuning algorithm to obtain a preliminary adjusted model.

[0050] In this embodiment of the invention, the parameters of the original large language model are initialized and an initial language model is constructed based on whether the selected fine-tuning algorithm is a federated optimization algorithm based on optimal transmission. The target subset of the dataset is segmented and encoded to obtain the corresponding input vectors and generate real data labels. Predicted labels are obtained through model inference, and the loss value between the predicted and real labels is calculated using a loss function. Then, by combining acceleration operators and resource-efficient operators, the model is iteratively optimized to generate adjusted language models for each client. The model parameters of each client are integrated by weighted averaging to obtain global model parameters, which are then used to further fine-tune the model, ultimately forming a preliminary fine-tuned model, providing a foundation for subsequent optimization.

[0051] In specific healthcare scenarios, patient data from different hospitals often exhibits heterogeneity. By adjusting large language models based on different fine-tuning algorithms, customized models can be efficiently developed for each hospital. While federated optimization algorithms based on optimal transmission cannot directly share data between hospitals, initialization schemes are generated through the configuration of compression simulators and initial adapters. Combined with the distribution of patient data, the model can maintain data privacy while effectively optimizing. For disease diagnosis tasks, word segmentation and encoding are performed using a target subset dataset, and the accuracy and robustness of disease prediction are optimized by comparing the model-generated predicted labels with the true labels. Finally, a weighted average of global model parameters is used to fuse the knowledge from various hospitals, resulting in an accurate and adaptable initial adjustment model capable of handling complex situations in different healthcare scenarios.

[0052] In specific fintech scenarios, where data sharing among financial institutions is strictly limited, federated optimization algorithms based on optimal transmission allow different banks to jointly train models using encrypted optimization protocols. Initial configurations are generated through tools such as compression simulators, ensuring each bank optimizes the model while maintaining data privacy. For credit assessment tasks, financial data from the target subset is segmented and encoded to generate corresponding financial tags. By comparing these tags with predicted tags, the model's performance across different customer groups is optimized. Finally, the model is fine-tuned using global parameter weighted averaging and resource-efficient operators, resulting in a preliminary adjusted model that integrates information from various banks and responds to market changes, thereby improving cross-institutional risk prediction capabilities.

[0053] In this embodiment of the invention, the step of adjusting the preset large language model using the target subset and the target fine-tuning algorithm to obtain a preliminary adjusted model includes: Determine whether the target fine-tuning algorithm is a federated optimization algorithm based on optimal transmission; If the target fine-tuning algorithm is a federated optimization algorithm based on optimal transmission, then obtain the compression simulator and the initial adapter, and generate an initialization configuration based on the compression simulator and the initial adapter; If the target fine-tuning algorithm is not a federated optimization algorithm based on optimal transmission, then obtain the initial learnable parameters and generate the initial configuration based on the initial learnable parameters; The parameters of the preset large language model are initialized using the initialization configuration to obtain the initial language model; The target subset of data is segmented and encoded to obtain multiple target segmentation codes; Generate real data tags based on the target word segmentation encoding; Obtain the predicted label of the target word segmentation code, and use a preset loss function to determine the loss value between the predicted label and the real data label; Obtain acceleration operators and resource-efficient operators, and use the loss value, acceleration operators, and resource-efficient operators to adjust the initial language model to obtain an adjusted language model; The global model parameters are obtained by taking a weighted average of the model parameters of the adjusted language model for all the target clients. The language model is fine-tuned using the global model parameters to obtain a preliminary fine-tuned model.

[0054] Specifically, before fine-tuning begins, it is determined whether the selected target fine-tuning algorithm is a federated optimization algorithm based on optimal transfer. If so, it means that the client cannot directly access the model parameters. Therefore, the system will obtain a compression simulator and an initial adapter, and use these two to build an initialization configuration for cross-client transfer and adaptation, thereby achieving efficient collaboration and alignment in the model update process. If the target fine-tuning algorithm is not based on the optimal transfer method (e.g., cue tuning or low-rank adaptive algorithm), the system will obtain the corresponding initial learnable parameters (such as cue vectors, LoRA weights, etc.), and generate the initialization configuration required for fine-tuning accordingly to support subsequent efficient parameter training, ensuring that the initialization method matches the algorithm characteristics, and guaranteeing the effectiveness and resource utilization efficiency of the fine-tuning process.

[0055] If cue-based tuning or low-rank adaptive algorithms are used, initialized learnable parameters (such as cue vectors or low-rank matrices) are injected into specific locations in the model. If a federated optimization algorithm based on optimal transfer is used, lightweight structural adaptation and alignment of the model are performed by loading parameters from a compression simulator and an initial adapter. The entire process does not change the main structure of the original model, but rather loads adjustable components on top of it to build an initial language model with fine-tuning capabilities, laying the foundation for subsequent training and optimization.

[0056] For each target subset of the dataset, text data is segmented and encoded, converting natural language into a vector form recognizable by the model, resulting in multiple target segmentation codes. Based on the task requirements (e.g., text classification, question answering, summarizing) for each code, corresponding ground truth labels are generated as the targets for supervised learning. The initial language model performs forward propagation on these encoded inputs, outputting corresponding predicted labels. Loss functions (e.g., cross-entropy, mean squared error) are used to calculate the loss between the predicted and ground truth labels, measuring the current model's output bias and providing a basis for subsequent model parameter optimization.

[0057] After obtaining the loss value, acceleration operators and resource-efficient operators are introduced to jointly optimize and adjust the initial language model. Acceleration operators aim to speed up the model training and convergence process, such as using gradient pruning and mixed-precision training techniques. Resource-efficient operators are used to reduce computational resource consumption, such as parameter sharing, low-rank updates, and weight compression. Combining these operators with the loss value, the learnable parameters are updated using gradients through backpropagation, thereby improving training efficiency and resource utilization while maintaining performance. This ultimately yields an adjusted language model, laying the foundation for subsequent federated aggregation or further fine-tuning.

[0058] After completing the local model tuning for each target client, the tuned language model parameters from all clients are weighted and averaged. The weights can be set based on factors such as sample size, data quality, or resource contribution, thereby fusing a unified global model parameter. This global parameter is then used to synchronously update the tuned language model for each client, i.e., to perform a globally consistent fine-tuning. This results in a preliminary fine-tuned model with better generalization ability and task adaptability, providing an optimization foundation for the next round of training or final deployment.

[0059] The initialization strategy is flexibly configured according to the target fine-tuning algorithm type, which effectively improves the quality of model initialization. The word segmentation and encoding steps ensure that the data structure and model input depth are well matched, thereby improving training efficiency. The error is quantified by using the loss function, and the model can still achieve better performance under limited computing resources through the collaborative optimization of acceleration operators and resource-efficient operators. By weighted aggregation and global fine-tuning of the parameters of each client model, the model's rapid convergence and generalization ability are enhanced, meeting the needs of efficient fine-tuning in multi-task and multi-terminal environments.

[0060] S5. Analyze the resource consumption of the adjustment process of the large language model to obtain the resource consumption situation, and optimize the performance of the preliminary adjustment model based on the resource consumption situation to obtain the target large language model.

[0061] In this embodiment of the invention, the fine-tuning task type is determined and corresponding analysis parameters are generated. Values ​​are assigned according to the model fine-tuning process, and the fine-tuning effect is analyzed. Simultaneously, computational resource consumption is recorded in real time, and a resource consumption graph is generated. The fine-tuning effect values ​​are mapped to the resource consumption graph to form a process parameter graph, and resource consumption is extracted accordingly. Based on the resource consumption, parameters and modules to be optimized are identified. The model structure is optimized through operations such as parameter adjustment, compression, and module pruning, ultimately obtaining a target large language model with better performance and resource utilization.

[0062] In specific healthcare scenarios, the resource consumption perception and optimization process can be applied to the model fine-tuning of intelligent consultation systems. By analyzing the model's resource consumption and performance in real time when recognizing patient intentions (such as registration, symptom consultation, and medication recommendations), the system enables visualized management of computational resource consumption and evaluation of fine-tuning efficiency. The system can automatically optimize model parameters and structure based on task complexity, such as pruning modules with minimal impact on diagnostic judgment, thereby reducing resource consumption while maintaining diagnostic accuracy. This approach is suitable for edge deployments or resource-constrained environments within hospital information systems.

[0063] In specific fintech scenarios, it can be applied to customer service robots or risk control systems to analyze and visually evaluate the resource consumption of the model when handling intent recognition tasks (such as loan inquiries, account anomaly reports, and transaction behavior analysis). Based on the fine-tuning effect and resource consumption relationship, the system can intelligently compress or optimize modules in the model that contribute less to business response, minimizing model deployment costs while ensuring the real-time performance and accuracy of services. It is suitable for financial business systems in high-concurrency scenarios.

[0064] Figure 3 This is a flowchart illustrating the resource consumption analysis process in a text intent analysis method provided in an embodiment of the present invention.

[0065] In this embodiment of the invention, the resource consumption analysis of the adjustment process of the large language model to obtain the resource consumption situation includes: Obtain the task type required for the fine-tuning task, and generate task analysis parameters based on the task type; The parameter values ​​of the task analysis parameters are determined based on the adjustment process of the large language model. The fine-tuning effect of the large language model is analyzed based on the parameter values ​​to obtain the fine-tuning effect value; The resource consumption during the adjustment process of the large language model is recorded in real time, and the real-time recorded data is visualized to obtain a resource consumption graph. The fine-tuning effect values ​​are mapped to the resource consumption graph to obtain the process parameter graph; Resource consumption information is generated based on the process parameter diagram.

[0066] In detail, the specific type of fine-tuning task (such as classification, generation, or optimization task) is identified, and corresponding task analysis parameters are generated accordingly. These parameters cover key factors such as model performance indicators, computational resource requirements, and training strategies. Combined with the actual adjustment process of the large language model, the specific values ​​of these parameters are dynamically determined, thereby accurately reflecting the characteristics and resource consumption of the current fine-tuning task and providing data support for subsequent performance evaluation and optimization.

[0067] Based on the established task analysis parameters, performance metrics of the large language model at each stage of fine-tuning are collected, including but not limited to accuracy, precision, recall, F1 score, loss function value, and convergence speed. These metrics are used to evaluate the model's performance on the specified fine-tuning task from multiple dimensions, analyze the differences in model performance on the training and validation sets, identify overfitting or underfitting issues, and, in conjunction with the specific requirements of the task (such as text generation quality, classification accuracy, etc.), calculate a value representing the overall fine-tuning effect, i.e., the fine-tuning effect value, through weighted or comprehensive methods. This value can be used to guide the adjustment of subsequent optimization strategies.

[0068] During the adjustment of the large language model, the system monitors and records key resource consumption indicators in real time, such as computation time, memory usage, GPU / CPU load and energy consumption, and visualizes these data in the form of charts to form a resource consumption graph.

[0069] In the resource consumption graph, resource usage indicators (such as computation time, memory usage, computing power consumption, etc.) corresponding to different time points or stages are quantified into specific values. The fine-tuning effect value, as a performance indicator, is converted into the same measurement scale through normalization or standardization. During the mapping process, the fine-tuning effect value at each time point is associated with the resource consumption value corresponding to that time point to form a two-dimensional coordinate point. The horizontal axis represents resource consumption, and the vertical axis represents the fine-tuning effect. Alternatively, it can intuitively show the trend of the fine-tuning effect changing with resource consumption. Based on the process parameter graph, the relationship between resource utilization efficiency and fine-tuning effect is analyzed, and the overall resource consumption situation is extracted and output to provide a basis for subsequent optimization and decision-making.

[0070] By comprehensively analyzing and visualizing the resource consumption during the tuning process of a large language model in real time, and combining it with the mapping of fine-tuning effect values, the relationship between model performance and resource usage can be intuitively reflected. This helps to accurately evaluate the efficiency and effectiveness of fine-tuning tasks, not only optimizing resource allocation and improving computational utilization, but also guiding subsequent tuning strategies to ensure the best fine-tuning effect under limited resource conditions, thereby improving the overall system performance and practical value.

[0071] In this embodiment of the invention, the step of optimizing the performance of the preliminary adjustment model based on the resource consumption to obtain the target large language model includes: Based on the resource consumption situation, generate parameters and modules to be optimized; The parameters to be optimized are adjusted according to the process parameter diagram to obtain the adjusted parameters; The adjustment parameters are compressed to obtain compressed parameters; Determine the contribution parameter value of the module to be optimized, and determine whether the contribution parameter value is greater than a preset contribution threshold; If the contribution parameter value is less than or equal to the contribution threshold, then the modules to be optimized corresponding to contribution parameter values ​​that are less than or equal to the contribution threshold are pruned. By reducing the number of adapter parameters in the remaining modules to be optimized, the adjusted modules are obtained. If the contribution parameter value is greater than the contribution threshold, then the number of adapter parameters in the module to be optimized is reduced to obtain the adjustment module; The initial adjustment model is optimized using the compression parameters and the adjustment module to obtain the target large language model.

[0072] In detail, based on resource consumption, the parameters and modules in the model that need to be optimized are identified. These parameters to be optimized are then reasonably adjusted in conjunction with the process parameter diagram to improve performance and reduce resource consumption. The adjusted parameters are then compressed to reduce model size and computational complexity, thus laying the foundation for subsequent model optimization and acceleration.

[0073] The importance of each parameter within the module to be optimized is scored based on metrics such as the product of gradient and weight. These scores are then weighted and summed or averaged to obtain the overall contribution parameter value. This contribution parameter value is compared with a contribution threshold. If the contribution parameter value is less than or equal to the threshold, the module to be optimized is considered to have a low contribution to model performance, and a pruning operation is performed. Specifically, some or all parameters in the module are deleted or compressed to reduce model size and computational resource consumption. For the remaining modules to be optimized after pruning, the number of adapter parameters in the remaining modules is further reduced in a refined manner. This is usually done through techniques such as pruning or parameter sharing to screen out adapter parameters that have a small contribution to model performance and delete or merge them, thereby reducing the parameter size of the module. If the contribution parameter value is greater than the contribution threshold, the same operation is performed on the modules to be optimized with contribution parameter values ​​greater than the threshold. Finally, a more streamlined and computationally efficient adjustment module is obtained to improve the overall model performance and resource utilization efficiency.

[0074] By integrating and optimizing the preliminary adjusted model using the obtained compression parameters and adjusted modules, and by replacing or merging the corresponding parameters and module structures, model redundancy and computational burden are reduced, while maintaining or improving model performance, thereby generating a more efficient and concise target large language model that meets the balance requirements of resources and performance.

[0075] By combining resource consumption analysis results and process parameter diagrams, the parameters and modules to be optimized in the large language model are adjusted in a targeted manner. Techniques such as compression, pruning, and adapter simplification are used to dynamically reduce the resource consumption of inefficient or redundant parts, while retaining the core modules that contribute significantly to performance. This significantly improves the model's running efficiency and deployment flexibility while ensuring model accuracy, meeting the practical needs of large models in computing-constrained environments.

[0076] S6. Obtain target text data in the target domain, and use the target large language model to perform text language analysis on the target text data to determine the target text intent.

[0077] In this embodiment of the invention, representative target text data, such as patient consultation dialogues and financial user inquiry records, are obtained from the target domain (e.g., healthcare or fintech). The target text data is preprocessed, including word segmentation, noise reduction, entity recognition, and format standardization. The cleaned text is then input into an optimized target large-scale language model. Leveraging powerful semantic understanding and context modeling capabilities, the model performs deep language analysis on the input text, extracting semantic features and contextual relationships. The model, combined with the domain intent labeling system learned during previous fine-tuning, identifies and classifies the intent of each piece of text data, ultimately outputting accurate target text intent to support downstream applications such as automatic question answering, business distribution, or intelligent recommendation.

[0078] In specific healthcare scenarios, it can be applied to intelligent triage and online consultation systems. By acquiring patients' natural language input (such as "My cough has worsened recently, I want to ask if it is pneumonia"), it uses an optimized medical big language model to perform semantic analysis, accurately identify their intent as "symptom consultation" or "disease self-diagnosis", and further extract relevant symptom keywords to assist in triage, recommend departments, or generate preliminary consultation suggestions, thereby improving the efficiency of the diagnosis and treatment process and the patient service experience.

[0079] In specific fintech scenarios, it is applicable to intelligent customer service and risk control early warning systems. By collecting users' text input in scenarios such as mobile banking and loan platforms (such as "My account just had an unauthorized transaction"), the optimized financial domain big data language model performs in-depth semantic analysis to identify whether the user's intent is "account abnormality appeal" or "risk alarm". Combined with historical behavior, it quickly classifies and processes the data to achieve automatic triage, risk warning or transfer to human response, thereby improving the intelligence and security of financial services.

[0080] By acquiring text data from the target domain and using a large target language model for language analysis to extract textual intent, the system's ability to understand user needs can be significantly improved, achieving high-precision and robust intent recognition. Compared to traditional methods, the approach of fine-tuning a large language model possesses stronger semantic understanding and contextual modeling capabilities, adapting to complex and diverse language expressions. This results in more accurate and intelligent service responses, enhancing user experience while improving model stability and the interpretability of textual intent.

[0081] It should be understood that the sequence number of each step in the above embodiments does not imply the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of the present invention.

[0082] like Figure 4 The diagram shown is a functional block diagram of a text intent analysis device provided in an embodiment of the present invention.

[0083] In this embodiment of the disclosure, a text intent analysis device is provided, which corresponds one-to-one with the text intent analysis method described in the above embodiments. For example... Figure 4 As shown, this text intent analysis device 100 can be installed in an electronic device. According to its functions, the text intent analysis device 100 includes a segmentation strategy selection module 101, a raw data segmentation module 102, a fine-tuning algorithm filtering module 103, a large language model adjustment module 104, a large language model optimization module 105, and a text intent analysis module 106. Detailed descriptions of each functional module are as follows: The partitioning strategy selection module 101 is used to acquire the original data of the target domain and the fine-tuning task requirements, and select a partitioning strategy for the original data according to the fine-tuning task requirements to obtain the target partitioning strategy. The original data partitioning module 102 is used to partition the original data according to the target partitioning strategy to obtain multiple target subsets; The fine-tuning algorithm filtering module 103 is used to obtain the client resource status, and filter multiple efficient fine-tuning algorithms of the obtained federated parameters according to the client resource status and the fine-tuning task requirements to obtain the target fine-tuning algorithm. The large language model adjustment module 104 is used to adjust the preset large language model using the target subset and the target fine-tuning algorithm to obtain a preliminary adjusted model; The large language model optimization module 105 is used to analyze the resource consumption of the adjustment process of the large language model, obtain the resource consumption situation, and optimize the performance of the preliminary adjusted model based on the resource consumption situation to obtain the target large language model. The text intent analysis module 106 is used to acquire target text data in the target domain, perform text language analysis on the target text data using the target large language model, and determine the target text intent.

[0084] In one embodiment, when the partitioning strategy selection module 101 performs partitioning strategy selection on the original data according to the fine-tuning task requirements to obtain the target partitioning strategy, it is used to: The original data is deduplicated to obtain deduplicated data; Identify irrelevant data in the deduplicated data that is unrelated to the target domain, and filter the irrelevant data in the deduplicated data to obtain filtered data; The filtered data is labeled according to the fine-tuning task requirements to obtain labeled data; Determine the distribution difference value of the labeled data, and determine whether the distribution difference value is greater than a preset difference threshold; If the distribution difference value is greater than the difference threshold, then the Dirichlet distribution partitioning strategy is used as the target partitioning strategy. If the distribution difference value is less than or equal to the difference threshold, then the uniform partitioning strategy will be used as the target partitioning strategy.

[0085] In one embodiment, when the original data partitioning module 102 performs the partitioning of the original data according to the target partitioning strategy to obtain multiple target subsets, it is used to: When the target partitioning strategy is a uniform partitioning strategy, the partitioning ratio is obtained according to the uniform partitioning strategy; The labeled data is divided according to the specified division ratio to obtain multiple target subsets; When the target partitioning strategy is a Dirichlet distribution partitioning strategy, the category labels and partitioning heterogeneity parameters of the labeled data are obtained; The allocation ratio of each category label is randomly generated according to the Dirichlet distribution partitioning strategy; The labeled data is divided according to the allocation ratio and the partitioning heterogeneity parameter to obtain multiple target subsets.

[0086] In one embodiment, when the fine-tuning algorithm filtering module 103 performs a process of filtering multiple acquired federated parameter efficient fine-tuning algorithms based on the client resource status and the fine-tuning task requirements to obtain the target fine-tuning algorithm, it is used to: Obtain the model access status of the target client and determine whether the model access status is accessible or inaccessible; If the model access status is accessible, then when the client resource status is resource-scarce, the hint tuning algorithm in the efficient fine-tuning algorithm of the obtained federated parameters will be used as the target fine-tuning algorithm. When the client resource status is sufficient, determine whether the fine-tuning task requirement is a fine-tuning task or a text coherence optimization task. If the fine-tuning task requires precise adjustment, then the low-rank adaptive algorithm in the efficient fine-tuning algorithm for federated parameters will be used as the target fine-tuning algorithm. If the fine-tuning task requirement is a text coherence optimization task, then the efficient fine-tuning algorithm based on federated parameters of prompt words will be used as the target fine-tuning algorithm. If the model access status is inaccessible, then the federated optimization algorithm based on optimal transport theory in the federated parameter efficient fine-tuning algorithm will be used as the target fine-tuning algorithm.

[0087] In one embodiment, when the large language model adjustment module 104 performs adjustments to the preset large language model using the target subset and the target fine-tuning algorithm to obtain a preliminary adjusted model, it is used to: Determine whether the target fine-tuning algorithm is a federated optimization algorithm based on optimal transmission; If the target fine-tuning algorithm is a federated optimization algorithm based on optimal transmission, then obtain the compression simulator and the initial adapter, and generate an initialization configuration based on the compression simulator and the initial adapter; If the target fine-tuning algorithm is not a federated optimization algorithm based on optimal transmission, then obtain the initial learnable parameters and generate the initial configuration based on the initial learnable parameters; The parameters of the preset large language model are initialized using the initialization configuration to obtain the initial language model; The target subset of data is segmented and encoded to obtain multiple target segmentation codes; Generate real data tags based on the target word segmentation encoding; Obtain the predicted label of the target word segmentation code, and use a preset loss function to determine the loss value between the predicted label and the real data label; Obtain acceleration operators and resource-efficient operators, and use the loss value, acceleration operators, and resource-efficient operators to adjust the initial language model to obtain an adjusted language model; The global model parameters are obtained by taking a weighted average of the model parameters of the adjusted language model for all the target clients. The language model is fine-tuned using the global model parameters to obtain a preliminary fine-tuned model.

[0088] In one embodiment, when the large language model optimization module 105 performs resource consumption analysis during the adjustment process of the large language model and obtains the resource consumption information, it is used to: Obtain the task type required for the fine-tuning task, and generate task analysis parameters based on the task type; The parameter values ​​of the task analysis parameters are determined based on the adjustment process of the large language model. The fine-tuning effect of the large language model is analyzed based on the parameter values ​​to obtain the fine-tuning effect value; The resource consumption during the adjustment process of the large language model is recorded in real time, and the real-time recorded data is visualized to obtain a resource consumption graph. The fine-tuning effect values ​​are mapped to the resource consumption graph to obtain the process parameter graph; Resource consumption information is generated based on the process parameter diagram.

[0089] In one embodiment, when the large language model optimization module 105 performs performance optimization on the initially adjusted model based on the resource consumption to obtain the target large language model, it is used to: Based on the resource consumption situation, generate parameters and modules to be optimized; The parameters to be optimized are adjusted according to the process parameter diagram to obtain the adjusted parameters; The adjustment parameters are compressed to obtain compressed parameters; Determine the contribution parameter value of the module to be optimized, and determine whether the contribution parameter value is greater than a preset contribution threshold; If the contribution parameter value is less than or equal to the contribution threshold, then the modules to be optimized corresponding to contribution parameter values ​​that are less than or equal to the contribution threshold are pruned. By reducing the number of adapter parameters in the remaining modules to be optimized, the adjusted modules are obtained. If the contribution parameter value is greater than the contribution threshold, then the number of adapter parameters in the module to be optimized is reduced to obtain the adjustment module; The initial adjustment model is optimized using the compression parameters and the adjustment module to obtain the target large language model.

[0090] In this invention, a text intent analysis device is first developed by acquiring raw data from the target domain and fine-tuning task requirements. Based on these requirements, a segmentation strategy is selected for the raw data to obtain a target segmentation strategy. This adaptive strategy selection not only improves the representativeness and diversity of the fine-tuned data but also optimizes the model training effect and generalization ability, contributing to the construction of a more robust and efficient large language model. The raw data is then segmented according to the target segmentation strategy to obtain multiple target subsets. Client resource status is acquired, and multiple acquired efficient federated parameter fine-tuning algorithms are screened based on the client resource status and the fine-tuning task requirements to obtain a target fine-tuning algorithm. The screening process flexibly matches the most suitable efficient federated parameter fine-tuning algorithm based on the client's model access permissions, resource conditions, and task requirements, exhibiting high adaptability and practical usability. Then, the target subsets and the target fine-tuning algorithm are used to adjust a preset large language model to obtain a preliminary adjusted model. An initialization strategy is flexibly configured according to the target fine-tuning algorithm type. To effectively improve the quality of model initialization, the word segmentation and encoding steps ensure deep matching between the data structure and the model input, thereby improving training efficiency. Through collaborative optimization of acceleration operators and resource-efficient operators, the model can still achieve superior performance under limited computing resources. Resource consumption analysis is performed on the adjustment process of the large language model to obtain resource consumption information. Based on the resource consumption information, the performance of the initially adjusted model is optimized to obtain the target large language model. By combining the resource consumption analysis results and process parameter graphs, the parameters and modules to be optimized in the large language model are adjusted in a targeted manner. Techniques such as compression, pruning, and adapter simplification are used to dynamically reduce the resource consumption of inefficient or redundant parts, while retaining the core modules that contribute significantly to performance. This significantly improves the model's running efficiency and deployment flexibility while ensuring model accuracy. Target text data in the target domain is obtained, and the target large language model is used to perform text language analysis on the target text data to determine the target text intent. This can significantly improve the system's ability to understand user needs and achieve high-precision and high-robust intent recognition. Compared to traditional methods, the approach of fine-tuning a large language model possesses stronger semantic understanding and contextual modeling capabilities, can adapt to complex and diverse language expressions, and can effectively improve the stability of large language model analysis performance and the interpretability of text intent. Specific limitations regarding a text intent analysis device can be found in the limitations of a text intent analysis method described above, and will not be repeated here. The various modules in the aforementioned text intent analysis device can be implemented entirely or partially through software, hardware, or a combination thereof. These modules can be embedded in or independent of the processor in a computer device in hardware form, or stored in the memory of a computer device in software form, so that the processor can call and execute the operations corresponding to each module.

[0091] In one embodiment, a computer device is provided, which may be a server, and its internal structure diagram may be as follows: Figure 5 As shown, the computer device includes a processor, memory, network interface, and database connected via a system bus. The processor provides computing and control capabilities. The memory includes non-volatile and / or volatile storage media and internal memory. The non-volatile storage media stores the operating system, computer programs, and database. The internal memory provides an environment for the operation of the operating system and computer programs stored in the non-volatile storage media. The network interface is used to communicate with external clients via a network connection. When executed by the processor, the computer program implements the functions or steps of a text intent analysis method on the server side.

[0092] In one embodiment, a computer device is provided, which may be a client, and its internal structure diagram may be as follows: Figure 6 As shown, the computer device includes a processor, memory, network interface, display screen, and input devices connected via a system bus. The processor provides computing and control capabilities. The memory includes non-volatile storage media and internal memory. The non-volatile storage media stores the operating system and computer programs. The internal memory provides an environment for the operation of the operating system and computer programs stored in the non-volatile storage media. The network interface is used to communicate with an external server via a network connection. When executed by the processor, the computer program implements the functions or steps of a text intent analysis method on the client side.

[0093] In one embodiment, a computer device is provided, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to perform the following steps: Obtain the original data and fine-tuning task requirements of the target domain, and select a partitioning strategy for the original data according to the fine-tuning task requirements to obtain the target partitioning strategy; The original data is divided according to the target partitioning strategy to obtain multiple target subsets; Obtain the client resource status, and based on the client resource status and the fine-tuning task requirements, filter multiple obtained federated parameters for efficient fine-tuning algorithms to obtain the target fine-tuning algorithm; The target subset dataset and the target fine-tuning algorithm are used to adjust the preset large language model to obtain a preliminary adjusted model; Resource consumption analysis is performed on the adjustment process of the large language model to obtain the resource consumption situation, and the performance of the preliminary adjusted model is optimized based on the resource consumption situation to obtain the target large language model; Obtain target text data in the target domain, and use the target large language model to perform text language analysis on the target text data to determine the target text intent.

[0094] In the several embodiments provided by this invention, it should be understood that the disclosed devices and apparatuses can be implemented in other ways. For example, the system embodiments described above are merely illustrative; for instance, the division of modules is only a logical functional division, and other division methods may be used in actual implementation.

[0095] Furthermore, the functional modules in the various embodiments of the present invention can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or in the form of hardware plus software functional modules.

[0096] Therefore, the embodiments should be considered exemplary and non-limiting in all respects, and the scope of the invention is defined by the appended claims rather than the foregoing description. Thus, all variations falling within the meaning and scope of equivalents of the claims are intended to be embraced within the invention. No appended diagram markings in the claims should be construed as limiting the scope of the claims.

[0097] It will be apparent to those skilled in the art that the present invention is not limited to the details of the exemplary embodiments described above, and that the present invention can be implemented in other specific forms without departing from the spirit or essential characteristics of the present invention.

[0098] In some embodiments of this example, a computer-readable storage medium is provided, on which a computer program is stored, characterized in that the computer program, when executed by a processor, implements the steps of the method described in the above embodiments.

[0099] The readable storage medium of the present invention stores a computer program, which, when executed by a processor of an electronic device, can perform the following: Obtain the original data and fine-tuning task requirements of the target domain, and select a partitioning strategy for the original data according to the fine-tuning task requirements to obtain the target partitioning strategy; The original data is divided according to the target partitioning strategy to obtain multiple target subsets; Obtain the client resource status, and based on the client resource status and the fine-tuning task requirements, filter multiple obtained federated parameters for efficient fine-tuning algorithms to obtain the target fine-tuning algorithm; The target subset dataset and the target fine-tuning algorithm are used to adjust the preset large language model to obtain a preliminary adjusted model; Resource consumption analysis is performed on the adjustment process of the large language model to obtain the resource consumption situation, and the performance of the preliminary adjusted model is optimized based on the resource consumption situation to obtain the target large language model; Obtain target text data in the target domain, and use the target large language model to perform text language analysis on the target text data to determine the target text intent.

[0100] It should be noted that the functions or steps that can be implemented by the computer-readable storage medium or computer device described above can be referred to the relevant descriptions on the server side and client side in the foregoing method embodiments. To avoid repetition, they will not be described one by one here.

[0101] Computer-readable storage media may also store at least one computer-executable program / instruction, such as computer-readable instructions. Computer-readable storage media include, but are not limited to, volatile memory and / or non-volatile memory. Volatile memory may include, for example, random access memory (RAM) and / or cache memory. Computer-readable storage media may include, for example, read-only memory (ROM), hard disk, flash memory, etc. For example, a non-transitory computer-readable storage medium may be connected to a computing device such as a computer, and then, when the computing device executes the computer-readable instructions stored on the computer-readable storage medium, the various methods described above can be performed.

[0102] In addition, the computer device may include (but is not limited to) a data bus, an input / output (I / O) bus, a display, and input / output devices (e.g., keyboard, mouse, speakers, etc.).

[0103] The processor can communicate with external devices via the I / O bus through wired or wireless networks.

[0104] In one embodiment, the at least one computer-executable instruction may also be compiled into or comprise a software product / computer program product, wherein one or more computer-executable instructions are executed by a processor to perform the steps of the various functions and / or methods in the embodiments described herein.

[0105] Those skilled in the art will understand that all or part of the processes in the methods of the above embodiments can be implemented by a computer program instructing related hardware. The computer program can be stored in a non-volatile computer-readable storage medium. When executed, the computer program can include the processes of the embodiments of the above methods. Any references to memory, storage, databases, or other media used in the embodiments provided in this application can include non-volatile and / or volatile memory. Non-volatile memory may include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM), or flash memory. Volatile memory may include random access memory (RAM) or external cache memory. By way of illustration and not limitation, RAM is available in a variety of forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), dual data rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), synchronous link DRAM (SLDRAM), RAMbus direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM), etc.

[0106] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the above-described division of functional units and modules is used as an example. In practical applications, the above functions can be assigned to different functional units and modules as needed, that is, the internal structure of the device can be divided into different functional units or modules to complete all or part of the functions described above.

[0107] In the embodiments provided in this disclosure, it should be understood that the disclosed apparatus and methods can also be implemented in other ways. The apparatus embodiments described above are merely illustrative; for example, the flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of apparatus, methods, and computer program products according to various embodiments of this disclosure. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of code containing one or more executable instructions for implementing a specified logical function. It should also be noted that in some alternative implementations, the functions marked in the blocks may occur in a different order than those marked in the drawings. For example, two consecutive blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in a block diagram and / or flowchart, and combinations of blocks in block diagrams and / or flowcharts, can be implemented using a dedicated hardware-based system that performs the specified function or action, or using a combination of dedicated hardware and computer instructions.

[0108] It should be noted that, in this disclosure, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitation, an element limited by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes that element.

[0109] The above-described embodiments are only used to illustrate the technical solutions of the present invention, and are not intended to limit it. Although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention, and should all be included within the protection scope of the present invention.

[0110] It should be noted that if any software tools or components not belonging to our company appear in the embodiments of this application, they are merely illustrative examples and do not represent actual use.

Claims

1. A text intent analysis method, characterized in that, The method includes: Obtain the original data and fine-tuning task requirements of the target domain, and select a partitioning strategy for the original data according to the fine-tuning task requirements to obtain the target partitioning strategy; The original data is divided according to the target partitioning strategy to obtain multiple target subsets; Obtain the client resource status, and based on the client resource status and the fine-tuning task requirements, filter multiple obtained federated parameters for efficient fine-tuning algorithms to obtain the target fine-tuning algorithm; The target subset dataset and the target fine-tuning algorithm are used to adjust the preset large language model to obtain a preliminary adjusted model; Resource consumption analysis is performed on the adjustment process of the large language model to obtain the resource consumption situation, and the performance of the preliminary adjusted model is optimized based on the resource consumption situation to obtain the target large language model; Obtain target text data in the target domain, and use the target large language model to perform text language analysis on the target text data to determine the target text intent.

2. The text intent analysis method as described in claim 1, characterized in that, The step of selecting a partitioning strategy for the original data based on the fine-tuning task requirements to obtain the target partitioning strategy includes: The original data is deduplicated to obtain deduplicated data; Identify irrelevant data in the deduplicated data that is unrelated to the target domain, and filter the irrelevant data in the deduplicated data to obtain filtered data; The filtered data is labeled according to the fine-tuning task requirements to obtain labeled data; Determine the distribution difference value of the labeled data, and determine whether the distribution difference value is greater than a preset difference threshold; If the distribution difference value is greater than the difference threshold, then the Dirichlet distribution partitioning strategy is used as the target partitioning strategy. If the distribution difference value is less than or equal to the difference threshold, then the uniform partitioning strategy will be used as the target partitioning strategy.

3. The text intent analysis method as described in claim 2, characterized in that, The original data is divided according to the target partitioning strategy to obtain multiple target subsets, including: When the target partitioning strategy is a uniform partitioning strategy, the partitioning ratio is obtained according to the uniform partitioning strategy; The labeled data is divided according to the specified division ratio to obtain multiple target subsets; When the target partitioning strategy is a Dirichlet distribution partitioning strategy, the category labels and partitioning heterogeneity parameters of the labeled data are obtained; The allocation ratio of each category label is randomly generated according to the Dirichlet distribution partitioning strategy; The labeled data is divided according to the allocation ratio and the partitioning heterogeneity parameter to obtain multiple target subsets.

4. The text intent analysis method as described in claim 1, characterized in that, The step of filtering multiple efficient fine-tuning algorithms for acquired federated parameters based on the client resource status and the fine-tuning task requirements to obtain the target fine-tuning algorithm includes: Obtain the model access status from the client and determine whether the model access status is accessible or inaccessible. If the model access status is accessible, then when the client resource status is resource-scarce, the hint tuning algorithm in the efficient fine-tuning algorithm of the obtained federated parameters will be used as the target fine-tuning algorithm. When the client resource status is sufficient, determine whether the fine-tuning task requirement is a fine-tuning task or a text coherence optimization task. If the fine-tuning task requires precise adjustment, then the low-rank adaptive algorithm in the efficient fine-tuning algorithm for federated parameters will be used as the target fine-tuning algorithm. If the fine-tuning task requirement is a text coherence optimization task, then the efficient fine-tuning algorithm based on federated parameters of prompt words will be used as the target fine-tuning algorithm. If the model access status is inaccessible, then the federated optimization algorithm based on optimal transport theory in the federated parameter efficient fine-tuning algorithm will be used as the target fine-tuning algorithm.

5. The text intent analysis method as described in claim 4, characterized in that, The process of adjusting the preset large language model using the target subset and the target fine-tuning algorithm to obtain a preliminary adjusted model includes: Determine whether the target fine-tuning algorithm is a federated optimization algorithm based on optimal transmission; If the target fine-tuning algorithm is a federated optimization algorithm based on optimal transmission, then obtain the compression simulator and the initial adapter, and generate an initialization configuration based on the compression simulator and the initial adapter; If the target fine-tuning algorithm is not a federated optimization algorithm based on optimal transmission, then obtain the initial learnable parameters and generate the initial configuration based on the initial learnable parameters; The parameters of the preset large language model are initialized using the initialization configuration to obtain the initial language model; The target subset of data is segmented and encoded to obtain multiple target segmentation codes; Generate real data tags based on the target word segmentation encoding; Obtain the predicted label of the target word segmentation code, and use a preset loss function to determine the loss value between the predicted label and the real data label; Obtain acceleration operators and resource-efficient operators, and use the loss value, acceleration operators, and resource-efficient operators to adjust the initial language model to obtain an adjusted language model; The model parameters of the adjusted language model are weighted and averaged across all clients to obtain the global model parameters; The language model is fine-tuned using the global model parameters to obtain a preliminary fine-tuned model.

6. The text intent analysis method as described in claim 5, characterized in that, The resource consumption analysis of the adjustment process of the large language model, to obtain the resource consumption situation, includes: Obtain the task type required for the fine-tuning task, and generate task analysis parameters based on the task type; The parameter values ​​of the task analysis parameters are determined based on the adjustment process of the large language model. The fine-tuning effect of the large language model is analyzed based on the parameter values ​​to obtain the fine-tuning effect value; The resource consumption during the adjustment process of the large language model is recorded in real time, and the real-time recorded data is visualized to obtain a resource consumption graph. The fine-tuning effect values ​​are mapped to the resource consumption graph to obtain the process parameter graph; Resource consumption information is generated based on the process parameter diagram.

7. The text intent analysis method as described in claim 6, characterized in that, The step of optimizing the performance of the preliminary adjustment model based on the resource consumption to obtain the target large language model includes: Based on the resource consumption situation, generate parameters and modules to be optimized; The parameters to be optimized are adjusted according to the process parameter diagram to obtain the adjusted parameters; The adjustment parameters are compressed to obtain compressed parameters; Determine the contribution parameter value of the module to be optimized, and determine whether the contribution parameter value is greater than a preset contribution threshold; If the contribution parameter value is less than or equal to the contribution threshold, then the modules to be optimized corresponding to contribution parameter values ​​that are less than or equal to the contribution threshold are pruned. By reducing the number of adapter parameters in the remaining modules to be optimized, the adjusted modules are obtained. If the contribution parameter value is greater than the contribution threshold, then the number of adapter parameters in the module to be optimized is reduced to obtain the adjustment module; The initial adjustment model is optimized using the compression parameters and the adjustment module to obtain the target large language model.

8. A text intent analysis device, characterized in that, The device includes: The partitioning strategy selection module is used to acquire the original data of the target domain and the fine-tuning task requirements, and select a partitioning strategy for the original data according to the fine-tuning task requirements to obtain the target partitioning strategy. The original data partitioning module is used to partition the original data according to the target partitioning strategy to obtain multiple target subsets. The fine-tuning algorithm filtering module is used to obtain the client resource status and filter multiple efficient fine-tuning algorithms based on the client resource status and the fine-tuning task requirements to obtain the target fine-tuning algorithm. The large language model adjustment module is used to adjust the preset large language model using the target subset and the target fine-tuning algorithm to obtain a preliminary adjusted model; The large language model optimization module is used to analyze the resource consumption of the adjustment process of the large language model, obtain the resource consumption situation, and optimize the performance of the initially adjusted model based on the resource consumption situation to obtain the target large language model. The text intent analysis module is used to acquire target text data in the target domain, perform text language analysis on the target text data using the target large language model, and determine the target text intent.

9. An electronic device, characterized in that, The electronic device includes: At least one processor; and, A memory communicatively connected to the at least one processor; wherein, The memory stores a computer program that can be executed by the at least one processor, the computer program being executed by the at least one processor to enable the at least one processor to perform a text intent analysis method as described in any one of claims 1 to 7.

10. A computer-readable storage medium storing a computer program, characterized in that, When the computer program is executed by the processor, it implements a text intent analysis method as described in any one of claims 1 to 7.

Images