Recommended verbal skill generation method and device, computer equipment and storage medium

Abstract

The invention discloses a recommended verbal skill generation method and device, computer equipment and a storage medium, and belongs to the technical field of artificial intelligence. According to the method, the pre-training model comprising the adapter layer is constructed, flexible application of a general language model and a specific scene strategy is effectively combined, the adapter layer is used as a bridge for connecting the network layer of the pre-training model, and rapid training of the specific scene model is realized; first, a general language model is trained by using first training data, and then iterative training is performed by using second training data for a specific scene to obtain a verbal skill recommendation model. The invention further relates to the technical field of block chains, and training data is stored on a block chain network. According to the method, the pre-training model structure is adopted, the universal language model is firstly trained, then the corresponding verbal skill recommendation models can be respectively trained for various specific scenes, and verbal skill recommendation can be accurately and efficiently generated.

Description

Technical Field

[0001] The present application belongs to the field of artificial intelligence technology, and specifically relates to a method, device, computer equipment and storage medium for generating recommendation words. Background Art

[0002] Auto insurance is an insurance product that protects the safety of vehicles and drivers, and is one of the essential services for car consumers. However, there are some problems in the current auto insurance market, such as the wide variety of auto insurance products, making it difficult for consumers to choose a suitable protection plan; auto insurance promoters lack professional knowledge and communication skills, making it difficult to provide effective consulting and guidance services; auto insurance marketing techniques lack innovation and personalization, making it difficult to attract and retain consumers' attention and interest, etc.

[0003] In response to the above problems, the traditional method of converting auto insurance is mainly through communication between salesmen and customers via telephone platforms. Since each seat needs to serve a large number of customers and the professional levels of different seats vary, manual methods require a lot of manpower, material and financial resources, and the conversion efficiency is not high. Summary of the invention

[0004] The purpose of the embodiments of the present application is to propose a method, device, computer equipment and storage medium for generating recommendation words, so as to solve the technical problems that the existing manual conversion of auto insurance requires a lot of manpower, material and financial resources, and the conversion efficiency is low.

[0005] In order to solve the above technical problems, the present application embodiment provides a method for generating recommendation words, which adopts the following technical solution:

[0006] A method for generating recommendation words, the method comprising:

[0007] Building a pre-trained model, wherein the pre-trained model includes several network layers;

[0008] Build an adapter layer between the network layers of the pre-trained model, where the adapter layer is built based on a small neural network;

[0009] Determining first training data according to a first training task, wherein the training task is a training task of a general scenario strategy;

[0010] Initializing model parameters of the pre-trained model, and performing model training on the initialized pre-trained model using the first training data to obtain a universal language model;

[0011] Determining second training data according to a second training task, wherein the training task is a training task for a specific scenario strategy;

[0012] Iteratively train the general language model using the second training data to obtain a speech recommendation model;

[0013] Receive instructions for generating auto insurance sales pitches, use the sales pitch recommendation model to generate auto insurance recommendation sales pitches, and output the auto insurance recommendation sales pitches.

[0014] Furthermore, after the step of constructing an adapter layer between the network layers of the pre-trained model, it also includes:

[0015] Obtaining network parameters of each network layer in the pre-trained model, wherein the network parameters are preset weights and biases of each network layer in the pre-trained model;

[0016] Construct a network parameter matrix based on the network parameters of each network layer to obtain a first weight matrix;

[0017] Obtaining adapter parameters of each adapter layer, wherein the adapter parameters are preset weights and biases of each adapter layer;

[0018] An adapter parameter matrix is ​​constructed based on adapter parameters of each adapter layer to obtain a second weight matrix.

[0019] Furthermore, the step of constructing a network parameter matrix based on the network parameters of each network layer to obtain a first parameter matrix specifically includes:

[0020] Mapping the network parameters of each network layer in the pre-trained model to a preset one-dimensional space to obtain a first parameter vector;

[0021] Combining all first parameter vectors in the one-dimensional space to obtain a first weight matrix;

[0022] The step of constructing an adapter parameter matrix based on adapter parameters of each adapter layer to obtain a second weight matrix specifically includes:

[0023] Mapping adapter parameters of each adapter layer to a preset one-dimensional space to obtain a second parameter vector;

[0024] All second parameter vectors in the one-dimensional space are combined to obtain a second weight matrix.

[0025] Furthermore, the step of determining the first training data according to the first training task specifically includes:

[0026] Obtain pre-training data, where the pre-training data is the dialogue text of historical auto insurance recommendations;

[0027] Identify features related to general scenario strategies in pre-training data and obtain general scenario-related features;

[0028] Identifying common scene-related features in the pre-training data to obtain first training data;

[0029] The step of determining the second training data according to the second training task specifically includes:

[0030] Identify features related to specific scenario strategies in pre-training data and obtain features related to specific scenarios;

[0031] The specific scene-related features in the pre-training data are identified to obtain the second training data.

[0032] Furthermore, the model parameters include a first weight matrix and a second weight matrix, the model parameters of the pre-trained model are initialized, and the initialized pre-trained model is trained using the first training data to obtain a universal language model, specifically including:

[0033] Initializing a first weight matrix and a second weight matrix;

[0034] Importing the first training data into the pre-trained model so that the pre-trained model learns common scene-related features;

[0035] In the process of learning general scene-related features by the pre-training model, the initialized first weight matrix and the second weight matrix are updated until the model is fitted to obtain a general language model.

[0036] Further, the step of using the second training data to iteratively train the general language model to obtain a speech recommendation model specifically includes:

[0037] Importing the second training data into the general language model so that the general language model learns the relevant features of the specific scenario;

[0038] In the process of the general language model learning the relevant features of a specific scenario, the first weight matrix and the second weight matrix of the general language model are updated until the model is fitted to obtain a speech recommendation model.

[0039] Furthermore, low-rank decomposition is used to control the updating of the first weight matrix and the second weight matrix. In the process of learning general scene-related features by the pre-training model, the steps of updating the initialized first weight matrix and the second weight matrix specifically include:

[0040] In the process of learning general scene-related features by the pre-training model, a first intrinsic rank is configured for the first weight matrix, and a second intrinsic rank is configured for the second weight matrix, wherein the second intrinsic rank is smaller than the first intrinsic rank;

[0041] The update parameters of the initialized first weight matrix and the second weight matrix are controlled by the first intrinsic rank and the second intrinsic rank.

[0042] In the process of the general language model learning the relevant features of a specific scenario, the step of updating the first weight matrix and the second weight matrix of the general language model specifically includes:

[0043] In the process of learning the scene-specific related features by the general language model, a third intrinsic rank is configured for the first weight matrix, and a fourth intrinsic rank is configured for the second weight matrix, wherein the fourth intrinsic rank is greater than the third intrinsic rank;

[0044] The update parameters of the first weight matrix and the second weight matrix of the universal language model are controlled by the third intrinsic rank and the fourth intrinsic rank.

[0045] In order to solve the above technical problems, the embodiment of the present application further provides a device for generating recommendation words, which adopts the following technical solution:

[0046] A device for generating recommendation words, comprising:

[0047] A model building module, used to build a pre-trained model, wherein the pre-trained model includes several network layers;

[0048] The adapter building module is used to build an adapter layer between the network layers of the pre-trained model, where the adapter layer is built based on a small neural network;

[0049] A first data module, used to determine first training data according to a first training task, wherein the training task is a training task of a general scenario strategy;

[0050] A first training module is used to initialize model parameters of a pre-trained model, and perform model training on the initialized pre-trained model using first training data to obtain a universal language model;

[0051] A second data module, used to determine second training data according to a second training task, wherein the training task is a training task of a specific scenario strategy;

[0052] A second training module is used to perform iterative model training on the general language model using the second training data to obtain a speech recommendation model;

[0053] The speech recommendation module is used to receive instructions for generating auto insurance speech, use the speech recommendation model to generate auto insurance recommendation speech, and output the auto insurance recommendation speech.

[0054] In order to solve the above technical problems, the embodiment of the present application further provides a computer device, which adopts the following technical solution:

[0055] A computer device includes a memory and a processor, wherein the memory stores computer-readable instructions, and when the processor executes the computer-readable instructions, the steps of the method for generating recommendation words as described in any one of the above items are implemented.

[0056] In order to solve the above technical problems, the embodiment of the present application further provides a computer-readable storage medium, which adopts the following technical solution:

[0057] A computer-readable storage medium having computer-readable instructions stored thereon, wherein the computer-readable instructions, when executed by a processor, implement the steps of the method for generating recommendation words as described in any one of the above.

[0058] Compared with the prior art, the embodiments of the present application have the following beneficial effects:

[0059] The present application discloses a method, device, computer equipment and storage medium for generating recommended speech, which belongs to the field of artificial intelligence technology. The present application constructs a pre-trained model including an adapter layer, effectively combines the flexible application of general language models and specific scenario strategies, uses the adapter layer as a bridge to connect the pre-trained model network layer, and uses a small neural network to achieve rapid training of specific scenario models, while ensuring model performance, reducing training costs. The present application uses two stages of training, first using the first training data to train a general language model, and then using the second training data for iterative training for specific scenarios, and finally obtains a speech recommendation model. The present application adopts the above-mentioned pre-trained model structure, by first training a general language model, and then training the corresponding speech recommendation models for a variety of specific scenarios, respectively, to accurately and efficiently generate speech recommendations. BRIEF DESCRIPTION OF THE DRAWINGS

[0060] In order to more clearly illustrate the scheme in the present application, a brief introduction is given below to the drawings required for use in the description of the embodiments of the present application. Obviously, the drawings described below are some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0061] Figure 1 An exemplary system architecture diagram is shown in which the present application can be applied;

[0062] Figure 2 A flowchart of an embodiment of a method for generating recommendation words according to the present application is shown;

[0063] Figure 3 A schematic diagram of the structure of an embodiment of a device for generating recommendation words according to the present application is shown;

[0064] Figure 4A schematic structural diagram of an embodiment of a computer device according to the present application is shown. DETAILED DESCRIPTION

[0065] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as those commonly understood by technicians in the technical field of the present application; the terms used in the specification of the application herein are only for the purpose of describing specific embodiments and are not intended to limit the present application; the terms "including" and "having" and any variations thereof in the specification and claims of the present application and the above-mentioned drawings are intended to cover non-exclusive inclusions. The terms "first", "second", etc. in the specification and claims of the present application or the above-mentioned drawings are used to distinguish different objects, not to describe a specific order.

[0066] Reference to "embodiments" herein means that a particular feature, structure, or characteristic described in conjunction with the embodiments may be included in at least one embodiment of the present application. The appearance of the phrase in various locations in the specification does not necessarily refer to the same embodiment, nor is it an independent or alternative embodiment that is mutually exclusive with other embodiments. It is explicitly and implicitly understood by those skilled in the art that the embodiments described herein may be combined with other embodiments.

[0067] In order to enable those skilled in the art to better understand the solution of the present application, the technical solution in the embodiments of the present application will be clearly and completely described below in conjunction with the accompanying drawings.

[0068] like Figure 1 As shown, the system architecture 100 may include terminal devices 101, 102, 103, a network 104 and a server 105. The network 104 is used to provide a medium for communication links between the terminal devices 101, 102, 103 and the server 105. The network 104 may include various connection types, such as wired, wireless communication links or optical fiber cables, etc.

[0069] Users can use terminal devices 101, 102, 103 to interact with server 105 through network 104 to receive or send messages, etc. Various communication client applications can be installed on terminal devices 101, 102, 103, such as web browser applications, shopping applications, search applications, instant messaging tools, email clients, social platform software, etc.

[0070] Terminal devices 101, 102, 103 can be various electronic devices with display screens and supporting web browsing, including but not limited to smart phones, tablet computers, e-book readers, MP3 players (Moving Picture Experts Group Audio Layer III), MP4 (Moving Picture Experts Group Audio Layer IV) players, laptop computers and desktop computers, etc.

[0071] Server 105 can be a server that provides various services, such as a background server that provides support for pages displayed on terminal devices 101, 102, and 103. The server can be an independent server or a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, content distribution networks (CDNs), and big data and artificial intelligence platforms.

[0072] It should be noted that the recommendation speech generation method provided in the embodiment of the present application is generally executed by a server, and accordingly, the recommendation speech generation device is generally set in the server.

[0073] It should be understood that Figure 1 The number of terminal devices, networks and servers in the embodiment is only for illustration. Any number of terminal devices, networks and servers may be provided according to implementation requirements.

[0074] Continue to refer Figure 2 , showing a flow chart of an embodiment of the method for generating recommendation words according to the present application. The present application embodiment can acquire and process relevant data based on artificial intelligence technology. Among them, artificial intelligence (AI) is the theory, method, technology and application system that uses digital computers or machines controlled by digital computers to simulate, extend and expand human intelligence, perceive the environment, acquire knowledge and use knowledge to obtain the best results.

[0075] The basic technologies of artificial intelligence generally include sensors, dedicated artificial intelligence chips, cloud computing, distributed storage, big data processing technology, operation / interaction systems, mechatronics and other technologies. Artificial intelligence software technology mainly includes computer vision technology, robotics technology, biometrics technology, speech processing technology, natural language processing technology, and machine learning / deep learning. The method for generating recommendation words includes the following steps:

[0076] S201, constructing a pre-training model, wherein the pre-training model includes several network layers.

[0077] Specifically, pre-trained models are usually built in advance based on deep learning and natural language processing (NLP) technologies. Pre-trained models learn common language features and patterns through preliminary training on large-scale text data. Pre-trained models usually include multiple neural network layers, such as the encoder and decoder layers in the Transformer architecture, which are interconnected through self-attention mechanisms (Self-Attention) and feedforward neural networks. The core goal of the pre-trained model is to capture the grammar, semantics, and contextual relationships of the language through language modeling tasks, such as predicting masked words or generating the next word. Common pre-trained models include Transformer models, BERT models, GPT models, and so on. By building pre-trained models, it is possible to demonstrate strong versatility and adaptability in a wide range of language tasks.

[0078] S202, constructing an adapter layer between network layers of the pre-trained model, wherein the adapter layer is constructed based on a small neural network.

[0079] Specifically, adapter layers are small neural network modules inserted between the network layers of the pre-trained model for fine-tuning on specific tasks. Adapter layers usually include a dimensionality reduction layer, a nonlinear activation function (such as ReLU), and a dimensionality increase layer, which capture the characteristics of specific tasks by adjusting the dimension of the input vector without changing the main parameters of the pre-trained model. This adapter layer structure enables the adapter layer to learn and adjust the characteristics of specific tasks while maintaining the general knowledge of the pre-trained model. The advantage of using adapter layers is parameter efficiency and flexibility, allowing the model to switch quickly between different tasks, significantly reducing computational overhead and storage requirements.

[0080] S203: Determine first training data according to a first training task, wherein the training task is a training task of a general scenario strategy.

[0081] Specifically, the first training data is determined based on the requirements of the initial training task, and an appropriate data set is selected and prepared for the first model training of the pre-training model. The purpose of the first model training is to obtain a general language model so that it can be fine-tuned based on the general language model according to specific scenarios to obtain a speech recommendation model for specific scenarios.

[0082] The first training data is usually large-scale general text data, covering a wide range of conversation texts and languages ​​to ensure that the model can learn rich language features and patterns. When selecting the first training data, you need to consider the diversity and coverage of the data so that the model can perform well in different situations. When obtaining the first training data, data preprocessing steps including cleaning, noise removal, word segmentation, and normalization are required to ensure data quality and consistency. By training on the first training data, the model can master the basic structure and general knowledge of the language, laying the foundation for subsequent fine-tuning of specific tasks.

[0083] In a specific embodiment of the present application, large-scale general text data is selected as the first training data, such as actual customer service records, FAQs, etc. Large-scale data means that the model can be exposed to more language phenomena and patterns, so as to understand and learn the language more accurately. These data have a wide range of application backgrounds and rich language phenomena, and can provide a variety of training samples for the model. Selecting general text data as training data can ensure that the model can perform well in different fields and situations. For example, in a customer service scenario, a customer's questions may involve multiple fields and topics, with different expressions and emotional tendencies. Only with diverse training data can the model be ensured to accurately understand and respond to these problems. Customers' questions may involve multiple products and services, with different usage scenarios and backgrounds. Only with training data with a wide coverage can the model be ensured to provide accurate answers and services in different situations.

[0084] S204, initializing model parameters of the pre-trained model, and performing model training on the initialized pre-trained model using the first training data to obtain a universal language model.

[0085] Specifically, initializing the pre-trained model involves setting the initial parameters of the model, which are usually obtained through a random initialization method. Subsequently, the pre-trained model is trained using the first training data to obtain a general language model. The training process includes forward propagation and back propagation. Forward propagation calculates the predicted output of the model, and back propagation adjusts the model parameters by calculating the gradient of the loss function to make the model perform better on the training data.

[0086] During the training process, the model is continuously iterated, and the loss function is minimized through optimization algorithms (such as gradient descent), thereby gradually improving the model's language understanding and generation capabilities. The final pre-trained model has strong general language features and context understanding capabilities, and can provide basic support for various natural language processing tasks.

[0087] In this embodiment, low-rank decomposition (LoRA) can be used to control the update of the pre-trained model parameters. Low-rank decomposition is a concept in mathematics or data processing, which generally refers to the decomposition of a high-rank matrix or tensor into a combination of multiple low-rank matrices or tensors. It can be achieved through various optimization algorithms and has a wide range of applications in multiple fields. In the recommendation system, low-rank decomposition can be used to represent the relationship between users and items as the product of low-rank matrices, thereby achieving personalized recommendations. For example, when the pre-trained model is trained using the first training data, a higher intrinsic rank score can be assigned to the network layer of the pre-trained model, while a lower intrinsic rank score can be assigned to the adapter layer, so that the network layer of the pre-trained model can fully learn the training tasks of the general scenario strategy during model training, while minimizing the impact of the training tasks of the general scenario strategy on the adapter layer.

[0088] S205: Determine second training data according to a second training task, wherein the training task is a training task for a specific scenario strategy.

[0089] Specifically, the second training data is determined based on the needs of a specific task, and a special data set is selected and prepared for fine-tuning the general language model. By identifying and collecting data that is highly relevant to a specific application scenario, namely the second training data, to ensure that the model can perform well on the task. When selecting the second training data, the focus is on its direct relevance and representativeness to the specific task. The data processing steps include cleaning, labeling, and formatting to ensure data quality and consistency. By training on the second training data, the model can be further improved from general language capabilities to the professional level of specific tasks, optimizing its performance in specific scenarios.

[0090] In a specific embodiment of the present application, taking the application of generating auto insurance scripts as an example, determining the second training data involves collecting and preparing a specialized data set that is highly relevant to auto insurance conversations. These data directly reflect common scenarios and customer needs in auto insurance business, which helps the model learn to generate more accurate and professional auto insurance scripts during the fine-tuning stage. The fine-tuning model can generate detailed and persuasive recommendation scripts based on the specific product information and customer interaction records learned from the second training data. For example, the second training data is text data when a customer inquires about the advantages of a specific auto insurance product. The general language model is fine-tuned through the training tasks of specific scenario strategies, that is, the second model training. The purpose of the second model training is to obtain a script recommendation model for a specific scenario, such as a script recommendation model related to the introduction of the advantages of auto insurance products. The second model training ensures that the model performs well in specific scenarios, can handle complex customer inquiries and provide high-quality services, thereby improving customer experience and business efficiency.

[0091] S206: Use the second training data to perform iterative model training on the general language model to obtain a speech recommendation model.

[0092] Specifically, the iterative training of the general language model using the second training data is to further optimize the model so that it performs more accurately and effectively in specific tasks or scenarios. During the iterative training process, the general language model repeatedly performs forward propagation calculations and backpropagation adjustments on the second training data to minimize the loss function and optimize the model parameters. This process usually includes multiple training cycles (epochs), and in each cycle the model gradually improves its understanding and adaptability to specific task requirements. Through iterative training, the model is able to learn more fine-grained features and patterns, thereby improving the generation accuracy and naturalness in the task of generating auto insurance scripts.

[0093] In this embodiment, low rank decomposition (LoRA) can be used to control the update of the parameters of the universal language model. For example, when the universal language model is trained using the second training data, a lower intrinsic rank score can be assigned to the network layer of the universal language model, while a higher intrinsic rank score can be assigned to the adapter layer. This allows the adapter layer to fully learn the training tasks of specific scenario strategies during model training, while minimizing the impact of the training tasks of specific scenario strategies on the network layer of the universal language model.

[0094] S207, receiving a car insurance script generation instruction, using a script recommendation model to generate a car insurance recommendation script, and outputting the car insurance recommendation script.

[0095] Specifically, using a trained recommendation model to generate and output auto insurance recommendation words is the process of actually applying the model to a specific task. This process involves taking user input (such as auto insurance-related queries or instructions) as the input of the model, and calculating the output words generated by the model through forward propagation. When generating words, the model uses the specific scenario knowledge and language features learned during the training process to generate text that is natural, coherent, and meets user needs. This step also includes quality inspection and optimization of the generated words to ensure their accuracy, professionalism, and customer-friendliness. Finally, the generated words are output for user reference and use.

[0096] In the above embodiment, the present application discloses a method for generating recommended speech. By constructing a pre-trained model including an adapter layer, the general language model and the flexible application of specific scenario strategies are effectively combined. The adapter layer is used as a bridge to connect the pre-trained model network layer, and a small neural network is used to realize the rapid training of the specific scenario model. While ensuring the model performance, the training cost is reduced. The present application uses two stages of training. First, a general language model is trained using the first training data, and then iterative training is performed using the second training data for specific scenarios to finally obtain a speech recommendation model. The present application adopts the above-mentioned pre-trained model structure. By first training a general language model, corresponding speech recommendation models can be trained for a variety of specific scenarios, respectively, so as to accurately and efficiently generate speech recommendations.

[0097] Furthermore, after the step of constructing an adapter layer between the network layers of the pre-trained model, it also includes:

[0098] Obtaining network parameters of each network layer in the pre-trained model, wherein the network parameters are preset weights and biases of each network layer in the pre-trained model;

[0099] Construct a network parameter matrix based on the network parameters of each network layer to obtain a first weight matrix;

[0100] Obtaining adapter parameters of each adapter layer, wherein the adapter parameters are preset weights and biases of each adapter layer;

[0101] An adapter parameter matrix is ​​constructed based on adapter parameters of each adapter layer to obtain a second weight matrix.

[0102] In this embodiment, an adapter layer is constructed based on the pre-trained model, and the network parameters and adapter parameters are further processed to optimize the performance of the model on a specific task. First, adapter layers are inserted between the network layers of the pre-trained model. These adapter layers adjust the input and output through a small neural network structure without changing the core structure and parameters of the pre-trained model. The introduction of the adapter layer is intended to enhance the model's adaptability to specific task data while retaining the versatility and efficiency of the pre-trained model.

[0103] The network parameters of each network layer in the pre-trained model are obtained. These parameters include preset weights and biases. By collecting and processing these parameters, a first weight matrix is ​​formed. The adapter parameters of each adapter layer are obtained. These parameters include the weight and bias settings of the adapter layer. These parameters are integrated into the adapter parameter matrix, i.e., the second weight matrix. This matrix is ​​constructed to adjust and optimize the function of the adapter layer to ensure that the adapter can effectively convert and adjust the input data appropriately in a specific task.

[0104] In the above embodiments, this application combines the general language understanding capabilities of the pre-trained model with the task-specific optimization of the adapter layer, so that the model can perform well in various practical application scenarios. By carefully processing network parameters and adapter parameters, the solution ensures the stability and efficiency of the model in tasks such as prediction, reasoning, and generation. For example, in the scenario of generating auto insurance scripts, this technology can ensure that the model can accurately understand and generate text that meets the needs of the auto insurance field, such as explaining insurance terms, guiding the claims process, etc.

[0105] Furthermore, the step of constructing a network parameter matrix based on the network parameters of each network layer to obtain a first parameter matrix specifically includes:

[0106] Mapping the network parameters of each network layer in the pre-trained model to a preset one-dimensional space to obtain a first parameter vector;

[0107] Combining all first parameter vectors in the one-dimensional space to obtain a first weight matrix;

[0108] In this embodiment, the parameters of each network layer in the pre-trained model are mapped to a preset one-dimensional space, and these one-dimensional vectors are combined into a first weight matrix. Specifically, the network parameters (such as weights and biases) of each layer of the pre-trained model are first converted into a one-dimensional vector. This mapping process simplifies the multi-dimensional parameters into a form that is easy to process and combine. Then, all one-dimensional parameter vectors are combined into a weight matrix, which summarizes the parameter information of all layers in the pre-trained model. In this way, the first weight matrix can comprehensively and systematically represent the parameter structure of the pre-trained model, and the first weight matrix is ​​used to adjust the output of each network layer in the pre-trained model.

[0109] Furthermore, the step of constructing an adapter parameter matrix based on the adapter parameters of each adapter layer to obtain a second weight matrix specifically includes:

[0110] Mapping adapter parameters of each adapter layer to a preset one-dimensional space to obtain a second parameter vector;

[0111] All second parameter vectors in the one-dimensional space are combined to obtain a second weight matrix.

[0112] In this embodiment, an adapter parameter matrix is ​​constructed to form a second weight matrix. The specific steps include: first, mapping the parameters of each adapter layer (such as weights and biases) to a one-dimensional space and converting them into a second parameter vector. This mapping process simplifies the multi-dimensional parameters, making them easier to process and analyze. Then, all one-dimensional second parameter vectors are combined into a comprehensive matrix, namely the second weight matrix. In this way, the second weight matrix systematically brings together the parameter information of all adapter layers, which helps to optimize and adjust on specific tasks and enhance the adaptability and expressiveness of the model.

[0113] In the above embodiment, by constructing the first weight matrix, the general parameters of the pre-trained model can be effectively integrated and managed, and a general language model can be obtained after training with a general corpus, which provides a solid foundation for subsequent task-specific fine-tuning, ensuring that the model quickly adapts to new tasks and maintains high performance. After constructing the second weight matrix, parameter optimization of the adapter layer is more convenient, allowing the model to quickly adapt to the needs of specific tasks and improve the performance of specific tasks (such as auto insurance script generation). By optimizing parameter management and enhancing adaptability, the performance, stability and flexibility of the model are significantly improved, ensuring its efficient application in different tasks.

[0114] Furthermore, the step of determining the first training data according to the first training task specifically includes:

[0115] Obtain pre-training data, where the pre-training data is the dialogue text of historical auto insurance recommendations;

[0116] Identify features related to general scenario strategies in pre-training data and obtain general scenario-related features;

[0117] Common scene-related features in the pre-training data are identified to obtain first training data.

[0118] In this embodiment, first, pre-training data containing historical auto insurance recommendation dialogue texts are obtained. These dialogue texts provide rich dialogue information for common scenarios. In other embodiments of the present application, pre-training data may also use dialogue texts from other scenarios. Then, the features related to common scenario strategies in these data are identified. These features may include common dialogue patterns, typical customer questions, and standard recommendation statements. By identifying these features, the pre-training data is screened and labeled to extract representative common scenario-related features. Finally, these identified feature data are sorted and combined to form the first training data. This process ensures that the model can learn common dialogue strategies and language patterns in a wide range of auto insurance recommendation scenarios, thereby improving its adaptability and accuracy in practical applications.

[0119] Furthermore, the step of determining the second training data according to the second training task specifically includes:

[0120] Identify features related to specific scenario strategies in pre-training data and obtain features related to specific scenarios;

[0121] The specific scene-related features in the pre-training data are identified to obtain the second training data.

[0122] In this embodiment, first, the features related to specific scenario strategies in the pre-training data are identified. These features include specific customer needs, special situation handling methods, and specific business rules. By analyzing these features, data closely related to specific scenarios can be extracted, such as data related to auto insurance claims consultation, data related to specific auto insurance product introductions, and so on. Then, these specific scenario-related features are identified, separated and sorted out from the pre-training data to form the second training data. This process ensures that the model can be fine-tuned for specific scenarios, making it more accurate and professional when handling specific tasks (such as complex auto insurance claims consultation or specific auto insurance product recommendations), thereby improving the effect and customer satisfaction in actual applications.

[0123] In the above embodiment, by obtaining the general scenario-related features in the pre-training data and constructing the first training data, it is ensured that the model can learn the general dialogue strategy and language mode in a wide range of auto insurance recommendation scenarios, thereby improving its adaptability and accuracy in practical applications. By obtaining the specific scenario-related features in the pre-training data and forming the second training data, it is ensured that the model can be fine-tuned for specific scenarios, making it more accurate and professional when handling specific tasks, thereby improving the effect and customer satisfaction in practical applications.

[0124] Furthermore, the model parameters include a first weight matrix and a second weight matrix, the model parameters of the pre-trained model are initialized, and the initialized pre-trained model is trained using the first training data to obtain a universal language model, specifically including:

[0125] Initializing a first weight matrix and a second weight matrix;

[0126] Importing the first training data into the pre-trained model so that the pre-trained model learns common scene-related features;

[0127] In the process of learning general scene-related features by the pre-training model, the initialized first weight matrix and the second weight matrix are updated until the model is fitted to obtain a general language model.

[0128] In this embodiment, first, the first weight matrix and the second weight matrix of the model are initialized to ensure that the model parameters are in an initial state. Then, the first training data is imported into the pre-trained model to enable it to learn general scenario-related features. These data contain general conversation texts, which help the model to master general language patterns and expressions. During the model training process, the parameters of the first weight matrix and the second weight matrix are continuously adjusted and updated through forward propagation and back propagation, and the performance of the model is gradually optimized until the model can fit the training data well. Ultimately, the fully trained pre-trained model becomes a model with general language understanding capabilities, laying the foundation for subsequent fine-tuning of specific tasks. In this way, the model can generate natural and accurate conversations in a wide range of auto insurance scenarios, improving its effectiveness and adaptability in practical applications.

[0129] Further, the step of using the second training data to iteratively train the general language model to obtain a speech recommendation model specifically includes:

[0130] Importing the second training data into the general language model so that the general language model learns the relevant features of the specific scenario;

[0131] In the process of the general language model learning the relevant features of a specific scenario, the first weight matrix and the second weight matrix of the general language model are updated until the model is fitted to obtain a speech recommendation model.

[0132] In this embodiment, first, the second training data containing features related to specific scenarios is imported into the general language model, so that the model begins to learn the language patterns and features of these specific scenarios. These data reflect specific auto insurance business needs, such as claims consultation, specific product recommendations, etc. During the training process, the model continuously adjusts the parameters of the first weight matrix and the second weight matrix through forward propagation and back propagation, so that it is gradually optimized to meet the needs of specific scenarios. This iterative training process continues until the model can fit the specific scenario data well, and finally forms a speech recommendation model optimized specifically for specific tasks. In this way, the model can generate more accurate and professional conversations when dealing with specific auto insurance scenarios, improving the quality and efficiency of customer service.

[0133] In the above embodiment, by importing the first training data into the pre-trained model, it learns the general scene-related features and obtains the general language model. By importing the second training data containing the specific scene-related features into the general language model, the model begins to learn the language patterns and features of these specific scenes, and obtains the specific scene-related speech recommendation model. In the above training process, low-rank decomposition (LoRA) is used to control the update of the general language model parameters, so that the network layer and the adapter layer in the model are as independent as possible. When the pre-trained model is trained using the first training data, a higher intrinsic rank score can be assigned to the network layer of the pre-trained model, and a lower intrinsic rank score can be assigned to the adapter layer, so that the network layer of the pre-trained model can fully learn the training tasks of the general scene strategy during model training, and the influence of the training tasks of the general scene strategy on the adapter layer can be reduced as much as possible. When the general language model is trained using the second training data, a lower intrinsic rank score can be assigned to the network layer of the general language model, and a higher intrinsic rank score can be assigned to the adapter layer, so that the adapter layer can fully learn the training tasks of the specific scene strategy during model training, and the influence of the training tasks of the specific scene strategy on the network layer of the general language model can be reduced as much as possible.

[0134] Furthermore, low-rank decomposition is used to control the updating of the first weight matrix and the second weight matrix. In the process of learning general scene-related features by the pre-training model, the steps of updating the initialized first weight matrix and the second weight matrix specifically include:

[0135] In the process of learning general scene-related features by the pre-training model, a first intrinsic rank is configured for the first weight matrix, and a second intrinsic rank is configured for the second weight matrix, wherein the second intrinsic rank is smaller than the first intrinsic rank;

[0136] The update parameters of the initialized first weight matrix and the second weight matrix are controlled by the first intrinsic rank and the second intrinsic rank.

[0137] In this embodiment, the update of the first weight matrix and the second weight matrix is ​​controlled by adopting low-rank decomposition to improve the efficiency and effect of model training. In the process of learning general scene-related features by the pre-training model, a larger intrinsic rank (i.e., the first intrinsic rank) is first configured for the first weight matrix, and a smaller intrinsic rank (i.e., the second intrinsic rank) is configured for the second weight matrix. Intrinsic rank refers to the effective rank of the matrix, which determines the complexity of matrix decomposition and the degree of freedom of parameter update. By configuring a larger intrinsic rank for the first weight matrix, it is ensured that the model has higher expressive power and flexibility in general scene learning, and by configuring a smaller intrinsic rank for the second weight matrix, the update range of the adapter layer parameters is effectively controlled, overfitting is avoided and training stability is improved.

[0138] During the model training process, the first weight matrix and the second weight matrix are decomposed into low-rank according to the preset intrinsic rank. By decomposing these matrices into low-rank forms, the original matrices are represented by fewer parameters, thereby controlling the complexity of parameter updates. As training progresses, the model updates the parameters of the weight matrices through forward propagation and back propagation when learning common scene-related features. These update processes are controlled by the intrinsic rank, ensuring that the parameter updates have enough degrees of freedom to capture data features while avoiding excessive complexity, thereby improving training efficiency and model generalization ability.

[0139] Furthermore, in the process of the general language model learning the specific scene-related features, the step of updating the first weight matrix and the second weight matrix of the general language model specifically includes:

[0140] In the process of learning the scene-specific related features by the general language model, a third intrinsic rank is configured for the first weight matrix, and a fourth intrinsic rank is configured for the second weight matrix, wherein the fourth intrinsic rank is greater than the third intrinsic rank;

[0141] The update parameters of the first weight matrix and the second weight matrix of the universal language model are controlled by the third intrinsic rank and the fourth intrinsic rank.

[0142] In this embodiment, when the general language model learns the relevant features of a specific scene, a lower intrinsic rank (third intrinsic rank) is configured for the first weight matrix, and a higher intrinsic rank (fourth intrinsic rank) is configured for the second weight matrix. The third intrinsic rank is low, which limits the update amplitude of the first weight matrix, so that it remains robust in a specific scene and avoids overfitting; the fourth intrinsic rank is high, allowing the second weight matrix to have sufficient flexibility to capture the complex features in a specific scene.

[0143] In actual operation, as the scene-specific data is input, the universal language model uses this data for forward propagation and back propagation, and controls the parameter updates of the first weight matrix and the second weight matrix through the third intrinsic rank and the fourth intrinsic rank, respectively. The low-rank decomposition method decomposes these matrices into low-rank forms, thereby reducing the complexity and number of parameter updates while maintaining the expressiveness of the model. The third intrinsic rank is low, and by limiting the parameter updates of the first weight matrix, it ensures the robustness and generalization ability of the model in specific scenarios. The fourth intrinsic rank is high, allowing more degrees of freedom to learn complex scene-specific features and enhance the adaptability and accuracy of the adapter layer.

[0144] In the above embodiment, by finely controlling the update of the weight matrix, the complexity and performance of the model are effectively balanced, ensuring that it can fully capture complex features while maintaining the robustness and efficiency of the model when processing specific scenario tasks. Ultimately, the optimized model performs better in specific scenarios, can provide accurate, professional and coherent speech generation services, and improve customer experience and business efficiency in practical applications.

[0145] In this embodiment, the electronic device (eg, Figure 1 The server shown in the figure) can receive instructions or obtain data through a wired connection or a wireless connection. It should be noted that the above wireless connection method may include but is not limited to 3G / 4G connection, WiFi connection, Bluetooth connection, WiMAX connection, Zigbee connection, UWB (ultra wideband) connection, and other wireless connection methods currently known or to be developed in the future.

[0146] It should be emphasized that in order to further ensure the privacy and security of the above training data, the above training data can also be stored in a blockchain node.

[0147] The blockchain referred to in this application is a new application model of computer technologies such as distributed data storage, peer-to-peer transmission, consensus mechanism, encryption algorithm, etc. Blockchain is essentially a decentralized database, a string of data blocks generated by cryptographic methods. Each data block contains a batch of network transaction information, which is used to verify the validity of its information (anti-counterfeiting) and generate the next block. Blockchain can include the underlying blockchain platform, platform product service layer, and application service layer.

[0148] Those skilled in the art can understand that all or part of the processes in the above-mentioned embodiments can be implemented by instructing related hardware through computer-readable instructions, and the computer-readable instructions can be stored in a computer-readable storage medium. When the computer-readable instructions are executed, they can include the processes of the embodiments of the above-mentioned methods. Among them, the aforementioned storage medium can be a non-volatile storage medium such as a disk, an optical disk, a read-only memory (ROM), or a random access memory (RAM).

[0149] It should be understood that, although the steps in the flowchart of the accompanying drawings are displayed in sequence as indicated by the arrows, these steps are not necessarily executed in sequence in the order indicated by the arrows. Unless otherwise specified herein, there is no strict order restriction on the execution of these steps, and they can be executed in other orders. Moreover, at least a part of the steps in the flowchart of the accompanying drawings may include multiple sub-steps or multiple stages, and these sub-steps or stages are not necessarily executed at the same time, but can be executed at different times, and their execution order is not necessarily sequential, but can be executed in turn or alternately with other steps or at least a part of the sub-steps or stages of other steps.

[0150] Further references Figure 3 , as a response to the above Figure 2 The present application provides an embodiment of a device for generating a recommendation speech. Figure 2 Corresponding to the method embodiment shown, the device can be specifically applied to various electronic devices.

[0151] like Figure 3 As shown, the recommendation speech generating device 300 described in this embodiment includes:

[0152] A model building module 301 is used to build a pre-trained model, wherein the pre-trained model includes a plurality of network layers;

[0153] An adapter construction module 302, used to construct an adapter layer between network layers of the pre-trained model, wherein the adapter layer is constructed based on a small neural network;

[0154] A first data module 303, configured to determine first training data according to a first training task, wherein the training task is a training task of a general scenario strategy;

[0155] A first training module 304 is used to initialize the model parameters of the pre-trained model, and perform model training on the initialized pre-trained model using the first training data to obtain a universal language model;

[0156] A second data module 305 is used to determine second training data according to a second training task, wherein the training task is a training task of a specific scenario strategy;

[0157] The second training module 306 is used to perform model iterative training on the general language model using the second training data to obtain a speech recommendation model;

[0158] The speech recommendation module 307 is used to receive the car insurance speech generation instruction, use the speech recommendation model to generate the car insurance recommendation speech, and output the car insurance recommendation speech.

[0159] In the above embodiment, the present application discloses a device for generating recommended speech. By constructing a pre-trained model including an adapter layer, the general language model and the flexible application of specific scenario strategies are effectively combined. The adapter layer is used as a bridge to connect the pre-trained model network layer, and a small neural network is used to realize rapid training of specific scenario models. While ensuring model performance, the training cost is reduced. The present application uses two stages of training. First, a general language model is trained using the first training data, and then iterative training is performed using the second training data for specific scenarios to finally obtain a speech recommendation model. The present application adopts the above-mentioned pre-trained model structure. By first training a general language model, corresponding speech recommendation models can be trained for a variety of specific scenarios, respectively, so as to accurately and efficiently generate speech recommendations.

[0160] To solve the above technical problems, the present application also provides a computer device. Figure 4 , Figure 4 This is a basic structural block diagram of the computer device in this embodiment.

[0161] The computer device 4 includes a memory 41, a processor 42, and a network interface 43 that are interconnected through a system bus. It should be noted that the figure only shows a computer device 4 with components 41-43, but it should be understood that it is not required to implement all the components shown, and more or fewer components can be implemented instead. Among them, those skilled in the art can understand that the computer device here is a device that can automatically perform numerical calculations and / or information processing according to pre-set or stored instructions, and its hardware includes but is not limited to microprocessors, application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), digital processors (DSPs), embedded devices, etc.

[0162] The computer device may be a computing device such as a desktop computer, a notebook, a PDA, a cloud server, etc. The computer device may interact with a user through a keyboard, a mouse, a remote controller, a touch pad, or a voice control device.

[0163] The memory 41 includes at least one type of readable storage medium, and the readable storage medium includes flash memory, hard disk, multimedia card, card-type memory (for example, SD or DX memory, etc.), random access memory (RAM), static random access memory (SRAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), programmable read-only memory (PROM), magnetic memory, disk, optical disk, etc. In some embodiments, the memory 41 can be an internal storage unit of the computer device 4, such as a hard disk or memory of the computer device 4. In other embodiments, the memory 41 can also be an external storage device of the computer device 4, such as a plug-in hard disk, a smart memory card (Smart Media Card, SMC), a secure digital (Secure Digital, SD) card, a flash card (FlashCard), etc. equipped on the computer device 4. Of course, the memory 41 can also include both the internal storage unit of the computer device 4 and its external storage device. In this embodiment, the memory 41 is generally used to store the operating system and various application software installed on the computer device 4, such as computer-readable instructions of the recommendation speech generation method, etc. In addition, the memory 41 can also be used to temporarily store various types of data that have been output or are to be output.

[0164] The processor 42 may be a central processing unit (CPU), a controller, a microcontroller, a microprocessor, or other data processing chip in some embodiments. The processor 42 is generally used to control the overall operation of the computer device 4. In this embodiment, the processor 42 is used to run computer-readable instructions stored in the memory 41 or process data, such as computer-readable instructions for running the recommendation speech generation method.

[0165] The network interface 43 may include a wireless network interface or a wired network interface. The network interface 43 is generally used to establish a communication connection between the computer device 4 and other electronic devices.

[0166] In the above embodiment, the present application discloses a computer device, which effectively combines the flexible application of the general language model and the specific scenario strategy by constructing a pre-trained model including an adapter layer, uses the adapter layer as a bridge to connect the pre-trained model network layer, and uses a small neural network to realize the rapid training of the specific scenario model, while ensuring the model performance, reducing the training cost. The present application uses two stages of training. First, a general language model is trained using the first training data, and then iterative training is performed using the second training data for specific scenarios to finally obtain a speech recommendation model. The present application adopts the above-mentioned pre-trained model structure. By first training a general language model, corresponding speech recommendation models can be trained for a variety of specific scenarios, respectively, so as to accurately and efficiently generate speech recommendations.

[0167] The present application also provides another embodiment, namely, providing a computer-readable storage medium, wherein the computer-readable storage medium stores computer-readable instructions, and the computer-readable instructions can be executed by at least one processor to enable the at least one processor to perform the steps of the recommendation speech generation method as described above.

[0168] In the above embodiment, the present application discloses a computer-readable storage medium, which effectively combines the flexible application of the general language model and the specific scenario strategy by constructing a pre-trained model including an adapter layer, uses the adapter layer as a bridge to connect the pre-trained model network layer, and uses a small neural network to realize the rapid training of the specific scenario model, while ensuring the model performance, reducing the training cost. The present application uses two stages of training. First, a general language model is trained using the first training data, and then iterative training is performed using the second training data for specific scenarios to finally obtain a speech recommendation model. The present application adopts the above-mentioned pre-trained model structure. By first training a general language model, corresponding speech recommendation models can be trained for a variety of specific scenarios, respectively, so as to accurately and efficiently generate speech recommendations.

[0169] Through the description of the above implementation methods, those skilled in the art can clearly understand that the above-mentioned embodiment methods can be implemented by means of software plus a necessary general hardware platform, and of course by hardware, but in many cases the former is a better implementation method. Based on such an understanding, the technical solution of the present application, or the part that contributes to the prior art, can be embodied in the form of a software product, which is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk), and includes a number of instructions for a terminal device (which can be a mobile phone, computer, server, air conditioner, or network device, etc.) to execute the methods described in each embodiment of the present application.

[0170] The present application can be used in many general or special computer system environments or configurations. For example: personal computers, server computers, handheld or portable devices, tablet devices, multiprocessor systems, microprocessor-based systems, set-top boxes, programmable consumer electronics, network PCs, minicomputers, mainframe computers, distributed computing environments including any of the above systems or devices, etc. The present application can be described in the general context of computer-executable instructions executed by a computer, such as program modules. Generally, program modules include routines, programs, objects, components, data structures, etc. that perform specific tasks or implement specific abstract data types. The present application can also be practiced in distributed computing environments, in which tasks are performed by remote processing devices connected through a communication network. In a distributed computing environment, program modules can be located in local and remote computer storage media including storage devices.

[0171] Obviously, the embodiments described above are only some embodiments of the present application, rather than all embodiments. The preferred embodiments of the present application are given in the accompanying drawings, but they do not limit the patent scope of the present application. The present application can be implemented in many different forms. On the contrary, the purpose of providing these embodiments is to make the understanding of the disclosure of the present application more thorough and comprehensive. Although the present application is described in detail with reference to the aforementioned embodiments, for those skilled in the art, it is still possible to modify the technical solutions recorded in the aforementioned specific implementation methods, or to perform equivalent replacement of some of the technical features therein. Any equivalent structure made using the contents of the specification and drawings of this application, directly or indirectly used in other related technical fields, is similarly within the scope of patent protection of this application.

Claims

1. A method for generating recommendation words. It is characterized in that The method comprises: Constructing a pre-trained model, wherein the pre-trained model includes a plurality of network layers; Building an adapter layer between the network layers of the pre-trained model, wherein the adapter layer is built based on a small neural network; Determining first training data according to a first training task, wherein the training task is a training task of a general scenario strategy; Initializing model parameters of the pre-trained model, and performing model training on the initialized pre-trained model using the first training data to obtain a universal language model; Determining second training data according to a second training task, wherein the training task is a training task for a specific scenario strategy; Using the second training data to perform model iterative training on the general language model to obtain a speech recommendation model; Receive an instruction for generating auto insurance script, use the script recommendation model to generate auto insurance recommendation script, and output the auto insurance recommendation script.

2. The method for generating recommendation words according to claim 1, It is characterized in that After the step of constructing an adapter layer between the network layers of the pre-trained model, the method further includes: Obtaining network parameters of each network layer in the pre-trained model, wherein the network parameters are preset weights and biases of each network layer in the pre-trained model; Constructing a network parameter matrix based on the network parameters of each network layer to obtain a first weight matrix; Acquire adapter parameters of each of the adapter layers, wherein the adapter parameters are preset weights and biases of each of the adapter layers; An adapter parameter matrix is ​​constructed based on the adapter parameters of each of the adapter layers to obtain a second weight matrix.

3. The method for generating recommendation words according to claim 2, It is characterized in that The step of constructing a network parameter matrix based on the network parameters of each network layer to obtain a first parameter matrix specifically includes: Mapping the network parameters of each network layer in the pre-trained model to a preset one-dimensional space to obtain a first parameter vector; Combining all first parameter vectors in the one-dimensional space to obtain the first weight matrix; The step of constructing an adapter parameter matrix based on the adapter parameters of each adapter layer to obtain a second weight matrix specifically includes: Mapping the adapter parameters of each adapter layer to a preset one-dimensional space to obtain a second parameter vector; All second parameter vectors in the one-dimensional space are combined to obtain the second weight matrix.

4. The method for generating recommendation words according to claim 2, It is characterized in that The step of determining the first training data according to the first training task specifically includes: Acquire pre-training data, wherein the pre-training is a dialogue text of historical auto insurance recommendations; Identifying features in the pre-training data that are related to the general scenario strategy to obtain general scenario-related features; Identifying common scene-related features in the pre-training data to obtain the first training data; The step of determining the second training data according to the second training task specifically includes: Identify features in the pre-training data that are related to the specific scenario strategy to obtain specific scenario-related features; Specific scene-related features in the pre-training data are identified to obtain the second training data.

5. The method for generating recommendation words according to claim 4, It is characterized in that The model parameters include the first weight matrix and the second weight matrix, the model parameters of the pre-trained model are initialized, and the steps of training the initialized pre-trained model using the first training data to obtain a universal language model specifically include: Initializing the first weight matrix and the second weight matrix; Importing the first training data into a pre-trained model so that the pre-trained model learns the general scenario-related features; In the process of the pre-training model learning the general scene-related features, the initialized first weight matrix and the second weight matrix are updated until the model is fitted to obtain the general language model.

6. The method for generating recommendation words according to claim 5, It is characterized in that The step of using the second training data to iteratively train the general language model to obtain a speech recommendation model specifically includes: Importing the second training data into the universal language model so that the universal language model learns the specific scenario-related features; In the process of the general language model learning the relevant features of the specific scenario, the first weight matrix and the second weight matrix of the general language model are updated until the model is fitted to obtain the speech recommendation model.

7. The method for generating recommendation words according to claim 6, It is characterized in that Low-rank decomposition is used to control the updating of the first weight matrix and the second weight matrix, and the step of updating the initialized first weight matrix and the second weight matrix during the process of the pre-training model learning the general scene-related features specifically includes: In the process of the pre-trained model learning the general scene-related features, configuring a first intrinsic rank for the first weight matrix and configuring a second intrinsic rank for the second weight matrix, wherein the second intrinsic rank is smaller than the first intrinsic rank; Controlling the update parameters of the initialized first weight matrix and the second weight matrix through the first intrinsic rank and the second intrinsic rank; The step of updating the first weight matrix and the second weight matrix of the universal language model during the process of the universal language model learning the specific scenario-related features specifically includes: In the process of the general language model learning the specific scene-related features, configuring a third intrinsic rank for the first weight matrix and configuring a fourth intrinsic rank for the second weight matrix, wherein the fourth intrinsic rank is greater than the third intrinsic rank; The update parameters of the first weight matrix and the second weight matrix of the universal language model are controlled by the third intrinsic rank and the fourth intrinsic rank.

8. A device for generating recommendation words, It is characterized in that The device comprises: A model building module, used to build a pre-trained model, wherein the pre-trained model includes several network layers; An adapter construction module, used to construct an adapter layer between the network layers of the pre-trained model, wherein the adapter layer is built based on a small neural network; A first data module, used to determine first training data according to a first training task, wherein the training task is a training task of a general scenario strategy; A first training module is used to initialize the model parameters of the pre-trained model, and use the first training data to perform model training on the initialized pre-trained model to obtain a universal language model; A second data module, used to determine second training data according to a second training task, wherein the training task is a training task of a specific scenario strategy; A second training module is used to perform model iterative training on the general language model using the second training data to obtain a speech recommendation model; The speech recommendation module is used to receive the car insurance speech generation instruction, use the speech recommendation model to generate the car insurance recommendation speech, and output the car insurance recommendation speech.

9. A computer device, It is characterized in that It includes a memory and a processor, wherein the memory stores computer-readable instructions, and when the processor executes the computer-readable instructions, the steps of the method for generating recommendation words as described in any one of claims 1 to 7 are implemented.

10. A computer-readable storage medium, It is characterized in that The computer-readable storage medium stores computer-readable instructions, and when the computer-readable instructions are executed by a processor, the steps of the method for generating recommendation words as described in any one of claims 1 to 7 are implemented.

Images