Data processing method and device, electronic equipment and computer readable storage medium
By calculating the delivery volume and training sample number of vehicles, the training sample set corresponding to the vehicle model is accurately extracted and the problem extraction model is optimized, which solves the problem of poor generalization ability of data processing models in the existing technology, and achieves more accurate data recording and service recommendations.
Patent Information
- Application Number
- CN202410027535.1
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-01-08
- Publication Date
- 2025-07-11
AI Technical Summary
The existing work order processing methods cannot accurately recommend solutions in terms of vehicle failure, vehicle body damage and maintenance. The accuracy of target training samples is low, resulting in poor generalization capabilities of data processing models and inaccurate system recording results.
By obtaining the delivery volume of each model, the total delivery volume of all models and the total number of training samples, the model data extraction formula is used to calculate the training samples of each model, and the corresponding training sample set is extracted from the original training sample set, and the target training sample set is combined into the target training sample set, and training is based on the pre-trained language processing model, the problem extraction model is obtained, and quality evaluation and optimization is performed through the BLEU and ROUGE algorithms.
The accuracy of the target training sample set is improved, the generalization ability of the problem extraction model is enhanced, more accurate data recording results are achieved, and the service efficiency and user satisfaction of work order processing are improved.
Smart Images
Figure CN120297400A_ABST
Abstract
Description
Technical Field
[0001] This application relates to the technical field of data processing, and in particular, to a data processing method, apparatus, electronic device, and computer-readable storage medium. Background Art
[0002] With the continuous development of the automotive field, vehicles have become a driving tool that almost every household will choose. There are many demand problems that need to be solved in aspects such as vehicle after-sales (for example: vehicle faults, body damage, maintenance, etc.). When vehicle users call to communicate and submit work orders, it is necessary to summarize the main problems of users and recommend solutions based on information such as vehicle problem descriptions and vehicle models, so as to improve service efficiency and user satisfaction.
[0003] In the existing work order processing process, artificial intelligence is combined with auto repair services to improve service quality. However, although the existing work order processing method can achieve automatic recording and sharing, its data processing model cannot recommend better solutions based on the vehicle model data of the vehicle. The accuracy of obtaining target training samples is relatively low, resulting in poor generalization ability of the data processing model and inaccurate system recording results. Summary of the Invention
[0004] This application provides a data processing method, apparatus, electronic device, and computer-readable storage medium for improving the accuracy of obtaining a target training sample set.
[0005] In a first aspect, an embodiment of this application provides a data processing method, and the method includes:
[0006] Obtain the delivery volume of each vehicle model, the total delivery volume of all vehicle models, and the total number of training samples;
[0007] Substitute the delivery volume of each vehicle model, the total delivery volume of all vehicle models, and the total number of training samples into the vehicle model data extraction formula to obtain the number of training samples for each vehicle model;
[0008] Extract the training sample set corresponding to each vehicle model from the original training sample set according to the number of training samples for each vehicle model;
[0009] Combine the training sample sets corresponding to each vehicle model to obtain the target training sample set;
[0010] Obtain a pre-trained language processing model;
[0011] Train the pre-trained language processing model based on the target training sample set to obtain a problem extraction model.
[0012] In some embodiments, substituting the delivery volume of each vehicle model, the total delivery volume of all vehicle models, and the number of training samples in the target training sample set into the vehicle model data extraction formula to obtain the number of training samples for each vehicle model includes:
[0013] Calculating the product of the delivery ratio of each vehicle model and the total number of training samples in the target training sample set to obtain the number of training samples for each vehicle model; the delivery ratio of any vehicle model is the ratio of the delivery volume of this vehicle model to the total delivery volume of all vehicle models.
[0014] In some embodiments, extracting the training sample set corresponding to each vehicle model from the training samples according to the number of training samples for each vehicle model includes:
[0015] Extracting the training samples corresponding to the first vehicle model in the original training sample set;
[0016] Sorting the classifications of the training samples corresponding to the first vehicle model according to the usage frequency;
[0017] Dividing the training samples corresponding to the first vehicle model into a first training sample set and a second training sample set according to the sorting result of the classifications of the training samples corresponding to the first vehicle model;
[0018] Obtaining a first quantity and a second quantity according to the number of training samples corresponding to the first vehicle model and a preset ratio;
[0019] Obtaining a first number of training samples from the first training sample set and a second number of training samples from the second training sample set;
[0020] Combining the first number of training samples and the second number of training samples to obtain the training sample set corresponding to the first vehicle model.
[0021] In some embodiments, dividing the training samples corresponding to the first vehicle model into a first training sample set and a second training sample set according to the sorting result of the classifications of the training samples corresponding to the first vehicle model includes:
[0022] Combining the training samples of the first preset number of classifications in the sorting result among the training samples corresponding to the first vehicle model into the first training sample set;
[0023] Combining the other training samples among the training samples corresponding to the first vehicle model except the training samples in the first training sample set into the second training sample set.
[0024] In some embodiments, obtaining a first number of training samples from the first training sample set and a second number of training samples from the second training sample set includes:
[0025] Randomly obtain a first number of training samples from the first training sample set, and randomly obtain a second number of training samples from the second training sample set.
[0026] In some embodiments, after obtaining the question extraction model, the method further includes:
[0027] Evaluating the quality of the question extraction model based on the bilingual mutual translation quality assessment algorithm BLEU and / or the recall-based evaluation algorithm ROUGE;
[0028] Optimizing the question extraction model according to the quality evaluation result of the question extraction model.
[0029] In a second aspect, an embodiment of the present application provides a data processing method, the method including:
[0030] Obtaining first session content;
[0031] Inputting the first session content into a question extraction model, and obtaining a question extraction result corresponding to the first session content output by the question extraction model;
[0032] Wherein, the question extraction model is a model obtained by training a pre-trained language processing model based on a target training sample set, the target training sample set is a training sample set obtained by combining the training sample sets corresponding to each vehicle model, the training sample sets corresponding to each vehicle model are extracted from the original training sample set according to the training sample quantity of each vehicle model, and the training sample quantity of each vehicle model is obtained by substituting the delivery quantity of each vehicle model, the total delivery quantity of all vehicle models, and the total number of training samples of the target training sample set into a vehicle model data extraction formula.
[0033] In some embodiments, before inputting the first session content into the question extraction model, the method further includes:
[0034] Removing stop words and / or punctuation marks from the first session content.
[0035] In a third aspect, an embodiment of the present application provides a data processing device, including:
[0036] An acquisition module, configured to acquire the delivery quantity of each vehicle model, the total delivery quantity of all vehicle models, and the total number of training samples;
[0037] A calculation module, configured to substitute the delivery quantity of each vehicle model, the total delivery quantity of all vehicle models, and the total number of training samples into a vehicle model data extraction formula to obtain the training sample quantity of each vehicle model;
[0038] An extraction module, configured to extract the training sample set corresponding to each vehicle model from the original training sample set according to the number of training samples of each vehicle model;
[0039] A combination module, configured to combine the training sample sets corresponding to each vehicle model to obtain the target training sample set;
[0040] The obtaining module is further configured to obtain a pre-trained language processing model;
[0041] A training module, configured to train the pre-trained language processing model based on the target training sample set to obtain a question extraction model.
[0042] In some embodiments, the calculation module is specifically configured to calculate the product of the delivery ratio of each vehicle model and the total number of training samples in the target training sample set to obtain the number of training samples of each vehicle model; the delivery ratio of any vehicle model is the ratio of the delivery volume of the vehicle model to the total delivery volume of all vehicle models.
[0043] In some embodiments, the extraction module is specifically configured to extract the training samples corresponding to the first vehicle model in the original training sample set; sort the classifications of the training samples corresponding to the first vehicle model according to the usage frequency; according to the sorting result of the classifications of the training samples corresponding to the first vehicle model, divide the training samples corresponding to the first vehicle model into a first training sample set and a second training sample set; obtain a first quantity and a second quantity according to the number of training samples corresponding to the first vehicle model and a preset ratio; obtain the first quantity of training samples from the first training sample set and obtain the second quantity of training samples from the second training sample set; combine the first quantity of training samples and the second quantity of training samples to obtain the training sample set corresponding to the first vehicle model.
[0044] In some embodiments, the extraction module is specifically configured to combine the training samples of the first preset number of classifications in the sorting result in the training samples corresponding to the first vehicle model into the first training sample set; combine the other training samples in the training samples corresponding to the first vehicle model except the training samples in the first training sample set into the second training sample set.
[0045] In some embodiments, the extraction module is specifically configured to randomly obtain the first quantity of training samples from the first training sample set and randomly obtain the second quantity of training samples from the second training sample set.
[0046] In some embodiments, the training module is further configured to, after obtaining the question extraction model, perform quality evaluation on the question extraction model based on the bilingual mutual translation quality evaluation algorithm BLEU and / or the recall rate-based evaluation algorithm ROUGE, and optimize the question extraction model according to the quality evaluation result of the question extraction model.
[0047] Fourthly, an embodiment of the present application provides a data processing device, including:
[0048] An acquisition unit, configured to acquire first session content;
[0049] A processing unit, configured to input the first session content into a question extraction model, and acquire a question extraction result corresponding to the first session content output by the question extraction model;
[0050] Wherein, the question extraction model is a model obtained by training a pre-trained language processing model based on a target training sample set, the target training sample set is a training sample set obtained by combining the training sample sets corresponding to each vehicle model, the training sample set corresponding to each vehicle model is extracted from the original training sample set according to the training sample quantity of each vehicle model, and the training sample quantity of each vehicle model is obtained by substituting the delivery volume of each vehicle model, the total delivery volume of all vehicle models, and the total number of training samples of the target training sample set into a vehicle model data extraction formula.
[0051] In some embodiments, the processing unit is further configured to remove stop words and / or punctuation marks in the first session content before inputting the first session content into the question extraction model.
[0052] Fifthly, an embodiment of the present application provides an electronic device, including: a processor, a memory, and a computer program stored on the memory and executable on the processor, where when the computer program is executed by the processor, any of the above data processing methods is implemented.
[0053] Sixthly, an embodiment of the present application provides a computer-readable storage medium, on which a computer program is stored, and when the computer program is executed by a processor, any of the above data processing methods is implemented.
[0054] The data processing method provided by the embodiments of the present application, after obtaining vehicle model data such as the delivery volume of each vehicle model, the total delivery volume of all vehicle models, and the number of training samples in the target training sample set, substitutes the delivery volume of each vehicle model, the total delivery volume of all vehicle models, and the number of training samples in the target training sample set into the custom vehicle model data extraction formula to obtain the number of training samples for each vehicle model; then extracts the training sample set corresponding to each vehicle model from the original training sample set according to the number of training samples for each vehicle model, and combines the training sample sets corresponding to each vehicle model to obtain the target training sample set. After obtaining the pre-trained language processing model, the pre-trained language processing model is trained based on the target training sample set to obtain the question extraction model. Since the embodiments of the present application substitute the delivery volume of each vehicle model, the total delivery volume of all vehicle models, and the number of training samples in the target training sample set into the custom vehicle model data extraction formula, the number of training samples for each vehicle model obtained by the embodiments of the present application is more accurate, and thus the target training samples obtained from the original data set based on the number of training samples for each vehicle model are also more accurate; the accuracy of obtaining the target training sample set is improved, and further the generalization ability of the question extraction model obtained after training the pre-trained language processing model with the target data set is stronger, and the data recording result is more accurate. BRIEF DESCRIPTION OF THE DRAWINGS
[0055] The drawings here are incorporated into the specification and form a part of this specification, showing embodiments consistent with the present application, and are used together with the specification to explain the principles of the present application.
[0056] In order to more clearly illustrate the technical solutions in the embodiments of the present application or the prior art, the following will briefly introduce the drawings required for use in the description of the embodiments or the prior art. Obviously, for those of ordinary skill in the art, other drawings can also be obtained based on these drawings without creative efforts.
[0057] Figure 1 is a flowchart of the steps of the data processing method provided by some embodiments of the present application;
[0058] Figure 2 is as Figure 1 shown in the specific flowchart of S130 in the data processing method;
[0059] Figure 3 is a flowchart of the steps of the data processing method provided by other embodiments of the present application;
[0060] Figure 4 is a schematic structural diagram of the automatic recording system provided by the embodiments of the present application;
[0061] Figure 5 is a schematic structural diagram of the data processing device provided by some embodiments of the present application;
[0062] Figure 6 FIG. 3 is a schematic structural diagram of a data processing device provided in some other embodiments of the present application;
[0063] Figure 7 FIG. 4 is a schematic structural diagram of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION OF THE EMBODIMENTS
[0064] The technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the accompanying drawings in the embodiments of the present application. Apparently, the described embodiments are only a part of the embodiments of the present application, rather than all the embodiments. All other embodiments obtained by those of ordinary skill in the art based on the embodiments of the present application without creative efforts shall fall within the protection scope of the present application.
[0065] In the following description, specific details such as specific system architectures, interfaces, and technologies are presented for the purpose of illustration rather than limitation, in order to thoroughly understand the present application.
[0066] The terms "first", "second", and "third" in the present application are only used for descriptive purposes, and cannot be construed as indicating or implying relative importance or implicitly specifying the quantity of the indicated technical features. Thus, features defined with "first", "second", and "third" may explicitly or implicitly include at least one of the described features. In the description of the present application, the meaning of "a plurality" is at least two, such as two, three, etc., unless otherwise specifically defined. All directional indications (such as up, down, left, right, front, back...) in the embodiments of the present application are only used to explain the relative positional relationship and movement conditions between components in a specific posture (as shown in the drawings). If the specific posture changes, the directional indications will also change accordingly. The terms "comprising" and "having" in the embodiments of the present application and any variations thereof are intended to cover non-exclusive inclusion. For example, a process, method, system, product, or device that includes a series of steps or units is not limited to the listed steps or units, but optionally further includes steps or units not listed, or optionally further includes other steps or components inherent to these processes, methods, products, or devices.
[0067] Referring to "embodiments" herein means that a particular feature, structure, or characteristic described in connection with the embodiments can be included in at least one embodiment of the present application. The phrase appears in various places in the specification and does not necessarily refer to the same embodiment, nor is it an independent or alternative embodiment mutually exclusive with other embodiments. Those skilled in the art will explicitly and implicitly understand that the embodiments described herein can be combined with other embodiments.
[0068] The present application will be described in detail below with reference to the accompanying drawings and embodiments.
[0069] An embodiment of a data processing method is provided in the present application. Referring to Figure 1 as shown, the data processing method includes the following steps:
[0070] S110. Obtain the delivery volume of each vehicle model, the total delivery volume of all vehicle models, and the number of training samples.
[0071] Among them, the number of training samples refers to the number of training samples in a preset sample data set for training a pre-trained language processing model. For example, if it is preset that the pre-trained language processing model needs to be trained with 1 million training samples, then the number of training samples can be 1 million.
[0072] As shown in Table 1 is an example of vehicle model data analysis:
[0073] Table 1
[0074] Vehicle Model Vehicle Model 1 Vehicle Model 2 Vehicle Model 3 All Vehicle Models Delivery Volume <![CDATA[V1]]> <![CDATA[V2]]> <![CDATA[V3]]> <![CDATA[v 总 >
[0075] Exemplarily, the delivery volume of vehicle model 1 in Table 1 is V1, the delivery volume of vehicle model 2 is V2, the delivery volume of vehicle model 3 is V3, and the total delivery volume of all vehicle models is V 总 .
[0076] S120. Substitute the delivery volume of each vehicle model, the total delivery volume of all vehicle models, and the total number of training samples into the vehicle model data extraction formula to obtain the number of training samples for each vehicle model.
[0077] In some embodiments, the above step S120 (substituting the delivery volume of each vehicle model, the total delivery volume of all vehicle models, and the total number of training samples into the vehicle model data extraction formula to obtain the number of training samples for each vehicle model) includes:
[0078] Calculate the product of the delivery ratio of each vehicle model and the total number of training samples to obtain the number of training samples for each vehicle model; the delivery ratio of any vehicle model is the ratio of the delivery volume of this vehicle model to the total delivery volume of all vehicle models.
[0079] That is, represent the delivery volume of a certain vehicle model as N(v c ), represent the total delivery volume of all vehicle models as N(v 总 ), represent the total number of training samples as M, and represent the number of training samples of this vehicle model as M c ;
[0080] Then there is: the delivery ratio of this vehicle model is The number of training samples of this vehicle model
[0081] S130. Extract the training sample sets corresponding to each vehicle model from the original training sample set according to the number of training samples of each vehicle model.
[0082] In some embodiments, the original training sample set may be a training sample set composed of all training samples. The original training sample set includes training samples of each vehicle model. However, when actually collecting training samples, it is impossible to collect all the conversation contents of all vehicle models, and the number of conversation contents generated by each vehicle model may also vary. Therefore, the proportion of training samples of each vehicle model in the original training sample set may not be the same as the delivery proportion of each vehicle model.
[0083] In some embodiments, the above step S130 only extracts the training sample set corresponding to a certain vehicle model calculated in step S120 from the original training sample set.
[0084] For example, if the number of training samples of vehicle model 1 calculated in step S120 is M1, then M1 training samples corresponding to vehicle model 1 are extracted from the original training sample set as the training sample set corresponding to vehicle model 1.
[0085] Continue to refer to Figure 1 , the data processing method provided in this embodiment further includes step S140, as follows:
[0086] S140. Combine the training sample sets corresponding to each vehicle model to obtain a target training sample set.
[0087] Exemplarily, taking three vehicle models, vehicle model 1, vehicle model 2, and vehicle model 3, as an example, that is, the target training sample set consists of the training sample set corresponding to vehicle model 1, the training sample set corresponding to vehicle model 2, and the training sample set corresponding to vehicle model 3.
[0088] If the training sample set corresponding to vehicle model 1 is {A1, B1......N1}, the training sample set corresponding to vehicle model 2 is {A2, B2......K2}, and the training sample set corresponding to vehicle model 3 is {A3, B3......L3}; then the target training sample set is {A1, B1......N1, A2, B2......K2, A3, B3......L3}.
[0089] S150. Obtain a pre-trained language processing model.
[0090] The pre-trained language processing model in the embodiments of the present application may be large models such as the dialogue robot ChatGLM, ChatGPT, or may also be self-constructed models such as FastTEXT, TextCNN, etc. The embodiments of the present application do not limit the pre-trained language processing model.
[0091] S160. Train a pre-trained language processing model using a target training sample set to obtain a question extraction model.
[0092] Exemplarily, if the pre-trained language processing model is the ChatGPT model, the ChatGPT model can be trained using the target training sample set to obtain the question extraction model.
[0093] In some embodiments, training a pre-trained language processing model using a target training sample set includes: sequentially inputting the conversation content of each training sample in the target training sample set into the pre-trained language processing model. After analyzing the first conversation content, the pre-trained language processing model extracts a first question result. Calculate the loss value based on the loss function, the first question result, and the second question result in the training sample. Fine-tune the pre-trained language processing model using the loss value. The pre-trained language processing model after being fine-tuned with all the sample data in the target training sample set is the question extraction model to be obtained in this embodiment.
[0094] In some embodiments, after performing the above step S160 to obtain the question extraction model, the above data processing method also provides methods for evaluating and optimizing the question processing model, specifically including:
[0095] Evaluate the quality of the question extraction model based on the Bilingual evaluation understudy (BLEU) and / or the Recall-Oriented Understudy for Gisting Evaluation (ROUGE); optimize the question extraction model according to the quality evaluation result of the question extraction model.
[0096] Among them, the score value range of the Bilingual evaluation understudy algorithm can be 0 - 1. The closer the score value is to 1, the higher the translation quality. The Recall-Oriented Understudy for Gisting Evaluation is a commonly used evaluation metric for machine translation and article summarization. ROUGE-N mainly calculates the recall rate on N-grams. Optimizing the question extraction model further improves the performance of the question extraction model.
[0097] The data processing method provided by the embodiments of the present application, after obtaining vehicle model data such as the delivery volume of each vehicle model, the total delivery volume of all vehicle models, and the number of training samples in the target training sample set, substitutes the delivery volume of each vehicle model, the total delivery volume of all vehicle models, and the number of training samples in the target training sample set into the custom vehicle model data extraction formula to obtain the number of training samples for each vehicle model; then extracts the training sample set corresponding to each vehicle model from the original training sample set according to the number of training samples for each vehicle model, and combines the training sample sets corresponding to each vehicle model to obtain the target training sample set. After obtaining the pre-trained language processing model, the pre-trained language processing model is trained based on the target training sample set to obtain a question extraction model. Since the embodiments of the present application substitute the delivery volume of each vehicle model, the total delivery volume of all vehicle models, and the number of training samples in the target training sample set into the custom vehicle model data extraction formula, the number of training samples for each vehicle model obtained by the embodiments of the present application is more accurate, and thus the target training samples obtained from the original data set based on the number of training samples for each vehicle model are also more accurate; the accuracy of obtaining the target training sample set is improved, and further, the generalization ability of the question extraction model obtained after training the pre-trained language processing model with the target data set is stronger, and the data recording result is more accurate.
[0098] As a refinement and extension of the above embodiments, the embodiments of the present application also provide a specific implementation manner of step S130 (extracting the training sample set corresponding to each vehicle model from the original training sample set according to the number of training samples for each vehicle model) in the above data processing method. Refer to Figure 2 As shown, the above step S130 includes the following specific steps:
[0099] S131. Extract the training samples corresponding to the first vehicle model from the original training sample set.
[0100] In step S131, the training samples corresponding to any vehicle model are extracted from the original training sample set to facilitate obtaining the training sample set of the vehicle model from the training samples corresponding to any vehicle model subsequently.
[0101] S132. Sort the classifications of the training samples corresponding to the first vehicle model according to the usage frequency.
[0102] In step S132, the classifications of the training samples of any vehicle model are sorted in descending order according to the usage frequency, so as to facilitate dividing the sorting result of the classifications of the training samples corresponding to the vehicle model in the subsequent steps.
[0103] S133. Divide the training samples corresponding to the first vehicle model into a first training sample set and a second training sample set according to the sorting result of the classifications of the training samples corresponding to the first vehicle model.
[0104] In some embodiments, the training samples of the top preset number of classifications in the sorting results of the training samples corresponding to the first vehicle model are combined into a first training sample set.
[0105] Exemplarily, the top preset number of classifications can be the classifications of the top 100 training samples with the highest usage frequency among the classifications of the training samples of this vehicle model.
[0106] That is, step S133 divides the training samples included in the classifications of the top 100 training samples with the highest usage frequency among the classifications of the training samples of this vehicle model into the first training sample set according to the sorting results of the classifications of the training samples corresponding to any vehicle model in step S132.
[0107] In some embodiments, the other training samples in the training samples corresponding to the first vehicle model except the training samples in the first training sample set are combined into a second training sample set.
[0108] Exemplarily, if the training samples of this vehicle model can include more than 4,000 classifications of the training samples of this vehicle model, then the first training sample set is the training samples included in the classifications of the top 100 training samples with the highest usage frequency among the classifications of the training samples of this vehicle model; the second training sample set is the training samples included in the other training sample classifications remaining after removing the classifications of the top 100 training samples with the highest usage frequency among the training samples of this vehicle model.
[0109] It should be noted that the embodiments of the present application do not limit the number of classifications of the training samples of this vehicle model with the highest usage frequency included in the first training sample, that is, there is no requirement for the above-mentioned top preset number; nor does it limit the number of classifications of the training samples of this vehicle model that can be included in the training samples of this vehicle model; the numbers mentioned in the above embodiments are only set for easier understanding of the technical solutions of the embodiments of the present application and are not unique.
[0110] The embodiments of the present application also provide classifications of the training samples of the vehicle model, and divide the classifications of the training samples of the vehicle model into the following four-level classifications, specifically including:
[0111] First-level classification: major problems such as mechanical and electrical appliances, software systems, etc.;
[0112] Second-level classification: minor problems such as mechanical and electrical appliance repair, mechanical and electrical appliance replacement, etc.;
[0113] Third-level classification: problem areas such as engine failure, vehicle compartment electrical appliances, vehicle body maintenance, etc.;
[0114] Fourth-level classification: specific problems such as poor starting, oil leakage, headlights not working, air conditioner not cooling, etc.
[0115] The following is an example of the third-level classification:
[0116] Mechanical and Electrical Appliances = {
[0117] "Mechanical and Electrical Appliance Repair": {
[0118] ″Engine Failure″: [″Poor Start-up″, ″Oil Leakage″, ″Cylinder Explosion″, ″Stall″],
[0119] ″Compartment Electrical Appliances″: [″Lights Not Working″, ″Air Conditioner Not Cooling″, ″Audio Failure″],
[0120] ″Body Maintenance″: [″Scratches″, ″Dents″, ″Broken Glass″],
[0121] }
[0122] }。
[0123] S134. Obtain a first quantity and a second quantity according to the number of training samples corresponding to the first vehicle model and a preset ratio.
[0124] In some embodiments, the preset ratio includes a first preset ratio and a second preset ratio; wherein, the sum of the first preset ratio and the second preset ratio is 1.
[0125] Specifically, after dividing the training samples corresponding to the first vehicle model into a first training sample set and a second training sample set according to the sorting result of the classification of the training samples corresponding to the first vehicle model in step S133, multiply the number of training samples corresponding to the first training sample set by the first preset ratio to obtain the first quantity, and multiply the number of training samples corresponding to the second training sample set by the second preset ratio to obtain the second quantity.
[0126] Exemplarily, represent the number of training samples corresponding to the first training sample set as Represent the number of training samples corresponding to the second training sample set as Represent the first preset ratio as α, and the second preset ratio as β; then: the first quantity is The second quantity is
[0127] S135. Obtain the first quantity of training samples from the first training sample set and obtain the second quantity of training samples from the second training sample set.
[0128] In some embodiments, the first preset ratio α can be 0.85, the second preset ratio β can be 0.15, and the sum of the first preset ratio α and the second preset ratio β is 1; it should be noted that the specific values of the first preset ratio and the second preset ratio in the embodiments of the present application are not limited, and are only set for easier understanding of this embodiment.
[0129] That is, from the first training sample set Obtain The first number of training samples is obtained from the first training sample set, and the second number of training samples is obtained from the second training sample set. is obtained from the second training sample set.
[0130] In some embodiments, the first number of training samples is randomly obtained from the first training sample set, and the second number of training samples is randomly obtained from the second training sample set.
[0131] S136. Combine the first number of training samples and the second number of training samples to obtain a training sample set for the first vehicle model.
[0132] Exemplarily, the first number can be The second number can be
[0133] Another data processing method is also provided in the embodiments of the present application. Referring to Figure 3 as shown, this data processing method includes the following steps:
[0134] S310. Obtain the first session content.
[0135] Specifically, the data type of the first session content is voice data or text data.
[0136] S320. Input the first session content into the question extraction model, and obtain the question extraction result corresponding to the first session content output by the question extraction model.
[0137] The question extraction model is a model obtained by training a pre-trained language processing model based on a target training sample set. The target training sample set is a training sample set obtained by combining the training sample sets corresponding to each vehicle model. The training sample sets corresponding to each vehicle model are extracted from the original training sample set according to the number of training samples of each vehicle model. The number of training samples of each vehicle model is obtained by substituting the delivery volume of each vehicle model, the total delivery volume of all vehicle models, and the total number of training samples of the target training sample set into the vehicle model data extraction formula.
[0138] In some embodiments, before inputting the first session content into the question extraction model, stop words and / or punctuation marks in the first session content are removed to make the question extraction result more accurate.
[0139] To make it easier to understand the working content of the question extraction model, embodiments of the present application provide an example of the input session content and the output question extraction result of the question extraction model, as follows:
[0140] Example 1.
[0141] First conversation content: The customer said, "Hello. I cleared the air conditioner filter before the maintenance a few days ago." The customer service said, "I'm here." The customer said, "How can I check if the air conditioner filter has been replaced for me?" The customer said, "I'm worried that after I cleared it, the 4S store will think I replaced it myself and won't replace it for me." The customer service said, "You can check the maintenance details in the 'My Orders' section of your APP." The customer said, "Okay." The customer service said, "You can see the content of your maintenance." The customer said, "So it wasn't replaced, right?" The customer service said, "It wasn't replaced." The customer said, "Do you have the contact information over there?" The customer service said, "Do you mean the contact information of the store?" The customer said, "Is the air conditioner filter included in the 999 package?" The customer said, "Yes." The customer said, "I see there is a free air filter." The customer service said, "You have a coupon for the air conditioner filter." The customer service said, "Was the air conditioner filter due for maintenance when you had the maintenance?" The customer said, "I got in touch with them." The customer said, "Send one over for me." The customer said, "Thank you." The customer service said, "Okay." The customer service said, "You're welcome. Is there anything else I can help you with?"
[0142] Problem extraction result: "The customer inquired about how to check the replacement record of the air conditioner filter and how to contact the 4S store to confirm whether it has been replaced. Inquired about the contact information of the store and whether the air conditioner filter is included in the 999 package."
[0143] Example 2
[0144] First conversation content: The customer service said, "Hello. I'm glad to serve you~ Let me check for you. Please wait a moment." The customer said, "Can I stop inserting this plug every day after it's fully charged?" The customer said, "Is it slow charging on the left and fast charging on the right?" The customer service said, "May I ask if your car is an Ideal car?" The customer said, "L9." The customer service said, "Okay. You need to turn it off. To prevent dust and rain from getting in." The customer said, "Is it slow charging on the left and fast charging on the right?" The customer service said, "Yes." The customer service said, "Hello. Is there anything else I can help you with?"
[0145] Problem extraction result: "The customer inquired whether they can stop inserting the charging plug every day and asked if it is slow charging on the left and fast charging on the right."
[0146] Example 3
[0147] First conversation content: The customer said, "Hello, are you there?" The customer service said, "Yes, I am." The customer said, "How much is this one?" The customer said, "Fast charge cover." The customer service said, "Are you currently in Sanya?" The customer service said, "I'll ask the master at the nearby store and also ask if there is any stock at the store." The customer said, "Yes." The customer said, "Can you order one for me?" The customer service said, "Okay, please wait a moment." The customer said, "Okay, thank you." The customer said, "[Putting hands together]." The customer service said, "You're welcome." The customer service said, "May I ask if this is the fast charge cover for L8?" The customer said, "Yes." The customer service said, "Okay." The customer service said, "There is no stock at the store now. Can we let the master at the store contact you after the goods arrive?" The customer said, "Okay, thank you." The customer service said, "You're welcome." The customer said, "Be sure to let the master contact me as soon as the goods arrive." The customer said, "Thank you." The customer service said, "Is it convenient to contact you at the mobile phone number ending with 9789?" The customer service said, "You're too kind." The customer said, "Okay, it's convenient." The customer service said, "Is there anything else I can help you with?" The customer service said, "Okay."
[0148] Problem extraction result: "The customer inquired about the price of the fast charge cover for L8 and whether there is any stock at the nearby store."
[0149] Example 4
[0150] First conversation content: The customer service said, "Yes, I am." The customer said, "I want to know how to log in to the account." The customer said, "The login has expired." The customer service said, "Do you mean the vehicle control account for the Ideal APP?" The customer service said, "Which software are you using on the car machine now?" The customer said, "Watching TV." The customer said, "It's still the previous one." The customer service said, "Are you using iQIYI?" The customer said, "Yes." The customer service said, "You open iQIYI, there should be a setting in the upper left corner, click in and there will be relevant login for the account. You can take a look." The customer said, "Okay, I'll do it later." The customer service said, "Mm-hmm." The customer service said, "Is there anything else I can help you with?"
[0151] Problem extraction result: "The customer inquired about how to handle the expired login of iQIYI Video."
[0152] Example 5
[0153] First conversation content: The customer said, "Are you there?" The customer service said, "Hello, I'm glad to serve you~ What can I do for you?" The customer said, "I want to make an appointment for maintenance." The customer service said, "Where is your vehicle used?" The customer said, "Yinzhou District, Ningbo." The customer service said, "Is it okay for you to come to the Ningbo Yinzhou Service Center for maintenance at 9 am on February 23rd?" The customer said, "Is the earliest time February 23rd?" The customer service said, "Yes. Do you think your time is convenient?" The customer said, "Can you pick up and deliver the car?" The customer service said, "You can enjoy the free pick-up and delivery service after purchasing the vehicle use service package." The customer service said, "Or you can also use the paid pick-up and delivery service." The customer said, "What is the price of the paid pick-up and delivery service?" The customer service said, "The price for the paid pick-up and delivery service within 20 kilometers is 60 yuan, and it increases by 4 yuan for each additional kilometer. You can refer to this." The customer said, "Is the total price for pick-up and delivery 60 yuan?" The customer service said, "It's for the single paid pick-up service." The customer service said, "Do you think the maintenance time is okay? If it's not convenient, you can provide the time you need to arrive at the store, and I will coordinate with the store for you."
[0154] Problem extraction result: "The customer wants to make an appointment for maintenance and inquires about whether pick-up and delivery are available and what the price of the paid pick-up and delivery service is."
[0155] Example 6
[0156] First conversation content: The customer said, "Are you there?" The customer service said, "Hello, I'm glad to serve you~ What can I do for you?" The customer said, "I want to make an appointment for maintenance." The customer service said, "Where is your vehicle used?" The customer said, "Yinzhou District, Ningbo." The customer service said, "Is it okay for you to come to the Ningbo Yinzhou Service Center for maintenance at 9 am on February 23rd?" The customer said, "Is the earliest time February 23rd?" The customer service said, "Yes. Do you think your time is convenient?" The customer said, "Can you pick up and deliver the car?" The customer service said, "You can enjoy the free pick-up and delivery service after purchasing the vehicle use service package." The customer service said, "Or you can also use the paid pick-up and delivery service." The customer said, "What is the price of the paid pick-up and delivery service?" The customer service said, "The price for the paid pick-up and delivery service within 20 kilometers is 60 yuan, and it increases by 4 yuan for each additional kilometer. You can refer to this." The customer said, "Is the total price for pick-up and delivery 60 yuan?" The customer service said, "It's for the single paid pick-up service." The customer service said, "Do you think the maintenance time is okay? If it's not convenient, you can provide the time you need to arrive at the store, and I will coordinate with the store for you."
[0157] Problem extraction result: "The customer inquires about the suspension height adjustment, if the automatic adjustment is turned off. When manually raising the height, will the suspension always maintain this height and at what speed increase will it lower."
[0158] Example 7
[0159] First conversation content: The customer said, "Hello." The customer service said, "Yes." The customer said, "I want to ask today, how can I turn off the air conditioner that turns on automatically?" The customer service said, "Please check the air conditioner page on the central control screen." The customer service said, "Is the automatic button on?" The customer said, "Oh, okay. I forgot about that part of the central control. Your reminder helped me." The customer service said, "Okay, please check it when it's convenient for you." The customer said, "Where is the automatic switch for the air conditioner in the central control area?" The customer service said, "Are there any passengers in the second row?" The customer said, "No." The customer said, "I'm the only one here now." The customer service said, "Please click on the front row and see if the automatic function is on." The customer said, "This situation has been going on for a long time. I always forget to ask you." The customer said, "No." The customer service said, "Please click on the back row and then lock the air conditioner." The customer said, "Okay. What's the principle behind this?" The customer service said, "I think it might be accidentally triggered in the back row. So please lock the back row first and see if the phenomenon still occurs." The customer service said, "Is the screen film applied?" The customer said, "There is no screen film." The customer service said, "Okay. What time did this situation occur just now?" The customer service said, "Okay, please keep observing." The customer said, "Okay, thank you." The customer service said, "You're welcome. Is there anything else I can help you with?" The customer service said, "[Fist and palm salute]" The customer said, "No, thank you." The customer service said, "You're welcome. If you have any problems later, please feel free to contact us at any time."
[0160] Problem extraction result: "The customer inquired about how to turn off the automatically turned-on air conditioner and where the automatic switch is in the central control area of the air conditioner."
[0161] Example 8
[0162] First conversation content: The customer said, "Make an appointment for painting." The customer service said, "Hello." The customer service said, "Which city do you use the car in and what time do you want to go?" The customer said, "How long will it probably take to finish the painting like this?" The customer said, "Quanzhou." The customer service said, "It may not be possible to finish the painting in one day. It may take about 2 days." The customer said, "Then please make an appointment for me." The customer said, "Give me a phone number." The customer said, "I want to ask specifically." The customer said, "How long will it take to finish?" The customer service said, "Then I'll let them contact you. If they ask you to make an appointment, please let me know." The customer said, "Okay." The customer service said, "Okay." The customer said, "How do I report the insurance?" The customer service said, "Your insurance company is Pacific Insurance. The phone number is 95500." The customer service said, "You can directly call this number to report the insurance." The customer said, "Can the insurance cover this situation where I hit it myself?" The customer service said, "If you have commercial insurance, it should be possible. You can call and consult."
[0163] Problem extraction result: "The customer inquired about the painting duration, the store phone number, how to report the insurance, and whether it can be compensated."
[0164] It should be noted that in the automotive field, there are many requirements and problems that need to be solved, such as vehicle failures, body damage, and maintenance. When the car owner calls to communicate and submit a work order, it is necessary to summarize the user's main problems and recommended solutions based on information such as problem descriptions and vehicle model data, so as to improve service efficiency and user satisfaction. In this embodiment, the problem extraction model automatically summarizes and induces the conversation content, and automatically records the extracted problem extraction results on the system, which improves the working efficiency of the system.
[0165] As Figure 4 shown, this embodiment applies the problem extraction model provided in the above embodiment to the work order automatic recording system in the automotive field. The work order automatic recording system includes:
[0166] Customer Relationship Management (CRM) system 41, data conversion module 42, and problem extraction model 43.
[0167] Among them, the customer inputs the problem to be consulted in the form of a conversation through the customer management system 41 into the problem extraction model 43. After analysis by the problem extraction model 43, the problem extraction result is extracted and returned to the customer management system to assist the customer service information recording. It should be noted that if the conversation content is text data, it is directly input into the problem extraction model 43; if the conversation content is voice data, the voice data needs to be converted into text data through the data conversion module 42 and then input into the problem extraction model 43.
[0168] The embodiment of the present application realizes the automatic recording of the automatic recording system through the problem extraction model, solves the difference in expert understanding compared with manual recording, and thus avoids the problem of inconsistent understanding of recording problems; based on the problem extraction model provided in the above embodiment, the intelligent analysis of the automatic recording system is realized.
[0169] The embodiment of the present application provides a data processing device. Please refer to Figure 5 , Figure 5 which is a schematic structural diagram of the data processing device provided by the embodiment of the present application. The data processing device 500 includes:
[0170] An acquisition module 51, configured to acquire the delivery volume of each vehicle model, the total delivery volume of all vehicle models, and the total number of training samples;
[0171] A calculation module 52, configured to substitute the delivery volume of each vehicle model, the total delivery volume of all vehicle models, and the total number of training samples into the vehicle model data extraction formula to obtain the number of training samples for each vehicle model;
[0172] An extraction module 53, configured to extract the training sample set corresponding to each vehicle model from the original training sample set according to the number of training samples of each vehicle model;
[0173] A combination module 54, configured to combine the training sample sets corresponding to each vehicle model to obtain the target training sample set;
[0174] The obtaining module 51 is further configured to obtain a pre-trained language processing model;
[0175] A training module 55, configured to train the pre-trained language processing model based on the target training sample set to obtain a question extraction model.
[0176] In some embodiments, the calculation module 52 is specifically configured to calculate the product of the delivery ratio of each vehicle model and the total number of training samples in the target training sample set to obtain the number of training samples of each vehicle model; the delivery ratio of any vehicle model is the ratio of the delivery volume of this vehicle model to the total delivery volume of all vehicle models.
[0177] In some embodiments, the extraction module 53 is specifically configured to extract the training samples corresponding to the first vehicle model in the original training sample set; sort the classifications of the training samples corresponding to the first vehicle model according to the usage frequency; according to the sorting result of the classifications of the training samples corresponding to the first vehicle model, divide the training samples corresponding to the first vehicle model into a first training sample set and a second training sample set; obtain a first quantity and a second quantity according to the number of training samples corresponding to the first vehicle model and a preset ratio; obtain a first quantity of training samples from the first training sample set and obtain a second quantity of training samples from the second training sample set; combine the first quantity of training samples and the second quantity of training samples to obtain the training sample set corresponding to the first vehicle model.
[0178] In some embodiments, the extraction module 53 is specifically configured to combine the training samples of the first preset number of classifications in the sorting result in the training samples corresponding to the first vehicle model into the first training sample set; combine the other training samples in the training samples corresponding to the first vehicle model except the training samples in the first training sample set into the second training sample set.
[0179] In some embodiments, the extraction module 53 is specifically configured to randomly obtain a first quantity of training samples from the first training sample set and randomly obtain a second quantity of training samples from the second training sample set.
[0180] In some embodiments, the training module 55 is further configured to, after obtaining the question extraction model, perform quality evaluation on the question extraction model based on the bilingual mutual translation quality evaluation algorithm BLEU and / or the evaluation algorithm ROUGE based on recall rate, and optimize the question extraction model according to the quality evaluation result of the question extraction model.
[0181] Another data processing device is provided in an embodiment of the present application. Please refer to Figure 6 , Figure 6 which is a schematic structural diagram of another data processing device provided in an embodiment of the present application. The data processing device 600 includes:
[0182] An acquisition unit 61, configured to acquire first session content;
[0183] A processing unit 62, configured to input the first session content into a question extraction model, and acquire a question extraction result corresponding to the first session content output by the question extraction model;
[0184] Wherein, the question extraction model is a model obtained by training a pre-trained language processing model based on a target training sample set, and the target training sample set is a training sample set obtained by combining training sample sets corresponding to each vehicle model. The training sample sets corresponding to each vehicle model are extracted from the original training sample set according to the number of training samples of each vehicle model, and the number of training samples of each vehicle model is obtained by substituting the delivery volume of each vehicle model, the total delivery volume of all vehicle models, and the total number of training samples of the target training sample set into a vehicle model data extraction formula.
[0185] In some embodiments, the processing unit 62 is further configured to remove stop words and / or punctuation marks in the first session content before inputting the first session content into the question extraction model.
[0186] An embodiment of the present application further provides an electronic device. Please refer to Figure 7 , Figure 7 which is a schematic structural diagram of the electronic device provided in an embodiment of the present application. The electronic device includes: a processor 701, a memory 702, and a computer program stored on the memory 702 and executable on the processor 701. When the computer program is executed by the processor 701, it implements the data processing method described in any one of the above embodiments.
[0187] Those skilled in the art should understand that the embodiments of the present application can be provided as a method, a system, or a computer program product. Therefore, the present application can adopt the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware aspects. Moreover, the present application can adopt the form of a computer program product implemented on one or more computer-usable storage media containing computer-usable program code.
[0188] The processor 701 may be a Central Processing Unit (CPU), or may also be other general-purpose processors, Digital Signal Processors (DSPs), Application Specific Integrated Circuits (ASICs), Field-Programmable Gate Arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The general-purpose processor may be a microprocessor or the processor may also be any conventional processor, etc.
[0189] The memory 702 may include non-permanent memory in the form of computer-readable media, random access memory (RAM) and / or non-volatile memory, such as read-only memory (ROM) or flash RAM. The memory is an example of computer-readable media.
[0190] The embodiments of the present application also provide a computer-readable storage medium. A computer program is stored on the computer-readable storage medium. When the computer program is executed by a processor, the data processing method described in any one of the above is implemented. The computer includes a personal computer, a tablet computer, a laptop computer, a handheld computer, a mobile phone or a server.
[0191] A computer-readable storage medium can be a computer-readable signal medium, a computer-readable storage medium, or any combination of the two. A computer-readable storage medium can be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, such as a magnetic card, an IC card, a USB flash drive, an SD card, etc. More specific examples of the computer-readable storage medium can include, but are not limited to: an electrical connection having one or more wires, a portable computer disk, a hard disk, a phase change memory (PRAM), a static random access memory (SRAM), a dynamic random access memory (DRAM), other types of random access memory (RAM), a read-only memory (ROM), an electrically erasable programmable read-only memory (EEPROM), a flash memory, or other memory technologies, a compact disc read-only memory (CD-ROM), a digital versatile disc (DVD), an optical storage device, a magnetic storage device, or any combination of the above. A computer-readable storage medium can be any tangible medium that contains or stores a program that can be used by or in conjunction with an instruction execution system, apparatus, or device. A computer-readable signal medium can include a data signal propagated in a baseband or as part of a carrier wave, which carries the computer-readable program code. Such a propagated data signal can take many forms, including but not limited to electromagnetic signals, optical signals, or any combination of the above. A computer-readable signal medium can also be any computer-readable storage medium other than a computer-readable storage medium, which can send, propagate, or transmit a program for use by or in conjunction with an instruction execution system, apparatus, or device. The program code contained on a computer-readable storage medium can be transmitted using any appropriate medium, including but not limited to: wireless, wired, optical fiber cable, radio frequency signal, etc.
[0192] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present application, and are not intended to limit them; although the present application has been described in detail with reference to the foregoing embodiments, those of ordinary skill in the art should understand that they can still modify the technical solutions described in the foregoing embodiments, or perform equivalent replacements on some or all of the technical features; and these modifications or replacements do not make the essence of the corresponding technical solutions deviate from the scope of the technical solutions of the embodiments of the present application.
Claims
1. A data processing method, characterized in that, including: Obtaining the delivery volume of each vehicle model, the total delivery volume of all vehicle models, and the total number of training samples; Substituting the delivery volume of each vehicle model, the total delivery volume of all vehicle models, and the total number of training samples into a vehicle model data extraction formula to obtain the number of training samples for each vehicle model; Extracting the training sample set corresponding to each vehicle model from the original training sample set according to the number of training samples for each vehicle model; Combining the training sample sets corresponding to each vehicle model to obtain a target training sample set; Obtaining a pre-trained language processing model; Training the pre-trained language processing model based on the target training sample set to obtain a question extraction model.
2. The method according to claim 1, characterized in that, The substituting the delivery volume of each vehicle model, the total delivery volume of all vehicle models, and the number of training samples of the target training sample set into a vehicle model data extraction formula to obtain the number of training samples for each vehicle model includes: Calculating the product of the delivery ratio of each vehicle model and the total number of training samples of the target training sample set to obtain the number of training samples for each vehicle model; the delivery ratio of any vehicle model is the ratio of the delivery volume of that vehicle model to the total delivery volume of all vehicle models.
3. The method according to claim 1, wherein The extracting the training sample set corresponding to each vehicle model from the training samples according to the number of training samples for each vehicle model includes: Extracting the training samples corresponding to the first vehicle model in the original training sample set; Sorting the classifications of the training samples corresponding to the first vehicle model according to the usage frequency; Dividing the training samples corresponding to the first vehicle model into a first training sample set and a second training sample set according to the sorting result of the classifications of the training samples corresponding to the first vehicle model; Obtaining a first quantity and a second quantity according to the number of training samples corresponding to the first vehicle model and a preset ratio; Obtaining a first number of training samples from the first training sample set and obtaining a second number of training samples from the second training sample set; Combining the first number of training samples and the second number of training samples to obtain the training sample set corresponding to the first vehicle model.
4. The method according to claim 3, characterized in that, The dividing the training samples corresponding to the first vehicle model into a first training sample set and a second training sample set according to the sorting result of the classifications of the training samples corresponding to the first vehicle model includes: Combining the training samples of the first preset number of classifications in the sorting result among the training samples corresponding to the first vehicle model into the first training sample set; Combining the other training samples among the training samples corresponding to the first vehicle model except the training samples in the first training sample set into the second training sample set.
5. The method according to claim 3, characterized in that, The obtaining a first number of training samples from the first training sample set and obtaining a second number of training samples from the second training sample set includes: Randomly obtaining a first number of training samples from the first training sample set and randomly obtaining a second number of training samples from the second training sample set.
6. The method according to claim 1, wherein After obtaining the question extraction model, the method further includes: Evaluating the quality of the question extraction model based on the bilingual mutual translation quality evaluation algorithm BLEU and / or the evaluation algorithm ROUGE based on recall rate; Optimizing the question extraction model according to the quality evaluation result of the question extraction model.
7. A data processing method, characterized in that, including: Obtain the first session content; Input the first session content into the question extraction model, and obtain the question extraction result corresponding to the first session content output by the question extraction model; Among them, the question extraction model is a model obtained by training a pre-trained language processing model based on a target training sample set. The target training sample set is a training sample set obtained by combining the training sample sets corresponding to each vehicle model. The training sample sets corresponding to each vehicle model are extracted from the original training sample set according to the training sample quantity of each vehicle model. The training sample quantity of each vehicle model is obtained by substituting the delivery volume of each vehicle model, the total delivery volume of all vehicle models, and the total number of training samples of the target training sample set into the vehicle model data extraction formula.
8. The method according to claim 7, wherein, Before inputting the first session content into the question extraction model, the method further includes: Remove the stop words and / or punctuation marks in the first session content.
9. A data processing device, characterized in that, Include: An acquisition module, configured to acquire the delivery volume of each vehicle model, the total delivery volume of all vehicle models, and the total number of training samples; A calculation module, configured to substitute the delivery volume of each vehicle model, the total delivery volume of all vehicle models, and the total number of training samples into the vehicle model data extraction formula to obtain the training sample quantity of each vehicle model; An extraction module, configured to extract the training sample sets corresponding to each vehicle model from the original training sample set according to the training sample quantity of each vehicle model; A combination module, configured to combine the training sample sets corresponding to each vehicle model to obtain the target training sample set; The acquisition module is further configured to acquire a pre-trained language processing model; A training module, configured to train the pre-trained language processing model based on the target training sample set to obtain a question extraction model.
10. A data processing device, characterized in that, Include: An acquisition unit, configured to acquire the first session content; A processing unit, configured to input the first session content into the question extraction model, and obtain the question extraction result corresponding to the first session content output by the question extraction model; Among them, the question extraction model is a model obtained by training a pre-trained language processing model based on a target training sample set. The target training sample set is a training sample set obtained by combining the training sample sets corresponding to each vehicle model. The training sample sets corresponding to each vehicle model are extracted from the original training sample set according to the training sample quantity of each vehicle model. The training sample quantity of each vehicle model is obtained by substituting the delivery volume of each vehicle model, the total delivery volume of all vehicle models, and the total number of training samples of the target training sample set into the vehicle model data extraction formula.
11. An electronic device, characterized in that, Include: A processor, a memory, and a computer program stored on the memory and executable on the processor. When the computer program is executed by the processor, it implements the data processing method according to any one of claims 1 to 8.
12. A computer-readable storage medium, characterized in that, A computer program is stored on the computer-readable storage medium. When the computer program is executed by the processor, it implements the data processing method according to any one of claims 1 to 8.