Data processing method, device, server and computer-readable storage medium

Abstract

This specification provides a data processing method, device, server and computer-readable storage medium. In one embodiment, based on the above data processing method, before implementation, multiple basic problems in the scene can be pre-determined, and corresponding atomic models can be constructed for the basic problems, and corresponding data packets can be configured; during implementation, the target problem in the target scene can be first determined according to the specific data processing involved in the target scene, and then the target atomic model can be matched from the above-mentioned pre-constructed multiple atomic models according to the target problem, and then the model can be trained on the basis of the target atomic model according to the target scene and the pre-configured data packet to obtain a target model suitable for data processing in the target scene. In this way, it is possible to efficiently train and obtain a suitable target model for data processing in multiple target scenes for multiple types of data processing.

Description

Technical Field

[0001] This specification belongs to the field of Internet technology, and in particular, to data processing methods, devices, servers, and computer-readable storage media. Background Art

[0002] For different data processing in different application scenarios, it is usually necessary to build and train corresponding processing models from scratch for each type of data processing in each application scenario in order to perform certain data processing in the scenario.

[0003] Therefore, there is an urgent need for a method that can process various types of data in various target scenarios and efficiently train suitable target models. Summary of the invention

[0004] This specification provides a data processing method, device, server and computer-readable storage medium, so that various types of data can be processed in various target scenarios, and suitable target models can be efficiently trained to improve the training efficiency of the target model.

[0005] A data processing method, device, server, and computer-readable storage medium provided in this specification are implemented as follows:

[0006] A data processing method comprises: determining a target problem in a target scenario; determining a target atomic model matching the target problem from a plurality of atomic models according to the target problem, and acquiring a target data packet corresponding to the target atomic model; wherein the atomic model comprises preset unit models corresponding to basic problems in basic scenarios, and the atomic models are also respectively configured with corresponding data packets; according to the target scenario and the target data packet, training the target atomic model to obtain a target model; wherein the target model is used for data processing in the target scenario.

[0007] A data processing method comprises: determining a target problem according to a target scenario; generating and sending a corresponding atomic model selection request to a server according to the target problem; wherein the atomic model selection request is used to request the server to determine a target atomic model matching the target problem from a plurality of atomic models, wherein the atomic model comprises a preset unit model corresponding to a basic problem in a basic scenario, and the atomic models are also respectively configured with corresponding data packets; receiving the target atomic model fed back by the server, and a target data packet corresponding to the target atomic model; and training the target atomic model according to the target scenario and the target data packet to obtain the target model.

[0008] A data processing method includes: receiving and responding to a processing request from a merchant, determining that a target scenario is a customer service group reply scenario, and a target question is to determine whether a text message posted by a customer in the customer service group requires a manual reply; generating and sending a corresponding model training request to a server according to the target question; wherein the model training request carries the target question and the target scenario; the server is used to respond to the model training request, and determine a target atomic model that matches the target question from multiple atomic models, and a target data packet corresponding to the target atomic model; the server is also used to train the target atomic model according to the target scenario and the target data packet to obtain a target model.

[0009] A data processing device includes: a first determination module, used to determine a target problem in a target scene; a second determination module, used to determine a target atomic model matching the target problem from multiple atomic models according to the target problem, and obtain a target data packet corresponding to the target atomic model; wherein the atomic model includes a preset unit model corresponding to a basic problem in a basic scene, and the atomic model is also respectively configured with a corresponding data packet; a training module, used to train the target atomic model according to the target scene and the target data packet to obtain a target model; wherein the target model is used to perform data processing in the target scene.

[0010] A server comprises a processor and a memory for storing instructions executable by the processor, wherein when the processor executes the instructions, a target problem in a target scenario is determined; according to the target problem, a target atomic model matching the target problem is determined from a plurality of atomic models, and a target data packet corresponding to the target atomic model is obtained; wherein the atomic model comprises preset unit models corresponding to basic problems in basic scenarios, and the atomic models are also respectively configured with corresponding data packets; according to the target scenario and the target data packet, the target atomic model is trained to obtain a target model; wherein the target model is used for data processing in the target scenario.

[0011] A computer-readable storage medium stores computer instructions, which, when executed, determine a target problem in a target scenario; determine a target atomic model that matches the target problem from multiple atomic models based on the target problem, and obtain a target data packet corresponding to the target atomic model; wherein the atomic model includes preset unit models corresponding to basic problems in basic scenarios, and the atomic models are also respectively configured with corresponding data packets; according to the target scenario and the target data packet, train the target atomic model to obtain a target model; wherein the target model is used to perform data processing in the target scenario.

[0012] This specification provides a data processing method, device, server and computer-readable storage medium. Before specific implementation, multiple basic problems are pre-split from basic scenarios with a large coverage range, and then corresponding atomic models are constructed for each basic problem in the basic scenario, and corresponding data packets are configured for each atomic model; during specific implementation, the target problem in the target scenario can be determined first, and then a suitable target atomic model is matched from the multiple atomic models pre-constructed according to the target problem, and a pre-configured target data packet corresponding to the target atomic model is obtained; and then, according to the target scenario and the target data packet, on the basis of the target atomic model, model training related to the target scenario can be performed to obtain a target model suitable for specific data processing in the target scenario, so as to use the model to perform data processing in the target scenario. Thus, it is possible to efficiently train and obtain applicable target models for multiple types of data processing in multiple target scenarios, thereby improving the training efficiency of the target model. BRIEF DESCRIPTION OF THE DRAWINGS

[0013] In order to more clearly illustrate the embodiments of this specification, the drawings required for use in the embodiments will be briefly introduced below. The drawings described below are only some embodiments recorded in this specification. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative labor.

[0014] Figure 1 It is a schematic diagram of an embodiment of a system structure composed of a data processing method provided by an embodiment of this specification;

[0015] Figure 2 It is a schematic diagram of an embodiment of applying the data processing method provided by the embodiments of this specification in a scenario example;

[0016] Figure 3 It is a schematic diagram of an embodiment of applying the data processing method provided by the embodiments of this specification in a scenario example;

[0017] Figure 4 It is a schematic diagram of an embodiment of applying the data processing method provided by the embodiments of this specification in a scenario example;

[0018] Figure 5 It is a schematic diagram of an embodiment of applying the data processing method provided by the embodiments of this specification in a scenario example;

[0019] Figure 6 It is a schematic diagram of an embodiment of applying the data processing method provided by the embodiments of this specification in a scenario example;

[0020] Figure 7 is a flowchart of a data processing method provided by an embodiment of this specification;

[0021] Figure 8 is a flowchart of a data processing method provided by another embodiment of the present specification;

[0022] Fig. 9 It is a schematic diagram of the structure of a server provided by an embodiment of this specification;

[0023] Fig.10 It is a schematic diagram of the structural composition of a data processing device provided by an embodiment of this specification. DETAILED DESCRIPTION

[0024] In order to enable those skilled in the art to better understand the technical solutions in this specification, the technical solutions in the embodiments of this specification will be clearly and completely described below in conjunction with the drawings in the embodiments of this specification. Obviously, the described embodiments are only part of the embodiments of this specification, not all of the embodiments. Based on the embodiments in this specification, all other embodiments obtained by ordinary technicians in this field without creative work should fall within the scope of protection of this specification.

[0025] The embodiment of this specification provides a data processing method, which can be applied to a system including a server and a terminal device. Figure 1 Different terminal devices can be connected to the server via wired or wireless means to perform specific data interaction.

[0026] Before specific implementation, the server can separate multiple basic problems from the basic scenario (for example, a scenario with a relatively wide coverage range) in advance, and construct corresponding atomic models for the above multiple basic problems to obtain multiple atomic models. At the same time, the server will also configure data packets for using the atomic models for the above multiple atomic models.

[0027] In specific implementation, facing different types of data processing involved in different target scenarios, multiple terminal devices (for example, terminal device 1, terminal device 2...terminal device n, etc.) can respectively initiate model training requests to the server to request the server to train a target model suitable for the target scenario.

[0028] The server can first receive and respond to each model training request to determine the target problem in each target scenario faced by each terminal device. Further, the server can determine the atomic model that matches the target problem from multiple atomic models as the target atomic model based on the target problem, and obtain the data packet corresponding to the target atomic model as the target data packet. Then, based on the target scenario and the target data packet, model training is performed on the basis of the target atomic model, so that the target model (for example, target model 1, target model 2...target model n) suitable for specific data processing in the target scenario faced by each terminal device can be trained quickly and efficiently.

[0029] The server may publish the above-mentioned multiple trained target models to the Internet, or send the above-mentioned multiple trained target models to corresponding terminal devices respectively.

[0030] In this way, the above-mentioned multiple terminal devices can easily and efficiently acquire and use the above-mentioned target model to perform specific data processing in the corresponding target scene.

[0031] In this embodiment, the server may specifically include a server that is applied to one side of the network platform and is responsible for data processing in the background that can realize functions such as data transmission and data processing. Specifically, the server may be, for example, an electronic device with data calculation, storage and network interaction functions. Alternatively, the server may also be a software program running in the electronic device that provides support for data processing, storage and network interaction. In this embodiment, the number of the servers is not specifically limited. The server may specifically be one server, or several servers, or a server cluster formed by several servers.

[0032] In this embodiment, the terminal device may specifically include a front-end device applied to the user side and capable of realizing functions such as data collection and data transmission. Specifically, the terminal device may be, for example, a desktop computer, a tablet computer, a laptop computer, a smart phone, etc. Alternatively, the terminal device may also be a software application that can be run in the above electronic devices. For example, it may be an application program running on a computer, etc.

[0033] For a specific scenario example, see Figure 2 As shown, the data processing method provided in the embodiments of this specification can be applied to receive and respond to the user's model training request, train and feed back the corresponding target model to the user, so that the user can efficiently obtain and use the target model to perform specific data processing in the target scenario.

[0034] In this scenario example, the server may be a server of a network platform that provides model training services. Specifically, the server may be connected to a first terminal device and a second terminal device. The first terminal device may be arranged on the side of an algorithm-side technician who is responsible for building an atomic model. The second terminal device may be arranged on the side of a front-end technician or a customer (hereinafter collectively referred to as a user) who needs to build or use a specific model.

[0035] The atomic model may specifically include a pre-built unit model corresponding to a basic problem in a basic scenario. Specifically, the atomic model may be understood as a minimum model structure that contains semantics, has been pre-trained, and can solve a corresponding basic problem.

[0036] Among them, the above-mentioned basic scenarios may specifically include a scenario with a wide coverage range (for example, the coverage range is greater than a preset coverage range threshold) and has good applicability and reference for a variety of specific scenarios. The above-mentioned basic problem can be specifically understood as an abstract summary of the relevant data processing involved in the basic scenario, and the extracted data processing problem with a certain representativeness and coverage, and with the common characteristics of processing problems belonging to a certain type. Usually a basic problem can be applicable to target problems in multiple different target scenarios at the same time.

[0037] It should be noted that the above-mentioned atomic model is different from the basic model or the initial model (for example, there is no combined scenario, and there is no trained initial binary classification model). The above-mentioned basic model or initial model usually does not contain any semantics; when training a model based on the above-mentioned basic model or initial model, it is necessary to build and train a new model from scratch in combination with the scenario. The above-mentioned atomic model is a model structure that has been established after a certain degree of learning and training on a basic problem in the basic scenario, which contains certain semantics and can solve a basic problem in the scenario. Of course, the above-mentioned atomic model is not equivalent to the real model. Although the above-mentioned atomic model already contains certain semantics, more attention is paid to factors such as the algorithm type and model structure used for the basic problem in the corresponding basic scenario.

[0038] In this scenario example, before specific implementation, refer to Figure 3 As shown, the technical staff on the algorithm side can first separate multiple basic problems from the basic scenario based on the basic scenario; and then build corresponding multiple atomic models for the above multiple basic problems.

[0039] Taking the customer service scenario as an example, the technician can use the first terminal device to process the data that often appears in the customer service scenario (which can be understood as a basic scenario that covers customer service scenarios of multiple stores with different product types, for example), abstract multiple related issues; and then perform data processing such as classification and induction on the above multiple related issues, so that multiple basic issues can be separated from the customer service scenario. For example, it can determine whether the text message sent by the customer contains sensitive words, whether the text message sent by the customer needs to be manually replied, or which type of automatic reply the text message sent by the customer belongs to, etc.

[0040] Through the above processing, the first terminal device can abstract multiple specific data processings that often appear in customer service scenarios into corresponding multiple basic questions. Among them, one basic question can correspond to one or more specific data processings in the customer service scenario.

[0041] Furthermore, the first terminal device can obtain historical data related to the customer service scenario as the first type of sample data by accessing the historical database of the network platform. For example, the chat records in the customer service group of different commodity type stores on a shopping website in history can be obtained as the first type of text data.

[0042] Next, the first terminal device may construct, based on the first type of sample data, atomic models corresponding to the multiple basic problems, respectively, which can solve the basic problems to a certain extent.

[0043] The following takes the construction of atomic model No. 1 for basic problem A as an example to explain how the first terminal device specifically constructs the atomic model. For the construction method of atomic models for other basic problems, it can be implemented by referring to the construction method of atomic model No. 1 for basic problem A, and this manual will not repeat it.

[0044] In specific implementation, the first terminal device may first select an algorithm component suitable for solving the basic problem A from existing algorithm components according to the basic problem A.

[0045] Specifically, the first terminal device may first determine the activity flow for solving the basic problem A based on the basic problem A and the relevant historical processing experience. Then, based on the activity flow, the data processing events involved in the activity flow and the execution order of the data processing events may be determined. Then, based on the data processing events and the execution order of the data processing events, the corresponding algorithm components may be selected and constructed.

[0046] For example, basic question A is to determine whether a text message sent by a customer requires a manual reply. Based on processing experience, basic question A can be analyzed and further determined as a classification problem, that is, to determine whether the type of text message sent by a customer requires a manual reply or does not require a manual reply. Then, from the existing algorithm components, for example, a binary classification algorithm component can be selected as the algorithm component for constructing the corresponding atomic model.

[0047] Of course, the above-mentioned method of constructing the algorithm component of the atomic model is only a schematic illustration. In specific implementation, according to the specific situation and processing requirements, other suitable methods can also be used to construct the corresponding algorithm component. For example, it is also possible to design a new algorithm as the corresponding algorithm component separately for the basic problem A through the first terminal device.

[0048] While constructing the algorithm component, the first terminal device may also perform standardization processing on the first type of sample data according to a preset standardization rule. The preset standardization rule may specifically include a standard format of data used to construct the atomic model. For example, the first terminal device may convert the data format of the first type of sample data into a unified preset data format defined by the preset standardization rule according to the preset standardization rule as the standardized first type of sample data.

[0049] Then, according to basic question A, the standardized first category sample data is labeled accordingly to obtain labeled first category sample data. Then, the labeled first category sample data can be used to train the algorithm components to obtain a model that meets the requirements and can be used to solve basic question A, as atomic model No. 1.

[0050] Among them, the above-mentioned atomic model No. 1 is obtained by training based on the first type of sample data after standardization. Therefore, the above-mentioned atomic model No. 1 has good versatility. In this way, it is relatively convenient to use other data in a unified data format obtained by processing based on preset standardization rules to directly perform secondary training on the above-mentioned atomic model No. 1.

[0051] It should be noted that the construction process of the above atomic model will focus more on the selection and training of the algorithm level. Therefore, the resulting atomic model often has an algorithm structure that can better solve basic problems. However, the above atomic model pays relatively less attention to the details of the scene during the construction process. Therefore, if the above atomic model is directly applied to specific data processing in a target scene, the effect is often not ideal.

[0052] In this scenario example, while constructing the above-mentioned atomic model No. 1, the first terminal device also configures the corresponding data packet for using atomic model No. 1 for the atomic model.

[0053] Specifically, the first terminal device can obtain and save the standardized first-category sample data used in training atomic model No. 1, and / or the standardized first-category sample data that is not used but is suitable for training or evaluating the atomic model as a sample set. The first terminal device can also save and establish a corresponding feature set based on the model parameters, model parameter weight values, and other data generated or used in the process of training atomic model No. 1.

[0054] At the same time, the first terminal device can also establish a training script file for training atomic model No. 1 (for example, training rules, how to process training sample data during training, model parameters used in the training process, etc.), an evaluation script file for evaluating atomic model No. 1 (for example, evaluation rules, evaluation indicators used in the evaluation process, etc.), and deployment environment information for deploying the operating environment of atomic model No. 1 (for example, CPU parameters and MEM parameters that atomic model No. 1 relies on when running) based on the training record data of atomic model No. 1 (for example, training logs, etc.), model features of atomic model No. 1, the memory environment for training atomic model No. 1, and the program code used when training atomic model No. 1.

[0055] In addition, the first terminal device may also save the algorithm components of atomic model No. 1, and / or, when subsequently training atomic model No. 1, may further expand the use of other associated algorithm components to establish an algorithm component library of atomic model No. 1.

[0056] Furthermore, one or more of the algorithm component library, training script file, evaluation script file, deployment environment information, sample set and feature set of atomic model No. 1 obtained in the above manner can be combined and packaged to obtain a data packet corresponding to atomic model No. 1. And establish a corresponding relationship between atomic model No. 1 and the data packet. For example, an identity for indicating atomic model No. 1 can be set on the data packet. Subsequently, the data packet corresponding to atomic model No. 1 can be found by retrieving the identity. Thereby, the corresponding data packet is configured for atomic model No. 1.

[0057] According to the above method, the first terminal device can construct multiple atomic models corresponding to multiple basic problems in the basic scenario, and configure corresponding data packets for the multiple atomic models, and then send the atomic models and data packets to the server.

[0058] The server receives and stores the above-mentioned multiple atomic models and data packets in a database of the network platform.

[0059] In this scenario example, when the user needs to train a suitable model for specific data processing in a specific target scenario, the user can interact with the server through the second terminal device and initiate a model training request so as to efficiently train a target model suitable for data processing in the target scenario, and then use the target model to process data in the target scenario.

[0060] Specifically, for example, Merchant A is a merchant of a clothing store on a shopping website. Merchant A owns and manages the customer service group of the clothing store. In the customer service group, customers can post some text messages to interact with Merchant A.

[0061] For example, a customer can post a text message in the customer service group to ask Merchant A whether the purchased clothing can be shipped free of charge, or ask Merchant A about the size of the clothing, or report to Merchant A that the received clothing has defects and ask Merchant A to provide corresponding solutions, etc.

[0062] There are many text messages in the above customer service group every day. Among them, some text messages can be automatically replied to without reminding merchant A directly using the reply text preset by merchant A, while other text messages need to remind merchant A to reply manually.

[0063] Currently, Merchant A hopes to train a target model that can automatically detect and identify text messages in the customer service group of his clothing store that require manual responses (i.e., specific data processing in a target scenario). This allows the target model to automatically identify text messages in the customer service group that require manual responses, so as to perform targeted response processing and reduce Merchant A's workload.

[0064] In specific implementation, Merchant A can first determine the corresponding target question through the second terminal device according to the above target scenario: determine whether the text information in the customer service group of the clothing store is a text information that requires a manual reply. Further, a corresponding model training request can be generated according to the above target question. The above model training request can carry the above target question. The second terminal device sends the above model training request to the server.

[0065] See also Figure 4 As shown, the server receives and determines the target problem according to the model training request. Then, the server can search the database that stores multiple atomic models according to the target problem, and find the basic problem B that is the same as the target problem or has a higher degree of similarity as the target problem as the matching problem. The atomic model No. 2 corresponding to the matching basic problem B is searched from the database as the target atomic model. At the same time, according to the identity information of the atomic model No. 2, the pre-configured data packet corresponding to the atomic model No. 2 is retrieved as the target data packet.

[0066] Furthermore, the server may first deploy a model operating environment suitable for atomic model No. 2 according to the deployment environment information in the target data packet. At the same time, the server may also generate and send a model training setting interface for atomic model No. 2 to the second terminal device. The second terminal device may display the above model training setting interface to merchant A.

[0067] The model training setting interface may specifically include a plurality of setting items for model training setting parameters. The model training setting parameters may specifically include parameter data for user-defined settings that affect model training. Users can customize a model training solution that meets their needs by setting corresponding model training setting parameters in the model training setting interface.

[0068] Specifically, the above-mentioned model training setting parameters may include one or more of the following: the type of training method, the type of training cycle, the training target, the model parameters, etc.

[0069] For example, the user can customize whether the model training process is incremental training or full training by setting the type of training method according to the specific target scenario. For another example, the user can customize whether the model training process is single training or periodic training by setting the type of training cycle in the model training setting interface. If the user chooses periodic training, the periodic training setting box can be further displayed in the above model training setting interface, and the user can flexibly set the training task cycle, the start time and end time of each cycle of the periodic training through the setting box according to the specific situation. For another example, the user can also customize the conditions for the end of model training by setting the training goal; or, by setting the model parameters, the model structure, model constraints, training features, etc. in the model training process can be flexibly adjusted. Of course, it should be noted that the model training setting parameters listed above are only a schematic description. In specific implementation, according to the specific situation and processing requirements, other types of parameters other than the parameters listed above can also be introduced as model training setting parameters. This specification does not limit this.

[0070] In addition, the above model training setting interface may further include a sample data import interface. Users can flexibly access their own sample data related to the target scene through the above import interface as needed to participate in the model training. In this way, through the above import interface, users can flexibly import sample data that is more effective or closer to the target scene to participate in the model training and obtain better training results.

[0071] In this scenario example, Merchant A can set the type of training mode to full, the type of training cycle to periodic training, and the training task cycle to 9:00 to 12:00 every Monday through the model training setting interface displayed by the second terminal device. At the same time, Merchant A can also use the model training setting interface to import sample data into the interface to connect to his own store customer service group, so that he can regularly obtain the chat records of the customer service group in the past week as sample data (recorded as the second type of sample data) every Monday, and then use the above sample data to regularly train and update the model to continuously improve the accuracy of the model.

[0072] Correspondingly, the server can receive the model training setting parameters set by merchant A and the second type of sample data provided by merchant A through the model training setting interface displayed by the second terminal device. Furthermore, the server can train atomic model No. 2 in the constructed operating environment according to the above model training setting parameters, the second type of sample data, and the pre-configured data package to obtain the target model that meets the requirements of merchant A.

[0073] Of course, Merchant A may not perform any setting operation in the model training setting interface. Accordingly, the server may perform model training on the No. 2 atomic model according to the preset model training setting parameters as default parameters.

[0074] During specific implementation, the server can also determine the target features based on the target problem in the target scenario. Of course, the features determined by the server can be further screened in combination with the model training setting parameters set by Merchant A to obtain more accurate target features. At the same time, the server can standardize the second-category sample data provided by Merchant A according to the sample set in the data packet, unify the data format of the newly accessed second-category sample data, and obtain standardized second-category sample data. The above-mentioned standardized second-category sample data and the standardized first-category sample data extracted from the sample set are combined to obtain training sample data for training the model. The above-mentioned training sample data is then labeled according to the target features to obtain labeled training sample data.

[0075] Furthermore, the server may perform model training on atomic model No. 2 using the training sample data after the above-mentioned marking process according to the training script file in the data packet in the operating environment to obtain the trained atomic model No. 2.

[0076] It should be noted that the training process of the above model, unlike the construction process of the atomic model, pays more attention to the details of the target scene, as well as the actual needs and usage experience of the user when using the model to process specific data in the target scene. Therefore, the trained atomic model No. 2 obtained by the above method is closer to the specific data processing in the target scene than the untrained atomic model No. 2. For example, in actual use, in addition to having higher accuracy, it will further show higher processing efficiency and relatively better user experience, which can better meet the actual needs of front-end users or technicians.

[0077] After training, refer to Figure 5 The server shown can also obtain sample data for evaluation from the sample set, and / or extract sample data for evaluation from the standardized second sample data as evaluation sample data. And according to the evaluation script file in the data packet, the trained atomic model No. 2 is specifically evaluated using the above evaluation sample data, and the corresponding evaluation index parameters are collected to obtain the corresponding evaluation results. A corresponding evaluation report is generated according to the evaluation results. Among them, the above evaluation report can specifically include evaluation index parameters such as coverage and accuracy.

[0078] Of course, before the evaluation, the server can also receive the model evaluation setting parameters customized by merchant A through the second terminal device. Accordingly, the server can evaluate the trained atomic model No. 2 in combination with the above model evaluation setting parameters to obtain a corresponding evaluation report.

[0079] The server can determine whether the currently trained atomic model No. 2 meets the preset requirements based on the evaluation index parameters in the above evaluation report. For example, it can be determined whether the trained atomic model No. 2 meets the preset requirements by determining whether the coverage of the trained atomic model No. 2 is greater than a preset coverage threshold, and / or determining whether the accuracy of the trained atomic model No. 2 is greater than a preset accuracy threshold, etc.

[0080] If it is determined in the above manner that the currently trained atomic model No. 2 meets the preset requirements, the training can be terminated and the currently trained atomic model No. 2 can be determined as the target model to be delivered to Merchant A for use.

[0081] If it is determined in the above manner that the currently trained atomic model No. 2 does not meet the preset requirements, secondary model training can be continued in the above manner based on the currently trained atomic model No. 2 until the trained model finally meets the preset requirements.

[0082] Since the above target model is based on the pre-prepared atomic model No. 2 that contains semantics and can solve the corresponding basic problems, the model is trained using the pre-configured data package combined with the data and parameters subsequently input by Merchant A. Therefore, the server does not need to re-search and build appropriate algorithm components from scratch, nor does it need to re-generate and configure relevant data such as the model's deployment environment information, training script files, and evaluation script files, which reduces the complexity of model training and the amount of data processing on the server, so that the target model suitable for data processing in the target scenario faced by Merchant A can be efficiently trained, thereby improving the training efficiency of the target model.

[0083] In addition, based on the above training method, the server performs secondary training on the basis of the atomic model to obtain the model. Therefore, the amount of data processing of the server when training the model can be effectively reduced, so that the server is able to simultaneously access and respond to more model training requests, and determine multiple matching target atomic models and corresponding data packets from multiple preset atomic models; and then, multiple models can be trained simultaneously based on the above multiple target atomic models and corresponding data packets to obtain multiple target models for data processing corresponding to different target scenarios.

[0084] In this scenario example, after the server has trained the target model that meets the preset requirements and is suitable for data processing in the target scenario of Merchant A in the above manner, refer to Figure 6 As shown, the server can also perform corresponding configuration processing on the above-mentioned target model, and then provide the configured target model to the second terminal device, so that merchant A can directly access and use the target model more conveniently, thereby further simplifying merchant A's operations and improving merchant A's user experience.

[0085] Specifically, the server can configure the corresponding model input interface, model output interface, model operating environment and related protocols for the target model to obtain the configured target model, and publish the configured target model on the network platform, and at the same time feed back the call link of the configured target model to the second terminal device.

[0086] Merchant A can directly call the configured target model based on the above call link through the second terminal device. Furthermore, it is also possible to easily connect the customer service group of the clothing store to the input interface of the configured model and connect the second terminal device to the output interface of the model.

[0087] In this way, at every preset time interval (for example, 5 minutes), the target model can automatically obtain the text information (i.e., target data) posted by customers to be identified and processed in the customer service group within the last 5 minutes through the above-mentioned model input interface, and run the model to identify and process the above-mentioned text information to determine whether there is text information that requires Merchant A to be reminded to make a manual reply, and obtain the corresponding recognition result as the model output.

[0088] The second terminal device can determine whether there is text information in the current customer service group that requires manual response from merchant A based on the recognition result output by the model. When it is determined that there is text information that requires manual response from merchant A, a prompt message or a prompt tone can be generated to remind merchant A that there is a text message sent by a customer in the current customer service group that requires manual response from the merchant himself, so that merchant A does not need to spend a lot of time and energy to judge and identify text messages posted by a large number of customers, effectively reducing the workload of merchant A, while ensuring that merchant A can promptly discover and respond to text messages posted by customers in the customer service group that require manual response.

[0089] See also Figure 7 As shown, the embodiment of this specification provides a data processing method. Among them, the method is specifically applied to the server side. When implemented specifically, the method may include the following contents.

[0090] S701: Determine a target problem in a target scenario.

[0091] In some embodiments, the above target scenario can specifically be a specific data processing scenario that the front-end user or technician is concerned about. Specifically, the above target scenario can be a reply scenario of a certain store customer service group, or a processing scenario of order data of a certain shopping website, or an identification scenario of transaction risks on a certain trading platform, etc. Of course, the target scenarios listed above are only a schematic illustration. During specific implementation, the above target scenarios can also include other types of scenarios according to specific circumstances and processing requirements. This specification does not limit this.

[0092] In some embodiments, the above-mentioned target problem can specifically be one or more data processing problems determined by abstracting and summarizing the relevant data processing involved in the target scenario of concern based on the specific needs of the front-end user or technical personnel.

[0093] For example, in a customer service group response scenario of a certain store, the front-end technician needs to build a model that can automatically identify the text messages posted by customers in the customer service group of the store that require manual responses from customer service. At this time, the target problem for this target scenario can be determined as: determine whether the type of text messages posted by customers in the customer service group of the store is the type that requires manual responses from customer service or the type that does not require manual responses from customer service.

[0094] In some embodiments, the server can determine the data processing problem in the target scenario as the target problem to be solved by the model to be trained based on the model training requirements of users or technicians on the technical side and the relevant data processing involved in the target scenario.

[0095] Alternatively, a second terminal device of a user or technician deployed at the technical end may sort out and analyze the relevant data processing involved in the target scenario according to the needs of the user or technician at the technical end, determine the corresponding target problem, and then generate and send a model training request carrying the target problem to the server. The server receives the above model training request and obtains the target problem carried by the model training request through data analysis.

[0096] S702: According to the target problem, determine a target atomic model that matches the target problem from multiple atomic models, and obtain a target data packet corresponding to the target atomic model; wherein the atomic model includes preset unit models corresponding to basic problems in basic scenarios, and the atomic models are also respectively configured with corresponding data packets.

[0097] In some embodiments, the atomic model may specifically include a pre-built unit model corresponding to a basic problem in a basic scenario. Specifically, the atomic model may be understood as a minimum model structure that contains semantics, has been pre-trained, and can solve a corresponding basic problem.

[0098] Among them, the above-mentioned basic scenarios may specifically include a scenario with a wide coverage range (for example, the coverage range is greater than a preset coverage range threshold) and has good applicability and reference for a variety of specific scenarios. The above-mentioned basic problem can be specifically understood as an abstract summary of the relevant data processing involved in the basic scenario, and the extracted data processing problem with a certain representativeness and coverage, and with the common characteristics of processing problems belonging to a certain type. Usually a basic problem can be applicable to target problems in multiple different target scenarios at the same time.

[0099] In some embodiments, the above-mentioned data packet can be specifically understood as a pre-configured data packet corresponding to a certain atomic model, which contains data related to the use of the atomic model. Specifically, the above-mentioned data packet may include one or more of the following listed data: an algorithm component library of a target atomic model, a training script file of a target atomic model, an evaluation script file of a target atomic model, deployment environment information of a target atomic model, a sample set of a target atomic model, a feature set of a target atomic model, etc. Of course, it should be noted that the data in the above-mentioned data packets are only a schematic illustration. During specific implementation, according to the specific circumstances and usage rules of the atomic model, the above-mentioned data packet may also contain other types of data. This specification does not limit this.

[0100] In some embodiments, before the specific implementation, the technicians on the algorithm side can split the basic scenario into multiple basic problems through the first terminal device. Then, for the above multiple basic problems, the corresponding atomic models are constructed respectively to obtain multiple atomic models, and the above multiple atomic models are configured with data packets using the atomic models. And the corresponding relationship between the atomic model and the data packet is established. For example, an identity identifier for indicating the corresponding atomic model is set on the data packet.

[0101] In some embodiments, during specific implementation, the target problem may be firstly used to find a basic problem that is the same or has a high degree of similarity as the matching basic problem, and then an atomic model corresponding to the matching basic problem may be found from multiple atomic models as the target atomic model. Then, according to the correspondence between the atomic model and the data packet, a pre-configured data packet corresponding to the target atomic model may be found as the target data packet.

[0102] In some embodiments, the above-mentioned target data packet may specifically include at least one of the following: an algorithm component library of the target atomic model, a training script file of the target atomic model, an evaluation script file of the target atomic model, deployment environment information of the target atomic model, a sample set of the target atomic model, a feature set of the target atomic model, etc.

[0103] In some embodiments, when the data processing involved in the target scenario is relatively complex, in specific implementation, multiple matching basic questions can be found simultaneously according to the target question in the target scenario. In this case, the target question can be a combination of multiple basic questions.

[0104] In this case, according to the target problem, multiple atomic models corresponding to the multiple basic problems included in the target problem can be found as the target atomic model. Subsequently, the target atomic model including the multiple atomic models can be trained to obtain the corresponding target model.

[0105] S703: Training the target atomic model according to the target scene and the target data packet to obtain a target model; wherein the target model is used to perform data processing in the target scene.

[0106] In some embodiments, the above-mentioned target model can be specifically understood as a complete model structure that has been trained and can be used for specific data processing in the target scenario and can effectively solve specific target problems in the target scenario.

[0107] For example, for a response scenario of a customer service group of a certain store, the target model may be a model that is well suited to the scenario and can accurately identify text messages posted by customers in the customer service group of the store that require manual responses from customer service.

[0108] In some embodiments, the target atomic model is trained according to the target scene and the target data packet to obtain the target model. When implemented specifically, the following contents may be included: acquiring data in the target scene as second category sample data; standardizing the second category sample data according to the target data packet to obtain standardized second category sample data; and training the target atomic model according to the target data packet and the standardized second category sample data.

[0109] In some embodiments, during specific implementation, the standardized first-category sample data contained in the sample set in the target data packet can be used as a reference, or the second-category sample data can be standardized accordingly according to the processing rules for sample data contained in the target data packet, so that the data format of the newly accessed second-category sample data can be unified into a data format that matches the target atomic model, so as to facilitate the subsequent more efficient use of the standardized second-category sample data to directly train the target atomic model, improve the training efficiency during subsequent training, and reduce the errors caused by the inconsistency of sample data during subsequent training.

[0110] In some embodiments, the training of the target atomic model according to the target data packet and the standardized second-category sample data may include the following when implemented: extracting the standardized first-category sample data from the sample set according to the target data packet; combining the standardized first-category sample data and the standardized second-category sample data to obtain training sample data; and training the target atomic model according to the target data packet using the training sample data. In this way, the data characteristics of the first-category sample data used to construct the atomic model and the second-category sample data close to the target scene can be comprehensively utilized to train a target model with better use effect.

[0111] In some embodiments, during specific implementation, depending on the specific situation, the standardized first-category sample data or the standardized second-category sample data can be used alone as training sample data to train the target atomic model to obtain the corresponding target model.

[0112] In some embodiments, the target atomic model is trained using the training sample data according to the target data packet. When implemented specifically, the following contents may also be included: according to the target scenario, the training sample data is labeled to obtain the labeled training sample data; according to the training script file in the target data packet, the target atomic model is trained using the labeled training sample data.

[0113] In some embodiments, before specific training, corresponding operating resources (such as CPU resources, etc.) can be allocated to the target atomic model according to the environment deployment information of the target atomic model in the target data packet; and then a matching operating environment can be deployed for the target atomic model based on the above operating resources, so that the target atomic model can be efficiently trained directly in the deployed operating environment later.

[0114] In some embodiments, according to the algorithm components on which some different target atomic models are based, the training sample data may not be labeled, and the training sample data may be directly input into the target atomic model for model training.

[0115] In some embodiments, in order to further meet the personalized needs of front-end users or technicians, the method may also include the following contents when implemented: receiving model training setting parameters; training the target atomic model using the labeled training sample data according to the training script file and the model training setting parameters.

[0116] In some embodiments, the above-mentioned model training setting parameters may specifically include parameter data for user-defined settings that affect model training.

[0117] In some embodiments, the model training setting parameters may specifically include at least one of the following: the type of training method, the type of training cycle, the training target, the model parameters, etc. Of course, it should be noted that the model training setting parameters listed above are only a schematic illustration. During specific implementation, according to specific circumstances and processing requirements, the above model training setting parameters may also include other types of parameter data. This specification does not limit this.

[0118] In some embodiments, during specific implementation, the server may display a model training setting interface to the front-end technician or user through the second terminal device. The front-end technician or user may set corresponding model training setting parameters through the above model training setting interface to customize a model training solution that meets their own needs. The server may receive the above model training setting parameters through the above model training setting interface, and subsequently may train the atomic model in accordance with the needs of the front-end technician or user in combination with the above model training setting parameters, thereby further meeting the user's diverse training needs.

[0119] In some embodiments, after the target atomic model is trained according to the training script file in the target data packet using the marked training sample data, the method may also include the following contents when implemented: performing model evaluation on the trained target atomic model according to the evaluation script file in the target data packet to determine the evaluation result for the trained target atomic model; determining whether the trained target atomic model meets the preset requirements based on the evaluation result; and determining the trained target atomic model as the target model when it is determined that the trained target atomic model meets the preset requirements.

[0120] In some embodiments, the above-mentioned evaluation report may specifically include evaluation index parameters such as coverage, accuracy, etc. Correspondingly, according to the evaluation results, determining whether the trained target atomic model meets the preset requirements, when specifically implemented, may include: determining the evaluation index parameters of the trained target atomic model according to the evaluation report; comparing the preset evaluation index parameter threshold with the evaluation index parameters of the trained target atomic model to obtain the corresponding comparison result; determining whether the trained target atomic model meets the preset requirements according to the comparison result.

[0121] For example, when it is determined that the coverage of the trained target atomic model is greater than a preset coverage threshold and the accuracy is greater than a preset accuracy threshold, it is determined that the currently trained target atomic model meets the preset requirements. At this time, the model training is stopped and the currently trained target atomic model is determined as the target model.

[0122] In some embodiments, after determining whether the trained target atomic model meets the preset requirements based on the evaluation results, the method may also include the following contents when implemented: when it is determined that the trained target atomic model does not meet the preset requirements, training the trained target atomic model according to the training script file in the target data packet.

[0123] For example, when it is determined that the coverage of the trained target atomic model is less than or equal to a preset coverage threshold, or the accuracy is less than or equal to a preset accuracy threshold, it is determined that the currently trained target atomic model does not meet the preset requirements. At this time, the model training will not be stopped. Instead, the currently trained target atomic model will be used as a basis to repeat the above training process and continue model training until the trained target atomic model meets the preset requirements.

[0124] In some embodiments, the model evaluation is performed on the trained target atomic model according to the evaluation script file in the target data packet. When implemented specifically, it may include the following contents: receiving model evaluation setting parameters; performing model evaluation on the trained target atomic model according to the evaluation script file in the target data packet and the model evaluation setting parameters.

[0125] In some embodiments, the above-mentioned model evaluation setting parameters may specifically include parameter data for users to customize and set, which affects the model evaluation.

[0126] In some embodiments, the model evaluation setting parameters may specifically include at least one of the following: the type of evaluation method, evaluation indicator parameters, evaluation cycle, etc. Of course, it should be noted that the model evaluation setting parameters listed above are only a schematic illustration. During specific implementation, according to specific circumstances and processing requirements, the above model evaluation setting parameters may also include other types of parameter data. This specification does not limit this.

[0127] In some embodiments, during specific implementation, the server may display a model evaluation setting interface to the front-end technician or user through the second terminal device. The front-end technician or user may set corresponding model evaluation setting parameters through the above model evaluation setting interface to customize a model evaluation solution that meets their own needs. The server may receive the above model evaluation setting parameters through the above model evaluation setting interface, and may subsequently evaluate the atomic model in accordance with the needs of the front-end technician or user in combination with the above model evaluation setting parameters to meet the user's diverse evaluation needs.

[0128] In some embodiments, after obtaining a target model that meets the preset requirements by training the target atomic model in the above manner, the method may further include the following when implemented: performing configuration processing on the target model according to the target scenario to obtain a configured target model; wherein the configuration processing includes at least one of the following: configuration of the input interface of the model, configuration of the output interface of the model, configuration of the operating environment of the model, etc. The configured target model is published.

[0129] In some embodiments, the configuration processing listed above is only a schematic illustration. In specific implementation, according to specific circumstances and processing requirements, the above configuration processing may further include other types of configuration processing. For example, the configuration of the data processing protocol involved in the model operation, the configuration of the model operation parameters, etc. This specification does not limit this.

[0130] In some embodiments, after publishing the configured target model, the method may also include the following when implemented: receiving and responding to a user's call request for the target model, performing data processing on the target data related to the target scene input by the user using the target model to obtain corresponding target data processing results; and feeding back the target data processing results.

[0131] In some embodiments, the server can publish the configured target model to the Internet or send it to the second terminal device in the above manner, so that the front-end user or technician can directly call the configured target model to process the data in the target scene, which simplifies the operation of the front-end technician or user and improves the user experience.

[0132] In some embodiments, for a more complex target scenario, the target problem may specifically include a combination of multiple basic problems. Accordingly, the target atomic model may include multiple atomic models corresponding to the multiple basic problems.

[0133] In some embodiments, for the above-mentioned more complex target scenarios, the above-mentioned method may also include the following contents when implemented: according to the target problem, multiple target atomic models and corresponding multiple target data packets are obtained; wherein each target atomic model in the multiple target atomic models is matched with one basic problem in the multiple basic problems respectively; according to the target scenario and the multiple target data packets, the multiple target atomic models are trained respectively to obtain multiple sub-models; and the multiple sub-models are combined to obtain a target model.

[0134] In some embodiments, before specific implementation, a first terminal device deployed on the side of the technician at the algorithm end can be used to pre-build multiple corresponding atomic models based on multiple basic problems separated from the basic scenario.

[0135] In some embodiments, the atomic model can be established specifically in the following manner: splitting multiple basic problems from a basic scenario; wherein the basic scenario includes a scenario whose coverage is greater than a preset coverage threshold; obtaining a first type of sample data from the basic scenario; and establishing atomic models corresponding to the multiple basic problems in the basic scenario based on the first type of sample data.

[0136] In some embodiments, the basic scene may specifically be the same scene as the target scene, or a scene that is similar to or related to the target scene.

[0137] In some embodiments, the first type of sample data may specifically be data collected in basic scenarios.

[0138] In some embodiments, the above-mentioned establishment of atomic models corresponding to the multiple basic problems in the basic scenarios may include the following contents when implemented: constructing corresponding algorithm components according to the basic problems; standardizing the first category of sample data according to preset standardization rules to obtain standardized first category sample data; establishing atomic models corresponding to the basic problems based on the standardized first category sample data and the algorithm components.

[0139] In some embodiments, during specific implementation, the first terminal device can combine experience and determine the activity flow for solving the basic problem based on the basic problem; determine the data processing events involved in the activity flow and the execution order of the data processing events based on the activity flow; and construct a corresponding algorithm component based on the data processing events and the execution order of the data processing events. The above algorithm component can be an existing algorithm component or an algorithm component that the first terminal device recreates for the basic problem.

[0140] In some embodiments, after the atomic model corresponding to the basic problem is established, the method may further include the following contents when implemented: configuring a data packet corresponding to the atomic model according to the basic problem and the atomic model.

[0141] In some embodiments, during the process of constructing an atomic model, relevant data used in constructing the atomic model may be acquired and saved to configure a data packet corresponding to the atomic model.

[0142] Specifically, for example, the first type of sample data used in the process of building the atomic model, the model features extracted in the process of building the atomic model, the deployment environment parameters when building the atomic model, the training script files and evaluation script files used in the construction, and the initial algorithm components based on which the atomic model is built, and other related data can be obtained to configure the corresponding data package.

[0143] As can be seen from the above, based on the above data processing method, before specific implementation, multiple basic problems are pre-split from basic scenes with a larger coverage range, and then corresponding atomic models are constructed for each basic problem in the basic scene, and corresponding data packets are configured for each atomic model; during specific implementation, the target problem in the target scene can be determined for data processing in a specific target scene, and then a suitable target atomic model is matched from the multiple atomic models pre-constructed above according to the target problem, and a pre-configured target data packet corresponding to the target atomic model is obtained; and then, based on the target scene and the target data packet, model training related to the target scene can be performed on the basis of the target atomic model, so as to obtain a target model suitable for specific data processing in the target scene, so as to use the model for data processing in the target scene. Thus, it is possible to efficiently train and obtain applicable target models for various types of data processing in various target scenes, thereby improving the training efficiency of the target model.

[0144] See also Figure 8 As shown, the embodiment of this specification also provides a data processing method. The method is specifically applied to the second terminal device side. When the method is specifically implemented, the method may include the following contents.

[0145] S801: Determine the target problem according to the target scenario.

[0146] S802: Generate and send a corresponding atomic model selection request to the server based on the target problem; wherein the atomic model selection request is used to request the server to determine a target atomic model that matches the target problem from multiple atomic models, and the atomic model includes preset unit models corresponding to basic problems in basic scenarios, and the atomic models are also respectively configured with corresponding data packets.

[0147] S803: Receive the target atomic model fed back by the server and a target data packet corresponding to the target atomic model.

[0148] S804: Training the target atomic model according to the target scene and the target data packet to obtain a target model.

[0149] In some embodiments, through the above method, the server can prepare in advance atomic models corresponding to multiple basic problems in the basic scenarios, and corresponding data packets. When the front-end user or technician needs to train a target model for a target problem in a specific target scenario, an atomic model selection request carrying the target problem can be sent to the server through a second terminal device arranged on the side of the front-end user or technician. The server can receive and respond to the above atomic model selection request, find a target atomic model that matches the target problem from multiple pre-built atomic models for the second terminal device, and the corresponding target data packet, and send it to the second terminal device. Furthermore, the second terminal device can train the target atomic model locally according to the target scenario and the target data packet to obtain the required target model.

[0150] In some embodiments, after obtaining the target model, the method may further include the following when implemented: acquiring target data in the target scene; and performing data processing on the target data using the target model.

[0151] In some embodiments, the target scenario may specifically include a customer service group reply scenario. Accordingly, the target data may specifically include text messages sent by customers in the customer service group. Of course, the target scenarios and target data listed above are only schematic illustrations. During specific implementation, according to specific circumstances and processing needs, the target scenarios may also include other types of scenarios (e.g., customer service answer scenarios for web page messages, customer service answer scenarios for email questions, etc.), and accordingly, the target data may also include other types of data. This specification does not limit this.

[0152] In some embodiments, the target data is processed using the target model. When implemented specifically, the following contents may be included: using the target model to identify and process the text message sent by the customer to determine whether the text message is target text message that requires a manual reply; if it is determined that the text message is target text message, generating reply prompt information; and sending the reply prompt information to the customer service in the customer service group.

[0153] Through the above method, the second terminal device can determine the target problem according to the specific data processing involved in the target scenario; according to the target problem, find the matching target atomic model and the target data packet from the preset atomic model through the server; then on the basis of the target atomic model, train the target atomic model according to the target data packet to obtain the required target model, thereby reducing the complexity of obtaining the target model and improving the training efficiency of the target model.

[0154] The embodiment of this specification also provides another data processing method. The method is specifically applied to the first terminal device side. When the method is specifically implemented, the method may include the following contents.

[0155] S1: separating a plurality of basic problems from a basic scenario; wherein the basic scenario includes a scenario whose coverage is greater than a preset coverage threshold.

[0156] S2: Acquire first type of sample data from the basic scene.

[0157] S3: According to the first type of sample data, atomic models corresponding to the multiple basic problems in the basic scenario are established, and corresponding data packets are configured for the atomic models.

[0158] S4: Send the atomic model and the corresponding data packet to the server, so that the atomic model can be used as a basis to efficiently train the corresponding data packet to obtain a target model suitable for data processing in the target scenario.

[0159] Through the above method, multiple corresponding atomic models and data packages can be built in advance based on the basic problems in the basic scenarios, which simplifies the workload of subsequent training of the target model and makes the subsequent training process of the target model more convenient and efficient.

[0160] The embodiment of this specification also provides another data processing method. The data processing method can be specifically applied to a second terminal device deployed on the merchant side. When the method is specifically implemented, it can include the following contents.

[0161] S1: Receive and respond to the merchant's processing request, determine that the target scenario is a customer service group reply scenario, and the target question is to determine whether the text message posted by the customer in the customer service group requires a manual reply.

[0162] S2: Generate and send a corresponding model training request to the server according to the target problem; wherein the model training request carries the target problem and the target scenario; the server is used to respond to the model training request, determine a target atomic model that matches the target problem from multiple atomic models, and a target data packet corresponding to the target atomic model; the server is also used to train the target atomic model according to the target scenario and the target data packet to obtain the target model.

[0163] In one embodiment, when the above method is implemented, it may also include: receiving a target model fed back by the server; wherein the target model is used to determine whether the text message posted by the customer in the customer service group requires a manual reply, and prompting the merchant with the text message that requires a manual reply.

[0164] In this embodiment, the second terminal device can obtain and run the above target model, so that the text information that requires manual response in the merchant customer service group can be discovered in a timely manner through the above target model, and the merchant can be prompted to respond in a timely manner.

[0165] The embodiments of this specification also provide a server, including a processor and a memory for storing processor executable instructions. When the processor is implemented, it can perform the following steps according to the instructions: determine a target problem in a target scenario; according to the target problem, determine a target atomic model that matches the target problem from multiple atomic models, and obtain a target data packet corresponding to the target atomic model; wherein the atomic model includes preset unit models corresponding to basic problems in basic scenarios, and the atomic models are also respectively configured with corresponding data packets; according to the target scenario and the target data packet, train the target atomic model to obtain a target model; wherein the target model is used to perform data processing in the target scenario.

[0166] In order to complete the above instructions more accurately, refer to Fig. 9 As shown, the embodiment of this specification also provides another specific server, wherein the server includes a network communication port 901, a processor 902 and a memory 903, and the above structures are connected through internal cables so that each structure can perform specific data interaction.

[0167] Among them, the network communication port 901 can be specifically used to receive a model training request sent by a second terminal device.

[0168] The processor 902 can be specifically used to respond to a model training request and determine a target problem in a target scene; according to the target problem, determine a target atomic model that matches the target problem from multiple atomic models, and obtain a target data packet corresponding to the target atomic model; wherein the atomic model includes a preset unit model corresponding to the basic problem in the basic scene, and the atomic model is also respectively configured with a corresponding data packet; according to the target scene and the target data packet, train the target atomic model to obtain a target model; wherein the target model is used to perform data processing in the target scene.

[0169] The memory 903 may be specifically used to store corresponding instruction programs.

[0170] In this embodiment, the network communication port 901 can be a virtual port that is bound to different communication protocols so that different data can be sent or received. For example, the network communication port can be a port responsible for web data communication, a port responsible for FTP data communication, or a port responsible for email data communication. In addition, the network communication port can also be a physical communication interface or communication chip. For example, it can be a wireless mobile network communication chip, such as GSM, CDMA, etc.; it can also be a Wifi chip; it can also be a Bluetooth chip.

[0171] In this embodiment, the processor 902 may be implemented in any appropriate manner. For example, the processor may take the form of a microprocessor or processor and a computer-readable medium storing a computer-readable program code (such as software or firmware) executable by the (micro)processor, a logic gate, a switch, an application specific integrated circuit (ASIC), a programmable logic controller, and an embedded microcontroller, etc. This specification does not limit this.

[0172] In this embodiment, the memory 903 may include multiple levels. In a digital system, anything that can store binary data can be a memory; in an integrated circuit, a circuit with a storage function that has no physical form is also called a memory, such as RAM, FIFO, etc.; in a system, a storage device with a physical form is also called a memory, such as a memory stick, TF card, etc.

[0173] The embodiments of this specification also provide a computer storage medium based on the above-mentioned data processing method, wherein the computer storage medium stores computer program instructions, and when the computer program instructions are executed, the following are achieved: determining a target problem in a target scene; according to the target problem, determining a target atomic model that matches the target problem from multiple atomic models, and obtaining a target data packet corresponding to the target atomic model; wherein the atomic model includes preset unit models corresponding to basic problems in basic scenes, and the atomic models are also respectively configured with corresponding data packets; according to the target scene and the target data packet, the target atomic model is trained to obtain a target model; wherein the target model is used to perform data processing in the target scene.

[0174] In this embodiment, the storage medium includes, but is not limited to, a random access memory (RAM), a read-only memory (ROM), a cache, a hard disk (HDD), or a memory card. The memory may be used to store computer program instructions. The network communication unit may be an interface for network connection communication set in accordance with the standard specified by the communication protocol.

[0175] In this embodiment, the functions and effects specifically implemented by the program instructions stored in the computer storage medium can be explained in comparison with other implementations and will not be described in detail here.

[0176] See also Fig.10 As shown, at the software level, the embodiments of this specification also provide a data processing device, which may specifically include the following structural modules.

[0177] The first determination module 1001 may be specifically used to determine a target problem in a target scenario.

[0178] The second determination module 1002 can be specifically used to determine a target atomic model that matches the target problem from multiple atomic models according to the target problem, and obtain a target data packet corresponding to the target atomic model; wherein the atomic model includes a preset unit model corresponding to the basic problem in the basic scenario, and the atomic models are also respectively configured with corresponding data packets.

[0179] The training module 1003 can be specifically used to train the target atomic model according to the target scene and the target data packet to obtain a target model; wherein the target model is used to perform data processing in the target scene.

[0180] It should be noted that the units, devices or modules described in the above embodiments can be implemented by computer chips or entities, or by products with certain functions. For the convenience of description, the above devices are described separately by functions divided into various modules. Of course, when implementing this specification, the functions of each module can be implemented in the same or more software and / or hardware, or the modules that implement the same function can be implemented by a combination of multiple sub-modules or sub-units. The device embodiments described above are only schematic. For example, the division of the units is only a logical function division. There may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms.

[0181] As can be seen from the above, the data processing device provided in the embodiment of this specification can perform model training on the basis of the target atomic model according to the target scenario and the pre-configured data packet to obtain a target model suitable for specific data processing of the target scenario. Thus, it is possible to efficiently train and obtain a suitable target model for various types of data processing in various target scenarios, thereby improving the training efficiency of the target model and meeting the diverse model training needs of users. In addition, the amount of data processing on the platform server side can be reduced, so that the platform server can simultaneously access and respond to multiple model training requests based on the existing data processing performance, and simultaneously train multiple different target models for multiple different users.

[0182] Although the present specification provides method operation steps as described in the embodiments or flow charts, more or less operation steps may be included based on conventional or non-creative means. The order of steps listed in the embodiments is only one way of executing the order of many steps, and does not represent a unique execution order. When the device or client product in practice is executed, it can be executed in sequence or in parallel according to the method shown in the embodiments or the drawings (for example, a parallel processor or a multi-threaded processing environment, or even a distributed data processing environment). The term "include", "include" or any other variant thereof is intended to cover non-exclusive inclusion, so that the process, method, product or device including a series of elements includes not only those elements, but also includes other elements that are not explicitly listed, or also includes elements inherent to such a process, method, product or device. In the absence of more restrictions, it is not excluded that there are other identical or equivalent elements in the process, method, product or device including the elements. The first, second, etc. words are used to represent the name, and do not represent any particular order.

[0183] Those skilled in the art also know that, in addition to implementing the controller in a purely computer-readable program code, the controller can be made to implement the same function in the form of logic gates, switches, application-specific integrated circuits, programmable logic controllers, and embedded microcontrollers by logically programming the method steps. Therefore, such a controller can be considered as a hardware component, and the devices for implementing various functions included therein can also be considered as structures within the hardware component. Or even, the devices for implementing various functions can be considered as both software modules for implementing the method and structures within the hardware component.

[0184] This specification may be described in the general context of computer-executable instructions executed by a computer, such as program modules. Generally, program modules include routines, programs, objects, components, data structures, classes, etc. that perform specific tasks or implement specific abstract data types. This specification may also be practiced in distributed computing environments where tasks are performed by remote processing devices connected through a communication network. In a distributed computing environment, program modules may be located in local and remote computer storage media including storage devices.

[0185] Through the description of the above embodiments, it can be known that those skilled in the art can clearly understand that the present specification can be implemented by means of software plus a necessary general hardware platform. Based on such an understanding, the technical solution of the present specification can essentially be embodied in the form of a software product, which can be stored in a storage medium such as ROM / RAM, a disk, an optical disk, etc., and includes a number of instructions for enabling a computer device (which can be a personal computer, a mobile terminal, a server, or a network device, etc.) to execute the methods described in each embodiment of the present specification or some parts of the embodiments.

[0186] The various embodiments in this specification are described in a progressive manner, and the same or similar parts between the various embodiments can be referred to each other, and each embodiment focuses on the differences from other embodiments. This specification can be used in many general or special computer system environments or configurations. For example: personal computers, server computers, handheld devices or portable devices, tablet devices, multi-processor systems, microprocessor-based systems, set-top boxes, programmable electronic devices, network PCs, minicomputers, mainframe computers, distributed computing environments including any of the above systems or devices, etc.

[0187] Although the present specification is described through embodiments, those skilled in the art will appreciate that there are many modifications and changes to the present specification without departing from the spirit of the present specification, and it is intended that the appended claims include these modifications and changes without departing from the spirit of the present specification.

Claims

1. A data processing method, comprising: Identify the target problem in the target scenario; According to the target problem, a target atomic model matching the target problem is determined from a plurality of atomic models, and a target data packet corresponding to the target atomic model is acquired; wherein the atomic model includes a preset unit model corresponding to a basic problem in a basic scenario, and the atomic models are also respectively configured with corresponding data packets; The target atomic model is trained according to the target scene and the target data packet to obtain a target model; wherein the target model is used for data processing in the target scene; specifically comprising: acquiring second-category sample data related to the target scene, and performing standardization processing to obtain standardized second-category sample data; combining the standardized second-category sample data with the standardized first-category sample data extracted based on the target data packet to train the target atomic model.

2. According to the method described in claim 1, the target data packet includes at least one of the following: an algorithm component library of the target atomic model, a training script file of the target atomic model, an evaluation script file of the target atomic model, deployment environment information of the target atomic model, a sample set of the target atomic model, and a feature set of the target atomic model.

3. The method according to claim 2, wherein the atomic model is established in the following manner: Multiple basic problems are separated from the basic scenario; among them, The basic scenario includes a scenario where the coverage is greater than a preset coverage threshold; Acquire first type of sample data from the basic scene; According to the first type of sample data, atomic models corresponding to the multiple basic problems in the basic scenario are established.

4. The method according to claim 3, establishing atomic models corresponding to the multiple basic problems in the basic scenario according to the first type of sample data, comprising: According to the basic problems, build corresponding algorithm components; According to a preset standardization rule, the first category of sample data is standardized to obtain standardized first category sample data; An atomic model corresponding to the basic problem is established based on the standardized first-category sample data and the algorithm component.

5. The method according to claim 4, after establishing the atomic model corresponding to the basic problem, the method further comprises: According to the basic problem and the atomic model, a data packet corresponding to the atomic model is configured.

6. The method according to claim 1, after obtaining the target model, further comprising: According to the target scenario, the target model is configured to obtain a configured target model; wherein the configuration process includes at least one of the following: configuration of the input interface of the model, configuration of the output interface of the model, and configuration of the operating environment of the model; Publish the configured target model.

7. The method according to claim 2, wherein the training of the target atomic model according to the target scene and the target data packet comprises: Acquire the data in the target scene as the second type of sample data; According to the target data packet, the second type of sample data is standardized to obtain standardized second type of sample data; The target atomic model is trained according to the target data packet and the standardized second-category sample data.

8. The method according to claim 7, wherein training the target atomic model according to the target data packet and the standardized second type sample data comprises: According to the target data packet, extracting standardized first-category sample data from the sample set; Combining the standardized first-category sample data and the standardized second-category sample data to obtain training sample data; The target atomic model is trained according to the target data packet and using the training sample data.

9. The method according to claim 8, wherein the training of the target atomic model using the training sample data according to the target data packet comprises: According to the target scenario, the training sample data is labeled to obtain labeled training sample data; According to the training script file in the target data packet, the target atomic model is trained using the labeled training sample data.

10. The method according to claim 9, further comprising: Receive model training setting parameters; According to the training script file and the model training setting parameters, the target atomic model is trained using the labeled training sample data.

11. According to the method of claim 10, the model training setting parameters include at least one of the following: the type of training method, the type of training cycle, the training target, and the model parameters.

12. The method according to claim 9, after training the target atomic model using the labeled training sample data according to the training script file in the target data packet, the method further comprises: According to the evaluation script file in the target data packet, a model evaluation is performed on the trained target atomic model to determine an evaluation result for the trained target atomic model; Determining whether the trained target atomic model meets preset requirements according to the evaluation results; When it is determined that the trained target atomic model meets the preset requirements, the trained target atomic model is determined as the target model.

13. The method according to claim 12, after determining whether the trained target atomic model meets the preset requirements according to the evaluation results, the method further comprises: When it is determined that the trained target atomic model does not meet the preset requirements, the trained target atomic model is trained according to the training script file in the target data packet.

14. The method according to claim 12, wherein the step of performing model evaluation on the trained target atomic model according to the evaluation script file in the target data packet comprises: Receive model evaluation setting parameters; Model evaluation is performed on the trained target atomic model according to the evaluation script file in the target data packet and the model evaluation setting parameters.

15. The method according to claim 6, after publishing the configured target model, the method further comprises: Receive and respond to a user's call request for a target model, and use the target model to process target data related to a target scene input by the user to obtain a corresponding target data processing result; Feedback the target data processing result. The method according to claim 1 , wherein the target question comprises a combination of a plurality of basic questions.

17. The method according to claim 16, further comprising: According to the target problem, a plurality of target atomic models and a corresponding plurality of target data packets are obtained; wherein each target atomic model in the plurality of target atomic models is matched with one basic problem in the plurality of basic problems respectively; According to the target scene and the multiple target data packets, respectively train the multiple target atomic models to obtain multiple sub-models; The multiple sub-models are combined to obtain a target model.

18. A data processing method, comprising: Determine the target problem based on the target scenario; According to the target problem, a corresponding atomic model selection request is generated and sent to the server; wherein the atomic model selection request is used to request the server to determine a target atomic model matching the target problem from multiple atomic models, the atomic model includes a preset unit model corresponding to the basic problem in the basic scene, and the atomic model is also respectively configured with a corresponding data packet; receiving the target atomic model fed back by the server and a target data packet corresponding to the target atomic model; The target atomic model is trained according to the target scene and the target data packet to obtain the target model, specifically comprising: acquiring second-category sample data related to the target scene and performing standardization processing to obtain standardized second-category sample data; combining the standardized second-category sample data with the standardized first-category sample data extracted based on the target data packet to train the target atomic model.

19. The method according to claim 18, after obtaining the target model, the method further comprises: Obtain target data in the target scene; The target model is used to perform data processing on the target data.

20. The method according to claim 19, wherein the target scenario comprises a customer service group reply scenario, and the target data comprises text messages sent by customers in the customer service group.

21. The method according to claim 20, wherein the step of processing the target data using the target model comprises: Using the target model to perform recognition processing on the text message sent by the customer to determine whether the text message is a target text message that requires a manual reply; In the case where it is determined that the text information is the target text information, generating a reply prompt information; The reply prompt information is sent to the customer service staff in the customer service group.

22. A data processing method, comprising: Receive and respond to the merchant's processing request, determine that the target scenario is the customer service group reply scenario, and the target question is to determine whether the text message posted by the customer in the customer service group requires a manual reply; According to the target problem, a corresponding model training request is generated and sent to the server; wherein the model training request carries the target problem and the target scenario; the server is used to respond to the model training request, and determine a target atomic model that matches the target problem from multiple atomic models, and a target data packet corresponding to the target atomic model; the server is also used to train the target atomic model according to the target scenario and the target data packet to obtain the target model; the server specifically obtains the second category sample data related to the target scenario, and performs standardization processing to obtain the standardized second category sample data; the standardized second category sample data is combined with the standardized first category sample data extracted based on the target data packet to train the target atomic model.

23. The method according to claim 22, further comprising: Receive the target model fed back by the server; wherein the target model is used to determine whether the text message posted by the customer in the customer service group requires a manual reply, and to prompt the merchant with the text message that requires a manual reply.

24. A data processing device, comprising: A first determination module is used to determine a target problem in a target scenario; A second determination module is used to determine a target atomic model matching the target problem from a plurality of atomic models according to the target problem, and obtain a target data packet corresponding to the target atomic model; wherein the atomic model includes a preset unit model corresponding to the basic problem in the basic scene, and the atomic models are also respectively configured with corresponding data packets; A training module, used for training the target atomic model according to the target scene and the target data packet to obtain a target model; wherein the target model is used for data processing in the target scene; The training module is specifically used to obtain second-category sample data related to the target scene, and perform standardization processing to obtain standardized second-category sample data; the standardized second-category sample data is combined with the standardized first-category sample data extracted based on the target data packet to train the target atomic model.

25. A server, comprising a processor and a memory for storing processor-executable instructions, wherein the processor implements the steps of the method according to any one of claims 1 to 17 when executing the instructions.

26. A computer-readable storage medium having computer instructions stored thereon, wherein the instructions, when executed, implement the steps of the method according to any one of claims 1 to 17.

Images