Online model optimization method, device, equipment, storage medium and program product

Abstract

The present application discloses an optimization method, device, equipment, storage medium and program product for an online model, and relates to the fields of artificial intelligence, machine learning and content recommendation. The method comprises: receiving a first message, the first message comprising identification information of an online model to be optimized; obtaining original sample data, and transforming fields in the original sample data to obtain transformed sample data; building a construction model identical to the online model to be optimized according to the identification information of the online model to be optimized; optimizing the online model to be optimized according to the original sample data, the transformed sample data and the construction model. The method reduces the complexity of model optimization and improves the efficiency of model optimization.

Description

Technical Field

[0001] The embodiments of the present application relate to artificial intelligence technology, and in particular to an online model optimization method, device, equipment, storage medium and program product, which can be used in the fields of machine learning and content recommendation. Background Art

[0002] For machine learning models, it is not only necessary for the model to have good predictive capabilities for existing data, but more importantly, it is also necessary to have good predictive capabilities for other unknown data. If the prediction accuracy of machine learning models is to be improved, it is necessary to continuously promote the optimization and iteration of the model, and to provide more accurate services through continuous tuning.

[0003] Currently, when optimizing machine learning models, the industry detects the sensitivity of the model to features from the feature dimension level. However, for complex combination feature models, cross-feature models, etc., detecting the sensitivity of the model to features is relatively complicated, resulting in a complex and inefficient model optimization process. Summary of the invention

[0004] The present application provides an online model optimization method, device, equipment, storage medium and program product that reduce the complexity of model optimization and improve the efficiency of model optimization.

[0005] According to one aspect of the present application, there is provided an online model optimization method, comprising:

[0006] Receiving a first message, wherein the first message includes identification information of an online model to be optimized;

[0007] Acquire original sample data, and transform fields in the original sample data to obtain transformed sample data;

[0008] According to the identification information of the online model to be optimized, building a construction model identical to the online model to be optimized;

[0009] The online model to be optimized is optimized according to the original sample data, the transformed sample data and the constructed model.

[0010] According to another aspect of the present application, there is provided an optimization device for an online model, comprising:

[0011] A receiving module, configured to receive a first message, wherein the first message includes identification information of an online model to be optimized;

[0012] An acquisition module, used to acquire original sample data and transform fields in the original sample data to obtain transformed sample data;

[0013] A model building module, used to build a construction model identical to the online model to be optimized according to the identification information of the online model to be optimized;

[0014] An optimization module is used to optimize the online model to be optimized based on the original sample data, the transformed sample data and the constructed model.

[0015] According to another aspect of the present application, there is provided an electronic device, comprising:

[0016] at least one processor; and

[0017] a memory communicatively connected to the at least one processor; wherein,

[0018] The memory stores instructions that can be executed by the at least one processor, and the instructions are executed by the at least one processor so that the at least one processor can execute the method described in the first aspect above.

[0019] According to another aspect of the present application, a non-transitory computer-readable storage medium storing computer instructions is provided, wherein the computer instructions are used to enable the computer to execute the method described in the first aspect.

[0020] According to another aspect of the present application, a computer program product is provided, which includes: a computer program, which is stored in a readable storage medium, and at least one processor of an electronic device can read the computer program from the readable storage medium, and the at least one processor executes the computer program so that the electronic device executes the method described in the first aspect.

[0021] According to the technical solution of the present application, the complexity of model optimization is reduced and the efficiency of model optimization is improved.

[0022] It should be understood that the content described in this section is not intended to identify the key or important features of the embodiments of the present application, nor is it intended to limit the scope of the present application. Other features of the present application will become easily understood through the following description. BRIEF DESCRIPTION OF THE DRAWINGS

[0023] The accompanying drawings are used to better understand the present solution and do not constitute a limitation of the present application.

[0024] Figure 1 It is a flowchart of an optimization method of an online model provided in an embodiment of the present application;

[0025] Figure 2 It is a process schematic diagram of an online model optimization method provided in an embodiment of the present application;

[0026] Figure 3It is a schematic diagram of fluctuations in the output results of the model provided in the embodiment of the present application;

[0027] Figure 4 It is a schematic diagram of fluctuations in the output results of the model provided in the embodiment of the present application;

[0028] Figure 5 It is a structural schematic diagram of an optimization device for an online model provided in an embodiment of the present application;

[0029] Figure 6 It is a schematic block diagram of an electronic device used to implement the online model optimization method of an embodiment of the present application. DETAILED DESCRIPTION

[0030] The following is a description of exemplary embodiments of the present application in conjunction with the accompanying drawings, including various details of the embodiments of the present application to facilitate understanding, which should be considered as merely exemplary. Therefore, it should be recognized by those of ordinary skill in the art that various changes and modifications can be made to the embodiments described herein without departing from the scope and spirit of the present application. Similarly, for the sake of clarity and conciseness, the description of well-known functions and structures is omitted in the following description.

[0031] Machine learning is a multi-disciplinary subject. Its core concern is to obtain rules from historical experience and apply them to new similar scenarios. The historical experience mentioned here and how to obtain rules and apply them all involve data (samples) and the selection and use of data. For machine learning models, not only do they need to have good predictive capabilities for existing data, but more importantly, they also need to have good predictive capabilities for other unknown data, that is, they also need to have good generalization capabilities.

[0032] In order to improve the accuracy of machine learning models, it is necessary to use appropriate data to evaluate the models to continuously promote the iteration of the models. Only by continuous tuning can better and more accurate services be provided. At present, the evaluation of machine learning models in the industry is to detect the sensitivity of the model to the features from the feature dimension level, and establish a mapping relationship from the features to the model estimation results. In this solution, it is necessary to determine how the features are extracted and the relationship between the sample fields and the corresponding features. For complex combined feature models, cross-feature models, etc., many sample fields and features are one-to-many relationships, and there are combinations, associations or transformations between features. As the number of features increases, the evaluation process will be more complicated and time-consuming, which is not conducive to model optimization.

[0033] In addition, this evaluation scheme for detecting the sensitivity of the model to features is not conducive to intuitively reflecting the impact of samples on the model. Taking a resource recommendation model as an example, a sample input to the model may include multiple fields such as resource title, resource interest points, resource classification, resource length, number of clicks, user interest points, user age, user education level, and user's historical click resource list. When extracting features from the sample, different features can be extracted based on these fields, such as extracting feature 1 {user interest points-resource interest points}, feature 2 {user education level, resource classification}, and feature 3 {user interest points-resource classification} as model input. For a sample, different models have different feature extraction rules. The same field may be used by multiple features of the same model, and a feature may contain multiple fields. For example, as shown in Table 1, for a model with 400 features, the number of sample fields used ranges from 100 to 500, and the frequency of use of the fields is as high as thousands of times. If the sensitivity of the model's estimated results is simply detected from the feature level, the impact of sample field fluctuations on the model cannot be intuitively reflected.

[0034] Table 1

[0035] Model Number of sample fields Sample field usage frequency Number of features Model 1 233 2471 408 Model 2 498 3220 435 Model 3 150 1108 466 Model 4 102 1522 257 Model 5 411 3193 724

[0036] In order to simplify the evaluation of machine learning models and improve the efficiency of model optimization, in an embodiment of the present application, for any online model, a set of identical construction models is built offline, and the fields of the original sample data input into the construction model are transformed, so as to quickly evaluate the sensitivity of the model to the sample fields, intuitively determine the impact of the transformation of the sample fields on the model's estimated results, and promote the optimization of the model based on the impact of the sample fields on the model.

[0037] The following will describe in detail the online model optimization method provided by the present application through specific embodiments. It is understandable that the following specific embodiments can be combined with each other, and the same or similar concepts or processes may not be described in detail in some embodiments.

[0038] Figure 1 1 is a flow chart of an online model optimization method provided according to an embodiment of the present application. The execution subject of the method is an online model optimization device, which can be implemented by software and / or hardware. For example, the device is a server. Figure 1 As shown, the method includes:

[0039] S101. Receive a first message, where the first message includes identification information of an online model to be optimized.

[0040] In an embodiment of the present application, the optimization of the online model can be triggered by the user, or it can also be triggered by other running programs. For example, a user can create and submit an optimization task for an online model through a front-end service, thereby triggering the server to execute subsequent steps. The user uses the front-end service through a terminal device. For example, the user can submit the identification information of the online model to be optimized on the front-end page, and the terminal device sends the identification information of the online model to be optimized to the server side through a first message. For example, other running programs trigger the optimization task of the online model, and the device running the program sends the identification information of the online model to be optimized to the server through a first message. The identification information of the online model to be optimized can be the interface information, address information, unique index and other identifiers of the online model to be optimized, which is not limited in the embodiment of the present application. The online model to be optimized can be a machine learning model running in any online system, which is not limited in the embodiment of the present application. For example, the online model to be optimized can be a resource recommendation model.

[0041] S102: Acquire original sample data, and transform fields in the original sample data to obtain transformed sample data.

[0042] The original sample data may be obtained based on the sample data submitted by the user, or may be obtained based on the input information of the online model to be optimized extracted from the online system. For example, when creating an optimization task for an online model, the user may submit sample data at the same time in addition to submitting the identification information of the online model to be optimized, that is, the server receives sample data in addition to the identification information of the online model to be optimized from the terminal device. For example, the server may also collect the input information of the online model to be optimized in real time based on the identification information of the online model to be optimized, and obtain the original sample data from the input information of the online model to be optimized, so that subsequent evaluation and optimization can better meet the actual situation of the online system.

[0043] The transformation of the fields in the original sample data may be at least one of deletion, trimming, blanking, adding, replacing or reversing. By performing the above transformations on the fields in the original sample, sample data different from the original sample data may be obtained.

[0044] S103: Building a construction model identical to the online model to be optimized based on the identification information of the online model to be optimized.

[0045] Since the online model to be optimized is a model in the running online system, in order to evaluate the online model to be optimized without affecting the normal operation of the online system, the embodiment of the present application adopts the method of building a construction model offline. The construction model is the same as the online model to be optimized. Therefore, the evaluation of the online model to be optimized can be achieved by evaluating the construction model.

[0046] S104: Optimize the online model to be optimized based on the original sample data, the transformed sample data and the constructed model.

[0047] By using the original sample data and the sample data after the fields are transformed as the input of the constructed model respectively, we can get different estimated results of the constructed model for each sample data output, so as to intuitively determine the sensitivity of the constructed model to the transformed fields in the original sample data, that is, we can determine the impact of the sample fields on the online model to be optimized, and thus optimize the online model to be optimized based on this.

[0048] The online model optimization method provided in the embodiment of the present application, for any online model, builds a set of construction models that are the same as the online model in an offline manner, transforms the fields of the original sample data input into the construction model, and quickly evaluates the sensitivity of the online model to the sample fields based on the estimation results of the construction model for the original sample data and the transformed sample data, intuitively determines the impact of the transformation of the sample fields on the estimation results of the model, promotes the optimization of the model based on the impact of the sample fields on the model, simplifies the evaluation of the machine learning model, and improves the efficiency of model optimization.

[0049] In addition, the method of the embodiment of the present application restores the online environment by building a construction model that is the same as the online model, and can more accurately realize the evaluation of the online system. When the original sample data is the real-time input of the online model, the evaluation result is closer to the actual situation of the online system.

[0050] The following combination Figure 2 Each step in the above embodiment is further described. Taking the online model to be optimized as a resource recommendation model as an example, the input of the model may include user information and resource information, and the output of the model is the user's point spread ratio q for each resource, and the point spread ratio q indicates the probability of the user clicking on the resource. Figure 2 As shown in , after the user creates and submits the optimization task of the online model on the front-end platform, the server performs sample processing, model estimation and result analysis.

[0051] The sample processing stage includes: the acquisition of original sample data and the transformation (mutation construction) of the original sample data. The acquisition of original sample data can also include two parts: sample acquisition and data analysis. The method of sample acquisition can be the user submission or traffic collection mentioned above. Traffic collection is to obtain the input information of the online model to be optimized; data analysis includes format analysis and data type conversion, etc. The sample data after sample acquisition and data analysis is the original sample data. In the transformation of the original sample data, the transformation method (mutation strategy) can include at least one of deletion, pruning, blanking, addition, replacement or inversion.

[0052] It should be noted that the server may first obtain the target transformation mode, and then process the target field in the original sample data accordingly according to the target transformation mode to obtain the transformed sample data. The target transformation mode may include at least one of the above transformation modes. The target transformation mode may be user-defined or randomly determined by the server, i.e. Figure 2 As shown, custom processing or random mutation. The original sample data can be called the original sample, and the transformed sample data can be called the constructed sample. For an original sample, different transformation methods or target fields can be used to construct multiple constructed samples.

[0053] Optionally, the user can select a target transformation mode when creating and submitting an optimization task of an online model, that is, the server receives a second message from the terminal device, the second message including the target transformation mode. Optionally, the server randomly selects at least one target transformation mode from a plurality of preset transformation modes.

[0054] When users select the target transformation method, they can also specify the target field that needs to be transformed, that is, the user can define the transformation of the target field according to the target transformation method, so that for certain specific fields, the impact of the transformation of these fields on the online model to be optimized can be evaluated. Alternatively, the user can specify the target transformation method without specifying the field, and the server will randomly transform the field according to the target transformation method, which is conducive to batch analysis of the impact of the field on the online model to be optimized. Alternatively, the user can specify the target field without specifying the target transformation method, which makes it easier to evaluate the impact of different transformations of the target field on the online model to be optimized. If the user does not specify, the server randomly selects the target field and the target transformation method for transformation to achieve batch analysis of the sample fields.

[0055] The model evaluation phase includes: environment preparation and sample estimation. During the environment preparation process, the server obtains the parameter information of the online model to be optimized based on the identification information of the online model to be optimized; based on the parameter information of the online model to be optimized, the server builds a construction model that is the same as the online model to be optimized. Specifically, it may include: Figure 2 The environment building, model synchronization, configuration modification and environment verification shown in . Afterwards, the server can input the original sample data and the transformed sample data into the constructed model to obtain the output results corresponding to the original sample data and the transformed sample data respectively. For example, for the resource recommendation model, the output result is the aforementioned point spread ratio q, which can also be called the estimated result.

[0056] In addition, during the environment preparation stage, a q_tool tool can be packaged with the construction model. Using the q_tool tool, the corresponding estimated results can be output in parallel for the input original samples and constructed samples, that is, the q_tool tool can be used to transform the original samples and multiple constructed samples in parallel, and output the original estimated results and the estimated results corresponding to the constructed samples. In addition, the q_tool tool can also reduce the calculation of samples. Taking the resource recommendation model as an example, the q_tool tool can be used to save the output of the model, that is, the point spread ratio q of the user for each resource, in the cache. If a subsequent sample includes the same user and resource, there is no need to perform the estimated calculation of the model, and the point spread ratio q in the cache can be directly used as the output result of the sample, thereby improving processing efficiency.

[0057] In the result analysis phase, the server optimizes the online model to be optimized based on the output results corresponding to the original sample data and the transformed sample data. For example, determine the difference between the output results corresponding to the original sample data and the output results corresponding to the transformed sample data; based on the difference, determine the impact factor of the transformed field in the transformed sample data on the online model to be optimized; based on the impact factor, optimize the online model to be optimized. In addition, the output result report can also be visualized, using charts and other forms to display the difference between the output results corresponding to the original sample data and the output results corresponding to the transformed sample data, so as to intuitively determine the impact of sample field changes on the model output results.

[0058] For example, Figure 3 and 4 As shown in Figure 1, the output of the model fluctuates after a target field in the sample is left blank, where Figure 3 The horizontal axis represents the identification of each recommended resource (article). Only the identification of some resources is shown in the figure. The vertical axis represents the difference in the point spread ratio q. Figure 4 The horizontal axis represents the identification of each recommended resource. Similarly, only some of the resource identifications are shown in the figure. In the vertical axis, the bar graph part represents the point spread ratio q corresponding to the original sample and the constructed sample, and the waveform part represents the difference in the point spread ratio q. The larger the fluctuation, the greater the impact of the field on the q value, which can also be understood as the higher importance of the field. By analyzing the fluctuation mean, maximum value and minimum value, etc., the size of the impact factor of the field on the model can be determined.

[0059] Through the method of the embodiment of the present application, the connection between the model and the minimum variable factor sample field can be established, thereby improving the interpretability of the model. In addition, the method is applicable to online scenarios. By analyzing the impact of the field on the model output, it is possible to better analyze the impact of sample field fluctuations on the market, and establish hierarchical monitoring of sample fields based on the influence of the field, which is convenient for mining fields that have a negative impact on the model, thereby more accurately optimizing the model and improving the robustness and generalization ability of the model.

[0060] Figure 5 Schematic diagram of the structure of the online model optimization device provided in the embodiment of the present application. Figure 5 As shown, the online model optimization device 500 includes:

[0061] The receiving module 501 is used to receive a first message, where the first message includes identification information of the online model to be optimized;

[0062] The acquisition module 502 is used to acquire original sample data and transform fields in the original sample data to obtain transformed sample data;

[0063] A model building module 503 is used to build a construction model that is the same as the online model to be optimized according to the identification information of the online model to be optimized;

[0064] The optimization module 504 is used to optimize the online model to be optimized based on the original sample data, the transformed sample data and the constructed model.

[0065] In one embodiment, the optimization module 504 includes:

[0066] An estimation unit, used for inputting the original sample data and the transformed sample data into the constructed model, and obtaining output results corresponding to the original sample data and the transformed sample data respectively;

[0067] The optimization unit is used to optimize the online model to be optimized according to the output results corresponding to the original sample data and the transformed sample data.

[0068] In one embodiment, the optimization unit comprises:

[0069] A first determining unit, used to determine the difference between the output result corresponding to the original sample data and the output result corresponding to the transformed sample data;

[0070] A second determining unit is used to determine, according to the difference, an influence factor of the transformed field in the transformed sample data on the online model to be optimized;

[0071] The optimization subunit is used to optimize the online model to be optimized according to the influencing factors.

[0072] In one embodiment, the model building module 503 includes:

[0073] A first acquisition unit, configured to acquire parameter information of the online model to be optimized according to identification information of the online model to be optimized;

[0074] The building unit is used to build a construction model that is the same as the online model to be optimized according to the parameter information of the online model to be optimized.

[0075] In one implementation, the acquisition module 502 includes:

[0076] A first receiving unit is used to receive original sample data; or,

[0077] The collection unit is used to determine the input information of the online model to be optimized as original sample data according to the identification information of the online model to be optimized.

[0078] In one implementation, the acquisition module 502 includes:

[0079] A second acquisition unit is used to acquire a target transformation mode, where the target transformation mode includes at least one of the following: deletion, cropping, blanking, adding, replacing or inversion;

[0080] The transformation unit is used to perform corresponding processing on the target field in the original sample data according to the target transformation method to obtain the transformed sample data.

[0081] In one embodiment, the second acquisition unit includes:

[0082] A second receiving unit is configured to receive a second message, wherein the second message includes a target conversion method; or

[0083] The selection unit is used to randomly select at least one target transformation method from a plurality of preset transformation methods.

[0084] The device of the embodiment of the present application can be used to execute the online model optimization method in the above method embodiment. Its implementation principle and technical effects are similar and will not be repeated here.

[0085] According to an embodiment of the present application, the present application also provides an electronic device and a non-transitory computer-readable storage medium storing computer instructions.

[0086] According to an embodiment of the present application, the present application also provides a computer program product, the program product includes: a computer program, the computer program is stored in a readable storage medium, at least one processor of an electronic device can read the computer program from the readable storage medium, and at least one processor executes the computer program so that the electronic device executes the solution provided by any of the above embodiments.

[0087] Figure 6 It is a schematic block diagram of an electronic device for realizing the optimization method of the online model of an embodiment of the present application. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workbenches, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device can also represent various forms of mobile devices, such as personal digital processing, cellular phones, smart phones, wearable devices and other similar computing devices. The components shown herein, their connections and relationships, and their functions are only examples, and are not intended to limit the implementation of the present disclosure described and / or required herein.

[0088] like Figure 6 As shown, the electronic device 600 includes a computing unit 601, which can perform various appropriate actions and processes according to a computer program stored in a read-only memory (ROM) 602 or a computer program loaded from a storage unit 608 into a random access memory (RAM) 603. In the RAM 603, various programs and data required for the operation of the device 600 can also be stored. The computing unit 601, the ROM 602, and the RAM 603 are connected to each other via a bus 604. An input / output (I / O) interface 605 is also connected to the bus 604.

[0089] A number of components in the device 600 are connected to the I / O interface 605, including: an input unit 606, such as a keyboard, a mouse, etc.; an output unit 607, such as various types of displays, speakers, etc.; a storage unit 608, such as a disk, an optical disk, etc.; and a communication unit 609, such as a network card, a modem, a wireless communication transceiver, etc. The communication unit 609 allows the device 600 to exchange information / data with other devices through a computer network such as the Internet and / or various telecommunication networks.

[0090] The computing unit 601 may be a variety of general and / or special processing components with processing and computing capabilities. Some examples of the computing unit 601 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various dedicated artificial intelligence (AI) computing chips, various computing units running machine learning model algorithms, digital signal processors (DSPs), and any appropriate processors, controllers, microcontrollers, etc. The computing unit 601 performs the various methods and processes described above, such as the optimization method of the online model. For example, in some embodiments, the optimization method of the online model may be implemented as a computer software program, which is tangibly contained in a machine-readable medium, such as a storage unit 608. In some embodiments, part or all of the computer program may be loaded and / or installed on the device 600 via the ROM 602 and / or the communication unit 609. When the computer program is loaded into the RAM 603 and executed by the computing unit 601, one or more steps of the optimization method of the online model described above may be performed. Alternatively, in other embodiments, the computing unit 601 may be configured to perform the optimization method of the online model by any other appropriate means (e.g., by means of firmware).

[0091] Various implementations of the systems and techniques described above herein can be implemented in digital electronic circuit systems, integrated circuit systems, field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), systems on chips (SOCs), load programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various implementations can include: being implemented in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which can be a special purpose or general purpose programmable processor that can receive data and instructions from a storage system, at least one input device, and at least one output device, and transmit data and instructions to the storage system, the at least one input device, and the at least one output device.

[0092] The program code for implementing the method of the present disclosure may be written in any combination of one or more programming languages. These program codes may be provided to a processor or controller of a general-purpose computer, a special-purpose computer, or other programmable data processing device, so that the program code, when executed by the processor or controller, enables the functions / operations specified in the flow chart and / or block diagram to be implemented. The program code may be executed entirely on the machine, partially on the machine, partially on the machine and partially on a remote machine as a stand-alone software package, or entirely on a remote machine or server.

[0093] In the context of the present disclosure, a machine-readable medium may be a tangible medium that may contain or store a program for use by or in conjunction with an instruction execution system, device, or equipment. A machine-readable medium may be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium may include, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, device, or equipment, or any suitable combination of the foregoing. A more specific example of a machine-readable storage medium may include an electrical connection based on one or more lines, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

[0094] To provide interaction with a user, the systems and techniques described herein can be implemented on a computer having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user; and a keyboard and pointing device (e.g., a mouse or trackball) through which the user can provide input to the computer. Other types of devices can also be used to provide interaction with the user; for example, the feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including acoustic input, voice input, or tactile input).

[0095] The systems and techniques described herein may be implemented in a computing system that includes back-end components (e.g., as a data server), or a computing system that includes middleware components (e.g., an application server), or a computing system that includes front-end components (e.g., a user computer with a graphical user interface or a web browser through which a user can interact with implementations of the systems and techniques described herein), or a computing system that includes any combination of such back-end components, middleware components, or front-end components. The components of the system may be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include: a local area network (LAN), a wide area network (WAN), and the Internet.

[0096] A computer system may include a client and a server. The client and the server are generally remote from each other and usually interact through a communication network. The relationship between the client and the server is generated by computer programs running on the corresponding computers and having a client-server relationship with each other. The server may be a cloud server, also known as a cloud computing server or cloud host, which is a host product in the cloud computing service system to solve the defects of difficult management and weak business scalability in traditional physical hosts and VPS services ("Virtual Private Server", or "VPS" for short). The server may also be a server of a distributed system, or a server combined with a blockchain.

[0097] It should be understood that the various forms of processes shown above can be used to reorder, add or delete steps. For example, the steps recorded in this application can be executed in parallel, sequentially or in different orders, as long as the expected results of the technical solution disclosed in this application can be achieved, and this document is not limited here.

[0098] The above specific implementations do not constitute a limitation on the protection scope of this application. It should be understood by those skilled in the art that various modifications, combinations, sub-combinations and substitutions can be made according to design requirements and other factors. Any modifications, equivalent substitutions and improvements made within the spirit and principles of this application should be included in the protection scope of this application.

Claims

1. An online model optimization method, comprising: Receive a first message, wherein the first message includes identification information of the online model to be optimized, wherein the identification information is submitted by a user on a front-end page and is sent to a server side by a device running the program through the first message, and the identification information is interface information, address information or a unique index of the online model to be optimized; Acquire original sample data, and transform fields in the original sample data to obtain transformed sample data; According to the identification information of the online model to be optimized, obtaining the parameter information of the online model to be optimized; according to the parameter information of the online model to be optimized, building a construction model identical to the online model to be optimized; Inputting the original sample data and the transformed sample data into the constructed model to obtain output results corresponding to the original sample data and the transformed sample data respectively; optimizing the online model to be optimized according to the output results corresponding to the original sample data and the transformed sample data; The transforming the fields in the original sample data to obtain the transformed sample data includes: receiving a second message, wherein the second message includes a target transformation mode, and randomly selecting at least two target transformation modes from a plurality of preset transformation modes to obtain a target transformation mode; the target transformation mode includes at least two of the following: deletion, cropping, blanking, adding, replacing, or inversion, wherein the user specifies a target field to be transformed when selecting the target transformation mode, and defines that the target field is transformed according to the target transformation mode; According to the target transformation method, the target field in the original sample data is processed accordingly to obtain the transformed sample data.

2. The method according to claim 1, wherein optimizing the online model to be optimized according to the output results corresponding to the original sample data and the transformed sample data comprises: Determine a difference between an output result corresponding to the original sample data and an output result corresponding to the transformed sample data; Determining, according to the difference, an influence factor of the transformed field in the transformed sample data on the online model to be optimized; The online model to be optimized is optimized according to the influencing factors.

3. The method according to any one of claims 1 to 2, wherein obtaining the original sample data comprises: Receiving the original sample data; or, According to the identification information of the online model to be optimized, the input information of the online model to be optimized is determined as the original sample data.

4. An online model optimization device, comprising: A receiving module, configured to receive a first message, wherein the first message includes identification information of an online model to be optimized, wherein the identification information is submitted by a user on a front-end page and is sent to a server side by a device running the program through the first message, and the identification information is interface information, address information or a unique index of the online model to be optimized; An acquisition module, used to acquire original sample data and transform fields in the original sample data to obtain transformed sample data; Model building module, including: A first acquisition unit, configured to acquire parameter information of the online model to be optimized according to identification information of the online model to be optimized; A building unit, used to build a construction model identical to the online model to be optimized according to the parameter information of the online model to be optimized; The optimization modules include: An estimation unit, used for inputting the original sample data and the transformed sample data into the constructed model, and obtaining output results corresponding to the original sample data and the transformed sample data respectively; An optimization unit, configured to optimize the online model to be optimized according to the output results corresponding to the original sample data and the transformed sample data; The acquisition module includes: a second acquisition unit and a transformation unit; The second acquiring unit includes: A second receiving unit is configured to receive a second message, wherein the second message includes a target transformation mode; a selecting unit is configured to randomly select at least two target transformation modes from a plurality of preset transformation modes; the target transformation modes include at least two of the following: deletion, cropping, blanking, adding, replacing or inverting, wherein when the user selects the target transformation mode, the user specifies a target field to be transformed, and defines that the target field is transformed according to the target transformation mode; The transformation unit is used to perform corresponding processing on the target field in the original sample data according to the target transformation mode to obtain the transformed sample data.

5. The device according to claim 4, wherein the optimization unit comprises: A first determining unit, configured to determine a difference between an output result corresponding to the original sample data and an output result corresponding to the transformed sample data; A second determining unit, configured to determine, based on the difference, an influence factor of the transformed field in the transformed sample data on the online model to be optimized; The optimization subunit is used to optimize the online model to be optimized according to the influencing factor.

6. The device according to any one of claims 4 to 5, wherein the acquisition module comprises: A first receiving unit, configured to receive the original sample data; or, The collection unit is used to determine the input information of the online model to be optimized as the original sample data according to the identification information of the online model to be optimized.

7. An electronic device comprising: at least one processor; and a memory communicatively coupled to the at least one processor; The memory stores instructions that can be executed by at least one processor, and the instructions are executed by at least one processor so that the at least one processor can execute any one of the methods in claims 1-3.

8. A non-transitory computer-readable storage medium storing computer instructions, wherein the computer instructions are used to cause the computer to execute the method according to any one of claims 1 to 3.

9. A computer program product, comprising a computer program, wherein when the computer program is executed by a processor, the method according to any one of claims 1 to 3 is implemented.

Images