Service interface quality evaluation method and device, equipment, medium and product

Abstract

The application relates to a service interface quality evaluation method and device, equipment, medium and product, and the method comprises the following steps: obtaining an advertisement script set, wherein the advertisement script set comprises a plurality of advertisement scripts generated by a preset service interface; obtaining an advertisement feature vector of each advertisement script according to advertisement feature information coding of the advertisement script, wherein positive and negative mark features are preset in the advertisement feature vector, and positive samples and negative samples of the advertisement script are correspondingly generated; predicting advertisement effectiveness indexes of the positive samples and the negative samples by using a preset advertisement effectiveness prediction model, wherein the model is trained to a convergence state in advance; and obtaining a quality index of the service interface according to comparison calculation of the advertisement effectiveness indexes of the positive samples and the negative samples. The application can determine a quality index for a function corresponding to a service interface used for generating an advertisement script in an e-commerce platform, early intervention evaluation of the related function is realized, and the e-commerce platform can conveniently optimize and upgrade the related function in a timely manner.

Description

Technical Field

[0001] This application relates to the field of e-commerce information technology, and in particular to a service interface quality assessment method and corresponding apparatus, computer equipment, computer-readable storage medium, and computer program product. Background Technology

[0002] E-commerce platforms typically have advertising systems in place. To facilitate ad placement for merchants, these systems continuously add various intelligent ad creative functions and provide corresponding service interfaces for merchants to use. These interfaces allow for the automatic generation of relevant ad copy based on the basic information provided by the merchant. Through these service interfaces, e-commerce platforms can offer merchants a large volume of diverse ad creatives.

[0003] To evaluate the effectiveness of a feature in an internet product, specifically to determine the impact of intelligent ad creative features on advertising business, it's necessary to compare the results of an ad using this feature versus not using it. A double-blind randomized A / B test is commonly used; however, A / B randomized tests are not suitable for all scenarios. In such cases, it's necessary to rely on observed data to answer causal questions.

[0004] Specifically, for the advertising business, in the early stages of the intelligent ad creative function's release, the main metric was the copy adoption rate. Once the adoption rate met the requirements, the function was quickly rolled out to all users. Therefore, it was no longer feasible to prevent some users from using the intelligent ad creative function, and thus, it was no longer possible to maintain a control group for A / B testing. Furthermore, the factors influencing the effectiveness of the ads generated based on this function are very complex, with confounding factors, making it impossible to evaluate the intelligent ad creative function through simple comparisons. Therefore, it is necessary to explore other methods to assess the actual performance of the intelligent ad creative function. Summary of the Invention

[0005] The purpose of this application is to solve the above-mentioned problems by providing a service interface quality assessment method and corresponding apparatus, computer equipment, computer-readable storage medium, and computer program product.

[0006] To suit the various purposes of this application, the following technical solution is adopted:

[0007] In one aspect, to suit one of the purposes of this application, a method for evaluating the quality of a service interface is provided, comprising the following steps:

[0008] Get the set of advertising copy, which includes multiple advertising copy generated by preset service interfaces;

[0009] The advertising feature vector is obtained by encoding the advertising feature information of each advertising copy. Positive and negative labeling features are preset in the advertising feature vector, and positive and negative samples of the advertising copy are generated accordingly.

[0010] The advertising effectiveness prediction model is used to predict the advertising effectiveness indicators of the positive and negative samples respectively. The advertising effectiveness prediction model is pre-trained to a convergent state.

[0011] The quality index of the service interface is obtained by comparing the advertising effectiveness indicators of positive and negative samples.

[0012] Optionally, before the step of predicting the advertising effectiveness indicators of the positive and negative samples respectively using a preset advertising effectiveness prediction model, the following steps are included:

[0013] Obtain the training dataset, which includes multiple ad copy that has been delivered and the ad performance metrics generated after the delivery.

[0014] The advertising feature vector of each advertising copy is obtained by encoding the advertising feature information. Based on whether the advertising copy was generated by a preset service interface, a positive or negative label feature is preset in the advertising feature vector to generate training samples.

[0015] The advertising effectiveness prediction model is iteratively trained until convergence using training samples corresponding to the advertising copy in the training dataset. The prediction results of the model are supervised by advertising effectiveness indicators generated after the training samples are deployed.

[0016] Optionally, the advertising feature vector of each ad copy can be obtained by encoding the advertising feature information, including the following steps:

[0017] The advertising feature information of the advertising copy is retrieved from the data interface provided by the advertising delivery system;

[0018] The advertising feature information is cleaned to obtain feature data, and the feature data is converted into corresponding feature values;

[0019] The feature values ​​corresponding to each specific configuration information are encoded into advertising feature vectors according to a preset sorting rule.

[0020] Optionally, the advertising feature information includes any of the following:

[0021] Merchant placement characteristic information is used to indicate the scale of advertising placement by the merchant user to which the corresponding advertising copy belongs;

[0022] Product configuration feature information is used to indicate the product corresponding to the corresponding advertising copy;

[0023] Audience configuration feature information is used to indicate the target user group for the corresponding advertising copy;

[0024] Budget settlement feature information is used to indicate the available funds for the corresponding advertising copy;

[0025] Bidding configuration feature information, used to indicate the bidding constraint information of the corresponding ad copy;

[0026] The delivery configuration feature information is used to indicate the delivery constraints of the corresponding advertising copy.

[0027] Optionally, the set of advertising copy includes advertising copy from the training dataset; or,

[0028] The advertising performance metrics are click-through rate, favorites rate, add-to-cart rate, conversion rate, or return on investment (ROI) used to indicate the effectiveness of advertising campaigns; or,

[0029] The advertising copy includes any one or more of the following: text, images, videos, and animations; or,

[0030] The service interface is used to call a preset copy template to generate advertising copy corresponding to given information, which includes product terms and / or brand terms of the target product.

[0031] Optionally, the quality index of the service interface can be obtained by comparing the advertising effectiveness metrics of positive and negative samples, including the following steps:

[0032] Calculate the mean values ​​of the advertising effectiveness indicators for the positive and negative samples corresponding to the advertising copy set, respectively;

[0033] The difference between the mean of the positive samples and the mean of the negative samples is used as the quality indicator of the service interface.

[0034] Optionally, after the step of calculating the quality index of the service interface based on the comparison of advertising effectiveness indicators of positive and negative samples, the following steps are included:

[0035] Determine whether the quality indicator is lower than a preset threshold. If it is lower than the preset threshold, send an alarm message to a preset communication interface.

[0036] On the other hand, to meet one of the purposes of this application, a service interface quality assessment device is provided, including a copy acquisition module, an encoding processing module, an indicator prediction module, and a comparison and evaluation module, wherein: the copy acquisition module is used to acquire an advertising copy set, including multiple advertising copy generated by a preset service interface; the encoding processing module is used to encode the advertising feature information of each advertising copy to obtain its advertising feature vector, and preset positive and negative label features in the advertising feature vector respectively, thereby generating positive and negative samples of the advertising copy; the indicator prediction module is used to predict the advertising performance indicators of the positive and negative samples respectively using a preset advertising performance prediction model, wherein the advertising performance prediction model is pre-trained to a convergent state; the comparison and evaluation module is used to calculate the quality indicators of the service interface by comparing the advertising performance indicators of the positive and negative samples.

[0037] Optionally, prior to the indicator prediction module, the module includes: a dataset acquisition module for acquiring a training dataset, which includes multiple deployed advertising copy and the advertising performance indicators generated after deployment; a sample generation module for obtaining an advertising feature vector based on the advertising feature information encoding of each advertising copy, and pre-setting positive or negative labeling features in the advertising feature vector according to whether the advertising copy was generated by a preset service interface, thereby generating training samples; and a training execution module for iteratively training the advertising performance prediction model to a convergent state using the training samples corresponding to the advertising copy in the training dataset, wherein the advertising performance indicators generated after deployment of the training samples are used to supervise the prediction results of the model.

[0038] Optionally, the encoding processing module or the sample generation module includes: an information retrieval unit, used to retrieve the advertising feature information of the advertising copy from the data interface provided by the advertising delivery system; a data cleaning unit, used to perform data cleaning on the advertising feature information to obtain feature data, and convert the feature data into corresponding feature values; and a feature encoding unit, used to encode the feature values ​​corresponding to each specific configuration information into advertising feature vectors according to a preset sorting rule.

[0039] Optionally, the advertising feature information includes any of the following: merchant placement feature information, used to indicate the advertising placement scale of the merchant user to which the corresponding advertising copy belongs; product configuration feature information, used to indicate the product corresponding to the corresponding advertising copy; audience configuration feature information, used to indicate the audience user group of the corresponding advertising copy; budget settlement feature information, used to indicate the available funds for the corresponding advertising copy; bidding configuration feature information, used to indicate the bidding constraint information of the corresponding advertising copy; and placement configuration feature information, used to indicate the placement constraint information of the corresponding advertising copy.

[0040] Optionally, the advertising copy set includes advertising copy from the training dataset; or, the advertising performance metrics are click-through rate, favorite rate, add-to-cart rate, conversion rate, or return on investment, which indicate the effectiveness of advertising campaigns; or, the advertising copy includes any one or more of text, images, videos, and animations; or, the service interface is used to call a preset copy template to generate advertising copy corresponding to given information, where the given information includes product keywords and / or brand keywords of the target product.

[0041] Optionally, the comparison and evaluation module includes: a mean statistics unit, used to calculate the mean values ​​of advertising effectiveness indicators for positive and negative samples corresponding to the advertising copy set respectively; and an indicator comparison unit, used to subtract the mean value corresponding to the negative samples from the mean value corresponding to the positive samples to obtain the difference, which is used as the quality indicator of the service interface.

[0042] Optionally, following the comparison and evaluation module, there is a decision processing module, used to determine whether the quality index is lower than a preset threshold, and when it is lower than the preset threshold, to send an alarm message to a preset communication interface.

[0043] In another aspect, a computer device provided for one of the purposes of this application includes a central processing unit and a memory, the central processing unit being used to invoke and run a computer program stored in the memory to perform the steps of the service interface quality assessment method described in this application.

[0044] In another aspect, a computer-readable storage medium is provided to suit another purpose of this application, which stores, in the form of computer-readable instructions, a computer program implemented according to the described service interface quality assessment method, which, when invoked by a computer, performs the steps included in the method.

[0045] In another aspect, a computer program product provided for another purpose of this application includes a computer program / instructions that, when executed by a processor, implement the steps of the service interface quality assessment method described in any embodiment of this application.

[0046] Compared with the prior art, this application has many advantages, including at least the following aspects:

[0047] First, this application applies the counterfactual inference principle to generate advertising feature vectors based on the advertising feature information of advertising copy generated by the service interface. For each advertising copy's feature vector, positive and negative labeled features are pre-set respectively, generating corresponding positive and negative samples to represent the data corresponding to the factual and counterfactual results. The positive and negative samples are then predicted using an advertising effectiveness prediction model to obtain corresponding advertising effectiveness indicators. Based on the comparison results of the advertising effectiveness indicators of the two types of samples, corresponding quality indicators are determined to measure the overall quality corresponding to the service interface, thus solving the problem of effectively measuring the advertising copy generation function implemented by the service interface.

[0048] Secondly, the advertising effectiveness prediction model of this application, after being trained, can be applied without relying on the advertising effectiveness indicators generated after the actual delivery of the advertising copy produced by the service interface to evaluate the function of the service interface. Instead, it only needs to rely on the advertising copy produced by the service interface to carry out comparative experiments. Therefore, it can intervene early during the gray-scale launch of the corresponding function of the service interface, without relying on the advertising copy to generate the corresponding advertising effectiveness data before intervention. This allows for early evaluation of the corresponding function and ensures that the corresponding function can be optimized and upgraded in a timely manner.

[0049] Furthermore, this application transfers the counterfactual inference principle to the field of e-commerce advertising service function evaluation. For the advertising effectiveness prediction model adopted in this application, it is suitable for predicting advertising effectiveness indicators based on advertising feature vectors determined according to the counterfactual inference principle. In fact, it enriches the function of the advertising effectiveness prediction model, making it suitable for evaluating the merits of relevant e-commerce advertising service functions, thereby creating economies of scale for e-commerce platforms. Attached Figure Description

[0050] The above and / or additional aspects and advantages of this application will become apparent and readily understood from the following description of the embodiments taken in conjunction with the accompanying drawings, wherein:

[0051] Figure 1 This is a flowchart illustrating a typical embodiment of the service interface quality assessment method of this application.

[0052] Figure 2 This is a flowchart illustrating the process of training the advertising effectiveness prediction model using the training dataset in this application.

[0053] Figure 3 This is a flowchart illustrating the process of obtaining an advertising feature vector based on the encoding of advertising feature information in an embodiment of this application.

[0054] Figure 4 This is a flowchart illustrating another embodiment of the service interface quality assessment method of this application.

[0055] Figure 5 This is a schematic block diagram of the service interface quality assessment device of this application;

[0056] Figure 6 This is a schematic diagram of the structure of a computer device used in this application. Detailed Implementation

[0057] The embodiments of this application are described in detail below. Examples of these embodiments are shown in the accompanying drawings, wherein the same or similar reference numerals denote the same or similar elements or elements having the same or similar functions throughout. The embodiments described below with reference to the accompanying drawings are exemplary and are only used to explain this application, and should not be construed as limiting this application.

[0058] Those skilled in the art will understand that, unless specifically stated otherwise, the singular forms “a,” “an,” “the,” and “the” used herein may also include the plural forms. It should be further understood that the term “comprising” as used in this application means the presence of the stated features, integers, steps, operations, elements, and / or components, but does not exclude the presence or addition of one or more other features, integers, steps, operations, elements, components, and / or groups thereof. It should be understood that when we say an element is “connected” or “coupled” to another element, it can be directly connected or coupled to the other element, or there may be intermediate elements. Furthermore, “connected” or “coupled” as used herein can include wireless connections or wireless coupling. The term “and / or” as used herein includes all or any units and all combinations of one or more associated listed items.

[0059] It will be understood by those skilled in the art that, unless otherwise defined, all terms used herein (including technical and scientific terms) have the same meaning as commonly understood by one of ordinary skill in the art to which this application pertains. It should also be understood that terms such as those defined in general dictionaries should be understood to have the same meaning as in the context of the prior art, and should not be interpreted in an idealized or overly formal sense unless specifically defined as herein.

[0060] Those skilled in the art will understand that the terms "client," "terminal," and "terminal device" as used herein include both devices that receive wireless signals, devices that only possess wireless signal receiver capabilities without transmission capabilities, and devices with receiving and transmitting hardware, devices that have receiving and transmitting hardware capable of bidirectional communication over a bidirectional communication link. Such devices may include: cellular or other communication devices such as personal computers or tablets, having single-line displays, multi-line displays, or cellular or other communication devices without multi-line displays; PCS (Personal Communications Service) that can combine voice, data processing, fax, and / or data communication capabilities; PDA (Personal Digital Assistant) that may include a radio frequency receiver, pager, internet / intranet access, web browser, notepad, calendar, and / or GPS (Global Positioning System) receiver; and conventional laptops and / or handheld computers or other devices that have and / or include radio frequency receivers. As used herein, "client," "terminal," and "terminal device" can be portable, transportable, installed in a means of transportation (air, sea, and / or land), or suitable and / or configured to operate locally and / or in a distributed manner, operating in any other location on Earth and / or in space. "Client," "terminal," and "terminal device" as used herein can also be a communication terminal, an internet access terminal, or a music / video playback terminal, such as a PDA, a MID (Mobile Internet Device), and / or a mobile phone with music / video playback capabilities, or a smart TV, set-top box, etc.

[0061] The hardware referred to by the names "server," "client," and "service node" in this application is essentially an electronic device with the equivalent capabilities of a personal computer. It is a hardware device with the necessary components revealed by the von Neumann architecture, such as a central processing unit (including an arithmetic logic unit and a control unit), memory, input devices, and output devices. The computer program is stored in its memory, and the central processing unit loads the program stored in the secondary storage into the main memory to run it, execute the instructions in the program, and interact with the input and output devices to complete specific functions.

[0062] It should be noted that the concept of "server" used in this application can also be extended to the case of server clusters. Based on the network deployment principles understood by those skilled in the art, the servers should be logically divided. Physically, these servers can be independent of each other but accessible through interfaces, or they can be integrated into a single physical computer or a computer cluster. Those skilled in the art should understand this flexibility and should not use it to constrain the implementation of the network deployment method in this application.

[0063] Unless otherwise expressly specified, one or more technical features of this application may be deployed on a server and accessed by a client through remote invocation of the online service interface provided by the server, or they may be directly deployed and run on a client for access.

[0064] Unless otherwise specified, the neural network models referenced or potentially referenced in this application may be deployed on a remote server and invoked remotely on the client, or deployed on a client with the capability to invoke directly. In some embodiments, when running on the client, the corresponding intelligence may be acquired through transfer learning in order to reduce the requirements on the client's hardware resources and avoid excessive consumption of the client's hardware resources.

[0065] Unless otherwise specified, all data involved in this application may be stored remotely on a server or on a local terminal device, as long as it is suitable for use by the technical solution of this application.

[0066] Those skilled in the art will understand that although the various methods in this application are described based on the same concept and thus present commonality among them, they can be performed independently unless otherwise specified. Similarly, the various embodiments disclosed in this application are all based on the same inventive concept; therefore, concepts expressed in the same way, as well as concepts that are appropriately changed for convenience but are expressed differently, should be understood equivalently.

[0067] Unless otherwise expressly stated, the various embodiments disclosed in this application can be combined in a cross-cutting manner to flexibly construct new embodiments, as long as such combination does not depart from the inventive spirit of this application and can meet the needs of the prior art or solve a certain deficiency in the prior art. Those skilled in the art should be aware of such modifications.

[0068] The service interface quality assessment method of this application can be programmed into a computer program product and deployed on a client or server for execution. For example, in an exemplary application scenario of this application, it can be deployed on the server of an e-commerce platform. In this way, the method can be executed by human-computer interaction with the process of the computer program product through a graphical user interface by accessing the interface opened after the computer program product is run.

[0069] Please see Figure 1 The service interface quality assessment method of this application, in its typical embodiment, includes the following steps:

[0070] Step S2100: Obtain the advertising copy set, which includes multiple advertising copy generated by preset service interfaces;

[0071] A set of advertising copy is prepared, consisting of advertising copy. In one embodiment, the advertising copy in the set is generated by a pre-defined service interface in the e-commerce platform, so that this application can evaluate the quality indicators of the copy generation function corresponding to the service interface purely based on the advertising copy generated by the service interface. In another embodiment, it is permissible to use the advertising copy generated by the service interface as the main body, mixed with a small amount of advertising copy from other sources, so that in the process of evaluating the quality indicators, the advertising copy from other sources can be used to harmonize the quality indicators, avoiding the determination of the quality indicators from relying entirely on the advertising copy generated by the service interface.

[0072] It is easy to understand that the service interface is an interface provided by the advertising delivery system of the e-commerce platform, which is suitable for generating advertising copy for users. Users can generate corresponding advertising copy through the service interface for advertising delivery.

[0073] In one embodiment, the service interface is configured to obtain given information input by the user. This given information may be a specified target product, or product terms and / or brand terms of the target product. Then, program instructions implemented by the copywriting generation function corresponding to the service interface call a preset copywriting template. The copywriting template includes irreplaceable original text and replaceable tags. The product terms and / or brand terms of the target product are used to replace the corresponding replaceable tags in the copywriting template, thereby obtaining advertising copy. After replacement, the advertising copy includes the original text and the product terms and / or brand terms. Therefore, in this embodiment, the advertising copy is in text form.

[0074] In another embodiment, the service interface is configured to obtain given information input by the user. This given information may be a specified target product, or a product image and / or advertising text for the target product. The advertising text may include advertising slogans and product terms and / or brand terms for the target product. Then, program instructions implemented by the copywriting generation function corresponding to the service interface call a preset copywriting template. The copywriting template includes original image / animation / video and other carrier resources, as well as indication information indicating the insertion position of the product image and / or advertising text within the carrier resources. Based on the indication information, the product image and / or advertising text of the target product are inserted into the corresponding position of the carrier resources, thereby obtaining the advertising copy. It is easy to understand that, depending on the original form of the carrier resources, the advertising copy can be expressed in any form, such as images, animations, or videos.

[0075] Therefore, the copywriting generation function that provides the service interface of this application is essentially an intelligent advertising creative function. It is implemented as an intelligent advertising creative service through a computer program and the service interface is opened to users. When users call the service interface, the corresponding advertising copy can be generated according to the given information. Users can use the generated advertising copy to place advertisements in the advertising system of e-commerce platforms.

[0076] In one embodiment, the set of advertising copy may contain only the index identifier corresponding to each advertising copy, and the corresponding advertising feature information can be retrieved from the advertising database through the index identifier; in another embodiment, the advertising feature information can be pre-collected and stored in the set of advertising copy for direct retrieval.

[0077] The advertising copy in the set of advertising copy can be either advertising copy that has been placed in the advertising placement system of the e-commerce platform, or advertising copy that has been configured in the advertising placement system but has not been placed.

[0078] Step S2200: Obtain the advertising feature vector of each advertising copy according to the advertising feature information encoding of each advertising copy, and pre-set positive and negative label features in the advertising feature vector respectively, and generate positive and negative samples of the advertising copy accordingly.

[0079] Each ad copy in the aforementioned ad copy collection originates from the advertising delivery system of an e-commerce platform. Therefore, the advertising database of the advertising delivery system stores the corresponding advertising feature information for each ad copy. As mentioned earlier, this advertising feature information can also be retrieved from the advertising database and pre-stored in the ad copy collection for convenient direct access.

[0080] The advertising feature information of the advertising copy, for example, may include any number of the following specific feature information: merchant placement feature information, product configuration feature information, audience configuration feature information, budget settlement feature information, bidding configuration feature information, and placement configuration feature information.

[0081] The aforementioned merchant placement characteristic information, used to indicate the advertising placement scale of the merchant user to which the corresponding advertising copy belongs, can be composed of multiple preferred characteristics, such as: the length of time since the merchant user who placed the corresponding advertising copy first placed the advertisement in the advertising placement system (Last). user The number of ads displayed by the merchant user within one or more preset historical time periods (Sum) user_ads Total amount spent by merchants and users on advertising within one or more preset historical timeframes. user_pay The merchant's advertising characteristics can be constructed using any one or more of the above-mentioned features, or by adding other similar features. It can be seen that these features primarily serve to indicate the scale of the merchant's advertising spending, providing reference information on the merchant's advertising expenditure capacity.

[0082] The product configuration feature information, used to indicate the product corresponding to the corresponding advertising copy, can be composed of multiple preferred features, such as: the semantic feature Product corresponding to the product title, product type, product image, etc. of the product corresponding to the advertising copy. i The length of time since the product was first listed on the e-commerce platform (Last) product The label feature corresponding to the product's image label. product The product configuration feature information can be constructed using any one or more of the above-mentioned features, or by adding other similar features. It can be seen that these features primarily function to indicate the product corresponding to the advertising copy, providing reference information for the advertising feature information from the perspective of the product's own configuration information.

[0083] The audience configuration feature information, used to indicate the target audience of the corresponding advertising copy, can be composed of multiple preferred features, such as: the gender of the target audience for the corresponding advertising copy in the advertising delivery system. people Age group people Region people The audience configuration feature information can be constructed by using any one or more of the above-mentioned features, or by adding other similar features. It can be seen that these features primarily function to indicate the target audience group corresponding to the advertising copy, providing reference information for the advertising feature information from the perspective of the target demographic of the advertising copy.

[0084] The budget settlement feature information, used to indicate the available funds for the corresponding advertising copy, can be composed of multiple preferred features. For example, the remaining budget balance of the advertising copy to which it belongs in the advertising delivery system as of the previous day. budget The daily budget set by the merchant user. oneday The budget settlement feature information can be constructed by using any one or more of the above-mentioned features, or by adding other similar features. It can be seen that these features primarily function to indicate whether the budget of the merchant user associated with the advertising copy is sufficient, providing reference information for the advertising feature information from the perspective of the available funds for the advertising copy.

[0085] The bidding configuration feature information mentioned above is used to indicate the bidding constraints of the corresponding ad copy. When merchants place ads in the ad delivery system, they typically configure the ad bidding amount, which represents the unit price of the current ad. per The current advertising bidding strategy per Information such as these can be used, employing any one or more of the above, or adding other similar items, to constitute the bidding configuration feature information. It can be seen that these features primarily function to indicate the spending intentions of the merchant users associated with the advertising copy, providing reference information for the advertising feature information from the perspective of the bidding constraints of the advertising copy.

[0086] The aforementioned ad placement configuration features are used to indicate the ad placement constraints for the corresponding ad copy. When merchants place ads on various ad placement systems, they typically configure the ad placement time period. timezone Advertising optimization methods method The budget settlement feature information can be constructed using any one or more of the above-mentioned items, or by adding other similar items. It can be seen that these features primarily function to indicate the corresponding merchant user's intention to control the placement of the advertising copy, providing reference information for the advertising feature information from the perspective of the placement constraints of the advertising copy.

[0087] Based on the above disclosure regarding the composition of advertising feature vectors for advertising copy, in one embodiment, all the feature information from the above examples can be organized and encoded in a preset order to form an initial feature vector for each advertising copy. Thus, each initial feature vector... It can be represented as:

[0088]

[0089] To use the initial feature vector of each advertising copy to generate the positive and negative samples needed for the comparison group, a positive label feature (Label) can be added to each of the initial feature vectors respectively. positive and negative label features negative Two ad feature vectors are constructed for each ad copy, and these are used as positive samples. and negative samples

[0090] Therefore, the expression for the advertising feature vector of the positive sample is as follows:

[0091]

[0092] The expression for the advertising feature vector of the negative sample is as follows:

[0093]

[0094] The positive tagging feature is used to assume that the corresponding advertising copy was generated by the service interface, even if the advertising copy was not generated by the service interface, thereby forcibly tagging the corresponding advertising copy as data generated by the service interface.

[0095] The negative labeling feature is used to assume that the corresponding advertising copy is not generated by the service interface, even if the advertising copy is generated by the service interface. Thus, the corresponding advertising copy is forcibly labeled as data not generated by the service interface.

[0096] Therefore, the positive and negative labeling features are two mutually exclusive labels in a representational sense. Adding such labeling features to the initial feature vector is essentially adding intervention variables to the initial feature vector. The introduction of these intervention variables generates samples, which influencing the prediction results of the model based on these samples. For ease of implementation, in one embodiment, the positive and negative labeling features can be binary represented using 1 and 0, respectively. Of course, those skilled in the art can flexibly choose different values ​​for feature representation based on the principle that the two labeling features are different.

[0097] By presetting the positive and negative labeling features in the advertising feature vector of the advertising copy to form positive and negative samples, each advertising copy will obtain two advertising feature vectors corresponding to different comparison groups. It is easy to understand that performing the same operation based on these two advertising feature vectors will yield different results, which can be used by this application to implement A / B group comparison test.

[0098] Step S2300: Use a preset advertising effectiveness prediction model to predict the advertising effectiveness indicators of the positive and negative samples respectively. The advertising effectiveness prediction model is pre-trained to a convergent state.

[0099] To enable the prediction of advertising effectiveness indicators based on the positive and negative samples, this application provides an advertising effectiveness prediction model suitable for predicting the corresponding advertising effectiveness indicators based on the samples.

[0100] For example, the advertising effectiveness prediction model can be implemented based on a classic machine learning model or a deep learning model. The classic machine learning model can be any algorithm such as Lasso Regression, XGBoost, or LightGBM, while the deep learning model can be a feedforward neural network, convolutional neural network, or DeepFM. The advertising prediction model can be pre-trained to convergence by someone skilled in the art using sufficient training samples based on the principles disclosed in this application, enabling it to learn to predict the corresponding advertising effectiveness index based on any positive or negative sample.

[0101] The aforementioned advertising performance metrics can be click-through rate, collection rate, add-to-cart rate, conversion rate, or return on investment, which are used to indicate the effectiveness of advertising campaigns.

[0102] The click-through rate (CTR) mentioned above is a commonly used term in internet advertising. It refers to the click-through rate of online advertisements (image ads / text ads / keyword ads / ranking ads / video ads, etc.), which is the actual number of clicks on the ad (strictly speaking, the number of times the ad reaches the target page) divided by the number of ad impressions (showcontent).

[0103] The collection rate refers to the ratio of the total number of product pages that are collected after clicking on an online advertisement to the total number of users who clicked on the page, i.e., the number of visitors.

[0104] The add-to-cart rate is similar to the collection rate, referring to the ratio of the total number of items added to the shopping cart from the product page reached after clicking on the online advertisement to the total number of users who clicked on the ad, i.e., the number of visitors.

[0105] The Conversion Rate (CVR) is the percentage of internet users who click on an online advertisement and visit the promoting website and subsequently convert, typically reflecting the direct revenue of the advertisement. It was initially proposed in the "China Online Marketing (Advertising) Effectiveness Evaluation Criteria" at the China Internet Association's Online Marketing Working Committee Member Conference held on June 18, 2009. The statistical period is usually hourly, daily, weekly, or monthly, and can be set as needed. The statistical objects include various advertising formats such as flash ads, image ads, text link ads, advertorials, email ads, video ads, and rich media ads. CVR = (Conversions / Clicks) * 100%.

[0106] The Return on Advertising Spend (ROAS) is a marketing metric that measures the effectiveness of online advertising. ROAS = Total Advertising Revenue / Advertising Cost.

[0107] It is easy to understand that any of the above advertising performance indicators can be directly calculated based on the corresponding data generated after the ad placement of each ad copy in the advertising placement system, according to the calculation principle of the advertising performance indicator. During the training phase of the advertising performance prediction model, the model is trained using corresponding training samples and the training samples themselves. Furthermore, the model is trained using the advertising performance indicator supervision model of the ad corresponding to the training sample. This allows the advertising performance prediction model to learn the ability to predict the corresponding advertising performance indicator of the sample based on the corresponding sample.

[0108] Accordingly, for each positive and negative sample of an advertisement in the advertising copy set, the positive and negative samples can be input one by one into the advertising effectiveness prediction model prepared in this application for prediction, so as to obtain the advertising effectiveness index predicted by the model.

[0109] As explained above, the entire set of advertising copy samples comprises two main groups. The first group consists of positive samples, each of which is intervened with pre-defined positive labeling features to assume that the corresponding advertising copy was generated by the service interface; this group is the fully intervened group. The second group consists of negative samples, each of which is intervened with pre-defined negative labeling features to assume that the corresponding advertising copy was not generated by the service interface; this group is the uninterrupted group. Each sample in both the fully intervened and uninterrupted groups can obtain its corresponding advertising performance index through the advertising performance prediction model.

[0110] Step S2400: Calculate the quality index of the service interface by comparing the advertising effectiveness indicators of positive and negative samples.

[0111] By using the advertising effectiveness metrics of samples from the fully intervened group and the uninterrupted group, a comparative experiment can be conducted. Through this comparison, an evaluation parameter can be determined as a quality indicator to measure the quality of the service interface.

[0112] In one embodiment, the quality indicators can be determined by conducting comparative experiments through the following specific steps:

[0113] Step S2410: Calculate the mean values ​​of the advertising effectiveness indicators for the positive and negative samples corresponding to the advertising copy set, respectively.

[0114] For the fully intervened group and the uninterrupted group, the mean of the advertising effectiveness index for all samples in each group can be calculated. Specifically, the mean of the advertising effectiveness index for all positive samples in the fully intervened group is obtained as the first mean, and the mean of the advertising effectiveness index for all negative samples in the uninterrupted group is obtained as the second mean. It is easy to understand that each mean reflects the overall potential benefit of the advertising copy in the advertising copy set once it is launched, under the conditions of service interface intervention and non-intervention.

[0115] Step S2420: Subtract the mean of the negative samples from the mean of the positive samples to obtain the difference, which is used as the quality indicator of the service interface.

[0116] To achieve comparison, the first mean corresponding to the positive sample is subtracted from the second mean corresponding to the negative sample to obtain a difference. This difference can be used as a quality indicator to evaluate the quality of the service interface. This quality indicator also reflects the difference in the overall effectiveness of the service interface in terms of intervention and non-intervention in advertising copy, thereby measuring the quality of the intelligent advertising creative function that provides the service interface.

[0117] In other embodiments, the calculation process can be modified appropriately. For example, the difference between the advertising effectiveness indicators of each positive and negative sample of each advertising copy can be calculated first, and then the average of the differences corresponding to all advertising copies can be calculated, and the average can be used as the quality indicator.

[0118] Based on the above embodiments, it can be seen that this application has many advantages, including at least:

[0119] First, this application applies the counterfactual inference principle to generate advertising feature vectors based on the advertising feature information of advertising copy generated by the service interface. For each advertising copy's feature vector, positive and negative labeled features are pre-set respectively, generating corresponding positive and negative samples to represent the data corresponding to the factual and counterfactual results. The positive and negative samples are then predicted using an advertising effectiveness prediction model to obtain corresponding advertising effectiveness indicators. Based on the comparison results of the advertising effectiveness indicators of the two types of samples, corresponding quality indicators are determined to measure the overall quality corresponding to the service interface, thus solving the problem of effectively measuring the advertising copy generation function implemented by the service interface.

[0120] Secondly, the advertising effectiveness prediction model of this application, after being trained, can be applied without relying on the advertising effectiveness metrics generated after the actual delivery of the advertising copy produced by the service interface to evaluate the function of the service interface. Instead, it only needs to rely on the advertising copy produced by the service interface to carry out comparative experiments. Therefore, it can intervene early in the gray period before a large number of advertising effectiveness metrics are generated after the corresponding function of the service interface is launched. It does not need to rely on the advertising copy to generate corresponding advertising effectiveness data before intervention. It can realize early evaluation of the corresponding function in advance and ensure that the corresponding function can be optimized and upgraded in a timely manner.

[0121] Furthermore, this application transfers the counterfactual inference principle to the field of e-commerce advertising service function evaluation. For the advertising effectiveness prediction model adopted in this application, it is suitable for predicting advertising effectiveness indicators based on advertising feature vectors determined according to the counterfactual inference principle. In fact, it enriches the function of the advertising effectiveness prediction model, making it suitable for evaluating the merits of relevant e-commerce advertising service functions, thereby creating economies of scale for e-commerce platforms.

[0122] Based on any of the above embodiments, please refer to Figure 2 Before step S2300, which involves using a preset advertising effectiveness prediction model to predict the advertising effectiveness indicators of the positive and negative samples respectively, the following steps are included:

[0123] Step S1100: Obtain the training dataset, which includes multiple ad copy that has been deployed and the ad performance metrics generated after deployment.

[0124] Prepare a training dataset for training the advertising effectiveness prediction model of this application, which includes advertising copy that has been placed in the advertising delivery system, and obtain the actual advertising effectiveness index generated after it is placed in the advertising delivery system. Thus, establish a mapping relationship between each advertising copy and its corresponding advertising effectiveness index in the training dataset.

[0125] Some of the advertising copy in the training dataset is generated by the intelligent advertising creative service that provides the service interface, while the other part may not be generated by the service interface, such as user-defined copy. A source tag can be added to each advertising copy in the training dataset to indicate whether the corresponding advertising copy was generated by the service interface. The source tag can be represented as a binary form of 1 or 0 so that it can be directly used as the positive or negative label feature in the future.

[0126] Similarly, based on the advertising performance indicators predicted by the advertising performance prediction model, the advertising performance indicators associated with the advertising copy in the training dataset are consistent with the type of advertising performance indicators that the model hopes to predict, and can be any one of click pass rate, collection rate, add-to-cart rate, conversion rate, and return on investment.

[0127] In one embodiment, after the training dataset is used to train the advertising effectiveness prediction model, the advertising copy in it can be used as the advertising copy set of this application.

[0128] During the gray-scale testing phase of the intelligent advertising creative service of this application, the number of generated and delivered advertising copy is relatively limited, and the number of advertisements for which the advertising delivery system generates corresponding advertising performance indicators is also relatively limited. However, because this application applies the counterfactual inference principle to train the advertising performance prediction model, the dependence on training samples required for model training is significantly reduced. Therefore, even if the advertising performance prediction model is trained using a limited number of advertising feature vectors corresponding to the advertising copy during the gray-scale testing phase, it is relatively easy to train the advertising performance prediction model to a convergent state. Of course, the specific sample size can be flexibly determined by those skilled in the art during the actual training process.

[0129] Step S1200: Obtain the advertising feature vector of each advertising copy according to the advertising feature information encoding of each advertising copy, and pre-set positive or negative label features in the advertising feature vector according to whether the advertising copy is generated by a preset service interface, and generate training samples.

[0130] Based on the principle disclosed in step S2200 above in this application, the advertising feature information of each advertisement copy corresponding to the advertisement in the training dataset can be obtained from the advertising delivery system. The feature data in the advertising feature information is encoded to generate the corresponding initial feature vector. Then, positive or negative label features are added to the initial feature vector to construct the corresponding advertising feature vector, and the corresponding positive or negative samples are obtained as training samples in the training stage.

[0131] It should be noted that the training phase of the advertising feature vector in the advertising effectiveness prediction model differs from the inference phase described above. The difference lies in the fact that, during the training phase, the pre-defined labeled features in the two advertising feature vectors of each advertising copy—namely, the positive and negative labeled features—do not require the application of counterfactual assumptions. Instead, they are labeled based on whether the corresponding advertising copy was generated by the service interface described in this application. Therefore, during the training phase, each advertising copy in the training dataset generates only one training sample, which is represented as a positive or negative sample depending on whether it was generated by the service interface.

[0132] It's easy to understand that, given that the source tag for each ad copy is already labeled in the training dataset, we can directly use the source tag for each ad copy as its corresponding label feature and pre-place it in the ad feature vectors corresponding to the positive and negative samples. That is, when the source tag for an ad copy is represented as "1", pre-placing the positive label feature "1" in the corresponding position of its initial feature vector will obtain the corresponding positive sample; when the source tag for an ad copy is represented as "0", pre-placing the negative label feature "0" in the corresponding position of its initial feature vector will obtain the corresponding negative sample.

[0133] After the above process, training samples of advertising copy in the training dataset can be obtained, which can be used to iteratively train the advertising effectiveness prediction model.

[0134] Step S1300: The advertising effectiveness prediction model is iteratively trained to convergence using training samples corresponding to the advertising copy in the training dataset. The advertising effectiveness index generated after the training samples are deployed is used to supervise the prediction results of the model.

[0135] When training the advertising performance prediction model using training samples corresponding to the advertising copy in the training dataset, a supervised training method is adopted. Each training sample, i.e., a positive or negative sample corresponding to the advertising copy, serves as the input to the advertising performance prediction model. The input content is the corresponding advertising feature vector. After semantic reasoning of the advertising feature vector, the model predicts the corresponding advertising performance index as the prediction result. The advertising performance index generated after the advertising copy is deployed serves as the supervisory label for the advertising performance prediction model, supervising the prediction results of the training samples.

[0136] In each iteration, the model's loss value is obtained by comparing the difference between the supervision label and the model's predicted output. This loss value is then used for backpropagation of the entire advertising effectiveness prediction model, correcting the weight parameters of each stage and updating the model's gradient to continuously approach convergence. If the model has not converged, training samples corresponding to the next ad copy are obtained from the training dataset, and the model is trained iteratively. When the loss value reaches or infinitely approaches a preset threshold, such as "0", the advertising effectiveness prediction model is considered to have been trained to a convergent state, and training can be terminated.

[0137] Based on the above embodiments, it can be understood that when training the advertising effectiveness prediction model, this application associates the marked features generated by whether the advertising copy is from the service interface of this application as features corresponding to factual information, and trains the model to a convergent state. When the model is subsequently used for inference, the counterfactual inference principle can be applied. By pre-setting positive and negative marked features in the advertising feature vector, intervention can be achieved, enabling the model to obtain advertising effectiveness indicators under two conditional assumptions for comparative testing. The quality indicators of the service interface can be determined through comparative testing, thereby realizing the quality assessment of the intelligent advertising creative service or function corresponding to the service interface.

[0138] Based on any of the above embodiments, please refer to Figure 3 In step S2200 and / or step S1200, obtaining the advertising feature vector based on the advertising feature information encoding of each advertising copy includes the following steps:

[0139] Step S3100: Retrieve the advertising feature information of the advertising copy from the data interface provided by the advertising delivery system;

[0140] The advertising delivery system can pre-implement a data interface for external access to the corresponding advertising feature information of each advertisement. When the data interface is invoked, the corresponding advertisement object is determined based on the given advertisement copy. Then, the advertising feature information of the advertisement object required by this application is invoked and returned to the caller. In this way, the advertising feature information corresponding to each advertisement copy in the training dataset and advertisement copy set of this application can be obtained through invocation.

[0141] Step S3200: Perform data cleaning on the advertising feature information to obtain feature data, and convert the feature data into corresponding feature values;

[0142] To extract the advertising feature vector corresponding to the advertising copy according to this application, especially according to the features required in the initial feature vector of the advertising feature vector, the advertising feature information can first be cleaned to extract the feature data.

[0143] The aforementioned feature data can be uniformly represented in numerical form. As can be seen from the principle disclosed in step S2200 of this application, each specific feature constituting the initial feature vector can be represented in the aforementioned numerical form. For cases where the original feature information is not in numerical form, such as the semantic feature "Product" in product configuration feature information... i Label features product And the audience gender in the audience configuration feature information. people Age group people Region people These can all be converted into corresponding numerical features. For example, for the semantic feature Product... i A pre-defined text feature extraction model can be used to extract text features from the product titles of corresponding products, obtaining their corresponding vectors, which can then be used as their numerical features. Similarly, for the region... people The corresponding codes for each region can be used as their respective data features. In this way, the advertising information is cleaned to obtain its feature data in numerical form, thus representing each specific piece of information as a feature value.

[0144] Step S3300: Encode the feature values ​​corresponding to each specific configuration information into an advertising feature vector according to the preset sorting rules.

[0145] After cleaning the advertising feature information, the feature values ​​are organized according to a preset encoding and sorting rule. This rule must remain consistent throughout the training and inference phases of the advertising effectiveness prediction model. This sorting rule forms the advertising feature vector corresponding to the advertising copy. Alternatively, during the training or inference phase, the advertising feature vector can be pre-defined with corresponding labeled features, i.e., positive or negative labeled features. This completes the encoding process for the advertising feature vector of each advertising copy.

[0146] As can be understood from the above embodiments, this application converts various feature information into feature values ​​by performing data cleaning on the advertising feature information corresponding to the advertising copy, and then organizes the feature values ​​into advertising feature vectors according to preset encoding rules, thereby realizing the encoding process of advertising feature information of advertising copy, simplifying the process, and realizing the comprehensive representation of various aspects of advertising information corresponding to advertising copy through advertising feature vectors, so that the advertising effectiveness prediction model can obtain effective samples for training or inference, and output effective advertising effectiveness indicators.

[0147] Based on any of the above embodiments, please refer to Figure 4 After step S2400, which involves calculating the quality index of the service interface based on a comparison of advertising effectiveness indicators of positive and negative samples, the following steps are included:

[0148] Step S2500: Determine whether the quality index is lower than a preset threshold. If it is lower than the preset threshold, send an alarm message to a preset communication interface.

[0149] As mentioned earlier, the quality indicators corresponding to the intelligent advertising creative services launched on e-commerce platforms are essentially evaluation results of the quality of the advertising copy generated by the service interface provided by the intelligent advertising creative services. If the quality indicator is low, it indicates that the revenue of the generated advertising copy is relatively poor; if the quality indicator is high, it indicates that the revenue of the generated advertising copy is relatively good. Accordingly, e-commerce platforms can adopt a preset threshold, which can be a measured threshold or an empirical threshold, flexibly set by those skilled in the art. The preset threshold is compared with the quality indicator. When the quality indicator is higher than the preset threshold, no action is taken, and the normal operation of the intelligent advertising creative service is maintained. Otherwise, when the quality indicator is lower than the preset threshold, an alarm message is sent to the e-commerce platform's management user by calling a preset communication interface to prompt the management user to optimize and upgrade the intelligent advertising creative service. After the optimization and upgrade are completed, the corresponding service interface is reopened.

[0150] As can be understood from the above embodiments, by using the quality indicators to control the open permissions of the service interface of the intelligent advertising creative service of this application, a closed loop of the utilization of quality indicators can be realized. Based on the quality indicators, the service interface can be called in a timely manner. When the intelligent advertising creative service is ineffective, early intervention can be made to stop the loss for the e-commerce platform, avoid user experience problems related to advertising, and ensure that high-quality services are provided to advertising users.

[0151] Please see Figure 5To meet one of the purposes of this application, a service interface quality assessment device is provided, which is a functional embodiment of the service interface quality assessment method of this application. The device includes a copy acquisition module 2100, an encoding processing module 2200, an indicator prediction module 2300, and a comparison and evaluation module 2400, wherein: the copy acquisition module 2100 is used to acquire an advertising copy set, including multiple advertising copy generated by a preset service interface; the encoding processing module 2200 is used to encode the advertising feature information of each advertising copy to obtain its advertising feature vector, and preset positive and negative label features in the advertising feature vector to generate positive and negative samples of the advertising copy accordingly; the indicator prediction module 2300 is used to predict the advertising performance indicators of the positive and negative samples respectively using a preset advertising performance prediction model, the advertising performance prediction model being pre-trained to a convergent state; the comparison and evaluation module 2400 is used to calculate the quality indicators of the service interface by comparing the advertising performance indicators of the positive and negative samples.

[0152] Based on any of the above embodiments, prior to the indicator prediction module 2300, the module includes: a dataset acquisition module for acquiring a training dataset, including multiple deployed advertising copy and the advertising performance indicators generated after deployment; a sample generation module for obtaining an advertising feature vector based on the advertising feature information encoding of each advertising copy, and pre-setting positive or negative labeling features in the advertising feature vector according to whether the advertising copy was generated by a preset service interface, thereby generating training samples; and a training execution module for iteratively training the advertising performance prediction model to a convergent state using the training samples corresponding to the advertising copy in the training dataset, wherein the advertising performance indicators generated after deployment of the training samples are used to supervise the prediction results of the model.

[0153] Based on any of the above embodiments, the encoding processing module 2200 or the sample generation module includes: an information retrieval unit, used to retrieve the advertising feature information of the advertising copy from the data interface provided by the advertising delivery system; a data cleaning unit, used to perform data cleaning on the advertising feature information to obtain feature data, and convert the feature data into corresponding feature values; and a feature encoding unit, used to encode the feature values ​​corresponding to each specific configuration information into advertising feature vectors according to a preset sorting rule.

[0154] Based on any of the above embodiments, the advertising feature information includes any of the following: merchant placement feature information, used to indicate the advertising placement scale of the merchant user to which the corresponding advertising copy belongs; product configuration feature information, used to indicate the product corresponding to the corresponding advertising copy; audience configuration feature information, used to indicate the audience user group of the corresponding advertising copy; budget settlement feature information, used to indicate the available funds for the corresponding advertising copy; bidding configuration feature information, used to indicate the bidding constraint information of the corresponding advertising copy; and placement configuration feature information, used to indicate the placement constraint information of the corresponding advertising copy.

[0155] Based on any of the above embodiments, the advertising copy set includes advertising copy from the training dataset; or, the advertising performance indicator is a click-through rate, collection rate, add-to-cart rate, conversion rate, or return on investment that indicates the effectiveness of advertising campaigns; or, the advertising copy includes any one or more of text, images, videos, and animations; or, the service interface is used to call a preset copy template to generate advertising copy corresponding to given information, wherein the given information includes product keywords and / or brand keywords of the target product.

[0156] Based on any of the above embodiments, the comparison and evaluation module 2400 includes: a mean statistics unit, used to calculate the mean of the advertising effectiveness indicators of the positive and negative samples corresponding to the advertising copy set respectively; and an indicator comparison unit, used to subtract the mean of the negative samples from the mean of the positive samples to obtain the difference, which is used as the quality indicator of the service interface.

[0157] Based on any of the above embodiments, the comparison and evaluation module 2400 further includes: a decision processing module, used to determine whether the quality index is lower than a preset threshold, and when it is lower than the preset threshold, to send an alarm message to a preset communication interface.

[0158] To address the aforementioned technical problems, embodiments of this application also provide computer equipment. For example... Figure 6The diagram shows the internal structure of a computer device. The computer device includes a processor, a computer-readable storage medium, a memory, and a network interface connected via a system bus. The computer-readable storage medium stores an operating system, a database, and computer-readable instructions. The database may store a sequence of control information. When executed by the processor, the computer-readable instructions enable the processor to implement a product search category identification method. The processor provides computing and control capabilities to support the operation of the entire computer device. The memory stores computer-readable instructions, which, when executed by the processor, enable the processor to execute the service interface quality assessment method of this application. The network interface of the computer device is used for communication with a terminal. Those skilled in the art will understand that… Figure 6 The structure shown is merely a block diagram of a portion of the structure related to the present application and does not constitute a limitation on the computer device to which the present application is applied. Specific computer devices may include more or fewer components than those shown in the figure, or combine certain components, or have different component arrangements.

[0159] In this embodiment, the processor is used to execute... Figure 5 The system defines the specific functions of each module and its sub-modules. The memory stores the program code and various data required to execute these modules or sub-modules. The network interface is used for data transmission between the user terminal and the server. In this embodiment, the memory stores the program code and data required to execute all modules / sub-modules in the service interface quality assessment device of this application. The server can call the server's program code and data to execute the functions of all sub-modules.

[0160] This application also provides a storage medium storing computer-readable instructions, which, when executed by one or more processors, cause the one or more processors to perform the steps of the service interface quality assessment method of any embodiment of this application.

[0161] This application also provides a computer program product, including a computer program / instructions that, when executed by one or more processors, implement the steps of the method described in any embodiment of this application.

[0162] Those skilled in the art will understand that all or part of the processes in the methods of the above embodiments of this application can be implemented by a computer program instructing related hardware. This computer program can be stored in a computer-readable storage medium, and when executed, it can include the processes of the embodiments of the methods described above. The aforementioned storage medium can be a magnetic disk, optical disk, read-only memory (ROM), or random access memory (RAM), etc.

[0163] In summary, this application can determine quality indicators for the functions corresponding to the service interfaces used to generate advertising copy in e-commerce platforms, enabling early intervention and evaluation of related functions, and facilitating timely optimization and upgrading of related functions by e-commerce platforms.

[0164] Those skilled in the art will understand that the steps, measures, and solutions in the various operations, methods, and processes discussed in this application can be alternated, modified, combined, or deleted. Furthermore, other steps, measures, and solutions in the various operations, methods, and processes discussed in this application can also be alternated, modified, rearranged, decomposed, combined, or deleted. Furthermore, steps, measures, and solutions in the prior art that are similar to those disclosed in this application can also be alternated, modified, rearranged, decomposed, combined, or deleted.

[0165] The above description is only a partial embodiment of this application. It should be noted that for those skilled in the art, several improvements and modifications can be made without departing from the principle of this application, and these improvements and modifications should also be considered within the scope of protection of this application.

Claims

1. A method for evaluating the quality of a service interface, characterized in that, Includes the following steps: Get the set of advertising copy, which includes multiple advertising copy generated by preset service interfaces; The advertising feature vector is obtained by encoding the advertising feature information of each advertising copy. For the same advertising copy, positive and negative labeling features are preset as intervention variables in the corresponding advertising feature vector. Positive samples representing the factual results of the corresponding advertising copy generated by the service interface and negative samples representing the counterfactual results of the corresponding advertising copy not generated by the service interface are generated accordingly. The advertising effectiveness prediction model is used to predict the advertising effectiveness indicators of the positive and negative samples respectively. The advertising effectiveness prediction model is pre-trained to a convergent state. The quality index of the service interface is obtained by comparing the advertising performance indicators of positive and negative samples, including: calculating the mean of the advertising performance indicators of the positive and negative samples corresponding to the advertising copy set respectively; and subtracting the mean of the negative samples from the mean of the positive samples to obtain the difference, which is used as the quality index of the service interface.

2. The service interface quality assessment method according to claim 1, characterized in that, Before the step of predicting the advertising effectiveness indicators of the positive and negative samples respectively using a preset advertising effectiveness prediction model, the following steps are included: Obtain the training dataset, which includes multiple ad copy that has been delivered and the ad performance metrics generated after the delivery. The advertising feature vector of each advertising copy is obtained by encoding the advertising feature information. Based on whether the advertising copy was generated by a preset service interface, a positive or negative label feature is preset in the advertising feature vector to generate training samples. The advertising effectiveness prediction model is iteratively trained until convergence using training samples corresponding to the advertising copy in the training dataset. The prediction results of the model are supervised by advertising effectiveness indicators generated after the training samples are deployed.

3. The service interface quality assessment method according to claim 1 or 2, characterized in that, The advertising feature vector is obtained by encoding the advertising feature information of each advertising copy. Includes the following steps: The advertising feature information of the advertising copy is retrieved from the data interface provided by the advertising delivery system; The advertising feature information is cleaned to obtain feature data, and the feature data is converted into corresponding feature values; The feature values ​​corresponding to each specific configuration information are encoded into advertising feature vectors according to the preset sorting rules.

4. The service interface quality assessment method according to claim 1 or 2, characterized in that, The advertising feature information includes any of the following: Merchant placement characteristic information is used to indicate the scale of advertising placement by the merchant user to which the corresponding advertising copy belongs; Product configuration feature information is used to indicate the product corresponding to the corresponding advertising copy; Audience configuration feature information is used to indicate the target user group for the corresponding advertising copy; Budget settlement feature information is used to indicate the available funds for the corresponding advertising copy; Bidding configuration feature information, used to indicate the bidding constraint information of the corresponding ad copy; The delivery configuration feature information is used to indicate the delivery constraints of the corresponding advertising copy.

5. The service interface quality assessment method according to claim 2, characterized in that: The advertising copy set contains advertising copy from the training dataset; or... The advertising performance metrics are click-through rate, favorites rate, add-to-cart rate, conversion rate, or return on investment (ROI) used to indicate the effectiveness of advertising campaigns; or, The advertising copy includes any one or more of the following: text, images, videos, and animations; or, The service interface is used to call a preset copy template to generate advertising copy corresponding to given information, which includes product terms and / or brand terms of the target product.

6. The service interface quality assessment method according to claim 1 or 2, characterized in that, After calculating the quality indicators of the service interface by comparing the advertising effectiveness indicators of positive and negative samples, the following steps are included: Determine whether the quality indicator is lower than a preset threshold. If it is lower than the preset threshold, send an alarm message to a preset communication interface.

7. A service interface quality assessment device, characterized in that, include: The copy acquisition module is used to acquire a set of advertising copy, which includes multiple advertising copy generated by preset service interfaces; The encoding processing module is used to encode the advertising feature information of each advertising copy to obtain its advertising feature vector. For the same advertising copy, positive and negative labeling features are preset as intervention variables in the corresponding advertising feature vector. Positive samples representing the factual results of the corresponding advertising copy generated by the service interface and negative samples representing the counterfactual results of the corresponding advertising copy not generated by the service interface are generated accordingly. The indicator prediction module is used to predict the advertising effectiveness indicators of the positive and negative samples respectively using a preset advertising effectiveness prediction model, wherein the advertising effectiveness prediction model is pre-trained to a convergent state. The comparison and evaluation module is used to calculate the quality index of the service interface based on the comparison of advertising performance indicators of positive and negative samples. This includes: calculating the mean of advertising performance indicators of positive and negative samples corresponding to the advertising copy set respectively; and subtracting the mean of negative samples from the mean of positive samples to obtain the difference, which is used as the quality index of the service interface.

8. A computer device comprising a central processing unit and a memory, characterized in that, The central processing unit is used to invoke and run a computer program stored in the memory to perform the steps of the method as described in any one of claims 1 to 6.

9. A computer-readable storage medium, characterized in that, It stores, in the form of computer-readable instructions, a computer program implemented according to any one of claims 1 to 6, which, when invoked by a computer, executes the steps included in the corresponding method.

Images